ecocloud provides an online platform giving researchers easy access to large volumes of curated data and tools using Cloud and Virtual Laboratory (VL) technology.
It gives you 10GB of persistent storage to keep your code/notebooks so they are ready to go when you start up a server (R or Python environment). You can import and export large datasets and results directly from your server to places like Dropbox.
With a couple of clicks, you can have a server running in the cloud with R (including RStudio) or Python as your coding language of choice. These servers use the JupyterLabs interface which includes connections to GitHub, Google Drive and Dropbox.
You can spawn a ready-to-go Virtual Desktop using TERN CoESRA which includes software such as QGIS and Biodiverse.
ecocloud also contains a portal that indexes data from more than 600 publishers and provides code snippets to access and interact with this data.
The first version of ecocloud was launched in September 2018, but it’s already helping researchers and students around Australia do more efficient and higher impact science.
Dr Nestor Bosch Guerra from the School of Animal Biology at the University of Western Australia is running a research project looking at national-scale trends in fish diversity, abundance and length using data from the GlobalArchive project.
“I’m working with data from over 20,000 remote video deployments, from 12 government and academic institutions, so my models kept failing due to the memory-limits on my computer,” says Nestor. “To solve my big data problem, I requested access to a large machine within ecocloud.”
“Access to an XXL machine with pre-installed libraries on ecocloud means that, with one click, I can now run complex big data analyses in seconds. I can also use the GitHub connection to move code and data in and out of my server environment.”
Nestor says that his analyses and models of the GlobalArchive (funded by the ARDC) fish data in ecocloud are expected to support State of the Environment reporting and add value to the millions of dollars that have already been invested in the dataset.
Nestor Bosch Guerra from the School of Animal Biology at the University of Western Australia uses ecocloud to run complex big data analyses in seconds
Dr Tim Langlois and Dr Jane Prince from the University of Western Australia Oceans Institute have used ecocloud to teach data analysis techniques to students from their Marine Ecology Masters program.
More than 20 students collected information on lobster and fish density in waters around Rottnest Island and, while still in the field, used their laptops to immediately analyse the data.
“Within five minutes students were logged in, connected to the GitHub repo and doing science,” reports Tim. “We all used R and everything worked! This would usually take an hour or two in a computer lab to pass around a USB stick with the data and code, get all students working in the same system, with the right packages installed and working directories set.”
“It’s a huge improvement and an Australian first.”
Tim Langlois (left) has used ecocloud to teach data analysis techniques in the field to students from University of Western Australia's Marine Ecology masters program (image courtesy of Tim Langlois)
ecocloud is also being used intensively by Dr Serena Lee from the Centre for Coastal Management at Griffith University.
Taking advantage of the support provided, Serena has been working closely with the ecocloud team to optimise her Python code and access and pre-process data in preparation for coastal modelling.
“ecocloud provided me with a well-defined and stable environment to work within, as well as access to a 60GB and 8CPU machine in the cloud,” says Serena.
Serena Lee from the Centre for Coastal Management at Griffith University has used ecocloud to access and pre-process data in preparation for coastal modelling (image courtesy of Griffith University)
The ecocloud project is a collaborative effort supported by the NCRIS-funded Australian Research Data Commons (ARDC) as the major project sponsor, and the Queensland Cyber Infrastructure Foundation (QCIF) as the implementation partner.
TERN is a founding partner of ecocloud alongside:
To ensure ecocloud continues to create solutions that meet your needs, we encourage you to take a look at the alpha release and submit comments, ideas and thoughts here.
Published in TERN newsletter April 2019