It’s easy to see why cloud bursting – where an application is run in a private cloud or data centre and then ‘bursts’ into a public cloud when demand dictates – could appeal to research universities.
It can provide institutions with an escape valve when their in-house resources are fully committed, helping to potentially speed up research and save costs.
In recent years adoption of cloud computing has been transforming research and education, and although change within academia can be slow, the latest UK Research & Innovation (UKRI) e-infrastructure report has shown a growing interest in community and public clouds.
“We also see that scientific computing teams at universities and research institutes are starting to look very seriously at virtualising their in-house compute clusters,” says Martin Hamilton, a member of UKRI’s Cloud for Research working group.
Although educational researchers tend to “thrash kit within an inch of its life”, Hamilton says there’s a “growing recognition that having the option of running a virtual machine (VM) image can make it easier for researchers to share and re-use code.”
However, there are divergent opinions within the research community as to how best cloud resources should be deployed. While bursting remains a go-to choice for some, others either remain reticent or have avoided the technology entirely in favour of a full-fledged cloud.
Cloud bursting advocates
Two of the world’s biggest champions of the cloud bursting approach are the University of Cambridge and, on the other side of the world, the National University of Singapore (NUS).
“NUS has a wide range of computing requirements, making it impractical for all resources and capacity requirements to be provided in-house,” says Tommy Hor, NUS’ chief information technology officer, speaking to Cloud Pro.
The National University of Singapore deploys cloud bursting to support its research projects
“Our researchers occasionally have ad-hoc service demands that require dedicated computing resources to speed up their work. We have started migrating our in-house pay-per-use service to the cloud, and this will give us greater financial agility and economies of scale.”
The University of Cambridge has gone as far as providing its own cloud bursting capabilities. Its Research Computing Services (RCS) operation has a dedicated private ‘public sector’ cloud designed specifically for scientific and technical computing.
“Researchers from across Cambridge University, plus UK universities and companies, use RCS for cloud bursting,” says Dr Paul Calleja, the university’s director of Research Computing. “Research undertaken includes large-scale genomic analysis for clinical diagnosis and simulations of jet engines.”
Cloud bursting challenges and limitations
But while cloud bursting has potential benefits, there are still problems to be ironed out. This includes interoperability issues between environments, pricing models and security.
“We recently saw a number of Docker images laden with malware removed from the public registry, opening black doors onto users’ machines and running cryptocurrency mining processes,” says UKRI’s Martin Hamilton.
“Things like this take on an even greater significance when we are talking about compute jobs to calculate stresses on airframes, analyse CT images looking for tumours, or model the effect a new drug will have on the human body.”
For the University of Bristol, cloud bursting is seen as a highly restrictive approach to deployment, one that needlessly increases the complexity of a network.
“In my opinion cloud bursting limits the use of the cloud to being just an extension of a local on-premise compute cluster,” says Dr Christopher Woods, leader of the university’s Research Software Engineering group, which is fully in the cloud.
“It also means you get the worst of both worlds – you’re running both a cluster and a cloud, so have twice the complexity.”
He adds that, in his experience, bursting can introduce problems when it comes to moving data between on-premise and the cloud, and that the “up-front-investment ‘batch queue’ way of using a cluster” isn’t always compatible with the on-demand way of paying for cloud computing services.
A stepping-stone to cloud
Cloud providers and organisations like Jisc are looking to address some of these issues by negotiating data egress waivers and special pricing agreements for universities.
However, as Dr Woods notes, universities may struggle with a change of payment model.
“The biggest issue is the money side. Universities are terribly slow at moving money around so it’s difficult to work out how the money would make its way from a researcher’s grant to the provider.
“A big question is how do they go from CAPEX to OPEX? Maybe this is why cloud bursting can be a good stepping-stone, as it lets universities effectively turn cloud into a CAPEX investment that’s been prepaid for.
“It’s a way to dip their toes in the water and get their heads around new contracts and procurement models,” he says.
Woods considers cloud bursting a “sticking plaster solution” that will disappear as more organisations trust their data to cloud providers and the option becomes cheaper than on-premise.
“My feeling is that the cost of cloud will be competitive by 2020 and that most universities will be fully on cloud by the end of 2025,” he says.
The iPlayer of cloud deployment
Woods says that cloud bursting, by definition, only offers a slice of the flexibility that full cloud deployment brings, something he suggests can be compared to TV streaming services.
“You get to run interactive simulations, interactive data analysis and publish interactive papers that can be re-run and re-used by others. The best way to describe the difference is that the cloud is the ‘Netflix of simulation’, while on-premise is like watching the BBC following a TV schedule.
“Cloud bursting is like iPlayer – a hybrid mix of terrestrial TV and on-demand streaming that’s unsatisfactory compared to just binge-watching whatever you want on Netflix on demand.”
The importance of engineers
Research software engineers like Woods at the University of Bristol are a relatively new kind of academic, using their DevOps mindset and technical knowledge to support other researchers.
Hamilton believes that this new mindset is going to be essential for research in the years to come, helping “researchers get to grips with the tools available and develop their scientific computing applications.”
In Woods’ experience, cloud providers are frequently only doing work with those institutions that are able to support projects with in-house research software engineers.
“You need to have that skill set within the university to make it work,” says Woods. “Academics want to solve a genome – they have no interest in putting together the supercomputer that will do that. You really need that layer of person to lead the way.
“Those institutions that have people that understand software and hardware – and can bring the two together – will be the ones to prosper and take advantage of everything cloud offers,” he adds.
Image: Shutterstock