Academics: Full cloud is like Netflix, bursting is just boring old iPlayer


Keri Allan

12 Jul, 2018

It’s easy to see why cloud bursting – where an application is run in a private cloud or data centre and then ‘bursts’ into a public cloud when demand dictates – could appeal to research universities.

It can provide institutions with an escape valve when their in-house resources are fully committed, helping to potentially speed up research and save costs.

In recent years adoption of cloud computing has been transforming research and education, and although change within academia can be slow, the latest UK Research & Innovation (UKRI) e-infrastructure report has shown a growing interest in community and public clouds.

“We also see that scientific computing teams at universities and research institutes are starting to look very seriously at virtualising their in-house compute clusters,” says Martin Hamilton, a member of UKRI’s Cloud for Research working group.

Although educational researchers tend to “thrash kit within an inch of its life”, Hamilton says there’s a “growing recognition that having the option of running a virtual machine (VM) image can make it easier for researchers to share and re-use code.”

However, there are divergent opinions within the research community as to how best cloud resources should be deployed. While bursting remains a go-to choice for some, others either remain reticent or have avoided the technology entirely in favour of a full-fledged cloud.

Cloud bursting advocates

Two of the world’s biggest champions of the cloud bursting approach are the University of Cambridge and, on the other side of the world, the National University of Singapore (NUS).

“NUS has a wide range of computing requirements, making it impractical for all resources and capacity requirements to be provided in-house,” says Tommy Hor, NUS’ chief information technology officer, speaking to Cloud Pro.

The National University of Singapore deploys cloud bursting to support its research projects

“Our researchers occasionally have ad-hoc service demands that require dedicated computing resources to speed up their work. We have started migrating our in-house pay-per-use service to the cloud, and this will give us greater financial agility and economies of scale.”

The University of Cambridge has gone as far as providing its own cloud bursting capabilities. Its Research Computing Services (RCS) operation has a dedicated private ‘public sector’ cloud designed specifically for scientific and technical computing.

“Researchers from across Cambridge University, plus UK universities and companies, use RCS for cloud bursting,” says Dr Paul Calleja, the university’s director of Research Computing. “Research undertaken includes large-scale genomic analysis for clinical diagnosis and simulations of jet engines.”

Cloud bursting challenges and limitations

But while cloud bursting has potential benefits, there are still problems to be ironed out. This includes interoperability issues between environments, pricing models and security.

“We recently saw a number of Docker images laden with malware removed from the public registry, opening black doors onto users’ machines and running cryptocurrency mining processes,” says UKRI’s Martin Hamilton.

“Things like this take on an even greater significance when we are talking about compute jobs to calculate stresses on airframes, analyse CT images looking for tumours, or model the effect a new drug will have on the human body.”

For the University of Bristol, cloud bursting is seen as a highly restrictive approach to deployment, one that needlessly increases the complexity of a network.

“In my opinion cloud bursting limits the use of the cloud to being just an extension of a local on-premise compute cluster,” says Dr Christopher Woods, leader of the university’s Research Software Engineering group, which is fully in the cloud.

“It also means you get the worst of both worlds – you’re running both a cluster and a cloud, so have twice the complexity.”

He adds that, in his experience, bursting can introduce problems when it comes to moving data between on-premise and the cloud, and that the “up-front-investment ‘batch queue’ way of using a cluster” isn’t always compatible with the on-demand way of paying for cloud computing services.

A stepping-stone to cloud

Cloud providers and organisations like Jisc are looking to address some of these issues by negotiating data egress waivers and special pricing agreements for universities.

However, as Dr Woods notes, universities may struggle with a change of payment model.

“The biggest issue is the money side. Universities are terribly slow at moving money around so it’s difficult to work out how the money would make its way from a researcher’s grant to the provider.

“A big question is how do they go from CAPEX to OPEX? Maybe this is why cloud bursting can be a good stepping-stone, as it lets universities effectively turn cloud into a CAPEX investment that’s been prepaid for.

“It’s a way to dip their toes in the water and get their heads around new contracts and procurement models,” he says.

Woods considers cloud bursting a “sticking plaster solution” that will disappear as more organisations trust their data to cloud providers and the option becomes cheaper than on-premise.

“My feeling is that the cost of cloud will be competitive by 2020 and that most universities will be fully on cloud by the end of 2025,” he says.

The iPlayer of cloud deployment

Woods says that cloud bursting, by definition, only offers a slice of the flexibility that full cloud deployment brings, something he suggests can be compared to TV streaming services.

“You get to run interactive simulations, interactive data analysis and publish interactive papers that can be re-run and re-used by others. The best way to describe the difference is that the cloud is the ‘Netflix of simulation’, while on-premise is like watching the BBC following a TV schedule.

“Cloud bursting is like iPlayer – a hybrid mix of terrestrial TV and on-demand streaming that’s unsatisfactory compared to just binge-watching whatever you want on Netflix on demand.”

The importance of engineers

Research software engineers like Woods at the University of Bristol are a relatively new kind of academic, using their DevOps mindset and technical knowledge to support other researchers.

Hamilton believes that this new mindset is going to be essential for research in the years to come, helping “researchers get to grips with the tools available and develop their scientific computing applications.”

In Woods’ experience, cloud providers are frequently only doing work with those institutions that are able to support projects with in-house research software engineers.

“You need to have that skill set within the university to make it work,” says Woods. “Academics want to solve a genome – they have no interest in putting together the supercomputer that will do that. You really need that layer of person to lead the way.

“Those institutions that have people that understand software and hardware – and can bring the two together – will be the ones to prosper and take advantage of everything cloud offers,” he adds.

Image: Shutterstock

IBM looks to further European cloud expansion with new customers and availability zones

IBM is looking to build upon recent cloud momentum – and the company is expanding in Europe after securing several new customers in healthcare, logistics, energy and more.

The announcements showcase how many prospective IBM clients are utilising the company’s cloud for its artificial intelligence, machine learning and blockchain capabilities. Credit Mutuel, a French bank, is deploying IBM Watson virtual assistants across all of its business lines – run on IBM’s cloud in France with a backup in Germany – while Koopman Logistics, based in the Netherlands, will aim to track and trace consignments across its supply chain through IBM’s blockchain.

Alongside them are Gruppo 24 Ore, a media firm based in Italy, Spanish digital health provider Teckel Medical, UK-based RS Components and lighting solutions firm Osram AG, based in Germany.

Last month at CeBIT, IBM announced 18 new availability zones across the North America, Europe and Asia Pacific regions, among other launches designed around security and privacy. “Our new availability zones and regions architecture is the next step in the evolution of our public cloud platform, and it’ll immediately reinforce and supplement the broad portfolio of infrastructure, platform, and software services that our clients trust to fuel their businesses,” wrote Andrew Hately, VP, DE and chief architect IBM Watson and Cloud Platform at the time.

Speaking to this publication back in February, John Considine, IBM general manager of cloud infrastructure services, cited the importance of extracting data to glean actionable insights for businesses as key – with the emerging technologies forming part of these extraction methods.

“One of our theories leading into the cloud, for the past few years, is that data is enormously important for the enterprises – and given more than 80% of the world’s data is still maintained behind the corporate firewall, our focus has been how… we enable the businesses to take advantage of that data, to combine it with new processing techniques, new data sets, and new capabilities,” Considine said.

“[It’s about] all the things associated with machine learning and deep learning, analytics and bringing all of these things together in a form that allows them to tap into those resources and deliver not only application modernisation, but really even process reinvention,” he added.

It is important to note, as Considine did, how much data is in less-than-easy spots. IBM punted out a similar statistic – that almost 80% of all enterprise data is still managed on the mainframe – when a new partnership with CA Technologies was announced last month.

The DNA of adaptability: How Kubernetes hosts and manages a plethora of different workloads

We live in an environment where everything is changing. Business requirements are changing. User demands are constantly in flux and always evolving. And our infrastructure is also continually changing. Frankly, the infrastructure has always been in a constant state of change, but in the past we pretended that we could get it to a point of stability — that we could reach a state of “done.” Once we finished setting up that totally stable infrastructure, then we could run everything on top with no problems, right?

IT is perpetually in firefighting mode because it treats change as the exception, not the rule. Yet, change is the only constant in our world.

The increased use of containers in recent years has come largely out of the value that the container image brought — having a deployable artefact (the Docker image) that bundled together all dependencies, from the operating system through middleware and the application components, enabled significant advancements in development and operational (DevOps) efficiencies. And the speed with which containers could be launched helped to expand and refine practices around infrastructure as code and immutable infrastructure. But containers alone do not address the need for constant adaptation.

Just like the infrastructure virtualisation that was ushered in by VMware 20 years ago and delivered as a service starting with AWS, the introduction and early adoption of containers has left so much of the way IT works largely unchanged. The use of automation has increased the very infrastructure as code and also resulted in the automation of existing practices — a script to install the docker runtime on three hosts, another to “docker run” three different microservice images, and another to adjust firewall rules to allow traffic through.

This automation still assumes a level of stability; after running the scripts we are “done” and things will just keep humming along. But when, for example, two of the docker hosts are suddenly unavailable, the team is once again in firefighting mode.

Enter container orchestration. The most popular container orchestration system in the industry today is Kubernetes, and with good reason. What makes Kubernetes and other similar systems really shine, is that the system operates in a mode that anticipates constant change.

The Kubernetes model is so effective because it allows a user to say “here’s my desired state. I want 2 instances of my user-facing web page, 3 instances of my catalogue service and 10 instances of my shopping cart service” and Kubernetes just makes it so. It is a declarative model for defining complex systems. Kubernetes constantly monitors the actual state of the system and any time it differs from the desired state it’ll remediate. Kubernetes has change-tolerance built into its DNA.

Another thing that taxes an IT team is the variability they have in their infrastructure. There are different server and storage platforms and an arguably even more varied set of networking solutions. Increasingly, enterprises are going hybrid, leveraging a combination of on premise and public cloud infrastructures. This means that not only must IT teams become experts in the management interfaces for many different clouds, the scripts they are writing to automate the myriad of different tasks must be written and maintained for each different infrastructure.

Kubernetes addresses this by providing abstractions over the top of the varied infrastructure assets, allowing Kubernetes consumers to leverage that infrastructure through common entities such as workloads (pods and replica sets), networks and network policies (NetworkPolicy) and storage (Storage Classes, Persistent Volume Claims). Kubernetes is designed to adapt to the infrastructure.

Finally, and perhaps the thing that gets me most excited about Kubernetes, is its extensibility. Out of the box Kubernetes already delivers a whole host of resource types — pods, storage classes, roles and so much more — and functionality to lifecycle manage those resources — replica sets, daemon sets, stateful sets and more, but particularly when it comes to stateful workloads like a database, cache, or indexing services, each one has unique needs. The way that Mongo DB protects data that it stores is quite different from the way that MySQL does, for example. Kubernetes allows for custom resource definitions (CRDs) and associated behaviours (one of the most popular means for this is via operator) to be added, effectively extending the reach of the platform. That is, Kubernetes can be adapted to host and manage a virtually endless set of different types of workloads.

When you look at the abstractions that Kubernetes provides it’s easy to think of it as a new API for infrastructure — its base primitives are compute, storage, and network, just as with server virtualisation. It is its tolerance for change that sets it apart.

Who is Kubernetes for?

Just like Docker and server virtualisation before that, initially Kubernetes has captured the mindshare of the developer. Particularly now that those developers are increasingly responsible for keeping their software running well in production, having an intelligent, autonomous system that helps them with those operational tasks is hugely valuable. App operations involves not only the day 1 task of deployment but also maintenance in the face of infrastructure changes, security vulnerabilities, and more.

Just as enterprise IT provides centralised, secure, compliant, and resilient virtualised infrastructure environments, the time has come for providing secure, compliant, and resilient container platforms.

It’s rare these days that I speak to an enterprise that does not have some, sometimes substantial, presence of container-centric efforts going on. Often it has grown out of a development group that has built its practices around containers. They’re building docker images for their apps but, because the enterprise does not already have a production platform that can run those images, the same app teams are managing the container platform. Just as enterprise IT provides centralised, secure, compliant, and resilient virtualised infrastructure environments, the time has come for providing secure, compliant, and resilient container platforms.

As Kubernetes becomes mission critical

With the capabilities that it brings for running and managing mission-critical workloads, Kubernetes itself must be equally resilient to change. If a security vulnerability is found that requires Kubernetes be upgraded, it must be patched quickly and with zero downtime for the workloads it is hosting.

If application capacity requirements suddenly spike, the Kubernetes capacity must be quickly expanded to meet the need. When the spike has passed, Kubernetes needs to be right-sized again to keep IT infrastructure costs in check.

These are exactly the challenges that Kubernetes is addressing for containerised workloads. The key is to use the same principles and techniques that Kubernetes uses for workloads to manage Kubernetes itself.

Gmail confirms private Gmail messages can be read by third parties


Bobby Hellard

4 Jul, 2018

Google has responded to The Wall Street Journal highlighting how common it is for third-party developers to view user Gmail messages.

The publication had previously reported that Google has a “dirty secret” by allowing developers to sift through Gmail due to users granting permission for third parties to do so. 

Google said it makes it possible for applications from other developers to integrate with Gmail, such as email clients, trip planners and customer relationship management systems so that users have options around how they access and use email.

As a result of this, private messages in Gmail can be read not only by third-party systems but also by humans not intended to be the recipients of such emails.

The search giant stressed that it continuously works to vet developers and their apps that integrate with Gmail before it opens up them for general access. It said it also provides both enterprise admins and individual consumers transparency and control over how their data is used.

“A vibrant ecosystem of non-Google apps gives you choice and helps you get the most out of your email,” said Suzanne Frey, Google Cloud’s director of security, trust and privacy.

“However, before a published, non-Google app can access your Gmail messages, it goes through a multi-step review process that includes automated and manual review of the developer, assessment of the app’s privacy policy and homepage to ensure it is a legitimate app, and in-app testing to ensure the app works as it says it does.”

In order to pass Google’s review process, non-Google apps must meet two key requirements. Firstly, apps should not misrepresent their identity and must be clear about how they are using your data and secondly, they must only request relevant data they need for their specific function, nothing more, and be clear about how they are using it.

The WSJ story did not unearth any wrongdoing from third-party apps or services using Gmail, but it has shone a light on a previously discreet industry practice that is under heavier scrutiny since Facebook’s Cambridge Analytica data privacy scandal.

Google is now taking steps to actively defend its own data management and user privacy practices to convince users and businesses that is a responsible steward of sensitive user data.

Picture: Google

Force Quit on a Mac: 3 Easy Ways to Close Frozen Applications

Ok, I get it. There is no equivalent to the PC’s Ctrl+Alt+Del shortcut on a Mac® to force quit an application. So how do I quit that annoying program that’s not responding? Luckily, Apple® has you covered and gives you multiple options. The shortcut actually exists, and moreover, there are a few other extremely convenient […]

The post Force Quit on a Mac: 3 Easy Ways to Close Frozen Applications appeared first on Parallels Blog.

Force Quit on a Mac: 3 Easy Ways to Close Frozen Applications

Ok, I get it. There is no equivalent to the PC’s Ctrl+Alt+Del shortcut on a Mac® to force quit an application. So how do I quit that annoying program that’s not responding? Luckily, Apple® has you covered and gives you multiple options. The shortcut actually exists, and moreover, there are a few other extremely convenient […]

The post Force Quit on a Mac: 3 Easy Ways to Close Frozen Applications appeared first on Parallels Blog.

IBM lands six major European cloud deals


Clare Hopping

4 Jul, 2018

IBM has announced partnerships with six European firms using its cloud services to grow their AI, blockchain and analytics businesses.

It will work closely with Dutch logistics firm Koopman Logistics to build its track and trace solution using IBM’s blockchain technology. Koopman transports consignments across Europe and it needed to implement a secure technology to replace its paper-based tracking process. Now it’s using IBM’s blockchain to track consignments using digital records.

The second partnership IBM announced is with Italian multimedia organisation Gruppo 24 Ore, which is using the company’s IBM Watson AI services hosted on the IBM Cloud to help tax professionals respond to questions about the Italian tax coding system. IBM Watson was implemented to process 1.5 million documents relating to the financial system and glean the data it needs to advise professionals.

French bank Crédit Mutuel is also using IBM Watson on IBM’s Cloud environment in France (with a back up in Germany) to power its virtual assistants that help the company’s 20,000 relationship managers advise their customers.

Digital health company Teckel Medical is running its digital health checker on IBM Cloud, while RS Components is making use of IBM’s Cloud platform, building its peer-to-peer marketplace in IBM’s London Cloud Garage, enabling startups to promote, test and sell their inventions online.

Finally, IBM has announced a partnership with Osram AG, a lighting solutions company that has switched its operation to a digital environment powered by IBM Cloud, resulting in greater operational savings and flexibility.

“Enterprises across Europe are gravitating to the IBM Cloud because it helps them modernize their existing infrastructures by gaining access to exciting technologies like AI, blockchain, IoT, analytics and more,” said Sebastian Krause, general manager IBM Cloud Europe. 

“At the same time, these companies value IBM’s deep industry and business process expertise, along with IBM’s commitment to the responsible management of their enterprise data.”

Image credit: IBM 

G-Cloud 10 arrives with 3,500 suppliers


Joe Curtis

3 Jul, 2018

G-Cloud 10 is now live, with more than 3,500 suppliers listed on the latest iteration of the framework.

More than 90% of the 3,505 companies who’ll be competing for public sector contracts are SMBs, according to the Crown Commercial Service (CCS), and there are 649 more vendors listed on the new version of G-Cloud than there were on its previous incarnation.

“Small businesses are the backbone of our economy, delivering innovative solutions in partnership with the public sector, fuelling economic growth and supporting the delivery of efficient, effective public services that meet the needs of citizens,” said Oliver Dowden, minister for implementation, who oversees CCS.

“The success of G-Cloud demonstrates how we are breaking down the barriers for SMBs who want to supply to government.”

Government figures show that G-Cloud has racked up £3.1 billion in sales since its launch in 2012, with 48% of that going to SMBs.

But SMBs have criticised the framework for its inability to allow them to change their prices on a given iteration – e.g. G-Cloud 9 – if their own costs increase.

Nevertheless, suppliers welcomed the launch of the latest version, with UKCloud founder Simon Hansford, whose firm has listed services since G-Cloud’s inception, saying: “With each iteration the framework has seen enhanced functionality and an increased volume of transactions as it has supported a thriving ecosystem of UK tech SMBs that have succeeded in winning business through it.”

G-Cloud allows public sector departments to put cloud contracts up to tender to a wider pool of bidders that are often smaller than the big tech firms that have historically benefitted from UK government spending.

The arrival of G-Cloud 10 was in doubt for some time, after the government originally said G-Cloud 9 would remain in place until May 2019, rethinking its decision earlier this year.

A new framework means suppliers can list new services they provide and adjust their prices.

Google Cloud investigates automated customer service practices after complaint

Google Cloud Platform has said it will conduct a detailed review of its abuse prevention processes after a customer complained about its treatment.

The unnamed customer, who works in the renewable energy industry, wrote in a Medium post that the company was a few days away from ‘losing everything’ after Google’s automated system pinged it for questionable activity.

Those who get pinged will receive a variety of emails – a ‘barrage’, as the customer put it – detailing that each service is down, the payments account is temporarily closed, and what needs to be done about it. Chat support is switched off, with a warning that unless a picture of the credit card and a government-issued photo ID of the card holder is uploaded within three days, the project will be deleted.

The customer warned about the consequences if the card holder – in this instance, the CFO – was not available, and around the automated nature of the system.

“I understand Google’s need to monitor and prevent suspicious activity. But how you handle things after some suspicious activity is detected matters a lot,” the post explains. “You need a human element here – one that cannot be replaced by any amount of code/AI. You just can’t turn things off and then ask for an explanation.”

In a statement, posted by Brian Bender, Google Cloud Platform engineering support regional lead, Google said it will be re-evaluating data sources used to assess potential fraudulent activity, implementing additional mechanisms for suspect accounts, and improve how it communicates account warnings. “Protecting our customers and systems are a top priority,” the statement added. “We sincerely apologise for this issue and are working quickly to make things better, not just for this customer but for all GCP customers.”

Given Google’s rise in the cloud infrastructure arena over the past 12 months – the company was listed in the leaders’ section for public cloud IaaS by Gartner in May – it is interesting to note that, for this particular customer, this was the first project built by them entirely on Google’s cloud. The customer was previously an AWS house; and while there was no technical reason cited for the change – both are ‘on-par’, as the customer put it – there was a note on the differing customer experiences.

“In our experience AWS handles billing issues in a much more humane way,” the customer explained. “They warn you about suspicious activity and give you time to explain and sort things out. They don’t kick you down the stairs.”

Other issues were sorted. Mike Kahn, Google Cloud customer engineer, noted the importance of having an enterprise user account rather than a consumer one – yet another commenter described this approach as having 'borderline contempt' for customers.

UK gov using emotion detecting AI for digital content


Bobby Hellard

3 Jul, 2018

The UK government is using a type of artificial intelligence that can detect emotion on social media to measure and understand how people feel on certain topics.

Web science firm FlyingBinary has released the “artificial emotional intelligence” service to the government’s G-Cloud marketplace, in partnership with emotional AI recognition company Emrays B.V.

“The web has become a noisy space as online content grows exponentially,” said Professor Jacqui Taylor, CEO of FlyingBinary. “Where once tools were in the hands of a social team it is increasingly difficult for humans using social media monitoring to understand the signals about a brand, initiatives, good news or issues.”

“This service uses AI technology to understand digital content from an emotional perspective and how resonant this is with an online audience before content is shared online.”

The two companies have deployed the artificial emotional intelligence engine as part of a newly awarded G-Cloud 10 service built for the UK government.

FlyingBinary has vast experience in web science, GDPR and security and has thus far supported almost 40,000 government organisations, helping them to understand the dynamics of emotions on the social web, and in the mass media space.

Emrays emotion AI, on the other hand, is said to be able to detect more than 20 distinct emotions in any digital content, which it says can help companies and governmental organisations measure and understand how people feel about any topic, ranging from companies, brands and concepts.

The engine learns collective patterns of emotional reactions to digital content publicly available on the web. The emotion AI analyses and “feels” content on par with humans, based on more than one billion data points it has already been trained on. It uses a diverse set of human emotions, such as love, anger, surprise and shock.

Taylor added that no personal data is used by the AI engine and that it focuses instead on the content itself and the human emotion expressed.

FlyingBinary was one of thousands of small businesses that won the chance to bid to supply cloud computing services to government bodies through the major government procurement framework.

G-Cloud 10, which is predicted by the government to have a potential worth of £600 million, gives the central government, local councils, NHS Trusts and other public sector bodies a way to purchase cloud-based services, such as web hosting from a single, central website.

Oliver Dowden, the Minister for Implementation, said: “Small businesses are the backbone of our economy, delivering innovative solutions in partnership with the public sector, fuelling economic growth and supporting the delivery of efficient, effective public services that meet the needs of citizens.

“The success of G-Cloud demonstrates how we are breaking down the barriers for SMEs who want to supply to the government.”

Picture: Shutterstock

The cloud news categorized.