Analysis The rapid adoption of container technology, DevOps practices, and microservices application architectures are three of the key drivers of modern digital transformation. Whether built in the cloud, on-premises, or in hybrid environments, containerisation has proved to be significantly more advantageous in terms of scalability, portability, and continuous development and improvement.

More recently, organisations have began to standardise on Kubernetes as their container orchestrator. Tinder recently announced the company is moving their infrastructure to Kubernetes. Soon after, Twitter announced its own migration from Mesos to Kubernetes.

While the reasons behind such a rapid adoption of Kubernetes has been well documented, security issues remain one of the biggest concerns for organisations. When you ignore your container and Kubernetes security, you might find yourself in the headlines for all the wrong reasons—just ask Tesla.

To better understand the trends in container and Kubernetes security and adoption, we conducted a survey of over 200 IT security and operations decision makers in November of 2018. We recently repeated the survey across nearly 400 individuals in security, DevOps, and product teams to gain additional insights into how organisations are adopting container technologies and how their security concerns have evolved.

The results are aligned with the prediction from Gartner that by 2022 more than 75% of global organisations will be running containerised applications in production – a significant increase from fewer than 30% today.

Kubernetes adoption grows by 50% in first half of 2019

Originally built by Google—based on the lessons learned from the Borg and Omega projects—Kubernetes was open-sourced in 2014 as a platform for automating deployment, scaling, and management of containerised applications. Google partnered with the Linux Foundation to form the Cloud Native Computing Foundation (CNCF) to manage the Kubernetes open-source project.

In an early sign of Kubernetes going mainstream, in 2016 Niantic released the massively popular mobile game Pokémon Go, which was built on Kubernetes and deployed in Google Container Engine. At launch, the game experienced usability issues caused by a massive user interest in U.S—the number of users logging in ended up being 50x the original estimation, and 10x the prediction for worst case scenario. By using the inherent scalability advantages of Kubernetes, Pokémon Go went on to successfully launch in Japan two weeks later despite traffic tripling what was experienced during the U.S launch.

Since then, Kubernetes usage has taken off. In our original survey conducted in November of 2018, 57% of respondents said they were orchestrating their containers with Kubernetes, which was at the time already more than any other orchestrator in the market. When we conducted the survey again in July 2019, the percentage of survey respondents who said they use Kubernetes as their orchestrator grew from 57% to 86% – a 50% increase.

And despite the fact that all major cloud providers offer their versions of managed Kubernetes service—with a primary value prop of being easier use—a sizeable portion of Kubernetes users opt for self managing their clusters. This is because self-managed Kubernetes provides greater flexibility to porting an existing Kubernetes application to another environment that’s using Kubernetes.

Kubernetes and container security concerns increase in lockstep with adoption

Security concerns continue to be one of the primary constraints for using containers and Kubernetes. 2019 saw the discovery of several high-severity container and Kubernetes vulnerabilities, including the runC vuln, a k8s privilege escalation flaw, a DoS vuln, and several other vulns that were announced earlier this month as part  of a CNCF audit.

Most respondents identify inadequate investment in security as their biggest concern about their company’s container strategy. Moving to a containerised/microservices architecture introduces several new container and Kubernetes security considerations, and existing security tooling isn’t suitable to address them.

Organisations need dedicated security controls purpose-built for containers, Kubernetes, and microservices, to meet their security and compliance obligations. For example, unlike traditional waterfall method of application development, modern app dev methodologies rely on continuous integration and continuous delivery (CI/CD) where security controls must be deeply embedded in the CI/CD pipeline for it to be effective.

Once again, respondents identified runtime as the life cycle phase that organisations are most worried about; however, most organisations understand that runtime failures are a function of missed security best practices during the build and deploy phases. Not surprisingly, more than half (57%) of respondents are more worried about what happens during the build and deploy phases. In other words, users realise they must "shift left" in their application of security best practices to build it right the first time.

Containers and Kubernetes are running everywhere

One of the interesting findings of the survey report was how diverse container and Kubernetes environments tend to be. While 70% of respondents run at least some of their containers on-premises, 75% of those running on-premises are also running some in the cloud, which means that any workable security solution has to span both environments.

Today, more than half of respondents (53%) are running in hybrid mode compared to 40% at the end of 2018. As a result, the percentage of organisations running containers only on-premises has dropped nearly in half (from 31% to just 17%), while cloud-only deployments have remained steady.

As expected, AWS continues its market dominance in container deployments, followed by Azure. Google comes in third but has gained considerable market share, growing from 18% to 28% over six months.

DevSecOps – not just a catchy term

Traditional security processes can become a barrier when building software using DevOps principals. The increasing complexity of security threats facing enterprises is leading to DevSecOps playing a crucial role.

Across all operations roles, the allocation of management responsibility by role remained consistent, but the jump in those citing DevSecOps as the responsible operator for container security is significant.

When isolating only those survey respondents who are in a security or compliance role, there is an even larger jump in allocation of responsibility to DevSecOps – 42% of respondents in a security or compliance role view DevSecOps as the right organisation to run container security programs.

Final thoughts

Despite the fact that container security is a significant hurdle, containerisation is not slowing down. The advantages of leveraging containers and Kubernetes—allowing engineers and DevOps teams to move fast, deploy software efficiently, and operate at unprecedented scale—is clearly overcoming the anxiety of security concerns.

Organisations are charging ahead with moving their containers to production. The percentage of organisations with more than 50% of their containers running in production environments has increased from 13% to 22% – a growth rate of 70%. In the same six months, those running less than 10% of their containers in production has fallen from 52% to 39%.

Organisations shouldn’t treat security as an afterthought. Unlocking the benefits of cloud-native technologies while maintaining strong security for mission critical application development infrastructure requires protecting the full container life cycle – across build, deploy and runtime phases.

Interested in hearing industry leaders discuss subjects like this and sharing their experiences and use-cases? Attend the Cyber Security & Cloud Expo World Series with upcoming events in Silicon Valley, London and Amsterdam to learn more.

Any discussion of Kubernetes is best started with an understanding of why we need Kubernetes. Kubernetes helps us manage containers, which dominate application development now because they enable portability, faster application development, and greater independence for developers. Once we started using containers in great volume, we needed a way to automate the setup, tear down, and management of containers – that's what Kubernetes does.

The industry has developed other orchestrators, but Kubernetes has emerged as the de facto standard for container orchestration. As much as nearly a year ago, 69% of organisations surveyed by the Cloud Native Computing Foundation (CNCF) were using Kubernetes to manage containers. Kubernetes started with the technical credibility of coming out of Google, and thousands of contributors have increased the robustness, scalability, and security features of Kubernetes.

A series of data points highlight the growth in popularity of Kubernetes. All the cloud providers offer a managed Kubernetes service. Amazon executives highlighted at the company’s recent AWS re:Invent conference that its managed Kubernetes service, AWS EKS, is the fastest growing service AWS has ever released. KubeCon, the industry conference hosted by the Cloud Native Computing Foundation (CNCF) has doubled in attendance every year, with more than 8000 people attending the recent North America conference. And scan any tech job aggregator like Indeed.com and you’ll see 1000s of companies seeking Kubernetes expertise for their IT architecture teams.

The mergers and acquisitions market provides another lens into the popularity of Kubernetes. IBM’s recent acquisition of Red Hat for $34 billion provides another indication of the popularity of Kubernetes. Most industry analysts said OpenShift, Red Hat’s commercial distribution of Kubernetes, drove a significant portion of that valuation. Also, recently, VMware acquired Heptio, which provided another popular distribution of Kubernetes. The purchase price is rumored to be $550 million, an astonishing amount for a company that hadn’t had the chance to generate much revenue yet.

Common misunderstandings about Kubernetes

Despite the massive popularity of Kubernetes, misunderstandings about the platform persist. One centres around how to work with Kubernetes. Most people running open source software have a “DIY” or “do it yourself” perspective – they’re used to digging into software and tuning all the dials and twisting all the knows. So, people often think they should be working directly in the Kubernetes platform. Often that’s not the best approach, however.

As Kubernetes continues its market dominance, organisations need to look for ways to apply a UI layer to the orchestrator to simplify management and security

Building support for high availability (HA) and resilience into Kubernetes, for example, is complicated – these areas provide a great reason to leverage abstraction layers on top of Kubernetes to simplify its operations and make it run in a more robust manner. People talk about Kubernetes needing a UI layer – another interface into it to make kind of needs a UX layer on top. A lot of the managed Kubernetes services provide this abstraction layer for getting the fundamentals set up, like setting up the master, the API server, and resilient data stores.

The same goes for the security layer. Kubernetes has a lot of power controls built in for networking policy enforcement, for example, but accessing them natively in Kubernetes means working in a YAML file. Having tooling on top that visualises the networking layer, as we do in the StackRox platform, makes the power of Kubernetes far more accessible to enterprises in a way similar to how Google Kubernetes Engine makes the control plane of Kubernetes more accessible.

Securing Kubernetes

Kubernetes provides powerful security capabilities around secrets management and network policy enforcement. Digging into network policy enforcement, you can use Kubernetes to limit what resources each asset can reach. By default, Kubernetes allows all assets to talk to all other assets, because the premise of Kubernetes is that it’s meant to aid application development, and as developers craft the microservices that are the building blocks for applications, Kubernetes defaults to letting all those services communicate.

Because the developers are working in Kubernetes, the security team should also use Kubernetes to help tighten down the environment – to limit those communications paths to reduce the blast radius if an attacker got in. Moving to least privilege is a fundamental tenet of security – any person or asset should be allowed to do only the functions necessary to its role and no more. Look for a container security platform that simplifies the process of moving Kubernetes to a least privilege model. The platform should highlight the allowed communications between assets, simulate new network policies, and recommend updated configurations that support least privilege and harden the environment.

Bringing it all together

As Kubernetes continues its market dominance, organisations should look for ways to apply a UI layer to the orchestrator to simplify functionality such as management and security. Despite its inherence security functions, Kubernetes also increases the attack surface, so organisation should look for security platforms that integrate deeply with Kubernetes to make accessing its security functions easier and provide mechanisms for reducing its attack surface.

Interested in hearing industry leaders discuss subjects like this and sharing their experiences and use-cases? Attend the Cyber Security & Cloud Expo World Series with upcoming events in Silicon Valley, London and Amsterdam to learn more.