Keumars Afifi-Sabet

Container orchestration platforms exist to make container use a whole lot easier. Running any application on a container will make it portable. However, when the time comes to scale or add services, you’re going to run into problems without a platform to manage and stitch it all together, and it will quickly become too difficult to handle.

When it comes to AWS, there are three main options – each with pros and cons. The choice you make will ultimately come down to your business needs and ongoing maintenance capabilities.

To help you decide, here are the pros and cons of each managed service:

ECS: the native choice

Elastic Container Service (ECS) was AWS’ first offering for managed container orchestration. For many, this is the easiest option, and it certainly has the least amount of components to get familiar with.

As a heavily integrated orchestration platform, it’s a great choice for anyone happy with the AWS ecosystem and who wants the benefits and familiarity of AWS services and support. It’s also cost-effective, as you don’t have to pay for the control plane and can use the built-in AWS code tools as well as enjoy fine-grained identity and access management (IAM) for Services and Tasks.

When your business wants to deploy an application onto ECS, the operations can be defined for each application individually, dictating for example which containers have access to S3 and which don’t.

When is ECS not the best choice?

As a proprietary AWS solution, cloning your applications to a different cloud vendor won’t be a simple task if you go with ECS. In addition, the orchestration platform has limited support for routing, currently supporting only path-based routing, and not host-based or header-based routing. Another factor to consider is that ECS is slower to respond to state changes than the others in the Big Three, so if you’re looking to a highly performant solution – it’s not going to be the right fit.

Who is ECS good for?

If you are looking for simplicity with good value for your investment, and these factors aren’t deal breakers for you, ECS is a great beginners option, and perfect for any business without experienced DevOps to operate their orchestration. I recommend it if you have a limited amount of services (<10) to deploy on the cloud. Without the bells and whistles which make the solution more complex, you might find ECS to be preferable for your company.

EKS: the Kubernetes choice

EKS is AWS’ offering of Kubernetes, the open-source container orchestration platform that has become popular. As EKS is a managed service by Amazon, this eliminates a lot of the hassle that comes with the initial installation and maintenance of Kubernetes going forward. Amazon EKS runs upstream Kubernetes. It’s not a different flavor, so you get the same functionality as if you created your own Kubernetes cluster, which makes the platform easy to clone if you want to run multi-cloud in the future.

As an open-source platform, EKS has the benefit of thousands of developers that are working on its technology constantly, actively contributing to functionality and new features. Unique selling points worth mentioning include namespace isolation, where you can split your cluster with logical boundaries, for example limiting developers to using a specific amount of resources of the cluster. Moreover, it provides the ability to run cron jobs and stateful workloads.

EKS offers a much faster deployment time than ECS, with results in a few seconds, allowing you to deploy several times a day and feedback fast for changes. Everything can be declared using the kubectl command line tool, and there are plenty of integrations. These include service-to-service communications and native scaling of both Pods and Worker Nodes, enabling your developers to focus on their business logic and deliver new features. I’d also highlight Helm, a package manager that provides the ability to bundle together several applications or business logic for deploying and updating a whole unit in one piece.

What should you watch out for with EKS?

It’s important to realize that Kubernetes isn’t the right choice for everyone. Your business will have the added cost of the control plane each month, and there is a much steeper learning curve than you would experience with ECS, and currently fewer integrations with AWS overall. Unlike ECS, the IAM to AWS is not built-in, so your developers or DevOps will need to install additional tools for this functionality.

The other serious limitation is pod density, a unique issue to EKS. Every container (pod) is bound to a certain private IP in your VPC, and if your application utilizes many replicas or microservices your cluster will scale but not due to the fact that your instance ran out of CPU or memory, rather that your instance ran out of IPs to allocate to the worker nodes.

This results in additional costs, and can be limiting as your developers will have limited IPs for smaller size instances used by the worker nodes. If your microservices scale quickly and by high volume, this is an important factor to consider.

Who is EKS right for?

The critical question here is, once the installation is complete, who is going to be responsible for taking ownership of it? Managing and maintaining EKS needs dedicated specialists, and if you don’t have the manpower, another option might be a better fit.

Fargate: The container on-demand choice

With Fargate, it’s a whole new game. You don’t have to create your own control plane or instances, there are no clusters needed, no need for infrastructure upgrades or maintenance. Instead, you specify how many resources you want to use, and pay as you go. This gives you the opportunity to focus on the design and build of your applications, rather than spending time worrying about the underlying infrastructure.

The best thing about Fargate is rapid horizontal scaling, the ability to scale on demand. Developers simply create containers and deploy to the Fargate service. Easy set-up, no learning curve.

Fargate is not suitable for stateful workloads, requiring your application to be stateless, which is one of the main reasons why some companies wouldn’t choose Fargate. Additionally, although the ability to scale to tens of thousands in no time is exciting, in reality not many businesses need this functionality, making the cost harder to justify.

Who would be a good fit for Fargate?

Only you can know if your budget suits choosing Fargate rather than investing in a DevOps team, and if the benefits of scaling on demand are worth the higher cost. This is most likely if you have just a handful of services.

For many, Fargate works well as a hybrid solution, allowing your applications to scale where necessary for on-demand tasks rather than using it 24/7. Another consideration is to isolate those workloads with sharp surges in resource usage and run them on Fargate to minimize the impact on the performance of your ECS or EKS clusters.

In closing, EKS is an increasingly popular choice for container orchestration, but that doesn’t mean it’s the right solution for your business needs. Remember, the more features and functionality there are, the more complexity it introduces, and the more resources you will need to manage your ecosystem. It’s in your best interest to make sure that you actually need the bells and whistles before choosing the shiny new thing.

Read more: AWS, Azure or Google: Do the differences between cloud providers really matter?

Interested in hearing industry leaders discuss subjects like this and sharing their experiences and use-cases? Attend the Cyber Security & Cloud Expo World Series with upcoming events in Silicon Valley, London and Amsterdam to learn more.

Demand for computer servers, disk storage systems, and networking hardware deployed within an enterprise hybrid cloud environment remains strong. Moreover, the investment in non-cloud on-premises infrastructure seems assured by the CIO and CTO need to deliver superior security and compliance with IT regulatory requirements in several key industries.

According to the latest worldwide market study by International Data Corporation (IDC), vendor revenue from sales of IT infrastructure products for cloud environments — including public and private cloud — declined 10.2 percent year-over-year in the second quarter of 2019 (2Q19), reaching $14.1 billion.

Cloud IT infrastructure market development

IDC also lowered its forecast for total spending on cloud IT infrastructure in 2019 to $63.6 billion, down 4.9 percent from last quarter's forecast and changing from expected growth to a year-over-year decline of 2.1 percent.

Vendor revenue from hardware infrastructure sales to public cloud environments in 2Q19 was down 0.9 percent compared to the previous quarter (1Q19) and down 15.1 percent year over year to $9.4 billion.

This segment of the market continues to be highly impacted by demand from a handful of hyperscale cloud service providers, whose spending on IT infrastructure tends to have significant upward and downward swings. That ongoing fluctuation creates volatility for the IT infrastructure vendors.

After a strong performance in 2018, IDC expects the public cloud IT infrastructure segment to cool down in 2019 with spending reaching $42 billion — that's a 6.7 percent decrease from 2018. Although it will continue to account for most of the spending on cloud IT environments, its share will decrease from 69.4 percent in 2018 to 66.1 percent in 2019.

In contrast, spending on private cloud IT infrastructure has shown more stable growth since IDC started tracking sales of IT infrastructure products in various deployment environments. In the second quarter of 2019, vendor revenues from private cloud environments increased 1.5 percent year-over-year reaching $4.6 billion. IDC expects spending in this segment to grow 8.4 percent year-over-year in 2019.

 

Overall, the IT infrastructure industry is at crossroads in terms of product sales to cloud vs. traditional IT environments. In 3Q18, vendor revenues from cloud IT environments climbed over the 50 percent mark for the first time but fell below this important tipping point since then.

In 2Q19, cloud IT environments accounted for 48.4 percent of vendor revenues. For the full year 2019, spending on cloud IT infrastructure will remain just below the 50 percent mark at 49 percent.

Longer-term, however, IDC expects that spending on cloud IT infrastructure will grow steadily and will sustainably exceed the level of spending on traditional IT infrastructure in 2020 and beyond.

Spending on the three technology segments in cloud IT environments is forecast to deliver growth for Ethernet switches while computing platforms and storage platforms are expected to decline in 2019.

Ethernet switches are expected to grow at 13.1 percent, while spending on storage platforms will decline at 6.8 percent and compute platforms will decline by 2.4 percent. Compute will remain the largest category of spending on cloud IT infrastructure at $33.8 billion.

Sales of IT infrastructure products into traditional (non-cloud) IT environments declined 6.6 percent from a year ago in Q219. For the full year 2019, worldwide spending on traditional non-cloud IT infrastructure is expected to decline by 5.8 percent, as the technology refresh cycle driving market growth in 2018 is winding down this year.

By 2023, IDC expects that traditional non-cloud IT infrastructure will only represent 41.8 percent of total worldwide IT infrastructure spending — that's down from 52 percent in 2018. This share loss and the growing share of cloud environments in overall spending on IT infrastructure is common across all regions.

Most regions grew their cloud IT Infrastructure revenues in 2Q19. Middle East & Africa was fastest growing at 29.3 percent year-over-year, followed by Canada at 15.6 percent year-over-year growth. Other growing regions in 2Q19 included Central & Eastern Europe (6.5 percent), Japan (5.9 percent), and Western Europe (3.1 percent).

Cloud IT infrastructure revenues were down slightly year-over-year in Asia-Pacific (excluding Japan) (APeJ) by 7.7 percent, Latin America by 14.2 percent, China by 6.9 percent, and the USA by 16.3 percent.

Outlook for cloud IT infrastructure investment

Long-term, IDC expects spending on cloud IT infrastructure to grow at a five-year compound annual growth rate (CAGR) of 6.9 percent, reaching $90.9 billion in 2023 and accounting for 58.2 percent of total IT infrastructure spend.

Public cloud data centres will account for 66 percent of this amount, growing at a 5.9 percent CAGR. Spending on private cloud infrastructure will grow at a CAGR of 9.2 percent.

Interested in hearing industry leaders discuss subjects like this and sharing their experiences and use-cases? Attend the Cyber Security & Cloud Expo World Series with upcoming events in Silicon Valley, London and Amsterdam to learn more.

Keumars Afifi-Sabet

Bobby Hellard

Digital Realty has announced it is to acquire Interxion in an $8.4 billion (£6.52bn) transaction which is touted as the largest data centre deal in history.

Interxion, a provider of European colocation data centre services, will join the Digital Realty team to create a ‘leading pan-European data centre presence’. The companies estimate the combined entity will cover more than two thirds of the GDP in Europe, with Interxion currently holding 53 carrier- and cloud-neutral facilities across 11 European countries.

The companies claim various benefits of the transaction: building upon Digital Realty’s record in hyperscale development and its associated benefits for enterprise customers is cited, as well as solving the public and hybrid cloud architectural requirements of a global customer base.

Digital Realty’s CEO, A. William Stein, will serve as chief executive of the combined company, while Interxion CEO David Ruberg will head up the EMEA business.

The deal beats Digital Realty’s own record when it purchased DuPont Fabros for $7.6bn in 2017.

“This strategic and complementary transaction builds upon Digital Realty’s established foundation of serving market demand for colocation, scale and hyperscale requirements in the Americas, EMEA and Asia Pacific and leverages Interxion’s European colocation and interconnection expertise, enhancing the combined company’s capabilities to enable customers to solve for the full spectrum of data centre requirements across a global platform,” said Stein in a statement.

Consolidation of the market was to be expected, according to analyst firm Synergy Research. In a note at the beginning of this year, the company said it expected to see ‘a lot more’ data centre M&A over the coming five years. Analysis of the transactions over the past two years saw a significant dip in overall value in 2018 compared with 2017, despite a greater number of deals.

Interested in hearing industry leaders discuss subjects like this and sharing their experiences and use-cases? Attend the Cyber Security & Cloud Expo World Series with upcoming events in Silicon Valley, London and Amsterdam to learn more.

This year marks 10 years since the term DevOps was first coined, during a now legendary presentation at a Toronto tech conference.

Anyone who’s seen the 90s brawler film Fight Club will know—the first rule of Fight Club is: you don’t talk about Fight Club. All those years ago, IT professionals weren’t part of Fight Club—but a small number formed their own underground DevOps club. But even though the first rule of DevOps Club is, “always talk about DevOps,” it’s taken a decade to catch on in a significant way.

Finally, DevOps doesn’t seem so underground. And like many actually-good-for-you approaches, the small movement of IT professionals pursuing a better vision for operations management, is being consumed by the vendor hype cycle. Capitalising on this, many vendors have plastered their websites with DevOps SEO terms to market a multitude of dubious DevOps-in-a-box solutions.

Those truly familiar with the process will already know—DevOps needs to be initiated internally. Its success doesn’t lie in third-party solutions; it lies in people and culture shifts. Despite lots of companies today claiming they’re “doing DevOps,” many still experience the problems solved by agile ways of working. In SolarWinds' 2018 IT Trends Report, IT pros named inadequate infrastructure and organisational strategy as the top two barriers to achieving optimal IT performance.

The good news is, hype cycles around trendy IT terms and solutions inevitably come to an end. In the next few years, when vendors and commentators have shifted their attention to the next buzzword, DevOps will be given room to grow organically and reach its true potential.

The waiting game

Once industry commentators have had their field day and moved on from DevOps, the developers who created the concept for themselves in the first place can own it once again. It’s these developers who, from firsthand experience, know about the IT challenges and goals today’s enterprises face. Once DevOps is aligned to address these common problems, it’ll become a concept you can better apply in IT environments.

All enterprises must understand, however, DevOps—real DevOps—will take time. Developers and Ops work in fundamentally different ways, which could spark arguments and disagreements in the first days, even months, of DevOps working. Management should hold their nerve and let these initial difficulties run their course—because once they have, these teams will be well on the way to DevOps nirvana.

DevOps is a process made by technologists, for technologists—external pressures will only hinder the process. Let’s take virtualisation as a case in point. Virtualisation was once the “cool” new concept every enterprise wanted to adopt. Like the situation today, vendors were putting pressure on data centre professionals to prematurely introduce virtualisation with subpar software and servers. Technologists resisted and waited until they were ready—these wholesale technological shifts aren’t simply “bolt-on” solutions. Now we can’t think of IT operations without virtualisation.

Despite its uniquely transformative potential, the adoption of DevOps is ultimately following the same blueprint as many new approaches before it. There’s been the successful early adopters, and increasingly (some) enterprises have taken up the charge. The suggests we’re moving closer to the hype decline phase, where even late adopters may finally get to enjoy the same longer weekends and faster change rates of the early adherents a decade ago.

The new standard

But where should DevOps be for you? And what would that look like?

In the next five years, traditional operations teams will discover at least a few DevOps practices help solve complex problems thrown up by new technologies. Better, it’s not necessary to accept the full dogma of DevOps or Agile to see systems benefits and new ways of working like DevOps.

The benefits of these remodeled IT environments are only growing in popularity. A 2018 survey of 2,400 developers and general IT professionals, conducted by the Cloud Native Computing Foundation, found the use of serverless technology had grown 22% in one year. As enterprises look to reap the benefits of digital transformation and cloud native technologies, the adoption rate of DevOps will only grow.

It’s not surprising cloud native and serverless environments are where DevOps have shown significant early results. Automation of standard tasks and the breaking down of data silos is precisely what’s needed to speed up delivery time and render critical problems easier to solve. Once enterprises begin to witness the impact of DevOps in these high-visibility scenarios—they’re likely to see value and want to extend DevOps ways into other areas of IT.

As adoption picks up, IT professionals and developers will increasingly expect all of their tools and technologies to be totally compatible with DevOps. The runaway success of tools like Gradle, Git, and Jenkins reflects the growing numbers of IT professionals who are prioritising tools speeding up deployment times and facilitate the collaborative ways of working that are fundamental to DevOps.      

It’s not too soon for vendors to start catering to DevOps—in fact, it’s high time they did. The annual Google Cloud Accelerate State of DevOps report analyses data from thousands of IT pros to provide a health check of the DevOps industry. This year’s report found the self-reported number of what Google classes as “elite DevOps performers” has almost tripled, now at 20% of all organisations.

All of these organisations in part identify by tools that compliment how their teams work. The rise of DevOps isn’t just inevitable—it’s evident. Vendors need to stop using “DevOps” as SEO juice and manfully incorporate DevOps-friendly features, especially for established enterprise technologies.

Give and take

Bringing about the future of DevOps will require IT professionals to make some unnatural adjustments. They need to learn and embrace automated deployment pipelines and learn at least the basics of code. But fundamentally, IT professionals need to come to terms with their own reservations about automation and experiment. That’s the path to understanding the value that DevOps brings.

Many hype-weary IT professionals might feel threatened by the automated solutions accompanying DevOps, and understandably so. But checking with peers, most discover even just the automation techniques of DevOps presents greater career opportunities. Increasingly, IT professionals are becoming accustomed to automating their “work” and realise the skills benefits of doing so. Some even report they’d never go back to waterfall operations again. Perhaps it’s just the reduction of overnight and weekend maintenance windows.

The hype cycle engulfing DevOps is finally subsiding, and this is a Great Thing. One by one, developers and operations engineers are discovering DevOps isn’t yet another overlay requirement like ISO, but a set of culture and process changes that realigns IT to meet the goals of today’s businesses. When we see a vendor slide mention CDV (Continuously Delivered Value™) we’ll know we’re on the way. IT teams have always understood this, and who doesn’t like tools increasing speed to market, greater innovation, and being associated with services users appreciate.

Read more: DevOps learnings: Why every successful marriage requires a solid foundation

Interested in hearing industry leaders discuss subjects like this and sharing their experiences and use-cases? Attend the Cyber Security & Cloud Expo World Series with upcoming events in Silicon Valley, London and Amsterdam to learn more.

Alphabet posted earnings which missed analyst expectations – yet as the company’s ‘other revenues’ bucket continues to grow, the word on Google Cloud remained positive from the executives.

Other revenues, of which Google Cloud is a part – the company continues to not show its full hand – reached $6.42 billion (£5.01bn) for Q319, an increase of 38.5% year on year and a rise of almost 4% from the previous quarter.

Profit for the overall business declined 23%, with the earnings of $10.12 per share falling well below Wall Street expectations of $12.42. However, total revenues of $40.5bn were seen as positive, with advertising revenues up 17% from this time last year.

Google Cloud’s highlights in Q3 were varied and legion. In terms of product and footprint, the company continued its European expansion with a launch in Poland last month, while the release of Dataproc on Kubernetes in the same month solidified Google’s leadership at container management for an enterprise level. On the partnership front, deals were struck in August with VMware, extending the companies’ collaboration, as well as with enterprise blockchain provider Cypherium.

Alphabet CEO Sundar Pichai was keen to evangelise the gains made by Google Cloud, particularly noting ‘customer momentum across multiple areas on [Google Cloud CEO] Thomas [Kurian’s] leadership’ to analysts.

Pichai elaborated on how Google Cloud customers fit in to other emerging areas when fielding an analyst question around quantum computing, an area in which Google declared ‘supremacy’ last month. “This is an important tool in the arsenal,” said Pichai. “While quantum will take many years to really start making a difference, we want to be at the cutting edge of driving it.

“I do think over time for sure, we do see a lot of interest from Cloud customers, particularly in cutting-edge verticals about quantum computing – so that’s an area where I think [we] will participate in as a business,” added Pichai.

Analysts had previously been asking Google to disclose specific figures around its Cloud business. In Q1, Goldman Sachs analyst Heather Bellini posed that very question, only to get a committed non-committal in response. This is understandable; as each of the cloud infrastructure giants count their beans with different methods, specific numbers may be seen as an apples versus oranges comparison. Microsoft continues to give Azure revenues in terms of percentages rather than an exact number, while AWS – which does give specifics – hit almost $9bn in its most recent quarter.

You can read the full Alphabet earnings release here.

Interested in hearing industry leaders discuss subjects like this and sharing their experiences and use-cases? Attend the Cyber Security & Cloud Expo World Series with upcoming events in Silicon Valley, London and Amsterdam to learn more.

Bobby Hellard

When organisations migrate to the cloud they have an application problem: deciding which apps to migrate and in what order as well as which ones to reconfigure as cloud-native. 
Once in the cloud they have a data problem: budgets that are flat or in decline and data volumes that are growing exponentially.

Where people go wrong is thinking ONLY about the application problem in advance. All too often when we cross the road we look left or right when we should be looking both left AND right.

It is wrong to think of cloud as a commodity. Cloud price wars have eased and the pace of decline in prices for cloud compute and storage has slowed almost to a halt. The reason for this shift is market maturity, with people having more faith in the cloud model than they once did. 451 Research analysts have said that the cloud has not yet become a commodity and as such, the cloud market is "not highly price-sensitive" at the moment, despite businesses wanting to get the best deals they possibly can.

CIOs are often overly focused on the cost of compute, where the cost of compute is not decelerating as fast as it did during the height of the price war. However, they should also be focusing on the cost of object and block storage. Prices for storage may have more scope to fall than for compute, but if you’re being charged for data and your data is growing exponentially then you have a problem.

Few, if any, organisations are throwing away any of their old data and new data is being added at an exponential rate – a rate that will only increase with 5G and IoT. This exponential explosion in the volume of data is a real problem.

Many of us are some way down the cloud path. Most of the initial gains that we experienced from moving to the cloud came from the low hanging fruit. Such gains came from transformational projects that could deliver immediate improvements in service or reductions in cost, or that addressed the most immediate challenges at hand.

Typically, though, we put off the biggest challenges, those that would require either organisational transformation, including interdepartmental collaboration and structural reform, or technological transformation, including re-engineering or refactoring applications from the ground up. 

For many organisations, the easy gains have already been realised and the real challenges lie ahead.

Indeed, many of the easy gains came from virtualised applications that could easily be ‘lifted and shifted’ to the cloud and connected to cloud-based block storage. Now with budgets that are flat or in decline and data volumes that continue to grow, there is a looming crisis relating not only to the ongoing cost of data storage, but also to the cost of both ingress and egress [The cost of moving data and applications into the cloud (ingress) or move anything out of the cloud (egress) or even moving it between regions].

Things should be fine for those that ‘looked both ways’ and ensured that such costs were calculated in advance and built into the business case. However, those that ‘only looked left’ will have been hit from the right by unexpected costs that are outpacing the growth of their budgets. Indeed, all too many CIOs have gone from being unintelligent in their use of data in legacy environments to unintelligent in their use of data in the cloud.

If you, like many, have been overly focused on the cost of infrastructure and compute, but as quickly as savings have already been realised (and the easy ones have all been realised already), you have started experiencing exponential data growth and with it cost, and you’re locked in by egress charges, then you’ve got a BIG problem. Even if you are using existing commercial infrastructure or commodity cloud services to cap infrastructure costs, if your data use is unintelligent or is growing fast, both of which are true in many organisations, then your costs will be spiralling in the wrong direction.

The only way out is to 1) rationalize your data and 2) find an intelligent longer-term solution for your data storage that doesn’t lock you in to a single cloud provider and doesn’t include egress or ingress charges.

Thankfully there are multi-cloud storage solutions, like HPE’s new Cloud Volumes service, that not only include AI to maximise the intelligence with which you manage your storage, but provide a direct link to both your own on prem systems as well as all the public cloud providers, but are also free of ingress and egress charges (once you bitten the bullet and met the initial one-off charge from your current cloud provider of moving any existing data onto this new platform).

Interested in hearing industry leaders discuss subjects like this and sharing their experiences and use-cases? Attend the Cyber Security & Cloud Expo World Series with upcoming events in Silicon Valley, London and Amsterdam to learn more.