Picture credit: Flickr/Anthony Jauneaud

In the past 25 years, one of the most proprietary technologies that has come to market is cloud computing. That’s the claim I made to the editor of this very publication back in July. The cloud’s promise of flexibility may prove to be a Trojan horse of vendor lock-in as you move up each layer of the vendor’s stack, consuming not just infrastructure, but also software and services.

In this article, I’d like to explain why there’s a risk of cloud lock-in and one robust tactic for avoiding it.

In the beginning

All the major cloud vendors began with infrastructure as a service (IaaS) offerings with two irresistible features: dramatically reduced infrastructure provisioning time, and the advantage of a pay-as-you-go elastic pricing model. This was incredibly well received by the market, and today it’s hard to imagine that most enterprise workloads won’t eventually be deployed on these offerings.

With a captive audience, these same vendors realised they could simply move up the stack, putting an ‘aaS’ on every layer. The most valued and most critical software component of all, the database, is very much the end game here as a database as a service (DBaaS). Amazon, Microsoft, and Google, among others, have developed wonderfully simple DBaaS offerings that eliminate much of the complexity and headache from running your own deployment in the cloud. The challenge is that the data always outlives the applications.

There is nothing wrong with the idea of DBaaS. Your business is probably using some of it right now.  Most organisations are resource constrained, especially when it comes to database admins. They are happy to give up some control and flexibility for convenience. In some cases their choice may be as stark as to either build their application on a DBaaS or not to build at all.

Many organisations are just recovering from an era when vendors used punitive and rigid licensing to force inflexible and outdated products on people. In the past 15 years we’ve seen the unstoppable march of Linux, as well as open source alternatives for every layer of the technology stack. While these options were initially viewed as inferior to their proprietary competitors, today open source is not only legitimate, it has become the innovator in many categories of technology.

Cloud vendors developed most of their offerings on an open source stack, and for good reason. It would be easy to view this as a continuation of the move away from vendor lock-in, but the truth is if you take a closer look at the pricing models, the egress charges, the interfaces, the absence of source code and so on, you’ll notice a familiar whiff coming from many of the cloud contracts. Prediction: DBaaS is going to be the new lock-in that everyone complains viciously about.

So here’s your challenge: you want to offer your team of developers the convenience of a DBaaS, but you want to keep complete control of your stack to avoid lock in and maximise flexibility. You also want to avoid an unsightly invoice from *insert cloud giant here* stapled to your forehead every month. What do you do?

The third way

Square Enix is one of the world’s leading providers of gaming experiences, publishing iconic titles like Tomb Raider and Final Fantasy. Collectively Square Enix games have sold hundreds of millions of units worldwide. It’s not just in gaming distribution that Square Enix is an innovator though. The operations team has also taken a progressive approach to delivering infrastructure to its army of designers and developers.  

Every game has its own set of functionality, so each team of developers uses dedicated infrastructure in a public cloud to store unique data sets for their game. Some functions are used across games, such as leaderboards, but most functions are specific to a given title. For example, Hitman Absolution introduced the ability for players to create their own contracts and share those with other players.

As the number and complexity of online games grew, Square Enix found it could not scale its infrastructure, which at that time was based on a relational database. The operations team needed to overcome that scaling issue and provide all the gaming studios with access to a high performance database. To do this, they migrated to a non-relational database and built a multi-tenant platform they call Online Suite. Online Suite is deployed as one instance of infrastructure that is shared across the company and studios. Essentially, the ops team built their own MongoDB as a service (MDaaS) which is delivered to all of Square Enix’s studios and developers.

The Online Suite provides an API that allows the studios to use MDaaS to store and manage metrics, player profiles, info cast information, leaderboards and competitions. The MDaaS is also used to enable players to share messages across all supported platform such as PlayStation, Xbox, PC, web, iOS, and Android. Essentially, the Online Suite supports any functionality that is shared across multiple games.

This gives them the best of both worlds: control and convenience. They are able to maintain full control of their self-managed environment, with the convenience that comes from a management platform consumed as a service from the cloud.

Square Enix can now scale dozens of database clusters on-demand and deliver 24×7 availability to its developers around the world, all with a single operations staffer. By adopting a multi-tenant DBaaS, Square Enix has been able to consolidate its database instances. This has improved performance and reliability while simplifying life for developers.

The way forward

Crucially, Square Enix has not lost any control. The ops team can still access the code throughout the stack, but they’ve hidden that complexity from their users. As far as the developers are concerned, they have the irresistible cloud experience that is flexible and elastic, but Square Enix has protected itself from lock-in by keeping ownership of the stack.

I’m not crazy. I don’t think this approach would work in every single organisation. I do hope that the example is instructive though. It’s not always a simple dichotomy between the burden of running your own stack or losing control and getting locked-in.

Cloud computing is dramatically changing the way we create services and products. It is a great tool but it’s also a Siren’s call of flexibility and cost savings which has the potential to trap you and to limit your options. But if you learn the lessons from our friends behind Tomb Raider, you might just be able to navigate a course out of cloud cuckoo land.