(c)iStock.com/AJ_Watt
Ponder this for a minute: our love affair with cloud technology and services, and the consequent struggle to keep cloud spending in check, is a lot like our daily eating habits.
Spinning up cloud services is almost as easy as grabbing food to go at your favourite burger joint. We’ve made fast food so convenient, affordable, and packed with calories, it’s become a national problem, because our consumption doesn’t match our nutritional needs or activity levels. Now imagine how much worse it would be if we had no ingredient lists, no easy way to count calories, and all of our weight gain for the month showed up in a single day.
It’s not a pretty picture, is it? Thank goodness for food group charts, nutrition labels, Weight Watchers, personal trainers, and fitness apps. Yet we find ourselves in a similar conundrum with cloud consumption.
We love the convenience and ability to spin up resources at the drop of a credit card, and the way we can easily scale resources up and down at will to match demand. On the other hand, it’s easy to have too much of a good thing. That’s how you end up with ‘cloud bloat.’
Cloud bloat happens when you consume resources you don’t really need. But instead of expanding your waist line, you’re expanding your “waste line” – in fact, research shows that up to 30% of cloud spend can be wasted.
So why do we have all this waste? Well, cloud makes it easy to spend without governance and control. Per unit costs seem so low and priced by the hour, how could they possibly add-up to tens of thousands of dollars each month?
How do you prevent cloud waste?
Most businesses rely on the monthly bill from their cloud provider or providers, which can pose problems. Cloud bills are in technical terms, rather than terms that make sense to your business. They are also after-the-fact, long past the point of enabling you to react to a problem in any timely fashion, and lack the historical context required to pinpoint problems or plan for future spend. While most cloud providers offer some sort of tagging capability that enables customers to categorise resources, compliance and consistency is difficult to enforce, making it hard to understand who used which resources and at what cost.
There are many different sources of waste, but some of the most common include purchasing oversized virtual machines (VMs), leaving VMs turned on when they are no longer needed, using expensive storage for infrequently accessed data, or just spinning up unauthorised resources, also known as shadow IT. In order to manage these resources, you first have to measure them – you need a “smart meter” for your cloud that will allow you to match usage with users and put costs against it, so you can see who is using how much of what, and what it is costing you. The next step is to use this information to ensure that you are using your cloud resources efficiently and effectively.
As you adopt more cloud services, and spend increases, managing cloud costs becomes a much larger and critical task. You need to consider how the data is to be used, how often, and by whom, in addition to the following factors:
- Which cloud platform(s) are used or planned?
- Public cloud, Private cloud, Multi- or hybrid cloud?
- What are our reporting requirements?
- Showback, chargeback, billing?
- Do we have customisable reports, dashboards for business users?
- How much data do we have?
- How accurate does the data need to be?
- Do we need “real-time” results?
- What is our budget and how does it compare to labor costs?
At Cloud Cruiser, we believe that there are four steps required to transform your data into meaningful intelligence, otherwise known as cloud analytics: normalisation, enrichment, relevancy and visualisation.
Normalising data into common buckets and units of measure across your different cloud platforms provides more holistic and useful analytics that ultimately drive better decisions. With enrichment, one of the key factors in data transformation is the ability to map your cloud data to organisational and financial information, to provide the business context that matters to you and that you need to drive effective decision-making.
In order to trim the cloud fat, your cloud consumers need fast access to timely, relevant information, so they can see where they’re spending and trim where necessary. Role-based permissions allow you to apply filters to cut down on extraneous information “noise” — and also protects company data by restricting access to confidential information. Similarly, raw data or data in spreadsheets lacks the context and the visual ‘ah-hah!’ you need to optimise your cloud spend and make better business decisions. Different roles require different levels of information, or different kinds of information – the finance folks typically are concerned with costs, while DevOps may be more focused on units – VMs, network throughput, storage volumes, and the like.
A picture is worth a thousand words. Or in this case, hundreds of thousands of rows of data when we’re talking about the cloud. The amount of data is growing exponentially, with more than 2.5 quintillion bytes of data being produced every day. It’s no wonder the BI and analytics market is predicted to reach $16.9B this year. Our poor, limited brains do much better when looking at pretty pictures than trying to find meaning in column after column of numbers.
Formulating a plan of attack
Every organisation is different, so it’s best to start out small and build from there. Start with the highest cost resources or the resources that have the greatest impact on your business. If CPU performance is mission-critical for your customer-facing apps, then target VM utilisation first. If keeping costs under budget for a particular project is your main objective, then target costs by project.
To help your organisation develop a better plan of attack, here are 12 tips to help manage cloud spend:
- If you have more than $20,000 of cloud spend per month, eliminate the spreadsheet and automate
- Collect cloud usage and spend on a daily basis
- Normalise data across cloud platforms, service bundles, and units of measure to see everything in a single, holistic manner
- Think of the end results you want first, and work backwards to identify users, visualisations and data requirements
- Add both organisational and financial context to your cloud data to give business meaning
- Eliminate any IT bottleneck – give end users self-service reporting access. Save time and money
- Develop use cases to ensure your tagging strategy considers both financial and operational goals
- People and processes are not enough to enforce consistent tagging – technology must be used to add, correct, or transform tags
- Data filtering should be enforced to ensure both relevancy and confidentiality
- Use the right visualisation chart for the story you’re trying to tell with your data
- Implement multiple levels of control to ensure usage and spending stays within pre-set thresholds
- Focus first on the areas of greatest spend and/or the areas of greatest impact to the business
Count your cloud calories
Don’t let your consumption get out of hand. Make it easy to count those “cloud calories”—empower your users with easy to read, easy to access data on cloud usage. Keep IT out of the way or users will resort to shadow IT (just like that secret stash of candy bars in your drawer). With relevant data presented visually and in context, users will do a better job of matching consumption to business activity. Encourage them to weigh in daily in order to spot spikes in usage or spending before they balloon into bloat.
We love the cloud. It’s dynamic, agile, scales and drives business value. However, the ease of access to resources can lead to waste, inefficiency, and out-of-control spending. Fortunately, with the right mix of people, process, and technology, you can defeat cloud bloat. A trim “waste line” and a healthy, sustainable approach to growth and technology use will keep your organisation agile and flexible.
