Enterprise data lakes are an essential component of many digital transformation projects. Numerous insights about customers, partners and other stakeholders are extracted from these significant commercial assets. Given the benefits, IT infrastructure and associated software investment will increase to support new use cases.

According to the latest worldwide market study by International Data Corporation (IDC), revenues for big data and business analytics (BDA) solutions will reach $260 billion in 2022 with a compound annual growth rate (CAGR) of 11.9 percent over the 2017-2022 forecast period.

BDA revenues are expected to total $166 billion in 2018 — that's an increase of 11.7 percent over 2017.

Big data and analytics market development

The industries making the largest investments in big data and business analytics solutions throughout the forecast are banking, discrete manufacturing, process manufacturing, professional services, and federal or central government. Combined, these five industries will account for nearly half ($81 billion) of worldwide BDA revenues this year.

They will also be the industries with the largest BDA opportunity in 2022 when their total investment is forecast to reach $129 billion. The industries that will deliver the fastest BDA revenue growth are retail (13.5 percent CAGR), banking (13.2 percent CAGR), and professional services (12.9 percent CAGR).

"At a high level, organizations are turning to big data and analytics solutions to navigate the convergence of their physical and digital worlds," said Jessica Goepfert, program vice president at IDC.

According to the IDC assessment, the transformation takes a different shape depending on the industry. For instance, within the banking and retail sector investments are about managing and reinvigorating the customer experience. Whereas in manufacturing, they're reinventing themselves to become more high-tech.

Furthermore, over half of all BDA revenues will go to IT and business services during the course of the forecast. Services-related revenues will also be among the fastest growing areas of opportunity with a combined CAGR of 13.2 percent.

Software investments will grow to more than $90 billion in 2022, led by purchases of end-user query, reporting, and analysis tools and relational data warehouse management tools.

Two of the fastest growing BDA technology categories will be cognitive or AI software platforms (36.5 percent CAGR) and non-relational analytic data stores (30.3 percent CAGR). BDA-related purchases of servers and storage will grow at a CAGR of 7.3 percent, reaching nearly $27 billion in 2022.

Outlook for regional market growth

The US market is the largest by far, delivering nearly $88 billion in BDA revenues this year and more than half of the worldwide total throughout the five-year forecast. Western Europe is the second largest market with 2018 revenues expected to reach $35 billion, followed by the Asia-Pacific region with $23.9 billion.

Japan will be the second largest country for BDA investments in 2018, followed by the United Kingdom, Germany, and China. The countries with the fastest growth in BDA solutions are Argentina (20.8 percent CAGR), Vietnam (19.8 percent CAGR), Philippines (19.5 percent CAGR), and Indonesia (19.4 percent CAGR).

San Mateo-headquartered cloud storage provider Cloudian has announced it has raised $94 million (£73m) in a series E funding round, hailing it as a validation that object storage is ‘an idea whose time truly has come.’

Cloudian’s business case has been around the benefits of object storage, which provides greater scalability than block storage, as well as favouring unstructured data, with metadata attached to each object more easily identifying and classifying them.

The funding, which for this round included input from Goldman Sachs and NTT DOCOMO Ventures among others, is not quite as simple as x raises y. Back in March, the big headline – as this publication duly reported – was that Cloudian had raised $125 million. Indeed it had – but $100m of that was ‘consumption-based financing’ – a financial buffer, if you will – with $25m as equity. This round can be seen as additional to the $25m already raised. Either way, the company’s funding now stands at $173 million.

As the technological landscape matures, with more enterprises looking to move data into the cloud, the benefits of object storage become even clearer. “We’ve seen this space grow significantly,” Jon Toor, Cloudian CMO tells CloudTech, citing recent IDC figures which showed the global enterprise storage market grew 34% during the first quarter of 2018. “The significant growth in enterprise storage year over year indicates there’s a lot of data moving into data centres. People are looking for new solutions.”

The theory is simple but deadly; as Cloudian is built on cloud technologies, the storage can be anywhere. This comes in especially handy when data is collected in a variety of places.

Michael Tso, CEO of Cloudian, explains that the concept of ‘data gravity’ is key. If data has been created somewhere, for instance in a factory or security camera, it is difficult to move large distances. “The need of having cloud-like storage technology, not in some centralised cloud but near where you’re creating this data, because it’s hard to move a lot of it, is one concept why object storage is really needed,” Tso tells CloudTech.

Case in point: Cloudian has recently picked up two leading Formula 1 teams as customers, which exemplifies this approach well. “They keep everything,” explains Michael Tso, Cloudian CEO. “All the data from the sensors in the car, all the practice runs, from all the races, all the video – that data will never get deleted, they can go back for different things, they can simulate and so forth.

“That’s a very good application for what we do – all the data eventually, no matter where it’s stored or where it comes from, is going to end up on a Cloudian platform because that is the final stop.”

From building a grand total of one car per year – well, two if you include both drivers – to millions, another recent Cloudian customer is a leading automotive manufacturer, in the process of being deployed worldwide. “They are using us to store all of the data for their structure automation, sensors, all the designs – it’s a one stop shop for all their data storage,” says Tso.

Perhaps surprisingly for a US-based cloud software provider, Cloudian has been seeing significant traction in Europe – to the extent where the company now has more customers in the continent, with revenues at approximately 50/50. Tso explains that his vision was ‘the world is flat [and] everything is connected’, and so growth was always intended to be organic. “When we started the company, we put one sales side in Europe, one in the US and one in Asia [and told them] – go for it,” he says. “Whoever made the biggest numbers got to hire the next guy.”

The increased importance of data security in Europe, chiefly thanks to the likes of GDPR, has been another boon. “That’s really I think what is driving the forces behind increased adoption of hybrid IT, where we are a key player,” says Tso. “Being able to bridge this cloud and on-prem, being able to control data flow and data access at a very fine granularity.

“Rather than in the public cloud, which has this approach of one size fits all, we’re seeing data security and privacy moving in different directions,” Tso adds. “The leadership of the European industry and government in that area is really driving our expansion into Europe.”

Earlier this year, the company pinned its mark on three trends driving the rise of object storage in 2018; the growth of artificial intelligence and the Internet of Things, massive data growth, as well as the rise of Amazon S3’s API as an industry standard. It would appear investors agree. “We believe Cloudian is well positioned to dominate the next generation of enterprise storage with its elegantly simple design that integrates both the data centre and cloud environments,” said Edouard Hervey, managing director at Goldman Sachs in a statement.

Read more: Cloudian: On acquiring Infinity Storage, multi-cloud and machine learning

• Software engineering manager is the highest paying position with an average salary of $163,500 with 31,621 open positions on Glassdoor today.
• Over 368,000 open positions are available across the 10 highest paying jobs on Glassdoor today.
• $147,000 is the average salary of the top 10 tech jobs on Glassdoor today.
• 12.7% of all open positions are for software engineers, making this job the most in-demand in tech today.

Glassdoor is best known for its candid, honest reviews of employers written anonymously by employees. It is now common practice and a good idea for anyone considering a position with a new employer to check them out on Glassdoor first. With nearly 40 million reviews on more than 770,000 companies. Glassdoor is now the 2nd most popular job site professionals rely on in the U.S., attracting approximately 59 million job seekers a month. The Chief Human Resources Officer of one of the largest and best-known cloud-based enterprise software companies told me recently she gets 2X more applications from Glassdoor for any given position than any other recruiting site or channel.

Earlier this month Glassdoor Economic Research published the results of research completed on how base pay compares between tech and non-tech jobs.  The research team gathered a sample of tech companies with at least 100 job postings on Glassdoor as of June 26, 2018. Glassdoor defined tech roles as those positions requiring knowledge of code, software or data. The study found the following to be the 10 highest paying tech jobs today:

Walmart eCommerce, Microsoft, Intel, Amazon, and Google have the highest concentration of tech jobs as a percentage of all positions open

Workday, Salesforce, Verizon, and IBM have the highest concentration of non-tech positions available today.

Source: Glassdoor Economic Research Blog, Landing a Non-Tech Job in Tech: Who’s Hiring Today? July 19, 2018

Atmosera delivers modern cloud services that maximize the advantages of cloud-based infrastructures. Offering private, hybrid, and public cloud solutions, Atmosera works closely with customers to engineer, deploy, and operate cloud architectures with advanced services that deliver strategic business outcomes. Atmosera’s expertise simplifies the process of cloud transformation and our 20+ years of experience managing complex IT environments provides our customers with the confidence and trust that they are being taken care of.

read more

At VMworld in Las Vegas, VMware CEO Pat Gelsinger, with the help of some of the best and brightest in the cloud industry, expanded on partnerships and products, as well as the evolution of the multi-cloud landscape.

For the second year running, Andy Jassy, CEO of Amazon Web Services (AWS), took to the stage to give an update on AWS’ growing partnership with VMware, alongside new features. The announcement of an expansion of VMware Cloud on AWS to Asia-Pacific was good – Jassy told the audience the service would be ‘largely’ across all regions, including GovCloud, by late 2019 – but even better was the announcement of Amazon Relational Database Services (RDS) on VMware.

“You’ll be able to provision databases, you’ll be able to scale the compute, or the memory, or the storage for those database instances, you’ll be able to patch the operating system or the database engines,” said Jassy. “I think it’s very exciting for our customers and I think it’s also a good example of where we’re continuing to deepen the partnership and listen to what customers want, and then innovate on their behalf.”

The service, which will be available in a few months, aims to take the capabilities of setting up relational databases in the cloud, but manage them on VMware’s on-premises environment. If users decide these databases would be better longer-term on AWS, then there is a smooth migration path in place.

Jassy noted MIT as a key customer on the primary use case of VMware Cloud on AWS – migrating on-premises applications to the cloud. The university has been able to migrate 3000 VMs from their data centres to the companies’ solution, taking only three months to do so.

The partnership between VMware and AWS, first announced two years ago and updated last year, is evidently blossoming. So much so that VMware half-borrowed a concept from a previous AWS keynote for its own. Whereas Jassy framed his 2016 re:Invent speech on superpowers – supersonic speed, immortality, x-ray vision – and how AWS seemingly enables them, Gelsinger focused on tech superpowers – cloud, mobile, AI/ML, and edge/IoT.

More importantly, while these technologies are changing the way we live and work, they work even better in tandem. “We really see that each one of them is a superpower in their own right, but they’re making each other more powerful,” said Gelsinger. “Cloud enables mobile connectivity, mobile creates more data, more data makes the AI better, AI enables more edge use cases, and more edge requires more cloud to store the data and do the computing.

“They’re reinforcing each other – these superpowers are reshaping every aspect of society.”

VMware’s long-standing vision, therefore, of ‘any device, any application, any cloud’, with intrinsic security, plays into this. Much was spoken about the VMware Cloud Foundation, which the company sees as being “the simplest path to the hybrid cloud”, as Gelsinger put it. The quickest way to get there is through hyperconverged infrastructure, to which VMware has announced an update to vSAN, to ease adoption through simplified operations and efficient infrastructure. The figures touted at the keynote – more than 15,000 customers and 50% of the Global 2000 – are impressive; Gelsinger said it was “clearly becoming the standard for how hyperconverged is done in the industry.”

Another product announcement, this time multi-cloud flavoured, came in the form of upgrades to the VMware NSX networking and security portfolio. According to the company’s earnings call last week, more than four in five of the Fortune 100 have now adopted NSX. NSX-T Data Center 2.3, which is expected to be available before November, will again aim to give customers greater ease of deployment, as well as extend multi-cloud networking and security to AWS and Microsoft Azure, ‘empowering customers that operate across multiple public clouds to take advantage of local availability zones and the unique services of different cloud providers’, as the press materials put it.

One more multi-cloud themed piece of news was that VMware had acquired Boston-based cloud service management provider CloudHealth Technologies. CloudHealth offers a variety of capabilities in its platform, from streamlined billing to scaling out, enabling organisations to manage their workloads across AWS, Microsoft Azure, Google Cloud Platform and VMware.

The company said last June, on the raising of its series D funding, that the IPO market was something it would watch as it aimed to ‘become the anchor company at the centre of the Boston software technology ecosystem for decades to come.’ Earlier this year CloudHealth announced ‘significant investments’ into Europe. Writing in a blog post, founder and CTO Joe Kinsella said that he ‘was gratified to learn early on in discussions that VMware and CloudHealth Technologies share a most important strand of corporate DNA: customer-first.’

“As part of VMware, we will be able to serve you better and offer you a richer set of choices to support your business transformation in the cloud,” Kinsella added.

Ultimately, Gelsinger sees multi-cloud as ‘the next act’ in VMware’s 20-year history, after the server era, BYOD, the network, and cloud migration. The VMware CEO cited a survey from Deloitte which stated the average business today was using eight public clouds.

“As you’re managing different tools, different teams, different architectures – how do you bridge across?” he asked. “This is what we will do in the multi-cloud era – we will help our community to bridge across and take advantage of these powerful cycles of innovation that are going on, but be able to use them across a consistent infrastructure and operational environment.”

You can check out the full list of VMworld news here.

Picture credits: VMware/Screenshot

The capital expenditure of the largest cloud infrastructure players continues to rise – and according to Synergy Research, the first half of 2018 has seen record figures being published.

Total capex for the first half of this year among the hyperscale operators hit $53 billion (£41.1bn), compared with $31bn this time last year. Q2’s figures did not quite match Q1, but this is down to an anomaly, Synergy argues, with Google confirming its buying of Manhattan’s Chelsea Market building, for $2.4bn, in March.

The top five spenders remain Google, Microsoft, Facebook, Apple, and Amazon – and have been for the past 10 quarters. Between them, these five companies account for more than 70% of hyperscale capex. Only in the third quarter of 2016 did overall spending break $15bn (below), with a particular ramp over the past 12 months.

It is worth noting too that the list of highest level players has been trimmed, from 24 to 20. Synergy explains that this is down to a variety of factors; from some companies being subsumed into others’, such as LinkedIn, to others not spending enough on capex to justify inclusion. In some cases, this is because they are moving more of their workloads onto AWS and Azure, to the detriment of their own data centre footprint.

Regardless, this is yet another indicator that the largest players in cloud infrastructure are not resting on their laurels. As this publication has reported, Google has looked to expansion in Finland and Singapore in the past three months, while Microsoft, in an experimental move, put a data centre underwater off the Orkney Islands.

“Hyperscale capex is one of the clearest indicators of the growth in cloud computing, digital enterprise and online lifestyles,” said John Dinsdale, a chief analyst at Synergy. “Capex has reached levels that were previously unthinkable for these massive data centre operators and it continues to climb.

“The largest of these hyperscale operators are building economic moats that smaller competitors have no chance of replicating,” Dinsdale added.

Today, in order to maintain competitive advantage, financial institutions need to be increasingly agile and quick in how they respond to fast-changing customer expectations and ultimately beat their competitors.

To this point, last month the EBA – European Banking Authority  published a Report on the Prudential Risks and Opportunities Arising for Institutions from Fintech. The report provides an analysis of the risks and opportunities relating to the adoption of new innovative technologies, providing seven fintech use cases, one of which is focused on outsourcing core banking and payment systems to the public, hybrid and private cloud.

The report looked at how cloud computing, which is an important enabling technology, is being leveraged by financial institutions to deliver innovative financial products and services.  In particular it highlights that in recent years there has been increasing interest from institutions in working with cloud service providers. And although that interest was initially focused on migrating non-core applications to the cloud, the EBA found that many financial institutions are now exploring how to migrate core mission critical systems to the cloud.

The report goes on to talk about how flexibility, scalability and agility are seen as the main benefits of public cloud, but adds that most cloud services have been standardised in order to allow services to be provided to a large number of customers in a highly automated manner on a large scale.

The underlying concern of course is that in such a security‑intensive and highly‑regulated industry, no one size ‘cloud’ fits all. So while it’s key that cloud providers standardise to very high service standards, those who also provide specialised service offerings and keep themselves open to individual use cases and customers’ requirements – e.g., for mission critical workloads ‑ clearly have an edge. This is precisely what Virtustream was built for, combined with a very high level of automation which reduces human intervention in the most complex IT operation processes, increasing efficiency and lowering risk exposure.  

The EPA report goes on to outline two main criterion that need to be met to ensure financial institutions are making the move to cloud correctly. These include “choosing the right cloud service partner (CSP) on its journey” and “ensuring the internal organisation can meet the needs for this transformation alongside its CSP partner”.

Choosing the right CSP

Financial institutions must carefully select the CSP that is right and suitable for their needs. This will depend on the project in question, the institution’s overall strategy and the regulatory requirements that the organisation must meet. The organisation must also consider what data is appropriate and necessary to migrate to the cloud; remembering that they don’t necessarily need to take an ‘all or nothing’ approach to cloud services. Likewise any CSP that an institution works with must have a firm understanding of the relevant compliance landscape. It is important to be able to demonstrate that a judgment call can be made when required. For example this involves documenting the reasonable action that has been taken to prevent or mitigate a data breach or loss, creating a full ‘audit trail’ and evidence of the company’s compliance.

This is where the CSP must have the deepest and broadest expertise on what it takes to migrate complex mission critical systems to the cloud, as we know quite well at Virtustream, having undertaken thousands of such migrations including the creation of an L3 extension of our users’ private data centres into our cloud nodes and integrating with their existing system monitoring and management tools via a broad set of APIs.

Likewise it is really important that the CSP is not only experienced but has a robust methodology and operating model. For example, in addition to our advisory services at Virtustream we also take a greatly optimised approach to cloud onboarding, migration and operation that includes:

• Assessment: Identifying all workloads across the application landscape, in order to analyse system configurations and interdependencies with an estimate of initial cost benefits
• Onboarding: Project planning and management, documentation of all applications and workloads, determination of the move sequences and thorough testing in order to identify any risks and issues, in order to finalise a full cutover plan
• Migration: The actual migration of production systems, technical checks for data consistency, conversion to production operations. GoLive™ migration checks, handover and transition to steady-state
• Managed services: A range of flexible choices which include infrastructure managed services and application managed services.  We also have expertise in a wide variety of databases, these include physical‑to‑virtual and virtual‑to‑virtual migrations, and database management

The role of IT teams

The report also went on to outline how the role of IT staff in financial institutions could possibly undergo a significant transformation with increased cloud outsourcing services, whereby roles convert into support and consultation for cloud service selection, engagement and management. This is where the adoption of an enterprise‑class cloud provider with managed public cloud services that deliver private cloud attributes is really important, as this strategically enables a new operating model for IT; one that is based on business outcomes and has close alignment between IT and the business.

What I mean by this is having an operating model in place that delivers the ability to quickly implement new ideas so that the organisation can tap into new revenue streams and acquire new customers; a model that lowers complexity and – with that ‑ also actively improves the risk posture.

Adopting a cloud operating model across all areas of the business is probably the most difficult part of the transformation. The key aspect to remember here is that it means working more closely with the business; it means adopting an IT operating model that is services and software product-oriented, not technology or project-oriented.

Looking to the sky

As cloud services become more integral to the whole organisation, so CSPs are going to quickly become part of the financial/banking infrastructure. However the risks involved in outsourcing data to the cloud carry wider potential consequences for any financial institution. This is why it is so important that regulatory bodies such as the EBA are able to respond to changes in the use of cloud and can continue to place strict compliance requirements on financial institutions and their partners.

To their credit, many CSPs have started to accept this as part of their ‘joint responsibility’ when they engage with a financial institution, but as cloud adoption continues to grow, financial institutions will need to carefully plan for and monitor their compliance, while CSPs look to provide an adaptable framework – one that is agile and able to flex to meet the ever-evolving needs of the finance industry.

Editor’s note: Find out more about the report and read it here (pdf, no opt-in).