Uncategorized

Airbnb: How big data is used to disrupt the hospitality industry

(Image Credit: iStockPhoto/GoodLifeStudio)

Writing on Airbnb’s ‘Nerds’ hub, Riley Newman, head of data science, says: “A datum is a record of an action or event, which in most cases reflects a decision made by a person. If you can recreate the sequence of events leading up to that decision, you can learn from it; it’s an indirect way of the person telling you what they like and don’t like – this property is more attractive than that one, I find these features useful but those… not so much. This sort of feedback can be a goldmine for decisions about community growth, product development and resource prioritization… we translate the customer’s ‘voice’ into a language more suitable for decision-making.”

The insight gained from this feedback enables Airbnb to ensure they concentrate efforts on signing up landlords in popular destinations at peak times, and structure pricing so that the use of their global network of properties is optimized. For example, data is used to determine the appropriate price of a room or apartment, based on a number of variables such as location, time of year, type of accommodation, transport links, etc. Airbnb use an algorithm to help their hosts determine the right price for their offering. This is particularly challenging given the sheer range of accommodation available and when you consider these are real homes, not bog-standard hotel rooms that can be easily rated on a star system. After all, what is desirable in a city apartment (Wi-Fi, good transport links, etc.) may be less important in a quaint cottage (where the guests may prefer peace and romantic decor over Wi-Fi and subway connections).

To help hosts set the price, Airbnb released a machine-learning platform called Aerosolve. The platform analyses images from the host’s photos (listings with photos of cosy bedrooms are more successful than those with stylish living rooms!) and automatically divides cities into micro-neighbourhoods. The platform also incorporates dynamic pricing tips that mimic hotel and airline pricing models.

Airbnb have also just unveiled Airpal: a user-friendly data analysis platform designed to allow all of their employees, not just those trained in data science, access to all of the company’s information, and tools to query it with.

What were the results?

As Newman says: “Measuring the impact of a data science team is ironically difficult, but one signal is that there’s now a unanimous desire to consult data for decisions that need to be made by technical and non-technical people alike.” This is demonstrated in the Airpal system; launched in 2014, Airpal has already been used by more than one-third of Airbnb employees to issue queries. This impressive statistic shows how central data has become to Airbnb’s decision making.

The growth of Airbnb is another indication that their clever use of data is paying off.

What data was used?

Data is primarily internal across a mixture of structured and unstructured formats: image data from host photos, location data, accommodation features (number of rooms/beds, Wi-Fi, hot tub, etc.), customer feedback and ratings, transaction data, etc. Some external data is analysed, too, for example accommodation in Edinburgh during the popular Edinburgh Festival will be priced higher than the same accommodation in a different month.

What are the technical details?

Airbnb hold their approximately 1.5 petabytes of data as Hive managed tables in Hadoop Distributed File System (HDFS) clusters, hosted on Amazon’s Elastic Compute Cloud (EC2) Web service. For querying data, Airbnb used to use Amazon Redshift but they’ve since switched to Facebook’s Presto database. As Presto is open source, this has allowed Airbnb to debug issues early on and share their patches upstream – something they couldn’t do with Redshift.

Going forward, Airbnb are hoping to move to real-time processing as opposed to batch processing, which will improve the detection of anomalies in payments and increase sophistication around matching and personalization.

Any challenges that had to be overcome?

One big challenge for the Airbnb data science team was keeping up with the company’s dramatic growth. Early in 2011, the team consisted of just three data scientists but, as the company was still quite small, the three could still pretty much meet with every individual employee and fulfil their data needs. By the end of the year, Airbnb had 10 international offices and hugely expanded teams, meaning the data team could no longer hope to partner with everyone across the company.

As Newman puts it: “We needed to find a way to democratize our work, broadening from individual interactions, to empowering teams, the company, and even our community.” This was achieved through investing in faster and more reliable technologies to cope with the expanding volume of data. They also moved basic data exploration and queries from data scientists to the teams throughout the company, with the help of dashboards and the Airpal query tool; this empowered Airbnb teams and freed up the data scientists from adhoc requests so they could focus on more impactful work. Educating the teams on how to use these tools has been key to helping them gain insights from the data.

What are the key learning points and takeaways?

Airbnb are a perfect example of a fast-growing company with ever-expanding Big Data needs. The ability to shift and adapt as the company have grown has, I think, been at the heart of their success. This highlights the non-static nature of Big Data and how your data strategy may need to change over time to cope with new demands.

It’s also great to see a data science team so well integrated with all parts of the organization (even if they can no longer meet with every employee!). This not only ensures the data scientists have an excellent understanding of the business’s goals but also emphasizes the importance of data-based decision making for employees right across the company. After all, it doesn’t matter how much data you have if no one acts upon it.

This is an edited extract from Big Data in Practice: How 45 Successful Companies Used Big Data Analytics to Deliver Extraordinary Results by Bernard Marr (published by Wiley)

Did you find anything interesting from this case study of Airbnb’s use of the cloud? Let us know in the comments.

Enterprise IT, Government, News & Analysis

Government report highlights only 29% of UK has cyber security policies

Overview#

Click to enlarge

The Department for Culture, Media and Sport has released findings from its annual Cyber Security Breaches Survey, where 69% of organizations believe security to be a top priority for the business, though only 29% have a formal written policy.

Within the large organizations category, those with 250 or more employees, 90% considered security as a ‘very high’ or ‘fairly high’ priority, though this percentage dropped to 69% when taking an average of the UK as a whole.

“The UK is a world-leading digital economy and this Government has made cyber security a top priority,” said Minister for the Digital Economy Ed Vaizey. “Too many firms are losing money, data and consumer confidence with the vast number of cyber-attacks. It’s absolutely crucial businesses are secure and can protect data. As a minimum, companies should take action by adopting the Cyber Essentials scheme which will help them protect themselves.”

Of the companies who participated in the survey, 24% said they had experienced a breach within the last twelve months, though this is higher for medium and large businesses, 51% and 65% respectively. Large organizations would appear to be the more attractive target for cyber criminals, with 25% of the larger organizations experiencing at least one attack per month over the last year. In terms of financials, the average breach costs organizations £3,480, though this increases to £36,500 for organizations in the large category.

Although a healthy proportion of organizations claim security is a top priority only 29% have written cyber security policies, and only 10% have formal incident management processes. The survey also highlighted only 17% have had their staff undergo some form of cyber security training in the last 12 months.

“One of the most shocking revelations in the Government’s research is the fact that just 10 per cent of UK businesses have an incident management plan in place,” said Jens Puhle, UK Managing Director of 8MAN. “Given that two thirds of large businesses were breached this year alone, organisations need to think in terms of “when”, not an “if” they are attacked, and it is vital they have a solid response plan in place.

How much of a priority is cloud security

Security priority – click to enlarge

“Businesses that are equipped with the ability to identify how the breach occurred and which systems were affected will be able to mitigate the damage the impact and resume normal operations much sooner. They will also be able to take control of the aftermath, disclosing the incident on their terms and working with the authorities to catch the perpetrator. Being unable to perform these basic tasks will make it much more likely that a business is seen as inviting disaster on itself and its customers through negligence, rather than as a blameless victim of crime.”

From an employee perspective, only 34% of organizations currently employ staff whose job role specifically includes information security or governance, which could be perceived as relatively low considering 67% believe security is a top priority. These jobs were most common within finance (60%) and education, health or social care (52%), sectors which could be viewed as having more stringent regulation surrounding data protection.

While hiring people with the right skills is an important step in becoming more secure Lee Meyrick, Director of Information Management at Nuix, believes these individuals also need to have a firm grasp how and where a company’s data resides, a task which might not be as simple as first imagined.

“The first step towards responding efficiently to breaches and closing information security gaps quickly, is understanding where important data is stored. This is easier said than done, as about 80% of organisational data is unstructured, meaning it’s in complex formats – such as emails, databases, photos, and presentations– that are difficult to search and understand.

Spend on security

Security spend – click to enlarge

“The key principle is making sure the only people who can access high-risk data are those who need to for day-to-day work. In order to achieve this, information security, information governance and records management specialists need to become “good shepherds” of their data.

“They should know where all their sheep are, segregate them into separate fields, make sure the fences between fields are sound and regularly check to ensure the sheep are healthy. In this way, even if a wolf manages to get into one of the fields, most of the flock will be safe.”

While the survey does demonstrate good intentions from organizations throughout the UK in respect to attitudes towards security, it would appear the practical implications from these intentions have largely remained unfulfilled to date. Large organizations would appear to have a more solid grip on security within their own environments, though this does not seem to extent to their own supply chain where only 13% of UK businesses set minimum cyber security standards for their suppliers.

The report states the attitudes within medium and large organizations towards security is positive, though more could be done to implement data encryption rules, offer staff training and having formal incident management processes. It also states more could be done to raise standards within their own supply chains, which could have a ripple effect on smaller organizations throughout the UK.

acquisition, IBM, M&A, Merger, Opinion

CIOs look to the cloud for seamless M&A

IBM speaker

Sebastian Krause, General Manager for IBM Cloud Europe

For senior CIOs, knowing how to respond to an M&A and divesture situation is key, as mergers, acquisitions and divestitures are a critical component of business strategy.

Projections for European M&A transactions show total deal values are set to rise from US$621 billion in 2014 to US$936 billion by 2017. M&A activity is likely to be bolstered by continued positive monetary policy, with additional cross-border M&A activity likely to take place as a result of a strong US dollar, primarily in Spain, Germany, and Italy.

Increasingly, businesses are using M&A to grow their organisation, achieve economies of scale, expand product portfolios, globalise and diversify.

In the intense negotiations around this business change, IT operations are likely to face dramatic reorganisation as various stakeholders analyse existing systems and look at the potential for efficiencies.

This is about survival and the IT division is likely to be under intense scrutiny during this period, under pressure to perform critical functions such as the integration or separation of critical systems and data, the provision of an uninterrupted service during the transition period, and the prompt delivery of synergy targets. IT strategy is therefore core to any successful M&A or divestiture plan and a critical contributor to its success or failure.

Increasingly, CIOs are under pressure to meet these challenges quickly and at lower cost. Their ability to do so can even impact the way analysts assess potential deals. IT dependent synergies have been found to be responsible for a large proportion (30 to 60%) of M&A benefits, but 70% of M&As fail to meet their synergy targets in the planned timeframe.

Realising these M&A and divestiture targets for enterprise IT environments is complex and requires a holistic approach that considers public, private, IaaS, PaaS, and SaaS as well as non-cloud delivery models.

Some CIOs may approach the situation by simply making adjustments to the existing IT landscape – from CRM, ERP through to office.

This can involve singling out certain components of an established Enterprise Resource Planning (ERP) system, cloning the existing ERP environment, deploying existing systems into the acquired business asset or transferring data between differing systems with the expectation that no issues with integration will arise. These approaches have certainly worked in the past, but can be costly, challenging to implement and disruptive.

This is why many CIOs are looking at a move towards cloud-based applications and infrastructure, which can take the pain out of the M&A process. Broadly, the drivers for moving to cloud services are increased agility, speed, innovation and lowering costs.

They can help organisations going through mergers and acquisitions to realise synergy benefits more quickly, simplify integration and accelerate the change programme, reduce costs through efficiencies, mitigate costly migration investments and encourage financial flexibility.

Top cloud benefits for M&A:

  • Achieving synergy more quickly: Cloud enabled applications simplify portability, integration and deployment.
  • Lowering costs: The cloud can provide temporary burst capacity for the migration.
  • Increased financial flexibility: Cloud provides a flexible cost model, allowing organisations to easily move between CAPEX and OPEX to impact EBITA and cash flow.
  • Simplifying changes: Cloud simplifies the creation of APIs to hide the underlying complexity of multiple, overlapping systems.
  • When preparing for an M&A or divestiture, it’s worth considering what the future IT model will look like, which APIs are needed to simplify required activities and how applications can be cloud enabled for portability and deployment.

Developing a repeatable platform that delivers these benefits and simplifies M&A activities will greatly improve an organisation’s ability to grow and be successful. It may even open up new opportunities that might not have been possible without the cost, flexibility, and scalability benefits that cloud solutions can deliver.

With businesses already realising real benefits, the cloud’s role in M&A is only set to grow. By building a cloud model that works, organisations can avoid reorganising IT operations for each merger or acquisition and ensure a much more seamless transition.

Through implementing an approach that can speed the execution and success of these deals, CIOs can look to deliver value from the IT department that goes far beyond just support, to true business leadership.

Written by Sebastian Krause, General Manager for IBM Cloud Europe

Adoption, Enterprise IT, hybrid cloud, multi-cloud, News & Analysis, Private Cloud, Public Cloud, Statistics, VMTurbo

57% of organizations still don’t have multi-cloud strategy – survey

Competition. Business concept illustrationResearch from VMTurbo has highlighted 57% of organizations have no multi-cloud strategy at all, where as 35% do not have a private cloud strategy and 28% lack one for public cloud.

Although hybrid cloud is considered one of the growing trends within the industry, the research suggests the noise behind multi-cloud strategies is coming from either a small number of customers, or from vendor organizations themselves. Of those who would be considered in the ‘Functional Multi-cloud Owner’ group, which only represented 10.4% of the respondents, almost half were using a two-cloud model, and just over a quarter were using a three-cloud model. The multi-cloud strategy was favoured by larger organizations in general.

“A lack of cloud strategy doesn’t mean an organization has studied and rejected the idea of the cloud; it means it has given adoption little or no thought at all,” said Charles Crouchman, CTO of VMTurbo. “As organizations make the journey from on-premise IT, to public and private clouds, and finally to multi- and hybrid clouds, it’s essential that they address this.

“Having a cloud strategy means understanding the precise costs and challenges that the cloud will introduce, knowing how to make the cloud approach work for you, and choosing technologies that will supplement cloud adoption. For instance, by automating workload allocation so that services are always provided with the best performance for the best cost. Without a strategy, organizations will be condemning themselves to higher-than-expected costs, and a cloud that never performs to its full potential.”

The survey also demonstrated the total cost of ownership is not fully understood within the community itself, less so within smaller organizations. SME’s planning to build private cloud environments estimated their budget to be in the region of $150,000 (average of all respondents), whereas the total bill for those who have already completed such projects averaged at $898,508.

The stat backs up thoughts of a number of organizations who believe there should be more of a business case behind the transition to the cloud than simply reducing CAPEX and OPEX. Last month, BCN spoke to Gwil Davies, Director & Cloud Lead in the EMEA IT Infrastructure Centre of Excellence at Deloitte, to understand the economics behind cloud computing. Davies believes a successful journey to the cloud is not just focused on reducing CAPEX and OPEX throughout the organization, but identifies where value can be achieved through a cloud-enabled business.

“I think it’s more important for organizations get a real understanding of how to use the cloud and perhaps not automatically assume that moving all of their current IT into cloud is going to be the cheaper solution.” said Davies.

The business case for the cloud is almost entirely dependent on the long-term ambitions of the business itself, though the survey does imply there is a need to further educate some corners of the IT industry on the benefits and perceived cost of private cloud. Cloud computing as a concept could be perceived to have penetrated the mainstream market, though the benefits may be less so.

Bharti Airtel, Enterprise IT, Micrsoft Azure, News & Analysis, Private Cloud

Bharti Airtel bolsters cloud capabilities with Microsoft partnership

Silhouette Businessman Holding PuzzleBharti Airtel has announced the launch of Connexion as well as a new collaboration with Microsoft to deliver Azure ExpressRoute to Indian businesses.

The new Connexion service is designed to maximize network performance over the cloud, whereas Azure ExpressRoute ensures a more secure and scalable connection between enterprises, cloud service providers, and data centre partners, through using a private connection as opposed to public internet. Microsoft claim the service increases reliability, speed and security, while also lowering latency.

“Over the years, at Airtel, we have been serving a vast array of global customers through our world class technology and innovative connectivity solutions,” said Ajay Chitkara, CEO of Global Business at Bharti Airtel. “Today, we are excited to further expand our value proposition for them with the launch of our ‘Connexion’, which is a direct private connectivity to cloud services.

“This platform is the right choice for the service providers and businesses seeking to make their IT infrastructure more agile and flexible. With ‘Connexion’ – we are confident of helping customers seamlessly and more securely connect to Microsoft Azure, by bringing down their network cost substantially and improving performance.”

The partnership further increases Airtel’s international cloud capabilities and ability to serve customers in the Middle East, South Asia and Asia-Pacific regions. Last month, the Airtel business also announced a partnership with GBI to build its influence within the Middle East. GBI operates a multilayer carrier neutral network, connecting the world to the Middle East, a region which is a long-term target for Airtel’s growth ambitions.

“This new partnership with GBI is a significant step in that direction,” said Chitkara. “GBI being a key network asset for the region will not only improve our customers’ experience and reach but would also enable GBI’s customers to experience a seamless extension on the Airtel Global network spanning across 50 countries across 5 continents”

AI, Artificial intelligence, BlueMix, cognitive computing, IBM, News & Analysis, SK C&C, Watson

IBM teams up with SK C&C to teach Watson learns Korean

Bukchon Hanok Village in Seoul, South Korea.SK C&C has continued Korea’s efforts to increase the usage and adoption of cloud computing within the region, announcing a new strategic alliance with IBM focused on the Watson cognitive computing platform.

As part of the agreement, IBM will train Watson to understand and comprehend Korean, and South Korea-based developers will create a number of localized API’s and services to increase adoption rates of such advanced cloud computing technologies in the region. Korean will be Watson’s eighth language, lining up with English, French, Italian, Spanish, Brazilian Portuguese, Japanese, and Arabic.

“Watson remains at the forefront of cognitive computing: advanced systems that learn at scale, understand with meaning, reason with purpose and interact with humans in natural ways,” said David Kenny, GM for IBM Watson. “The South Korean marketplace is moving quickly to embrace the disruptive opportunities from next generation technology.

“Our strategic alliance with SK Holdings C&C will put cognitive services in the hands of more businesses and developers, allowing them to apply Watson within their organizations to help transform entire industries and professions.”

Korea has been making positive strides in recent months to increase the adoption rate of cloud computing within the country, announcing a number of initiatives in March. Adoption rates are reported to be as low as 6.4% within the country currently, which could be perceived as low considering the number of tech companies which has grown out of Korea, though the government is planning to increase this to 13% over the next twelve months. Over the same period, the government also plans to increase the number of Korean cloud companies from 353 to 500.

While this announcement focused on cloud computing as a broader technology set, the government also announced plans to invest 100 billion won (approximately $87.2 million) to foster the development of supercomputers. The Ministry of Science, ICT, and Future Planning said it would invest 10 billion won annually for the next 10 years to boost the growth of artificial intelligence, big data, the Internet-of-Things technologies and other emerging industries through supercomputers. The ambition is to create a supercomputer with a data-processing speed of 1 petaflop (PF) in five years, eventually reaching 30 PF by 2025.

As part of the partnership between IBM and SK C&C, the telco will run Watson and Bluemix from its Pangyo Cloud Centre, to foster the growth of cognitive computing and artificial intelligence. More specifically SK C&C is hoping the introduction of the technologies will improve mobile device experience, as well as consumers’ call centre interactions. SK C&C will also become IBM’s preferred distributor for cognitive solutions in South Korea.

“This alliance highlights SK’s dedication to growing our artificial intelligence-based data services business, strengthening our Ai leadership position, as well as spurring innovation and Ai adoption across Korea,” said Park Jung-ho, CEO of SK Holdings C&C.

The partnership between IBM and SK C&C is one of a number of examples of IBM’s efforts to broaden the appeal to the international audience. SK C&C will assist in developing Watson’s advanced conversational capabilities in Korean, in the same way SoftBank is aiding for Japanese, Mubadala for Arabic and GBM in South America. Each of these companies, including SK C&C, are developing local communities of developers to build, explore and create new applications in their native languages. Korean language Watson services are expected to become available early next year.

Uncategorized

Docker Containers and #Microservices | @DevOpsSummit #DevOps #Docker

From the conception of Docker containers to the unfolding microservices revolution we see today, here is a brief history of what I like to call ‘containerology’.
In 2013, we were solidly in the monolithic application era. I had noticed that a growing amount of effort was going into deploying and configuring applications. As applications had grown in complexity and interdependency over the years, the effort to install and configure them was becoming significant. But the road did not end with a single deployment, no, the installation and configuration work was repeated over and over again, not only for each software release but for each and every environment that we promote applications to until finally being deposited into production, where this we repeat this exercise one last time.

read more

Uncategorized

Ten Ways to Flash Forward By @Kevin_Jackson | @CloudExpo #Cloud

Not to long ago I was honored to be included as a storage expert in the Dell ebook, “10 Ways to Flash Forward: Future-Ready Storage Insights from the Experts.” This publication provided insight into some of the most recent advances in enterprise storage and gave some excellent recomendations for your advance storage strategy. The full document is available for free download at http://marketing.dell.com/storage-ebook-toprank-Flash-Forward

read more

Uncategorized

Understanding IoT | @ThingsExpo #IoT #IIoT #M2M #DigitalTransformation

There is an ever-growing explosion of new devices that are connected to the Internet using “cloud” solutions. This rapid growth is creating a massive new demand for efficient access to data. And it’s not just about connecting to that data anymore.
This new demand is bringing new issues and challenges and it is important for companies to scale for the coming growth. And with that scaling comes the need for greater security, gathering and data analysis, storage, connectivity and, of course, the need to find a platform (solution) that is capable of quickly and easily managing all of these complex necessities.

read more

The cloud news categorized.