Archivo de la categoría: Google

Anthony Yeh, containers, Database, Google, Infrastructure as a Service, MySQL, News & Analysis, Open Source, YouTube

YouTube brings Vitess MySQL scaling magic to Kubernetes

YouTube is working to integrate a beefed up version of MySQL with Kubernetes

YouTube is working to integrate a beefed up version of MySQL with Kubernetes

YouTube is working to integrate Vitess, which improves the ability of MySQL databases to scale in containerised environments, with Kubernetes, an open source container deployment and management tool.

Vitess, which is available as an open source project and pitched as a high-concurrency alternative to NoSQL and vanilla MySQL databases, uses a BSON-based protocol which creates very lightweight connections (around 32KB), and its pooling feature uses Go’s concurrency support to map these lightweight connections to a small pool of MySQL connections; Vitess can handle thousands of connections.

It also handles horizontal and vertical sharding, and can dynamically re-write queries that could impede the database performance.

Anthony Yeh, a software engineer at YouTube said the company is currently using the service to handle metadata for the company’s video service, which handles billions of daily video views and 300 hours of new video uploads per minute.

“Your new website is growing exponentially. After a few rounds of high fives, you start scaling to meet this unexpected demand. While you can always add more front-end servers, eventually your database becomes a bottleneck.”

“Vitess is available as an open source project and runs best in a containerized environment. With Kubernetes and Google Container Engine as your container cluster manager, it’s now a lot easier to get started. We’ve created a single deployment configuration for Vitess that works on any platform that Kubernetes supports,” he explained in a blog post on the Google Cloud Platform website. “In this environment, Vitess provides a MySQL storage layer with improved durability, scalability, and manageability.”

Yeh said the company is just getting started with the Kubernetes integration, but once users will be able to deploy Vitess in containers with Kubernetes on any cloud platform supported by it.

cloud logging, Google, News & Analysis, operations management, Vendor

Google adds log analysis to cloud platform

Google is giving enterprises more tools to troubleshoot persnickety workloads and apps

Google is giving enterprises more tools to troubleshoot persnickety workloads and apps

Google has added a service enabling users of its cloud platform to analyse data logs from both Compute Engine and App Engine, which the company said would help users optimise management operators.

Google already offers a cloud monitoring tools that gives enterprises visibility into networking and compute resources associated with their instances (the company acquired these capabilities from Stackdriver last year), but much more can be gleaned from the massive number of logs that workloads generate – particularly when investigating consistent service errors.

That’s where the cloud logging service comes into play, enabling users of Google’s cloud services to view, analyse and export log data in real time.

“Businesses generate a staggering amount of log data that contains rich information on systems, applications, user requests, and administrative actions. When managed effectively, this treasure trove of data can help you investigate and debug system issues, gain operational and business insights and meet security and compliance needs,” explained Deepak Tiwari, product manager at Google in a recent blog post.

“But log management is challenging. You need to manage very high volumes of streaming data, provision resources to handle peak loads, scale fast and efficiently and have the capability to analyse data in real-time,” he said.

A number of Google’s rivals (Microsoft, AWS) already offer log analysis tools for cloud service users but Google said its tool, which is currently in beta (and free for the time being), comes pre-integrated with a number of its services (BigQuery for analysis, Google Cloud Storage for longer term log data storage) and can be deployed quickly.

Cloud Services, Cloud Storage, Google

Nearline: Google’s Low-Cost Cloud Storage Service For Cold Data

Google is launching a new cloud storage service that is expected to change how many companies of any size view online storage. Google Cloud Storage Nearline allows businesses to store data they or their customers do not often need, and for a low cost of $0.01 per gigabyte at rest.

 

Unlike other cloud storage services where it can take very long to retrieve your data, Google promises that on Nearline your data will be available in only three seconds. They believe the gap between the cost of online and offline storage must decrease according to their director of product management for the Cloud Platform team, Tom Kershaw.

 

Businesses may need or want to keep all of their records for as long as possible. Once they have been moved offline however, it becomes difficult to quickly find the desired record. Google is hoping to blue the line between cold storage and online storage so that businesses do not have to delete or move their files to more complicated storage locations.

 

The low cost of this storage service, which is competitive with Glacier by Amazon, is due to the fact that Google is able to host all of its data on a single system, regardless of location. This is unusual for a cloud storage service because historically, service providers have built two separate systems. The hardest thing about offline storage is transferring the data between these two systems.

 

Nearline uses the same system as the rest of Google’s storage products, including the same encryption and security features. They also share API’s with the standard storage service. It is expected that early adopters of this service will use it primarily for photo, video and document storage.

 

In order to reach a broader market, Google has partnered with many storage companies, most notable Iron Mountain. This partnership will allow users to send in their hard drives and have the securely uploaded onto Nearline.

iron mtn

The post Nearline: Google’s Low-Cost Cloud Storage Service For Cold Data appeared first on Cloud News Daily.

Amazon AWS, Amazon Web Service, DropBox, Google, Google Drive, Microsoft, New Relic, Splunk

Amazon AWS Moving ‘Up the Stack’ to Applications

Amazon Web Services has entered the applications end of the cloud world with several recent releases:

  • Log monitoring and admin with Logs for CloudWatch
  • Collaboration and file sharing with Zocalo
  • Mobile application development with Cognito, Mobile Analytics and a new Mobile SDK

Logs for Cloudwatch works with the AWS CloudWatch network monitoring console to collect log file activities which can then be stored and analyzed in AWS Kinesis. The new tool automatically moves logs from instances and aggregates them into a central service where exceptions can be set directly on those applications.

Third-party products already that, and companies like Splunk, Logentries, and New Relic , which launched its new Insights real-time analytics tool just hours before the AWS news, will all be watching this very carefully (probably also very nervously).

The new AWS Zocalo collaboration/file-sharing plans are further proof that Amazon knows it must be a broad platform player to compete against two mega platform rivals – Google and Microsoft, as well as two younger, well-funded but more limited contenders in Dropbox and Box. Zocalo thus targets Google Drive and Microsoft OneDrive, which are part of a much bigger portfolio of end-user products at those companies.

Cloud & IT Management, Cloud computing, Free, Google, Google App Engine, Kubernetes, LXC, Open Source, Tools

Google Adds Docker Image Support to App Engine, Announces Kubernetes Container Manager

Google continues to up the cloud ante by adding a set of extensions that allow Google App Engine developers to build and deploy Docker images in Managed VMs. Developers can use these extensions to easily access the large and growing library of Docker images, and the Docker community can easily deploy containers into a completely managed environment with access to services such as Cloud Datastore.

From the Google Cloud Platform Blog:

“Based on our experience running Linux containers within Google, we know how important it is to be able to efficiently schedule containers at Internet scale. To that end, we’re announcing Kubernetes, a lean yet powerful open-source container manager that deploys containers into a fleet of machines, provides health management and replication capabilities, and makes it easy for containers to connect to one another and the outside world. We’ll continue to build out the feature set, while collaborating with the Docker community to incorporate the best ideas from Kubernetes into Docker.”

 

Full details here.

Cloud & IT Management, Cloud computing, Docker, Free, Google, Google App Engine, Kubernetes, LXC, Open Source, Tools

Google Adds Docker Image Support to App Engine, Announces Kubernetes Container Manager

Google continues to up the cloud ante by adding a set of extensions that allow Google App Engine developers to build and deploy Docker images in Managed VMs. Developers can use these extensions to easily access the large and growing library of Docker images, and the Docker community can easily deploy containers into a completely managed environment with access to services such as Cloud Datastore.

From the Google Cloud Platform Blog:

“Based on our experience running Linux containers within Google, we know how important it is to be able to efficiently schedule containers at Internet scale. To that end, we’re announcing Kubernetes, a lean yet powerful open-source container manager that deploys containers into a fleet of machines, provides health management and replication capabilities, and makes it easy for containers to connect to one another and the outside world. We’ll continue to build out the feature set, while collaborating with the Docker community to incorporate the best ideas from Kubernetes into Docker.”

 

Full details here.

Advertising, Box, DropBox, File Sharing, Filesharing, Google, Google Adword, Intralink

How an Adwords Campaign Accidentally Exposed Dropbox and Box User’s Confidential Files

We previously reported on a Dropbox Security Snafu (and their correction for it). Now we’re learning more about how it came about, and how it was discovered.

There are several ways users can inadvertently leak confidential files, but the one that is the real head-scratcher is a combination of a user entering the URL of a Dropbox or Box file-sharing link in their browser’s “search box” rather than the “URL box”, combined with Google AdWords campaigns by competitors who want their ads to appear with people “search” for Dropbox or Box (pretty standard stuff).

The sites running such a campaign then — completely innocently — see what users are searching for, and what they are “searching for” turns out to be fully-clickable URLs to files that often contain sensitive personal or company data.

If you think that’s too rare a scenario to worry about, think again:

In one short and entirely innocently designed ad campaign alone, we found that about 5 per cent of hits represented full links to shared files, half of which required no password to download. This amounted to over 300 documents from a small campaign, including several tax returns, a mortgage application, bank information and personal photos. In one case, corporate information including a business plan was uncovered.

That’s from Richard Anstey of Intralink, the people who stumbled on the issue.

Look at this to see (redacted) images of one person’s tax return, and another’s mortgage application. Identity theft, anyone?

Read more about how Intralink discovered all this, along with some good advice on protecting yourself.

TL;DR: sensitive file? Use a sharing application that offers a password or PIN option.

Amazon, Amazon AWS, Cloud computing, Data center, Google

Amazon, Google: a Battle to Dominate the Cloud

The cloud is just a vast mass of computers connected to the internet, on which people or companies can rent processing power or data storage as they need it.

All the warehouses of servers that run the whole of the internet, all the software used by companies the world over, and all the other IT services companies hire others to provide, or which they provide internally, will be worth some $1.4 trillion in 2014, according to Gartner Research—some six times Google and Amazon’s combined annual revenue last year.

When that time comes, all the world’s business IT needs will be delivered as a service, like electricity; you won’t much care where it was generated, as long as the supply is reliable.

Way back in 2006, Amazon had the foresight to start renting out portions of its own, already substantial cloud—the data centers on which it was running Amazon.com—to startups that wanted to pay for servers by the hour, instead of renting them individually, as was typical at the time. Because Amazon was so early, and so aggressive—it has lowered prices for its cloud services 42 times since first unveiling them, according to the company—it first defined and then swallowed whole the market for cloud computing and storage.

Even though Amazon’s external cloud business is much bigger than Google’s, Google still has the biggest total cloud infrastructure—the most servers and data centers. Tests of Amazon’s and Google’s clouds show that by one measure at least—how fast data is transferred from one virtual computer to another inside the cloud—Google’s cloud is seven to nine times faster than Amazon’s.

The question is, is Amazon’s lead insurmountable?

 

Amazon AWS, Amazon Web Services, Cloud computing, Google, Microsoft, Microsoft Office, Opinion

A Big, Perhaps Watershed Week of Cloud Annoucements

  • Google harmonized its cloud computing business to a single entity, with a pricing model intended to hold customers by enticing them to build ever cheaper and more complex software. 
  • Cisco announced it would spend $1 billion on a “cloud of clouds” project. 
  • Microsoft’s new CEO made his first big public appearance, offering Office for the Apple iPad, partly as a way to sell more of its cloud-based Office 365 product.
  • Amazon Web Services announced the general release of its cloud-based desktop computing business, as well as a deal with to offer cloud-based enterprise software tools to industries like healthcare and manufacturing.

For more detail and opinions read this, and listen to this.