In the past, we ran small data on expensive big boxes, and it worked fine. For example, most database applications were sized in gigabytes or a few terabytes. Today, we’re moving to a world of Big Data measured in petabytes, running on lots of inexpensive small boxes. Data volumes are growing at 50 percent per year worldwide and at more than 100 percent compounded annually in many companies. This growth presents a fundamentally different computer science problem, and it requires a redesign of the way we build and operate data centers.
Simply put, rapid data growth is outstripping legacy IT designs, and the new world is all about distributed systems: commodity hardware, scale-out design, open standards, Ethernet, programmability, automation and self-service. The problem: your favorite large IT vendors don’t have products that look like this, and their pace of innovation is too slow to catch up to the need.