Due of the rise of Hadoop, many enterprises are now deploying their first small clusters of 10 to 20 servers. At this small scale, the complexity of operating the cluster looks and feels like general data center servers. It is not until the clusters scale, as they inevitably do, when the pain caused by the exponential complexity becomes apparent. We’ve seen this problem occur time and time again.
In his session at Big Data Expo, Greg Bruno, Vice President of Engineering and co-founder of StackIQ, described why clusters are so different from farms of single-purpose servers that reside in traditional data centers, and why without an automated solution that can address the cluster requirements, real pain is coming and failure is certain.