(c)iStock.com/VisualCommunications
How do you see what isn’t physically there? It is emerging as a major problem facing enterprises, as cloud increasingly becomes the new normal.
On-premise networks and physical data centres are disappearing fast, with core business applications and processes being migrated to cloud architectures. The latest Cisco Global Cloud Index states that by 2020, 92% of workloads will be processed in public and private cloud data centres, and just 8% in physical data centres.
The rapid move to cloud is understandable since it is so alluring: they are elastic, cost less to operate and manage, and is scalable, enhancing business agility. But in engineering there is no such thing as getting something for nothing. You always have to give something up in order to gain something. The trick, of course, is getting something you value greatly while giving up something that does not mean as much to you. For cloud migration, these benefits are being realised at the expense of visibility and insight into those cloud environments.
When we surveyed a range of businesses on their virtualisation practices, just 37% monitored their virtualised environments with the same rigor as their physical networks. So there is a big visibility gap when it comes to the cloud. While that may seem like a good trade-off today, it may feel different if and when things go wrong. Most organisations cannot take that chance, so they need to bridge the visibility gap quickly. They need it for better control, to maintain security no matter where their data goes, and of course to ensure reliability of their core business applications.
It would seem like addressing this visibility gap could be fixed easily by inserting virtual network taps into the virtualised environment and sending the traffic to their monitoring, analytics, and security tools. Unfortunately, doing this would quickly flood these tools with data, because internal East-West traffic in virtual data centres typically represents 80% of the total traffic. It would be like connecting a lawn sprinkler to a fire hydrant. Identifying and extracting only relevant traffic is key, but how can you do that efficiently? More so, how can your virtual taps handle scaling up and down as virtual machines emerge and dissolve? Let’s take a closer look at the key requirements for visibility and monitoring in virtual environments.
Expanding your (visibility) horizon
There are four key points to consider when deploying virtual taps so you end up with meaningful, granular access to critical application traffic on virtualised networks.
Horizontal scale: Cloud environments are attractive because they can scale up and down rapidly as user demands and workloads change. When placing the virtual taps in the virtual network, you need to be sure they can scale up to accommodate rapid growth in traffic volumes as well as user numbers and data interactions. The taps should do this automatically, without needing IT intervention. Virtualisation means agility, so if an application or service expands to handle 10x or 100x the number of users, make sure the virtual tap you are using can scale elastically – without impacting application performance.
Securing in the dark: Virtualised networks are typically segmented using virtual firewalls to protect key applications and services from attack, and to prevent lateral movement in the virtualised environment that could compromise data or resources. So the virtual taps you use needs to be able to see the application and network traffic flowing between segments. With this comprehensive insight, you can ensure that the appropriate security rules and policies governing each segment are being enforced.
More containers: As virtual machine use grows, container use multiplies even faster – by as much as 10-fold or more, as each application may employ multiple containers. If your organisation is using container-based virtualisation to boost application performance, the virtual tap must be able to access traffic in the container environment.
DevOps elasticity: When your DevOps team puts out a new build – which, remember, doesn’t just cover new applications and services, but also updates to existing ones – then that update propagates across the virtual environment. Individual virtual machines, containers, and by association their hosted applications, have shorter and shorter lifespans requiring continual awareness of the actual state of the environment. It is vital that these changes not block the entire traffic path, or take the virtual tap down with it. As an example, consider how to archive and retrieve monitored traffic from a container that no longer exists. The tap is your sentinel, which has to maintain pervasive access to traffic to enable you to see what is happening on the virtual network: it must be fault-tolerant, even if the application it is monitoring fails.
These four points apply when monitoring any virtualised environment, whether public cloud, private cloud or software defined wide-area networks (SD-WANs): the virtual taps and the overall visibility solution need to be completely environment-agnostic.
Elastic visibility
Once the virtual taps have been deployed to extract traffic from the virtual machines in your environments, you are ready to start processing packets. This volume of traffic needs filtering and controlling using a network packet broker. That keeps duplicate data from overwhelming monitoring and security tools and ensures they scale up/down as needed. Data traffic should be broken up into manageable pieces using packet filtering, grooming and brokering processes, to make sure the security systems and analytics tools are seeing everything.
Elastically-scalable access is achievable for all the data crossing their virtual networks and clouds as well as intelligent distribution to analytics and compliance tools. Leaving your data unmonitored is not smart business. You do not have to give up visibility to gain cloud speed and cost advantages. With the right architecture, you can have both so you should not compromise.