How our Managed Services Team Responded to Heartbleed

By Jay Keating, Vice President of Managed Services


A lot has been written about the Heartbleed bug impacting versions of OpenSSL software in recent weeks. For an in-depth description of what Heartbleed is and how to respond to the vulnerability, you can refer to or any number of 3rd party reports. This blog won’t review the actual weakness, but rather describe how our Managed IT Services team responded to the incident in support of our customers.

As software and hardware vendors release code updates to deal with vulnerabilities such as Heartbleed, our Managed IT Services team assesses overall risk in context to mitigating factors and then recommends a course of action. In the case of Heartbleed, since the scope was so broad, we prioritized our assessment in the following tiers:

  1. We quickly evaluated our internal systems and support tools. This important step had to happen immediately so our own management tools weren’t creating risk or concern for our customers.
  2. We then evaluated all Managed IT Services customers’ Internet-facing devices such as firewalls and web servers. All devices that were exposed have been patched at this point, and customers were notified of the risk and mitigation plan immediately. In this case, given the extent of the exposure, we declared emergency maintenance windows with our customers rather than wait for pre-approved monthly maintenance windows.
  3. Finally, with the Internet-facing systems no longer vulnerable, we have refocused on our customers’ internal networks for the next round of assessments and mitigation. As we work through this phase, we will once again work collaboratively with our Managed IT Services customers to coordinate an acceptable maintenance window as soon as possible.

With Heartbleed still top of mind, I suggest a few moments of reflection to think through how your organization responded.  Here are some questions to help frame your review:

  • Do you have a formal security incident response program in place and was it useful in responding to Heartbleed? If not, who will manage your response and what process will be followed?
  • How long did it take you to fully understand your risks? How long did you expect it to take?
  • Do you have support and maintenance contracts in place for all components of your infrastructure, and are the support contact details documented within your security incident response plan?
  • Who is responsible for internal and external communication in case you need to declare emergency maintenance periods?
  • What will your staffing plan be if your team goes into extended hours of operation in response to a security threat?
  • Who are your key IT delivery partners and what resources could they bring to your assistance if you need help with assessment, planning, communication, mitigation, and / or recovery? 

Just answering those six questions will improve your response program. Let us know if we can help.

Learn more about how your organization can properly manage your IT environment