(c)iStock.com/spxChrome

DevOps engineers are notoriously difficult to find. If you needed further proof of this fact, a new study by Indeed.com has revealed that DevOps engineer is the #1 hardest IT job to fill in North America, leading a list that includes software and mobile engineers.

An organisation’s inability to hire – and retain – systems engineers, build automation engineers, and other titles usually grouped under “DevOps” is a major roadblock to digital transformation efforts; in fact, the majority of organisations say the biggest roadblock to cloud migration is finding the right IT talent, not security, cost, or legacy systems.

There is certainly no easy answer — but here are several ways that organisations are attempting to reduce its impact.

The power of automation

Most companies hire DevOps engineers to automate deployment for frequent or continuous deployment. In reality, this means that much of a DevOps engineer’s time is spent deploying and configuring daily builds, troubleshooting failed builds, and communicating with developers and project managers — all while the long-term work of automating deployment and configuration tasks falls by the wayside.

It is possible that the term “DevOps engineer” itself contributes to this confusion and poor prioritisation; many say there is (or should be) no such thing as a DevOps engineer and they should more properly be called by their exact function in your team, like storage engineer, deployment automation engineer, and so on.

The value of deployment automation and the progress towards some variety of “push button” deployment to test environments is obvious; a survey by Puppet found that high performing IT teams deploy thirty times more frequently than low performing teams. Infrastructure automation is often lower on the priority list but of equal importance, and involves the ability of virtual machines to scale, self-heal, and configure themselves. Anecdotally, our experience is that most organisations do the bare minimum (auto scaling), while the vast majority of infrastructure maintenance tasks are still highly manual.

The fact that your DevOps engineers — or if you prefer different titles: build automation engineers, Linux administrators, Puppet engineers, etc. — do not have time to automate tasks (that could save them more time in the future) is clearly a problem. Your sluggish progress on deployment automation drains resources every day. But your lack of infrastructure automation can quickly become a punishing business problem when you find that auto scaling fails, or you forgot to update a package, or your SSL cert is not automatically renewed, or your environment is not automated to deal with your cloud provider’s infrequent outages. Slow deployment pipelines are bad, but broken infrastructure is worse.

Such events cause what we will call “reactive automation”, a sudden burst of interest in infrastructure automation that quickly fades when everything goes back to normal. Unfortunately, the templates and configuration management scripts that automate infrastructure buildout and maintenance themselves must be maintained, and if no one is paying attention, another infrastructure failure is bound to happen.

The result is a team of stressed, overworked engineers that wish they could focus on the “cool stuff”, but are instead stuck in firefighting mode: exactly the opposite of what you want to happen.

“Hire more DevOps engineers”

When faced with overworked engineers, the natural answer is: let’s hire more. But of course you are already doing that. Most companies have a standing open position for DevOps engineer. Is there another answer?

The second answer is training some of your existing systems engineers in new tools and new cultural frameworks. This certainly needs to happen, but will take some time. The other answer is outsourcing. Outsourcing can mean any number of things, but there are two flavours that best complement DevOps teams. The first is outsourcing infrastructure automation. The second is outsourcing day-to-day, boring, repetitive infrastructure maintenance tasks and around-the-clock monitoring.

Infrastructure automation is in many ways the ideal set of tasks to outsource; the line in the sand between your team’s responsibilities (the application) and the outsourced team (the infrastructure) is relatively clear for most applications, and there is often little time, initiative, or advanced experience to automate infrastructure in-house. Your in-house engineers keep doing what they are doing — managing daily builds, interfacing with developers — and the outsourced team co-manages the templates and scripts that control scalability, security, and failover. This team should also integrate this automation with your existing deployment pipeline.

This works out even better if the same team manages day-to-day patching, monitoring, alerting, log management, change management, etc., much like a traditional professional services or managed services team. These are items that distract your valuable DevOps engineers from more important tasks, and also wake them up at 3am when something goes wrong. When you outsource, you are still fulfilling “you build it, you own it” principle, but at least you have a team telling you when things break and helping you fix it faster.

Managed service providers (MSPs) are not what they used to be — in a good way. Among its many positive effects, the cloud has forced MSPs to evolve and provide more value. Now you can use it to your advantage.

The enterprise DevOps team

As DevOps makes its way to the enterprise, the nature and definition of “DevOps team” will change. Enterprises will continue to struggle to attract talent away from big tech. You will likely see more differentiation in what DevOps means, as traditional network engineers become cloud network experts and Puppet engineers become cloud configuration management masters, leading to a complex medley of “traditional” and “cloud” skills.

Adopting DevOps involves adopting a certain amount of risk, and enterprises want to control that risk. They will rely more heavily on outsourced talent to supplement growing internal teams. This will help them achieve higher deployment velocity and automation more quickly, and put guardrails in place to prevent new DevOps teams from costly mistakes.

DevOps engineers will always be hard to find. Great tech talent and great talent generally is hard to find. The key is knowing how to protect your business against the drought.

The post DevOps Engineer: #1 Hardest Job to Fill appeared first on Logicworks Gathering Clouds.