Organisations continue to move from monolithic to microservices – but observability a struggle

We've all heard the exhortations from vendors and industry players alike – get into DevOps, containers and serverless for IT and application delivery before it's too late. But while the benefits are evident, companies are getting bogged down in troubleshooting and debugging software issues.

That's according to a new report from Scalyr. The study, which polled 155 software development practitioners, found that the majority (71%) are getting used to DevOps-type practices, pushing code into production at least weekly. Almost a third of respondents said they did so at least once a day.

Yet while greater efficiencies are coming out at one end, observability and analysis shoots up at the other. Almost half of those polled said they have five or more observability tools, with 58% of respondents in a DevOps role doing so. When it comes to log management, the biggest concern for software developers is ad-hoc query speed; this was cited by 54% of overall respondents, with that number going up to 68% for those who mostly work in microservices.

Two in five (40%) said their companies' engineers spent the majority of their time troubleshooting software issues, with the number rising to almost three quarters (73%) of companies who push out code on a daily basis. A quarter said they spent the majority of their log management investigation time waiting for queries to complete.

The figures show that time saved in one area is essentially relative – developers are still wasting a lot of time doing something else. "As organisations make the shift from more traditional architectures to microservices and deliver code more frequently, they spend more time troubleshooting and debugging software, working to understand complex data sources, and operational visibility is paramount," wrote Jamie Barnett, Scalyr chief customer officer in a blog post.

"Our takeaway is that organisations that are experiencing this shift to modern software delivery need to take a hard look at their observability tools and processes to make sure they can keep pace and evolve to support modern, high-speed, distributed software engineering practices," Barnett added.

This chimes in with other recent research on the topic. According to the Ponemon Institute last month, the gap between what organisations ideally want in DevOps and microservices practices versus what they are actually able to deliver is costing them millions each year.