Microsoft hopes that open sourcing the compression technology embedded in its Azure cloud servers will pave the way for the technology’s adoption into a range of other devices.
The company is making the algorithms, hardware design specifications and the source code behind its compression tech, dubbed Project Zipline, available for manufacturers and engineers to integrate into silicon components.
Microsoft announced this move to mark the start of the Open Compute Project’s (OCP) annual summit. Microsoft is a prominent member of the programme, which was started by Facebook in 2011 and includes the likes of IBM, Intel, and Google.
Project Zipline is being released to the OCP to combat the challenges posed by an exploding volume of data that exists in the ‘global datasphere’, in both private and public realms, the company said. Businesses are also increasingly finding themselves burdened with mountains of internal data that should be better managed and utilised.
“The enterprise is fast becoming the world’s data steward once again,” said Microsoft’s general manager for Azure hardware infrastructure Kushagra Vaid.
“In the recent past, consumers were responsible for much of their own data, but their reliance on and trust of today’s cloud services, especially from connectivity, performance, and convenience perspectives, continues to increase and the desire to store and manage data locally continues to decrease.
“We are open sourcing Project Zipline compression algorithms, hardware design specifications, and Verilog source code for register transfer language (RTL) with initial content available today and more coming soon.
“This contribution will provide collateral for integration into a variety of silicon components (e.g. edge devices, networking, offload accelerators etc.) across the industry for this new high-performance compression standard.”
According to the firm, the compression algorithm yields up to twice as high compression ratios versus the widely used Zlib-L4 64KB compression model. Contributing RTL at this level of detail, Vaid added, sets a new precedent for frictionless collaboration, and can open the door for hardware innovation at the silicon level.
Members of the OCP will be able to run their own Project Zipline trials and contribute to the further development of the algorithm, and its hardware specifications.
Microsoft hopes that its technology will be integrated into a variety of silicon components and devices, in the future. These could range from smart SSDs to archival systems, to cloud appliances, as well as IoT and edge devices.
Making its compression technology available represents Microsoft’s latest contribution to OCP, more than five years after the company first began contributing to the open source project. Incremental contributions have been made ever since, with the company, for instance, delivering its Open CloudServer specs to the project in October 2014.