What is NVMe over Fabrics (NVMe-oF)?

The evolvement of the NVMe interface protocol is a boon to SSD-based storage arrays. It further powered SSDs (solid state drives) to obtain high performance and reduced latency for accessing data. Benefits further extended by the NVMe over Fabrics network protocol brings NVMe feature retained over the network fabric, while accessing the storage array remotely. Let’s understand how.

While leveraging NVMe protocol with storage arrays consists of high-speed NAND and SSDs, a latency was experienced when NVMe based storage arrays were accessed through shared storage or storage area networks (SAN). In SAN, data should be transferred between the host (initiator) and the NVMe-enabled storage array (target) over Ethernet, RDMA technologies (iWARP/RoCE), or Fibre Channel. Latency was caused due a translation of SCSI commands into NVMe commands, in the data transportation process.

To address this bottleneck, NVM Express introduced the NVMe over Fabrics protocol, to get replaced with iSCSI as a storage networking protocol. With this, the benefits of NVMe were taken onto network fabrics in a SAN-kind of architecture to have a complete end-to-end NVMe-based storage model which is highly efficient for modern workloads. NVMe-oF supports all available network fabrics technologies, such as RDMA (RoCE, iWARP), Fibre Channel (FC-NVMe), Infiniband, Future Fabrics, and Intel Omni-Path architecture.

NVMe over Fabrics and OpenStack

As we know, OpenStack consists of a library of open source projects for the centralised management of data centre operations. OpenStack provides an ideal environment to implement an efficient NVMe-based storage model for high throughput. OpenStack Nova and Cinder are components used in proposed NVMe-oF with an OpenStack solution. This consists of creation and integration of Cinder NVME-oF target driver, along with OpenStack Nova.

OpenStack Cinder is a block storage service project for OpenStack deployments mainly used to create services which provide persistent storage to cloud-based applications. It provides APIs to users to access storage resources without disclosing storage location information.

OpenStack Nova is a component within OpenStack which helps provide on-demand access to compute resources like virtual machines, containers, and bare metal services. In NVMe-oF with OpenStack solutions, Nova is attaching NVMe volumes to VMs.

Support of NVMe-oF in OpenStack is available from the ‘Rocky’ release. A proposed solution requires RDMA NICs and supports kernel initiator and kernel target.

NVMe-oF targets supported

Based on the proposed solution above, we get two choices to implement NVMe-oF with OpenStack; first, with a kernel NVMe-oF target driver which is supported as of the OpenStack ‘R’ release, and second Intel’s SPDK (storage performance development kit) based NVMe-oF implementation containing SPDK NVMe-oF target driver and the SPDK LVOL (Logical Volume Manager) backend. This is anticipated to be in the OpenStack ‘S’ release.

Kernel NVMe-oF target: Here is the implementation consisting of support for kernel target and kernel initiator. But the kernel-based NVMe-oF target implementation has limitations in terms of number of IOPs per CPU core. Also, kernel-based NVMe-oF suffers latency issues due to CPU interrupts, many systems calling to read data, and time taken to transfer data between threads.

Fig – Kernel Based NVMe-oF + OpenStack Implementation

SPDK NVMe-oF target: Why SPDK? SPDK architecture achieved high performance for NVMe-oF with OpenStack by moving all necessary application drivers to userspaces (apart from the kernel) and enables operation in polled mode instead of interrupt mode and lockless (avoiding the use of CPU cycles synchronising data between threads) processing.

Let’s understand what it means.

In SPDK implementation, storage drivers which are utilised for storage operations like storing, updating and deleting data are isolated from the kernel space where general purpose computing processes run. This isolation of storage drivers from kernel saves time required for processing in the kernel, and enables CPU cycles to spend more time for execution of storage drivers at user space. This avoids interruption and locking of storage drivers with other general-purpose computing drivers in kernel space.

In a typical I/O model, application requests a read/write data access and waits until the I/O cycle to complete. In polled mode, once the application places a request for data access, it goes at other execution and comes back after a defined interval to check completion of an earlier request. This reduces latency and process overheads, and further improves the efficiency of I/O operations.

By summarising, SPDK is specially designed to extract performance from non-volatile media, containing tools and libraries for scalable and efficient storage applications utilised userspace, and polled mode components to enable millions of I/Os per core. SPDK architecture is open source BSD licensed blocks optimised for bringing out high throughput from the latest generation of CPUs and SSDs.

Fig – SPDK Architecture

Why SPDK NVMe-oF target?

As per the performance benchmarking report of NVMe-oF using SPDK, it has been seen that:

• Throughput scales up and latency decreases almost linearly with the scaling of SPDK NVMe-oF target and initiator I/O cores
• SPDK NVMe-oF target performed up to 7.3x better with regards to IOPS/core than Linux Kernel NVMe-oF target while running 4x 100% random write workload with increasing number of connections (16) per NVMe-oF subsystem
• SPDK NVMe-oF initiator is 3x faster than Kernel NVMe-oF initiator with null bdev-based backend
• SPDK reduces NVMe-oF software overheads by up to 10x
• SPDK saturates 8 NVMe SSDs with a single CPU core

Fig – SPDK vs. Kernel NVMe-oF I/O Efficiency

SPDK NVMe-oF implementation

This is the first implementation of NVMe-oF integrating with OpenStack (Cinder and Nova) which leverages NVMe-oF target driver and SPDK LVOL (Logical Volume Manager)-based SDS storage backend. This provides a high-performance alternative to kernel LVM and kernel NVMe-oF target.

Fig – SPDK Based NVMe-oF + OpenStack Implementation

The implementation was demonstrated at OpenStack Summit 2018 Vancouver. You can watch the demonstration video here.

If compared with Kernel-based implementations, SPDK reduces NVMe-oF software overheads and yields high throughput and performance. Let’s see how this will be added to the upcoming OpenStack ‘S’ release.

This article is based on a session at OpenStack Summit 2018 Vancouver – OpenStack and NVMe-over-Fabrics – Network connected SSDs with local performance. The session was presented by Tushar Gohad (Intel), Moshe Levi (Mellanox) and Ivan Kolodyazhny (Mirantis).

The post OpenStack and NVMe-over-Fabrics – Getting High Performance for Network Connected SSDs appeared first on Calsoft Inc. Blog.

The requirements of cloud native VNFs (virtual network functions) for telecom are different than IT applications – and VNF deployment using microservices and containers can help realising cloud-native NFV implementation success.

The best application for NFV is how it will be integrated, architected and further matured to strengthen 5G implementation for telecom service providers. Based on the current pitfalls related to VNF deployment and orchestration, making cloud-native VNF is the only solution in front of service providers today.

Yet telecom applications’ requirements of VNFs are different than any cloud-native IT application. Telecom VNF applications are built for data plane/packet processing functions, along with control, signalling and media processing. An error, or harm to VNF may break down the network and will impact the number of subscribers. Due to such a critical processing requirement, VNFs in telecom should be resilient, offer ultra-high performance, low latency, scalability, and capacity. Telecom VNFs need to be a real-time application having latency sensitivity to fulfil network data, control and signalling processing requirements.

Decomposition of cloud-native VNFs into microservices

VNFs are network functions-embedded software taken out of network peripherals and hosted on virtual machines as an application. Any kind of update to VNFs raises a time-consuming manual effort which hammers overall NFV infrastructure operations. To get ready for cloud native, bundled VNF software needs to be microservices-based, wherein monolithic VNFs are decomposed into different smaller sets of collaborative services having diverse but related functionalities, maintaining their own states, having different infrastructure resources consumption requirements, should be communicated, automatically scaled and orchestrated using well-defined APIs.

There are various benefits of microservice-based VNF decomposition:

• Decomposed VNF sub-services are deployed on hardware which is best suited to be efficiently run and managed. It can scale as needed
• Any error or glitch in the microservice causes failure to only that specific function, which allows easy troubleshooting and enables high availability
• Decomposition allows reusability of service within VNF lifecycle in NFV environment. It also allows some services to get rollout quickly
• Whole VNF becomes lightweight as functions like load balancing and deep packet inspection (DPI) are stripped out from the core application

As VNFs get divided in microservices, service providers may face operation complexity as the number grows. To manage all microservices well in production environment, high level automation needs to be implemented with NFV MANO layer and cloud orchestrator.

Evaluating deployment method of VNF using virtual machine and containers

Containers are a form of virtualisation at the operating system level. It encapsulates application dependencies, required libraries and configuration in a package which is isolated from other containers in the same operating system. Containers allow applications to run in an independent way and can be easily portable.

As a move towards cloud-native, VNF microservices can be deployed in containers which enable the continuous delivery/deployment of large, complex applications. But this approach is still in the early stage for cloud-native NFV.

Concerns with using containers for VNF

To use in NFV, there are certain concerns of using container technology:

• The ecosystem is still evolving and immature compared with virtual machines
• Security risks are involved with containers – all containers in OS share a single kernel, any breach on kernel OS breaks down all containers dependent on it
• Isolating a fault is not easy with containers and a fault can be replicated to subsequent containers

Service providers who may want to use containers in an NFV environment may face challenges in multi-tenancy support, multi-network plane support, forwarding throughput, and limited orchestration capabilities. It is still possible to use containers in mobile edge computing (MEC) environments, which is going to co-exist with NFV in 5G in the future. MEC will be taking user plane function near to the edge of the network, closer to user application to provide very low latency, agility and enable real-time use cases like IoT, augmented reality, or virtual reality.

Containers can possibly be used along with virtual machines in an NFV environment as well. The deployment of VNFs can be virtual machine only, containers only, hybrid – where a container will run in virtual machines providing security and isolation features – and heterogeneous mode, where some VNFs will run in VM, some in containers, alongside a mix of both.

Service providers can evaluate their deployment methods as per their requirements at NFV infrastructure level.

Benefits of containers for cloud-native NFV path

Having a container in place to host microservices can allow active schedule and management to optimise resource utilisation. Container orchestration engines enable provisioning of host resources to containers, assigning containers to hosts, instantiate and reschedule containers. With containers, service providers can realise successful implementation of DevOps methodologies, allowing ease in automation tasks like scaling, upgrading, healing, and become resilient.

A major benefit of containerised microservices is the ability to orchestrate the containers so that separate lifecycle management processes can be applied to each service. This allows for each service to be versioned and upgraded singularly, as opposed to upgrading the entire VNF in virtual machine. While upgrading a whole application or VNF, a container scheduler determines which individual services have changed and deploys only those specific services.

Containers enable cloud-native ability into NFV infrastructure with added performance, portability and agility benefits, for telecom-specific application deployment and orchestration. To have fully featured cloud-native 5G networks, it is imperative for service providers to have containers to deploy more than virtual machines. But service providers will seek further research and development from open source communities like ONAP and OPNFV.

How containers impact NFV at application, infrastructure, and process levels

Applications (VNFs):
– It packages microservices along with its dependencies, libraries and configuration, and make it isolated
– Containers can build quickly with existing images in place for microservices
– Enables faster time to market due to highly automated deployment
– Programmable API enables a complete DevOps approach to be implemented with VNF development, deployment and lifecycle management

Infrastructure (VNF orchestration):
– Containers are portable packages which can move from one environment to another
– Containers can scale in/scale out as per requirement at NFV infrastructure
– Enables higher density
– Enables multi-tenancy to serve multiple requests
– Ease in upgrades and rollbacks as containers allow versioning

Process (VNF deployment):
– Containers can be immutable and can be pushed to any platform
– Allows smooth transition from dev to test to ops
– Enables highly efficient automation
– With containers, service providers can drive continuous integration/deployment to VNF onboarding and lifecycle management

Containers play a vital role on the path to achieve a complete 5G network built with highly automated cloud-native NFV. Successful deployment of 5G will depend on how service providers build a strategy around usage of containers in NFV infrastructure. Aside from the security risks involved in using containers, there might be use case challenges of containers in telecom applications that demand much higher performance. Containerisation can be possibly implemented in mobile edge computing to provide its benefits, but full integration will be expected by service providers to enable cloud-native NFV.

References

• The evolution to cloud-native NFV: early adoption brings benefits with a flexible approach
• Evolving the Mobile Core to Being Cloud Native
• Cloud-Native NFV Architecture for Agile Service Creation & Scaling
• Distributed VNF Management – Architecture and Use Cases
• Are You Being Served? Containers, Microservices, Cloud-Native Telco

The post Evaluating Container Based VNF Deployment For Cloud Native NFV appeared first on Sagar Nangare.

Virtual network functions (VNFs) are software implementations of network function equipment packaged in virtual machines, on top of commercial off the shelf hardware NFV infrastructure. VNFs are a core part of NFV – as we know the base of NFV was to virtualise the network functions and software based to reduce cost and gain full control over network operations with added agility and flexibility benefits. We can say that the majority of NFV operations are focused towards how VNFs can be served in NFV infrastructure to introduce new services for consumers. In future, we can expect major developments will be related to VNFs only.

VNFs and NFV are separated by the fact that VNF are provided by external vendors or open source communities to service providers who are transitioning their infrastructure to NFV. There may be several VNFs which combine to form a single service for NFV. This adds complexity to the overall NFV purpose of agility, where VNFs from different vendors need to deploy in NFV infrastructure having a different operational model.

VNFs developed by different vendors have different methodologies for complete deployment in existing NFV environments. Onboarding VNFs remains a challenge due to a lack of standard processes for complete management from development to deployment and monitoring.

At a basic level, traditional VNFs come with limitations such as:

• VNFs consume huge amounts of hardware in order to be highly available
• VNFs are developed, configured and tested to run for specified NFV hardware infrastructure
• Needs manual installation, configuration and deployment on NFVi
• API not provided for VNF to enable automated scaling, configuration to serve the sudden spike in demand for utilisation
• Not supporting multi-tenancy, VNFs cannot be easily shared in infrastructure for reuse

Building cloud-native VNFs is a solution for vendors and this is a revolution in software development to have all cloud-native characteristics to VNFs. Features we can expect as cloud-native VNFs are containerised functions, microservices-based, dynamically managed and specifically designed for orchestration. The major differentiator of cloud-native VNFs from traditional VNFs can be self-management capability and scalability.

Building cloud-native VNFs overcomes the limitations of traditional VNFs and gives the following benefits. Cloud-native VNFs have APIs which enables:

• Automated installation and configuration
• Automated scaling when dynamic requirement from network
• Self-healing or fault tolerant
• Automated monitoring and analysis of VNFs for errors, capacity management and performance
• Automated upgrading and updating VNFs for applying new releases and patches
• Standard and simplified management enables less power consumption; reduction of unnecessary allocated resources
• Reusability and sharing of processes within VNFs can be achieved. VNFs can be easily shared within an NFV environment

NFV is a key technology used in the development of 5G networks. But NFV is going through a maturation stage where NFV solution providers are resolving many challenges like automated deployment, and VNF onboarding. Developing VNF and deploying into NFV infrastructure sounds simple, but it raises various questions when it comes to scale, configuring or updating VNFs. Any task related to VNFs need manual intervention, leads to more time consumption for launching or updating new services for service providers.

To deliver the promise of agility by NFV in 5G needs exceptional automation at every level of NFV deployment. Building cloud-native VNFs seems to be the solution – but it is at a very early stage.

The post Importance of Cloud-Native VNFs for NFV Success appeared first on Sagar Nangare.