Kubernetes and Data Protection Hits Mainstream with Container Storage Interface (CSI) 1.17

In this blog, we will discuss the growth of containers and how to protect these valuable environments. Kubernetes adoption has rapidly become a part of our daily lives as consumers — usually without our even being aware of it. Kubernetes is at the heart of most digital transformation projects, helping companies large and small to deliver an amazing customer experience.

This technology helps everything along its digital journey, whether ordering a cup of coffee to be picked up, or streaming your favorite video content. There are many examples of how Kubernetes adoption enriched our lives up until a few months ago, and now it has truly become ubiquitous.

In these last few months, many of us might be ordering that coffee, dinner, or groceries, all from our phones or tablets. This is Kubernetes in action. In this respect, Kubernetes is helping us maintain social distancing, while also making it easier for us to navigate our needs. We are all becoming a part of this new paradigm which is helping us to be safe. Enabled by technology, we  are using online freedoms in different ways, and we are even more dependent on availability and speed of access. How these constructs continue to expand and operate is a result of the flexibility that containers provide, and with robust data protection replication options, running new systems from dev/test to production, many organizations are now expanding their online capacity to deal with the volume of online demands.

To understand what lies beneath a flexible environment, let’s start with some background on its evolution. Containers allow abstractions. This gives developers the ability to build software with amazing agility. However, at some point, the environment needs to talk to the storage system, be that external or hyper-converged. In the early days of Kubernetes the environment needed to interface with each storage system differently, similar to how each cellphone manufacturer has its own charger interface; but this is proprietary and unwieldy for those with multiple storage system options.

To address this, the Kubernetes community came up with something called CSI (Container Storage Interface), an interface that enables the environment to talk to all these different storage systems in a uniform way for automating the provisioning, attaching and mounting of block and file storage. In addition, Kubernetes storage interest groups identified snapshot operations as critical functionality for many stateful workloads, as it enables tools to back up the data.

Kubernetes volume snapshot was introduced to navigate this challenge. Kubernetes 1.17 [beta] is now enabled by default and can be part of standard Kubernetes deployments. This feature is becoming widely viable across multiple distributions and, as they gradually adopt the Kubernetes 1.17 version for taking snapshots at the file-level system, this enhances a data protection solution that integrates with the CSI driver. So, this also includes the plain vanilla versions of the Kubernetes distribution, OpenShift, Diamante, Anthos, TKG, and so forth. This is great for many enterprises, which are finally able to extend their enterprise grade data protection solution to Kubernetes environments.

A number of companies are releasing applications in monthly (or even more frequent) builds as part of the DevOps transformation. To accommodate this rapid pace of change, each build check-in includes a combination of application code version, application data schema changes and associated application definition and configuration.

In this rapid pace of change, the probability of failure is high and having a good data protection solution is essential to ensure an equally fast recovery to a stable state. However, most data protection solutions focus primarily on the application data, and these lead to configuration drifts during rollbacks. It is important to protect the associated environment configuration spread across various Kubernetes objects, including ConfigMaps, Secrets, and others. In addition, it is important to have the data protection solutions be available in a self-service mode for DevOps and application owners to manage their own rollbacks and data protection policies for the Kubernetes environments. The IT systems admin would still retain the full access using a common interface that manages the entire infrastructure, including bare metal, virtualized and containerized environments, to help with broader systemic failures.

In summary, Kubernetes environments are being adopted widely across organizations as they move to a microservices, agile and DevOps culture to help with rapid transformations. Organizations recognize that by building these environments with their end users in mind they can provide a better experience. With Kubernetes 1.17 and CSI standards maturing, now is the time for enterprises to adopt them, and look for an enterprise-grade data protection solution to help with protecting these environments.