Kubernetes + Plural

At Plural, we're trying to make the deployment of whatever software a developer organization wants to use a simple, consistent process.  For anyone who has had experience wrangling distributed systems since perhaps three years ago, the experience was anything but that, and this goal would have been practically unattainable. Fortunately, with the advent of containerization and now kubernetes, there is finally a platform powerful enough for us to manage this.  But how?  Let's first take an analogy to manufacturing process breakthroughs from the prior century.

The Ford Model T was groundbreaking not so much because it was a better car, but because it was a car made out of standardized, interchangeable parts. It had strict definitions for things like tires, body paneling, and engine components. This enabled easy spin-up of new factories to produce Model T's, training of employees at those factories, and low servicing costs when they broke down.

Similarly to the Ford Model T, Kubernetes brought standardization to distributed systems. Pre-Kubernetes, everything was bespoke – engineers managed distributed systems using a motley assortment of one-off cloud consoles, bash scripts, and Python scripts.

This is the documentation for installing Looker on-prem. It's an outrageous amount of instructions that no one can be expected to follow correctly. And Looker isn't even particularly old tech. This is the kind of installation that devops engineers are regularly expected to deal with.

With Kubernetes, for the first time, you had a way of describing deployment logic, config management, networking, rbac policies, and ACL rules, that was interchangeable across on-prem or any cloud provider. You could declaratively specify what a distributed system should look like within a set of yaml files, package these files up, and move them around between different clusters. Standardization enabled portability.

The Rise of Managed Kubernetes

Kubernetes abstracted away certain details of distributed systems management, but it also exposed others. Prior to 2018 and the introduction of managed Kubernetes services, in order to use Kubernetes, you would have to run a Kubernetes cluster yourself. This involved dealing with things like node lifecycle management, deploying new hosts, making sure the hosts were kept up to date with the latest OS patches and container runtime versions, and setting up observability for the cluster.

Today, you can pay a Kubernetes managed service like EKS or GKE $100/month to do that all for you. What's left then is the detail around the deployment and maintenance of specific applications like Grafana, Sentry, or Airflow. Each of these applications still has its own quirks, features, and operational modes.

At Plural, we're tackling this problem by building tooling that automates the process of installing a whole suite of open-source applications and their dependencies.

In doing this we have been able to leverage the Kubernetes Operator Platform, which allows developers to write extensions of the Kubernetes API for managing specific applications. For instance, the Postgres operator provides a robust way to deploy the database, a prerequisite for many larger full-stack applications, on Kubernetes. We have also written several lightweight operators specific to Plural, and include other popular open-source ones.

Ultimately, Plural provides value by handling both Kubernetes and the configuration and integration of third-party, non-Kubernetes resources like S3 buckets, Single Sign-On, and security. The end result is that companies can get their core infra, deployed and maintained out-of-the-box on managed Kubernetes, at roughly the same amount of effort, but half the cost, of an AWS managed service. It's a state of affairs that would have been impossible even a few years ago, but is possible now.

Yiren Lu

Yiren Lu