What is a Kubernetes Operator?

If you're running a simple, stateless application like a web server on Kubernetes, it's very easy for the platform to replace or repair that application automatically because each instance is interchangeable – there's no state to worry about.

But what if you're running something more complex, like a full-stack application that needs a database and has some concept of state?

Well, deploying a database on Kubernetes is in theory pretty easy – you find an image and then deploy it using a statefulset in a K8 cluster. But, running that database in production over time is hard. When your database needs to be resized or upgraded, unlike web servers, you can't just add a new one. You need to configure the new member to join with the existing members, define read/write follower relationships, and add secrets. For each of these tasks a dev-ops engineer or SRE would have to manually intervene, either creating new yaml or editing existing ones.

In other words, even though the promise of Kubernetes is automation, running a stateful complex application on Kubernetes in production still involves a lot of manual labor.

Kubernetes Operators Framework

Kubernetes operators are an attempt to solve for the on-going challenge of running a stateful complex application on Kubernetes. If you recall from earlier, the central working paradigm of Kubernetes is the control loop, where a controller:

1. observes the state of the resources in your cluster
2. diffs the current state of your cluster vs. what you want it to be
3. resolves that diff by acting on it.

But Kubernetes' list of built-in/default resources is short, and includes only very broad concepts like Pods, Deployments, Service, Persistent Volume Claims and Stateful set.

The Kubernetes Operator Framework allows developers to define custom resource definitions (CRD's) that are specific to the internals of complex stateful applications.

For example, the following yaml defines a new CRD kind, SampleDB.

---
#Defines the CRD 'SampleDB'
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: sampledb.samplecontroller.k8s.io
spec:
  group: samplecontroller.k8s.io
  version: v1alpha1
  names:
    kind: SampleDB
    plural: SampleDBs
  scope: Namespaced
---
# SampleDB Kind
apiVersion: samplecontroller.k8s.io/v1alpha1
kind: SampleDB
metadata:
  name: example-sample-db
spec:
  deploymentName: example-sample-db
  replicas: 1

The CRD alone won't do anything; the developer then needs to implement a custom controller in Go to do stuff with/to the SampleDB CRD, something that looks like this:

deployment, err := r.createDeployment(watchlist, redis)
if err != nil {
	return ctrl.Result{}, err
}
svc, err := r.createService(watchlist)
if err != nil {
	return ctrl.Result{}, err
}

applyOpts := []client.PatchOption{client.ForceOwnership, client.FieldOwner("watchlist-controller")}

err = r.Patch(ctx, &deployment, client.Apply, applyOpts...)
if err != nil {
	return ctrl.Result{}, err
}

err = r.Patch(ctx, &svc, client.Apply, applyOpts...)
if err != nil {
	return ctrl.Result{}, err
}

The role of the controller is to listen for changes on the custom resource it manages and manipulate the state of the system in response.

For example, if you add a new SampleDB, the operator might set up a PersistentVolumeClaims to provide durable database storage, a StatefulSet to run SampleDB and a Job to handle initial configuration. Upon deletion, the operator makes sure that the StatefulSet and Volumes are also removed.

Together, the CRD and custom controller are deployed onto the cluster as an operator. Many Kubernetes operators for popular stateful applications like Postgres or MySQL have been open-sourced, which means as a Kubernetes user, you don't actually have to write your own. You can just install and deploy the operator on your cluster (or specify it in your helm chart) and from there, the cluster will take over. You are then able to perform various application-specific actions against the Kubernetes API using kubetcl. For instance, creating a postgres cluster having deployed the postgres operator:

kubectl delete postgresql acid-minimal-cluster

Kubernetes operator tutorial

Operators are an invaluable building block for dealing with specific applications on Kubernetes, and extensively used at Plural. For a walkthrough of how to build a Kubernetes Operator, check out this tutorial from Kubebuilder: writing operators generally involves a large amount of boilerplate. Kubebuilder helps speed up the process by auto-generating much of that.

If you’re interested in learning more about how Plural works and how we can help your engineering team deploy open-source applications in a cloud production environment, reach out to myself and the rest of the team over at Plural.

Ready to effortlessly deploy and operate open source applications in minutes? Get started with Plural today.

Join us on our Discord channel for questions, discussions, and to meet the rest of the community.