Building a Kubernetes CRD

January 2023

10 min read
Kubernetes CustomResourceDefinitions (CRDs) are a powerful mechanism for exposing custom objects as native Kubernetes primitives. This article will provide an overview of CRDs and demonstrate how to build a CRD that reveals the dependency details of a deployed microservice.
How to build a Kubernetes CRD
INtroduction

Kuberentes CRDs

By default, Kubernetes exposes a number of resources that you can view and manage using Kubernetes commands. A Kubernetes CRD is an object you expose as a first-class citizen and can manage with standard Kubernetes tools.

A CRD defines and manages objects and includes basic operations that can be performed on them. Additionally, you can create operators to implement custom operations on those objects.

Build a Kubernetes CRD for a CI/CD pipeline

To see this in action, let’s build our own custom resource definition. We’ll build a custom resource definition that captures information about the microservice’s external service dependencies. This can include dependencies for storing data, such as databases and in-memory caches. It can also include dependencies on cloud services, such as secrets managers (for example, Key Vault in Microsoft Azure). 

Our custom resource will be an object containing information about our external dependencies. We will capture the following information for our service:

  • Dependency name
  • Dependency URL
  • Dependency type (should be one of datastore, cloudservice, or microservice)

To do this, we need to create a custom resource definition and apply it to our Kubernetes cluster using the kubectl apply command. Then, we’ll create an instance of the custom resource and upload it using the kubectl apply command.

STEPS to perform

To complete this walkthrough, you’ll need access to a Kubernetes environment. We are using a local environment powered by minikube.

Create a CustomResourceDefinition

CustomResourceDefinitions are YAML-formatted files. Here’s an example definition for our external dependencies resource structure. Save this as a file named dependencies.yaml

apiVersion: "apiextensions.k8s.io/v1"
kind: CustomResourceDefinition
metadata:
  name: dependencies.example.com
spec:
  group: example.com
  scope: Namespaced
  names:
    plural: dependencies
    singular: dependency
    kind: Dependency
    shortNames:
      - dep
  versions:
    - name: v1
      served: true
      storage: true
      schema: 
        openAPIV3Schema:
          type: object
          properties:
            dependency:
              type: object
              properties:
                name:
                  type: string
                  minimum: 3
                url:
                  type: string
                  minimum: 10
                type:
                  type: string
                  minimum: 3
 

apiVersion specifies the version for the CRD extension itself. There are older versions (for example, v1beta1) that uses slightly different YAML formats. Additionally, not all versions may be supported in your deployment of Kubernetes. So make sure the version and syntax specification align. In this example, we’re using CRD version v1.

The metadata section scopes our CRD for API purposes. Kubernetes will automatically create API endpoints that enable basic CRUD (Create, Read, Update, Delete) operations on our objects. The name property will become part of the path of our newly created APIs, which we’ll be able to access at api/dependencies.example.com/v1/.

Next comes the specification for our CRD, defined under spec. Before defining our object, we include some meta-information:

  • group must match the latter part of the namespace we chose under name
  • scope determines whether our CRD is scoped to the cluster or scoped to our namespace. 
  • names controls how we’ll refer to our objects in API calls and using the Kubernetes CLI.

We can also define a shortName to help save on typing. 

After this, we specify our CRD under versions. We can publish multiple versions so that we don’t break backward compatibility with existing tools or CI/CD scripts. The served flag means this version is enabled. 

storage –  must be used for the most recent version to designate it’s the version format used for storage in etcd. Kubernetes will convert requests for previous versions to the appropriate version spec on the fly.

Then, under schema, we define our custom resource using OpenAPI definition syntax. You’ll recognise this format if you’ve created OpenAPI specifications or used Swagger

Now, we will discuss how you can use additional OpenAPI definition features to strengthen type safety and runtime parameter checking.

Upload our definition and create objects

After creating our CustomResourceDefinition, we can upload it to our Kubernetes cluster using the kubectl apply command:

kubectl apply -f dependencies.yaml 

If your CRD is syntactically correct, Kubernetes will create it immediately. The CRD is simply a template for a custom resource. To store resources, you must create them in a separate YAML file and upload them to your cluster. The file below creates a dependency object that conforms to our custom resource definition. Save this to a file named dependency-cr.yaml.

apiVersion: "example.com/v1"
kind: Dependency
metadata:
  name: api-dependency
dependency:
  name: storageapi
  url: https://example.com/storageapi/v1/
  type: microservice 

Key considerations:

  • The apiVersion field is our spec group combined with the version of our custom resource definition.
  • name will appear in URL paths, so Kubernetes enforces the use of RFC 1123 syntax. This means you must use only lowercase alphanumeric characters (a-z, 0-9) or the ‘-’ and ‘.’ characters.
  • dependency and all of its fields must correspond to the OpenAPI definition we published earlier. 

Save this to a file named dependency-cr.yaml and upload it to Kubernetes: 

kubectl apply -f dependency-cr.yaml 

Operate on a custom resource with kubectl

Once published, you can use kubectl commands to manage your dependency objects. To list the object you just created, run the below command: 

kubectl get dependencies 
Amnics-MacBook-Pro: tmp amnic$ kubectl get dependencies

NAME                AGE
api-dependency       9s 

For dependencies, we can use the shortName we defined earlier:

kubectl get dep 

You can use other basic kubectl commands to manage your custom resource. To view the full details of the resource, run the below command:

kubectl get dep api-dependency -o yaml 

To delete it, run the below command: 

kubectl delete dep api-dependency 

Apply data constraints

We can enhance our CRD by incorporating further restrictions on the values that can be assigned to specific fields. For instance, as previously mentioned, we will only permit three options for the type field: datastore, cloudservice, and microservice.

Add the below snippet to our file dependencies.yaml:

enum:
 - datastore
 - cloudservice
 - microservice
 

After adding the above snippet to our file, it should look like:

type: string
minimum: 3
enum:
 - datastore
 - cloudservice
 - microservice
 

You can force update your existing CRD like so: 

kubectl replace -f dependencies.yaml 

Now, try and publish a custom dependency resource with an unsupported value for type. Create another file, dependency-cr-1.yaml, with the below content: 

apiVersion: "example.com/v1"
kind: Dependency
metadata:
  name: api-dependency-1
dependency:
  name: storageapi
  url: https://example.com/storageapi/v1/
  type: macroservice 

Now, let’s try to apply the newly made changes:

kubectl apply -f dependency-cr-1.yaml 

This command will result in an error indicating that macroservice is not a supported type.

CONCLUSION

In this blog, we demonstrated how to set up and utilize custom resources in Kubernetes. In our next post, we will delve deeper by constructing a custom Kubernetes operator.

Amnic Share Icon
Building Blocks Jan 2023 Thumbnail
Building Blocks | January 2023
Read More
Hero banner image for December 2022
Building Blocks | December 2022
Read More