Kubernetes CustomResourceDefinitions (CRDs) are a powerful mechanism for exposing custom objects as native Kubernetes primitives. This article will provide an overview of CRDs and demonstrate how to build a CRD that reveals the dependency details of a deployed microservice.
INtroduction
Kubernetes CRDs
In the realm of Kubernetes, a multitude of resources are readily accessible and manageable through standard Kubernetes commands. Yet, when the need arises to extend Kubernetes capabilities for custom objects, Custom Resource Definitions (CRDs) emerge as a pivotal solution.
Kubernetes Custom Resource Definitions (CRDs) transform an object into a first-class citizen within the Kubernetes ecosystem. This means you can harness the full potential of standard Kubernetes tools to interact with and oversee these custom objects, just as you would with the built-in Kubernetes resources.
Build a Kubernetes CRD for a CI/CD pipeline
To witness Kubernetes in action, let’s embark on a journey to create our own Custom Resource Definition (CRD). In this endeavor, we’ll craft a CRD that serves as a repository for critical data about your microservice’s external service dependencies. These dependencies span a wide spectrum, including data storage components like databases and in-memory caches, as well as connections to vital cloud services, such as secrets managers like Key Vault in Microsoft Azure.
Our custom resource will be an object containing information about our external dependencies. We will capture the following information for our service:
- Dependency name
- Dependency URL
- Dependency type (should be one of
datastore,cloudservice, ormicroservice)
To do this, we need to create a custom resource definition and apply it to our Kubernetes cluster using the kubectl apply command. Then, we’ll create an instance of the custom resource and upload it using the kubectl apply command.
STEPS to perform
To complete this walkthrough, you’ll need access to a Kubernetes environment. We are using a local environment powered by minikube.
Create a CustomResourceDefinition
CustomResourceDefinitions are YAML-formatted files. Here’s an example definition for our external dependencies resource structure. Save this as a file named dependencies.yaml.
apiVersion: "apiextensions.k8s.io/v1"
kind: CustomResourceDefinition
metadata:
name: dependencies.example.com
spec:
group: example.com
scope: Namespaced
names:
plural: dependencies
singular: dependency
kind: Dependency
shortNames:
- dep
versions:
- name: v1
served: true
storage: true
schema:
openAPIV3Schema:
type: object
properties:
dependency:
type: object
properties:
name:
type: string
minimum: 3
url:
type: string
minimum: 10
type:
type: string
minimum: 3
apiVersion specifies the version for the CRD extension itself. There are older versions (for example, v1beta1) that uses slightly different YAML formats. Additionally, not all versions may be supported in your deployment of Kubernetes. So make sure the version and syntax specification align. In this example, we’re using CRD version v1.
The metadata section scopes our CRD for API purposes. Kubernetes will automatically create API endpoints that enable basic CRUD (Create, Read, Update, Delete) operations on our objects. The name property will become part of the path of our newly created APIs, which we’ll be able to access at api/dependencies.example.com/v1/.
Next comes the specification for our CRD, defined under spec. Before defining our object, we include some meta-information:
groupmust match the latter part of the namespace we chose undername.scopedetermines whether our CRD is scoped to the cluster or scoped to our namespace.namescontrols how we’ll refer to our objects in API calls and using the Kubernetes CLI.
We can also define a shortName to help save on typing.
After this, we specify our CRD under versions. We can publish multiple versions so that we don’t break backward compatibility with existing tools or CI/CD scripts. The served flag means this version is enabled.
storage – must be used for the most recent version to designate it’s the version format used for storage in etcd. Kubernetes will convert requests for previous versions to the appropriate version spec on the fly.
Then, under schema, we define our custom resource using OpenAPI definition syntax. You’ll recognise this format if you’ve created OpenAPI specifications or used Swagger.
Now, we will discuss how you can use additional OpenAPI definition features to strengthen type safety and runtime parameter checking.
Upload our definition and create objects
After creating our CustomResourceDefinition, we can upload it to our Kubernetes cluster using the kubectl apply command:
kubectl apply -f dependencies.yaml If your CRD is syntactically correct, Kubernetes will create it immediately. The CRD is simply a template for a custom resource. To store resources, you must create them in a separate YAML file and upload them to your cluster. The file below creates a dependency object that conforms to our custom resource definition. Save this to a file named dependency-cr.yaml.
apiVersion: "example.com/v1"
kind: Dependency
metadata:
name: api-dependency
dependency:
name: storageapi
url: https://example.com/storageapi/v1/
type: microservice Key considerations:
- The
apiVersionfield is our spec group combined with the version of our custom resource definition. namewill appear in URL paths, so Kubernetes enforces the use of RFC 1123 syntax. This means you must use only lowercase alphanumeric characters (a-z, 0-9) or the ‘-’ and ‘.’ characters.dependencyand all of its fields must correspond to the OpenAPI definition we published earlier.
Save this to a file named dependency-cr.yaml and upload it to Kubernetes:
kubectl apply -f dependency-cr.yaml Operate on a custom resource with kubectl
Once published, you can use kubectl commands to manage your dependency objects. To list the object you just created, run the below command:
kubectl get dependencies Amnics-MacBook-Pro: tmp amnic$ kubectl get dependencies
NAME AGE
api-dependency 9s For dependencies, we can use the shortName we defined earlier:
kubectl get dep You can use other basic kubectl commands to manage your custom resource. To view the full details of the resource, run the below command:
kubectl get dep api-dependency -o yaml To delete it, run the below command:
kubectl delete dep api-dependency Apply data constraints
We can enhance our CRD by incorporating further restrictions on the values that can be assigned to specific fields. For instance, as previously mentioned, we will only permit three options for the type field: datastore, cloudservice, and microservice.
Add the below snippet to our file dependencies.yaml:
enum:
- datastore
- cloudservice
- microservice
After adding the above snippet to our file, it should look like:
type: string
minimum: 3
enum:
- datastore
- cloudservice
- microservice
You can force update your existing CRD like so:
kubectl replace -f dependencies.yaml Now, try and publish a custom dependency resource with an unsupported value for type. Create another file, dependency-cr-1.yaml, with the below content:
apiVersion: "example.com/v1"
kind: Dependency
metadata:
name: api-dependency-1
dependency:
name: storageapi
url: https://example.com/storageapi/v1/
type: macroservice Now, let’s try to apply the newly made changes:
kubectl apply -f dependency-cr-1.yaml This command will result in an error indicating that macroservice is not a supported type.
CONCLUSION
In this blog, we demonstrated how to set up and utilize custom resources in Kubernetes. In our next post, we will delve deeper by constructing a custom Kubernetes operator.
