Canary Style Deployments in Kubernetes

I will be using Digital Oceans managed Kubernetes service for this. It’s low cost compared to other cloud providers.

This example uses the Ingress Nginx controller for Kubernetes. Here is the installation guide depending on your cloud provider. The diagram below illustrates the flow we want to implement. We will only focus from the ingress controller down.

Here is the config file that will deploy the first app that is Pod A. I will share the entire file for this pod, it will contain the deployment, service and ingress.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello-world
  labels:
    app: hello-world
spec:
  replicas: 1
  selector:
    matchLabels:
      app: hello-world
  template:
    metadata:
      labels:
        app: hello-world
    spec:
      containers:
      - name: hello-world
        image: gcr.io/google-samples/node-hello:1.0
        imagePullPolicy: Always
        ports:
          - containerPort: 8080
        readinessProbe:
          httpGet:
            path: /
            port: 8080
          initialDelaySeconds: 5
          periodSeconds: 5
        resources:
           limits:
             memory: "128Mi"
             cpu: "0.5m"
---
apiVersion: v1
kind: Service
metadata:
  name: hello-world-service
spec:
  ports:
  - port: 80
    targetPort: 8080
  selector:
    app: hello-world
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: hello-world-ingress
  annotations:
spec:
  ingressClassName: nginx
  rules:
  - host: k8s.littycities.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: hello-world
            port:
              number: 80

The host is set to k8s.littycities.com. So when a user goes to that fake subdomain it would hit Pod A. Now below I will share the config for the Pod B deployment and service. I’ll share the ingress for this service below it since its the one that will be setup for a canary style deployment.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello-world-3
  labels:
    app: hello-world-3
spec:
  replicas: 1
  selector:
    matchLabels:
      app: hello-world-3
  template:
    metadata:
      labels:
        app: hello-world-3
    spec:
      containers:
      - name: hello-world-3-container
        image: ghcr.io/aldomatic/docker-image-host-github
        imagePullPolicy: Always
        ports:
          - containerPort: 3000
        readinessProbe:
          httpGet:
            path: /
            port: 3000
          initialDelaySeconds: 300
          periodSeconds: 30
        resources:
         # Specifying the resourses that we might need for our application
           limits:
             memory: "128Mi"
             cpu: "0.5m"
---
apiVersion: v1
kind: Service
metadata:
  name: hello-world-3-service
spec:
  ports:
  - port: 80
    targetPort: 3000
  selector:
    app: hello-world-3

Now the good stuff. Here is the config for Pod B. This will allow us to access Pod B via a custom header value. This is the only way we can access this service. If we don’t pass it then we get Pod A.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: hello-world-3-ingress
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: "x-canary"
spec:
  rules:
  - host: k8s.littycities.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: hello-world-3-service
            port:
              number: 80

If the create/apply of these files goes well you should be able to access Pod B via the x-canary header. If you don’t pass in the header value then you default to Pod A.

curl -H “x-canary: always” http://k8s.littycity.com

This is nice way to launch pre-prod release or test out a new version of your application. I have made several assumptions here. but reach out if anything is not clear.