Rollback Kubernetes deployments

Rollback functionality in Kubernetes allows you to revert a deployment to a previous version if something goes wrong with a newer deployment. This feature is vital for ensuring application stability and mitigating the risk of downtime caused by faulty releases.

In this guide, we’ll explore Kubernetes rollbacks in depth, including the theory, detailed steps for implementation, multiple scenarios, and platform-specific nuances.

What is a Kubernetes Rollback deployment?

Kubernetes keeps a history of deployments in the form of revisions, enabling you to revert to a previous state if required. Each deployment update (e.g., changing an image, replicas, or environment variables) creates a new revision. The rollback process allows you to:

1. Restore a deployment to a specific revision.
2. Automatically undo a failed update.
3. Debug and resolve deployment issues effectively.

Understanding Deployment Revisions

Each deployment maintains a history of revisions, which can be viewed using:

kubectl rollout history deployment <DEPLOYMENT_NAME>

Example output:

REVISION  CHANGE-CAUSE
1         kubectl set image deployment/nginx-deployment nginx=nginx:1.19.0
2         kubectl set image deployment/nginx-deployment nginx=nginx:1.20.0
3         kubectl set image deployment/nginx-deployment nginx=nginx:1.21.0

• Revision 1: Initial deployment.
• Revision 2: Updated to nginx:1.20.0.
• Revision 3: Updated to nginx:1.21.0.

Rollback Kubernetes Deployments Commands

• Rollback to the previous revision:

  kubectl rollout undo deployment <DEPLOYMENT_NAME>
  

• Rollback to a specific revision:

  kubectl rollout undo deployment <DEPLOYMENT_NAME> --to-revision=<REVISION_NUMBER>
  

• View the deployment status:

  kubectl rollout status deployment <DEPLOYMENT_NAME>
  

Step-by-Step Rollback Implementation

1. Prerequisites

• Kubernetes CLI (kubectl): Ensure kubectl is installed and configured to access your cluster.
• Access to the cluster: Verify access with:

  kubectl cluster-info
  

2. Deployment Setup

Create a Deployment

Create a sample deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.19.0
        ports:
        - containerPort: 80

Apply the deployment:

kubectl apply -f nginx-deployment.yaml

Verify the deployment:

kubectl get deployments

3. Update the Deployment

Update the deployment to a newer image version:

kubectl set image deployment/nginx-deployment nginx=nginx:1.20.0 --record

Check the rollout status:

kubectl rollout status deployment/nginx-deployment

List deployment revisions:

kubectl rollout history deployment/nginx-deployment

4. Trigger a Rollback

Rollback to the Previous Revision

If the new version (e.g., nginx:1.20.0) causes issues, rollback to the previous version:

kubectl rollout undo deployment/nginx-deployment

Confirm the rollback:

kubectl get deployments

Rollback to a Specific Revision

Rollback to a specific revision (e.g., Revision 1):

kubectl rollout undo deployment/nginx-deployment --to-revision=1

5. Verify the Rollback

View the current image version:

kubectl describe deployment nginx-deployment | grep Image

Check the deployment status:

kubectl rollout status deployment/nginx-deployment

Scenarios for Rollback

Scenario 1: Immediate Rollback After a Failed Update

Problem:

After updating a deployment (e.g., nginx:1.20.0), the application crashes due to a bug.

Solution:

1. Check the deployment status:

   kubectl rollout status deployment/nginx-deployment
   

2. Rollback to the previous version:

   kubectl rollout undo deployment/nginx-deployment
   

3. Verify the rollback:

   kubectl get pods
   

Expected Outcome:

The deployment restores to the last working version (e.g., nginx:1.19.0), resolving the issue.

Scenario 2: Rollback to a Stable Revision

Problem:

A recent update to nginx:1.21.0 introduces performance issues. You want to rollback to a stable version (e.g., nginx:1.19.0).

Solution:

1. Check deployment history:

   kubectl rollout history deployment/nginx-deployment
   

2. Rollback to Revision 1:

   kubectl rollout undo deployment/nginx-deployment --to-revision=1
   

3. Confirm the rollback:

   kubectl describe deployment nginx-deployment | grep Image
   

Expected Outcome:

The deployment is restored to nginx:1.19.0.

Scenario 3: Rollback During a Deployment

Problem:

A new version is being deployed, but errors are detected mid-rollout.

Solution:

1. Pause the rollout:

   kubectl rollout pause deployment/nginx-deployment
   

2. Undo the changes:

   kubectl rollout undo deployment/nginx-deployment
   

3. Resume the rollout (optional):

   kubectl rollout resume deployment/nginx-deployment
   

Expected Outcome:

The deployment halts, reverts to the last stable state, and resumes operations.

Rollback Across Different Operating Systems

The rollback process is the same across different operating systems as long as kubectl is configured correctly. Below are OS-specific considerations:

Linux

• Install kubectl:

  sudo apt-get install kubectl
  

• Ensure sufficient permissions to access the Kubernetes cluster:

  sudo kubectl get deployments
  

MacOS

• Install kubectl using Homebrew:

  brew install kubectl
  

• Use the ~/.kube/config file to manage cluster contexts.

Windows

• Install kubectl using Chocolatey:

  choco install kubernetes-cli
  

• Use PowerShell to run commands:

  kubectl get pods
  

Best Practices for Rollbacks

1. Always Record Updates: Use the --record flag when making changes to deployments to track the change cause:

   kubectl set image deployment/nginx-deployment nginx=nginx:1.20.0 --record
   

2. Monitor Rollouts: Monitor the rollout status after updates:

   kubectl rollout status deployment/nginx-deployment
   

3. Backup Configurations: Export deployment configurations before updates:

   kubectl get deployment nginx-deployment -o yaml > backup.yaml
   

4. Limit Revision History: Configure the revisionHistoryLimit to manage the number of stored revisions:

   spec:
     revisionHistoryLimit: 5
   

5. Use Health Checks: Define readiness and liveness probes to detect issues early:

   readinessProbe:
     httpGet:
       path: /
       port: 80
   

Limitations of Kubernetes Rollbacks

1. Stateful Sets: Rollbacks are not directly supported for StatefulSets. Manual intervention may be required.
2. Data Dependencies: Rollbacks won’t revert database or external system changes.
3. Revision History Limit: Revisions older than the revisionHistoryLimit are purged and cannot be rolled back.

Conclusion

Kubernetes rollbacks provide a robust mechanism to maintain application stability during updates. By understanding how to use the rollback commands effectively, you can ensure minimal downtime and rapid recovery in case of deployment failures.

Key Takeaways

1. Rollbacks restore deployments to a previous working state.
2. Always monitor deployment rollouts and maintain revision history.
3. Automate rollbacks with CI/CD pipelines for faster recovery.

This guide equips you with the knowledge and tools to handle deployment rollbacks efficiently across various scenarios and operating systems. Let me know if you need further clarification or additional examples!

References

• Kubernetes Official Documentation: Rolling Back a Deployment
• Kubernetes Official Documentation: kubectl rollout undo Command Reference
• Learnk8s: How do you rollback deployments in Kubernetes?
• What Are Kubernetes Containers?