Istio production rollout checklist

A practical, step-by-step checklist for safely rolling Istio into production.

1. Pre-Rollout Readiness

✅ Architecture & Scope

• Microservices architecture is stable
• Kubernetes cluster is healthy and monitored
• Services use supported protocols (HTTP / gRPC / TCP)
• Target namespaces for Istio are clearly defined
• External dependencies identified (databases, SaaS, third-party APIs)

✅ Team & Process Readiness

• Team understands basic Istio concepts
• On-call ownership defined
• Rollback plan documented
• Change window approved
• Runbooks updated

2. Cluster & Resource Preparation

✅ Capacity Planning

• CPU headroom available
• Memory headroom available
• Pod resource requests and limits reviewed
• Node autoscaling tested

Envoy sidecars typically add ~50–100MB memory per pod

✅ Kubernetes Baseline

• PodDisruptionBudgets configured
• Liveness probes configured
• Readiness probes configured
• HPA tested
• NetworkPolicies reviewed

3. Istio Installation (Production-Safe)

✅ Installation Profile

• Using default or custom production profile
• demo profile NOT used
• Ingress gateway installed
• Control plane configured for HA

✅ Control Plane Health

• istiod running with multiple replicas
• No crash loops
• Leader election functioning
• istioctl analyze shows no critical errors

4. Namespace & Sidecar Strategy

✅ Namespace Enablement

• Start with a low-risk namespace
• Sidecar injection enabled via namespace label
• Cluster-wide injection avoided

✅ Injection Validation

• Pods contain exactly one Envoy sidecar
• No pod startup delays
• Application logs unchanged

5. Observability (Required)

✅ Metrics

• Request latency visible
• Error rates visible (4xx / 5xx)
• Throughput metrics visible

✅ Tracing

• Distributed tracing enabled
• Sampling rate reviewed
• Trace propagation verified

✅ Dashboards

• Kiali accessible
• Prometheus scraping verified
• Grafana dashboards validated

6. Traffic Management (Start Simple)

✅ Baseline Policies

• Timeouts defined
• Conservative retries configured
• Load balancing verified

🚫 Avoid Initially

• Canary releases
• Traffic mirroring
• Fault injection
• Complex routing rules

7. Security Rollout (Gradual Zero Trust)

✅ Phase 1: PERMISSIVE mTLS

• mTLS enabled in PERMISSIVE mode
• No service communication failures
• External traffic tested

✅ Phase 2: STRICT mTLS

• All services verified compatible
• Legacy workloads handled
• STRICT mTLS enabled incrementally

✅ Authorization Policies

• Default-deny NOT enabled initially
• Service identities validated
• Policies reviewed with security team

8. Ingress & Egress Safety

✅ Ingress

• TLS termination strategy defined
• Rate limiting configured
• Health checks verified

✅ Egress

• External traffic paths documented
• Egress rules defined (if using REGISTRY_ONLY)
• DNS resolution validated

9. Performance & Stability Validation

✅ Load Testing

• Baseline tests before Istio
• Load tests after Istio rollout
• Latency impact measured and accepted

✅ Failure Testing

• Pod restart behavior tested
• Network latency simulated
• Dependency failure behavior validated

10. Rollout & Expansion Strategy

✅ Production Rollout

• Expand namespace by namespace
• Metrics monitored after each rollout
• Rollout paused on error spikes

✅ Rollback Readiness

• Namespace label removal tested
• Sidecar removal verified
• Istio uninstall tested in staging

11. Post-Rollout Hardening

✅ Security

• STRICT mTLS enforced where possible
• Authorization policies applied
• Certificate rotation verified

✅ Operations

• Alerts configured
• Runbooks finalized
• Upgrade strategy defined
• Istio versions tracked

12. Ongoing Maintenance

• Monitor Envoy memory usage
• Review retry policies regularly
• Audit traffic rules quarterly
• Test disaster recovery
• Stay within supported Istio versions

Final Go-Live Approval

• Metrics stable
• Error rates acceptable
• Latency acceptable
• Rollback tested
• Team trained