Graceful Shutdowns for Services on Kubernetes

If you’re running services in the cloud or on Kubernetes, your services need to be able to handle shutdowns.

A pod can shut down for many reasons, such as being evicted from scaling down nodes.

There’s a grace period that you can configure with terminationGracePeriodSeconds. It defaults to 30 seconds. You can read more about pod lifecycles here.

You want requests to your API services to be completed within the grace period. Otherwise, the pod will be forcefully shut down and return an error to the caller.

The general pattern is to design your applications and services to handle SIGTERM. One of the first things Kubernetes does is send SIGTERM when it tries to stop a pod. Once the grace period ends, it’ll send a SIGKILL.

Once you get the SIGTERM, your service should stop accepting new requests, complete in-flight requests, and close any stateful resources, such as database connections.

Some major frameworks, such as Spring Boot, already have a flag you can enable to shut down gracefully.

Resources

• Kubernetes best practices: terminating with grace

---