Container termination grace period is not getting honoured
We are using windows containers managed by the Kubernetes where we have the terminationGracePeriodSeconds
as 10mins. To handle the shutdown notification, we are doing something like this https://gist.github.com/darstahl/fbb80c265dcfd1b327aabcc0f3554e56. Once we catch the SIGTERM
signal, we are doing our operations of gracefully shutting down all the services. In that sequence, we have a Sleep of 30sec duration.
The issue is container is getting terminated when it hits the Sleep, and not continuing the further operations, though the overall termination grace period is very large.
Any idea on the issue? and how to fix it?
As I mentioned in comments I think what you need is a PreStop hook.
Container hooks
There are two hooks that are exposed to Containers:
PostStart
This hook executes immediately after a container is created. However, there is no guarantee that the hook will execute before the container ENTRYPOINT. No parameters are passed to the handler.
PreStop
This hook is called immediately before a container is terminated due to an API request or management event such as liveness probe failure, preemption, resource contention and others. A call to the preStop hook fails if the container is already in terminated or completed state. It is blocking, meaning it is synchronous, so it must complete before the call to delete the container can be sent. No parameters are passed to the handler.
A more detailed description of the termination behavior can be found in Termination of Pods.
Additionally there are few tutorials with examples how PreStop hook.