ImagePullBackOff on the New Node Pool
The report, in their words
Roughly half of a service's pods are Running and the other half are stuck in ImagePullBackOff. The deployment went out four hours ago and was fine. This morning the cluster gained a second node pool for capacity, and every pod that lands on the new nodes fails to pull the image.
Pull evidence
One item at a time, and nothing here tells you which one matters. Deciding what is worth looking at is most of the diagnosis.
kubectl describe pod — the pull events
kubectl get pods -o wide
How the cluster authenticates to the registry
The two node pools' identities
Does the tag exist in the registry?
Network reachability from a pool-b node
Registry rate limiting
Node disk on pool-b
0 of 8 inspected. You are not required to open all of them — a real investigation is judged on how few you needed.
What is your diagnosis?
Commit to one. Nothing below is shown until you do.
Guessing wrong and being told exactly why is the point of this page. Reading the answer first is not practice.