This error typically happens when BuildKit is not shut down correctly. Some of our community members have seen this when running the Buildkit pods on ephemeral VMs (e.g. spot instances) or when using autoscalers like Karpenter.
Buildkit is a stateful component (e.g. it keeps a cache of images and previous builds), so we don’t recommend deploying it to ephemeral pods, or attaching them to autoscalers.
If you run into the issue described above, we recommend that you manually delete the pod.
kubectl -n <ns> exec -it <buildkit-pod> -- sh -lc 'rm -f /var/lib/buildkit/buildkitd.lock'
kubectl -n <ns> delete pod <buildkit-pod>
If the issue continues to happen, delete the lock file manually, and then delete the podBuildKit instance as well as the BuildKit pod.
kubectl -n <ns> exec -it <buildkit-pod> -c buildkitd -- sh -lc 'rm -f /var/lib/buildkit/buildkitd.lock'
kubectl -n <ns> delete pod <buildkit-pod>