Provider.DeletePod() currently sends the remote /delete request asynchronously and immediately removes the pod from p.pods.
If the request fails because the Slurm plugin or network is temporarily unavailable:
If the request fails because the Slurm plugin or network is temporarily unavailable:
- the error is only logged;
- the pod is no longer tracked by the virtual kubelet;
- the status loop stops reporting it;
- no later retry is attempted;
- the corresponding Slurm job may continue running indefinitely
Deletion should be made reliable by either waiting for the remote deletion to succeed before removing the pod from p.pods, or by adding a durable retry mechanism. The pod should remain eligible for cleanup retries until the remote side confirms deletion.
Please add tests covering transient /delete failures and successful retry on a later attempt.
Provider.DeletePod()currently sends the remote/deleterequest asynchronously and immediately removes the pod fromp.pods.If the request fails because the Slurm plugin or network is temporarily unavailable:
If the request fails because the Slurm plugin or network is temporarily unavailable:
Deletion should be made reliable by either waiting for the remote deletion to succeed before removing the pod from
p.pods, or by adding a durable retry mechanism. The pod should remain eligible for cleanup retries until the remote side confirms deletion.Please add tests covering transient
/deletefailures and successful retry on a later attempt.