Can't delete scan or scan CRD #1101
Description
🐞 Bug report
Describe the bug
I was testing my setup using IAM roles. I had a few problems setting up a scan due to permissions on S3. I fixed them but then I want to delete a scan object I created and the delete gets stuck.
I run:
% kubectl get scan
NAME TYPE STATE FINDINGS
zap-full-scan-admin zap-full-scan
% kubectl delete scan zap-full-scan-admin
scan.execution.securecodebox.io "zap-full-scan-admin" deleted
... then it gets stuck here for one hour until my AWS session expires then:
error: You must be logged in to the server (Unauthorized)
exit 1
I tried uninstalling the zap scanner and operator using helm, both uninstalled fine, but the zap scan was still there and still wouldn't delete.
So I went to the uninstallation steps page ( https://docs.securecodebox.io/docs/getting-started/uninstallation ) and deleted lurkers (there were no parsers), roles, binding and service accounts. All went good.
However when I tried to delete the CRDs it got stuck in the scheduled scans:
% kubectl delete crd cascadingrules.cascading.securecodebox.io \
parsedefinitions.execution.securecodebox.io \
scancompletionhooks.execution.securecodebox.io \
scans.execution.securecodebox.io \
scantypes.execution.securecodebox.io \
scheduledscans.execution.securecodebox.io
customresourcedefinition.apiextensions.k8s.io "cascadingrules.cascading.securecodebox.io" deleted
customresourcedefinition.apiextensions.k8s.io "parsedefinitions.execution.securecodebox.io" deleted
customresourcedefinition.apiextensions.k8s.io "scancompletionhooks.execution.securecodebox.io" deleted
customresourcedefinition.apiextensions.k8s.io "scans.execution.securecodebox.io" deleted
customresourcedefinition.apiextensions.k8s.io "scantypes.execution.securecodebox.io" deleted
customresourcedefinition.apiextensions.k8s.io "scheduledscans.execution.securecodebox.io" deleted
... then if got stuck here and won't finish so I ctrl+C, but notice the scans.execution.securecodebox.io which says deleted.
% kubectl get crds
NAME CREATED AT
alertmanagerconfigs.monitoring.coreos.com 2021-08-04T01:47:22Z
alertmanagers.monitoring.coreos.com 2021-08-04T01:47:25Z
eniconfigs.crd.k8s.amazonaws.com 2021-06-09T02:15:07Z
podmonitors.monitoring.coreos.com 2021-08-04T01:47:27Z
probes.monitoring.coreos.com 2021-08-04T01:47:30Z
prometheuses.monitoring.coreos.com 2021-08-04T01:47:33Z
prometheusrules.monitoring.coreos.com 2021-08-04T01:47:35Z
provisioners.karpenter.sh 2021-12-14T05:33:10Z
scans.execution.securecodebox.io 2022-04-07T06:54:40Z <<<<<<<<<< Won't delete
securitygrouppolicies.vpcresources.k8s.aws 2021-06-09T02:15:11Z
servicemonitors.monitoring.coreos.com 2021-08-04T01:47:37Z
thanosrulers.monitoring.coreos.com 2021-08-04T01:47:39Z
Now, I described the scan object and found it is marked for deletion:
% kubectl describe scan zap-full-scan-admin
Name: zap-full-scan-admin
Namespace: default
Labels: organization=OWASP
Annotations: <none>
API Version: execution.securecodebox.io/v1
Kind: Scan
Metadata:
Creation Timestamp: 2022-04-08T00:11:04Z
Deletion Grace Period Seconds: 0
Deletion Timestamp: 2022-04-08T00:13:07Z <<<<< so it is marked for deletion so should be deleted but it isn't
Finalizers:
s3.storage.securecodebox.io
...
I decided to test and create another scan to compare with the first scan object and got:
% kubectl apply -f zap-full-scan-admin2.yaml
Error from server (MethodNotAllowed): error when creating "zap-full-scan-admin.yaml": create not allowed while custom resource definition is terminating
So I got the scan CRD definition:
% kubectl describe crd scans.execution.securecodebox.io
Name: scans.execution.securecodebox.io
Namespace:
Labels: <none>
Annotations: controller-gen.kubebuilder.io/version: v0.4.1
API Version: apiextensions.k8s.io/v1
Kind: CustomResourceDefinition
Metadata:
Creation Timestamp: 2022-04-07T06:54:40Z
Deletion Grace Period Seconds: 0
Deletion Timestamp: 2022-04-12T00:29:03Z
Finalizers:
customresourcecleanup.apiextensions.k8s.io
... a loong definition
Status:
Accepted Names:
Kind: Scan
List Kind: ScanList
Plural: scans
Singular: scan
Conditions:
Last Transition Time: 2022-04-07T06:54:40Z
Message: no conflicts found
Reason: NoConflicts
Status: True
Type: NamesAccepted
Last Transition Time: 2022-04-07T06:54:40Z
Message: the initial names have been accepted
Reason: InitialNamesAccepted
Status: True
Type: Established
Last Transition Time: 2022-04-12T00:29:03Z
Message: CustomResource deletion is in progress
Reason: InstanceDeletionInProgress
Status: True
Type: Terminating <<<<< So it is deleting... but its stuck for some reason
Stored Versions:
v1
Events: <none>
In conclusion, I got a scan CRD and a scan object that won't delete and are stuck for some unknown reason.
Steps To Reproduce
Described the problem above.
Expected behavior
I expect CRDs and Scans to delete correctly and be removed from kubernetes resources.
System (please complete the following information):
Screenshots / Logs
Additional context
Metadata
Metadata
Assignees
Type
Projects
Status