Skip to main content
Solved

Stale Pods and Secrets

A

Hello Team,

 

We notice a lot of stale pods (created on a random date in previous months of Jan, Feb and March) with the name ‘backup-data-stats-*’ and stale secrets (again created in previous months)  ‘k10-content-store-passphrase-*’ in kasten namespace. Is this a normal behaviour? Do we have to do some regular clean up for kasten namespaces? 

Thanks.

Arun Prasad

Kasten version

4.5.3
Kubernetes Version v1.21.6+c180a7c
Kubernetes Release Type OpenShift

Best answer by Hagag

Hello @Arun Prasad 

k10 creates ‘backup-data-stats-*’ Pods get the physical size and logical size of the repository.

it is created everytime kopia snapshot is performed and ‘k10-content-store-passphrase’  contains the password of ‘backup-data-stats’ in order to connect to kopia repo.

and it is terminating after the kopia snapshot completed.

 

Please check the status of your runaction either from the k10 dashboard or CLI

kubectl get runactions.actions.kio.kasten.io  -o custom-columns=NAME:.metadata.name,"CREATED AT":.metadata.creationTimestamp,STATUS:.status.state

 

you might also need to kubectl describe backup-data-stats-* 

 

Thanks

Ahmed Hagag

View original
Did this topic help you find an answer to your question?

Geoff Burke
Forum|alt.badge.img+22
  • Geoff Burke
  • Veeam Legend, Veeam Vanguard
  • April 25, 2022

I am interested in this as well as I saw the same thing.

 

VMCA2024, VMCE2023, VMCE2024-SP,CKA, LFCS, PCA,TARS
Rick Vanover
1 person likes this
  • Rick Vanover
    Rick Vanover
Hagag
Forum|alt.badge.img+2
  • Hagag
  • Experienced User
  • April 26, 2022

Hello @Arun Prasad 

k10 creates ‘backup-data-stats-*’ Pods get the physical size and logical size of the repository.

it is created everytime kopia snapshot is performed and ‘k10-content-store-passphrase’  contains the password of ‘backup-data-stats’ in order to connect to kopia repo.

and it is terminating after the kopia snapshot completed.

 

Please check the status of your runaction either from the k10 dashboard or CLI

kubectl get runactions.actions.kio.kasten.io  -o custom-columns=NAME:.metadata.name,"CREATED AT":.metadata.creationTimestamp,STATUS:.status.state

 

you might also need to kubectl describe backup-data-stats-* 

 

Thanks

Ahmed Hagag

Geoff Burke
1 person likes this
  • Geoff Burke
    Geoff Burke
A

Hi @Hagag,

Thank you for responding. I will have a look into the runs. Just for my clarification, are you trying to say that, whatever activity kopia is trying to do and if that fails, it could result in dangling pods and secrets? 

 

Regards,

Arun Prasad

Hagag
Forum|alt.badge.img+2
  • Hagag
  • Experienced User
  • April 27, 2022

Hello @Arun Prasad it might be one of the reasons , that is why we need to check :) 

 

waiting for your feedback

Regards,

Ahmed Hagag

A

Hi @Hagag,

We updated the cluster yesterday and I don’t see the pods anymore but the secrets are still present. I ran the command that you provided to extract the run information and attached the output.  Since the pods are gone, I couldnt run the describe backup-data-stats command.

Please let me know if you need any other information or logs.  Thanks your help!

Regards,

Arun Prasad

1 Attachments
A

Seeing this as well. 50+ dangling `backup-data-stats-*` pods, many over 9d old.

I deleted the pods but did so once before and they started accumulating again.

Is it safe to delete the ‘k10-content-store-passphrase-*’ secrets as well?

I

is it safe to delete the pods ? i got like 75 right now :( 

Comment


Terms & Conditions