Skip to main content
Solved

Prevent kanister-pvc buildup when using reclaimPolicy: Retain

  • February 14, 2022
  • 1 comment
  • 332 views

My default storage class has was setup with reclaimPolicy: Retain.

 

Everytime Kasten exports backups, it will create copy-vol* pods which request kanister-pvc’s with the same storage class as what they are copying. This means that everytime a 20G volume is backed up, a new 20G PV is requested and used up. This quickly leads to running out of storage.

I’ve already set up kasten 4.5.9 with a storage class that has “reclaimPolicy: Delete”, but since it the copy-vol containers seem to mirror the SC of existing PVs that doesn’t help.

 

Is there anyway to prevent the PV buildup as a result of backups? For now I’m manually patching all released PVs with “reclaimPolicy: Delete”.

Best answer by jaiganeshjk

Thank you for posting this question,

TL;DR

You can patch the export enabled policy using the kubectl command below by replacing the <policyName> and the <storageClassName> with the name of the storageclass with delete policy.

kubectl patch policies.config.kio.kasten.io <policyName> -p '[{"op": "add","path": "/spec/actions/1/exportParameters/exportData/exporterStorageClassName", "value": "<storageClassName>"}]' --type json

Long Answer

There is an override setting that you can use in your policy to specify a storageClass that is to be used for the exports. We don’t have an option to specify this in the UI policy form yet.

Below are the example that shows how to add these parameters by editing the Policy manually.

This will override the storageClass used for the export operation.

apiVersion: config.kio.kasten.io/v1alpha1
kind: Policy
metadata:
  name: postgresql-backup-1
  namespace: kasten-io
spec:
  actions:
  - action: backup
  - action: export
    exportParameters:
      exportData:
        enabled: true
        # Storage class to use for any temporary PVCs created
        # during the snapshot conversion process. If not specified, the
        # storage class of the source volume is used.
        exporterStorageClassName: gp2-delete
      frequency: '@hourly'
      migrationToken:
        name: postgresql-backup-1-migration-token-7vrgc
        namespace: kasten-io
      profile:
        name: test-jai
        namespace: kasten-io
      receiveString: xxxxxxx
    retention: {}
  createdBy: kasten-io:k10-k10
  frequency: '@hourly'
  retention:
    daily: 7
    hourly: 24
    monthly: 12
    weekly: 4
    yearly: 7
  selector:
    matchExpressions:
    - key: k10.kasten.io/appNamespace
      operator: In
      values:
      - postgresql
      - kasten-io-cluster

Above example holds good only when your namespace has PVCs using a single storageClass.

You can use the below example if you have multiple PVCs using different storageClasses in same namespace.

apiVersion: config.kio.kasten.io/v1alpha1
kind: Policy
metadata:
  name: postgresql-backup-1
  namespace: kasten-io
spec:
  actions:
  - action: backup
  - action: export
    exportParameters:
      exportData:
        enabled: true
        overrides:
            # Override setting of a specific storage class
          - storageClassName: gp2-retain
            enabled: true
            exporterStorageClassName: gp2-delete
          - storageClassName: gp3-retain
            enabled: true
            exporterStorageClassName: gp3-delete
            # Setting to disable the export for a specific storageClass
          - storageClassName: nfs-client
            enabled: false
      frequency: '@hourly'
      migrationToken:
        name: postgresql-backup-1-migration-token-7vrgc
        namespace: kasten-io
      profile:
        name: test-jai
        namespace: kasten-io
      receiveString: xxxxxxx
    retention: {}
  createdBy: kasten-io:k10-k10
  frequency: '@hourly'
  retention:
    daily: 7
    hourly: 24
    monthly: 12
    weekly: 4
    yearly: 7
  selector:
    matchExpressions:
    - key: k10.kasten.io/appNamespace
      operator: In
      values:
      - postgresql
      - kasten-io-cluster

Please look at the documentation to see all the fields that can be configured with the policy API

View original

1 comment

jaiganeshjk
Forum|alt.badge.img+2
  • Experienced User
  • 274 comments
  • Answer
  • February 14, 2022

Thank you for posting this question,

TL;DR

You can patch the export enabled policy using the kubectl command below by replacing the <policyName> and the <storageClassName> with the name of the storageclass with delete policy.

kubectl patch policies.config.kio.kasten.io <policyName> -p '[{"op": "add","path": "/spec/actions/1/exportParameters/exportData/exporterStorageClassName", "value": "<storageClassName>"}]' --type json

Long Answer

There is an override setting that you can use in your policy to specify a storageClass that is to be used for the exports. We don’t have an option to specify this in the UI policy form yet.

Below are the example that shows how to add these parameters by editing the Policy manually.

This will override the storageClass used for the export operation.

apiVersion: config.kio.kasten.io/v1alpha1
kind: Policy
metadata:
  name: postgresql-backup-1
  namespace: kasten-io
spec:
  actions:
  - action: backup
  - action: export
    exportParameters:
      exportData:
        enabled: true
        # Storage class to use for any temporary PVCs created
        # during the snapshot conversion process. If not specified, the
        # storage class of the source volume is used.
        exporterStorageClassName: gp2-delete
      frequency: '@hourly'
      migrationToken:
        name: postgresql-backup-1-migration-token-7vrgc
        namespace: kasten-io
      profile:
        name: test-jai
        namespace: kasten-io
      receiveString: xxxxxxx
    retention: {}
  createdBy: kasten-io:k10-k10
  frequency: '@hourly'
  retention:
    daily: 7
    hourly: 24
    monthly: 12
    weekly: 4
    yearly: 7
  selector:
    matchExpressions:
    - key: k10.kasten.io/appNamespace
      operator: In
      values:
      - postgresql
      - kasten-io-cluster

Above example holds good only when your namespace has PVCs using a single storageClass.

You can use the below example if you have multiple PVCs using different storageClasses in same namespace.

apiVersion: config.kio.kasten.io/v1alpha1
kind: Policy
metadata:
  name: postgresql-backup-1
  namespace: kasten-io
spec:
  actions:
  - action: backup
  - action: export
    exportParameters:
      exportData:
        enabled: true
        overrides:
            # Override setting of a specific storage class
          - storageClassName: gp2-retain
            enabled: true
            exporterStorageClassName: gp2-delete
          - storageClassName: gp3-retain
            enabled: true
            exporterStorageClassName: gp3-delete
            # Setting to disable the export for a specific storageClass
          - storageClassName: nfs-client
            enabled: false
      frequency: '@hourly'
      migrationToken:
        name: postgresql-backup-1-migration-token-7vrgc
        namespace: kasten-io
      profile:
        name: test-jai
        namespace: kasten-io
      receiveString: xxxxxxx
    retention: {}
  createdBy: kasten-io:k10-k10
  frequency: '@hourly'
  retention:
    daily: 7
    hourly: 24
    monthly: 12
    weekly: 4
    yearly: 7
  selector:
    matchExpressions:
    - key: k10.kasten.io/appNamespace
      operator: In
      values:
      - postgresql
      - kasten-io-cluster

Please look at the documentation to see all the fields that can be configured with the policy API


Comment