Skip to main content
  • EN

Currently…

kubectl get storageclass
NAME                              PROVISIONER                RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
csi-cinder-classic                cinder.csi.openstack.org   Delete          Immediate           true                   244d
csi-cinder-high-speed (default)   cinder.csi.openstack.org   Delete          Immediate           true                   244d

kubectl get volumesnapshotclass
NAME                                       DRIVER                     DELETIONPOLICY   AGE
csi-cinder-snapclass                       cinder.csi.openstack.org   Delete           245d
csi-cinder-snapclass-in-use-v1             cinder.csi.openstack.org   Delete           162d
csi-cinder-snapclass-v1                    cinder.csi.openstack.org   Delete           162d
k10-clone-csi-cinder-snapclass-in-use-v1   cinder.csi.openstack.org   Retain           16d

kubectl get volumesnapshotclass csi-cinder-snapclass-v1 -o json
{
    "apiVersion": "snapshot.storage.k8s.io/v1",
    "deletionPolicy": "Delete",
    "driver": "cinder.csi.openstack.org",
    "kind": "VolumeSnapshotClass",
    "metadata": {
        "annotations": {
            "k10.kasten.io/is-snapshot-class": "true"
        },
        "creationTimestamp": "2022-06-07T11:25:02Z",
        "generation": 1,
        "name": "csi-cinder-snapclass-v1",
        "resourceVersion": "4744139956",
        "uid": "c1b3f9a2-940b-4c6a-be6b-fd4c32f32664"
    }

kubectl get volumesnapshotclass csi-cinder-snapclass-in-use-v1 -o json
{
    "apiVersion": "snapshot.storage.k8s.io/v1",
    "deletionPolicy": "Delete",
    "driver": "cinder.csi.openstack.org",
    "kind": "VolumeSnapshotClass",
    "metadata": {
        "annotations": {
            "k10.kasten.io/is-snapshot-class": "true"
        },
        "creationTimestamp": "2022-06-07T11:25:03Z",
        "generation": 1,
        "name": "csi-cinder-snapclass-in-use-v1",
        "resourceVersion": "5052736106",
        "uid": "722f2170-0799-4458-a803-9055737ca308"
    },
    "parameters": {
        "force-create": "true"
    }
}

The above mentioned annotations work fine for me so far. BUT the kasten10 docs state that only one storageclass should have the annotation. But i need them for -v1 AND -v1-in-use class for snapshots to work. Is my setup still correct or do i just need to annotate csi-cinder-highspeed and dont annotate the snapclasses ? *confused*

 

And what is k10-clone-csi-cinder-snapclass-in-use-v1 for? It has been created by k10… do i really need it ?

 

Thx in advance..

Hello @markbon 
I think your configuration is ok , For each CSI driver, ensure that a VolumeSnapshotClass has been added with K10 annotation (k10.kasten.io/is-snapshot-class: "true").


Why k10-clone-csi-cinder is created?

 

Note that CSI snapshots are not durable. In particular, CSI snapshots have a namespaced VolumeSnapshot object and a non-namespaced VolumeSnapshotContent object. With the default (and recommended) deletionPolicy, if there is a deletion of a volume or the namespace containing the volume, the cleanup of the namespaced VolumeSnapshot object will lead to the cascading delete of the VolumeSnapshotContent object and therefore the underlying storage snapshot.

Setting deletionPolicy to Delete isn't sufficient either as some storage systems will force snapshot deletion if the associated volume is deleted (snapshot lifecycle is not independent of the volume). Similarly, it might be possible to force-delete snapshots through the storage array's native management interface. Enabling backups together with volume snapshots is therefore required for a durable backup.

K10 creates a clone of the original VolumeSnapshotClass with the DeletionPolicy set to 'Retain'. When restoring a CSI VolumeSnapshot, an independent replica is created using this cloned class to avoid any accidental deletions of the underlying VolumeSnapshotContent.

@Hagag thx for you detailed reply. I think i get now how the snapshot work inside kubernetes iitself…

I still got the a problem with exporting the complete namespace. After my configuration is correct this should be possible ? 

The Error Message in Kasten10 looks like this:

Failed snapshot for namespace

 Error performing operator snapshot

 ActionSet Failed

 Failed while waiting for Pod kanister-job-s5vl8 to complete: Pod failed or did not transition into complete state: Pod kanister-job-s5vl8 failed. Pod status: &PodStatus{Phase:Failed,Conditions:i]PodCondition{PodCondition{Type:Initialized,Status:True,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2022-11-16 14:15:13 +0000 UTC,Reason:,Message:,},PodCondition{Type:Ready,Status:False,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2022-11-16 14:15:23 +0000 UTC,Reason:PodFailed,Message:,},PodCondition{Type:ContainersReady,Status:False,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2022-11-16 14:15:23 +0000 UTC,Reason:PodFailed,Message:,},PodCondition{Type:PodScheduled,Status:True,LastProbeTime:0001-01-01 00:00:00 +0000 UTC,LastTransitionTime:2022-11-16 14:15:13 +0000 UTC,Reason:,Message:,},},Message:,Reason:,HostIP:51.75.91.189,PodIP:10.2.6.134,StartTime:2022-11-16 14:15:13 +0000 UTC,ContainerStatuses:a]ContainerStatus{ContainerStatus{Name:container,State:ContainerState{Waiting:nil,Running:nil,Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2022-11-16 14:15:22 +0000 UTC,FinishedAt:2022-11-16 14:15:22 +0000 UTC,ContainerID:containerd://4b160bf1a53a66c8e26ee2572ee9d6833bb6cdd0c490f6ed95556867f4d6a85c,},},LastTerminationState:ContainerState{Waiting:nil,Running:nil,Terminated:nil,},Ready:false,RestartCount:0,Image:ghcr.io/kanisterio/kanister-kubectl-1.18:0.84.0,ImageID:ghcr.io/kanisterio/kanister-kubectl-1.18@sha256:1d079e2aec5d1e8064d7560a1a600c511f7b5a55c3f756cdf7b131d3a0349f9c,ContainerID:containerd://4b160bf1a53a66c8e26ee2572ee9d6833bb6cdd0c490f6ed95556867f4d6a85c,Started:*false,},},QOSClass:BestEffort,InitContainerStatuses:a]ContainerStatus{},NominatedNodeName:,PodIPs:P]PodIP{PodIP{IP:10.2.6.134,},},EphemeralContainerStatuses:a]ContainerStatus{},}

 

@markbon 

It seems you are using kanister blueprints to backup the application,
the pod "kanister-job-s5vl8"  that created during the backup in your app namespace is failed to be in Running status

might be a problem with pulling the image "Image:ghcr.io/kanisterio/kanister-kubectl-1.18:0.84.0”

Please try to track the pods initiated in your app namespace and do kubectl describe to get more insight about the error 

kubectl get pods -n <NAMESPACE> -w

you can create a while loop to be able to catch the pod before its termination.  

while true; do kubectl describe pod -l createdBy=kanister -n <NAMESPACE> ; sleep 2 ; done

kubectl describe pod -l createdBy=kanister -n <NAMESPACE>

Comment


Terms & Conditions