Skip to main content
  • EN
Solved

Backup fails on snapshot phase for csi csi-isilon.dellemc.com

Hi,

we have ocp cluster where csi csi-isilon.dellemc.com is installed, when running the backup, it failed on the snapshot phase, the preflight check for the storage class shows the below:

 

proot@wat-kasten]# ./k10tools primer storage check csi -f ./csi_check.yaml
Using "./csi_check.yaml" file content as config source
I0725 11:44:26.276281  541000 request.go:697] Waited for 1.047160547s due to client-side throttling, not priority and fairness, request: GET:https://api.ocpprod01.otg.om:6443/apis/operator.openshift.io/v1alpha1
Failed to discover pod configuration for Pod (wat-ocpprod01-bas1p-v.otg.om): (pods "wat-ocpprod01-bas1p-v.otg.om" not found)
I0725 11:44:36.326277  541000 request.go:697] Waited for 6.144135655s due to client-side throttling, not priority and fairness, request: GET:https://api.ocpprod01.otg.om:6443/apis/appconnect.ibm.com/v1beta1
Creating application
W0725 11:44:37.134185  541000 warnings.go:70] would violate PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "kubestr-csi-original-pod" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "kubestr-csi-original-pod" must set securityContext.capabilities.drop=d"ALL"]), runAsNonRoot != true (pod or container "kubestr-csi-original-pod" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "kubestr-csi-original-pod" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
  -> Created pod (kubestr-csi-original-pod8rkhk) and pvc (kubestr-csi-original-pvcslzj6)
Taking a snapshot
Cleaning up resources
CSI Snapshot Walkthrough:
  Using annotated VolumeSnapshotClass (nfs-app-pscl-sc01)
  Failed to create Snapshot: CSI Driver failed to create snapshot for PVC (kubestr-csi-original-pvcslzj6) in Namespace (default): Failed to check and update snapshot content: failed to take snapshot of the volume ocpprod01-vols-96460bac09=_=_=2016=_=_=Production=_=_=WAT-PS01: "rpc error: code = InvalidArgument desc =  runid=16965899 rpc error: code = InvalidArgument desc =  runid=16965899 source volume id is invalid"  -  Error
Error: {"message":"Failed to create Snapshot: CSI Driver failed to create snapshot for PVC (kubestr-csi-original-pvcslzj6) in Namespace (default): Failed to check and update snapshot content: failed to take snapshot of the volume ocpprod01-vols-96460bac09=_=_=2016=_=_=Production=_=_=WAT-PS01: \"rpc error: code = InvalidArgument desc =  runid=16965899 rpc error: code = InvalidArgument desc =  runid=16965899 source volume id is invalid\"","function":"kasten.io/k10/kio/tools/k10primer.(*TestRetVal).Errors","linenumber":168,"file":"kasten.io/k10/kio/tools/k10primer/k10primer.go:168"}
 

 

the storage class is described in the below:

hroot@wat-kasten]# k get sc nfs-app-pscl-sc01 -o yaml
allowVolumeExpansion: true
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  annotations:
    k10.kasten.io/volume-snapshot-class: nfs-app-pscl-sc01
    kubectl.kubernetes.io/last-applied-configuration: |
      {"allowVolumeExpansion":true,"apiVersion":"storage.k8s.io/v1","kind":"StorageClass","metadata":{"annotations":{},"name":"nfs-app-pscl-sc01"},"parameters":{"AccessZone":"Production","AzServiceIP":"prod-wat-ps01.otg.om","IsiPath":"/ifs/data/wat-ps01","RootClientEnabled":"false"},"provisioner":"csi-isilon.dellemc.com","reclaimPolicy":"Delete","volumeBindingMode":"Immediate"}
  creationTimestamp: "2023-12-27T13:00:11Z"
  name: nfs-app-pscl-sc01
  resourceVersion: "818562779"
  uid: 098635a7-1ba3-4b20-8892-37aafc1964f1
parameters:
  AccessZone: Production
  AzServiceIP: prod-wat-ps01.otg.om
  IsiPath: /ifs/data/wat-ps01
  RootClientEnabled: "false"
provisioner: csi-isilon.dellemc.com
reclaimPolicy: Delete
volumeBindingMode: Immediate
 

 and the volume snapshotclass is :

eroot@wat-kasten]# k get volumesnapshotclass nfs-app-pscl-sc01 -o yaml
apiVersion: snapshot.storage.k8s.io/v1
deletionPolicy: Delete
driver: csi-isilon.dellemc.com
kind: VolumeSnapshotClass
metadata:
  annotations:
    k10.kasten.io/is-snapshot-class: "true"
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"snapshot.storage.k8s.io/v1","deletionPolicy":"Delete","driver":"csi-isilon.dellemc.com","kind":"VolumeSnapshotClass","metadata":{"annotations":{},"name":"nfs-app-pscl-sc01"}}
  creationTimestamp: "2024-05-15T10:04:24Z"
  generation: 1
  name: nfs-app-pscl-sc01
  resourceVersion: "805829914"
  uid: d96d2d68-7308-43ca-8351-b7ff410d4c2a
 

can anyone help here?

2 comments

S
Userlevel 3
Badge +1

@Ali Kanso 

Have you tried to take a Native Kubernetes Volumesnapshot of any volume provisioned from isilon StorageClass using Volumesnapshotclass and check if the volumesnapshot is being created and ready to use ?

https://kubernetes.io/docs/concepts/storage/volume-snapshots/#volumesnapshots

 

apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
  name: new-snapshot-test
spec:
  volumeSnapshotClassName: nfs-app-pscl-sc01
  source:
    persistentVolumeClaimName: <pvc-test>

Regards
Satish

A
Userlevel 4

Hi,

we resolved the issue by adding the IsiPath in volumesnapshotclass yaml file as below:

# For kubernetes version 20 (v1 snaps)
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshotClass
metadata:
  name: isilon-snapclass
driver: csi-isilon.dellemc.com

# Configure what happens to a VolumeSnapshotContent when the VolumeSnapshot object
# it is bound to is to be deleted
# Allowed values:
#   Delete: the underlying storage snapshot will be deleted along with the VolumeSnapshotContent object.
#   Retain: both the underlying snapshot and VolumeSnapshotContent remain.
deletionPolicy: Delete

parameters:
  # The base path of the volumes on Isilon cluster for which snapshot is being created.
  # This path should be same as the IsiPath from the storageClass.
  # Optional: false
  IsiPath: /ifs/data/csi

 

for reference: https://github.com/dell/csi-powerscale/blob/v2.7.0/samples/volumesnapshotclass/isilon-volumesnapshotclass-v1.yaml

 

Comment


Terms & Conditions