Hi @yashg ,

Thank You for reaching out to us. 

Based on the error message , the “Copy-Vol-Data-x” Pod is not not transitioning to Running . Its waiting for the PVC “kio-8e899f23e87911ec9d687ef14d2bf471-7 “ to be attached where as its Pending . 

This could cause only if the underlying storage is unable to provision the PVC. 

1. Can you describe the PVC “kio-8e899f23e87911ec9d687ef14d2bf471-7 “ and check if you can see any events for it
2. describe copy-vol-data pod 
3. Check events under kasten-io namespace and application namespace

Regards
Satish

Hi @Satish ,

I noticed the PVC is showing bounded sometimes then suddenly it goes to Pending state and its getting re-created again and again , Pods are also getting re-launched again and again (maybe re-trying export process).

Attaching describe command outputs for PVC and the pod.

For FYI, the cluster has more than sufficient compute resources and currently running on 4 nodes.

Thanks

Name:          kio-6dd09648e93911ec9d687ef14d2bf471-17
Namespace:     kasten-io
StorageClass:  gp2
Status:        Bound
Volume:        kio-6dd09648e93911ec9d687ef14d2bf471-17
Labels:        <none>
Annotations:   kasten.io/jobid: 6dd1a696-e939-11ec-845c-12c68f26d14c
               kasten.io/manifestid: 6dd09648-e939-11ec-9d68-7ef14d2bf471
               pv.kubernetes.io/bind-completed: yes
Finalizers:    tkubernetes.io/pvc-protection]
Capacity:      10737418240
Access Modes:  RWO
VolumeMode:    Filesystem
Used By:       copy-vol-data-5gs94
Events:        <none>
 

yash@yash-VirtualBox:~$ kubectl -n kasten-io describe pod copy-vol-data-xhhq2
Name:         copy-vol-data-xhhq2
Namespace:    kasten-io
Priority:     0
Node:         aks-agentpool-33550208-vmss000005/10.224.0.6
Start Time:   Sat, 11 Jun 2022 09:19:15 +0530
Labels:       createdBy=kanister
Annotations:  <none>
Status:       Pending
IP:           
IPs:          <none>
Containers:
  container:
    Container ID:  
    Image:         ghcr.io/kanisterio/kanister-tools:0.79.0
    Image ID:      
    Port:          <none>
    Host Port:     <none>
    Command:
      bash
      -c
      tail -f /dev/null
    State:          Waiting
      Reason:       ContainerCreating
    Ready:          False
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /mnt/vol_data/kio-6dd09648e93911ec9d687ef14d2bf471-17 from vol-kio-6dd09648e93911ec9d687ef14d2bf471-17 (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-c2xkn (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  vol-kio-6dd09648e93911ec9d687ef14d2bf471-17:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  kio-6dd09648e93911ec9d687ef14d2bf471-17
    ReadOnly:   false
  kube-api-access-c2xkn:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/memory-pressure:NoSchedule op=Exists
                             node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason                  Age                 From                     Message
  ----     ------                  ----                ----                     -------
  Warning  FailedScheduling        13m                 default-scheduler        0/4 nodes are available: 4 pod has unbound immediate PersistentVolumeClaims.
  Warning  FailedScheduling        13m (x1 over 13m)   default-scheduler        0/4 nodes are available: 4 pod has unbound immediate PersistentVolumeClaims.
  Normal   Scheduled               13m                 default-scheduler        Successfully assigned kasten-io/copy-vol-data-xhhq2 to aks-agentpool-33550208-vmss000005
  Normal   NotTriggerScaleUp       13m                 cluster-autoscaler       pod didn't trigger scale-up: 1 pod has unbound immediate PersistentVolumeClaims
  Normal   SuccessfulAttachVolume  12m                 attachdetach-controller  AttachVolume.Attach succeeded for volume "kio-6dd09648e93911ec9d687ef14d2bf471-17"
  Warning  FailedMount             4m25s               kubelet                  Unable to attach or mount volumes: unmounted volumes=�vol-kio-6dd09648e93911ec9d687ef14d2bf471-17], unattached volumes=-kube-api-access-c2xkn vol-kio-6dd09648e93911ec9d687ef14d2bf471-17]: timed out waiting for the condition
  Warning  FailedMount             2m7s (x4 over 11m)  kubelet                  Unable to attach or mount volumes: unmounted volumes= vol-kio-6dd09648e93911ec9d687ef14d2bf471-17], unattached volumes=�vol-kio-6dd09648e93911ec9d687ef14d2bf471-17 kube-api-access-c2xkn]: timed out waiting for the condition
  Warning  FailedMount             22s (x14 over 12m)  kubelet                  MountVolume.MountDevice failed for volume "kio-6dd09648e93911ec9d687ef14d2bf471-17" : rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun6(lun: 6), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/kio-6dd09648e93911ec9d687ef14d2bf471-17/globalmount
 

@yashg ,

Based on the events below, i see its failing the mount of volume to the POD. Can you look into the Azure CSi Driver pod logs in details whats causing volume not to get mounted?

  Warning  FailedMount             4m25s               kubelet                  Unable to attach or mount volumes: unmounted volumes=svol-kio-6dd09648e93911ec9d687ef14d2bf471-17], unattached volumes=skube-api-access-c2xkn vol-kio-6dd09648e93911ec9d687ef14d2bf471-17]: timed out waiting for the condition
  Warning  FailedMount             2m7s (x4 over 11m)  kubelet                  Unable to attach or mount volumes: unmounted volumes=svol-kio-6dd09648e93911ec9d687ef14d2bf471-17], unattached volumes=svol-kio-6dd09648e93911ec9d687ef14d2bf471-17 kube-api-access-c2xkn]: timed out waiting for the condition
  Warning  FailedMount             22s (x14 over 12m)  kubelet                  MountVolume.MountDevice failed for volume "kio-6dd09648e93911ec9d687ef14d2bf471-17" : rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun6(lun: 6), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/kio-6dd09648e93911ec9d687ef14d2bf471-17/globalmount

@Satish 

It is showing this error, not sure why.

 

GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun2(lun: 2), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/kio-06feb876eba011ec9d687ef14d2bf471-5/globalmount

0614 05:33:54.852402 1 utils.go:77] GRPC call: /csi.v1.Node/NodeUnpublishVolume

I0614 05:33:54.852434 1 utils.go:78] GRPC request: {"target_path":"/var/lib/kubelet/pods/e707d076-6f55-42d5-a0e9-6b809a532c3b/volumes/kubernetes.io~csi/kio-06feb876eba011ec9d687ef14d2bf471-13/mount","volume_id":"/subscriptions/4e745d58-627d-48cb-b1e6-9e3118cebad6/resourceGroups/mc_swa-backup-aks-rg_swa-backup-aks_westus2/providers/Microsoft.Compute/disks/vol-08f5b9fb-eba3-11ec-a9ec-6633efe5cf0c"}

I0614 05:33:54.852551 1 nodeserver.go:303] NodeUnpublishVolume: unmounting volume /subscriptions/4e745d58-627d-48cb-b1e6-9e3118cebad6/resourceGroups/mc_swa-backup-aks-rg_swa-backup-aks_westus2/providers/Microsoft.Compute/disks/vol-08f5b9fb-eba3-11ec-a9ec-6633efe5cf0c on /var/lib/kubelet/pods/e707d076-6f55-42d5-a0e9-6b809a532c3b/volumes/kubernetes.io~csi/kio-06feb876eba011ec9d687ef14d2bf471-13/mount

I0614 05:33:54.852607 1 mount_helper_common.go:99] "/var/lib/kubelet/pods/e707d076-6f55-42d5-a0e9-6b809a532c3b/volumes/kubernetes.io~csi/kio-06feb876eba011ec9d687ef14d2bf471-13/mount" is a mountpoint, unmounting

I0614 05:33:54.852627 1 mount_linux.go:294] Unmounting /var/lib/kubelet/pods/e707d076-6f55-42d5-a0e9-6b809a532c3b/volumes/kubernetes.io~csi/kio-06feb876eba011ec9d687ef14d2bf471-13/mount

W0614 05:33:54.854907 1 mount_helper_common.go:133] Warning: "/var/lib/kubelet/pods/e707d076-6f55-42d5-a0e9-6b809a532c3b/volumes/kubernetes.io~csi/kio-06feb876eba011ec9d687ef14d2bf471-13/mount" is not a mountpoint, deleting

I0614 05:33:54.854976 1 nodeserver.go:309] NodeUnpublishVolume: unmount volume /subscriptions/4e745d58-627d-48cb-b1e6-9e3118cebad6/resourceGroups/mc_swa-backup-aks-rg_swa-backup-aks_westus2/providers/Microsoft.Compute/disks/vol-08f5b9fb-eba3-11ec-a9ec-6633efe5cf0c on /var/lib/kubelet/pods/e707d076-6f55-42d5-a0e9-6b809a532c3b/volumes/kubernetes.io~csi/kio-06feb876eba011ec9d687ef14d2bf471-13/mount successfully

I0614 05:33:54.854993 1 utils.go:84] GRPC response: {}

I0614 05:33:54.958001 1 utils.go:77] GRPC call: /csi.v1.Node/NodeUnstageVolume

I0614 05:33:54.958029 1 utils.go:78] GRPC request: {"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/kio-06feb876eba011ec9d687ef14d2bf471-13/globalmount","volume_id":"/subscriptions/4e745d58-627d-48cb-b1e6-9e3118cebad6/resourceGroups/mc_swa-backup-aks-rg_swa-backup-aks_westus2/providers/Microsoft.Compute/disks/vol-08f5b9fb-eba3-11ec-a9ec-6633efe5cf0c"}

I0614 05:33:54.958085 1 nodeserver.go:202] NodeUnstageVolume: unmounting /var/lib/kubelet/plugins/kubernetes.io/csi/pv/kio-06feb876eba011ec9d687ef14d2bf471-13/globalmount

I0614 05:33:54.958117 1 mount_helper_common.go:99] "/var/lib/kubelet/plugins/kubernetes.io/csi/pv/kio-06feb876eba011ec9d687ef14d2bf471-13/globalmount" is a mountpoint, unmounting

I0614 05:33:54.958131 1 mount_linux.go:294] Unmounting /var/lib/kubelet/plugins/kubernetes.io/csi/pv/kio-06feb876eba011ec9d687ef14d2bf471-13/globalmount

W0614 05:33:56.894225 1 mount_helper_common.go:133] Warning: "/var/lib/kubelet/plugins/kubernetes.io/csi/pv/kio-06feb876eba011ec9d687ef14d2bf471-13/globalmount" is not a mountpoint, deleting

I0614 05:33:56.894337 1 nodeserver.go:207] NodeUnstageVolume: unmount /var/lib/kubelet/plugins/kubernetes.io/csi/pv/kio-06feb876eba011ec9d687ef14d2bf471-13/globalmount successfully

I0614 05:33:56.894361 1 utils.go:84] GRPC response: {}

@yashg , Can you get me the yaml outputs for below

1. PVC you are protecting & the one pending created by k10
2. csidrivers (kubectl get csidrivers -oyaml)
3. storageclass (kubectl get sc -oyaml)

Addition to it we have a Primer Script which checks if the CSI Driver is configured correctly to backup. Below are quick steps which you can run on the “Storage Class”

curl -s https://docs.kasten.io/tools/k10_primer.sh > primer

bash primer -c “storage csi-checker -s <storageclass> --runAsUser=1000”

Regards
Satish

@Satish 

Please find the attached outputs and also I ran the primer script for gp2 storage class which provisioned some PVC‘s and it was failing with below error: 

 

Starting CSI Checker. Could take up to 5 minutes

W0615 07:54:42.905776 7 warnings.go:70] storage.k8s.io/v1beta1 CSIDriver is deprecated in v1.19+, unavailable in v1.22+; use storage.k8s.io/v1 CSIDriver

CSI Snapshot Walkthrough:

Not a supported CSI driver (kubernetes.io/azure-disk) - Error

1.  PVC describe yaml output

Name:          kio-6dd09648e93911ec9d687ef14d2bf471-17
Namespace:     kasten-io
StorageClass:  gp2
Status:        Bound
Volume:        kio-6dd09648e93911ec9d687ef14d2bf471-17
Labels:        <none>
Annotations:   kasten.io/jobid: 6dd1a696-e939-11ec-845c-12c68f26d14c
               kasten.io/manifestid: 6dd09648-e939-11ec-9d68-7ef14d2bf471
               pv.kubernetes.io/bind-completed: yes
Finalizers:    ekubernetes.io/pvc-protection]
Capacity:      10737418240
Access Modes:  RWO
VolumeMode:    Filesystem
Used By:       copy-vol-data-5gs94
Events:        <none>

2. kubectl get csidrivers -oyaml

apiVersion: v1
items:
- apiVersion: storage.k8s.io/v1
  kind: CSIDriver
  metadata:
    annotations:
      csiDriver: v1.18.0
      kubectl.kubernetes.io/last-applied-configuration: |
        {"apiVersion":"storage.k8s.io/v1","kind":"CSIDriver","metadata":{"annotations":{"csiDriver":"v1.18.0","snapshot":"v4.2.1"},"labels":{"addonmanager.kubernetes.io/mode":"Reconcile","kubernetes.io/cluster-service":"true"},"name":"disk.csi.azure.com"},"spec":{"attachRequired":true,"fsGroupPolicy":"File","podInfoOnMount":false}}
      snapshot: v4.2.1
    creationTimestamp: "2022-06-09T13:21:27Z"
    labels:
      addonmanager.kubernetes.io/mode: Reconcile
      kubernetes.io/cluster-service: "true"
    name: disk.csi.azure.com
    resourceVersion: "500"
    uid: 1f81f0f3-d9c2-4f5d-940a-50589cb5785e
  spec:
    attachRequired: true
    fsGroupPolicy: File
    podInfoOnMount: false
    requiresRepublish: false
    storageCapacity: false
    volumeLifecycleModes:
    - Persistent
- apiVersion: storage.k8s.io/v1
  kind: CSIDriver
  metadata:
    annotations:
      csiDriver: v1.18.0
      kubectl.kubernetes.io/last-applied-configuration: |
        {"apiVersion":"storage.k8s.io/v1","kind":"CSIDriver","metadata":{"annotations":{"csiDriver":"v1.18.0","snapshot":"v4.2.1"},"labels":{"addonmanager.kubernetes.io/mode":"Reconcile","kubernetes.io/cluster-service":"true"},"name":"file.csi.azure.com"},"spec":{"attachRequired":false,"fsGroupPolicy":"ReadWriteOnceWithFSType","podInfoOnMount":true,"volumeLifecycleModes":e"Persistent","Ephemeral"]}}
      snapshot: v4.2.1
    creationTimestamp: "2022-06-09T13:21:27Z"
    labels:
      addonmanager.kubernetes.io/mode: Reconcile
      kubernetes.io/cluster-service: "true"
    name: file.csi.azure.com
    resourceVersion: "504"
    uid: 25e6a0b7-c7e3-4e39-a7c5-ef738b96630d
  spec:
    attachRequired: false
    fsGroupPolicy: ReadWriteOnceWithFSType
    podInfoOnMount: true
    requiresRepublish: false
    storageCapacity: false
    volumeLifecycleModes:
    - Persistent
    - Ephemeral
- apiVersion: storage.k8s.io/v1
  kind: CSIDriver
  metadata:
    creationTimestamp: "2022-06-09T13:41:33Z"
    name: rook-ceph.cephfs.csi.ceph.com
    resourceVersion: "6027"
    uid: 798a56e7-1e58-414f-9618-621dc643c599
  spec:
    attachRequired: true
    fsGroupPolicy: ReadWriteOnceWithFSType
    podInfoOnMount: false
    requiresRepublish: false
    storageCapacity: false
    volumeLifecycleModes:
    - Persistent
- apiVersion: storage.k8s.io/v1
  kind: CSIDriver
  metadata:
    creationTimestamp: "2022-06-09T13:41:33Z"
    name: rook-ceph.rbd.csi.ceph.com
    resourceVersion: "6026"
    uid: 5c54129b-699c-4c78-a497-83f5065782f6
  spec:
    attachRequired: true
    fsGroupPolicy: ReadWriteOnceWithFSType
    podInfoOnMount: false
    requiresRepublish: false
    storageCapacity: false
    volumeLifecycleModes:
    - Persistent
kind: List
metadata:
  resourceVersion: ""

3. kubectl get sc -o yaml

apiVersion: v1
items:
- allowVolumeExpansion: true
  apiVersion: storage.k8s.io/v1
  kind: StorageClass
  metadata:
    creationTimestamp: "2022-06-09T13:21:26Z"
    labels:
      addonmanager.kubernetes.io/mode: EnsureExists
      kubernetes.io/cluster-service: "true"
    name: azurefile
    resourceVersion: "411"
    uid: 0b2a57ad-601e-40a2-824e-ece61a54eba7
  mountOptions:
  - mfsymlinks
  - actimeo=30
  parameters:
    skuName: Standard_LRS
  provisioner: file.csi.azure.com
  reclaimPolicy: Delete
  volumeBindingMode: Immediate
- allowVolumeExpansion: true
  apiVersion: storage.k8s.io/v1
  kind: StorageClass
  metadata:
    creationTimestamp: "2022-06-09T13:21:26Z"
    labels:
      addonmanager.kubernetes.io/mode: EnsureExists
      kubernetes.io/cluster-service: "true"
    name: azurefile-csi
    resourceVersion: "416"
    uid: 806549c3-b571-4072-b5a8-eee81d464bb5
  mountOptions:
  - mfsymlinks
  - actimeo=30
  parameters:
    skuName: Standard_LRS
  provisioner: file.csi.azure.com
  reclaimPolicy: Delete
  volumeBindingMode: Immediate
- allowVolumeExpansion: true
  apiVersion: storage.k8s.io/v1
  kind: StorageClass
  metadata:
    creationTimestamp: "2022-06-09T13:21:26Z"
    labels:
      addonmanager.kubernetes.io/mode: EnsureExists
      kubernetes.io/cluster-service: "true"
    name: azurefile-csi-premium
    resourceVersion: "417"
    uid: ac1d3014-a086-4d5b-91a2-8b964e73d142
  mountOptions:
  - mfsymlinks
  - actimeo=30
  parameters:
    skuName: Premium_LRS
  provisioner: file.csi.azure.com
  reclaimPolicy: Delete
  volumeBindingMode: Immediate
- allowVolumeExpansion: true
  apiVersion: storage.k8s.io/v1
  kind: StorageClass
  metadata:
    creationTimestamp: "2022-06-09T13:21:26Z"
    labels:
      addonmanager.kubernetes.io/mode: EnsureExists
      kubernetes.io/cluster-service: "true"
    name: azurefile-premium
    resourceVersion: "412"
    uid: 4a5bf367-6426-442f-bbef-e9e162e3d45c
  mountOptions:
  - mfsymlinks
  - actimeo=30
  parameters:
    skuName: Premium_LRS
  provisioner: file.csi.azure.com
  reclaimPolicy: Delete
  volumeBindingMode: Immediate
- allowVolumeExpansion: true
  apiVersion: storage.k8s.io/v1
  kind: StorageClass
  metadata:
    annotations:
      storageclass.kubernetes.io/is-default-class: "true"
    creationTimestamp: "2022-06-09T13:21:26Z"
    labels:
      addonmanager.kubernetes.io/mode: EnsureExists
      kubernetes.io/cluster-service: "true"
    name: default
    resourceVersion: "413"
    uid: d11c78ef-59d5-443f-9d85-06eb098be534
  parameters:
    skuname: StandardSSD_LRS
  provisioner: disk.csi.azure.com
  reclaimPolicy: Delete
  volumeBindingMode: WaitForFirstConsumer
- apiVersion: storage.k8s.io/v1
  kind: StorageClass
  metadata:
    annotations:
      kubectl.kubernetes.io/last-applied-configuration: |
        {"apiVersion":"storage.k8s.io/v1","kind":"StorageClass","metadata":{"annotations":{},"name":"gp2"},"parameters":{"kind":"Managed","storageaccounttype":"Premium_LRS"},"provisioner":"kubernetes.io/azure-disk","reclaimPolicy":"Delete"}
    creationTimestamp: "2022-06-09T13:23:49Z"
    name: gp2
    resourceVersion: "1188"
    uid: 27c6b8fa-51db-4a8e-8dcc-be838eddbd9e
  parameters:
    kind: Managed
    storageaccounttype: Premium_LRS
  provisioner: kubernetes.io/azure-disk
  reclaimPolicy: Delete
  volumeBindingMode: Immediate
- allowVolumeExpansion: true
  apiVersion: storage.k8s.io/v1
  kind: StorageClass
  metadata:
    creationTimestamp: "2022-06-09T13:21:26Z"
    labels:
      addonmanager.kubernetes.io/mode: EnsureExists
      kubernetes.io/cluster-service: "true"
    name: managed
    resourceVersion: "409"
    uid: 1ae7f095-ccc4-4600-9a60-4cae95777385
  parameters:
    cachingmode: ReadOnly
    kind: Managed
    storageaccounttype: StandardSSD_LRS
  provisioner: disk.csi.azure.com
  reclaimPolicy: Delete
  volumeBindingMode: WaitForFirstConsumer
- allowVolumeExpansion: true
  apiVersion: storage.k8s.io/v1
  kind: StorageClass
  metadata:
    creationTimestamp: "2022-06-09T13:21:26Z"
    labels:
      addonmanager.kubernetes.io/mode: EnsureExists
      kubernetes.io/cluster-service: "true"
    name: managed-csi
    resourceVersion: "414"
    uid: cccea515-c1a1-4780-a9b7-6734619d56f4
  parameters:
    skuname: StandardSSD_LRS
  provisioner: disk.csi.azure.com
  reclaimPolicy: Delete
  volumeBindingMode: WaitForFirstConsumer
- allowVolumeExpansion: true
  apiVersion: storage.k8s.io/v1
  kind: StorageClass
  metadata:
    creationTimestamp: "2022-06-09T13:21:26Z"
    labels:
      addonmanager.kubernetes.io/mode: EnsureExists
      kubernetes.io/cluster-service: "true"
    name: managed-csi-premium
    resourceVersion: "415"
    uid: 083c33d9-ddd5-4ac7-b369-4d5b407d45c6
  parameters:
    skuname: Premium_LRS
  provisioner: disk.csi.azure.com
  reclaimPolicy: Delete
  volumeBindingMode: WaitForFirstConsumer
- allowVolumeExpansion: true
  apiVersion: storage.k8s.io/v1
  kind: StorageClass
  metadata:
    creationTimestamp: "2022-06-09T13:21:26Z"
    labels:
      addonmanager.kubernetes.io/mode: EnsureExists
      kubernetes.io/cluster-service: "true"
    name: managed-premium
    resourceVersion: "410"
    uid: 253f1daa-c31e-4fb0-bf96-6606a541fec2
  parameters:
    cachingmode: ReadOnly
    kind: Managed
    storageaccounttype: Premium_LRS
  provisioner: disk.csi.azure.com
  reclaimPolicy: Delete
  volumeBindingMode: WaitForFirstConsumer
- apiVersion: storage.k8s.io/v1
  kind: StorageClass
  metadata:
    annotations:
      meta.helm.sh/release-name: testenv5-ceph-objectstore
      meta.helm.sh/release-namespace: testenv5
    creationTimestamp: "2022-06-09T16:07:03Z"
    labels:
      app.kubernetes.io/managed-by: Helm
    name: testenv5-cephs3
    resourceVersion: "54911"
    uid: 18e72caf-1b85-4447-9401-dca130c117c1
  parameters:
    objectStoreName: testenv5-objectstore
    objectStoreNamespace: testenv5
    region: us-east-1
  provisioner: testenv5.ceph.rook.io/bucket
  reclaimPolicy: Delete
  volumeBindingMode: Immediate
kind: List
metadata:
  resourceVersion: ""

Thanks and Regards,

Yash 

@yashg 

looks like premier script is failing because of the CSI Driver (kubernetes.io/azure-disk) . Possibly an unsupported driver.  I will double check on this . 

========

Starting CSI Checker. Could take up to 5 minutesW0615 07:54:42.905776 7 warnings.go:70] storage.k8s.io/v1beta1 CSIDriver is deprecated in v1.19+, unavailable in v1.22+; use storage.k8s.io/v1 CSIDriverCSI Snapshot Walkthrough:Not a supported CSI driver (kubernetes.io/azure-disk) - Error

========

Below are supported azure csidrivers. 

https://kubernetes-csi.github.io/docs/drivers.html