Hello,
In version 7.0.13, kasten backup jobs are failing during the export phase
error:
cause: '{"cause":{"cause":{"cause":{"cause":{"cause":{"cause":{"cause":{"cause":{"cause":{"message":"command
terminated with exit code 1.\nstdout:
{\"id\":\"9ba1b21e2c3cf426425e7a1ea0b2a14f\",\"source\":{\"host\":\"e81a2ed4-79e9-42b3-84ac-a8b05153f716.central-db.central-db\",\"userName\":\"k10-admin\",\"path\":\"/mnt/vol_data/kanister-pvc\"},\"description\":\"\",\"startTime\":\"2024-11-07T14:17:42.792216969Z\",\"endTime\":\"2024-11-07T14:17:56.725058015Z\",\"rootEntry\":{\"name\":\"kanister-pvc\",\"type\":\"d\",\"mode\":\"020000775\",\"mtime\":\"2024-10-10T11:22:26.741964546Z\",\"gid\":70,\"obj\":\"k426a49ed67610fa2c342dde0e67a76f1\",\"summ\":{\"size\":219511781,\"files\":1941,\"symlinks\":0,\"dirs\":29,\"maxTime\":\"2024-11-07T14:14:49.962036091Z\",\"numFailed\":0}}}\nstderr:
| 1 hashing, 0 hashed (0, 0 cached (0
estimating...\r\n/ 3 hashing, 17 hashed (2.4 MB), 0 cached (0
355.9 KB, estimated 219.5 MB (1.1%) 17m16s left\r\n* 0 hashing, 1941
hashed (219.5 MB), 0 cached (0
(100.0%) 0s left\r\n\u001b131mERROR\u001b\0m flush error: error waiting
for async writes: unable to create pending pack: unable to get session ID:
unable to write session marker: unable to write session marker:
s9a0da7cb5e2789d77b6da3ab6ccad0cc-s298444e6acb0feca12e: error determining
sharded path: error getting sharding parameters for storage: unable to
complete
GetBlobFromPath:/mnt/data/k10/72730d98-9daf-4a6b-a32e-350775389119/migration/repo/e81a2ed4-79e9-42b3-84ac-a8b05153f716/.shards
despite 10 retries: open
/mnt/data/k10/72730d98-9daf-4a6b-a32e-350775389119/migration/repo/e81a2ed4-79e9-42b3-84ac-a8b05153f716/.shards:
permission
denied"},"file":"github.com/kanisterio/kanister@v0.0.0-20241025011050-227a815a2c61/pkg/kube/exec.go","function":"github.com/kanisterio/kanister/pkg/kube.ExecWithOptions","linenumber":156,"message":"Failed
to exec command in
pod"},"file":"kasten.io/k10/kio/kanister/function/kio_copy_volume_data.go:384","function":"kasten.io/k10/kio/kanister/function.CopyVolumeData.copyVolumeDataPodExecFunc.func2","linenumber":384,"message":"Failed
to create and upload
backup"},"file":"kasten.io/k10/kio/kanister/function/kio_copy_volume_data.go:166","function":"kasten.io/k10/kio/kanister/function.CopyVolumeData","linenumber":166,"message":"Failed
to execute copy volume data pod
function"},"file":"kasten.io/k10/kio/exec/internal/snapshotconverters/ac_gvc_converter.go:249","function":"kasten.io/k10/kio/exec/internal/snapshotconverters.(*GVCConverterInternalAPIImpl).genericVolumeCopy","linenumber":249,"message":"failed
running
copyVolumeData"},"file":"kasten.io/k10/kio/exec/internal/snapshotconverters/ac_gvc_converter.go:170","function":"kasten.io/k10/kio/exec/internal/snapshotconverters.(*GVCConverterInternalAPIImpl).CopySnapshotRestoredInPVC","linenumber":170,"message":"failed
running
genericVolumeCopy"},"file":"kasten.io/k10/kio/exec/internal/snapshotconverters/ac_gvc_converter.go:77","function":"kasten.io/k10/kio/exec/internal/snapshotconverters.(*GVCConverter).Convert","linenumber":77,"message":"Error
creating portable
snapshot"},"fields":b{"name":"type","value":"CSI"},{"name":"id","value":"k10-csi-snap-jbbcf5xdchrrl8tx"}],"file":"kasten.io/k10/kio/exec/phases/phase/artifactcopier.go:546","function":"kasten.io/k10/kio/exec/phases/phase.(*ArtifactCopier).convertSnapshots.func1","linenumber":546,"message":"Failed
to export snapshot
data"},"file":"kasten.io/k10/kio/exec/phases/phase/artifactcopier.go:276","function":"kasten.io/k10/kio/exec/phases/phase.(*ArtifactCopier).Copy","linenumber":276,"message":"Error
converting
snapshots"},"file":"kasten.io/k10/kio/exec/phases/phase/export.go:168","function":"kasten.io/k10/kio/exec/phases/phase.(*exportRestorePointPhase).Run","linenumber":168,"message":"Failed
to copy artifacts"}'