Hello,
i want to export the VolumeSnapshots after a backup action to my S3 Bucket.
Following is my Policy:
apiVersion: config.kio.kasten.io/v1alpha1
kind: Policy
metadata:
name: live-backup
namespace: mynamespace
uid: ef09f734-0b8b-4a28-a8ee-711b287d4ec3
resourceVersion: "1534281042"
generation: 34
creationTimestamp: 2023-10-23T11:18:55Z
labels:
argocd.argoproj.io/instance: staplerbatterie
annotations:
k10.kasten.io/revalidated-at: 2023-12-05T12:14:11Z
spec:
comment: staplerbatterie Backup Policy
frequency: "@hourly"
paused: false
actions:
- action: backup
backupParameters:
profile:
name: s3-profile
namespace: kasten-io
filters:
includeResources:
- name: data-mariadb-live-0
resource: persistentvolumeclaims
- name: live-cache-0
resource: persistentvolumeclaims
- name: live-log-0
resource: persistentvolumeclaims
excludeResources: []
- action: export
exportParameters:
frequency: "@hourly"
profile:
name: s3-profile
namespace: kasten-io
receiveString: bIzAPpoanm.....
exportData:
enabled: false
retention:
hourly: 2
subFrequency:
minutes:
- 25
hours:
- 10
weekdays:
- 7
days:
- 31
months:
- 12
retention:
hourly: 2
daily: 0
weekly: 0
monthly: 0
yearly: 0
selector:
matchExpressions:
- key: k10.kasten.io/appNamespace
operator: In
values:
- mynamespace
It seems like it tries to connect to a Kopia Repository, but I dont know why, since I configure my S3 Bucket as exportParameters in the Policy configuration.
I get this exception:
kind: ExportAction
apiVersion: actions.kio.kasten.io/v1alpha1
metadata:
name: run-zpzl2tc4sthgf8w
namespace: mynamespace
uid: 7d9ea36f-9382-11ee-8344-ba58ab829f00
resourceVersion: "421619"
creationTimestamp: 2023-12-05T15:25:15Z
labels:
k10.kasten.io/isMetadataExport: "true"
k10.kasten.io/policyName: mynamespace-live-backup
k10.kasten.io/policyNamespace: mynamespace
k10.kasten.io/runActionName: run-zpzl2tc4st
status:
state: Failed
startTime: 2023-12-05T15:25:15Z
endTime: 2023-12-05T15:28:13Z
restorePoint:
name: ""
result:
name: ""
error:
cause: '{"cause":{"cause":{"cause":{"cause":{"message":"Failed to exec command
in pod: command terminated with exit code
1"},"file":"kasten.io/k10/kio/kopia/repository.go:657","function":"kasten.io/k10/kio/kopia.ConnectToKopiaRepository","linenumber":657,"message":"Failed
to connect to the backup
repository"},"file":"kasten.io/k10/kio/kopiaapiserver/api_server.go:246","function":"kasten.io/k10/kio/kopiaapiserver.SetupAPIServerForCollectionExport","linenumber":246,"message":"Failed
to initialize Kopia API
server"},"file":"kasten.io/k10/kio/collections/kopia/handler_kopia.go:123","function":"kasten.io/k10/kio/collections/kopia.(*KopiaHandler).setupAPIServerForExport","linenumber":123,"message":"Failed
to prepare Kopia API server for collections
export"},"file":"kasten.io/k10/kio/exec/phases/phase/migrate.go:155","function":"kasten.io/k10/kio/exec/phases/phase.(*migrateSendPhase).Run","linenumber":155,"message":"Failed
to export collection"}'
message: Job failed to be executed
exceptions:
- cause: '{"cause":{"cause":{"cause":{"cause":{"cause":{"cause":{"cause":{"message":"Failed
to exec command in pod: command terminated with exit code
1"},"file":"kasten.io/k10/kio/kopia/repository.go:587","function":"kasten.io/k10/kio/kopia.ConnectToKopiaRepositoryNoStateChange","linenumber":587,"message":"Failed
to connect to the backup
repository"},"file":"kasten.io/k10/kio/kopiaapiserver/api_server.go:445","function":"kasten.io/k10/kio/kopiaapiserver.setupAPIServerForCollectionRead","linenumber":445,"message":"Failed
to connect to collections Kopia
repository"},"file":"kasten.io/k10/kio/kopiaapiserver/api_server.go:390","function":"kasten.io/k10/kio/kopiaapiserver.SetupAPIServerForCollectionRead","linenumber":390,"message":"Failed
to initialize Kopia API
server"},"file":"kasten.io/k10/kio/collections/kopia/handler_kopia.go:213","function":"kasten.io/k10/kio/collections/kopia.(*KopiaHandler).setupAPIServerForImport","linenumber":213,"message":"Failed
to prepare Kopia API server for collections
read"},"file":"kasten.io/k10/kio/collections/kopia/manager.go:418","function":"kasten.io/k10/kio/collections/kopia.(*KopiaManager).ListByTags","linenumber":418,"message":"Failed
to connect to kopia for
read"},"fields":[{"name":"ManifestID","value":"7d9ea36f-9382-11ee-8344-ba58ab829f00"}],"file":"kasten.io/k10/kio/exec/phases/phase/migrate_cleanup.go:83","function":"kasten.io/k10/kio/exec/phases/phase.(*migrateCleanupPhase).run","linenumber":83,"message":"Failed
to list snapshots"},"fields":[],"message":"Failed to cleanup
collections"}'
message: Failure in collections cleanup phase. Collections may be orphaned.
actionDetails:
phases:
- attempt: 3
endTime: 2023-12-05T15:28:13Z
errors:
- cause: '{"cause":{"cause":{"cause":{"cause":{"message":"Failed to exec command
in pod: command terminated with exit code
1"},"file":"kasten.io/k10/kio/kopia/repository.go:657","function":"kasten.io/k10/kio/kopia.ConnectToKopiaRepository","linenumber":657,"message":"Failed
to connect to the backup
repository"},"file":"kasten.io/k10/kio/kopiaapiserver/api_server.go:246","function":"kasten.io/k10/kio/kopiaapiserver.SetupAPIServerForCollectionExport","linenumber":246,"message":"Failed
to initialize Kopia API
server"},"file":"kasten.io/k10/kio/collections/kopia/handler_kopia.go:123","function":"kasten.io/k10/kio/collections/kopia.(*KopiaHandler).setupAPIServerForExport","linenumber":123,"message":"Failed
to prepare Kopia API server for collections
export"},"file":"kasten.io/k10/kio/exec/phases/phase/migrate.go:155","function":"kasten.io/k10/kio/exec/phases/phase.(*migrateSendPhase).Run","linenumber":155,"message":"Failed
to export collection"}'
message: Job failed to be executed
- cause: '{"cause":{"cause":{"cause":{"cause":{"message":"Failed to exec command
in pod: command terminated with exit code
1"},"file":"kasten.io/k10/kio/kopia/repository.go:657","function":"kasten.io/k10/kio/kopia.ConnectToKopiaRepository","linenumber":657,"message":"Failed
to connect to the backup
repository"},"file":"kasten.io/k10/kio/kopiaapiserver/api_server.go:246","function":"kasten.io/k10/kio/kopiaapiserver.SetupAPIServerForCollectionExport","linenumber":246,"message":"Failed
to initialize Kopia API
server"},"file":"kasten.io/k10/kio/collections/kopia/handler_kopia.go:123","function":"kasten.io/k10/kio/collections/kopia.(*KopiaHandler).setupAPIServerForExport","linenumber":123,"message":"Failed
to prepare Kopia API server for collections
export"},"file":"kasten.io/k10/kio/exec/phases/phase/migrate.go:155","function":"kasten.io/k10/kio/exec/phases/phase.(*migrateSendPhase).Run","linenumber":155,"message":"Failed
to export collection"}'
message: Job failed to be executed
- cause: '{"cause":{"cause":{"cause":{"cause":{"message":"Failed to exec command
in pod: command terminated with exit code
1"},"file":"kasten.io/k10/kio/kopia/repository.go:657","function":"kasten.io/k10/kio/kopia.ConnectToKopiaRepository","linenumber":657,"message":"Failed
to connect to the backup
repository"},"file":"kasten.io/k10/kio/kopiaapiserver/api_server.go:246","function":"kasten.io/k10/kio/kopiaapiserver.SetupAPIServerForCollectionExport","linenumber":246,"message":"Failed
to initialize Kopia API
server"},"file":"kasten.io/k10/kio/collections/kopia/handler_kopia.go:123","function":"kasten.io/k10/kio/collections/kopia.(*KopiaHandler).setupAPIServerForExport","linenumber":123,"message":"Failed
to prepare Kopia API server for collections
export"},"file":"kasten.io/k10/kio/exec/phases/phase/migrate.go:155","function":"kasten.io/k10/kio/exec/phases/phase.(*migrateSendPhase).Run","linenumber":155,"message":"Failed
to export collection"}'
message: Job failed to be executed
exceptions:
- cause: '{"cause":{"cause":{"cause":{"cause":{"cause":{"cause":{"cause":{"message":"Failed
to exec command in pod: command terminated with exit code
1"},"file":"kasten.io/k10/kio/kopia/repository.go:587","function":"kasten.io/k10/kio/kopia.ConnectToKopiaRepositoryNoStateChange","linenumber":587,"message":"Failed
to connect to the backup
repository"},"file":"kasten.io/k10/kio/kopiaapiserver/api_server.go:445","function":"kasten.io/k10/kio/kopiaapiserver.setupAPIServerForCollectionRead","linenumber":445,"message":"Failed
to connect to collections Kopia
repository"},"file":"kasten.io/k10/kio/kopiaapiserver/api_server.go:390","function":"kasten.io/k10/kio/kopiaapiserver.SetupAPIServerForCollectionRead","linenumber":390,"message":"Failed
to initialize Kopia API
server"},"file":"kasten.io/k10/kio/collections/kopia/handler_kopia.go:213","function":"kasten.io/k10/kio/collections/kopia.(*KopiaHandler).setupAPIServerForImport","linenumber":213,"message":"Failed
to prepare Kopia API server for collections
read"},"file":"kasten.io/k10/kio/collections/kopia/manager.go:418","function":"kasten.io/k10/kio/collections/kopia.(*KopiaManager).ListByTags","linenumber":418,"message":"Failed
to connect to kopia for
read"},"fields":[{"name":"ManifestID","value":"7d9ea36f-9382-11ee-8344-ba58ab829f00"}],"file":"kasten.io/k10/kio/exec/phases/phase/migrate_cleanup.go:83","function":"kasten.io/k10/kio/exec/phases/phase.(*migrateCleanupPhase).run","linenumber":83,"message":"Failed
to list snapshots"},"fields":[],"message":"Failed to cleanup
collections"}'
message: Failure in collections cleanup phase. Collections may be orphaned.
name: Exporting Metadata
startTime: 2023-12-05T15:25:16Z
state: failed
updatedTime: 2023-12-05T15:28:13Z
- attempt: 3
endTime: 2023-12-05T15:28:13Z
errors:
- cause: '{"cause":{"cause":{"cause":{"cause":{"message":"Failed to exec command
in pod: command terminated with exit code
1"},"file":"kasten.io/k10/kio/kopia/repository.go:657","function":"kasten.io/k10/kio/kopia.ConnectToKopiaRepository","linenumber":657,"message":"Failed
to connect to the backup
repository"},"file":"kasten.io/k10/kio/kopiaapiserver/api_server.go:246","function":"kasten.io/k10/kio/kopiaapiserver.SetupAPIServerForCollectionExport","linenumber":246,"message":"Failed
to initialize Kopia API
server"},"file":"kasten.io/k10/kio/collections/kopia/handler_kopia.go:123","function":"kasten.io/k10/kio/collections/kopia.(*KopiaHandler).setupAPIServerForExport","linenumber":123,"message":"Failed
to prepare Kopia API server for collections
export"},"file":"kasten.io/k10/kio/exec/phases/phase/migrate.go:155","function":"kasten.io/k10/kio/exec/phases/phase.(*migrateSendPhase).Run","linenumber":155,"message":"Failed
to export collection"}'
message: Job failed to be executed
- cause: '{"cause":{"cause":{"cause":{"cause":{"message":"Failed to exec command
in pod: command terminated with exit code
1"},"file":"kasten.io/k10/kio/kopia/repository.go:657","function":"kasten.io/k10/kio/kopia.ConnectToKopiaRepository","linenumber":657,"message":"Failed
to connect to the backup
repository"},"file":"kasten.io/k10/kio/kopiaapiserver/api_server.go:246","function":"kasten.io/k10/kio/kopiaapiserver.SetupAPIServerForCollectionExport","linenumber":246,"message":"Failed
to initialize Kopia API
server"},"file":"kasten.io/k10/kio/collections/kopia/handler_kopia.go:123","function":"kasten.io/k10/kio/collections/kopia.(*KopiaHandler).setupAPIServerForExport","linenumber":123,"message":"Failed
to prepare Kopia API server for collections
export"},"file":"kasten.io/k10/kio/exec/phases/phase/migrate.go:155","function":"kasten.io/k10/kio/exec/phases/phase.(*migrateSendPhase).Run","linenumber":155,"message":"Failed
to export collection"}'
message: Job failed to be executed
- cause: '{"cause":{"cause":{"cause":{"cause":{"message":"Failed to exec command
in pod: command terminated with exit code
1"},"file":"kasten.io/k10/kio/kopia/repository.go:657","function":"kasten.io/k10/kio/kopia.ConnectToKopiaRepository","linenumber":657,"message":"Failed
to connect to the backup
repository"},"file":"kasten.io/k10/kio/kopiaapiserver/api_server.go:246","function":"kasten.io/k10/kio/kopiaapiserver.SetupAPIServerForCollectionExport","linenumber":246,"message":"Failed
to initialize Kopia API
server"},"file":"kasten.io/k10/kio/collections/kopia/handler_kopia.go:123","function":"kasten.io/k10/kio/collections/kopia.(*KopiaHandler).setupAPIServerForExport","linenumber":123,"message":"Failed
to prepare Kopia API server for collections
export"},"file":"kasten.io/k10/kio/exec/phases/phase/migrate.go:155","function":"kasten.io/k10/kio/exec/phases/phase.(*migrateSendPhase).Run","linenumber":155,"message":"Failed
to export collection"}'
message: Job failed to be executed
exceptions:
- cause: '{"cause":{"cause":{"cause":{"cause":{"cause":{"cause":{"cause":{"message":"Failed
to exec command in pod: command terminated with exit code
1"},"file":"kasten.io/k10/kio/kopia/repository.go:587","function":"kasten.io/k10/kio/kopia.ConnectToKopiaRepositoryNoStateChange","linenumber":587,"message":"Failed
to connect to the backup
repository"},"file":"kasten.io/k10/kio/kopiaapiserver/api_server.go:445","function":"kasten.io/k10/kio/kopiaapiserver.setupAPIServerForCollectionRead","linenumber":445,"message":"Failed
to connect to collections Kopia
repository"},"file":"kasten.io/k10/kio/kopiaapiserver/api_server.go:390","function":"kasten.io/k10/kio/kopiaapiserver.SetupAPIServerForCollectionRead","linenumber":390,"message":"Failed
to initialize Kopia API
server"},"file":"kasten.io/k10/kio/collections/kopia/handler_kopia.go:213","function":"kasten.io/k10/kio/collections/kopia.(*KopiaHandler).setupAPIServerForImport","linenumber":213,"message":"Failed
to prepare Kopia API server for collections
read"},"file":"kasten.io/k10/kio/collections/kopia/manager.go:418","function":"kasten.io/k10/kio/collections/kopia.(*KopiaManager).ListByTags","linenumber":418,"message":"Failed
to connect to kopia for
read"},"fields":[{"name":"ManifestID","value":"7d9ea36f-9382-11ee-8344-ba58ab829f00"}],"file":"kasten.io/k10/kio/exec/phases/phase/migrate_cleanup.go:83","function":"kasten.io/k10/kio/exec/phases/phase.(*migrateCleanupPhase).run","linenumber":83,"message":"Failed
to list snapshots"},"fields":[],"message":"Failed to cleanup
collections"}'
message: Failure in collections cleanup phase. Collections may be orphaned.
name: Monitoring Actions
startTime: 2023-12-05T15:25:16Z
state: succeeded
updatedTime: 2023-12-05T15:28:13Z
progress: 100
spec:
subject:
apiVersion: internals.kio.kasten.io/v1alpha1
kind: Manifest
name: 7d8f046a-9382-11ee-8344-ba58ab829f00
scheduledTime: 2023-12-05T15:25:00Z
frequency: "@hourly"
receiveString: bIzAPpoan.....
profile:
name: s3-profile
namespace: kasten-io
exportData:
enabled: false
Also my Retentionconfigurations is not working as I expect.
I expect that I never have more than 2 backups in my K10 system and so never have more than 2 VolumeSnapshots in my cluster (make by K10). But in reality, no backups get deleted ever, it keeps growing.
I have another K10 Disaster Policy, here the export to my S3 location works as I expect.
Using Version 6.5.0.