Hi,
Using Kasten K10 with amazon s3 location, my aws s3 bucket keeps increasing quite a lot. Even if I removed hundreds of resotrepointcontent, I dont see any drops in the S3 bucket usage. I also decreased the retension in my backup policies. Hoeverer I dont see any improvements.
Anybody experienced this behavior?
Best answer by Hagag
What is the Current volume size of your application? ( not the snapshots )
Kopia Blobs only get deleted when they are not any longer referenced by any snapshot. So you won’t notice a huge drop in your repo size
let me explain more,
Imagine the first snapshot had files/contents [File1, File2, File3, File4, File5] and this will be linked to Kopia blob1
blob1= [File1, File2, File3, File4, File5]
then you have 2nd snapshot that contains [File1, File2, File3, File4, File5, File6, File7, File8] and this will be linked to the Kopia blob as below
blob2= [ File6, File7, File8]
enventually you have 2 blobs
blob1 is related to snapshot1 & snapshot2
blob2 is related to snapshot2
if you deleted resotorepointcontent for snapshot1, you still have blob1 referenced to snapshot2 and you still need it for restoring snapshot2. that is why you won`t notice a huge drop in your repo size.
Ahmed Hagag
+2Hello
K10 using Kopia for exporting your backup to S3, and Here is the content deletion process used by Kopia
First snapshot deletion only removes the snapshot manifest, not the content/data
There is something called snapshot garbage collection which finds and deletes unreferenced content ( content is only marked as deleted )
Then Index compaction removes index entries marked as deleted
content removal occurs during blob compaction ( Copies referenced contents to new blobs & delete old blobs
Blobs only get deleted when they are not any longer referenced by any snapshot. So you won’t notice a huge drop in your repo size
note that Full Maintenance Tasks are enabled by default and will execute every 24 hours. so better to wait for the next day and check your bucket size.
Ahmed Hagag
I deleted the backups (restorePointContent) already some weeks ago. I think I should see a drop as I literaly reclaimed 6TB of “Snapshot Storage” (local snapshots) and I saw the drop on the K10 dashboard. For any reason, I have no storage reclaimed on the object storage graph.
like the documentation said, for example:
# delete restore point content 'sample-app-backup-rpc-raq923'
kubectl delete restorepointcontents.apps.kio.kasten.io sample-app-backup-rpc-raq923
restorepointcontent.apps.kio.kasten.io/sample-app-backup-rpc-raq923 deleted
In addition to policy-based deletion of RestorePoints and RestorePointContents, you can explicitly delete a RestorePointContent. Deleting a RestorePointContent will cause its bound RestorePoints and all associated resources (e.g. persistent volumes, application-level artifacts, etc.) to be reclaimed.
Deleting a RestorePointContent resource creates a RetireAction that can be examined to monitor progress of the retirement.
+2What is the Current volume size of your application? ( not the snapshots )
Kopia Blobs only get deleted when they are not any longer referenced by any snapshot. So you won’t notice a huge drop in your repo size
let me explain more,
Imagine the first snapshot had files/contents [File1, File2, File3, File4, File5] and this will be linked to Kopia blob1
blob1= [File1, File2, File3, File4, File5]
then you have 2nd snapshot that contains [File1, File2, File3, File4, File5, File6, File7, File8] and this will be linked to the Kopia blob as below
blob2= [ File6, File7, File8]
enventually you have 2 blobs
blob1 is related to snapshot1 & snapshot2
blob2 is related to snapshot2
if you deleted resotorepointcontent for snapshot1, you still have blob1 referenced to snapshot2 and you still need it for restoring snapshot2. that is why you won`t notice a huge drop in your repo size.
Ahmed Hagag
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.
