Recommanded way to backup Postgres (StackGres)

You want to learn about Kanister blueprints in K10 = https://docs.kasten.io/latest/kanister/kanister.html

https://Kanister.io is now a CNCF Sandbox project originally created by Kasten!

There are many approaches to Postgres and there are example blueprints on the Kanister.io web site. There many Operator approaches to Postgres, as well!

So far EnterpriseDB’s Postgres operator has an officially supported Kanister blueprint, but I’m not aware of any community contribution for StackGres. Please review what’s out there, there are many possible next steps: come to a Kanister community meeting, propose an issue, adapt existing blueprints to your needs, and please contribute back.

--Mark

We also use StackGres for our databases.

Now we tried Kasten and found that our DB StatefulSets are not correctly recognized as ready by Kasten.

When I start a backup, it waits for 3 running replicas:
3 replicas specified and only 0 are running
But 3 Pods are running fine inside the StatefulSet, the SGCluster is healthy and works fine.

 

- cause:
    cause:
      cause:
        cause:
          cause:
            message: "Specified 3 replicas and only 0 are running: Context done while
              polling: context deadline exceeded"
          fields:
            - name: namespace
              value: test
            - name: name
              value: test
          file: kasten.io/k10/kio/exec/phases/phase/snapshot.go:426
          function: kasten.io/k10/kio/exec/phases/phase.WaitOnWorkloadReady
          linenumber: 426
          message: Statefulset not in ready state. Retry the operation once Statefulset is
            ready
        fields:
          - name: workloadName
            value: test
          - name: workloadNamespace
            value: test
        file: kasten.io/k10/kio/exec/phases/backup/snapshot_data_phase.go:1158
        function: kasten.io/k10/kio/exec/phases/backup.WaitForWorkloadWithSkipWait
        linenumber: 1158
        message: Error while waiting for workload to be ready
      fields:
        - name: workloadName
          value: test
        - name: workloadNamespace
          value: test
      file: kasten.io/k10/kio/exec/phases/backup/snapshot_data_phase.go:1173
      function: kasten.io/k10/kio/exec/phases/backup.WaitForWorkload
      linenumber: 1173
      message: Error while waiting for workload to become ready
    file: kasten.io/k10/kio/exec/phases/backup/snapshot_data_phase.go:372
    function: kasten.io/k10/kio/exec/phases/backup.processVolumeArtifacts
    linenumber: 372
    message: Error encountered waiting for workload
  message: Job failed to be executed
- cause:
    cause:
      cause:
        cause:
          cause:
            message: "Specified 1 replicas and only 0 are running: could not get
              StatefulSet{Namespace: test, Name: test}: client rate limiter Wait
              returned an error: rate: Wait(n=1) would exceed context deadline"
          fields:
            - name: namespace
              value: test
            - name: name
              value: test
          file: kasten.io/k10/kio/exec/phases/phase/snapshot.go:426
          function: kasten.io/k10/kio/exec/phases/phase.WaitOnWorkloadReady
          linenumber: 426
          message: Statefulset not in ready state. Retry the operation once Statefulset is
            ready
        fields:
          - name: workloadName
            value: test
          - name: workloadNamespace
            value: test
        file: kasten.io/k10/kio/exec/phases/backup/snapshot_data_phase.go:1158
        function: kasten.io/k10/kio/exec/phases/backup.WaitForWorkloadWithSkipWait
        linenumber: 1158
        message: Error while waiting for workload to be ready
      fields:
        - name: workloadName
          value: test
        - name: workloadNamespace
          value: test
      file: kasten.io/k10/kio/exec/phases/backup/snapshot_data_phase.go:1173
      function: kasten.io/k10/kio/exec/phases/backup.WaitForWorkload
      linenumber: 1173
      message: Error while waiting for workload to become ready
    file: kasten.io/k10/kio/exec/phases/backup/snapshot_data_phase.go:372
    function: kasten.io/k10/kio/exec/phases/backup.processVolumeArtifacts
    linenumber: 372
    message: Error encountered waiting for workload
  message: Job failed to be executed
- cause:
    cause:
      cause:
        cause:
          cause:
            message: "Specified 3 replicas and only 0 are running: could not get
              StatefulSet{Namespace: test, Name: test}: client rate limiter Wait
              returned an error: context deadline exceeded"
          fields:
            - name: namespace
              value: test
            - name: name
              value: test
          file: kasten.io/k10/kio/exec/phases/phase/snapshot.go:426
          function: kasten.io/k10/kio/exec/phases/phase.WaitOnWorkloadReady
          linenumber: 426
          message: Statefulset not in ready state. Retry the operation once Statefulset is
            ready
        fields:
          - name: workloadName
            value: test
          - name: workloadNamespace
            value: test
        file: kasten.io/k10/kio/exec/phases/backup/snapshot_data_phase.go:1158
        function: kasten.io/k10/kio/exec/phases/backup.WaitForWorkloadWithSkipWait
        linenumber: 1158
        message: Error while waiting for workload to be ready
      fields:
        - name: workloadName
          value: test
        - name: workloadNamespace
          value: test
      file: kasten.io/k10/kio/exec/phases/backup/snapshot_data_phase.go:1173
      function: kasten.io/k10/kio/exec/phases/backup.WaitForWorkload
      linenumber: 1173
      message: Error while waiting for workload to become ready
    file: kasten.io/k10/kio/exec/phases/backup/snapshot_data_phase.go:372
    function: kasten.io/k10/kio/exec/phases/backup.processVolumeArtifacts
    linenumber: 372
    message: Error encountered waiting for workload
  message: Job failed to be executed

Because the best way to engage the Kanister project with a feature request is a Kanister issue, I’ve copied your post. Please comment on and follow https://github.com/kanisterio/kanister/issues/2927

Thanks for posting it to Github.
I have added some more informations to reproduce the issue. But not sure why it was already closed.

Hi @MaierS ,

I had the team review it and this appears to be Veeam Kasten (formerly K10) related, so out of scope for them. Sorry, that means I channeled your issue to the wrong place and they closed it.
 

I didn’t have a chance to follow up until now. So let’s see if any of Kasten support can troubleshoot the logs above, I’ll alert them to take a look.

We were able to reproduce this issue with the stackgres operator and are trying to find the cause for this. We will file a bug internally to track and resolve this. Will keep you posted once we have a fix for this. 

As mentioned in the previously linked issue on GitHub: this issue has been resolved with PR #3209 which is included in Kasten K10 release 7.0.14.

That’s correct ​@alwyna Thanks for updating the thread. 

This is fixed in 7.0.14 version of Veeam Kasten.