Hi,

I am struggling with a problem. 

Disk space for service Catalog is low at 21%

Disk space for service Jobs is low at 21%

Disk space for service Logging is low at 21%

I can’t, for the life of me, work out how to fix this! Nothing I have come across seems to work.

I can see this if I query the Helm chart:

bash-4.4$ helm show all kasten/k10

<snip>

  persistence:
    mountPath: "/mnt/k10state"
    enabled: true
    ## If defined, storageClassName: <storageClass>
    ## If set to "-", storageClassName: "", which disables dynamic provisioning
    ## If undefined (the default) or set to null, no storageClassName spec is
    ##   set, choosing the default provisioner.  (gp2 on AWS, standard on
    ##   GKE, AWS & OpenStack)
    ##
    storageClass: ""
    accessMode: ReadWriteOnce
    size: 20Gi
    metering:
      size: 2Gi
    catalog:
      size: ""
    jobs:
      size: ""
    logging:
      size: ""
    grafana:
      # Default value is set to 5Gi. This is the same as the default value
      # from previous releases <= 4.5.1 where the Grafana sub chart used to
      # reference grafana.persistence.size instead of the global values.
      # Since the size remains the same across upgrades, the Grafana PVC
      # is not deleted and recreated which means no Grafana data is lost
      # while upgrading from <= 4.5.1
      size: 5Gi

<snip>

That matches the size I see for the pod (just showing the catalog one here):

bash-4.4$ kubectl -n kasten-io exec catalog-svc-cffb6bcdf-fl4tj -- df -h
Defaulted container "catalog-svc" out of: catalog-svc, kanister-sidecar, upgrade-init (init), schema-upgrade-check (init)
Filesystem      Size  Used Avail Use% Mounted on
overlay          17G   13G  4.0G  76% /
tmpfs            64M     0   64M   0% /dev
tmpfs           2.9G     0  2.9G   0% /sys/fs/cgroup
/dev/vda1        17G   13G  4.0G  76% /mnt/k10state
shm              64M     0   64M   0% /dev/shm
tmpfs           5.7G   12K  5.7G   1% /run/secrets/kubernetes.io/serviceaccount
tmpfs           2.9G     0  2.9G   0% /proc/asound
tmpfs           2.9G     0  2.9G   0% /proc/acpi
tmpfs           2.9G     0  2.9G   0% /proc/scsi
tmpfs           2.9G     0  2.9G   0% /sys/firmware
bash-4.4$

But if I try and change the pod size, it just  doesn’t work (this is from trying an upgrade and a fresh install - same result):

bash-4.4$ helm upgrade --install k10 kasten/k10 --namespace=kasten-io --set external.Gateway.create=true --set global.persistence.size=45G --set global.persistence.catalog.size=45G
Release "k10" does not exist. Installing it now.
NAME: k10
LAST DEPLOYED: Mon Sep 25 08:40:59 2023
NAMESPACE: kasten-io
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Thank you for installing Kasten?s K10 Data Management Platform 6.0.5!

Documentation can be found at https://docs.kasten.io/.

How to access the K10 Dashboard:

To establish a connection to it use the following `kubectl` command:

`kubectl --namespace kasten-io port-forward service/gateway 8080:8000`

The Kasten dashboard will be available at: `http://127.0.0.1:8080/k10/#/`

The pod size is still 20G.

I have tried using “45Gi”, 45Gi, “45G” and 45G, but nothing seems to change the values in Helm to anything other than 20G. 

I am using a default storage claim:

bash-4.4$ kubectl get sc
NAME                 PROVISIONER                RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
standard (default)   k8s.io/minikube-hostpath   Delete          Immediate           true                   37d
 

This is K10 running on K8s on RHEL8.8.

What am I missing?

Thanks in advance for any pointers!