k10 on k3s - Deploy issues - crypto-sv

Doing a deployment first time for Kasteen outside using “operator” within OCP, and that also via IAC.

Single server Ubuntu 22.04 + k3s + Cilium Load Balancer ingress services.

A couple issues:

1. When deploying, it I can now get far enough to be able to login but only when creating a temporary token vs the permanent one I built into the deployment.
2. When I need to run deployment fresh.  I run delete and it will not clear out namespace.  Delete namespace hangs “terminating” and I have to run force delete.
3. Deploy has error on several services that show errors.. But most reference error below about “crypto-sv”  (similar to this post without response Kasten fresh installation on K8S cluster issues with the Crypto-svc | Veeam Community Resource Hub)

Delete / Cleanup Hacks due to delete issue

```

kubectl delete ns kasten-io 

# break above after 5min as it hangs “terminating”

# Force ns delete when kasten fails to clean up:

NS=`kubectl get ns |grep Terminating | awk 'NR==1 {print $1}'` && kubectl get namespace "$NS" -o json   | tr -d "\n" | sed "s/\"finalizers\": \[[^]]\+\]/\"finalizers\": []/"   | kubectl replace --raw /api/v1/namespaces/$NS/finalize -f -

# remove all pv within old namespacek kasten-io

k get pv |grep kasten-io | cut -d' ' -f1

```

Error in UI on several services

```

[&DeploymentCondition{Type:Available,Status:False,Reason:MinimumReplicasUnavailable,Message:Deployment does not have minimum availability.,LastUpdateTime:2024-01-27 16:18:01 +0000 UTC,LastTransitionTime:2024-01-27 16:18:01 +0000 UTC,} &DeploymentCondition{Type:Progressing,Status:False,Reason:ProgressDeadlineExceeded,Message:ReplicaSet "crypto-svc-5bc6d9fdd5" has timed out progressing.,LastUpdateTime:2024-01-27 16:28:02 +0000 UTC,LastTransitionTime:2024-01-27 16:28:02 +0000 UTC,}]

```

Deployment with CICD:  (create namespace and secret, rest is feed of valeus to helm)

```

apiVersion: v1

kind: Namespace

metadata:

  name: kasten-io

  labels:

    name: kasten-io

---

apiVersion: v1

kind: Secret

metadata:

  name: veeam-sa

  namespace: kasten-io

  annotations:

    kubernetes.io/service-account.name: "veeam-sa"

type: kubernetes.io/service-account-token

data:

  extra: R0R1QD2sYWI= # echo -n "kastenveeam" |base64

```

Values file

```
---
rbac:
  create: true
serviceAccount:
  # Specifies whether a ServiceAccount should be created
  create: true
  # The name of the ServiceAccount to use.
  # If not set and create is true, a name is derived using the release and chart names.
  name: "veeam-sa"

global:
  # These are the default values for picking k10 images. They can be overridden
  # to specify a particular registy and tag.
  image:
    registry: gcr.io/kasten-images
    tag: ''
    pullPolicy: Always
  airgapped:
    repository: ''
  persistence:
    mountPath: "/media/md0/containers/k10state"  # path for RAID array with capacity
    enabled: true
    storageClass: "pandoras-md0-local-path-sc"
    accessMode: ReadWriteOnce
ingress:
  create: true
  tls:
    enabled: true
    secretName: "veeam-sa" # Not sure if this is total number of refernces needed for secrets token
  class: "" #Ingress controller type --> Assume this will use default
  host: "k3spandorak10.acme.local" #ingress object host name
  urlPath: "/k10" #url path for k10 gateway
  pathType: "ImplementationSpecific"

eula:
  accept: true #true value if EULA accepted
  company: "acme"  # not in values example but required in chart
  email: "tec@acme.com"

externalGateway:
  create: true
  # Any standard service annotations
  annotations:
    "io.cilium/lb-ipam-ips": "172.16.103.90"
    "meta.helm.sh/release-namespace": "kasten-io"
  # Host and domain name for the K10 API server
  fqdn:
    name: "acme.local"
    #Supported types route53-mapper, external-dns
    type: "external-dns"
auth:
  tokenAuth:
    enabled: true

```

CI Deploy 

```

.build-stage:

  stage: build

  script:

    ## Run ansible playbook flow to stage for helm deployments

    - ansible-playbook pods.yml -vvv

    ## Deploy backup - Kasten

    - helm repo add kasten https://charts.kasten.io/

    - helm install k3sk10 kasten/k10 -f ./backup/files/kasten_values.yaml --create-namespace --namespace=kasten-io

```

deploy results

```

$ helm repo add kasten https://charts.kasten.io/
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: ./files/pandora_k3s_kubeconfig.yml
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: ./files/pandora_k3s_kubeconfig.yml
"kasten" has been added to your repositories
$ helm install k3sk10 kasten/k10 -f ./backup/files/kasten_values.yaml --create-namespace --namespace=kasten-io
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: ./files/pandora_k3s_kubeconfig.yml
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: ./files/pandora_k3s_kubeconfig.yml
NAME: k3sk10
LAST DEPLOYED: Sat Jan 27 17:51:33 2024
NAMESPACE: kasten-io
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Thank you for installing Kasten’s K10 Data Management Platform 6.5.3!
Documentation can be found at https://docs.kasten.io/.
How to access the K10 Dashboard:
You are using the system's default ingress controller. Please ask your
administrator for instructions on how to access the cluster.
WebUI location:  https://k3spandorak10.acme.local//k10
In addition,
The K10 Dashboard is accessible via http://acme.local/k3sk10/#/
In addition,
To establish a connection to it use the following `kubectl` command:
`kubectl --namespace kasten-io port-forward service/gateway 8080:80`
The Kasten dashboard will be available at: `http://127.0.0.1:8080/k3sk10/#/`
Cleaning up project directory and file based variables

```