Skip to main content
  • EN

I’m experienced with Kubernetes but new to K10. I’m deploying it on my cluster via the K10 Helm chart. I’m running into a problem with the Ingress settings. I’m using cert-manager to provision a TLS cert via Let’s Encrypt, which I do regularly. Something is wrong with the Ingress resource generated by the Helm chart that means the http01 solver of cert-manager does not complete successfully.

This is the relevant part of my values for the Helm chart: 

global:
  ingress:
    create: true
    urlPath: "" #url path for k10 gateway

ingress:
  create: true
  tls:
    enabled: true
    secretName: ingress-tls
 class: public
  host: backup.mydomain.com
  urlPath: "" #url path for k10 gateway
  pathType: ImplementationSpecific
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod

Cert remains unprovisioned:

$ kubectl get cert
NAME          READY   SECRET        AGE
ingress-tls   False   ingress-tls   25m

The generated Ingress resource:

$ kubectl get ing k10-ingress -o yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
    meta.helm.sh/release-name: k10
    meta.helm.sh/release-namespace: kasten-io
  creationTimestamp: "2023-09-05T19:08:51Z"
  generation: 9
  labels:
    app: k10
    app.kubernetes.io/instance: k10
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: k10
    helm.sh/chart: k10-6.0.6
    heritage: Helm
    release: k10
  name: k10-ingress
  namespace: kasten-io
  resourceVersion: "89575587"
  uid: 0af872d8-85e3-4634-b580-004b5da7c821
spec:
  ingressClassName: public
  rules:
  - host: backup.mydomain.com
    http:
      paths:
      - backend:
          service:
            name: gateway
            port:
              number: 8000
        path: /k10/
        pathType: ImplementationSpecific
  tls:
  - hosts:
    - backup.mydomain.com
    secretName: ingress-tls
status:
  loadBalancer:
    ingress:
    - ip: 127.0.0.1

I can’t personally see anything wrong with that Ingress record but all my other Ingresses on the cluster work and can successfully provision TLS certs

@jaiganeshjk 

@jgazeley Thanks for posting your question here.
I understand that you are facing issues in cert-manager certificate provisioning while creating ingress with K10.

Do you see any issue/events in the certificate CR resource describe output provisioned by the cert-manager for this ingress ?

kubectl describe certificates ingress-tls -n kasten-io

 

Hey @jaiganeshjk. There isn’t much in the output of that command:

 

$ kubectl describe certificates ingress-tls -n kasten-io
Name:         ingress-tls
Namespace:    kasten-io
Labels:       app=k10
              app.kubernetes.io/instance=k10
              app.kubernetes.io/managed-by=Helm
              app.kubernetes.io/name=k10
              helm.sh/chart=k10-6.0.7
              heritage=Helm
              release=k10
Annotations:  <none>
API Version:  cert-manager.io/v1
Kind:         Certificate
Metadata:
  Creation Timestamp:  2023-09-15T19:43:24Z
  Generation:          1
  Owner References:
    API Version:           networking.k8s.io/v1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  Ingress
    Name:                  k10-ingress
    UID:                   0af872d8-85e3-4634-b580-004b5da7c821
  Resource Version:        91780513
  UID:                     5f7be6b6-8305-4d9f-ba35-7c6c9e426c86
Spec:
  Dns Names:
    backup.gazeley.uk
  Issuer Ref:
    Group:      cert-manager.io
    Kind:       ClusterIssuer
    Name:       letsencrypt-prod
  Secret Name:  ingress-tls
  Usages:
    digital signature
    key encipherment
Status:
  Conditions:
    Last Transition Time:        2023-09-15T19:43:24Z
    Message:                     Issuing certificate as Secret does not exist
    Observed Generation:         1
    Reason:                      DoesNotExist
    Status:                      False
    Type:                        Ready
    Last Transition Time:        2023-09-15T19:43:24Z
    Message:                     Issuing certificate as Secret does not exist
    Observed Generation:         1
    Reason:                      DoesNotExist
    Status:                      True
    Type:                        Issuing
  Next Private Key Secret Name:  ingress-tls-zlzpd
Events:
  Type    Reason     Age    From                                       Message
  ----    ------     ----   ----                                       -------
  Normal  Issuing    2m54s  cert-manager-certificates-trigger          Issuing certificate as Secret does not exist
  Normal  Generated  2m54s  cert-manager-certificates-key-manager      Stored new private key in temporary Secret resource "ingress-tls-zlzpd"
  Normal  Requested  2m54s  cert-manager-certificates-request-manager  Created new CertificateRequest resource "ingress-tls-1"

 

I solved this issue. cert-manager was unable to provision a certificate using the http01 solver due to strict networkPolicies that were applied to the kasten-io namespace.

I realised that the k10 helm chart has a default values entry:

networkPolicy:
  create: true

I changed this to False and it works now. Usually networkPolicy should be set to a default of False to avoid problems like this, and the admin can opt in if they require it.

Ideally the networkPolicy should also include a rule to allow the http01 solver, if the admin wishes to use it.

Hey Folks,

Just wondering if some other option might become available. I setup a smallstep ca in podman and then cert-manager in K3S, have it leveraging ACME for auto certificates but had the same issue with Kasten. 

It works if I remove the network polices but otherwise the request for certificate remains pending. 

On a side note if you want to leverage your internal lab DNS server inside K3S you need to add a custom configmap for coredns:

apiVersion: v1
kind: ConfigMap
metadata:
  name: coredns-custom
  namespace: kube-system
data:
  lab1.local: |
    lab5.local {
      forward . 192.168.0.199
    }

 

Comment


Terms & Conditions