• EN
Solved

Helm chart Ingress problem with cert-manager

  • 9 September 2023
  • 4 comments
  • 110 views
J

I’m experienced with Kubernetes but new to K10. I’m deploying it on my cluster via the K10 Helm chart. I’m running into a problem with the Ingress settings. I’m using cert-manager to provision a TLS cert via Let’s Encrypt, which I do regularly. Something is wrong with the Ingress resource generated by the Helm chart that means the http01 solver of cert-manager does not complete successfully.

This is the relevant part of my values for the Helm chart: 

global:
  ingress:
    create: true
    urlPath: "" #url path for k10 gateway

ingress:
  create: true
  tls:
    enabled: true
    secretName: ingress-tls
 class: public
  host: backup.mydomain.com
  urlPath: "" #url path for k10 gateway
  pathType: ImplementationSpecific
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod

Cert remains unprovisioned:

$ kubectl get cert
NAME          READY   SECRET        AGE
ingress-tls   False   ingress-tls   25m

The generated Ingress resource:

$ kubectl get ing k10-ingress -o yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
    meta.helm.sh/release-name: k10
    meta.helm.sh/release-namespace: kasten-io
  creationTimestamp: "2023-09-05T19:08:51Z"
  generation: 9
  labels:
    app: k10
    app.kubernetes.io/instance: k10
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: k10
    helm.sh/chart: k10-6.0.6
    heritage: Helm
    release: k10
  name: k10-ingress
  namespace: kasten-io
  resourceVersion: "89575587"
  uid: 0af872d8-85e3-4634-b580-004b5da7c821
spec:
  ingressClassName: public
  rules:
  - host: backup.mydomain.com
    http:
      paths:
      - backend:
          service:
            name: gateway
            port:
              number: 8000
        path: /k10/
        pathType: ImplementationSpecific
  tls:
  - hosts:
    - backup.mydomain.com
    secretName: ingress-tls
status:
  loadBalancer:
    ingress:
    - ip: 127.0.0.1

I can’t personally see anything wrong with that Ingress record but all my other Ingresses on the cluster work and can successfully provision TLS certs

icon

Best answer by jgazeley 24 October 2023, 21:18

View original

4 comments

Madi.Cristil
Userlevel 7
Badge +7

@jaiganeshjk 

Madi Cristil
jaiganeshjk
Userlevel 6
Badge +2

@jgazeley Thanks for posting your question here.
I understand that you are facing issues in cert-manager certificate provisioning while creating ingress with K10.

Do you see any issue/events in the certificate CR resource describe output provisioned by the cert-manager for this ingress ?

kubectl describe certificates ingress-tls -n kasten-io

 

Jaiganesh
J

Hey @jaiganeshjk. There isn’t much in the output of that command:

 

$ kubectl describe certificates ingress-tls -n kasten-io
Name:         ingress-tls
Namespace:    kasten-io
Labels:       app=k10
              app.kubernetes.io/instance=k10
              app.kubernetes.io/managed-by=Helm
              app.kubernetes.io/name=k10
              helm.sh/chart=k10-6.0.7
              heritage=Helm
              release=k10
Annotations:  <none>
API Version:  cert-manager.io/v1
Kind:         Certificate
Metadata:
  Creation Timestamp:  2023-09-15T19:43:24Z
  Generation:          1
  Owner References:
    API Version:           networking.k8s.io/v1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  Ingress
    Name:                  k10-ingress
    UID:                   0af872d8-85e3-4634-b580-004b5da7c821
  Resource Version:        91780513
  UID:                     5f7be6b6-8305-4d9f-ba35-7c6c9e426c86
Spec:
  Dns Names:
    backup.gazeley.uk
  Issuer Ref:
    Group:      cert-manager.io
    Kind:       ClusterIssuer
    Name:       letsencrypt-prod
  Secret Name:  ingress-tls
  Usages:
    digital signature
    key encipherment
Status:
  Conditions:
    Last Transition Time:        2023-09-15T19:43:24Z
    Message:                     Issuing certificate as Secret does not exist
    Observed Generation:         1
    Reason:                      DoesNotExist
    Status:                      False
    Type:                        Ready
    Last Transition Time:        2023-09-15T19:43:24Z
    Message:                     Issuing certificate as Secret does not exist
    Observed Generation:         1
    Reason:                      DoesNotExist
    Status:                      True
    Type:                        Issuing
  Next Private Key Secret Name:  ingress-tls-zlzpd
Events:
  Type    Reason     Age    From                                       Message
  ----    ------     ----   ----                                       -------
  Normal  Issuing    2m54s  cert-manager-certificates-trigger          Issuing certificate as Secret does not exist
  Normal  Generated  2m54s  cert-manager-certificates-key-manager      Stored new private key in temporary Secret resource "ingress-tls-zlzpd"
  Normal  Requested  2m54s  cert-manager-certificates-request-manager  Created new CertificateRequest resource "ingress-tls-1"

 

J

I solved this issue. cert-manager was unable to provision a certificate using the http01 solver due to strict networkPolicies that were applied to the kasten-io namespace.

I realised that the k10 helm chart has a default values entry:

networkPolicy:
  create: true

I changed this to False and it works now. Usually networkPolicy should be set to a default of False to avoid problems like this, and the admin can opt in if they require it.

Ideally the networkPolicy should also include a rule to allow the http01 solver, if the admin wishes to use it.

Comment


Terms & Conditions