Skip to main content
  • EN

Hi All,

 

I installed Kasten on K3s and if i open the dashboard it is working fine, the issue is svclb-gateway-ext POD is always in pending state: 

AME                                  READY   STATUS    RESTARTS   AGE
svclb-gateway-ext-mpsb5               0/1     Pending   0          2d1h
svclb-gateway-ext-rr2xt               0/1     Pending   0          2d1h
kanister-svc-c7bc74b9d-vx266          1/1     Running   0          2d1h
frontend-svc-b58b8b657-gbfv6          1/1     Running   0          2d1h
executor-svc-54d78ddcc7-xl8kg         2/2     Running   0          2d1h
executor-svc-54d78ddcc7-c6zgf         2/2     Running   0          2d1h
auth-svc-69fdd659dc-qmrf6             1/1     Running   0          2d1h
state-svc-74b44f8b69-9j68q            1/1     Running   0          2d1h
executor-svc-54d78ddcc7-5q8j8         2/2     Running   0          2d1h
config-svc-5d79f9c786-bd9bj           1/1     Running   0          2d1h
logging-svc-59455454cf-ts7b8          1/1     Running   0          2d1h
prometheus-server-75fcc94d5c-ltt2b    2/2     Running   0          2d1h
catalog-svc-5f58597479-c6mfg          2/2     Running   0          2d1h
k10-grafana-55cf89786-cgz9n           1/1     Running   0          2d1h
crypto-svc-5585dd8884-gqqpm           2/2     Running   0          2d1h
aggregatedapis-svc-59ff7cd849-nh27m   1/1     Running   0          2d1h
jobs-svc-7f995cbb74-klvjv             1/1     Running   0          2d1h
metering-svc-687659cf94-bt4w7         1/1     Running   0          2d1h
dashboardbff-svc-86546965d9-8gxsh     1/1     Running   1          2d1h
gateway-78659b78bf-r2h2b              1/1     Running   1          2d1h
svclb-gateway-ext-5b98g               0/1     Pending   0          69s
 

if Describe the POD i got the below error:

Events:
  Type     Reason            Age   From               Message
  ----     ------            ----  ----               -------
  Warning  FailedScheduling  38h   default-scheduler  0/3 nodes are available: 1 node(s) didn't have free ports for the requested pod ports, 2 node(s) didn't match Pod's node affinity/selector.

please your advise of anyone face like this issue

 

BR,

Ali      

 

@Aly Idriss Thanks for posting this question here.

svclb-gateway-ext-xxxxx is not a part of K10 helm chart.

 

I am not sure if there’s an external component creates those pods.

@jaiganeshjk, i can see a daemonset installed with kasten, 

NAME                               DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
daemonset.apps/svclb-gateway-ext   3         3         0       3            0           <none>          2d2h

 

# kubectl describe daemonset.apps/svclb-gateway-ext -n kasten-io
Name:           svclb-gateway-ext
Selector:       app=svclb-gateway-ext
Node-Selector:  <none>
Labels:         objectset.rio.cattle.io/hash=a0ef3b73da0b70b99b09dd18f63ff190f89953ce
                svccontroller.k3s.cattle.io/nodeselector=false
Annotations:    deprecated.daemonset.template.generation: 1
                objectset.rio.cattle.io/applied:
                  H4sIAAAAAAAA/5xUTW/jNhD9K8WcKUWOk9gS0MMiySFo1zFsby9BEIzIkc2aIgVypI1h6L8XVLxrp81HsUfNx9Pje4/cw1ZbBQXcINXOLolBADb6L/JBOwsFYNOEs24EAmpiVMgIxR...
                objectset.rio.cattle.io/id: svccontroller
                objectset.rio.cattle.io/owner-gvk: /v1, Kind=Service
                objectset.rio.cattle.io/owner-name: gateway-ext
                objectset.rio.cattle.io/owner-namespace: kasten-io
Desired Number of Nodes Scheduled: 3
Current Number of Nodes Scheduled: 3
Number of Nodes Scheduled with Up-to-date Pods: 3
Number of Nodes Scheduled with Available Pods: 0
Number of Nodes Misscheduled: 0
Pods Status:  0 Running / 3 Waiting / 0 Succeeded / 0 Failed
Pod Template:
  Labels:  app=svclb-gateway-ext
           svccontroller.k3s.cattle.io/svcname=gateway-ext
  Containers:
   lb-port-80:
    Image:      rancher/klipper-lb:v0.2.0
    Port:       80/TCP
    Host Port:  80/TCP
    Environment:
      SRC_PORT:    80
      DEST_PROTO:  TCP
      DEST_PORT:   80
      DEST_IP:     10.43.15.230
    Mounts:        <none>
  Volumes:         <none>
Events:            <none>
 

Yea @Aly Idriss ,

Since you mentioned you are trying out K10 in K3s, It might be something that is created by Rancher for any external loadBalancer service you create.

I see that they use something called `Klipper Loadbalancer` for LoadBalancer services in K3s which uses hostPort to expose the service.

All your 3 worker nodes might have already been using port 80, and it might be the reason for the pods staying in pending state.

https://rancher.com/docs/k3s/latest/en/networking/#service-load-balancer

Thanks @jaiganeshjk , it is very wierd  because i installed K3s without traefik and why it is appearing under Kasten namespace not default or kube-syste.

since I installed metallb-system as external LB, do you think it is safe to remove K3s LB service.

BR,

Ali

Yes It is safe to remove it. You might have to disable ServiceLB along with the Traefik while installing K3s.

It seems that if you have a Service with type LoadBalancer, this serviceLB controller spins up a daemonSet for this.

https://github.com/k3s-io/k3s/issues/2323#issuecomment-701218153

thank you very much @jaiganeshjk 

Comment


Terms & Conditions