• EN
Solved

ImagePullBackOff ghcr.io/kanisterio/kanister-kubectl

  • 5 October 2023
  • 4 comments
  • 115 views
B
Userlevel 1

Hello,

I have installed Kasten v6.0.7 with Helm Chart.
I use the air gap installation for pull images in local repository with this config :

global:
  image:
    registry: <private_repository>:<port>
    repository: kasten-images
    pullPolicy: IfNotPresent
  airgapped:
    repository: <private_repository>:<port>/kasten-images
    pullPolicy: IfNotPresent

and

kanisterToolsImage:
  registry: <private_repository>:<port>
  repository: kanisterio
  image: kanister-tools
  pullPolicy: IfNotPresent

 

It works for all images exept one : ghcr.io/kanisterio/kanister-kubectl-1.18
I have this error events :

55s (x4 over 2m23s)    Normal    Pulling          Pod/kanister-job-bzts6                                                      Pulling image "ghcr.io/kanisterio/kanister-kubectl-1.18:0.96.0"
46s (x4 over 2m15s)    Normal    Pulling          Pod/kanister-job-fwwp7                                                      Pulling image "ghcr.io/kanisterio/kanister-kubectl-1.18:0.96.0"
46s (x6 over 2m27s)    Warning   Failed           Pod/kanister-job-mg8c4                                                      Error: ImagePullBackOff
46s                    Warning   Failed           Pod/kanister-job-fwwp7                                                      Failed to pull image "ghcr.io/kanisterio/kanister-kubectl-1.18:0.96.0": rpc error: code = Unknown desc = failed to pull and unpack image "ghcr.io/kanisterio/kanister-kubectl-1.18:0.96.0": failed to resolve reference "ghcr.io/kanisterio/kanister-kubectl-1.18:0.96.0": failed to do request: Head "https://ghcr.io/v2/kanisterio/kanister-kubectl-1.18/manifests/0.96.0": dial tcp 140.82.121.34:443: connect: connection refused
...
3m26s                  Warning   ActionSetFailed Action: backup   Blueprint/k10-pgo-bp-0.0.3                                                  Failed to execute phase: v1alpha1.Phase{Name:"pgoBackup", State:"pending", Output:map[string]interface {}(nil)}: Failed while waiting for Pod kanister-job-wb8vq to be ready: Pod failed to become ready in time: Failed to fetch logs from the pod: container "container" in pod "kanister-job-wb8vq" is waiting to start: trying and failing to pull image
3m26s                  Warning   ActionSetFailed Action: backup   ActionSet/k10-backup-k10-pgo-bp-0.0.3-postgres-storage5gh5   Failed to execute phase: v1alpha1.Phase{Name:"pgoBackup", State:"pending", Output:map[string]interface {}(nil)}: Failed while waiting for Pod kanister-job-wb8vq to be ready: Pod failed to become ready in time: Failed to fetch logs from the pod: container "container" in pod "kanister-job-wb8vq" is waiting to start: trying and failing to pull image
3m24s                  Warning   ActionSetFailed Action: backup   Blueprint/k10-pgo-bp-0.0.3                                                  Failed to execute phase: v1alpha1.Phase{Name:"pgoBackup", State:"pending", Output:map[string]interface {}(nil)}: Failed while waiting for Pod kanister-job-6qkb9 to be ready: Pod failed to become ready in time: Pod did not transition into running state. Timeout:15m0s  Namespace:kasten-io, Name:kanister-job-6qkb9: client rate limiter Wait returned an error: context deadline exceeded


Is there another place to change the ghcr.io repo to my local repo ?

 


Thanks

Regards

icon

Best answer by Hagag 5 October 2023, 22:31

View original

4 comments

Hagag
Userlevel 5
Badge +2

@bryan.audo-ext 
 

By default, the k10tools command will use the publicly available kanister-tools image at ghcr.io/kanisterio/kanister-tools:<Kanister version>. Since this image is not available in air-gapped environments, to override the default image, set the KANISTER_TOOLS environment variable to the kanister-tools image that is available in the air-gapped environment's local registry. The kanister-tools version can be found by running this command in a shell that has access to the public internet and with the appropriate K10 version. docker run --rm -it gcr.io/kasten-images/k10offline:<K10 version> list-images | grep kanister-tools.

Example:

export KANISTER_TOOLS=<your local registry>/<your local repository name>/kanister-tools:k10-<Kanister version>

B
Userlevel 1

Hello @Hagag 

 

Thank you for your reply.

 

The problem is not kasten-images/kanister-tools.
The KANISTER_TOOLS variable is already overloaded in the kasten-io deployment via the k10-config configmap :

deployment kanister-svc:

...
- name: KANISTER_TOOLS
  valueFrom:
    configMapKeyRef:
      key: overwriteKanisterTools
      name: k10-config
...

cm k10-config:

...
overwriteKanisterTools: <private_repository>:<port>/kasten-images/kanister-tools:k10-0.95.0
...

 

It's kanisterio/kanister-kubectl which always use ghcr.io by default.

I tried adding a environment variable KANISTER_KUBECTL to the kasten-io deployment, but with no success 🙂

Is there an environment variable for kanister-kubectl ?


Regards

Hagag
Userlevel 5
Badge +2

@bryan.audo-ext  

I notice that you're employing PGOBackup is that correct ?
k10 is generating a blueprint for this backup type within the kasten-io namespace. and you might need to modify the image repository and tag within the blueprint.

Could you please provide a list of blueprints within the kasten-io namespace? You can achieve this by using either of the following commands:

kubectl get blueprints -n kasten-io
or

kubectl get blueprints.cr.kanister.io -n kasten-io


 

B
Userlevel 1

Hello @Hagag,

Yes, it works now ! It was indeed on the CRD blueprints that the ghcr.io repo was declared.
I modified it via kubectl edit blueprints -n kasten-io k10-pgo-bp-0.0.3 and replaced the image: ghcr.io with my local repo.

Perhaps to be modified when creating the CRD if air gap mode is used in a future version ?

 

Thank's for your help.


Regards

Comment


Terms & Conditions