Я получаю это сообщение об ошибке на кубернетах, когда пытаюсь вытащить изображение:
0s Normal Pulling pod/zeppelin-server-6f5646f56d-6slj2 Pulling image "apache/zeppelin:0.9.0"
0s Warning Failed pod/zeppelin-server-6f5646f56d-6slj2 Failed to pull image "apache/zeppelin:0.9.0": rpc error: code = Unknown desc = failed to pull and unpack image "docker.io/apache/zeppelin:0.9.0": failed to copy: httpReaderSeeker: failed open: server message: invalid_token: authorization failed
0s Warning Failed pod/zeppelin-server-6f5646f56d-6slj2 Error: ErrImagePull
Здесь kubectl get events -w:
<unknown> Normal Scheduled pod/zeppelin Successfully assigned zeppelin/zeppelin to k3d-k3s-linkerd-server
47s Normal Pulling pod/zeppelin Pulling image "apache/zeppelin:0.9.0"
19s Warning Failed pod/zeppelin-server-6f5646f56d-6slj2 Failed to pull image "apache/zeppelin:0.9.0": rpc error: code = Unknown desc = failed to pull and unpack image "docker.io/apache/zeppelin:0.9.0": failed to copy: httpReaderSeeker: failed open: server message: invalid_token: authorization failed
19s Warning Failed pod/zeppelin-server-6f5646f56d-6slj2 Error: ErrImagePull
С другой стороны:
$ kubectl run zeppelin --image=apache/zeppelin:0.9.0 --restart=Never -v=10
I0531 12:13:29.882735 18907 loader.go:375] Config loaded from file: /home/jeusdi/.config/k3d/k3s-linkerd/kubeconfig.yaml
I0531 12:13:29.884168 18907 cached_discovery.go:114] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/servergroups.json
I0531 12:13:29.884623 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/autoscaling/v1/serverresources.json
I0531 12:13:29.884623 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/autoscaling/v2beta1/serverresources.json
I0531 12:13:29.884633 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/authentication.k8s.io/v1/serverresources.json
I0531 12:13:29.884693 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/storage.k8s.io/v1/serverresources.json
I0531 12:13:29.884716 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/events.k8s.io/v1beta1/serverresources.json
I0531 12:13:29.884728 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/linkerd.io/v1alpha2/serverresources.json
I0531 12:13:29.884773 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/authorization.k8s.io/v1beta1/serverresources.json
I0531 12:13:29.884789 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/scheduling.k8s.io/v1/serverresources.json
I0531 12:13:29.884799 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/k3s.cattle.io/v1/serverresources.json
I0531 12:13:29.884826 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/v1/serverresources.json
I0531 12:13:29.884858 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/admissionregistration.k8s.io/v1beta1/serverresources.json
I0531 12:13:29.884863 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/apps/v1/serverresources.json
I0531 12:13:29.884868 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/apiextensions.k8s.io/v1/serverresources.json
I0531 12:13:29.884894 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/caching.internal.knative.dev/v1alpha1/serverresources.json
I0531 12:13:29.884922 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/apiextensions.k8s.io/v1beta1/serverresources.json
I0531 12:13:29.884929 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/batch/v1/serverresources.json
I0531 12:13:29.884922 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/apiregistration.k8s.io/v1/serverresources.json
I0531 12:13:29.884958 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/storage.k8s.io/v1beta1/serverresources.json
I0531 12:13:29.884979 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/networking.k8s.io/v1beta1/serverresources.json
I0531 12:13:29.884988 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/policy/v1beta1/serverresources.json
I0531 12:13:29.885002 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/apiregistration.k8s.io/v1beta1/serverresources.json
I0531 12:13:29.885008 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/flagger.app/v1beta1/serverresources.json
I0531 12:13:29.885025 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/batch/v1beta1/serverresources.json
I0531 12:13:29.885037 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/admissionregistration.k8s.io/v1/serverresources.json
I0531 12:13:29.885043 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/rbac.authorization.k8s.io/v1/serverresources.json
I0531 12:13:29.885069 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/extensions/v1beta1/serverresources.json
I0531 12:13:29.885076 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/certificates.k8s.io/v1beta1/serverresources.json
I0531 12:13:29.885085 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/flagger.app/v1alpha3/serverresources.json
I0531 12:13:29.885109 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/rbac.authorization.k8s.io/v1beta1/serverresources.json
I0531 12:13:29.885121 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/networking.k8s.io/v1/serverresources.json
I0531 12:13:29.885152 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/coordination.k8s.io/v1beta1/serverresources.json
I0531 12:13:29.885153 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/flagger.app/v1alpha1/serverresources.json
I0531 12:13:29.885182 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/linkerd.io/v1alpha1/serverresources.json
I0531 12:13:29.885182 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/tap.linkerd.io/v1alpha1/serverresources.json
I0531 12:13:29.885222 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/autoscaling/v2beta2/serverresources.json
I0531 12:13:29.885224 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/scheduling.k8s.io/v1beta1/serverresources.json
I0531 12:13:29.885247 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/split.smi-spec.io/v1alpha1/serverresources.json
I0531 12:13:29.885255 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/node.k8s.io/v1beta1/serverresources.json
I0531 12:13:29.885273 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/authentication.k8s.io/v1beta1/serverresources.json
I0531 12:13:29.885297 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/coordination.k8s.io/v1/serverresources.json
I0531 12:13:29.885310 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/discovery.k8s.io/v1beta1/serverresources.json
I0531 12:13:29.885311 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/tekton.dev/v1alpha1/serverresources.json
I0531 12:13:29.885363 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/traefik.containo.us/v1alpha1/serverresources.json
I0531 12:13:29.885365 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/tekton.dev/v1beta1/serverresources.json
I0531 12:13:29.885381 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/metrics.k8s.io/v1beta1/serverresources.json
I0531 12:13:29.885478 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/authorization.k8s.io/v1/serverresources.json
I0531 12:13:29.885705 18907 cached_discovery.go:71] returning cached discovery info from /home/jeusdi/.kube/cache/discovery/localhost_6443/helm.cattle.io/v1/serverresources.json
I0531 12:13:29.889299 18907 request.go:1068] Request Body: {"kind":"Pod","apiVersion":"v1","metadata":{"name":"zeppelin","creationTimestamp":null,"labels":{"run":"zeppelin"}},"spec":{"containers":[{"name":"zeppelin","image":"apache/zeppelin:0.9.0","resources":{}}],"restartPolicy":"Never","dnsPolicy":"ClusterFirst"},"status":{}}
I0531 12:13:29.889359 18907 round_trippers.go:423] curl -k -v -XPOST -H "Content-Type: application/json" -H "User-Agent: kubectl/v1.18.3 (linux/amd64) kubernetes/2e7996e" -H "Accept: application/json, */*" -H "Authorization: Basic YWRtaW46NjA4OGZhYmEzNjc3OGFkMGJhNzJiMzhiNzJjZTE5NWQ=" 'https://localhost:6443/api/v1/namespaces/zeppelin/pods'
I0531 12:13:29.912123 18907 round_trippers.go:443] POST https://localhost:6443/api/v1/namespaces/zeppelin/pods 201 Created in 22 milliseconds
I0531 12:13:29.912223 18907 round_trippers.go:449] Response Headers:
I0531 12:13:29.912233 18907 round_trippers.go:452] Content-Length: 1241
I0531 12:13:29.912244 18907 round_trippers.go:452] Content-Type: application/json
I0531 12:13:29.912250 18907 round_trippers.go:452] Date: Sun, 31 May 2020 10:13:29 GMT
I0531 12:13:29.912324 18907 request.go:1068] Response Body: {"kind":"Pod","apiVersion":"v1","metadata":{"name":"zeppelin","namespace":"zeppelin","selfLink":"/api/v1/namespaces/zeppelin/pods/zeppelin","uid":"a01e2b82-2c47-4424-b1e0-d6bedcfe2d50","resourceVersion":"365387","creationTimestamp":"2020-05-31T10:13:29Z","labels":{"run":"zeppelin"}},"spec":{"volumes":[{"name":"default-token-49tq6","secret":{"secretName":"default-token-49tq6","defaultMode":420}}],"containers":[{"name":"zeppelin","image":"apache/zeppelin:0.9.0","resources":{},"volumeMounts":[{"name":"default-token-49tq6","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Never","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}],"priority":0,"enableServiceLinks":true},"status":{"phase":"Pending","qosClass":"BestEffort"}}
pod/zeppelin created
Проблема только с apache/zeppelin. Я развернул другие модули:
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
spark-master-0 1/1 Running 1 41h
spark-worker-1 1/1 Running 1 41h
spark-worker-0 1/1 Running 1 41h
zeppelin 0/1 ContainerCreating 0 5m14s