Защищенный паролем Elasticsearch на Kubernetes вызывает ошибку "не удалось присоединиться к {asticsearch-master-x}" " - PullRequest
Новая
       24

Защищенный паролем Elasticsearch на Kubernetes вызывает ошибку "не удалось присоединиться к {asticsearch-master-x}" "

0 голосов
/ 14 апреля 2020

Развернуть Elasticsearch и Kibana было легко, используя официальный график руля. Теперь я хочу защитить его с помощью аутентификации. Это больше не должно быть проблемой, поскольку Система безопасности была открыта в мае 2019 года и доступна бесплатно . Поэтому я включил xpack security , а также добавил аутентификацию зашифрованного сертификата, поскольку это требование .

Файл yaml моих значений выглядит следующим образом: 

# Elastic repo blocked by proxy
image: "elasticsearch"
imageTag: "7.6.1"

ingress:
  enabled: true
  tls:
    - hosts:
      - elasticsearch-test.k8s.internal
  paths: ['/']
  hosts:
    - elasticsearch-test.k8s.internal

# Also possible with secrets: https://github.com/elastic/helm-charts/blob/master/elasticsearch/examples/config/values.yaml
extraEnvs:
  - name: ELASTIC_PASSWORD
    value: testpw
  - name: ELASTIC_USERNAME
    value: testuser

esConfig:
  elasticsearch.yml: |
    xpack.security.enabled: true
    xpack.security.transport.ssl.enabled: true
    xpack.security.transport.ssl.verification_mode: certificate
    xpack.security.transport.ssl.keystore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
    xpack.security.transport.ssl.truststore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12

secretMounts:
  - name: elastic-certificates
    secretName: elastic-certificates
    path: /usr/share/elasticsearch/config/certs

На основе примера в официальной диаграмме Хелма я создаю слегка измененный файл, который генерирует сертификаты и сохранить их как секреты Kubernetes: 

STACK_VERSION := 7.6.1
ELASTICSEARCH_IMAGE := elasticsearch:$(STACK_VERSION)
ELASTIC_USER := testuser
ELASTIC_PASSWORD := testpw
HELM_NAME := elasticsearch-test
NAMESPACE := elasticsearch-test

install:
    helm install --name $(HELM_NAME) elastic/elasticsearch --namespace $(NAMESPACE) -f elasticsearch-values.yml

secrets:
    docker rm -f elastic-helm-charts-certs || true
    rm -f elastic-certificates.p12 elastic-certificate.pem elastic-stack-ca.p12 || true

    docker run --name elastic-helm-charts-certs -i -w /app \
        $(ELASTICSEARCH_IMAGE) \
        /bin/sh -c " \
            elasticsearch-certutil ca --out /app/elastic-stack-ca.p12 --pass '' && \
            elasticsearch-certutil cert --name security-master --dns security-master --ca /app/elastic-stack-ca.p12 --pass '' --ca-pass '' --out /app/elastic-certificates.p12"
    docker cp elastic-helm-charts-certs:/app/elastic-certificates.p12 ./ && \
    docker rm -f elastic-helm-charts-certs
    openssl pkcs12 -nodes -passin pass:'' -in elastic-certificates.p12 -out elastic-certificate.pem && \
    kubectl create secret generic elastic-certificates --from-file=elastic-certificates.p12 && \
    kubectl create secret generic elastic-certificate-pem --from-file=elastic-certificate.pem && \
    kubectl create secret generic elastic-credentials  --from-literal=password=$(ELASTIC_PASSWORD) --from-literal=username=$(ELASTIC_USER)
    rm -f elastic-certificates.p12 elastic-certificate.pem elastic-stack-ca.p12

purge:
    kubectl delete secrets elastic-credentials elastic-certificates elastic-certificate-pem || true
    helm del --purge $(HELM_NAME)

clean:
    $(MAKE) purge

Но соединение между участником кластера Elasticsearch не удалось: 

{"type": "server", "timestamp": "2020-04-14T14:40:54,038Z", "level": "WARN", "component": "o.e.t.TransportService", "cluster.name": "elasticsearch", "node.name": "elasticsearch-master-0", "message": "Transport response handler not found of id [1]" }
{"type": "server", "timestamp": "2020-04-14T14:40:54,162Z", "level": "WARN", "component": "o.e.t.TransportService", "cluster.name": "elasticsearch", "node.name": "elasticsearch-master-0", "message": "Transport response handler not found of id [2]" }
{"type": "server", "timestamp": "2020-04-14T14:40:56,679Z", "level": "INFO", "component": "o.e.c.c.JoinHelper", "cluster.name": "elasticsearch", "node.name": "elasticsearch-master-0", "message": "failed to join {elasticsearch-master-2}{0frrBAvhS_Swe6RFqarh3A}{f4eYyU4ARKSz3kXvNKc4ig}{10.42.3.52}{10.42.3.52:9300}{dilm}{ml.machine_memory=2147483648, ml.max_open_jobs=20, xpack.installed=true} with JoinRequest{sourceNode={elasticsearch-master-0}{-F2BUfJLSyCggyDg7uo6pg}{N9KOsW7DSK-fvrj6nJIMEA}{10.42.5.194}{10.42.5.194:9300}{dilm}{ml.machine_memory=2147483648, xpack.installed=true, ml.max_open_jobs=20}, optionalJoin=Optional[Join{term=4, lastAcceptedTerm=3, lastAcceptedVersion=22, sourceNode={elasticsearch-master-0}{-F2BUfJLSyCggyDg7uo6pg}{N9KOsW7DSK-fvrj6nJIMEA}{10.42.5.194}{10.42.5.194:9300}{dilm}{ml.machine_memory=2147483648, xpack.installed=true, ml.max_open_jobs=20}, targetNode={elasticsearch-master-2}{0frrBAvhS_Swe6RFqarh3A}{f4eYyU4ARKSz3kXvNKc4ig}{10.42.3.52}{10.42.3.52:9300}{dilm}{ml.machine_memory=2147483648, ml.max_open_jobs=20, xpack.installed=true}}]}", 
"stacktrace": ["org.elasticsearch.transport.NodeNotConnectedException: [elasticsearch-master-2][10.42.3.52:9300] Node not connected",
"at org.elasticsearch.transport.ConnectionManager.getConnection(ConnectionManager.java:191) ~[elasticsearch-7.6.1.jar:7.6.1]",
"at org.elasticsearch.transport.TransportService.getConnection(TransportService.java:618) ~[elasticsearch-7.6.1.jar:7.6.1]",
"at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:590) [elasticsearch-7.6.1.jar:7.6.1]",
"at org.elasticsearch.cluster.coordination.JoinHelper.sendJoinRequest(JoinHelper.java:279) [elasticsearch-7.6.1.jar:7.6.1]",
"at org.elasticsearch.cluster.coordination.JoinHelper.sendJoinRequest(JoinHelper.java:212) [elasticsearch-7.6.1.jar:7.6.1]",
"at org.elasticsearch.cluster.coordination.JoinHelper.lambda$new$2(JoinHelper.java:136) [elasticsearch-7.6.1.jar:7.6.1]",
"at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler$1.doRun(SecurityServerTransportInterceptor.java:257) [x-pack-security-7.6.1.jar:7.6.1]",
"at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.6.1.jar:7.6.1]",
"at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:225) [elasticsearch-7.6.1.jar:7.6.1]",
"at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler.lambda$messageReceived$0(SecurityServerTransportInterceptor.java:306) [x-pack-security-7.6.1.jar:7.6.1]",
"at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63) [elasticsearch-7.6.1.jar:7.6.1]",
"at org.elasticsearch.xpack.security.authz.AuthorizationService.authorizeSystemUser(AuthorizationService.java:378) [x-pack-security-7.6.1.jar:7.6.1]",
"at org.elasticsearch.xpack.security.authz.AuthorizationService.authorize(AuthorizationService.java:186) [x-pack-security-7.6.1.jar:7.6.1]",
"at org.elasticsearch.xpack.security.transport.ServerTransportFilter$NodeProfile.lambda$inbound$1(ServerTransportFilter.java:130) [x-pack-security-7.6.1.jar:7.6.1]",
"at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63) [elasticsearch-7.6.1.jar:7.6.1]",
"at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$authenticateAsync$2(AuthenticationService.java:248) [x-pack-security-7.6.1.jar:7.6.1]",
"at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$lookForExistingAuthentication$6(AuthenticationService.java:310) [x-pack-security-7.6.1.jar:7.6.1]",
"at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lookForExistingAuthentication(AuthenticationService.java:321) [x-pack-security-7.6.1.jar:7.6.1]",
"at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.authenticateAsync(AuthenticationService.java:245) [x-pack-security-7.6.1.jar:7.6.1]",
"at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.access$000(AuthenticationService.java:196) [x-pack-security-7.6.1.jar:7.6.1]",
"at org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:139) [x-pack-security-7.6.1.jar:7.6.1]",
"at org.elasticsearch.xpack.security.transport.ServerTransportFilter$NodeProfile.inbound(ServerTransportFilter.java:121) [x-pack-security-7.6.1.jar:7.6.1]",
"at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler.messageReceived(SecurityServerTransportInterceptor.java:313) [x-pack-security-7.6.1.jar:7.6.1]",
"at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:63) [elasticsearch-7.6.1.jar:7.6.1]",
"at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:264) [elasticsearch-7.6.1.jar:7.6.1]",
"at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:692) [elasticsearch-7.6.1.jar:7.6.1]",
"at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.6.1.jar:7.6.1]",
"at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]",
"at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]",
"at java.lang.Thread.run(Thread.java:830) [?:?]"] }

То, что я уже пробовал

  • Настройка protocol: https
  • В esConfig.elasticsearch.yml добавлено следующее: 
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.truststore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
xpack.security.http.ssl.keystore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
  • Проверено, что сама безопасность xpack включена: curl 'http://localhost:9200/_cat/plugins?v&pretty' в одном модуле Elasticsearch выдает security_exception: missing authentication credentials for REST request [/_cat/plugins?v&pretty]

Docker images

Elasti c имеет собственный Docker реестр в docker.elastic.co/elasticsearch/elasticsearch. Но он заблокирован в нашей корпоративной сети, поэтому я попробовал официальный образ эластичного поиска на Docker хабе . Так как это официально и up2date, я предполагаю, что это те же изображения, что и на собственном репозитории Elastics. Я не смог найти информацию о различиях или других причинах, по которым официальные изображения хабов не должны использоваться.

...