Запуск стабильной / рудной карты Helm Chart на Kubernetes 1.14 на EKS Fails Liveness Probe - PullRequest
1 голос
/ 11 октября 2019

Я пытаюсь развернуть Vanilla MineCraft Server с stable / minecraft , используя Helm на Kubernetes 1.14, работающем на AWS EKS, но я либо получаю CrashLoopBackOff, либо Liveness Probe Failures. Это кажется мне странным, поскольку я развертываю диаграмму, как указано в документации:

helm install --name mine-release --set minecraftServer.eula=TRUE --namespace=mine-release stable/minecraft

Уже предпринята попытка отладки:

  1. Попытка уменьшения и увеличения памяти helm install --name mine-release --set resources.requests.memory="1024Mi" --set minecraftServer.memory="1024M" --set minecraftServer.eula=TRUE --namespace=mine-release stable/minecraft
  2. Пробовал просматривать журналы через kubectl logs mine-release-minecraft-56f9c8588-xn9pv --namespace mine-release, но эта ошибка всегда появляется
Error from server: Get https://10.0.143.216:10250/containerLogs/mine-release/mine-release-minecraft-56f9c8588-xn9pv/mine-release-minecraft: dial tcp 10.0.143.216:10250: i/o timeout

Чтобы получить больше контекста, вывод kubectl describe pods mine-release-minecraft-56f9c8588-xn9pv --namespace mine-release для описания модуля и событий приведен ниже:

Name:               mine-release-minecraft-56f9c8588-xn9pv
Namespace:          mine-release
Priority:           0
PriorityClassName:  <none>
Node:               ip-10-0-143-216.ap-southeast-2.compute.internal/10.0.143.216
Start Time:         Fri, 11 Oct 2019 08:48:34 +1100
Labels:             app=mine-release-minecraft
                    pod-template-hash=56f9c8588
Annotations:        kubernetes.io/psp: eks.privileged
Status:             Running
IP:                 10.0.187.192
Controlled By:      ReplicaSet/mine-release-minecraft-56f9c8588
Containers:
  mine-release-minecraft:
    Container ID:   docker://893f622e1129937fab38dc902e25e95ac86c2058da75337184f105848fef773f
    Image:          itzg/minecraft-server:latest
    Image ID:       docker-pullable://itzg/minecraft-server@sha256:00f592eb6660682f327770d639cf10692b9617fa8b9a764b9f991c401e325105
    Port:           25565/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Fri, 11 Oct 2019 08:50:56 +1100
    Last State:     Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Fri, 11 Oct 2019 08:50:03 +1100
      Finished:     Fri, 11 Oct 2019 08:50:53 +1100
    Ready:          False
    Restart Count:  2
    Requests:
      cpu:      500m
      memory:   1Gi
    Liveness:   exec [mcstatus localhost:25565 status] delay=30s timeout=1s period=5s #success=1 #failure=3
    Readiness:  exec [mcstatus localhost:25565 status] delay=30s timeout=1s period=5s #success=1 #failure=3
    Environment:
      EULA:                          true
      TYPE:                          VANILLA
      VERSION:                       1.14.4
      DIFFICULTY:                    easy
      WHITELIST:                     
      OPS:                           
      ICON:                          
      MAX_PLAYERS:                   20
      MAX_WORLD_SIZE:                10000
      ALLOW_NETHER:                  true
      ANNOUNCE_PLAYER_ACHIEVEMENTS:  true
      ENABLE_COMMAND_BLOCK:          true
      FORCE_gameMode:                false
      GENERATE_STRUCTURES:           true
      HARDCORE:                      false
      MAX_BUILD_HEIGHT:              256
      MAX_TICK_TIME:                 60000
      SPAWN_ANIMALS:                 true
      SPAWN_MONSTERS:                true
      SPAWN_NPCS:                    true
      VIEW_DISTANCE:                 10
      SEED:                          
      MODE:                          survival
      MOTD:                          Welcome to Minecraft on Kubernetes!
      PVP:                           false
      LEVEL_TYPE:                    DEFAULT
      GENERATOR_SETTINGS:            
      LEVEL:                         world
      ONLINE_MODE:                   true
      MEMORY:                        512M
      JVM_OPTS:                      
      JVM_XX_OPTS:                   
    Mounts:
      /data from datadir (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-j8zql (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  datadir:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  mine-release-minecraft-datadir
    ReadOnly:   false
  default-token-j8zql:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-j8zql
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason                  Age                    From                                                      Message
  ----     ------                  ----                   ----                                                      -------
  Warning  FailedScheduling        2m25s                  default-scheduler                                         pod has unbound immediate PersistentVolumeClaims (repeated 3 times)
  Normal   Scheduled               2m24s                  default-scheduler                                         Successfully assigned mine-release/mine-release-minecraft-56f9c8588-xn9pv to ip-10-0-143-216.ap-southeast-2.compute.internal
  Warning  FailedAttachVolume      2m22s (x3 over 2m23s)  attachdetach-controller                                   AttachVolume.Attach failed for volume "pvc-b48ba754-eba7-11e9-b609-02ed13ff0a10" : "Error attaching EBS volume \"vol-08b29bb4eeca4df56\"" to instance "i-00ae1f5b96eed8e6a" since volume is in "creating" state
  Normal   SuccessfulAttachVolume  2m18s                  attachdetach-controller                                   AttachVolume.Attach succeeded for volume "pvc-b48ba754-eba7-11e9-b609-02ed13ff0a10"
  Warning  Unhealthy               60s                    kubelet, ip-10-0-143-216.ap-southeast-2.compute.internal  Readiness probe failed: Traceback (most recent call last):
  File "/usr/bin/mcstatus", line 11, in <module>
    sys.exit(cli())
  File "/usr/lib/python2.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python2.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python2.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python2.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/mcstatus/scripts/mcstatus.py", line 58, in status
    response = server.status()
  File "/usr/lib/python2.7/site-packages/mcstatus/server.py", line 49, in status
    connection = TCPSocketConnection((self.host, self.port))
  File "/usr/lib/python2.7/site-packages/mcstatus/protocol/connection.py", line 129, in __init__
    self.socket = socket.create_connection(addr, timeout=timeout)
  File "/usr/lib/python2.7/socket.py", line 575, in create_connection
    raise err
socket.error: [Errno 99] Address not available
  Normal   Pulling    58s (x2 over 2m14s)  kubelet, ip-10-0-143-216.ap-southeast-2.compute.internal  pulling image "itzg/minecraft-server:latest"
  Normal   Killing    58s                  kubelet, ip-10-0-143-216.ap-southeast-2.compute.internal  Killing container with id docker://mine-release-minecraft:Container failed liveness probe.. Container will be killed and recreated.
  Normal   Started    55s (x2 over 2m11s)  kubelet, ip-10-0-143-216.ap-southeast-2.compute.internal  Started container
  Normal   Pulled     55s (x2 over 2m11s)  kubelet, ip-10-0-143-216.ap-southeast-2.compute.internal  Successfully pulled image "itzg/minecraft-server:latest"
  Normal   Created    55s (x2 over 2m11s)  kubelet, ip-10-0-143-216.ap-southeast-2.compute.internal  Created container
  Warning  Unhealthy  25s (x2 over 100s)   kubelet, ip-10-0-143-216.ap-southeast-2.compute.internal  Readiness probe failed: Traceback (most recent call last):
  File "/usr/bin/mcstatus", line 11, in <module>
    sys.exit(cli())
  File "/usr/lib/python2.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python2.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python2.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python2.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/mcstatus/scripts/mcstatus.py", line 58, in status
    response = server.status()
  File "/usr/lib/python2.7/site-packages/mcstatus/server.py", line 61, in status
    raise exception
socket.error: [Errno 104] Connection reset by peer
  Warning  Unhealthy  20s (x8 over 95s)  kubelet, ip-10-0-143-216.ap-southeast-2.compute.internal  Readiness probe failed:
  Warning  Unhealthy  17s (x5 over 97s)  kubelet, ip-10-0-143-216.ap-southeast-2.compute.internal  Liveness probe failed:

Я немного подробнее о своей настройке Kubernetes:

Kubernetes версии 1.14 и узлах, работающих на m5.larges

1 Ответ

1 голос
/ 11 октября 2019

Я сделал репродукцию вашей проблемы, и ответ: Проверка готовности и живучести.

Ваша диаграмма не успевает встать, поэтому после того, как тест готовности вернет false, тест на живучесть убьети попробуйте сделать это снова и снова.

livenessProbe: указывает, работает ли контейнер. В случае сбоя проверки работоспособности кублет убивает Контейнер, и Контейнер подвергается политике перезапуска. Если Контейнер не предоставляет проверку живучести, по умолчанию используется состояние Успех.

readinessProbe: Указывает, готов ли Контейнер к запросам на обслуживание. В случае сбоя проверки готовности контроллер конечных точек удаляет IP-адрес модуля из конечных точек всех служб, соответствующих модулю. Состояние готовности по умолчанию до начальной задержки - Отказ. Если Контейнер не обеспечивает проверку готовности, состояние по умолчанию - Успех.

Вы можете использовать вашу команду после моего редактирования

helm install --name mine-release --set resources.requests.memory="1024Mi" --set minecraftServer.memory="1024M" --set minecraftServer.eula=TRUE --set livenessProbe.initialDelaySeconds=90 --set livenessProbe.periodSeconds=15 --set readinessProbe.initialDelaySeconds=90 --set readinessprobe.periodSeconds=15 --namespace=mine-release stable/minecraft

ИЛИ

Используйте helm fetch для загрузки диаграммы управления на свой компьютер

helm fetch stable/minecraft --untar 

вместо изменения значений в команде установки helm, вы можете использовать текстовый редактор, например vi или nano, и обновлять все в minecraft / values.yaml

vi/nano ./minecraft/values.yaml

файл minecraft / values.yaml после редактирования

# ref: https://hub.docker.com/r/itzg/minecraft-server/
image: itzg/minecraft-server
imageTag: latest

## Configure resource requests and limits
## ref: http://kubernetes.io/docs/user-guide/compute-resources/
##
resources:
  requests:
    memory: 1024Mi
    cpu: 500m

nodeSelector: {}

tolerations: []

affinity: {}

securityContext:
  # Security context settings
  runAsUser: 1000
  fsGroup: 2000
# Most of these map to environment variables. See Minecraft for details:
# https://hub.docker.com/r/itzg/minecraft-server/
livenessProbe:
  command:
    - mcstatus
    - localhost:25565
    - status
  initialDelaySeconds: 90
  periodSeconds: 15
readinessProbe:
  command:
    - mcstatus
    - localhost:25565
    - status
  initialDelaySeconds: 90
  periodSeconds: 15
minecraftServer:
  # This must be overridden, since we can't accept this for the user.
  eula: "TRUE"
  # One of: LATEST, SNAPSHOT, or a specific version (ie: "1.7.9").
  version: "1.14.4"
  # This can be one of "VANILLA", "FORGE", "SPIGOT", "BUKKIT", "PAPER", "FTB", "SPONGEVANILLA"
  type: "VANILLA"
  # If type is set to FORGE, this sets the version; this is ignored if forgeInstallerUrl is set
  forgeVersion:
  # If type is set to SPONGEVANILLA, this sets the version
  spongeVersion:
  # If type is set to FORGE, this sets the URL to download the Forge installer
  forgeInstallerUrl:
  # If type is set to BUKKIT, this sets the URL to download the Bukkit package
  bukkitDownloadUrl:
  # If type is set to SPIGOT, this sets the URL to download the Spigot package
  spigotDownloadUrl:
  # If type is set to PAPER, this sets the URL to download the PaperSpigot package
  paperDownloadUrl:
  # If type is set to FTB, this sets the server mod to run
  ftbServerMod:
  # Set to true if running Feed The Beast and get an error like "unable to launch forgemodloader"
  ftbLegacyJavaFixer: false
  # One of: peaceful, easy, normal, and hard
  difficulty: easy
  # A comma-separated list of player names to whitelist.
  whitelist:
  # A comma-separated list of player names who should be admins.
  ops:
  # A server icon URL for server listings. Auto-scaled and transcoded.
  icon:
  # Max connected players.
  maxPlayers: 20
  # This sets the maximum possible size in blocks, expressed as a radius, that the world border can obtain.
  maxWorldSize: 10000
  # Allows players to travel to the Nether.
  allowNether: true
  # Allows server to announce when a player gets an achievement.
  announcePlayerAchievements: true
  # Enables command blocks.
  enableCommandBlock: true
  # If true, players will always join in the default gameMode even if they were previously set to something else.
  forcegameMode: false
  # Defines whether structures (such as villages) will be generated.
  generateStructures: true
  # If set to true, players will be set to spectator mode if they die.
  hardcore: false
  # The maximum height in which building is allowed.
  maxBuildHeight: 256
  # The maximum number of milliseconds a single tick may take before the server watchdog stops the server with the message. -1 disables this entirely.
  maxTickTime: 60000
  # Determines if animals will be able to spawn.
  spawnAnimals: true
  # Determines if monsters will be spawned.
  spawnMonsters: true
  # Determines if villagers will be spawned.
  spawnNPCs: true
  # Max view distance (in chunks).
  viewDistance: 10
  # Define this if you want a specific map generation seed.
  levelSeed:
  # One of: creative, survival, adventure, spectator
  gameMode: survival
  # Message of the Day
  motd: "Welcome to Minecraft on Kubernetes!"
  # If true, enable player-vs-player damage.
  pvp: false
  # One of: DEFAULT, FLAT, LARGEBIOMES, AMPLIFIED, CUSTOMIZED
  levelType: DEFAULT
  # When levelType == FLAT or CUSTOMIZED, this can be used to further customize map generation.
  # ref: https://hub.docker.com/r/itzg/minecraft-server/
  generatorSettings:
  worldSaveName: world
  # If set, this URL will be downloaded at startup and used as a starting point
  downloadWorldUrl:
  # force re-download of server file
  forceReDownload: false
  # If set, the modpack at this URL will be downloaded at startup
  downloadModpackUrl:
  # If true, old versions of downloaded mods will be replaced with new ones from downloadModpackUrl
  removeOldMods: false
  # Check accounts against Minecraft account service.
  onlineMode: true
  # If you adjust this, you may need to adjust resources.requests above to match.
  memory: 1024M
  # General JVM options to be passed to the Minecraft server invocation
  jvmOpts: ""
  # Options like -X that need to proceed general JVM options
  jvmXXOpts: ""
  serviceType: LoadBalancer
  rcon:
    # If you enable this, make SURE to change your password below.
    enabled: false
    port: 25575
    password: "CHANGEME!"
    serviceType: LoadBalancer

  query:
    # If you enable this, your server will be "published" to Gamespy
    enabled: false
    port: 25565

## Additional minecraft container environment variables
##
extraEnv: {}

persistence:
  ## minecraft data Persistent Volume Storage Class
  ## If defined, storageClassName: <storageClass>
  ## If set to "-", storageClassName: "", which disables dynamic provisioning
  ## If undefined (the default) or set to null, no storageClassName spec is
  ##   set, choosing the default provisioner.  (gp2 on AWS, standard on
  ##   GKE, AWS & OpenStack)
  ##
  # storageClass: "-"
  dataDir:
    # Set this to false if you don't care to persist state between restarts.
    enabled: true
    Size: 1Gi

podAnnotations: {}

Затем мы используем helm install

helm install --name mine-release --namespace=mine-release ./minecraft -f ./minecraft/values.yaml

Результатыиз helm install:

NAME:   mine-release
LAST DEPLOYED: Fri Oct 11 14:52:17 2019
NAMESPACE: mine-release
STATUS: DEPLOYED

RESOURCES:
==> v1/PersistentVolumeClaim
NAME                            STATUS   VOLUME    CAPACITY  ACCESS MODES  STORAGECLASS  AGE
mine-release-minecraft-datadir  Pending  standard  0s

==> v1/Pod(related)
NAME                                    READY  STATUS   RESTARTS  AGE
mine-release-minecraft-f4558bfd5-mwm55  0/1    Pending  0         0s

==> v1/Secret
NAME                    TYPE    DATA  AGE
mine-release-minecraft  Opaque  1     0s

==> v1/Service
NAME                    TYPE          CLUSTER-IP   EXTERNAL-IP  PORT(S)          AGE
mine-release-minecraft  LoadBalancer  10.0.13.180  <pending>    25565:32020/TCP  0s

==> v1beta1/Deployment
NAME                    READY  UP-TO-DATE  AVAILABLE  AGE
mine-release-minecraft  0/1    1           0          0s


NOTES:
Get the IP address of your Minecraft server by running these commands in the
same shell:

!! NOTE: It may take a few minutes for the LoadBalancer IP to be available. !!

You can watch for EXTERNAL-IP to populate by running:
  kubectl get svc --namespace mine-release -w mine-release-minecraft

Результаты из логов:

[12:53:45] [Server-Worker-1/INFO]: Preparing spawn area: 98%
[12:53:45] [Server thread/INFO]: Time elapsed: 26661 ms
[12:53:45] [Server thread/INFO]: Done (66.833s)! For help, type "help"
[12:53:45] [Server thread/INFO]: Starting remote control listener
[12:53:45] [RCON Listener #1/INFO]: RCON running on 0.0.0.0:25575
...