Я создал mon go -statefulset в kubernetes, и все 3 модуля работают,
NAME READY STATUS RESTARTS AGE
mongo-0 1/1 Running 0 40m
mongo-1 1/1 Running 0 40m
mongo-2 1/1 Running 0 40m
mysql-5456cbb767-t8g2g 1/1 Running 0 3h45m
nfs-client-provisioner-5d77dc5bd-mcz8p 1/1 Running 0 42m
, однако после инициализации и перенастройки первого члена набора реплик с правильным DNS-именем может не добавлять оставшиеся наборы реплик.
>rs.initiate()
>var cfg = rs.conf()
>cfg.members[0].host="mongo‑0.mongo:27017"
>rs.reconfig(cfg)
Добавление второго члена дает мне эту ошибку
rs0:PRIMARY> rs.add("mongo‑1.mongo:27017")
{
"operationTime" : Timestamp(1590453966, 1),
"ok" : 0,
"errmsg" : "Quorum check failed because not enough voting nodes responded; required 2 but only the following 1 voting nodes responded: mongo-0:27017; the following nodes did not respond affirmatively: mongo‑1.mongo:27017 failed with Error connecting to mongo‑1.mongo:27017 :: caused by :: Could not find address for mongo‑1.mongo:27017: SocketException: Host not found (authoritative)",
"code" : 74,
"codeName" : "NodeNotFound",
"$clusterTime" : {
"clusterTime" : Timestamp(1590453966, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
}
rs0:PRIMARY>
Также использование hostname -f на всех узлах возвращает правильное имя хоста узлов. например.
ubuntu@k8s-master:~$ hostname -f
k8s-master
Есть идеи, как решить эту проблему?
Сервисы развернуты
ubuntu@k8s-master:~$ kubectl get svc --all-namespaces=true -o wide
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
default kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 20h <none>
default mongo ClusterIP None <none> 27017/TCP 7h59m app=mongo
default mysql ClusterIP None <none> 3306/TCP 11h app=mysql
default nrf-instance-service NodePort 10.96.18.198 <none> 9090:30005/TCP 10h app=nrf-instance
kube-system kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 20h k8s-app=kube-dns
kube-system metrics-server ClusterIP 10.111.247.62 <none> 443/TCP 11h k8s-app=metrics-server
kubernetes-dashboard dashboard-metrics-scraper ClusterIP 10.97.205.147 <none> 8000/TCP 20h k8s-app=dashboard-metrics-scraper
kubernetes-dashboard kubernetes-dashboard NodePort 10.100.212.222 <none> 443:32648/TCP 20h k8s-app=kubernetes-dashboard
Также DNS проверяется каждый месяц go pod
ubuntu@k8s-master:~$ kubectl exec -ti mongo-0 -- cat /etc/resolv.conf
nameserver 10.96.0.10
search default.svc.cluster.local svc.cluster.local cluster.local
options ndots:5
ubuntu@k8s-master:~$ kubectl exec -ti mongo-1 -- cat /etc/resolv.conf
nameserver 10.96.0.10
search default.svc.cluster.local svc.cluster.local cluster.local
options ndots:5
ubuntu@k8s-master:~$ kubectl exec -ti mongo-2 -- cat /etc/resolv.conf
nameserver 10.96.0.10
search default.svc.cluster.local svc.cluster.local cluster.local
options ndots:5
ubuntu@k8s-master:~$ kubectl exec -ti mongo-0 -- nslookup kubernetes.default
Server: 10.96.0.10
Address: 10.96.0.10#53
Name: kubernetes.default.svc.cluster.local
Address: 10.96.0.1
ubuntu@k8s-master:~$ kubectl exec -ti mongo-1 -- nslookup kubernetes.default
Server: 10.96.0.10
Address: 10.96.0.10#53
Name: kubernetes.default.svc.cluster.local
Address: 10.96.0.1
ubuntu@k8s-master:~$ kubectl exec -ti mongo-2 -- nslookup kubernetes.default
Server: 10.96.0.10
Address: 10.96.0.10#53
Name: kubernetes.default.svc.cluster.local
Address: 10.96.0.1
содержимое statefulset.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: mongo
spec:
selector:
matchLabels:
app: mongo
serviceName: "mongo"
replicas: 3
template:
metadata:
labels:
app: mongo
spec:
terminationGracePeriodSeconds: 10
containers:
- name: mongo
image: mongo
command:
- mongod
- "--bind_ip_all"
- "--replSet"
- rs0
ports:
- containerPort: 27017
volumeMounts:
- name: mongo-volume
mountPath: /data/db
volumeClaimTemplates:
- metadata:
name: mongo-volume
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 5Gi
kubectl get endpoints mon go -oyaml
apiVersion: v1
kind: Endpoints
metadata:
annotations:
endpoints.kubernetes.io/last-change-trigger-time: "2020-05-26T08:27:18Z"
creationTimestamp: "2020-05-26T00:42:16Z"
labels:
app: mongo
service.kubernetes.io/headless: ""
managedFields:
- apiVersion: v1
fieldsType: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.: {}
f:endpoints.kubernetes.io/last-change-trigger-time: {}
f:labels:
.: {}
f:app: {}
f:service.kubernetes.io/headless: {}
f:subsets: {}
manager: kube-controller-manager
operation: Update
time: "2020-05-26T08:27:18Z"
name: mongo
namespace: default
resourceVersion: "114014"
selfLink: /api/v1/namespaces/default/endpoints/mongo
uid: 0a7faaa4-9de7-4a14-b101-04fc5884d23c
subsets:
- addresses:
- hostname: mongo-1
ip: 10.244.1.16
nodeName: k8s-worker-node2
targetRef:
kind: Pod
name: mongo-1
namespace: default
resourceVersion: "114012"
uid: 249f3cf6-aa2b-4d4e-a736-671dd1942fc7
- hostname: mongo-2
ip: 10.244.2.28
nodeName: k8s-worker-node1
targetRef:
kind: Pod
name: mongo-2
namespace: default
resourceVersion: "113775"
uid: 796a80d9-889e-4fdd-88f2-0baf3ed080c6
- hostname: mongo-0
ip: 10.244.2.31
nodeName: k8s-worker-node1
targetRef:
kind: Pod
name: mongo-0
namespace: default
resourceVersion: "113825"
uid: 8948c536-43fd-46c4-9132-d5afe145ede7
ports:
- name: mongo
port: 27017
protocol: TCP
Добавление с помощью
rs0:PRIMARY> rs.add("mongo‑1.mongo.default.svc.cluster.local:27017")
{
"operationTime" : Timestamp(1590526166, 1),
"ok" : 0,
"errmsg" : "Quorum check failed because not enough voting nodes responded; required 2 but only the following 1 voting nodes responded: mongo-0:27017; the following nodes did not respond affirmatively: mongo‑1.mongo.default.svc.cluster.local:27017 failed with Error connecting to mongo‑1.mongo.default.svc.cluster.local:27017 :: caused by :: Could not find address for mongo‑1.mongo.default.svc.cluster.local:27017: SocketException: Host not found (authoritative)",
"code" : 74,
"codeName" : "NodeNotFound",
"$clusterTime" : {
"clusterTime" : Timestamp(1590526166, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
}
rs0:PRIMARY>