При попытке восстановить тома EBS из моментального снимка возвращает статус утерянного. мы используем ключи AWS KMS CMK с политикой, имеющей разрешение kms *. Операция резервного копирования прошла нормально .. операция восстановления способна восстановить все k8s ресурсы, кроме PV C.
k get pvc -n nginx-example
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
nginx-logs Lost pvc-bda55207-a1e5-11ea-b7e6-02b82f6b7f4e 0 gp2-encrypt 4m22s
k get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pvc-bda55207-a1e5-11ea-b7e6-02b82f6b7f4e 1Gi RWO Retain Released nginx-example/nginx-logs gp2-encrypt 33m
Мы заметили UID PV и PV C не совпадают после восстановления PV C.
Учетная запись службы, используемая модулем velero, имеет следующую политику:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"ec2:DeleteSnapshot",
"kms:Decrypt",
"ec2:CreateTags",
"kms:GenerateDataKeyWithoutPlaintext",
"s3:ListBucket",
"kms:GenerateDataKeyPairWithoutPlaintext",
"ec2:DescribeSnapshots",
"kms:GenerateDataKeyPair",
"kms:ReEncryptFrom",
"ec2:CreateVolume",
"s3:ListMultipartUploadParts",
"s3:PutObject",
"s3:GetObject",
"s3:AbortMultipartUpload",
"ec2:DescribeVolumes",
"ec2:CreateSnapshot",
"kms:GenerateDataKey",
"kms:ReEncryptTo",
"s3:DeleteObject"
],
"Resource": "*"
}
]
}
, мы используем приведенный ниже yaml для определения класса хранения и PV C
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: gp2-encrypt
parameters:
type: gp2
encrypted: "true"
fsType: ext4
kmsKeyId: arn:aws:kms:us-east-XXXXXX
provisioner: kubernetes.io/aws-ebs
reclaimPolicy: Retain
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: nginx-logs
namespace: nginx-example
labels:
app: nginx
spec:
storageClassName: gp2-encrypt
accessModes:
- ReadWriteOnce
resources:
requests:
storage: [50Mi]
Ниже представлены логи от velero pods ..
> time="2020-05-29T19:59:04Z" level=info msg="Starting restore of backup
> cluster-addons/nginx-backup-5" logSource="pkg/restore/restore.go:394"
> restore=cluster-addons/nginx-backup-5-20200529155858
> time="2020-05-29T19:59:04Z" level=info msg="Restoring cluster level
> resource 'persistentvolumes'" logSource="pkg/restore/restore.go:779"
> restore=cluster-addons/nginx-backup-5-20200529155858
> time="2020-05-29T19:59:04Z" level=info msg="Getting client for /v1,
> Kind=PersistentVolume" logSource="pkg/restore/restore.go:821"
> restore=cluster-addons/nginx-backup-5-20200529155858
> time="2020-05-29T20:09:04Z" level=info msg="Restoring resource
> 'persistentvolumeclaims' into namespace 'nginx-example'"
> logSource="pkg/restore/restore.go:777"
> restore=cluster-addons/nginx-backup-5-20200529155858
> time="2020-05-29T20:09:04Z" level=info msg="Getting client for /v1,
> Kind=PersistentVolumeClaim" logSource="pkg/restore/restore.go:821"
> restore=cluster-addons/nginx-backup-5-20200529155858
> time="2020-05-29T20:09:04Z" level=info msg="Executing item action for
> persistentvolumeclaims" logSource="pkg/restore/restore.go:1030"
> restore=cluster-addons/nginx-backup-5-20200529155858
> time="2020-05-29T20:09:04Z" level=info msg="Executing
> AddPVFromPVCAction" cmd=/velero
> logSource="pkg/restore/add_pv_from_pvc_action.go:44" pluginName=velero
> restore=cluster-addons/nginx-backup-5-20200529155858
> time="2020-05-29T20:09:04Z" level=info msg="Adding PV
> pvc-bda55207-a1e5-11ea-b7e6-02b82f6b7f4e as an additional item to
> restore" cmd=/velero
> logSource="pkg/restore/add_pv_from_pvc_action.go:66" pluginName=velero
> restore=cluster-addons/nginx-backup-5-20200529155858
> time="2020-05-29T20:09:04Z" level=info msg="Skipping
> persistentvolumes/pvc-bda55207-a1e5-11ea-b7e6-02b82f6b7f4e because
> it's already been restored." logSource="pkg/restore/restore.go:910"
> restore=cluster-addons/nginx-backup-5-20200529155858
> time="2020-05-29T20:09:04Z" level=info msg="Executing item action for
> persistentvolumeclaims" logSource="pkg/restore/restore.go:1030"
> restore=cluster-addons/nginx-backup-5-20200529155858
> time="2020-05-29T20:09:04Z" level=info msg="Executing
> ChangeStorageClassAction" cmd=/velero
> logSource="pkg/restore/change_storageclass_action.go:63"
> pluginName=velero restore=cluster-addons/nginx-backup-5-20200529155858
> time="2020-05-29T20:09:04Z" level=info msg="Attempting to restore
> PersistentVolumeClaim: nginx-logs"
> logSource="pkg/restore/restore.go:1136"
> restore=cluster-addons/nginx-backup-5-20200529155858
> time="2020-05-29T20:09:04Z" level=info msg="Done executing
> ChangeStorageClassAction" cmd=/velero
> logSource="pkg/restore/change_storageclass_action.go:74"
> pluginName=velero restore=cluster-addons/nginx-backup-5-20200529155858
>
> The cloudtrail does not have much information. Would you please let us
> know any additional. settings needed here?