Можно ли создавать модули, службы, контроллеры реплик Kubernetes et c на AWS cloudfromation? - PullRequest
0 голосов
/ 08 апреля 2020

Облачная информация AWS поддерживает создание модулей, сервисов, контроллеров реплики и т. Д. c Kubernetes, или настройка кластеров EKS и рабочих узлов и использование Kubectl для создания ресурсов - единственный способ?

Ответы [ 2 ]

1 голос
/ 09 апреля 2020

Не из коробки, но вы можете это сделать, если в CloudFormation вы используете собственный тип ресурса, поддерживаемый лямбда-функцией.

В быстром запуске AWS EKS есть пример :

AWSTemplateFormatVersion: "2010-09-09"
Description: "deploy an example workload into an existing kubernetes cluster (qs-1p817r5f9)"
Parameters:
  KubeConfigPath:
    Type: String
  KubeConfigKmsContext:
    Type: String
    Default: "EKSQuickStart"
  KubeClusterName:
    Type: String
  NodeInstanceProfile:
    Type: String
  QSS3BucketName:
    AllowedPattern: ^[0-9a-zA-Z]+([0-9a-zA-Z-]*[0-9a-zA-Z])*$
    ConstraintDescription: Quick Start bucket name can include numbers, lowercase
      letters, uppercase letters, and hyphens (-). It cannot start or end with a hyphen
      (-).
    Default: aws-quickstart
    Description: S3 bucket name for the Quick Start assets. This string can include
      numbers, lowercase letters, uppercase letters, and hyphens (-). It cannot start
      or end with a hyphen (-).
    Type: String
  QSS3KeyPrefix:
    AllowedPattern: ^[0-9a-zA-Z-/.]*$
    ConstraintDescription: Quick Start key prefix can include numbers, lowercase letters,
      uppercase letters, hyphens (-), dots(.) and forward slash (/).
    Default: quickstart-amazon-eks/
    Description: S3 key prefix for the Quick Start assets. Quick Start key prefix
      can include numbers, lowercase letters, uppercase letters, hyphens (-), dots(.) and
      forward slash (/).
    Type: String
  QSS3BucketRegion:
    Default: 'us-east-1'
    Description: The AWS Region where the Quick Start S3 bucket (QSS3BucketName) is
      hosted. When using your own bucket, you must specify this value.
    Type: String
  LambdaZipsBucketName:
    Description: 'OPTIONAL: Bucket Name where the lambda zip files should be placed,
          if left blank a bucket will be created.'
    Type: String
    Default: ''
  K8sSubnetIds:
    Type: List<AWS::EC2::Subnet::Id>
  VPCID:
    Type: AWS::EC2::VPC::Id
  ControlPlaneSecurityGroup:
    Type: AWS::EC2::SecurityGroup::Id
Conditions:
  CreateLambdaZipsBucket: !Equals
    - !Ref 'LambdaZipsBucketName'
    - ''
  UsingDefaultBucket: !Equals [!Ref QSS3BucketName, 'aws-quickstart']
Resources:
  WorkloadStack:
    Type: AWS::CloudFormation::Stack
    Properties:
      TemplateURL: !Sub
        - 'https://${S3Bucket}.s3.${S3Region}.${AWS::URLSuffix}/${QSS3KeyPrefix}templates/example-workload.template.yaml'
        - S3Region: !If [UsingDefaultBucket, !Ref 'AWS::Region', !Ref QSS3BucketRegion]
          S3Bucket: !If [UsingDefaultBucket, !Sub '${QSS3BucketName}-${AWS::Region}', !Ref QSS3BucketName]
      Parameters:
        KubeManifestLambdaArn: !GetAtt KubeManifestLambda.Arn
        HelmLambdaArn: !GetAtt HelmLambda.Arn
        KubeConfigPath: !Ref KubeConfigPath
        KubeConfigKmsContext: !Ref KubeConfigKmsContext
        KubeClusterName: !Ref KubeClusterName
        NodeInstanceProfile: !Ref NodeInstanceProfile
  CopyZips:
    Type: Custom::CopyZips
    Properties:
      ServiceToken: !GetAtt 'CopyZipsFunction.Arn'
      DestBucket: !Ref LambdaZipsBucketName
      SourceBucket: !If [UsingDefaultBucket, !Sub '${QSS3BucketName}-${AWS::Region}', !Ref QSS3BucketName]
      Prefix: !Ref 'QSS3KeyPrefix'
      Objects:
        - functions/packages/Helm/lambda.zip
        - functions/packages/DeleteBucketContents/lambda.zip
        - functions/packages/KubeManifest/lambda.zip
        - functions/packages/LambdaEniCleanup/lambda.zip
  VPCLambdaCleanup:
    Type: Custom::LambdaCleanup
    Properties:
      ServiceToken: !GetAtt VPCLambdaCleanupLambdaFunction.Arn
      Region: !Ref "AWS::Region"
      LambdaFunctionNames:
        - !Ref KubeManifestLambda
  VPCLambdaCleanupLambdaFunction:
    DependsOn: CopyZips
    Type: "AWS::Lambda::Function"
    Properties:
      Handler: lambda_function.lambda_handler
      MemorySize: 128
      Role: !GetAtt LambdaCleanUpFunctionRole.Arn
      Runtime: python3.7
      Timeout: 900
      Code:
        S3Bucket: !Ref LambdaZipsBucketName
        S3Key: !Sub '${QSS3KeyPrefix}functions/packages/LambdaEniCleanup/lambda.zip'
  HelmLambda:
    DependsOn: CopyZips
    Type: AWS::Lambda::Function
    Properties:
      Handler: lambda_function.lambda_handler
      MemorySize: 128
      Role: !GetAtt ManifestRole.Arn
      Runtime: python3.6
      Timeout: 900
      Code:
        S3Bucket: !Ref LambdaZipsBucketName
        S3Key: !Sub '${QSS3KeyPrefix}functions/packages/Helm/lambda.zip'
      VpcConfig:
        SecurityGroupIds: [ !Ref EKSLambdaSecurityGroup ]
        SubnetIds: !Ref K8sSubnetIds
  KubeManifestLambda:
    DependsOn: CopyZips
    Type: AWS::Lambda::Function
    Properties:
      Handler: lambda_function.lambda_handler
      MemorySize: 128
      Role: !GetAtt ManifestRole.Arn
      Runtime: python3.6
      Timeout: 900
      Code:
        S3Bucket: !Ref LambdaZipsBucketName
        S3Key: !Sub '${QSS3KeyPrefix}functions/packages/KubeManifest/lambda.zip'
      VpcConfig:
        SecurityGroupIds: [ !Ref EKSLambdaSecurityGroup ]
        SubnetIds: !Ref K8sSubnetIds
  DeleteBucketContentsLambda:
    DependsOn: CopyZips
    Type: AWS::Lambda::Function
    Properties:
      Handler: lambda_function.lambda_handler
      MemorySize: 128
      Role: !GetAtt DeleteBucketContentsRole.Arn
      Runtime: python3.7
      Timeout: 900
      Code:
        S3Bucket: !Ref LambdaZipsBucketName
        S3Key: !Sub '${QSS3KeyPrefix}functions/packages/DeleteBucketContents/lambda.zip'
  CopyZipsFunction:
    Type: AWS::Lambda::Function
    Properties:
      Description: Copies objects from a source S3 bucket to a destination
      Handler: index.handler
      Runtime: python3.7
      Role: !GetAtt CopyZipsRole.Arn
      Timeout: 900
      Code:
        ZipFile: |
          import json
          import logging
          import threading
          import boto3
          import cfnresponse
          def copy_objects(source_bucket, dest_bucket, prefix, objects):
              s3 = boto3.client('s3')
              for o in objects:
                  key = prefix + o
                  copy_source = {
                      'Bucket': source_bucket,
                      'Key': key
                  }
                  print('copy_source: %s' % copy_source)
                  print('dest_bucket = %s'%dest_bucket)
                  print('key = %s' %key)
                  s3.copy_object(CopySource=copy_source, Bucket=dest_bucket,
              Key=key)
          def delete_objects(bucket, prefix, objects):
              s3 = boto3.client('s3')
              objects = {'Objects': [{'Key': prefix + o} for o in objects]}
              s3.delete_objects(Bucket=bucket, Delete=objects)
          def timeout(event, context):
              logging.error('Execution is about to time out, sending failure response to CloudFormation')
              cfnresponse.send(event, context, cfnresponse.FAILED, {}, physical_resource_id)
          def handler(event, context):
              physical_resource_id = None
              if "PhysicalResourceId" in event.keys():
                physical_resource_id = event["PhysicalResourceId"]
              # make sure we send a failure to CloudFormation if the function is going to timeout
              timer = threading.Timer((context.get_remaining_time_in_millis()
              / 1000.00) - 0.5, timeout, args=[event, context])
              timer.start()
              print('Received event: %s' % json.dumps(event))
              status = cfnresponse.SUCCESS
              try:
                  source_bucket = event['ResourceProperties']['SourceBucket']
                  dest_bucket = event['ResourceProperties']['DestBucket']
                  prefix = event['ResourceProperties']['Prefix']
                  objects = event['ResourceProperties']['Objects']
                  if event['RequestType'] == 'Delete':
                      delete_objects(dest_bucket, prefix, objects)
                  else:
                      copy_objects(source_bucket, dest_bucket, prefix, objects)
              except Exception as e:
                  logging.error('Exception: %s' % e, exc_info=True)
                  status = cfnresponse.FAILED
              finally:
                  timer.cancel()
                  cfnresponse.send(event, context, status, {}, physical_resource_id)
  LambdaZipsBucket:
    Type: AWS::S3::Bucket
    Condition: CreateLambdaZipsBucket
  LambdaCleanUpFunctionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
          - Action: ['sts:AssumeRole']
            Effect: Allow
            Principal:
              Service: [lambda.amazonaws.com]
        Version: '2012-10-17'
      Path: /
      Policies:
        - PolicyName: LambdaRole
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Action:
                  - 'logs:CreateLogGroup'
                  - 'logs:CreateLogStream'
                  - 'logs:PutLogEvents'
                Effect: Allow
                Resource: !Sub "arn:${AWS::Partition}:logs:*:*:*"
              - Action:
                  - 'ec2:*'
                Effect: Allow
                Resource: "*"
  DeleteBucketContentsRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: lambda.amazonaws.com
            Action: sts:AssumeRole
      ManagedPolicyArns:
        - !Sub 'arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole'
      Policies:
        - PolicyName: deletebucketcontents
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action: s3:*
                Resource:
                  - !Sub 'arn:${AWS::Partition}:s3:::${LambdaZipsBucketName}/*'
                  - !Sub 'arn:${AWS::Partition}:s3:::${LambdaZipsBucketName}'
  CopyZipsRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: lambda.amazonaws.com
            Action: sts:AssumeRole
      ManagedPolicyArns:
        - !Su  'arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole'
      Policies:
        - PolicyName: lambda-copier
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action: s3:GetObject
                Resource: !Sub
                  - 'arn:${AWS::Partition}:s3:::${S3Bucket}/${QSS3KeyPrefix}*'
                  - S3Bucket: !If [UsingDefaultBucket, !Sub '${QSS3BucketName}-${AWS::Region}', !Ref QSS3BucketName]
              - Effect: Allow
                Action:
                  - s3:PutObject
                  - s3:DeleteObject
                Resource: !Sub 'arn:${AWS::Partition}:s3:::${LambdaZipsBucketName}/${QSS3KeyPrefix}*'
  ManifestRole:
    Type: "AWS::IAM::Role"
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service: lambda.amazonaws.com
            Action: sts:AssumeRole
      Policies:
        - PolicyName: eksStackPolicy
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action: s3:GetObject
                Resource: !Sub
                  - "arn:${AWS::Partition}:s3:::${BucketName}/*"
                  - S3Bucket: !If [UsingDefaultBucket, !Sub '${QSS3BucketName}-${AWS::Region}', !Ref QSS3BucketName]
              - Effect: Allow
                Action:
                  - logs:CreateLogGroup
                  - logs:CreateLogStream
                  - logs:PutLogEvents
                  - ec2:CreateNetworkInterface
                  - ec2:DescribeNetworkInterfaces
                  - ec2:DeleteNetworkInterface
                Resource:
                  - "*"
              - Action: "kms:decrypt"
                Effect: Allow
                Resource: "*"
              - Action: "s3:GetObject"
                Effect: Allow
                Resource: "*"
  EKSLambdaSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Security group for lambda to communicate with cluster API
      VpcId: !Ref VPCID
  ClusterControlPlaneSecurityGroupIngress:
    Type: AWS::EC2::SecurityGroupIngress
    Properties:
      Description: Allow lambda to communicate with the cluster API Server
      GroupId: !Ref ControlPlaneSecurityGroup
      SourceSecurityGroupId: !Ref EKSLambdaSecurityGroup
      IpProtocol: tcp
      ToPort: 443
      FromPort: 443

Работает, создавая пользовательский ресурс лямбда-функции KubeManifestLambda и HelmLambda с установленными соответственно kubectl и helm, обе из которых настроены на роль, позволяющую им получать доступ к кластеру EKS k8s.

Затем эти пользовательские ресурсы можно использовать для развертывания манифестов k8s и диаграмм управления с пользовательскими значениями, например в этом примере .

 KubeManifestExample:
    Type: "Custom::KubeManifest"
    Version: '1.0'
    Properties:
      # The lambda function that executes the manifest against the cluster. This is created in one of the parent stacks
      ServiceToken: !Ref KubeManifestLambdaArn
      # S3 path to the encrypted config file eg. s3://my-bucket/kube/config.encrypted
      KubeConfigPath: !Ref KubeConfigPath
      # context for KMS to use when decrypting the file
      KubeConfigKmsContext: !Ref KubeConfigKmsContext
      # Kubernetes manifest
      Manifest:
        apiVersion: v1
        kind: ConfigMap
        metadata:
          # If name is not specified it will be automatically generated,
          # and can be retrieved with !GetAtt LogicalID.name
          #
          # name: test
          #
          # if namespace is not specified, "default" namespace will be used
          namespace: kube-system
        data:
          # examples of consuming outputs of the HelmExample resource below's output. Creates an implicit dependency, so
          # this resource will only launch once the HelmExample resource has completed successfully
          ServiceCatalogReleaseName: !Ref HelmExample
          ServiceCatalogKubernetesServiceName: !GetAtt HelmExample.Service0

Это даже позволяет ссылаться на другие Ресурсы формирования облаков, такие как экземпляры RDS, которые создаются как часть рабочей нагрузки.

0 голосов
/ 09 апреля 2020

Вы можете использовать CloudFormation для создания кластера EKS и рабочих узлов, но вы должны использовать kubectl для любых операций в кластере, таких как создание служб, модулей, развертываний и т. Д. c ..... вы не можете использовать CloudFormation для этого

...