Как раскрутить контейнер для сельдерея с помощью SQS - PullRequest
Новая
       65

Как раскрутить контейнер для сельдерея с помощью SQS

0 голосов
/ 05 августа 2020

Я пытаюсь настроить Celery для общения с AWS SQS. Сельдерей положите журнал, как показано ниже, и после этого контейнер опускается и снова запускается в l oop. Зарегистрируйтесь, как показано ниже:

обрезать ... 

[2020-08-05 09:32:35,715: DEBUG/MainProcess] Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane
[2020-08-05 09:32:35,798: DEBUG/MainProcess] Changing event name from before-call.apigateway to before-call.api-gateway
[2020-08-05 09:32:35,799: DEBUG/MainProcess] Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict
[2020-08-05 09:32:35,801: DEBUG/MainProcess] Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration
[2020-08-05 09:32:35,801: DEBUG/MainProcess] Changing event name from before-parameter-build.route53 to before-parameter-build.route-53
[2020-08-05 09:32:35,802: DEBUG/MainProcess] Changing event name from request-created.cloudsearchdomain.Search to request-created.cloudsearch-domain.Search
[2020-08-05 09:32:35,802: DEBUG/MainProcess] Changing event name from docs.*.autoscaling.CreateLaunchConfiguration.complete-section to docs.*.auto-scaling.CreateLaunchConfiguration.complete-section
[2020-08-05 09:32:35,805: DEBUG/MainProcess] Changing event name from before-parameter-build.logs.CreateExportTask to before-parameter-build.cloudwatch-logs.CreateExportTask
[2020-08-05 09:32:35,805: DEBUG/MainProcess] Changing event name from docs.*.logs.CreateExportTask.complete-section to docs.*.cloudwatch-logs.CreateExportTask.complete-section
[2020-08-05 09:32:35,806: DEBUG/MainProcess] Changing event name from before-parameter-build.cloudsearchdomain.Search to before-parameter-build.cloudsearch-domain.Search
[2020-08-05 09:32:35,806: DEBUG/MainProcess] Changing event name from docs.*.cloudsearchdomain.Search.complete-section to docs.*.cloudsearch-domain.Search.complete-section
[2020-08-05 09:32:35,806: DEBUG/MainProcess] Setting config variable for region to {'eu-west-1'}
[2020-08-05 09:32:35,808: DEBUG/MainProcess] Loading JSON file: /usr/local/lib/python3.8/site-packages/botocore/data/endpoints.json
[2020-08-05 09:32:35,815: DEBUG/MainProcess] Event choose-service-name: calling handler <function handle_service_name_alias at 0x7fed0d144940>
[2020-08-05 09:32:35,899: DEBUG/MainProcess] Loading JSON file: /usr/local/lib/python3.8/site-packages/botocore/data/sqs/2012-11-05/service-2.json
[2020-08-05 09:32:35,902: DEBUG/MainProcess] Event creating-client-class.sqs: calling handler <function add_generate_presigned_url at 0x7fed096053a0>
[2020-08-05 09:32:36,907: DEBUG/MainProcess] removing tasks from inqueue until task handler finished
 
 -------------- celery@ip-10-10-12-215.eu-central-1.compute.internal v4.4.6 (cliffs)
--- ***** ----- 
-- ******* ---- Linux-4.14.177-139.254.amzn2.x86_64-x86_64-with-glibc2.2.5 2020-08-05 09:32:35
- *** --- * --- 
- ** ---------- [config]
- ** ---------- .> app:         app_name:0x7fed0d6f18e0
- ** ---------- .> transport:   sqs://aws_access_key:**@localhost//
- ** ---------- .> results:     disabled://
- *** --- * --- .> concurrency: 2 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** ----- 
 -------------- [queues]
                .> celery           exchange=celery(direct) key=celery
                
[tasks]
  . app_name.tasks.check_new_users
  . app_name.tasks.send_mail_for_new_user
  . app_name.tasks.test_task
  . celery.accumulate
  . celery.backend_cleanup
  . celery.chain
  . celery.chord
  . celery.chord_unlock
  . celery.chunks
  . celery.group
  . celery.map
  . celery.starmap

Настройки в Django следующие: (переменные взяты из SSM) 

AWS_REGION = os.getenv("AWS_REGION")
BROKER_URL = os.getenv("BROKER_URL", "amqp://app_name-rabbitmq//")
BROKER_TRANSPORT = 'sqs'
BROKER_TRANSPORT_OPTIONS = {
    "region": {AWS_REGION},
    "polling_interval": 60,
    "queue_name_prefix": "prod-"
}
CELERY_BROKER_URL = BROKER_URL
CELERY_BROKER_TRANSPORT_OPTIONS = BROKER_TRANSPORT_OPTIONS

CELERY_ACCEPT_CONTENT = ['json', 'yaml']
CELERY_RESULT_SERIALIZER = 'json'
CELERY_TASK_SERIALIZER = 'json'

CELERY_ENABLE_REMOTE_CONTROL = False
CELERY_SEND_EVENTS = False

CELERY_RESULT_BACKEND = None
RESULT_BACKEND = None

CELERY_IMPORTS = ("app_name.tasks",)
CELERYBEAT_SCHEDULE = {
    "check_new_users": {"task": "tasks.app_name.check_new_users", "schedule": crontab(hour="9,17", minute=0,)},
}

Есть ли какие-либо у кого-то была такая ситуация и можно было помочь? Роль IAM - это на время полный доступ к SQS. РЕДАКТИРОВАТЬ: Если требуются какие-либо другие подробности, сообщите мне.

...