Я пытаюсь использовать библиотеку sqlalchemy в лямбде, которую я хочу загрузить. При этом используется проект AWS Lambda, созданный с использованием инструментария AWS в Pycharm. Библиотека sqlalchemy используется только для вставки таблиц pandas в mysql (в таблицах более 300 столбцов, а в будущем будет добавлено еще больше, поэтому явное определение имен столбцов не вариант). Обратите внимание, что библиотека включена в файл needs.txt.
Лямбда работает нормально при запуске на моем локальном компьютере с использованием параметра «Построить функцию внутри контейнера» внутри конфигураций запуска. Я считаю, что этот параметр эквивалентен использованию команды:
sam build --use-container
Однако конфигурация для удаленного развертывания лямбды не включает параметр «Функция сборки внутри контейнера». Таким образом, я получаю следующую ошибку при удаленном развертывании моей функции:
2019-04-17 13:23:07 Using SAM Template at /Users/davidpell/PycharmProjects/lambda/template.yaml
2019-04-17 13:23:07 Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane
2019-04-17 13:23:07 Changing event name from before-call.apigateway to before-call.api-gateway
2019-04-17 13:23:07 Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict
2019-04-17 13:23:07 Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration
2019-04-17 13:23:07 Changing event name from before-parameter-build.route53 to before-parameter-build.route-53
2019-04-17 13:23:07 Changing event name from request-created.cloudsearchdomain.Search to request-created.cloudsearch-domain.Search
2019-04-17 13:23:07 Changing event name from docs.*.autoscaling.CreateLaunchConfiguration.complete-section to docs.*.auto-scaling.CreateLaunchConfiguration.complete-section
2019-04-17 13:23:07 Changing event name from before-parameter-build.logs.CreateExportTask to before-parameter-build.cloudwatch-logs.CreateExportTask
2019-04-17 13:23:07 Changing event name from docs.*.logs.CreateExportTask.complete-section to docs.*.cloudwatch-logs.CreateExportTask.complete-section
2019-04-17 13:23:07 Changing event name from before-parameter-build.cloudsearchdomain.Search to before-parameter-build.cloudsearch-domain.Search
2019-04-17 13:23:07 Changing event name from docs.*.cloudsearchdomain.Search.complete-section to docs.*.cloudsearch-domain.Search.complete-section
2019-04-17 13:23:07 Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane
2019-04-17 13:23:07 Changing event name from before-call.apigateway to before-call.api-gateway
2019-04-17 13:23:07 Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict
2019-04-17 13:23:07 Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration
2019-04-17 13:23:07 Changing event name from before-parameter-build.route53 to before-parameter-build.route-53
2019-04-17 13:23:07 Changing event name from request-created.cloudsearchdomain.Search to request-created.cloudsearch-domain.Search
2019-04-17 13:23:07 Changing event name from docs.*.autoscaling.CreateLaunchConfiguration.complete-section to docs.*.auto-scaling.CreateLaunchConfiguration.complete-section
2019-04-17 13:23:07 Changing event name from before-parameter-build.logs.CreateExportTask to before-parameter-build.cloudwatch-logs.CreateExportTask
2019-04-17 13:23:07 Changing event name from docs.*.logs.CreateExportTask.complete-section to docs.*.cloudwatch-logs.CreateExportTask.complete-section
2019-04-17 13:23:07 Changing event name from before-parameter-build.cloudsearchdomain.Search to before-parameter-build.cloudsearch-domain.Search
2019-04-17 13:23:07 Changing event name from docs.*.cloudsearchdomain.Search.complete-section to docs.*.cloudsearch-domain.Search.complete-section
2019-04-17 13:23:07 'build' command is called
2019-04-17 13:23:07 No Parameters detected in the template
2019-04-17 13:23:07 2 resources found in the template
2019-04-17 13:23:07 Found Serverless function with name='HelloWorldFunction' and CodeUri='hello_world/'
2019-04-17 13:23:08 Building resource 'HelloWorldFunction'
2019-04-17 13:23:08 Loading workflow module 'aws_lambda_builders.workflows'
2019-04-17 13:23:08 Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python', dependency_manager='pip', application_framework=None)'
2019-04-17 13:23:08 Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)'
2019-04-17 13:23:08 Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)'
2019-04-17 13:23:08 Registering workflow 'GoDepBuilder' with capability 'Capability(language='go', dependency_manager='dep', application_framework=None)'
2019-04-17 13:23:08 Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go', dependency_manager='modules', application_framework=None)'
2019-04-17 13:23:08 Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java', dependency_manager='gradle', application_framework=None)'
2019-04-17 13:23:08 Registering workflow 'JavaMavenWorkflow' with capability 'Capability(language='java', dependency_manager='maven', application_framework=None)'
2019-04-17 13:23:08 Registering workflow 'DotnetCliPackageBuilder' with capability 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)'
2019-04-17 13:23:08 Found workflow 'PythonPipBuilder' to support capabilities 'Capability(language='python', dependency_manager='pip', application_framework=None)'
2019-04-17 13:23:08 Running workflow 'PythonPipBuilder'
2019-04-17 13:23:08 Running PythonPipBuilder:ResolveDependencies
2019-04-17 13:23:08 calling pip download -r /Users/davidpell/PycharmProjects/lambda/hello_world/requirements.txt --dest /var/folders/c4/jqqz7trx02l9w1qkbx8dzxvr0000gn/T/tmpgl5r1tqp
2019-04-17 13:23:13 Full dependency closure: {numpy==1.16.2(wheel), six==1.12.0(wheel), requests==2.21.0(wheel), pytz==2019.1(wheel), pymysql==0.9.3(wheel), urllib3==1.24.1(wheel), python-dateutil==2.8.0(wheel), setuptools==41.0.0(wheel), sqlalchemy==1.3.3(sdist), pip==19.0.3(wheel), idna==2.8(wheel), chardet==3.0.4(wheel), certifi==2019.3.9(wheel), pandas==0.24.2(wheel)}
2019-04-17 13:23:13 initial compatible: {six==1.12.0(wheel), requests==2.21.0(wheel), pytz==2019.1(wheel), pymysql==0.9.3(wheel), urllib3==1.24.1(wheel), python-dateutil==2.8.0(wheel), setuptools==41.0.0(wheel), pip==19.0.3(wheel), idna==2.8(wheel), chardet==3.0.4(wheel), certifi==2019.3.9(wheel)}
2019-04-17 13:23:13 initial incompatible: {numpy==1.16.2(wheel), pandas==0.24.2(wheel), sqlalchemy==1.3.3(sdist)}
2019-04-17 13:23:13 Downloading missing wheels: {sqlalchemy==1.3.3(sdist), numpy==1.16.2(wheel), pandas==0.24.2(wheel)}
2019-04-17 13:23:13 calling pip download --only-binary=:all: --no-deps --platform manylinux1_x86_64 --implementation cp --abi cp37m --dest /var/folders/c4/jqqz7trx02l9w1qkbx8dzxvr0000gn/T/tmpgl5r1tqp sqlalchemy==1.3.3
2019-04-17 13:23:14 calling pip download --only-binary=:all: --no-deps --platform manylinux1_x86_64 --implementation cp --abi cp37m --dest /var/folders/c4/jqqz7trx02l9w1qkbx8dzxvr0000gn/T/tmpgl5r1tqp numpy==1.16.2
2019-04-17 13:23:15 calling pip download --only-binary=:all: --no-deps --platform manylinux1_x86_64 --implementation cp --abi cp37m --dest /var/folders/c4/jqqz7trx02l9w1qkbx8dzxvr0000gn/T/tmpgl5r1tqp pandas==0.24.2
2019-04-17 13:23:16 compatible wheels after second download pass: {numpy==1.16.2(wheel), six==1.12.0(wheel), requests==2.21.0(wheel), pytz==2019.1(wheel), pymysql==0.9.3(wheel), urllib3==1.24.1(wheel), python-dateutil==2.8.0(wheel), setuptools==41.0.0(wheel), pip==19.0.3(wheel), idna==2.8(wheel), chardet==3.0.4(wheel), certifi==2019.3.9(wheel), pandas==0.24.2(wheel)}
2019-04-17 13:23:16 Build missing wheels from sdists (C compiling True): {sqlalchemy==1.3.3(sdist)}
2019-04-17 13:23:16 calling pip wheel --no-deps --wheel-dir /var/folders/c4/jqqz7trx02l9w1qkbx8dzxvr0000gn/T/tmpgl5r1tqp /var/folders/c4/jqqz7trx02l9w1qkbx8dzxvr0000gn/T/tmpgl5r1tqp/SQLAlchemy-1.3.3.tar.gz
2019-04-17 13:23:17 compatible after building wheels (no C compiling): {numpy==1.16.2(wheel), six==1.12.0(wheel), requests==2.21.0(wheel), pytz==2019.1(wheel), pymysql==0.9.3(wheel), urllib3==1.24.1(wheel), python-dateutil==2.8.0(wheel), setuptools==41.0.0(wheel), pip==19.0.3(wheel), idna==2.8(wheel), chardet==3.0.4(wheel), certifi==2019.3.9(wheel), pandas==0.24.2(wheel)}
2019-04-17 13:23:17 Build missing wheels from sdists (C compiling False): {sqlalchemy==1.3.3(sdist)}
2019-04-17 13:23:17 calling pip wheel --no-deps --wheel-dir /var/folders/c4/jqqz7trx02l9w1qkbx8dzxvr0000gn/T/tmpgl5r1tqp /var/folders/c4/jqqz7trx02l9w1qkbx8dzxvr0000gn/T/tmpgl5r1tqp/SQLAlchemy-1.3.3.tar.gz
2019-04-17 13:23:19 compatible after building wheels (C compiling): {numpy==1.16.2(wheel), six==1.12.0(wheel), requests==2.21.0(wheel), pytz==2019.1(wheel), pymysql==0.9.3(wheel), urllib3==1.24.1(wheel), python-dateutil==2.8.0(wheel), setuptools==41.0.0(wheel), pip==19.0.3(wheel), idna==2.8(wheel), chardet==3.0.4(wheel), certifi==2019.3.9(wheel), pandas==0.24.2(wheel)}
2019-04-17 13:23:19 Final compatible: {numpy==1.16.2(wheel), six==1.12.0(wheel), requests==2.21.0(wheel), pytz==2019.1(wheel), pymysql==0.9.3(wheel), urllib3==1.24.1(wheel), python-dateutil==2.8.0(wheel), setuptools==41.0.0(wheel), pip==19.0.3(wheel), idna==2.8(wheel), chardet==3.0.4(wheel), certifi==2019.3.9(wheel), pandas==0.24.2(wheel)}
2019-04-17 13:23:19 Final incompatible: {numpy==1.16.2(wheel), pandas==0.24.2(wheel)}
2019-04-17 13:23:19 Final missing wheels: {sqlalchemy==1.3.3(sdist)}
2019-04-17 13:23:20 PythonPipBuilder:ResolveDependencies failed
Traceback (most recent call last):
File "/usr/local/Cellar/aws-sam-cli/0.14.2/libexec/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 39, in execute
self.scratch_dir
File "/usr/local/Cellar/aws-sam-cli/0.14.2/libexec/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 143, in build_dependencies
requirements_path, artifacts_dir_path, scratch_dir_path)
File "/usr/local/Cellar/aws-sam-cli/0.14.2/libexec/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 212, in build_site_packages
raise MissingDependencyError(packages_without_wheels)
aws_lambda_builders.workflows.python_pip.packager.MissingDependencyError: {sqlalchemy==1.3.3(sdist)}
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/Cellar/aws-sam-cli/0.14.2/libexec/lib/python3.7/site-packages/aws_lambda_builders/workflow.py", line 236, in run
action.execute()
File "/usr/local/Cellar/aws-sam-cli/0.14.2/libexec/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 42, in execute
raise ActionFailedError(str(ex))
aws_lambda_builders.actions.ActionFailedError: {sqlalchemy==1.3.3(sdist)}
Build Failed
Error: PythonPipBuilder:ResolveDependencies - {sqlalchemy==1.3.3(sdist)}
Достопримечательность
calling pip download --only-binary=:all: --no-deps --platform manylinux1_x86_64 --implementation cp --abi cp37m --dest /var/folders/c4/jqqz7trx02l9w1qkbx8dzxvr0000gn/T/tmpgl5r1tqp sqlalchemy==1.3.3
Удаление параметра --only-binary=:all: позволит этой отдельной команде функционировать, однако я не знаю, поможет ли это.
Как я могу построить свою лямбда-функцию, которая включает зависимость sqlalchemy?