Apache Airflow PythonVirtualenvOperator Не удалось построить колесо для df2gspread - PullRequest
Новая
       3

Apache Airflow PythonVirtualenvOperator Не удалось построить колесо для df2gspread

0 голосов
/ 23 марта 2020

Я пытаюсь запланировать сценарий python, используя apache airflow, и мой сценарий требует дополнительных пакетов. Поэтому я использую PythonVirtualenvOperator, но скрипт завершает работу, пытаясь установить пакет df2gspread. Вот пример.

Содержимое DAG 

import time
from pprint import pprint

from airflow import DAG
from airflow.operators.python_operator import PythonVirtualenvOperator
from airflow.utils.dates import days_ago

args = {
    'owner': 'airflow',
    'start_date': days_ago(2),
}

dag = DAG(
    dag_id='test_example',
    default_args=args,
    schedule_interval=None,
    tags=['example']
)


def callable_virtualenv():
    from colorama import Fore
    from time import sleep
    import df2gspread
    print(Fore.RED + 'some red text')
    for i in range(10):
        print(i)
        sleep(1)
    print('Finished')


virtualenv_task = PythonVirtualenvOperator(
    task_id="virtualenv_python",
    python_callable=callable_virtualenv,
    requirements=[
        "colorama==0.4.0",
        "df2gspread==1.0.4",
    ],
    system_site_packages=False,
    python_version='3.7',
    dag=dag,
)

И результат работы этого DAG: 

ERROR: Failed building wheel for df2gspread

Colorama устанавливается без каких-либо проблем и в то же время df2gspread вызывает ошибку

UPD: вот полный вывод команды установки 

  Building wheels for collected packages: df2gspread
  Building wheel for df2gspread (setup.py): started
  Building wheel for df2gspread (setup.py): finished with status 'error'
  ERROR: Command errored out with exit status 1:
   command: /venv/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/_z/6v0wj4dj7kldx67qgnw4m3640000gn/T/pip-install-2xx3cr8h/df2gspread/setup.py'"'"'; __file__='"'"'/private/var/folders/_z/6v0wj4dj7kldx67qgnw4m3640000gn/T/pip-install-2xx3cr8h/df2gspread/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/_z/6v0wj4dj7kldx67qgnw4m3640000gn/T/pip-wheel-yu4021ye
       cwd: /private/var/folders/_z/6v0wj4dj7kldx67qgnw4m3640000gn/T/pip-install-2xx3cr8h/df2gspread/
  Complete output (6 lines):
  usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: setup.py --help [cmd1 cmd2 ...]
     or: setup.py --help-commands
     or: setup.py cmd --help

  error: invalid command 'bdist_wheel'
  ----------------------------------------
  ERROR: Failed building wheel for df2gspread
  Running setup.py clean for df2gspread
Failed to build df2gspread
Installing collected packages: colorama, argparse, uritemplate, httplib2, pyasn1, pyasn1-modules, rsa, six, oauth2client, google-api-python-client, chardet, urllib3, idna, certifi, requests, gspread, python-dateutil, pytz, numpy, pandas, df2gspread
  WARNING: The scripts pyrsa-decrypt, pyrsa-encrypt, pyrsa-keygen, pyrsa-priv2pub, pyrsa-sign and pyrsa-verify are installed in '/var/folders/_z/6v0wj4dj7kldx67qgnw4m3640000gn/T/venvcmzzx47t/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script chardetect is installed in '/var/folders/_z/6v0wj4dj7kldx67qgnw4m3640000gn/T/venvcmzzx47t/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The scripts f2py, f2py3 and f2py3.7 are installed in '/var/folders/_z/6v0wj4dj7kldx67qgnw4m3640000gn/T/venvcmzzx47t/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
    Running setup.py install for df2gspread: started
    Running setup.py install for df2gspread: finished with status 'done'
...