Невозможно загрузить данные из pandas фрейма данных в таблицу AWS athena из-за «ошибки пакета импорта» - PullRequest
Новая
       23

Невозможно загрузить данные из pandas фрейма данных в таблицу AWS athena из-за «ошибки пакета импорта»

3 голосов
/ 14 марта 2020

Когда я пытаюсь импортировать пакет awswrangler в sagemaker, я получаю следующую ошибку 

import awswrangler as aws

Ошибка 

ImportError                               Traceback (most recent call last)
<ipython-input-9-cc67bb4c1dd7> in <module>
----> 1 import awswrangler as aws

/opt/app-root/lib/python3.6/site-packages/awswrangler/__init__.py in <module>
     15 from awswrangler.emr import EMR  # noqa
     16 from awswrangler.glue import Glue  # noqa
---> 17 from awswrangler.pandas import Pandas  # noqa
     18 from awswrangler.redshift import Redshift  # noqa
     19 from awswrangler.s3 import S3  # noqa

/opt/app-root/lib/python3.6/site-packages/awswrangler/pandas.py in <module>
     17 from boto3 import client  # type: ignore
     18 from botocore.exceptions import ClientError, HTTPClientError  # type: ignore
---> 19 from pandas.io.common import infer_compression  # type: ignore
     20 from pyarrow import parquet as pq  # type: ignore
     21 from s3fs import S3FileSystem  # type: ignore

ImportError: cannot import name 'infer_compression'

Мотыга, чтобы это исправить?

Ответы [ 2 ]

1 голос
/ 16 марта 2020

Вы, вероятно, испортили свою среду conda. У меня он отлично работает на экземпляре ноутбука fre sh со средой conda_ python3. 

sh-4.2$ source activate python3
(python3) sh-4.2$ pip install awswrangler

Collecting awswrangler
  Downloading https://files.pythonhosted.org/packages/e9/99/b3ba9811e1a5f346da484f2dff40924613ec481df5d463e30bc3fd71096e/awswrangler-0.3.2.tar.gz (61kB)
    100% |████████████████████████████████| 71kB 10.1MB/s ta 0:00:01
Collecting numpy~=1.18.1 (from awswrangler)
  Downloading https://files.pythonhosted.org/packages/62/20/4d43e141b5bc426ba38274933ef8e76e85c7adea2c321ecf9ebf7421cedf/numpy-1.18.1-cp36-cp36m-manylinux1_x86_64.whl (20.1MB)
    100% |████████████████████████████████| 20.2MB 2.4MB/s eta 0:00:01
Collecting pandas~=1.0.1 (from awswrangler)
  Downloading https://files.pythonhosted.org/packages/f9/b9/9ad570258ce4fe504bd23002154f9e6f09bf7110359d271e4ba1664f7281/pandas-1.0.2-cp36-cp36m-manylinux1_x86_64.whl (10.1MB)
    100% |████████████████████████████████| 10.1MB 6.7MB/s eta 0:00:01
Collecting pyarrow~=0.16.0 (from awswrangler)
  Downloading https://files.pythonhosted.org/packages/ba/10/93fad5849418eade4a4cd581f8cd27be1bbe51e18968ba1492140c887f3f/pyarrow-0.16.0-cp36-cp36m-manylinux1_x86_64.whl (62.9MB)
    100% |████████████████████████████████| 62.9MB 766kB/s eta 0:00:01    17% |█████▌                          | 10.7MB 61.0MB/s eta 0:00:01
Requirement already satisfied: botocore>=1.13.34 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from awswrangler) (1.14.14)
Requirement already satisfied: boto3>=1.10.34 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from awswrangler) (1.11.14)
Collecting s3fs~=0.4.0 (from awswrangler)
  Downloading https://files.pythonhosted.org/packages/72/5c/ec84c7ec49fde2c3b0d885ecae4504fa40fc77fef7684e9f2939c50f9b94/s3fs-0.4.0-py3-none-any.whl
Collecting tenacity~=6.0.0 (from awswrangler)
  Downloading https://files.pythonhosted.org/packages/bf/d4/8cab4b5239ddb62d950243abff9e88046bb61737ce3eee8694b3d748560c/tenacity-6.0.0-py2.py3-none-any.whl
Collecting pg8000~=1.13.2 (from awswrangler)
  Downloading https://files.pythonhosted.org/packages/16/32/ae895597e43bc968e0e3e63860e9932b851115457face0d06d7f451b71fc/pg8000-1.13.2-py3-none-any.whl
Collecting pymysql~=0.9.3 (from awswrangler)
  Downloading https://files.pythonhosted.org/packages/ed/39/15045ae46f2a123019aa968dfcba0396c161c20f855f11dea6796bcaae95/PyMySQL-0.9.3-py2.py3-none-any.whl (47kB)
    100% |████████████████████████████████| 51kB 27.9MB/s ta 0:00:01
Requirement already satisfied: python-dateutil>=2.6.1 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from pandas~=1.0.1->awswrangler) (2.7.3)
Requirement already satisfied: pytz>=2017.2 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from pandas~=1.0.1->awswrangler) (2018.4)
Requirement already satisfied: six>=1.0.0 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from pyarrow~=0.16.0->awswrangler) (1.11.0)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from botocore>=1.13.34->awswrangler) (0.9.4)
Requirement already satisfied: urllib3<1.26,>=1.20; python_version != "3.4" in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from botocore>=1.13.34->awswrangler) (1.23)
Requirement already satisfied: docutils<0.16,>=0.10 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from botocore>=1.13.34->awswrangler) (0.14)
Requirement already satisfied: s3transfer<0.4.0,>=0.3.0 in /home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages (from boto3>=1.10.34->awswrangler) (0.3.3)
Collecting fsspec>=0.6.0 (from s3fs~=0.4.0->awswrangler)
  Downloading https://files.pythonhosted.org/packages/dd/1f/7028dacd3c28f34ce48130aae73a88fa5cc27b6b0e494fcf2739f7954d9d/fsspec-0.6.2-py3-none-any.whl (62kB)
    100% |████████████████████████████████| 71kB 35.6MB/s ta 0:00:01
Collecting scramp==1.1.0 (from pg8000~=1.13.2->awswrangler)
  Downloading https://files.pythonhosted.org/packages/bb/ef/6bdba6756ba7ccb81187833504ebba0511af750a2d9beaa04e4b56c3974f/scramp-1.1.0-py3-none-any.whl
Building wheels for collected packages: awswrangler
  Running setup.py bdist_wheel for awswrangler ... done
  Stored in directory: /home/ec2-user/.cache/pip/wheels/13/1a/52/966931a7453271ead15b0df3ea440e2f435d453cab8f6e99e3
Successfully built awswrangler
Installing collected packages: numpy, pandas, pyarrow, fsspec, s3fs, tenacity, scramp, pg8000, pymysql, awswrangler
  Found existing installation: numpy 1.14.3
    Uninstalling numpy-1.14.3:
      Successfully uninstalled numpy-1.14.3
  Found existing installation: pandas 0.24.2
    Uninstalling pandas-0.24.2:
      Successfully uninstalled pandas-0.24.2
  Found existing installation: s3fs 0.1.5
    Uninstalling s3fs-0.1.5:
      Successfully uninstalled s3fs-0.1.5
Successfully installed awswrangler-0.3.2 fsspec-0.6.2 numpy-1.18.1 pandas-1.0.2 pg8000-1.13.2 pyarrow-0.16.0 pymysql-0.9.3 s3fs-0.4.0 scramp-1.1.0 tenacity-6.0.0
You are using pip version 10.0.1, however version 20.0.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

(python3) sh-4.2$ python
Python 3.6.5 |Anaconda, Inc.| (default, Apr 29 2018, 16:14:56)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import awswrangler
>>>
0 голосов
/ 18 апреля 2020

Просто обязательно перезапустите ноутбук после установки изнутри с помощью !pip install awswrangler

или используйте предлагаемый жизненный цикл в документах

...