AttributeError: модуль '__main__' не имеет атрибута 'text_process' - PullRequest
Новая
       36

AttributeError: модуль '__main__' не имеет атрибута 'text_process'

0 голосов
/ 24 января 2019

Контекст

Я развертываю свою модель ML на Lambda; после успешного развертывания, когда я нажимаю URL-адрес шлюза, я получаю 500 ответов, и журналы 

Traceback (most recent call last):
  File "/tmp/serverless-ml/flask/app.py", line 2292, in wsgi_app
  response = self.full_dispatch_request()
  File "/tmp/serverless-ml/flask/app.py", line 1815, in full_dispatch_request
  rv = self.handle_user_exception(e)
  File "/tmp/serverless-ml/flask/app.py", line 1718, in handle_user_exception
  reraise(exc_type, exc_value, tb)
  File "/tmp/serverless-ml/flask/_compat.py", line 35, in reraise
  raise value
  File "/tmp/serverless-ml/flask/app.py", line 1813, in full_dispatch_request
  rv = self.dispatch_request()
  File "/tmp/serverless-ml/flask/app.py", line 1799, in dispatch_request
  return self.view_functions[rule.endpoint](**req.view_args)
  File "/tmp/serverless-ml/api/app.py", line 35, in index
  prediction = predict(data)
  File "/tmp/serverless-ml/api/app.py", line 54, in predict
  model = load_model(MODEL_FILE_NAME)
  File "/tmp/serverless-ml/api/app.py", line 22, in helper
  memo[x] = f(x)
  File "/tmp/serverless-ml/api/app.py", line 48, in load_model
  model = joblib.load(open(key, 'rb'))
  File "/tmp/serverless-ml/sklearn/externals/joblib/numpy_pickle.py", line 588, in load
  obj = _unpickle(fobj)
  File "/tmp/serverless-ml/sklearn/externals/joblib/numpy_pickle.py", line 526, in _unpickle
  obj = unpickler.load()
  File "/var/lang/lib/python3.6/pickle.py", line 1050, in load
  dispatch[key[0]](self)
  File "/var/lang/lib/python3.6/pickle.py", line 1338, in load_global
  klass = self.find_class(module, name)
  File "/var/lang/lib/python3.6/pickle.py", line 1392, in find_class
  return getattr(sys.modules[module], name)
AttributeError: module '__main__' has no attribute 'text_process'

модель трубопровода требует text_process

Pipeline(memory=None,
     steps=[('bow', CountVectorizer(analyzer=<function text_process at 0x110cbb6a8>, binary=False,
        decode_error='strict', dtype=<class 'numpy.int64'>,
        encoding='utf-8', input='content', lowercase=True, max_df=1.0,
        max_features=None, min_df=1, ngram_range=(1, 1), preprocessor=None,...f=False, use_idf=True)), ('classifier', MultinomialNB(alpha=1.0, class_prior=None, fit_prior=True))])

Я использую Python 3.6

Ожидаемое поведение

Нет ошибки

Фактическое поведение

AttributeError: модуль ' main ' не имеет атрибута 'text_process'

Возможное исправление

Понятия не имею!

Шаги для воспроизведения

api/app.py

from flask import Flask
from flask import request
from flask import json
import boto3
import pickle
from sklearn.externals import joblib

from nltk.corpus import stopwords
import string

def text_process(mess):
    mess=mess.lower()
    nopunc =[char for char in mess if char not in string.punctuation]
    nopunc=''.join(nopunc)
    return [word for word in nopunc.split() if word.lower() not in stopwords.words('english')]

MODEL_FILE_NAME = 'models/pipeline.pkl'

app = Flask(__name__)

#S3 = boto3.client('s3', region_name='eu-central-1')


def memoize(f):
    memo = {}

    def helper(x):
        if x not in memo:
            memo[x] = f(x)
        return memo[x]

    return helper



@app.route('/', methods=['POST'])
def index():
    print(request.get_json(silent=True));
    body_dict = request.get_json(silent=True)
    data = body_dict['data']

    prediction = predict(data)

    result = {'prediction': prediction}
    return json.dumps(result)


@memoize
def load_model(key):

    model = joblib.load(open(key, 'rb'))

    return model


def predict(data):
    model = load_model(MODEL_FILE_NAME)

    return model.predict(data).tolist()


if __name__ == '__main__':
    # listen on all IPs
    app.run(host='0.0.0.0')

код отлично работает на моей локальной машине

Ваше окружение

* Используемая версия Zappa: * Операционная система и версия Python: * Выход pip freeze: 

boto3==1.9.83
botocore==1.12.83
certifi==2018.11.29
cfn-flip==1.1.0.post1
chardet==3.0.4
Click==7.0
docutils==0.14
durationpy==0.5
Flask==1.0.2
future==0.16.0
hjson==3.0.1
idna==2.8
itsdangerous==1.1.0
Jinja2==2.10
jmespath==0.9.3
kappa==0.6.0
lambda-packages==0.20.0
MarkupSafe==1.1.0
nltk==3.4
numpy==1.16.0
placebo==0.8.2
python-dateutil==2.6.1
python-slugify==1.2.4
PyYAML==3.13
requests==2.21.0
s3transfer==0.1.13
scikit-learn==0.20.2
scipy==1.2.0
singledispatch==3.4.0.3
six==1.12.0
sklearn==0.0
toml==0.10.0
tqdm==4.19.1
troposphere==2.4.1
Unidecode==1.0.23
urllib3==1.24.1
Werkzeug==0.14.1
wsgi-request-logger==0.4.6
zappa==0.47.1
  • Ссылка на ваш проект (необязательно):
  • Ваш zappa_settings.py: 
{
    "dev": {
        "app_function": "api.app.app",
        "aws_region": ".....",
        "profile_name": "default",
        "project_name": "serverless-ml",
        "runtime": "python3.6",
        "slim_handler": true,
        "s3_bucket": "zappa-...."
    }
}
...