У меня есть модель doc2ve c, которая управляет моим рекомендательным приложением. Я построил модель doc2ve c и сохранил ее в ведре s3. Теперь, когда я открываю веб-приложение, модель должна быть загружена обратно из s3, но этого не происходит. Я использовал AWS Elasticbean stalk для развертывания своего веб-приложения. Итак, я проверил журналы Elasticbean stalk на наличие ошибок. Это ошибка, которую он показывает мне, когда я пытаюсь развернуть
-------------------------------------
/var/log/httpd/error_log
-------------------------------------
[Fri May 22 12:53:57.768095 2020] [:error] [pid 8177] [remote 172.31.23.26:48093] File "/usr/lib64/python3.6/pickle.py", line 1338, in load_global
[Fri May 22 12:53:57.768098 2020] [:error] [pid 8177] [remote 172.31.23.26:48093] klass = self.find_class(module, name)
[Fri May 22 12:53:57.768104 2020] [:error] [pid 8177] [remote 172.31.23.26:48093] File "/usr/lib64/python3.6/pickle.py", line 1388, in find_class
[Fri May 22 12:53:57.768107 2020] [:error] [pid 8177] [remote 172.31.23.26:48093] __import__(module, level=0)
[Fri May 22 12:53:57.768127 2020] [:error] [pid 8177] [remote 172.31.23.26:48093] ModuleNotFoundError: No module named 'numpy.random._pickle'
[Fri May 22 12:53:57.768138 2020] [:error] [pid 8177] [remote 172.31.23.26:48093]
[Fri May 22 12:53:57.768141 2020] [:error] [pid 8177] [remote 172.31.23.26:48093] During handling of the above exception, another exception occurred:
[Fri May 22 12:53:57.768144 2020] [:error] [pid 8177] [remote 172.31.23.26:48093]
[Fri May 22 12:53:57.768150 2020] [:error] [pid 8177] [remote 172.31.23.26:48093] Traceback (most recent call last):
[Fri May 22 12:53:57.768177 2020] [:error] [pid 8177] [remote 172.31.23.26:48093] File "/opt/python/current/app/application.py", line 81, in <module>
[Fri May 22 12:53:57.768181 2020] [:error] [pid 8177] [remote 172.31.23.26:48093] model_d2v = d2v.load_d2v('doc2vec_model', ENV)
[Fri May 22 12:53:57.768187 2020] [:error] [pid 8177] [remote 172.31.23.26:48093] File "/opt/python/current/app/microservices/d2v_services.py", line 21, in load_d2v
[Fri May 22 12:53:57.768190 2020] [:error] [pid 8177] [remote 172.31.23.26:48093] model=joblib.load(model_name)
[Fri May 22 12:53:57.768196 2020] [:error] [pid 8177] [remote 172.31.23.26:48093] File "/opt/python/run/venv/local/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 605, in load
[Fri May 22 12:53:57.768199 2020] [:error] [pid 8177] [remote 172.31.23.26:48093] obj = _unpickle(fobj, filename, mmap_mode)
[Fri May 22 12:53:57.768204 2020] [:error] [pid 8177] [remote 172.31.23.26:48093] File "/opt/python/run/venv/local/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 529, in _unpickle
[Fri May 22 12:53:57.768208 2020] [:error] [pid 8177] [remote 172.31.23.26:48093] obj = unpickler.load()
[Fri May 22 12:53:57.768213 2020] [:error] [pid 8177] [remote 172.31.23.26:48093] File "/usr/lib64/python3.6/pickle.py", line 1050, in load
[Fri May 22 12:53:57.768216 2020] [:error] [pid 8177] [remote 172.31.23.26:48093] dispatch[key[0]](self)
[Fri May 22 12:53:57.768222 2020] [:error] [pid 8177] [remote 172.31.23.26:48093] File "/usr/lib64/python3.6/pickle.py", line 1338, in load_global
[Fri May 22 12:53:57.768230 2020] [:error] [pid 8177] [remote 172.31.23.26:48093] klass = self.find_class(module, name)
[Fri May 22 12:53:57.768237 2020] [:error] [pid 8177] [remote 172.31.23.26:48093] File "/usr/lib64/python3.6/pickle.py", line 1388, in find_class
[Fri May 22 12:53:57.768240 2020] [:error] [pid 8177] [remote 172.31.23.26:48093] __import__(module, level=0)
[Fri May 22 12:53:57.768250 2020] [:error] [pid 8177] [remote 172.31.23.26:48093] ModuleNotFoundError: No module named 'numpy.random._pickle'
[Fri May 22 12:54:07.192783 2020] [:error] [pid 8177] loading...doc2vec_model
[Fri May 22 12:54:12.237091 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] mod_wsgi (pid=8177): Target WSGI script '/opt/python/current/app/application.py' cannot be loaded as Python module.
[Fri May 22 12:54:12.237334 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] mod_wsgi (pid=8177): Exception occurred processing WSGI script '/opt/python/current/app/application.py'.
[Fri May 22 12:54:12.238604 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] Traceback (most recent call last):
[Fri May 22 12:54:12.238658 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] File "/opt/python/current/app/microservices/d2v_services.py", line 14, in load_d2v
[Fri May 22 12:54:12.238664 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] model=joblib.load(model_name)
[Fri May 22 12:54:12.238675 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] File "/opt/python/run/venv/local/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 605, in load
[Fri May 22 12:54:12.238679 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] obj = _unpickle(fobj, filename, mmap_mode)
[Fri May 22 12:54:12.238686 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] File "/opt/python/run/venv/local/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 529, in _unpickle
[Fri May 22 12:54:12.238690 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] obj = unpickler.load()
[Fri May 22 12:54:12.238695 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] File "/usr/lib64/python3.6/pickle.py", line 1050, in load
[Fri May 22 12:54:12.238699 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] dispatch[key[0]](self)
[Fri May 22 12:54:12.238705 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] File "/usr/lib64/python3.6/pickle.py", line 1338, in load_global
[Fri May 22 12:54:12.238709 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] klass = self.find_class(module, name)
[Fri May 22 12:54:12.238714 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] File "/usr/lib64/python3.6/pickle.py", line 1388, in find_class
[Fri May 22 12:54:12.238718 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] __import__(module, level=0)
[Fri May 22 12:54:12.238737 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] ModuleNotFoundError: No module named 'numpy.random._pickle'
[Fri May 22 12:54:12.238793 2020] [:error] [pid 8177] [remote 172.31.15.191:48093]
[Fri May 22 12:54:12.238800 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] During handling of the above exception, another exception occurred:
[Fri May 22 12:54:12.238803 2020] [:error] [pid 8177] [remote 172.31.15.191:48093]
[Fri May 22 12:54:12.238810 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] Traceback (most recent call last):
[Fri May 22 12:54:12.238840 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] File "/opt/python/current/app/application.py", line 81, in <module>
[Fri May 22 12:54:12.238845 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] model_d2v = d2v.load_d2v('doc2vec_model', ENV)
[Fri May 22 12:54:12.238851 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] File "/opt/python/current/app/microservices/d2v_services.py", line 21, in load_d2v
[Fri May 22 12:54:12.238855 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] model=joblib.load(model_name)
[Fri May 22 12:54:12.238872 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] File "/opt/python/run/venv/local/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 605, in load
[Fri May 22 12:54:12.238883 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] obj = _unpickle(fobj, filename, mmap_mode)
[Fri May 22 12:54:12.238889 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] File "/opt/python/run/venv/local/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 529, in _unpickle
[Fri May 22 12:54:12.238892 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] obj = unpickler.load()
[Fri May 22 12:54:12.238897 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] File "/usr/lib64/python3.6/pickle.py", line 1050, in load
[Fri May 22 12:54:12.238901 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] dispatch[key[0]](self)
[Fri May 22 12:54:12.238906 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] File "/usr/lib64/python3.6/pickle.py", line 1338, in load_global
[Fri May 22 12:54:12.238909 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] klass = self.find_class(module, name)
[Fri May 22 12:54:12.238914 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] File "/usr/lib64/python3.6/pickle.py", line 1388, in find_class
[Fri May 22 12:54:12.238917 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] __import__(module, level=0)
[Fri May 22 12:54:12.238928 2020] [:error] [pid 8177] [remote 172.31.15.191:48093] ModuleNotFoundError: No module named 'numpy.random._pickle'
[Fri May 22 12:56:22.918445 2020] [mpm_prefork:notice] [pid 8172] AH00169: caught SIGTERM, shutting down
Я попытался импортировать 'numpy .random._pickle', но это не сработало.
Код, который я использовал для загрузки модели:
import subprocess
from sklearn.externals import joblib
import gensim
model = load_d2v("doc2vec_model")
def load_d2v(fname):
model_name = fname
s3_base_path='s3://sd-flikku/datalake/current_doc2vec_model'
path = s3_base_path+'/'+model_name
command = "aws s3 cp {} {}".format(path,model_name).split()
print('loading...'+model_name)
subprocess.call(command)
model=joblib.load(model_name)
return model
Не могли бы вы мне с этим помочь? Заранее спасибо.