Ошибка выполнения при использовании XGBoostSageMakerEstimator из sagemaker_pyspark в AWS EMR Notebook - PullRequest
Новая
       50

Ошибка выполнения при использовании XGBoostSageMakerEstimator из sagemaker_pyspark в AWS EMR Notebook

0 голосов
/ 25 сентября 2019

Я пытаюсь использовать SageMaker Python SDK с PySpark на ноутбуке EMR (Jupyter).При попытке использовать XGBoostSageMakerEstimator, как показано ниже, 

from sagemaker_pyspark.algorithms import XGBoostSageMakerEstimator

xgboost_estimator = XGBoostSageMakerEstimator(
    sagemakerRole=IAMRole(someRoleArn),
    trainingInstanceType='ml.m4.xlarge',
    trainingInstanceCount=1,
    endpointInstanceType='ml.m4.xlarge',
    endpointInitialInstanceCount=1)

Я получаю следующую ошибку, из-за которой мне не удалось найти решение. 

Exception ignored in: <bound method JavaWrapper.__del__ of <sagemaker_pyspark.wrapper.ScalaMap object at 0x7fd3d9e96240>>
Traceback (most recent call last):
  File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/ml/wrapper.py", line 40, in __del__
AttributeError: 'ScalaMap' object has no attribute '_java_obj'
Exception ignored in: <bound method JavaWrapper.__del__ of <sagemaker_pyspark.wrapper.ScalaMap object at 0x7fd3d9e96240>>
Traceback (most recent call last):
  File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/ml/wrapper.py", line 40, in __del__
AttributeError: 'ScalaMap' object has no attribute '_java_obj'
Exception ignored in: <bound method JavaWrapper.__del__ of <sagemaker_pyspark.wrapper.Option object at 0x7fd3d9e9d3c8>>
Traceback (most recent call last):
  File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/ml/wrapper.py", line 40, in __del__
AttributeError: 'Option' object has no attribute '_java_obj'
Exception ignored in: <bound method JavaWrapper.__del__ of <sagemaker_pyspark.wrapper.Option object at 0x7fd3d9e9d128>>
Traceback (most recent call last):
  File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/ml/wrapper.py", line 40, in __del__
AttributeError: 'Option' object has no attribute '_java_obj'
Exception ignored in: <bound method JavaWrapper.__del__ of <sagemaker_pyspark.wrapper.Option object at 0x7fd3d9e9d0f0>>
Traceback (most recent call last):
  File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/ml/wrapper.py", line 40, in __del__
AttributeError: 'Option' object has no attribute '_java_obj'
Exception ignored in: <bound method JavaWrapper.__del__ of <sagemaker_pyspark.wrapper.Option object at 0x7fd3d9e9d080>>
Traceback (most recent call last):
  File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/ml/wrapper.py", line 40, in __del__
AttributeError: 'Option' object has no attribute '_java_obj'
Exception ignored in: <bound method JavaWrapper.__del__ of <sagemaker_pyspark.wrapper.Option object at 0x7fd3d9e96ef0>>
Traceback (most recent call last):
  File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/ml/wrapper.py", line 40, in __del__
AttributeError: 'Option' object has no attribute '_java_obj'

Любая помощь для устранения этой проблемыбудет принята с благодарностью.

Использование:

  • EMR (emr-5.26.0) Кластер с Spark 2.4.3
  • EMR Notebook, подключенный к кластеру
  • sagemaker_pyspark предустановлен с emr-5.26.0
...