Запуск featuretools dfs с n_jobs> 1, иногда дает мне (иногда запускается без ошибки) tornado.iostream.StreamClosedError: Stream is closed.
Я хочу запустить featuretools dfs n_job> = 2 без ошибок в центах linux
ошибка полного стека ниже:
Traceback (most recent call last):
File "mlp-platform/python/component/automl/test/automl_full_test.py", line 113, in <module>
result1, features = FeatureEvolutionFeatureTools(params=params).run()
File "/yarn/mlp-platform/python/component/automl/feature_evolution.py", line 42, in run
trans_primitives=transformer_name, **dfs_init_option)
File "/yarn/mlp-platform/python/component/automl/feature_evolution.py", line 54, in dfs_wrapper
tornado.application - ERROR - Exception in callback functools.partial(<function wrap.<locals>.null_wrapper at 0x2ba94b588d08>, <Future finished exception=StreamClosedError('Stream is closed',)>)
Traceback (most recent call last):
File "/usr/lib64/python3.6/site-packages/tornado/gen.py", line 1141, in run
yielded = self.gen.throw(*exc_info)
File "/usr/lib64/python3.6/site-packages/tornado/tcpclient.py", line 232, in connect
af, addr, stream = yield connector.start(connect_timeout=timeout)
File "/usr/lib64/python3.6/site-packages/tornado/gen.py", line 1133, in run
value = future.result()
File "/usr/lib64/python3.6/site-packages/tornado/tcpclient.py", line 112, in on_connect_done
stream = future.result()
tornado.iostream.StreamClosedError: Stream is closed
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.6/site-packages/tornado/ioloop.py", line 758, in _run_callback
ret = callback()
File "/usr/lib64/python3.6/site-packages/tornado/stack_context.py", line 300, in null_wrapper
return fn(*args, **kwargs)
File "/usr/lib64/python3.6/site-packages/tornado/gen.py", line 1233, in inner
self.run()
File "/usr/lib64/python3.6/site-packages/tornado/gen.py", line 1173, in run
future_set_exc_info(self.result_future, sys.exc_info())
File "/usr/lib64/python3.6/site-packages/tornado/concurrent.py", line 643, in future_set_exc_info
future.set_exception(exc_info[1])
asyncio.base_futures.InvalidStateError: invalid state return func(**kwargs)
File "/usr/lib/python3.6/site-packages/featuretools/utils/entry_point.py", line 40, in function_wrapper
raise e
File "/usr/lib/python3.6/site-packages/featuretools/utils/entry_point.py", line 32, in function_wrapper
return_value = func(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/featuretools/synthesis/dfs.py", line 276, in dfs
progress_callback=progress_callback)
File "/usr/lib/python3.6/site-packages/featuretools/computational_backends/calculate_feature_matrix.py", line 266, in calculate_feature_matrix
progress_callback=progress_callback)
File "/usr/lib/python3.6/site-packages/featuretools/computational_backends/calculate_feature_matrix.py", line 534, in parallel_calculate_chunks
client.replicate([_es, _saved_features])
File "/usr/lib/python3.6/site-packages/distributed/client.py", line 3078, in replicate
**kwargs
File "/usr/lib/python3.6/site-packages/distributed/client.py", line 767, in sync
self.loop, func, *args, callback_timeout=callback_timeout, **kwargs
File "/usr/lib/python3.6/site-packages/distributed/utils.py", line 345, in sync
raise exc.with_traceback(tb)
File "/usr/lib/python3.6/site-packages/distributed/utils.py", line 329, in f
result[0] = yield future
File "/usr/lib64/python3.6/site-packages/tornado/gen.py", line 1133, in run
value = future.result()
concurrent.futures._base.CancelledError
Я разобрался ниже
стереть код в featuretools calc_feature_matrix.py, 595 строк
if client is not None:
client.close()
без ошибок
- с n_jobs = 1, без ошибок
- в окне, без ошибок
Я использую
featuretools 0.13.2
python 3.6.8