Я установил mesos / sparkcluster со следующей версией mesos 1.9.0 с искрой 2.4.5 на centos8. Я попытался запустить следующий spark-job / appl / spark / bin / spark-submit --verbose - кластер с режимом развертывания --executor-memory 1G - master mesos: //mesos.master: 7077 / appl / spark -shared / lib / pi.py 10
Все задания завершаются с кодом завершения 127.
sh: строка 0: cd: слишком много аргументов sh: bin / spark-submit : Нет такого файла или каталога
Может ли быть, что centos8 не поддерживается? Я понятия не имею больше. Большое спасибо заранее!
Вот журнал ошибок:
I0326 20:57:30.080410 66713 fetcher.cpp:562] Fetcher Info: {"cache_directory":"/tmp/mesos/fetch/mesos","items":[{"action":"BYPASS_CACHE","uri":{"cache":false,"extract":true,"value":"/appl/spark-shared/lib/pi.py"}},{"action":"BYPASS_CACHE","uri":{"cache":false,"extract":true,"value":"file:///appl/spark-shared/lib/spark-2.4.5-bin-hadoop2.7.tgz"}}],"sandbox_directory":"/appl/work/mesos/slaves/2489fe38-4b0d-4794-b720-20628066b58d-S0/frameworks/44d55039-16e0-4db4-b9d2-bbc73af27ff5-0000/executors/driver-20200326205729-0006/runs/5f5da7bf-4825-497e-a4fc-41cc3e5f6fa0","stall_timeout":{"nanoseconds":60000000000},"user":"mesos"}
I0326 20:57:30.101250 66713 fetcher.cpp:459] Fetching URI '/appl/spark-shared/lib/pi.py'
I0326 20:57:30.101300 66713 fetcher.cpp:290] Fetching '/appl/spark-shared/lib/pi.py' directly into the sandbox directory
W0326 20:57:30.105692 66713 fetcher.cpp:332] Copying instead of extracting resource from URI with 'extract' flag, because it does not seem to be an archive: /appl/spark-shared/lib/pi.py
I0326 20:57:30.105773 66713 fetcher.cpp:618] Fetched '/appl/spark-shared/lib/pi.py' to '/appl/work/mesos/slaves/2489fe38-4b0d-4794-b720-20628066b58d-S0/frameworks/44d55039-16e0-4db4-b9d2-bbc73af27ff5-0000/executors/driver-20200326205729-0006/runs/5f5da7bf-4825-497e-a4fc-41cc3e5f6fa0/pi.py'
I0326 20:57:30.105785 66713 fetcher.cpp:459] Fetching URI 'file:///appl/spark-shared/lib/spark-2.4.5-bin-hadoop2.7.tgz'
I0326 20:57:30.105790 66713 fetcher.cpp:290] Fetching 'file:///appl/spark-shared/lib/spark-2.4.5-bin-hadoop2.7.tgz' directly into the sandbox directory
I0326 20:57:31.888286 66713 fetcher.cpp:618] Fetched 'file:///appl/spark-shared/lib/spark-2.4.5-bin-hadoop2.7.tgz' to '/appl/work/mesos/slaves/2489fe38-4b0d-4794-b720-20628066b58d-S0/frameworks/44d55039-16e0-4db4-b9d2-bbc73af27ff5-0000/executors/driver-20200326205729-0006/runs/5f5da7bf-4825-497e-a4fc-41cc3e5f6fa0/spark-2.4.5-bin-hadoop2.7.tgz'
I0326 20:57:31.888345 66713 fetcher.cpp:623] Successfully fetched all URIs into '/appl/work/mesos/slaves/2489fe38-4b0d-4794-b720-20628066b58d-S0/frameworks/44d55039-16e0-4db4-b9d2-bbc73af27ff5-0000/executors/driver-20200326205729-0006/runs/5f5da7bf-4825-497e-a4fc-41cc3e5f6fa0'
Overwriting environment variable 'MESOS_NATIVE_JAVA_LIBRARY'
I0326 20:57:32.194804 66769 exec.cpp:164] Version: 1.9.0
I0326 20:57:32.219038 66779 exec.cpp:237] Executor registered on agent 2489fe38-4b0d-4794-b720-20628066b58d-S0
I0326 20:57:32.224376 66776 executor.cpp:190] Received SUBSCRIBED event
I0326 20:57:32.225698 66776 executor.cpp:194] Subscribed executor on mesos.master
I0326 20:57:32.226032 66776 executor.cpp:190] Received LAUNCH event
I0326 20:57:32.228232 66776 executor.cpp:722] Starting task driver-20200326205729-0006
I0326 20:57:32.255175 66776 executor.cpp:738] Forked command at 66782
sh: line 0: cd: too many arguments
sh: bin/spark-submit: No such file or directory
I0326 20:57:32.500557 66773 executor.cpp:1039] Command exited with status 127 (pid: 66782)
I0326 20:57:33.521730 66781 process.cpp:935] Stopped the socket accept loop