Я могу войти в jupyterhub, однако, когда я запускаю сервер, я получаю следующее:
start-jupyterhub.sh: line 1: /opt/jupyterhub/miniconda/etc: Is a directory
[I 2020-05-07 10:35:11.567 JupyterHub app:2240] Running JupyterHub version 1.1.0
[I 2020-05-07 10:35:11.568 JupyterHub app:2271] Using Authenticator: dummyauthenticator.dummyauthenticator.DummyAuthenticator
[I 2020-05-07 10:35:11.568 JupyterHub app:2271] Using Spawner: yarnspawner.spawner.YarnSpawner-0.4.0
[I 2020-05-07 10:35:11.568 JupyterHub app:2271] Using Proxy: jupyterhub.proxy.ConfigurableHTTPProxy-1.1.0
[I 2020-05-07 10:35:11.703 JupyterHub app:1349] Loading cookie_secret from /etc/jupyterhub/jupyterhub_cookie_secret
[I 2020-05-07 10:35:12.005 JupyterHub proxy:461] Generating new CONFIGPROXY_AUTH_TOKEN
[I 2020-05-07 10:35:12.053 JupyterHub app:1655] Not using whitelist. Any authenticated user will be allowed.
10
[I 2020-05-07 10:35:12.129 JupyterHub app:2311] Initialized 0 spawners in 0.037 seconds
[W 2020-05-07 10:35:12.131 JupyterHub proxy:517] Found proxy pid file: /var/jupyterhub/jupyterhub-proxy.pid
[W 2020-05-07 10:35:12.151 JupyterHub proxy:529] Proxy no longer running at pid=63909
[W 2020-05-07 10:35:12.172 JupyterHub proxy:643] Running JupyterHub without SSL. I hope there is SSL termination happening somewhere else...
[I 2020-05-07 10:35:12.172 JupyterHub proxy:646] Starting proxy @ http://127.0.0.1:8090
10:35:14.717 [ConfigProxy] info: Proxying http://127.0.0.1:8090 to (no default)
10:35:14.795 [ConfigProxy] info: Proxy API at http://127.0.0.1:8001/api/routes
10:35:15.003 [ConfigProxy] info: 200 GET /api/routes
[I 2020-05-07 10:35:15.039 JupyterHub app:2556] Hub API listening on http://*:8081/hub/
[I 2020-05-07 10:35:15.039 JupyterHub app:2558] Private Hub API connect url http://hadoop-master:8081/hub/
10:35:15.041 [ConfigProxy] info: 200 GET /api/routes
[I 2020-05-07 10:35:15.042 JupyterHub proxy:320] Checking routes
[I 2020-05-07 10:35:15.042 JupyterHub proxy:400] Adding default route for Hub: / => http://hadoop-master:8081
10:35:15.045 [ConfigProxy] info: Adding route / -> http://hadoop-master:8081
10:35:15.046 [ConfigProxy] info: Route added / -> http://hadoop-master:8081
[I 2020-05-07 10:35:15.047 JupyterHub app:2631] JupyterHub is now running at http://127.0.0.1:8090
10:35:15.048 [ConfigProxy] info: 201 POST /api/routes/
[I 2020-05-07 10:36:17.252 JupyterHub log:174] 302 GET / -> /hub/ (@127.0.0.1) 3.80ms
[I 2020-05-07 10:36:19.068 JupyterHub log:174] 302 GET /hub/ -> /hub/spawn (hadoopuser@127.0.0.1) 114.49ms
[W 2020-05-07 10:36:19.388 JupyterHub spawner:946] Setting port from user.server is deprecated as of JupyterHub 0.7.
[I 2020-05-07 10:36:21.998 JupyterHub log:174] 302 GET /hub/spawn -> /hub/spawn-pending/hadoopuser (hadoopuser@127.0.0.1) 2721.90ms
[I 2020-05-07 10:36:22.655 JupyterHub pages:347] hadoopuser is pending spawn
[I 2020-05-07 10:36:23.406 JupyterHub log:174] 200 GET /hub/spawn-pending/hadoopuser (hadoopuser@127.0.0.1) 760.24ms
10:36:24.645 [ConfigProxy] error: 503 GET /hub/static/components/font-awesome/fonts/fontawesome-webfont.woff2?v=4.7.0 socket hang up
[I 2020-05-07 10:36:24.710 JupyterHub log:174] 200 GET /hub/error/503?url=%2Fhub%2Fstatic%2Fcomponents%2Ffont-awesome%2Ffonts%2Ffontawesome-webfont.woff2%3Fv%3D4.7.0 (@192.168.126.129) 43.54ms
10:36:24.772 [ConfigProxy] error: 503 GET /favicon.ico socket hang up
[I 2020-05-07 10:36:24.801 JupyterHub log:174] 200 GET /hub/error/503?url=%2Ffavicon.ico (@192.168.126.129) 7.77ms
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/conf/YarnConfiguration
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.conf.YarnConfiguration
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 7 more
[E 2020-05-07 10:36:26.567 JupyterHub user:640] Unhandled error starting hadoopuser's server: Failed to start java process
ERROR:asyncio:Task exception was never retrieved
future: <Task finished coro=<BaseHandler.spawn_single_user() done, defined at /opt/jupyterhub/miniconda/lib/python3.7/site-packages/jupyterhub/handlers/base.py:741> exception=DriverError('Failed to start java process')>
Traceback (most recent call last):
File "/opt/jupyterhub/miniconda/lib/python3.7/site-packages/jupyterhub/handlers/base.py", line 939, in spawn_single_user
timedelta(seconds=self.slow_spawn_timeout), finish_spawn_future
File "/opt/jupyterhub/miniconda/lib/python3.7/site-packages/jupyterhub/handlers/base.py", line 852, in finish_user_spawn
await spawn_future
File "/opt/jupyterhub/miniconda/lib/python3.7/site-packages/jupyterhub/user.py", line 656, in spawn
raise e
File "/opt/jupyterhub/miniconda/lib/python3.7/site-packages/jupyterhub/user.py", line 560, in spawn
url = await gen.with_timeout(timedelta(seconds=spawner.start_timeout), f)
File "/opt/jupyterhub/miniconda/lib/python3.7/site-packages/yarnspawner/spawner.py", line 208, in start
client = await self._get_client()
File "/opt/jupyterhub/miniconda/lib/python3.7/site-packages/yarnspawner/spawner.py", line 146, in _get_client
None, lambda: skein.Client(**kwargs)
File "/opt/jupyterhub/miniconda/lib/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/opt/jupyterhub/miniconda/lib/python3.7/site-packages/yarnspawner/spawner.py", line 146, in <lambda>
None, lambda: skein.Client(**kwargs)
File "/opt/jupyterhub/miniconda/lib/python3.7/site-packages/skein/core.py", line 343, in __init__
java_options=java_options)
File "/opt/jupyterhub/miniconda/lib/python3.7/site-packages/skein/core.py", line 256, in _start_driver
raise DriverError("Failed to start java process")
skein.exceptions.DriverError: Failed to start java process
[I 2020-05-07 10:36:26.725 JupyterHub log:174] 200 GET /hub/api/users/hadoopuser/server/progress (hadoopuser@127.0.0.1) 2059.16ms
Это мой сайт пряжи. xml
GNU nano 2.9.3 yarn-site.xml
limitations under the License. See accompanying LICENSE file.
-->
<configuration>
<property>
<description>CLASSPATH for YARN applications. A comma-separated list o$
<name>yarn.application.classpath</name>
<value>
/usr/local/hadoop/etc/hadoop/,
/usr/local/hadoop/share/hadoop/common/lib/*,
/usr/local/hadoop/share/hadoop/common/*,
/usr/local/hadoop/share/hadoop/hdfs,
/usr/local/hadoop/share/hadoop/hdfs/lib/*,
/usr/local/hadoop/share/hadoop/hdfs/*,
/usr/local/hadoop/share/hadoop/mapreduce/lib/*,
/usr/local/hadoop/share/hadoop/mapreduce/*,
/usr/local/hadoop/share/hadoop/yarn,
/usr/local/hadoop/share/hadoop/yarn/lib/*,
/usr/local/hadoop/share/hadoop/yarn/*
</value>
</property>
</configuration>
Вот что происходит когда я запускаю 'yarn classpath' на одном из моих узлов:
/usr/local/hadoop/etc/hadoop:/usr/local/hadoop/share/hadoop/common/lib/*:/usr/local/hadoop/share/hadoop/common/*:/usr/local/hadoop/share/hadoop/hdfs:/usr/local/hadoop/share/hadoop/hdfs/lib/*:/usr/local/hadoop/share/hadoop/hdfs/*:/usr/local/hadoop/share/hadoop/mapreduce/lib/*:/usr/local/hadoop/share/hadoop/mapreduce/*:/usr/local/hadoop/share/hadoop/yarn:/usr/local/hadoop/share/hadoop/yarn/lib/*:/usr/local/hadoop/share/hadoop/yarn/*
Я искал буквально повсюду и не могу понять этого! Если вы ответите, пожалуйста, постарайтесь сделать это как можно более нетехническим, поскольку я новичок в этом! :)