Я пытаюсь использовать кластер hdinsight для создания озера данных.Для первого шага я использую sqoop для импорта таблицы в озере.Sqoop может перечислять таблицы из моей базы данных, но когда я запускаю импорт, используя его, задание сокращения карты застревает на 0%
. Я уже пытался указать адрес привязки как 0.0.0.0 в mysql.cnf * 1003.*
Я получаю следующее исключение
19/04/29 08:16:48 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
19/04/29 08:16:48 INFO tool.CodeGenTool: Beginning code generation
19/04/29 08:16:51 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `ekocsp` AS t LIMIT 1
19/04/29 08:16:53 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `ekocsp` AS t LIMIT 1
19/04/29 08:16:54 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/hdp/2.6.5.3008-11/hadoop-mapreduce
Note: /tmp/sqoop-eko/compile/56e7ea96415499a807366357d7084a26/ekocsp.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
19/04/29 08:16:57 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-eko/compile/56e7ea96415499a807366357d7084a26/ekocsp.jar
19/04/29 08:16:57 WARN manager.MySQLManager: It looks like you are importing from mysql.
19/04/29 08:16:57 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
19/04/29 08:16:57 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
19/04/29 08:16:57 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
19/04/29 08:16:58 INFO mapreduce.ImportJobBase: Beginning import of ekocsp
19/04/29 08:17:00 INFO client.AHSProxy: Connecting to Application History server at headnodehost/10.0.0.13:10200
19/04/29 08:17:01 INFO client.RequestHedgingRMFailoverProxyProvider: Looking for the active RM in [rm1, rm2]...
19/04/29 08:17:01 INFO client.RequestHedgingRMFailoverProxyProvider: Found active RM [rm1]
19/04/29 08:17:11 INFO db.DBInputFormat: Using read commited transaction isolation
19/04/29 08:17:11 INFO mapreduce.JobSubmitter: number of splits:1
19/04/29 08:17:12 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1556524594135_0004
19/04/29 08:17:15 INFO impl.YarnClientImpl: Submitted application application_1556524594135_0004
19/04/29 08:17:15 INFO mapreduce.Job: The url to track the job: http://hn0-ekotes.04ynzcm1urqu3jeuzaoywk5wzh.cx.internal.cloudapp.net:8088/proxy/application_1556524594135_0004/
19/04/29 08:17:15 INFO mapreduce.Job: Running job: job_1556524594135_0004
19/04/29 08:17:32 INFO mapreduce.Job: Job job_1556524594135_0004 running in uber mode : false
19/04/29 08:17:32 INFO mapreduce.Job: map 0% reduce 0%
19/04/29 08:19:49 INFO mapreduce.Job: Task Id : attempt_1556524594135_0004_m_000000_0, Status : FAILED
Error: java.lang.RuntimeException: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:167)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:170)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:164)
Caused by: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:220)
at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:165)
... 9 more
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:409)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1127)
at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:356)
at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2502)
at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2539)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2321)
at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:832)
at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:46)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:409)
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:417)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:344)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:302)
at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:213)
... 10 more
Caused by: java.net.ConnectException: Connection timed out (Connection timed out)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at java.net.Socket.connect(Socket.java:538)
at java.net.Socket.<init>(Socket.java:434)
at java.net.Socket.<init>(Socket.java:244)
at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:258)
at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:306)
... 26 more
19/04/29 08:20:33 INFO mapreduce.Job: Job job_1556524594135_0004 failed with state KILLED due to: Application killed by user.
19/04/29 08:20:33 INFO mapreduce.Job: Counters: 0
19/04/29 08:20:33 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
19/04/29 08:20:33 INFO mapreduce.ImportJobBase: Transferred 0 bytes in 214.1728 seconds (0 bytes/sec)
19/04/29 08:20:33 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
19/04/29 08:20:33 INFO mapreduce.ImportJobBase: Retrieved 0 records.
19/04/29 08:20:33 ERROR tool.ImportTool: Error during import: Import job failed!