Kafka Connect с Mysql с использованием Kafka версии 2.12-2.4.1 - PullRequest
Новая
       23

Kafka Connect с Mysql с использованием Kafka версии 2.12-2.4.1

0 голосов
/ 30 апреля 2020

Я совершенно новичок в Kafka, пытаюсь получить данные из Mysql с помощью Kafka, для этого я использовал два jars: kafka-connect-jdb c -5.3.1 и mysql -connector- java -8.0.17 хранится в папке lib Kafka (путь: .... \ kafka_2.12-2.4.1 \ libs), я использую Windows10 с java -8.

Я следовал этому уроку: https://www.youtube.com/watch?v=r7LUbtOFcQI

После того, как я начал выпускать соединитель ниже, проблема. Будем благодарны за любые решения. 

E:\2020\kafka_2.12-2.4.1>.\bin\windows\connect-distributed.bat .\config\connect-distributed.properties
[2020-04-30 17:46:49,773] WARN could not get type for name org.jboss.resource.adapter.jdbc.vendor.MySQLExceptionSorter from any class loader (org.reflections.Reflections)
org.reflections.ReflectionsException: could not get type for name org.jboss.resource.adapter.jdbc.vendor.MySQLExceptionSorter
        at org.reflections.ReflectionUtils.forName(ReflectionUtils.java:390)
        at org.reflections.Reflections.expandSuperTypes(Reflections.java:381)
        at org.reflections.Reflections.<init>(Reflections.java:126)
        at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader$InternalReflections.<init>(DelegatingClassLoader.java:428)
        at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanPluginPath(DelegatingClassLoader.java:327)
        at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanUrlsAndAddPlugins(DelegatingClassLoader.java:263)
        at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.initPluginLoader(DelegatingClassLoader.java:211)
        at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.initLoaders(DelegatingClassLoader.java:204)
        at org.apache.kafka.connect.runtime.isolation.Plugins.<init>(Plugins.java:60)
        at org.apache.kafka.connect.cli.ConnectDistributed.startConnect(ConnectDistributed.java:91)
        at org.apache.kafka.connect.cli.ConnectDistributed.main(ConnectDistributed.java:78)
Caused by: java.lang.ClassNotFoundException: org.jboss.resource.adapter.jdbc.vendor.MySQLExceptionSorter
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at org.reflections.ReflectionUtils.forName(ReflectionUtils.java:388)
        ... 10 more
[2020-04-30 17:46:49,837] WARN could not get type for name org.osgi.framework.BundleListener from any class loader (org.reflections.Reflections)
org.reflections.ReflectionsException: could not get type for name org.osgi.framework.BundleListener
        at org.reflections.ReflectionUtils.forName(ReflectionUtils.java:390)
        at org.reflections.Reflections.expandSuperTypes(Reflections.java:381)
        at org.reflections.Reflections.<init>(Reflections.java:126)
        at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader$InternalReflections.<init>(DelegatingClassLoader.java:428)
        at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanPluginPath(DelegatingClassLoader.java:327)
        at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanUrlsAndAddPlugins(DelegatingClassLoader.java:263)
        at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.initPluginLoader(DelegatingClassLoader.java:211)
        at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.initLoaders(DelegatingClassLoader.java:204)
        at org.apache.kafka.connect.runtime.isolation.Plugins.<init>(Plugins.java:60)
        at org.apache.kafka.connect.cli.ConnectDistributed.startConnect(ConnectDistributed.java:91)
        at org.apache.kafka.connect.cli.ConnectDistributed.main(ConnectDistributed.java:78)
Caused by: java.lang.ClassNotFoundException: org.osgi.framework.BundleListener
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at org.reflections.ReflectionUtils.forName(ReflectionUtils.java:388)
        ... 10 more
[2020-04-30 17:46:49,848] WARN could not get type for name io.netty.internal.tcnative.CertificateVerifier from any class loader (org.reflections.Reflections)
org.reflections.ReflectionsException: could not get type for name io.netty.internal.tcnative.CertificateVerifier

Почтовый запрос, который я отправляю 

{
    "name": "jdbc_source_mysql_03",
    "config": {
        "connector.class": "io.confluent.connect.jdbc.JdbcSourceConnector",
        "connection.url": "jdbc:mysql://localhost:3306/user",
        "connection.user": "root",
        "connection.password": "root",
        "topic.prefix": "mysql-02-",
        "table.whitelist": "test",
        "mode": "incrementing",
        "incrementing.column.name": "id",
        "poll.interval.ms": "1000",
        "timestamp.column.name": "modified",
        "tasks.max": "2",
        "name": "jdbc_source_mysql_03"
    },
    "tasks": [],
    "type": "source"
...