Невозможно создать какие-либо журналы при запуске потребителя kafka после установки cd c на sql server 2019 и регистрации debezium sql server connector

Question

Раньше я уже устанавливал CD C на SQL сервере на основе этой ссылки и регистрировал Debezium SQL серверный коннектор для подключения Kafka на основе этой ссылки . Я запускаю сценарий потребителя Kafka и делаю некоторые данные модификации (вставка, обновление и удаление) на сервере SQL, но потребитель Kafka не отображает никаких JSON журналов о данных модификации.

Это мой файл конфигурации sqlserver-debezium.json :

{
   "name":"sqlserver-debezium",
   "config":{
      "connector.class":"io.debezium.connector.sqlserver.SqlServerConnector",
      "database.user":"sa",
      "database.dbname":"cdc_sqlserver",
      "database.history.kafka.bootstrap.servers":"10.140.0.33:9092",
      "database.history.kafka.topic":"debezium_sqlserver",
      "database.server.name":"sqlserver-debezium",
      "database.port":"1433",
      "table.whitelist":"cdc_sqlserver_ipul.dbo.customers",
      "database.hostname":"10.140.15.214",
      "database.password":"test123",
      "name":"sqlserver-debezium",
      "include.query":"true",
      "snapshot.mode":"when_needed"
   }
}

Я добавляю коннектор через хост-машину следующим образом (точно так же, как в руководстве):

curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" localhost:8083/connectors/ -d @sqlserver-debezium.json

Запуск скрипта Kafka consumer:

kafka-console-consumer --bootstrap-server localhost:9092 --topic sqlserver-connector --from-beginning

Буду признателен за любую помощь.

Answer 1

Эта проблема решена. Итак, я изменяю значение переменной snapshot.mode на initial, удаляю переменную include.query в sqlserver-debezium. json, повторяю попытку соединителя регистра и делаю некоторые изменения (вставка, обновление, удаление) в базе данных сервера sql, затем отображаю json такие журналы

{"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":true,"field":"first_name"
},{"type":"string","optional":true,"field":"last_name"},{"type":"string","optional":true,"field":"email"}],"optional":true,"name":"btpns_sqlserver_ipul.dbo.customer
s.Value","field":"before"},{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":true,"field":"first_name"},{"type":
"string","optional":true,"field":"last_name"},{"type":"string","optional":true,"field":"email"}],"optional":true,"name":"btpns_sqlserver_ipul.dbo.customers.Value","
field":"after"},{"type":"struct","fields":[{"type":"string","optional":false,"field":"version"},{"type":"string","optional":false,"field":"connector"},{"type":"stri
ng","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"ts_ms"},{"type":"string","optional":true,"name":"io.debezium.data.Enum","version":1,"
parameters":{"allowed":"true,last,false"},"default":"false","field":"snapshot"},{"type":"string","optional":false,"field":"db"},{"type":"string","optional":false,"f
ield":"schema"},{"type":"string","optional":false,"field":"table"},{"type":"string","optional":true,"field":"change_lsn"},{"type":"string","optional":true,"field":"
commit_lsn"},{"type":"int64","optional":true,"field":"event_serial_no"}],"optional":false,"name":"io.debezium.connector.sqlserver.Source","field":"source"},{"type":
"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"},{"type":"struct","fields":[{"type":"string","optional":false,"field":"id"},
{"type":"int64","optional":false,"field":"total_order"},{"type":"int64","optional":false,"field":"data_collection_order"}],"optional":true,"field":"transaction"}],"
optional":false,"name":"btpns_sqlserver_ipul.dbo.customers.Envelope"},"payload":{"before":{"id":9,"first_name":"Andi","last_name":"Thompson","email":"kitt@acme.com"
},"after":{"id":9,"first_name":"Deni","last_name":"Thompson","email":"kitt@acme.com"},"source":{"version":"1.2.0-SNAPSHOT","connector":"sqlserver","name":"btpns-sql
server-ipul","ts_ms":1588740858363,"snapshot":"false","db":"btpns_cdc_sqlserver_ipul","schema":"dbo","table":"customers","change_lsn":"0000002d:00003da0:0002","comm
it_lsn":"0000002d:00003da0:0003","event_serial_no":2},"op":"u","ts_ms":1588740859225,"transaction":null}}
{"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":true,"field":"first_name"
},{"type":"string","optional":true,"field":"last_name"},{"type":"string","optional":true,"field":"email"}],"optional":true,"name":"btpns_sqlserver_ipul.dbo.customer
s.Value","field":"before"},{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":true,"field":"first_name"},{"type":
"string","optional":true,"field":"last_name"},{"type":"string","optional":true,"field":"email"}],"optional":true,"name":"btpns_sqlserver_ipul.dbo.customers.Value","
field":"after"},{"type":"struct","fields":[{"type":"string","optional":false,"field":"version"},{"type":"string","optional":false,"field":"connector"},{"type":"stri
ng","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"ts_ms"},{"type":"string","optional":true,"name":"io.debezium.data.Enum","version":1,"
parameters":{"allowed":"true,last,false"},"default":"false","field":"snapshot"},{"type":"string","optional":false,"field":"db"},{"type":"string","optional":false,"f
ield":"schema"},{"type":"string","optional":false,"field":"table"},{"type":"string","optional":true,"field":"change_lsn"},{"type":"string","optional":true,"field":"
commit_lsn"},{"type":"int64","optional":true,"field":"event_serial_no"}],"optional":false,"name":"io.debezium.connector.sqlserver.Source","field":"source"},{"type":
"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"},{"type":"struct","fields":[{"type":"string","optional":false,"field":"id"},
{"type":"int64","optional":false,"field":"total_order"},{"type":"int64","optional":false,"field":"data_collection_order"}],"optional":true,"field":"transaction"}],"
optional":false,"name":"btpns_sqlserver_ipul.dbo.customers.Envelope"},"payload":{"before":{"id":9,"first_name":"Deni","last_name":"Thompson","email":"kitt@acme.com"
},"after":null,"source":{"version":"1.2.0-SNAPSHOT","connector":"sqlserver","name":"btpns-sqlserver-ipul","ts_ms":1588744428530,"snapshot":"false","db":"btpns_cdc_s
qlserver_ipul","schema":"dbo","table":"customers","change_lsn":"0000002d:00005400:0002","commit_lsn":"0000002d:00005400:0005","event_serial_no":1},"op":"d","ts_ms":
1588744430725,"transaction":null}}

Answer 2

Не могли бы вы дважды проверить настройку table.whitelist? Должно быть только dbo.customers.