前往小程序,Get更优阅读体验!
立即前往
首页
学习
活动
专区
工具
TVP
发布
社区首页 >专栏 >Debezium kafka connector 运行报错1236

Debezium kafka connector 运行报错1236

作者头像
XING辋
发布2019-07-19 10:55:18
2.1K0
发布2019-07-19 10:55:18
举报
文章被收录于专栏:M莫的博客

今天发现stag环境kafka connector运行报错( curl 172.18.1.1:8083/connectors/order-center-connector/status)

12345678910111213141516

{ "name": "order-center-connector", "connector": { "state": "RUNNING", "worker_id": "172.18.1.1:8083" }, "tasks": [ { "state": "FAILED", "trace": "org.apache.kafka.connect.errors.ConnectException: The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires. Error code: 1236; SQLSTATE: HY000.\n\tat io.debezium.connector.mysql.AbstractReader.wrap(AbstractReader.java:200)\n\tat io.debezium.connector.mysql.AbstractReader.failed(AbstractReader.java:167)\n\tat io.debezium.connector.mysql.BinlogReader$ReaderThreadLifecycleListener.onCommunicationFailure(BinlogReader.java:955)\n\tat com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:921)\n\tat com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:559)\n\tat com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:793)\n\tat java.lang.Thread.run(Thread.java:745)\nCaused by: com.github.shyiko.mysql.binlog.network.ServerException: The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires.\n\tat com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:882)\n\t... 3 more\n", "id": 0, "worker_id": "172.18.1.1:8083" } ], "type": "source" }

搜索查了下,按照规避Debezium master purged GTID问题的说法,分析了分析博主是正确的,但是按照他的实现方式(如下)还是不能够解决问题。

12345678910111213141516171819202122

curl -H "Content-Type:application/json" -XPUT 'http://172.18.1.1:8083/connectors/order-center-connector/config' -d ' { "connector.class": "io.debezium.connector.mysql.MySqlConnector", "tasks.max": "1", "database.hostname": "sha2app220.stag.rs.com", "database.port": "3306", "database.user": "debezium", "database.password": "password1234567890", "database.server.id": "19991", "database.server.name": "trade_order_0", "database.whitelist": "db_order", "include.schema.changes": "false", "snapshot.mode": "schema_only", "snapshot.locking.mode": "none", "gtid.source.includes":"631357f9-95a0-11e6-a28c-005056b50019.*,631c2fd3-95a0-11e6-a28c-005056b51ac6.*,d9af3b6a-cfe3-11e7-a3ce-005056ab1bc3:1-36631495,fefa6b5b-d00f-11e7-a4ee-005056ab5e4d.*", "database.history.kafka.bootstrap.servers": "base6511.stag.rs.com:9092,base6512.stag.rs.com:9092,base6513.stag.rs.com:9092", "database.history.kafka.topic": "dbhistory.trade_order_0", "decimal.handling.mode": "string", "table.whitelist": "db_order.tx_order,db_order.tx_order_addition,db_order.tx_customer_service", "database.history.store.only.monitored.tables.ddl":"true", "database.history.skip.unparseable.ddl":"true" }'

最后发现解决这个问题比较简单,只需要把之前的order-center-connector删除掉,重新建立不同名称的connector就行了,如下

1234567891011121314151617181920

curl -H "Content-Type:application/json" -XPUT 'http://172.18.1.1:8083/connectors/order-center-connector1/config' -d ' { "connector.class": "io.debezium.connector.mysql.MySqlConnector", "tasks.max": "1", "database.hostname": "sha2app220.stag.rs.com", "database.port": "3306", "database.user": "debezium", "database.password": "password1234567890", "database.server.id": "19991", "database.server.name": "trade_order_0", "database.whitelist": "db_order", "include.schema.changes": "false", "snapshot.mode": "schema_only", "snapshot.locking.mode": "none", "database.history.kafka.topic": "dbhistory.trade_order_0", "decimal.handling.mode": "string", "table.whitelist": "db_order.tx_order,db_order.tx_order_addition,db_order.tx_customer_service", "database.history.store.only.monitored.tables.ddl":"true", "database.history.skip.unparseable.ddl":"true" }'

本文参与 腾讯云自媒体同步曝光计划,分享自作者个人站点/博客。
原始发表:2019-07-18,如有侵权请联系 cloudcommunity@tencent.com 删除

本文分享自 作者个人站点/博客 前往查看

如有侵权,请联系 cloudcommunity@tencent.com 删除。

本文参与 腾讯云自媒体同步曝光计划  ,欢迎热爱写作的你一起参与!

评论
登录后参与评论
0 条评论
热度
最新
推荐阅读
相关产品与服务
数据库
云数据库为企业提供了完善的关系型数据库、非关系型数据库、分析型数据库和数据库生态工具。您可以通过产品选择和组合搭建,轻松实现高可靠、高可用性、高性能等数据库需求。云数据库服务也可大幅减少您的运维工作量,更专注于业务发展,让企业一站式享受数据上云及分布式架构的技术红利!
领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档