Mysql通过Canal同步Elasticsearch( 三 ) _生活百科

文章插图
PUT /area_city{"mappings": {"properties": {"id":{"type": "long"},"pid":{"type": "integer"},"deep":{"type": "integer"},"name":{"type": "text"},"pinyin_prefix":{"type": "text"},"pinyin":{"type": "text"},"ext_id":{"type": "text"},"ext_name":{"type": "text"}}}}安装 Canal-Adapter安装镜像BASH# 拉取镜像$ docker pull slpcat/canal-adapter:v1.1.5# 随意启动，用于挂载配置文件$ docker run --name canal-adapter -p 8081:8081 -d slpcat/canal-adapter:v1.1.5# 创建存储挂载的目录$ mkdir /usr/kang/docker-canal-adapter# 挂载文件$ docker cp canal-adapter:/opt/canal-adapter /home/haha/docker-canal-adapter# 删除容器$ docker stop canal-adapter$ docker rm canal-adapter# 建议添加权限chmod 777 /home/haha/docker-canal-adapter# 启动新的容器$ docker run --name canal-adapter -p 8081:8081 -v /home/haha/docker-canal-adapter/canal-adapter:/opt/canal-adapter -d slpcat/canal-adapter:v1.1.5配置canal-adapter 文件'

文章插图
需要修改的是，conf目录下的 applicatiopn.yml 、es7目录下的 mytest_user.yml
编辑 application.yml:

文章插图

BASH# 切换到挂载目录$ cd /home/haha/docker-canal-adapter/canal-adapter$ cd conf/$ vim application.ymlYAMLserver:port: 8081spring:jackson:date-format: yyyy-MM-dd HH:mm:sstime-zone: GMT+8default-property-inclusion: non_nullcanal.conf:mode: tcp #tcp kafka rocketMQ rabbitMQflatMessage: truezookeeperHosts:syncBatchSize: 1000retries: 0timeout:accessKey:secretKey:consumerProperties:# canal tcp consumer编辑此处为 canal.deployer 的地址，canal.deployer的默认端口为 11111# 用于获取 canal 的数据进行实时同步canal.tcp.server.host: 192.168.80.80:11111canal.tcp.zookeeper.hosts:canal.tcp.batch.size: 500canal.tcp.username:canal.tcp.password:# kafka consumer#kafka.bootstrap.servers: 127.0.0.1:9092#kafka.enable.auto.commit: false#kafka.auto.commit.interval.ms: 1000#kafka.auto.offset.reset: latest#kafka.request.timeout.ms: 40000#kafka.session.timeout.ms: 30000#kafka.isolation.level: read_committed#kafka.max.poll.records: 1000# rocketMQ consumer#rocketmq.namespace:#rocketmq.namesrv.addr: 127.0.0.1:9876#rocketmq.batch.size: 1000#rocketmq.enable.message.trace: false#rocketmq.customized.trace.topic:#rocketmq.access.channel:#rocketmq.subscribe.filter:# rabbitMQ consumer#rabbitmq.host:#rabbitmq.virtual.host:#rabbitmq.username:#rabbitmq.password:#rabbitmq.resource.ownerId:srcDataSources:defaultDS: # 修改此处为 数据库信息url: jdbc:mysql://192.168.80.80:3306/estest?useUnicode=trueusername: rootpassword: 123456789canalAdapters:- instance: example # canal instance Name or mq topic namegroups:- groupId: g1outerAdapters:- name: logger#- name: rdb#key: mysql1#properties:#jdbc.driverClassName: com.mysql.jdbc.Driver#jdbc.url: jdbc:mysql://127.0.0.1:3306/mytest2?useUnicode=true#jdbc.username: root#jdbc.password: 121212#- name: rdb#key: oracle1#properties:#jdbc.driverClassName: oracle.jdbc.OracleDriver#jdbc.url: jdbc:oracle:thin:@localhost:49161:XE#jdbc.username: mytest#jdbc.password: m121212#- name: rdb#key: postgres1#properties:#jdbc.driverClassName: org.postgresql.Driver#jdbc.url: jdbc:postgresql://localhost:5432/postgres#jdbc.username: postgres#jdbc.password: 121212#threads: 1#commitSize: 3000#- name: hbase#properties:#hbase.zookeeper.quorum: 127.0.0.1#hbase.zookeeper.property.clientPort: 2181#zookeeper.znode.parent: /hbase- name: es7 # es7文件夹配置,还有es6#key: fgnKeyhosts: 192.168.80.80:9200 # 127.0.0.1:9200 for rest mode es 集群地址, 逗号分隔properties:mode: rest # or rest 可指定transport模式或者rest模式## security.auth: test:123456 #only used for rest modecluster.name: elasticsearch #指定es的集群名称#- name: kudu#key: kudu#properties:#kudu.master.address: 127.0.0.1 # ',' split multi address