建站 手机网站,网页文章 在wordpress,泷澄建设集团网站,有哪些建设网站的大公司概述
Kingbase FlySync是面向同城/异地灾备、数据库平滑升级替换、数据集中共享与分发、应用上云迁移、数据库负载均衡等场景的数据同步产品。该产品基于增量日志解析技术#xff0c;性能高、时延低、资源占用极少#xff0c;能够实现异构数据源之间大规模增量数据的任意方向…概述
Kingbase FlySync是面向同城/异地灾备、数据库平滑升级替换、数据集中共享与分发、应用上云迁移、数据库负载均衡等场景的数据同步产品。该产品基于增量日志解析技术性能高、时延低、资源占用极少能够实现异构数据源之间大规模增量数据的任意方向流转和实时同步帮助用户打破数据孤岛轻松共享分发数据。Kingbase FlySync支持丰富的软硬件平台和数据源同步拓扑可以一对一、一对多、多对一、级联等方式任意组合延伸并可在数据同步过程中保证端到端的事务级数据完整性和高可用性。
准备环境
两台虚拟机
虚拟机IP数据库软件版本角色操作系统K2node1192.168.40.111(Kingbase) KingbaseES V8R6主库CenterOS7.2K2node2192.168.40.112(Kingbase) KingbaseES V8R6备库CenterOS7.2
数据库安装要求
项目node1node2数据库版本KingbaseES V8R6KingbaseES V8R6安装路径/KingbaseES/V8//KingbaseES/V8/data路径/data/data端口5432154321用户名systemsystem密码kingbasekingbase
目标
同步程序安装部署使用命令方式完成数据迁移并保证数据一致。按照以下规范完成部署
类别源端目标端replicater安装路径/home/flysync/replicator/home/flysync/replicatorreplicator端口rmi-port11000/kufl-port:3112rmi-port:11000/kufl-port3112kfs服务名kingbase8_sourcekingbase8_targetKFS 同步所使用的用户flysync/flysyncflysync/flysync映射规则1examexam[0017]映射规则2exam.courseexam0017.course0017
资源
1.测试虚拟机下载地址包含node1,node2
虚拟机快照都还原至优化数据库root用户密码为kingbase.123kingbase用户密码为kingbase
点击下载百度网盘
2.同步工具下载地址
点击下载百度网盘
3.临时授权下载地址
点击下载百度网盘
4.ruby工具下载地址
点击下载百度网盘
5.EXAMv0.11.sql下载地址
点击下载百度网盘
实操同步软件安装部署
1.node1准备安装环境
(1)增加flysync 用户并设置密码
[rootnode1 ~]# adduser flysync
[rootnode1 ~]# usermod -G kingbase flysync
[rootnode1 ~]# echo flysync | passwd --stdin flysync
更改用户 flysync 的密码 。
passwd所有的身份验证令牌已经成功更新。
[rootnode1 ~]#(2)调整flysync的最大文件句柄数open files、允许创建的最大进程数量max user processes
[rootnode1 ~]# cat /etc/security/limits.conf EOF
flysync - nofile 65535
flysync - nproc 8096
EOF
[rootnode1 ~]# cat /etc/security/limits.conf
# /etc/security/limits.conf
#
#This file sets the resource limits for the users logged in via PAM.
#It does not affect resource limits of the system services.
#
#Also note that configuration files in /etc/security/limits.d directory,
#which are read in alphabetical order, override the settings in this
#file in case the domain is the same or more specific.
#That means for example that setting a limit for wildcard domain here
#can be overriden with a wildcard setting in a config file in the
#subdirectory, but a user specific setting here can be overriden only
#with a user specific setting in the subdirectory.
#
#Each line describes a limit for a user in the form:
#
#domain type item value
#
#Where:
#domain can be:
# - a user name
# - a group name, with group syntax
# - the wildcard *, for default entry
# - the wildcard %, can be also used with %group syntax,
# for maxlogin limit
#
#type can have the two values:
# - soft for enforcing the soft limits
# - hard for enforcing hard limits
#
#item can be one of the following:
# - core - limits the core file size (KB)
# - data - max data size (KB)
# - fsize - maximum filesize (KB)
# - memlock - max locked-in-memory address space (KB)
# - nofile - max number of open file descriptors
# - rss - max resident set size (KB)
# - stack - max stack size (KB)
# - cpu - max CPU time (MIN)
# - nproc - max number of processes
# - as - address space limit (KB)
# - maxlogins - max number of logins for this user
# - maxsyslogins - max number of logins on the system
# - priority - the priority to run user process with
# - locks - max number of file locks the user can hold
# - sigpending - max number of pending signals
# - msgqueue - max memory used by POSIX message queues (bytes)
# - nice - max nice priority allowed to raise to values: [-20, 19]
# - rtprio - max realtime priority
#
#domain type item value
##* soft core 0
#* hard rss 10000
#student hard nproc 20
#faculty soft nproc 20
#faculty hard nproc 50
#ftp hard nproc 0
#student - maxlogins 4#add by kingbase
kingbase soft nproc 65536
kingbase hard nproc 65536
kingbase soft nofile 65536
kingbase hard nofile 65536
#kingbase soft stack 10240
#kingbase hard stack 32768
kingbase soft core unlimited
kingbase hard core unlimitedflysync - nofile 65535
flysync - nproc 8096
[rootnode1 ~]# (3)hosts 文件配置
[rootnode1 ~]# cat /etc/hosts EOF
192.168.40.111 node1
192.168.40.112 node2
EOF
[rootnode1 ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.40.111 node1
192.168.40.112 node2
[rootnode1 ~]# (4)network配置
[rootnode1 ~]# cat /etc/sysconfig/network EOFNETWORKING yesEOF
[rootnode1 ~]# cat /etc/sysconfig/network
# Created by anaconda
NETWORKING yes
HOSTNAME node1
[rootnode1 ~]# (5)上传并安装ruby工具
使用FTP工具上传rbenv_2.2.2_linux_x64.zip到/home/flysync目录下
[rootnode1 flysync]# ls -l
总用量 6832
-rw-r--r-- 1 root root 6993781 11月 16 10:28 rbenv_2.2.2_linux_x64.zip
[rootnode1 flysync]# pwd
/home/flysync
[rootnode1 flysync]# [rootnode1 flysync]# chown flysync:flysync /home/flysync/rbenv_2.2.2_linux_x64.zip
[rootnode1 flysync]#解压rbenv_2.2.2_linux_x64.zip包
[rootnode1 flysync]su flysync
[flysyncnode1 flysync]# unzip -q /home/flysync/rbenv_2.2.2_linux_x64.zip
[flysyncnode1 flysync]# ls -l
总用量 6836
-rwxr-xr-x 1 root root 340 11月 4 2019 fspm
drwxrwxr-x 3 flysync flysync 17 11月 4 2019 rbenv
-rw-r--r-- 1 flysync flysync 6993781 11月 16 10:28 rbenv_2.2.2_linux_x64.zip
[rootnode1 flysync]#把ruby添加到环境变量
[flysyncnode1 flysync]# vi ~/.bash_profile
[flysyncnode1 flysync]# source ~/.bash_profile
[flysyncnode1 flysync]# cat ~/.bash_profile
[flysyncnode1 ~]$ cat ~/.bash_profile
# .bash_profile# Get the aliases and functions
if [ -f ~/.bashrc ]; then. ~/.bashrc
fi# User specific environment and startup programsPATH$PATH:$HOME/.local/bin:$HOME/bin
export RUBY_HOME/home/flysync/rbenv/ruby
export PATH$RUBY_HOME/bin:$PATH
export PATH[flysyncnode1 flysync]# ruby -v
ruby 2.2.2p95 (2015-04-13 revision 50295) [x86_64-linux]
[flysyncnode1 flysync]# (6)node1数据库配置
登录数据库
[kingbasenode1 ~]$ cd /KingbaseES/V8/Server/bin/
[kingbasenode1 bin]$ ./ksql -Usystem test
ksql (V8.0)
输入 help 来获取帮助信息.
test# 添加flysync用户并授权
test# CREATE USER flysync PASSWORD flysync;
CREATE ROLE
test# ALTER USER flysync REPLICATION;
ALTER ROLE
test# GRANT ALL PRIVILEGES ON DATABASE TEST TO flysync ;
GRANT
test# ALTER USER flysync SUPERUSER;
ALTER ROLE
test# 配置服务器参数
test# ALTER SYSTEM SET wal_level logical;
ALTER SYSTEM
test# ALTER SYSTEM SET wal_keep_segments 4;
ALTER SYSTEM
test# ALTER SYSTEM SET max_wal_senders 10;
ALTER SYSTEM
test# ALTER SYSTEM SET max_replication_slots 10;
ALTER SYSTEM
test# 退出数据库并修改数据库认证文件
test# \q
[kingbasenode1 bin]$ cat /data/sys_hba.conf EOF
host replication all 0.0.0.0/0 scram-sha-256
EOF
[kingbasenode1 bin]$ cat /data/sys_hba.conf
# Kingbase Client Authentication Configuration File
#
#
# Refer to the Client Authentication section in the Kingbase
# documentation for a complete description of this file. A short
# synopsis follows.
#
# This file controls: which hosts are allowed to connect, how clients
# are authenticated, which Kingbase user names they can use, which
# databases they can access. Records take one of these forms:
#
# local DATABASE USER METHOD [OPTIONS]
# host DATABASE USER ADDRESS METHOD [OPTIONS]
# hostssl DATABASE USER ADDRESS METHOD [OPTIONS]
# hostnossl DATABASE USER ADDRESS METHOD [OPTIONS]
#
# (The uppercase items must be replaced by actual values.)
#
# The first field is the connection type: local is a Unix-domain
# socket, host is either a plain or SSL-encrypted TCP/IP socket,
# hostssl is an SSL-encrypted TCP/IP socket, and hostnossl is a
# plain TCP/IP socket.
#
# DATABASE can be all, sameuser, samerole, replication, a
# database name, or a comma-separated list thereof. The all
# keyword does not match replication. Access to replication
# must be enabled in a separate record (see example below).
#
# USER can be all, a user name, a group name prefixed with , or a
# comma-separated list thereof. In both the DATABASE and USER fields
# you can also write a file name prefixed with to include names
# from a separate file.
#
# ADDRESS specifies the set of hosts the record matches. It can be a
# host name, or it is made up of an IP address and a CIDR mask that is
# an integer (between 0 and 32 (IPv4) or 128 (IPv6) inclusive) that
# specifies the number of significant bits in the mask. A host name
# that starts with a dot (.) matches a suffix of the actual host name.
# Alternatively, you can write an IP address and netmask in separate
# columns to specify the set of hosts. Instead of a CIDR-address, you
# can write samehost to match any of the servers own IP addresses,
# or samenet to match any address in any subnet that the server is
# directly connected to.
#
# METHOD can be trust, reject, md5, password, scram-sha-256,
# gss, sspi, ident, peer, pam, ldap, radius or cert.
# Note that password sends passwords in clear text; md5 or
# scram-sha-256 are preferred since they send encrypted passwords.
#
# OPTIONS are a set of options for the authentication in the format
# NAMEVALUE. The available options depend on the different
# authentication methods -- refer to the Client Authentication
# section in the documentation for a list of which options are
# available for which authentication methods.
#
# Database and user names containing spaces, commas, quotes and other
# special characters must be quoted. Quoting one of the keywords
# all, sameuser, samerole or replication makes the name lose
# its special character, and just match a database or username with
# that name.
#
# This file is read on server startup and when the server receives a
# SIGHUP signal. If you edit the file on a running system, you have to
# SIGHUP the server for the changes to take effect, run sys_ctl reload,
# or execute SELECT pg_reload_conf().
#
# Put your actual configuration here
# ----------------------------------
#
# If you want to allow non-local connections, you need to add more
# host records. In that case you will also need to make Kingbase
# listen on a non-local interface via the listen_addresses
# configuration parameter, or via the -i or -h command line switches.# CAUTION: Configuring the system for local trust authentication
# allows any local user to connect as any Kingbase user, including
# the database superuser. If you do not trust all your local users,
# use another authentication method.# TYPE DATABASE USER ADDRESS METHOD# local is for Unix domain socket connections only
local all all trust
# IPv4 local connections:
host all all 127.0.0.1/32 scram-sha-256
host all all 0.0.0.0/0 scram-sha-256
# IPv6 local connections:
host all all ::1/128 scram-sha-256
host all all ::0/0 scram-sha-256
# Allow replication connections from localhost, by a user with the
# replication privilege.
local replication all trust
host replication all 127.0.0.1/32 scram-sha-256
host replication all ::1/128 scram-sha-256
host replication all 0.0.0.0/32 scram-sha-256
host replication all 0.0.0.0/0 scram-sha-256
[kingbasenode1 bin]$ (7)重启数据库使配置生效
[kingbasenode1 bin]$ sys_ctl restart -D /data
waiting for server to shut down.... done
server stopped
waiting for server to start....2023-11-16 03:43:14.182 GMT [5233] 警告: max_connections should be less than or equal than 10 (restricted by license)
2023-11-16 03:43:14.182 GMT [5233] 提示: the value of max_connect is set 10
2023-11-16 03:43:14.183 GMT [5233] 警告: max_connections should be less than or equal than 10 (restricted by license)
2023-11-16 03:43:14.183 GMT [5233] 提示: the value of max_connect is set 10
2023-11-16 11:43:14.189 CST [5233] 日志: sepapower extension initialized
2023-11-16 11:43:14.191 CST [5233] 日志: 太多后台工作进程
2023-11-16 11:43:14.191 CST [5233] 详细信息: 当前设置里最多可以注册2个后台工作进程.
2023-11-16 11:43:14.191 CST [5233] 提示: 考虑增大配置参数 max_worker_processes的值.
2023-11-16 11:43:14.191 CST [5233] 日志: 太多后台工作进程
2023-11-16 11:43:14.191 CST [5233] 详细信息: 当前设置里最多可以注册2个后台工作进程.
2023-11-16 11:43:14.191 CST [5233] 提示: 考虑增大配置参数 max_worker_processes的值.
2023-11-16 11:43:14.192 CST [5233] 日志: 正在启动 KingbaseES V008R006C005B0023 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-46), 64-bit
2023-11-16 11:43:14.192 CST [5233] 日志: 正在监听IPv4地址0.0.0.0端口 54321
2023-11-16 11:43:14.192 CST [5233] 日志: 正在监听IPv6地址::端口 54321
2023-11-16 11:43:14.193 CST [5233] 日志: 在Unix套接字 /tmp/.s.KINGBASE.54321上侦听
2023-11-16 11:43:14.244 CST [5233] 日志: 日志输出重定向到日志收集进程
2023-11-16 11:43:14.244 CST [5233] 提示: 后续的日志输出将出现在目录 sys_log中.done
server started
[kingbasenode1 bin]$ (8)导入测试数据
[kingbasenode1 install]$ ksql -U flysync -d test -f /install/EXAM_v0.11.sql
ksql:/install/EXAM_v0.11.sql:8: 注意: 串联删除10个其它对象
描述: 递归删除 序列 exam.seq_student_sno
递归删除 表 exam.student
递归删除 表 exam.course
递归删除 表 exam.instructor
递归删除 表 exam.instructor_history
递归删除 表 exam.score
递归删除 视图 exam.v_score_information
递归删除 function exam.get_avg_score(numeric)
递归删除 procedure exam.retire_instructor(numeric)
递归删除 表 exam.tmpbak01
DROP SCHEMA
CREATE SCHEMA
SET
CREATE SEQUENCE
CREATE TABLE
CREATE TABLE
CREATE TABLE
CREATE TABLE
CREATE TABLE
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
COMMENT
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
CREATE VIEW
CREATE FUNCTION
CREATE PROCEDURE
CREATE TRIGGER
ksql:/install/EXAM_v0.11.sql:202: 警告: 没有事物在运行中
COMMIT
[kingbasenode1 install]$
(9)最后检查下环境
检查机器名
[kingbasenode1 install]$ exit
登出
[rootnode2 flysync]# uname -n
node2
检查操作系统属性检查及配置
[rootnode1 ~]$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 11210
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 65535
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 8096
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
检查IP 地址
[rootnode1 flysync]# hostname --ip-address
192.168.40.111 192.168.40.111
[rootnode1 flysync]# 检查其他机器的 IP 地址是否正确
[rootnode1 ~]$ ping -c 4 node1
PING node1 (192.168.40.111) 56(84) bytes of data.
64 bytes from node1 (192.168.40.111): icmp_seq1 ttl64 time0.036 ms
64 bytes from node1 (192.168.40.111): icmp_seq2 ttl64 time0.110 ms
64 bytes from node1 (192.168.40.111): icmp_seq3 ttl64 time0.036 ms
64 bytes from node1 (192.168.40.111): icmp_seq4 ttl64 time0.037 ms--- node1 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2999ms
rtt min/avg/max/mdev 0.036/0.054/0.110/0.033 ms
[flysyncnode1 ~]$ ping -c 4 node2
PING node2 (192.168.40.112) 56(84) bytes of data.
64 bytes from node2 (192.168.40.112): icmp_seq1 ttl64 time1.65 ms
64 bytes from node2 (192.168.40.112): icmp_seq2 ttl64 time1.43 ms
64 bytes from node2 (192.168.40.112): icmp_seq3 ttl64 time0.262 ms
64 bytes from node2 (192.168.40.112): icmp_seq4 ttl64 time0.301 ms--- node2 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3003ms
rtt min/avg/max/mdev 0.262/0.913/1.656/0.637 ms
检查时区
[rootnode1 ~]$ date
2023年 11月 16日 星期四 11:48:34 CST2.node2准备安装环境
(1)增加flysync 用户并设置密码
[rootnode2 ~]# userdel -r flysync
[rootnode2 ~]# adduser flysync
[rootnode2 ~]# usermod -G kingbase flysync
[rootnode2 ~]# echo flysync |passwd --stdin flysync
更改用户 flysync 的密码 。
passwd所有的身份验证令牌已经成功更新。
[rootnode2 ~]#
(2)调整flysync的最大文件句柄数open files、允许创建的最大进程数量max user processes
[rootnode2 ~]# cat /etc/security/limits.conf EOF
flysync - nofile 65535
flysync - nproc 8096
EOF
[rootnode2 ~]# cat /etc/security/limits.conf
# /etc/security/limits.conf
#
#This file sets the resource limits for the users logged in via PAM.
#It does not affect resource limits of the system services.
#
#Also note that configuration files in /etc/security/limits.d directory,
#which are read in alphabetical order, override the settings in this
#file in case the domain is the same or more specific.
#That means for example that setting a limit for wildcard domain here
#can be overriden with a wildcard setting in a config file in the
#subdirectory, but a user specific setting here can be overriden only
#with a user specific setting in the subdirectory.
#
#Each line describes a limit for a user in the form:
#
#domain type item value
#
#Where:
#domain can be:
# - a user name
# - a group name, with group syntax
# - the wildcard *, for default entry
# - the wildcard %, can be also used with %group syntax,
# for maxlogin limit
#
#type can have the two values:
# - soft for enforcing the soft limits
# - hard for enforcing hard limits
#
#item can be one of the following:
# - core - limits the core file size (KB)
# - data - max data size (KB)
# - fsize - maximum filesize (KB)
# - memlock - max locked-in-memory address space (KB)
# - nofile - max number of open file descriptors
# - rss - max resident set size (KB)
# - stack - max stack size (KB)
# - cpu - max CPU time (MIN)
# - nproc - max number of processes
# - as - address space limit (KB)
# - maxlogins - max number of logins for this user
# - maxsyslogins - max number of logins on the system
# - priority - the priority to run user process with
# - locks - max number of file locks the user can hold
# - sigpending - max number of pending signals
# - msgqueue - max memory used by POSIX message queues (bytes)
# - nice - max nice priority allowed to raise to values: [-20, 19]
# - rtprio - max realtime priority
#
#domain type item value
##* soft core 0
#* hard rss 10000
#student hard nproc 20
#faculty soft nproc 20
#faculty hard nproc 50
#ftp hard nproc 0
#student - maxlogins 4# End of file
#add by kingbase
kingbase soft nproc 65536
kingbase hard nproc 65536
kingbase soft nofile 65536
kingbase hard nofile 65536
kingbase soft stack 10240
kingbase hard stack 32768
kingbase soft core unlimited
kingbase hard core unlimited
flysync - nofile 65535
flysync - nproc 8096
[rootnode2 ~]# (3)hosts 文件配置
[rootnode2 ~]# cat /etc/hosts EOF
192.168.40.111 node1
192.168.40.112 node2
EOF
[rootnode2 ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.40.111 node1
192.168.40.112 node2
[rootnode2 ~]# (4)network配置
[rootnode1 ~]# cat /etc/sysconfig/network EOF
NETWORKING yes
EOF
[rootnode2 ~]# cat /etc/sysconfig/network
# Created by anaconda
NETWORKING yes
HOSTNAME node2
[rootnode2 ~]# (5)拷贝并安装ruby工具
bash
[flysyncnode2 ~]# scp rootnode1:/home/flysync/rbenv_2.2.2_linux_x64.zip /home/flysync/
The authenticity of host node1 (192.168.40.111) cant be established.
ECDSA key fingerprint is b8:54:e0:74:c5:e2:33:46:0d:7d:39:91:e3:b5:5a:6d.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added node1,192.168.40.111 (ECDSA) to the list of known hosts.
rootnode1s password:
rbenv_2.2.2_linux_x64.zip 100% 6830KB 6.7MB/s 00:00
[flysyncnode2 ~]# cd /home/flysync/
[flysyncnode2 flysync]# ls rbenv_2.2.2_linux_x64.zip
rbenv_2.2.2_linux_x64.zip
[flysyncnode2 flysync]#解压rbenv_2.2.2_linux_x64.zip包
[rootnode2 flysync]su flysync
[flysyncnode2 flysync]# unzip -q /home/flysync/rbenv_2.2.2_linux_x64.zip
[flysyncnode2 flysync]# ls -l
总用量 6836
-rwxr-xr-x 1 root root 340 11月 4 2019 fspm
drwxrwxr-x 3 flysync flysync 17 11月 4 2019 rbenv
-rw-r--r-- 1 flysync flysync 6993781 11月 16 10:28 rbenv_2.2.2_linux_x64.zip
[rootnode1 flysync]#把ruby添加到环境变量
[flysyncnode2 ~]$ vi ~/.bash_profile
[flysyncnode2 ~]$ source ~/.bash_profile
[flysyncnode2 ~]$ cat ~/.bash_profile
# .bash_profile# Get the aliases and functions
if [ -f ~/.bashrc ]; then. ~/.bashrc
fi# User specific environment and startup programs
export RUBY_HOME/home/flysync/rbenv/ruby
export PATH$RUBY_HOME/bin:$PATH
PATH$PATH:$HOME/.local/bin:$HOME/binexport PATH
[flysyncnode2 flysync]# ruby -v
ruby 2.2.2p95 (2015-04-13 revision 50295) [x86_64-linux]
[flysyncnode2 flysync]# (6)node2数据库添加用户并授权
登录数据库
[flysyncnode2 ~]$ exit
exit
[rootnode2 flysync]# su kingbase
[kingbasenode2 flysync]$
[kingbasenode2 flysync]$ cd /KingbaseES/V8/Server/bin/
[kingbasenode2 bin]$ ./ksql -Usystem test
ksql (V8.0)
输入 help 来获取帮助信息.test#
添加flysync用户并授权
test# CREATE USER flysync PASSWORD flysync;
CREATE ROLE
ttest# GRANT ALL PRIVILEGES ON DATABASE TEST TO flysync ;
GRANT
test# ALTER USER flysync SUPERUSER;
ALTER ROLE
test# (7)最后检查下环境
检查机器名
[kingbasenode1 install]$ exit
登出
[rootnode2 flysync]# uname -n
node2检查操作系统属性检查及配置
[rootnode2 flysync]# ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 9838
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 9838
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
[rootnode2 flysync]# 检查IP 地址
[rootnode2 flysync]# hostname --ip-address
192.168.40.112
[rootnode2 flysync]# 检查其他机器的 IP 地址是否正确
[rootnode2 flysync]# ping -c 4 node1
PING node1 (192.168.40.111) 56(84) bytes of data.
64 bytes from node1 (192.168.40.111): icmp_seq1 ttl64 time0.296 ms
64 bytes from node1 (192.168.40.111): icmp_seq2 ttl64 time0.274 ms
64 bytes from node1 (192.168.40.111): icmp_seq3 ttl64 time0.370 ms
64 bytes from node1 (192.168.40.111): icmp_seq4 ttl64 time0.325 ms--- node1 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2999ms
rtt min/avg/max/mdev 0.274/0.316/0.370/0.038 ms
[rootnode2 flysync]# ping -c 4 node2
PING node2 (192.168.40.112) 56(84) bytes of data.
64 bytes from node2 (192.168.40.112): icmp_seq1 ttl64 time0.028 ms
64 bytes from node2 (192.168.40.112): icmp_seq2 ttl64 time0.043 ms
64 bytes from node2 (192.168.40.112): icmp_seq3 ttl64 time0.038 ms
64 bytes from node2 (192.168.40.112): icmp_seq4 ttl64 time0.036 ms--- node2 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3000ms
rtt min/avg/max/mdev 0.028/0.036/0.043/0.006 ms
[rootnode2 flysync]# 检查时区
[rootnode2 flysync]# date
2023年 11月 16日 星期四 13:19:32 CST
[rootnode2 flysync]# 3.node1节点部署replicator同步程序
(1).上传并解压replicator同步程序
[rootnode1 ~]# su flysync
[flysyncnode1 install]# cd /install/
[flysyncnode1 install]# ls KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz
KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz
[flysyncnode1 install]$ tar -zxf KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz
[flysyncnode1 install]$ ls KingbaseFlySync-V002R002C004B20230320-replicator
cluster-home extensions flysync-replicator tools
[rootnode1 install]# (2).拷贝驱动包
[flysyncnode1 install]$ cd KingbaseFlySync-V002R002C004B20230320-replicator/flysync-replicator/lib
[flysyncnode1 lib]$ cp /install/KingbaseFlySync-V002R002C004B20230320-replicator/extensions/jdbc/kingbase8-8.6.0.jar .
[flysyncnode1 lib]$ ls -l |grep kingbase8-8.6.0.jar
-rw-rw-r-- 1 flysync flysync 1019069 11月 16 13:35 kingbase8-8.6.0.jar
[flysyncnode1 lib]$ (3).配置静默安装文件
[flysyncnode1 lib]$ cd /install/KingbaseFlySync-V002R002C004B20230320-replicator/flysync-replicator/samples/ini/
[flysyncnode1 ini]$ cp flysync_kingbase8_source.ini ~/flysync.ini
[flysyncnode1 ini]$ vi ~/flysync.ini 参数如下
[defaults]
# 安装目录
install-directory~/replicator
# 环境变量脚本位置
profile-script~/.bash_profile
# rmi端口,默认11000
rmi-port11000# 服务名
[kingbase8_source]
# 监听kufl获取的端口
kufl-port3112
# master主机名
masternode1
# 当前主机名
membersnode1# 数据源类型
datasource-typekingbase
# 同步数据库信息
replication-hostnode1
replication-userflysync
replication-passwordflysync
replication-port54321# 同步服务角色
rolemaster## 指定连接的数据库名称; 源端时需指定解析方式
kingbase-dbnametest
kingbase-extractor-methodxlogical## kingbase 版本号
datasource-version8## 过滤器开关配置
# svc_extractor_filtersreplicate,casetransform,dropcolumn,rename
svc_extractor_filtersreplicate## 集群参数
# propertyreplicator.datasource.global.connectionSpec.use_clustertrue
# propertyreplicator.datasource.global.connectionSpec.slave_add192.168.28.176,192.168.28.176
# propertyreplicator.datasource.global.connectionSpec.slave_port1521,1521### 过滤器详细配置
## replicate 表过滤
# propertyreplicator.filter.replicate.doPUBLIC.*
# propertyreplicator.filter.replicate.ignorePUBLIC.T1
## casetransform 大小写转换
# propertyreplicator.filter.casetransform.to_upper_casetrue
propertyreplicator.filter.replicate.doexam.*
## 优化参数
# 达到多少条记录后分事务默认 500
propertyreplicator.extractor.dbms.maxRowsByBlock500
# 若多少秒没有新的事务产生那么就直接返回该事务不再等待合并
propertyreplicator.extractor.dbms.maxWaitingTime2
# 大事务优化参数
propertyreplicator.extractor.dbms.minRowsPerBlock4000# 自动恢复尝试次数系统 OFFLINE 时尝试将系统重新置于ONLINE状态的次数
repl_auto_recovery_max_attempts0
# 系统 OFFLINE 时尝试 ONLINE 之前的等待时间s(秒)、m(分钟)、h(小时)
repl_auto_recovery_delay_interval30s
# 系统自动恢复次数重置时间s(秒)、m(分钟)、h(小时)
repl_auto_recovery_reset_interval300s(4).执行安装
[flysyncnode1 ini]$ cd /install/KingbaseFlySync-V002R002C004B20230320-replicator/
[flysyncnode1 KingbaseFlySync-V002R002C004B20230320-replicator]$ ./tools/fspm install
..
WARN node1 ntpd is not running. It is important that configured hosts have time synchronised. (NtpdRunningCheck)
WARN node1 Linux swappiness is currently set to 30, on restart it will be 1, consider setting this to 10 or under to avoid swapping. (SwappinessCheck)
...License : /home/flysync/replicator/license.dat
.#####################################################################
# Next Steps
#####################################################################
Unless automatically started, you must start the FlySync services before the
cluster will be available./home/flysync/replicator/flysync/cluster-home/bin/startallWait a minute for the services to start up and configure themselves. After
that you may proceed.We have added FlySync environment variables to ~/.bash_profile.
Run source ~/.bash_profile to rebuild your environment.Once your services start successfully replication will begin.
To look at services and perform administration, run the following command
from any database server./home/flysync/replicator/flysync/flysync-replicator/bin/fsrepctl servicesConfiguration is now complete. For further information, please consult
FlySync documentation.NOTE Command successfully completed
[flysyncnode1 KingbaseFlySync-V002R002C004B20230320-replicator]$ (5).上传并更新同步软件授权
[flysyncnode1 KingbaseFlySync-V002R002C004B20230320-replicator]$ cp /install/license.dat ~/replicator/license.dat
[flysyncnode1 KingbaseFlySync-V002R002C004B20230320-replicator]$ ls ~/replicator/license.dat
/home/flysync/replicator/license.dat(6).让环境变量生效
[flysyncnode1 KingbaseFlySync-V002R002C004B20230320-replicator]$ source ~/.bash_profile
[flysyncnode1 KingbaseFlySync-V002R002C004B20230320-replicator]$ (7).配置源端过滤策略
只抽取exam 模式下的增量数据
[flysyncnode1 KingbaseFlySync-V002R002C004B20230320-replicator]$ vi ~/flysync.ini
cat ~/flysync.ini | grep -E svc_extractor_filters|replicate.do
# svc_extractor_filtersreplicate,casetransform,dropcolumn,rename
svc_extractor_filtersreplicate
# propertyreplicator.filter.replicate.doPUBLIC.*
propertyreplicator.filter.replicate.doexam.*
[flysyncnode1 KingbaseFlySync-V002R002C004B20230320-replicator]$ (8).启动同步程序并置于offline 状态
[flysyncnode1 KingbaseFlySync-V002R002C004B20230320-replicator]$ replicator start offline
Starting FlySync Replicator Service...
Waiting for FlySync Replicator Service......
running: PID:23406(9).查看同步软件的运行状态
[flysyncnode1 KingbaseFlySync-V002R002C004B20230320-replicator]$ fsrepctl -service kingbase8_source services
Processing services command...
NAME VALUE
---- -----
appliedLastSeqno: -1
appliedLatency : -1.0
role : master
serviceName : kingbase8_source
serviceType : local
started : true
state : OFFLINE:NORMAL
Finished services command...
[flysyncnode1 KingbaseFlySync-V002R002C004B20230320-replicator]$
4.node2节点部署replicator同步程序
(1).上传并解压replicator同步程序
[rootnode2 ~]# su flysync
[flysyncnode2 root]$ cd /install/
[flysyncnode2 install]$ ls KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz
KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz
[flysyncnode2 install]$ tar -zxf KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz
[flysyncnode2 install]$ ls KingbaseFlySync-V002R002C004B20230320-replicator
cluster-home extensions flysync-replicator tools
[flysyncnode2 install]$
(2).拷贝驱动包
[flysyncnode2 install]$ cd KingbaseFlySync-V002R002C004B20230320-replicator/flysync-replicator/lib
[flysyncnode2 lib]$ cp /install/KingbaseFlySync-V002R002C004B20230320-replicator/extensions/jdbc/kingbase8-8.6.0.jar .
[flysyncnode2 lib]$ ls -l |grep kingbase8-8.6.0.jar
-rw-rw-r-- 1 flysync flysync 1019069 11月 16 22:30 kingbase8-8.6.0.jar
[flysyncnode2 lib]$ (3).配置静默安装文件
[flysyncnode2 lib]$ cd /install/KingbaseFlySync-V002R002C004B20230320-replicator/flysync-replicator/samples/ini/
[flysyncnode2 ini]$ cp flysync_kingbase8_target.ini ~/flysync.ini
[flysyncnode2 ini]$ vi ~/flysync.ini
[flysyncnode2 ini]$ cat ~/flysync.ini
[defaults]
# 安装目录
install-directory~/replicator
# 环境变量脚本位置
profile-script~/.bash_profile
# rmi端口,默认11000
rmi-port11000# 服务名
[kingbase8_target]
# 监听kufl获取的端口
kufl-port3112
# master主机名
masternode1
# 当前主机名
membersnode2##指定连接master监听的kufl端口
master-kufl-port3112# 数据源类型
datasource-typekingbase
# 同步数据库信息
replication-hostnode2
replication-userflysync
replication-passwordflysync
replication-port54321# 同步服务角色
roleslave## 指定连接的数据库名称
kingbase-dbnametest## kingbase 版本号
datasource-version8## 过滤器开关配置
# svc-remote-filterscasetransform,dropcolumn,rename
svc-remote-filtersrename
## 集群参数
# propertyreplicator.datasource.global.connectionSpec.use_clustertrue
# propertyreplicator.datasource.global.connectionSpec.slave_add192.168.28.176,192.168.28.176
# propertyreplicator.datasource.global.connectionSpec.slave_port1521,1521### 过滤器详细配置
## replicate 表过滤
# propertyreplicator.filter.replicate.doPUBLIC.*
# propertyreplicator.filter.replicate.ignorePUBLIC.T1
## casetransform 大小写转换
# propertyreplicator.filter.casetransform.to_upper_casetrue## 优化参数
# 开启小事务合并入库
propertyreplicator.applier.dbms.optimizeRowEventstrue
# 指定单表一次入库的数据量
propertyreplicator.applier.dbms.maxRowBatchSize5000# 自动恢复尝试次数系统 OFFLINE 时尝试将系统重新置于ONLINE状态的次数
repl_auto_recovery_max_attempts0
# 系统 OFFLINE 时尝试 ONLINE 之前的等待时间s(秒)、m(分钟)、h(小时)
repl_auto_recovery_delay_interval30s
# 系统自动恢复次数重置时间s(秒)、m(分钟)、h(小时)
repl_auto_recovery_reset_interval300s
[flysyncnode2 ini]$
(4).执行安装
[flysyncnode2 ini]$ cd /install/KingbaseFlySync-V002R002C004B20230320-replicator/
[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$ ./tools/fspm install
..
WARN node2 ntpd is not running. It is important that configured hosts have time synchronised. (NtpdRunningCheck)
WARN node2 Linux swappiness is currently set to 30, on restart it will be 0, consider setting this to 10 or under to avoid swapping. (SwappinessCheck)
License : /home/flysync/replicator/license.dat
.#####################################################################
# Next Steps
#####################################################################
Unless automatically started, you must start the FlySync services before the
cluster will be available./home/flysync/replicator/flysync/cluster-home/bin/startallWait a minute for the services to start up and configure themselves. After
that you may proceed.We have added FlySync environment variables to ~/.bash_profile.
Run source ~/.bash_profile to rebuild your environment.Once your services start successfully replication will begin.
To look at services and perform administration, run the following command
from any database server./home/flysync/replicator/flysync/flysync-replicator/bin/fsrepctl servicesConfiguration is now complete. For further information, please consult
FlySync documentation.NOTE Command successfully completed
[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$ (5).上传并更新同步软件授权
[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$ cp /install/license.dat ~/replicator/license.dat
[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$ ls ~/replicator/license.dat
/home/flysync/replicator/license.dat
[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$ (6).让环境变量生效
[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$ source ~/.bash_profile
[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$ (7).配置映射策略
将exam 模式下的增量数据映射到exam0017下。将exam.course 表的增量数据映射到 exam0017.course0017下。
[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$ vi ~/replicator/filters-config/rename.csv
[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$ cat ~/replicator/filters-config/rename.csv
#######################################
# RenameFilter sample definition file #
######################################## CSV format:
#
# originalSchema,originalTable,originalColumn,newSchema,newTable,newColumn
#
# Left side (first three columns) may contain a special character:
#
# * - match all [schemas/tables/columns] (part-string matching not supported)
#
# Right side (last three columns) may contain a special character:
#
# - - leave as original# In case of multiple entries matching the same request, order of preference is
# important. Consider the following example:
#
# schemac,*,*,schemac1,-,-
# schemac,tablew,*,schemac2,-,-
#
# In this example all tables are moved to schemac1 except tablew, which is
# moved to schemac2. Heres the rules being followed:
#
# Order of preference when looking up schema name:
#
# 1. schema.table
# 2. schema.*
#
# Order of preference when looking up table name:
#
# 1. schema.table
# 2. *.table
#
# Order of preference when looking up column name:
#
# 1. schema.table
# 2. schema.*
# 3. *.table
# 4. *.*
#
# By the way, order of preference is fixed in order to have a fast lookup.# Notes:
#
# * Case sensitivity matters. If it is required to ignore case, use another
# filter in the pipeline before to transform case.
#
# * For renaming columns, make sure to have column names in KUFL events. For
# some DBMS types a separate filter (eg. ColumnNameFilter) might be needed.############
# Examples #
############# Rename schema for all tables.
schemac,*,*,schemac1,-,-# This table moved to a different schema.
schemac,tablew,*,schemac2,-,-# Table renamed and moved to a different schema.
schemav,tablee,*,schemav1,tableee,-# Rename table in a particular schema.
schemab,tabler,*,-,tabler1,-# Rename each table with a given name across all schemas.
*,tablet,*,-,tablet1,-# Rename column of this particular table.
schemaz,tableq,cola,-,-,colaa # Column renaming doesnt change schema/table.# Rename column of each table with a given name across all schemas.
*,tablew,colb,-,-,colbb# Rename column occurring in any table across all schemas.
*,*,colc,-,-,colcc# Rename column across all tables in the given schema.
schemax,*,cold,-,-,coldd# Renamed schema, but one table is left in it and some of its columns renamed.
scheman,*,*,scheman2,-,-
scheman,tabley,*,-,tabley2,- # Leave table in original schema.
scheman,tabley,colj,-,-,colj2
scheman,tabley,colk,-,-,colk2# Renamed schema and one table with one column in it.
schemam,*,*,schemam2,-,-
schemam,tableu,*,schemam2,tableu2,- # Move renamed table to new schema too.
schemam,tableu,colj,-,-,colj2 # Column renaming doesnt change schema/table.
# --以下为新增内容--
exam,*,*,exam0017,-,-
exam,course,*,exam0017,course0017,-
flysync_kingbase8_source,trep_commit_seqno,*,flysync_kingbase8_target,-,-
flysync_kingbase8_source,consistency,*,flysync_kingbase8_target,-,-
flysync_kingbase8_source,heartbeat,*,flysync_kingbase8_target,-,-
flysync_kingbase8_source,trep_shard,*,flysync_kingbase8_target,-,-
flysync_kingbase8_source,trep_shard_channel,*,flysync_kingbase8_target,-,-
[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$
(8).启动同步程序并置于offline 状态
[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$ replicator start offline
Starting FlySync Replicator Service...
Waiting for FlySync Replicator Service......
running: PID:53248(9).查看同步软件的运行状态
[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$ fsrepctl -service kingbase8_target services
Processing services command...
NAME VALUE
---- -----
appliedLastSeqno: -1
appliedLatency : -1.0
role : slave
serviceName : kingbase8_target
serviceType : local
started : true
state : OFFLINE:NORMAL
Finished services command...
[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$ 5.KFS存量数据搬迁
(1)检查状态
源端node1
[flysyncnode1 ~]$ fsrepctl -service kingbase8_source services
Processing services command...
NAME VALUE
---- -----
appliedLastSeqno: -1
appliedLatency : -1.0
role : master
serviceName : kingbase8_source
serviceType : local
started : true
state : OFFLINE:NORMAL
Finished services command...
[flysyncnode1 ~]$
目标端node2
[flysyncnode2 ~]$ fsrepctl -service kingbase8_target services
Processing services command...
NAME VALUE
---- -----
appliedLastSeqno: -1
appliedLatency : -1.0
role : slave
serviceName : kingbase8_target
serviceType : local
started : true
state : OFFLINE:NORMAL
Finished services command...
[flysyncnode2 ~]$ ddlscan 和loader迁移模式选其中一种就好 (2).ddlscan结构搬迁
a.新建模式exam0017
[rootnode2 ~]# su kingbase
[kingbasenode2 root]$ exit
exit
[rootnode2 ~]# su - kingbase
上一次登录四 11月 16 22:58:52 CST 2023pts/0 上
[kingbasenode2 ~]$ cd /KingbaseES/V8/Server/bin/
[kingbasenode2 bin]$ ./ksql -U flysync -d test
ksql (V8.0)
输入 help 来获取帮助信息.test# CREATE SCHEMA exam0017;
CREATE SCHEMA
test# \q
[kingbasenode2 bin]$ b.开始迁移数据
[kingbasenode2 bin]$ exit
[rootnode2 ~]# su - flysync
上一次登录四 11月 16 23:04:05 CST 2023pts/0 上
[flysyncnode2 ~]$ ddlscan -target.service kingbase8_target -source.user flysync -source.pass flysync -source.db test -source.dbtype kingbase8 -source.host node1 -source.port 54321 -source.schema exam -target.db test -mgType 0| [ - main] INFO ddlscan.DDLScanWorker Tables or TableFile not specified - extracting everything!| [ - main] INFO ddlscan.DDLScanWorker Connect to source database with url: jdbc:kingbase8://node1:54321/test| [ - main] INFO database.AbstractDatabase Loading database driver: com.kingbase8.Driver| [ - main] INFO filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:| [ - main] INFO filter.RenameDefinitions schemac.*.* - schemac1.-.-| [ - main] INFO filter.RenameDefinitions schemac.tablew.* - schemac2.-.-| [ - main] INFO filter.RenameDefinitions schemav.tablee.* - schemav1.tableee.-| [ - main] INFO filter.RenameDefinitions schemab.tabler.* - -.tabler1.-| [ - main] INFO filter.RenameDefinitions *.tablet.* - -.tablet1.-| [ - main] INFO filter.RenameDefinitions schemaz.tableq.cola - -.-.colaa| [ - main] INFO filter.RenameDefinitions *.tablew.colb - -.-.colbb| [ - main] INFO filter.RenameDefinitions *.*.colc - -.-.colcc| [ - main] INFO filter.RenameDefinitions schemax.*.cold - -.-.coldd| [ - main] INFO filter.RenameDefinitions scheman.*.* - scheman2.-.-| [ - main] INFO filter.RenameDefinitions scheman.tabley.* - -.tabley2.-| [ - main] INFO filter.RenameDefinitions scheman.tabley.colj - -.-.colj2| [ - main] INFO filter.RenameDefinitions scheman.tabley.colk - -.-.colk2| [ - main] INFO filter.RenameDefinitions schemam.*.* - schemam2.-.-| [ - main] INFO filter.RenameDefinitions schemam.tableu.* - schemam2.tableu2.-| [ - main] INFO filter.RenameDefinitions schemam.tableu.colj - -.-.colj2| [ - main] INFO filter.RenameDefinitions exam.*.* - exam0017.-.-| [ - main] INFO filter.RenameDefinitions exam.course.* - exam0017.course0017.-| [ - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_commit_seqno.* - flysync_kingbase8_target.-.-| [ - main] INFO filter.RenameDefinitions flysync_kingbase8_source.consistency.* - flysync_kingbase8_target.-.-| [ - main] INFO filter.RenameDefinitions flysync_kingbase8_source.heartbeat.* - flysync_kingbase8_target.-.-| [ - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard.* - flysync_kingbase8_target.-.-| [ - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard_channel.* - flysync_kingbase8_target.-.-| [ - main] INFO ddlscan.DDLScanWorker There is 5 tables to migration.| [ - main] INFO ddlscan.TargetDDLScanWorker Connect to target database with url: jdbc:kingbase8://node2:54321/test| [ - main] INFO ddlscan.TargetDDLScanWorker Drop tables on target database.| [ - main] INFO ddlscan.DDLScanCtrl
--------------------------------------------------------------------------------
TOTAL: 5
SUCCESS: 5
FAILURE: 0
TOTAL TIME: 1s
FINISHED AT: 2023-11-16 23:04:18
OUT: /tmp/ddlscan_20231116_230417.sql
ERROR:
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|NUM |SOURCE_TABLE |TARGET_TABLE |STATUS |MESSAGE |
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|1 |exam.course |exam0017.course0017 |Success | |
|2 |exam.instructor |exam0017.instructor |Success | |
|3 |exam.instructor_history |exam0017.instructor_history |Success | |
|4 |exam.score |exam0017.score |Success | |
|5 |exam.student |exam0017.student |Success | |
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------[flysyncnode2 ~]$ (3).loader数据搬迁(极速模式)
[flysyncnode2 ~]$ loader -source.service kingbase8_source -source.user flysync -source.pass flysync \-source.db test -source.dbtype kingbase8 -source.host node1 \-source.port 54321 -source.schema exam \-source.rmiHost node1 -source.rmiPort 11000 \-target.rmiHost node2 -target.rmiPort 11000 \-target.service kingbase8_target -mgType 0 -clean| [ - main] INFO database.AbstractDatabase Loading database driver: com.kingbase8.Driver| [ - main] INFO loader.LoaderWorker Tables not specified - extracting everything!| [ - main] INFO loader.LoaderWorker Import table exam.course| [ - main] INFO loader.LoaderWorker Import table exam.instructor| [ - main] INFO loader.LoaderWorker Import table exam.instructor_history| [ - main] INFO loader.LoaderWorker Import table exam.score| [ - main] INFO loader.LoaderWorker Import table exam.student| [ - main] INFO conf.ReplicatorRuntime Replicator role: slave| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting consistencyFailureStop to true| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnNames to true| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnTypes to true| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting replicator.applier.failOnZeroRowUpdate to warn| [kingbase8_target - main] INFO pipeline.Pipeline Configuring pipeline: slave| [kingbase8_target - main] INFO datasource.DataSourceService Configuring data source: nameglobal| [kingbase8_target - main] INFO datasource.DataSourceManager Loading data source: nameglobal classNamecom.kingbase.flysync.replicator.datasource.KingbaseDataSource| [kingbase8_target - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings| [kingbase8_target - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [kingbase8_target - main] INFO datasource.DataSourceService Configuring data source: nameextractor| [kingbase8_target - main] INFO datasource.DataSourceManager Loading data source: nameextractor classNamecom.kingbase.flysync.replicator.datasource.AliasDataSource| [kingbase8_target - main] INFO datasource.DataSourceService Configuring data source: namefile_metadata| [kingbase8_target - main] INFO datasource.DataSourceManager Loading data source: namefile_metadata classNamecom.kingbase.flysync.replicator.datasource.FileDataSource| [kingbase8_target - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings| [kingbase8_target - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [kingbase8_target - main] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: loader-to-q| [kingbase8_target - main] INFO extractor.ExtractorWrapper Configuring raw extractor and heartbeat filter| [kingbase8_target - main] INFO event.EventMetadataFilter Use default schema for unknown SQL statements: false| [kingbase8_target - main] INFO extractor.ExtractorWrapper Master auto-repositioning on source_id change is enabled; extractor will reposition current log position if last extracted source_id differs from current source_id| [kingbase8_target - main] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: q-to-dbms| [kingbase8_target - main] INFO conf.ReplicatorRuntime Preparing pipeline for use: slave| [kingbase8_target - main] INFO pipeline.Pipeline Preparing pipeline: slave| [kingbase8_target - main] INFO datasource.DataSourceService Preparing and initializing data source: namefile_metadata| [kingbase8_target - main] INFO datasource.FileDataSource Service directory does not exist, creating: /home/flysync/replicator/metadata/applier/kingbase8_target| [kingbase8_target - main] INFO datasource.FileDataSource Initializing data source files: servicekingbase8_target directory/home/flysync/replicator/metadata/applier| [kingbase8_target - main] INFO datasource.FileCommitSeqno Initializing file-based seqno tracking: directory/home/flysync/replicator/metadata/applier/kingbase8_target filecommitseqno.0| [kingbase8_target - pool-1-thread-1] INFO pipeline.DeferredShutdownTask Waiting for pipeline to shut down: slave| [kingbase8_target - main] INFO datasource.FileCommitSeqno Validated that trep_commit_seqno file count matches channels: files1 channels1| [kingbase8_target - main] INFO datasource.DataSourceService Preparing and initializing data source: nameglobal| [kingbase8_target - main] INFO datasource.DataSourceService Setting catalog handling for privileged slave/relay: nameglobal| [kingbase8_target - main] INFO datasource.DataSourceService Disabling logging of updates for slave/relay: nameglobal| [kingbase8_target - main] INFO pipeline.StageTaskGroup Preparing tasks for stage: q-to-dbms| [kingbase8_target - main] INFO filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.*.* - schemac1.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.tablew.* - schemac2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemav.tablee.* - schemav1.tableee.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemab.tabler.* - -.tabler1.-| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablet.* - -.tablet1.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemaz.tableq.cola - -.-.colaa| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablew.colb - -.-.colbb| [kingbase8_target - main] INFO filter.RenameDefinitions *.*.colc - -.-.colcc| [kingbase8_target - main] INFO filter.RenameDefinitions schemax.*.cold - -.-.coldd| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.*.* - scheman2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.* - -.tabley2.-| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colj - -.-.colj2| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colk - -.-.colk2| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.*.* - schemam2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.* - schemam2.tableu2.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.colj - -.-.colj2| [kingbase8_target - main] INFO filter.RenameDefinitions exam.*.* - exam0017.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions exam.course.* - exam0017.course0017.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_commit_seqno.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.consistency.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.heartbeat.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard_channel.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO applier.JdbcApplier Connecting to data source| [kingbase8_target - main] INFO applier.JdbcApplier Suppressing logging on privileged slave| [kingbase8_target - main] WARN pipeline.StageTaskGroup [main-q-to-dbms-0] Last event data not available; Setting extractor to current position| [kingbase8_target - main] INFO pipeline.StageTaskGroup Preparing tasks for stage: loader-to-q| [kingbase8_target - main] INFO extractor.ExtractorWrapper Preparing raw extractor and heartbeat filter| [kingbase8_target - main] WARN pipeline.StageTaskGroup [main-loader-to-q-0] Last event data not available; Setting extractor to current position| [kingbase8_target - main] INFO pipeline.Pipeline Starting pipeline: slave| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Starting stage task thread| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Starting stage task thread| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Task has been cancelled| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno5 eventidNIL| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Task event count: 6| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Task has been cancelled| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno5 eventidNIL| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Task event count: 6| [kingbase8_target - pool-1-thread-1] INFO pipeline.DeferredShutdownTask Pipeline has shut down, dispatching offline event: slave| [kingbase8_target - main] INFO pipeline.Pipeline Shutting down pipeline: slave| [kingbase8_target - main] INFO pipeline.Pipeline Releasing pipeline: slave| [kingbase8_target - main] INFO pipeline.StageTaskGroup Releasing tasks for stage: loader-to-q| [kingbase8_target - main] INFO extractor.ExtractorWrapper Releasing raw extractor and heartbeat filter| [kingbase8_target - main] INFO pipeline.StageTaskGroup Releasing tasks for stage: q-to-dbms| [kingbase8_target - main] INFO conf.ReplicatorRuntime Replicator role: slave| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting consistencyFailureStop to true| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnNames to true| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnTypes to true| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting replicator.applier.failOnZeroRowUpdate to warn| [kingbase8_target - main] INFO pipeline.Pipeline Configuring pipeline: slave| [kingbase8_target - main] INFO datasource.DataSourceService Configuring data source: nameglobal| [kingbase8_target - main] INFO datasource.DataSourceManager Loading data source: nameglobal classNamecom.kingbase.flysync.replicator.datasource.KingbaseDataSource| [kingbase8_target - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings| [kingbase8_target - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [kingbase8_target - main] INFO datasource.DataSourceService Configuring data source: nameextractor| [kingbase8_target - main] INFO datasource.DataSourceManager Loading data source: nameextractor classNamecom.kingbase.flysync.replicator.datasource.AliasDataSource| [kingbase8_target - main] INFO datasource.DataSourceService Configuring data source: namefile_metadata| [kingbase8_target - main] INFO datasource.DataSourceManager Loading data source: namefile_metadata classNamecom.kingbase.flysync.replicator.datasource.FileDataSource| [kingbase8_target - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings| [kingbase8_target - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [kingbase8_target - main] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: loader-to-q| [kingbase8_target - main] INFO extractor.ExtractorWrapper Configuring raw extractor and heartbeat filter| [kingbase8_target - main] INFO event.EventMetadataFilter Use default schema for unknown SQL statements: false| [kingbase8_target - main] INFO extractor.ExtractorWrapper Master auto-repositioning on source_id change is enabled; extractor will reposition current log position if last extracted source_id differs from current source_id| [kingbase8_target - main] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: q-to-dbms| [kingbase8_target - main] INFO conf.ReplicatorRuntime Preparing pipeline for use: slave| [kingbase8_target - main] INFO pipeline.Pipeline Preparing pipeline: slave| [kingbase8_target - main] INFO datasource.DataSourceService Preparing and initializing data source: namefile_metadata| [kingbase8_target - main] INFO datasource.FileDataSource Initializing data source files: servicekingbase8_target directory/home/flysync/replicator/metadata/applier| [kingbase8_target - main] INFO datasource.FileCommitSeqno Validated that trep_commit_seqno file count matches channels: files1 channels1| [kingbase8_target - main] INFO datasource.DataSourceService Preparing and initializing data source: nameglobal| [kingbase8_target - main] INFO datasource.DataSourceService Setting catalog handling for privileged slave/relay: nameglobal| [kingbase8_target - main] INFO datasource.DataSourceService Disabling logging of updates for slave/relay: nameglobal| [kingbase8_target - pool-5-thread-1] INFO pipeline.DeferredShutdownTask Waiting for pipeline to shut down: slave| [kingbase8_target - main] INFO pipeline.StageTaskGroup Preparing tasks for stage: q-to-dbms| [kingbase8_target - main] INFO filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.*.* - schemac1.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.tablew.* - schemac2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemav.tablee.* - schemav1.tableee.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemab.tabler.* - -.tabler1.-| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablet.* - -.tablet1.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemaz.tableq.cola - -.-.colaa| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablew.colb - -.-.colbb| [kingbase8_target - main] INFO filter.RenameDefinitions *.*.colc - -.-.colcc| [kingbase8_target - main] INFO filter.RenameDefinitions schemax.*.cold - -.-.coldd| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.*.* - scheman2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.* - -.tabley2.-| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colj - -.-.colj2| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colk - -.-.colk2| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.*.* - schemam2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.* - schemam2.tableu2.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.colj - -.-.colj2| [kingbase8_target - main] INFO filter.RenameDefinitions exam.*.* - exam0017.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions exam.course.* - exam0017.course0017.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_commit_seqno.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.consistency.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.heartbeat.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard_channel.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO applier.JdbcApplier Connecting to data source| [kingbase8_target - main] INFO applier.JdbcApplier Suppressing logging on privileged slave| [kingbase8_target - main] WARN pipeline.StageTaskGroup [main-q-to-dbms-0] Last event data not available; Setting extractor to current position| [kingbase8_target - main] INFO filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.*.* - schemac1.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.tablew.* - schemac2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemav.tablee.* - schemav1.tableee.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemab.tabler.* - -.tabler1.-| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablet.* - -.tablet1.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemaz.tableq.cola - -.-.colaa| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablew.colb - -.-.colbb| [kingbase8_target - main] INFO filter.RenameDefinitions *.*.colc - -.-.colcc| [kingbase8_target - main] INFO filter.RenameDefinitions schemax.*.cold - -.-.coldd| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.*.* - scheman2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.* - -.tabley2.-| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colj - -.-.colj2| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colk - -.-.colk2| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.*.* - schemam2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.* - schemam2.tableu2.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.colj - -.-.colj2| [kingbase8_target - main] INFO filter.RenameDefinitions exam.*.* - exam0017.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions exam.course.* - exam0017.course0017.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_commit_seqno.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.consistency.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.heartbeat.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard_channel.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO applier.JdbcApplier Connecting to data source| [kingbase8_target - main] INFO applier.JdbcApplier Suppressing logging on privileged slave| [kingbase8_target - main] WARN pipeline.StageTaskGroup [main-q-to-dbms-1] Last event data not available; Setting extractor to current position| [kingbase8_target - main] INFO filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.*.* - schemac1.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.tablew.* - schemac2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemav.tablee.* - schemav1.tableee.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemab.tabler.* - -.tabler1.-| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablet.* - -.tablet1.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemaz.tableq.cola - -.-.colaa| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablew.colb - -.-.colbb| [kingbase8_target - main] INFO filter.RenameDefinitions *.*.colc - -.-.colcc| [kingbase8_target - main] INFO filter.RenameDefinitions schemax.*.cold - -.-.coldd| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.*.* - scheman2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.* - -.tabley2.-| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colj - -.-.colj2| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colk - -.-.colk2| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.*.* - schemam2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.* - schemam2.tableu2.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.colj - -.-.colj2| [kingbase8_target - main] INFO filter.RenameDefinitions exam.*.* - exam0017.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions exam.course.* - exam0017.course0017.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_commit_seqno.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.consistency.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.heartbeat.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard_channel.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO applier.JdbcApplier Connecting to data source| [kingbase8_target - main] INFO applier.JdbcApplier Suppressing logging on privileged slave| [kingbase8_target - main] WARN pipeline.StageTaskGroup [main-q-to-dbms-2] Last event data not available; Setting extractor to current position| [kingbase8_target - main] INFO pipeline.StageTaskGroup Preparing tasks for stage: loader-to-q| [kingbase8_target - main] INFO extractor.ExtractorWrapper Preparing raw extractor and heartbeat filter| [kingbase8_target - main] WARN pipeline.StageTaskGroup [main-loader-to-q-0] Last event data not available; Setting extractor to current position| [kingbase8_target - main] INFO pipeline.Pipeline Starting pipeline: slave| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Starting stage task thread| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Starting stage task thread| [kingbase8_target - main-q-to-dbms-2] INFO pipeline.SingleThreadStageTask Starting stage task thread| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Task has been cancelled| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno5 eventidNIL| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Task event count: 6| [kingbase8_target - main-q-to-dbms-0] INFO applier.JdbcApplier nologging: exam0017.course0017| [kingbase8_target - main-q-to-dbms-0] INFO applier.JdbcApplier nologging: exam0017.score| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Task has been cancelled| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Performing rollback of possible partial transaction: seqno(unavailable)| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno5 eventidNIL| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Task event count: 3| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Starting stage task thread| [kingbase8_target - main-q-to-dbms-1] INFO applier.JdbcApplier nologging: exam0017.instructor| [kingbase8_target - main-q-to-dbms-2] INFO applier.JdbcApplier nologging: exam0017.instructor_history| [kingbase8_target - main-q-to-dbms-2] INFO pipeline.SingleThreadStageTask Task has been cancelled| [kingbase8_target - main-q-to-dbms-2] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread| [kingbase8_target - main-q-to-dbms-2] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno5 eventidNIL| [kingbase8_target - main-q-to-dbms-1] INFO applier.JdbcApplier nologging: exam0017.student| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Task has been cancelled| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Performing rollback of possible partial transaction: seqno(unavailable)| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno5 eventidNIL| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Task event count: 3| [kingbase8_target - main-q-to-dbms-2] INFO pipeline.SingleThreadStageTask Task event count: 2| [kingbase8_target - pool-5-thread-1] INFO pipeline.DeferredShutdownTask Pipeline has shut down, dispatching offline event: slave| [kingbase8_target - main] INFO pipeline.Pipeline Shutting down pipeline: slave| [kingbase8_target - main] INFO pipeline.Pipeline Releasing pipeline: slave| [kingbase8_target - main] INFO pipeline.StageTaskGroup Releasing tasks for stage: loader-to-q| [kingbase8_target - main] INFO extractor.ExtractorWrapper Releasing raw extractor and heartbeat filter| [kingbase8_target - main] INFO pipeline.StageTaskGroup Releasing tasks for stage: q-to-dbms| [kingbase8_target - pipeline-0] INFO conf.ReplicatorRuntime Replicator role: slave| [kingbase8_target - pipeline-0] INFO conf.ReplicatorRuntime Setting consistencyFailureStop to true| [kingbase8_target - pipeline-0] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnNames to true| [kingbase8_target - pipeline-0] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnTypes to true| [kingbase8_target - pipeline-0] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn| [kingbase8_target - pipeline-0] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn| [kingbase8_target - pipeline-0] INFO conf.ReplicatorRuntime Setting replicator.applier.failOnZeroRowUpdate to warn| [kingbase8_target - pipeline-0] INFO pipeline.Pipeline Configuring pipeline: slave| [kingbase8_target - pipeline-0] INFO datasource.DataSourceService Configuring data source: nameglobal| [kingbase8_target - pipeline-0] INFO datasource.DataSourceManager Loading data source: nameglobal classNamecom.kingbase.flysync.replicator.datasource.KingbaseDataSource| [kingbase8_target - pipeline-0] INFO datasource.AbstractDataSource No cvsType provided; using default settings| [kingbase8_target - pipeline-0] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [kingbase8_target - pipeline-0] INFO datasource.DataSourceService Configuring data source: nameextractor| [kingbase8_target - pipeline-0] INFO datasource.DataSourceManager Loading data source: nameextractor classNamecom.kingbase.flysync.replicator.datasource.AliasDataSource| [kingbase8_target - pipeline-0] INFO datasource.DataSourceService Configuring data source: namefile_metadata| [kingbase8_target - pipeline-0] INFO datasource.DataSourceManager Loading data source: namefile_metadata classNamecom.kingbase.flysync.replicator.datasource.FileDataSource| [kingbase8_target - pipeline-0] INFO datasource.AbstractDataSource No cvsType provided; using default settings| [kingbase8_target - pipeline-0] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [kingbase8_target - pipeline-0] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: loader-to-q| [kingbase8_target - pipeline-0] INFO extractor.ExtractorWrapper Configuring raw extractor and heartbeat filter| [kingbase8_target - pipeline-0] INFO event.EventMetadataFilter Use default schema for unknown SQL statements: false| [kingbase8_target - pipeline-0] INFO extractor.ExtractorWrapper Master auto-repositioning on source_id change is enabled; extractor will reposition current log position if last extracted source_id differs from current source_id| [kingbase8_target - pipeline-0] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: q-to-dbms| [kingbase8_target - pipeline-0] INFO conf.ReplicatorRuntime Preparing pipeline for use: slave| [kingbase8_target - pipeline-0] INFO pipeline.Pipeline Preparing pipeline: slave| [kingbase8_target - pipeline-0] INFO datasource.DataSourceService Preparing and initializing data source: namefile_metadata| [kingbase8_target - pipeline-0] INFO datasource.FileDataSource Initializing data source files: servicekingbase8_target directory/home/flysync/replicator/metadata/applier| [kingbase8_target - pipeline-0] INFO datasource.FileCommitSeqno Validated that trep_commit_seqno file count matches channels: files1 channels1| [kingbase8_target - pipeline-0] INFO datasource.DataSourceService Preparing and initializing data source: nameglobal| [kingbase8_target - pipeline-0] INFO datasource.DataSourceService Setting catalog handling for privileged slave/relay: nameglobal| [kingbase8_target - pipeline-0] INFO datasource.DataSourceService Disabling logging of updates for slave/relay: nameglobal| [kingbase8_target - pool-11-thread-1] INFO pipeline.DeferredShutdownTask Waiting for pipeline to shut down: slave| [kingbase8_target - pipeline-0] INFO pipeline.StageTaskGroup Preparing tasks for stage: q-to-dbms| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions schemac.*.* - schemac1.-.-| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions schemac.tablew.* - schemac2.-.-| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions schemav.tablee.* - schemav1.tableee.-| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions schemab.tabler.* - -.tabler1.-| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions *.tablet.* - -.tablet1.-| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions schemaz.tableq.cola - -.-.colaa| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions *.tablew.colb - -.-.colbb| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions *.*.colc - -.-.colcc| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions schemax.*.cold - -.-.coldd| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions scheman.*.* - scheman2.-.-| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions scheman.tabley.* - -.tabley2.-| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions scheman.tabley.colj - -.-.colj2| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions scheman.tabley.colk - -.-.colk2| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions schemam.*.* - schemam2.-.-| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions schemam.tableu.* - schemam2.tableu2.-| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions schemam.tableu.colj - -.-.colj2| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions exam.*.* - exam0017.-.-| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions exam.course.* - exam0017.course0017.-| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_commit_seqno.* - flysync_kingbase8_target.-.-| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions flysync_kingbase8_source.consistency.* - flysync_kingbase8_target.-.-| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions flysync_kingbase8_source.heartbeat.* - flysync_kingbase8_target.-.-| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard.* - flysync_kingbase8_target.-.-| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard_channel.* - flysync_kingbase8_target.-.-| [kingbase8_target - pipeline-0] INFO applier.JdbcApplier Connecting to data source| [kingbase8_target - pipeline-0] INFO applier.JdbcApplier Suppressing logging on privileged slave| [kingbase8_target - pipeline-0] WARN pipeline.StageTaskGroup [pipeline-0-q-to-dbms-0] Last event data not available; Setting extractor to current position| [kingbase8_target - pipeline-0] INFO pipeline.StageTaskGroup Preparing tasks for stage: loader-to-q| [kingbase8_target - pipeline-0] INFO extractor.ExtractorWrapper Preparing raw extractor and heartbeat filter| [kingbase8_target - pipeline-0] WARN pipeline.StageTaskGroup [pipeline-0-loader-to-q-0] Last event data not available; Setting extractor to current position| [kingbase8_target - pipeline-0] INFO pipeline.Pipeline Starting pipeline: slave| [kingbase8_target - pipeline-0-loader-to-q-0] INFO pipeline.SingleThreadStageTask Starting stage task thread| [kingbase8_target - pipeline-0-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Starting stage task thread| [kingbase8_target - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from exam.course:0-3| [kingbase8_target - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from exam.instructor:0-7| [kingbase8_target - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from exam.instructor_history:0-1| [kingbase8_target - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from exam.score:0-15| [kingbase8_target - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from exam.student:0-8| [kingbase8_target - pipeline-0-loader-to-q-0] INFO pipeline.SingleThreadStageTask Task has been cancelled| [kingbase8_target - pipeline-0-loader-to-q-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread| [kingbase8_target - pipeline-0-loader-to-q-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno5 eventidNIL| [kingbase8_target - pipeline-0-loader-to-q-0] INFO pipeline.SingleThreadStageTask Task event count: 6| [kingbase8_target - pipeline-0-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Task has been cancelled| [kingbase8_target - pipeline-0-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread| [kingbase8_target - pipeline-0-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno5 eventidNIL| [kingbase8_target - pipeline-0-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Task event count: 6| [kingbase8_target - pool-11-thread-1] INFO pipeline.DeferredShutdownTask Pipeline has shut down, dispatching offline event: slave| [kingbase8_target - pipeline-0] INFO pipeline.Pipeline Shutting down pipeline: slave| [kingbase8_target - pipeline-0] INFO pipeline.Pipeline Releasing pipeline: slave| [kingbase8_target - pipeline-0] INFO pipeline.StageTaskGroup Releasing tasks for stage: loader-to-q| [kingbase8_target - pipeline-0] INFO extractor.ExtractorWrapper Releasing raw extractor and heartbeat filter| [kingbase8_target - pipeline-0] INFO pipeline.StageTaskGroup Releasing tasks for stage: q-to-dbms| [kingbase8_target - main] INFO loader.TargetLoaderWorker Tables imported| [kingbase8_target - main] INFO loader.LoaderWorker | -------------------------打印执行结果开始-------------------------| [kingbase8_target - main] INFO loader.LoaderWorker | exam.instructor_history : success| [kingbase8_target - main] INFO loader.LoaderWorker | exam.course : success| [kingbase8_target - main] INFO loader.LoaderWorker | exam.student : success| [kingbase8_target - main] INFO loader.LoaderWorker | exam.score : success| [kingbase8_target - main] INFO loader.LoaderWorker | exam.instructor : success| [kingbase8_target - main] INFO loader.LoaderWorker | -------------------------打印执行结果结束-------------------------| [kingbase8_target - main] INFO conf.ReplicatorRuntime Replicator role: slave| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting consistencyFailureStop to true| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnNames to true| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnTypes to true| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting replicator.applier.failOnZeroRowUpdate to warn| [kingbase8_target - main] INFO pipeline.Pipeline Configuring pipeline: slave| [kingbase8_target - main] INFO datasource.DataSourceService Configuring data source: nameglobal| [kingbase8_target - main] INFO datasource.DataSourceManager Loading data source: nameglobal classNamecom.kingbase.flysync.replicator.datasource.KingbaseDataSource| [kingbase8_target - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings| [kingbase8_target - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [kingbase8_target - main] INFO datasource.DataSourceService Configuring data source: nameextractor| [kingbase8_target - main] INFO datasource.DataSourceManager Loading data source: nameextractor classNamecom.kingbase.flysync.replicator.datasource.AliasDataSource| [kingbase8_target - main] INFO datasource.DataSourceService Configuring data source: namefile_metadata| [kingbase8_target - main] INFO datasource.DataSourceManager Loading data source: namefile_metadata classNamecom.kingbase.flysync.replicator.datasource.FileDataSource| [kingbase8_target - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings| [kingbase8_target - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [kingbase8_target - main] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: loader-to-q| [kingbase8_target - main] INFO extractor.ExtractorWrapper Configuring raw extractor and heartbeat filter| [kingbase8_target - main] INFO event.EventMetadataFilter Use default schema for unknown SQL statements: false| [kingbase8_target - main] INFO extractor.ExtractorWrapper Master auto-repositioning on source_id change is enabled; extractor will reposition current log position if last extracted source_id differs from current source_id| [kingbase8_target - main] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: q-to-dbms| [kingbase8_target - main] INFO conf.ReplicatorRuntime Preparing pipeline for use: slave| [kingbase8_target - main] INFO pipeline.Pipeline Preparing pipeline: slave| [kingbase8_target - main] INFO datasource.DataSourceService Preparing and initializing data source: namefile_metadata| [kingbase8_target - main] INFO datasource.FileDataSource Initializing data source files: servicekingbase8_target directory/home/flysync/replicator/metadata/applier| [kingbase8_target - main] INFO datasource.FileCommitSeqno Validated that trep_commit_seqno file count matches channels: files1 channels1| [kingbase8_target - main] INFO datasource.DataSourceService Preparing and initializing data source: nameglobal| [kingbase8_target - main] INFO datasource.DataSourceService Setting catalog handling for privileged slave/relay: nameglobal| [kingbase8_target - main] INFO datasource.DataSourceService Disabling logging of updates for slave/relay: nameglobal| [kingbase8_target - pool-15-thread-1] INFO pipeline.DeferredShutdownTask Waiting for pipeline to shut down: slave| [kingbase8_target - main] INFO pipeline.StageTaskGroup Preparing tasks for stage: q-to-dbms| [kingbase8_target - main] INFO filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.*.* - schemac1.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.tablew.* - schemac2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemav.tablee.* - schemav1.tableee.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemab.tabler.* - -.tabler1.-| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablet.* - -.tablet1.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemaz.tableq.cola - -.-.colaa| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablew.colb - -.-.colbb| [kingbase8_target - main] INFO filter.RenameDefinitions *.*.colc - -.-.colcc| [kingbase8_target - main] INFO filter.RenameDefinitions schemax.*.cold - -.-.coldd| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.*.* - scheman2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.* - -.tabley2.-| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colj - -.-.colj2| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colk - -.-.colk2| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.*.* - schemam2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.* - schemam2.tableu2.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.colj - -.-.colj2| [kingbase8_target - main] INFO filter.RenameDefinitions exam.*.* - exam0017.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions exam.course.* - exam0017.course0017.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_commit_seqno.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.consistency.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.heartbeat.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard_channel.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO applier.JdbcApplier Connecting to data source| [kingbase8_target - main] INFO applier.JdbcApplier Suppressing logging on privileged slave| [kingbase8_target - main] WARN pipeline.StageTaskGroup [main-q-to-dbms-0] Last event data not available; Setting extractor to current position| [kingbase8_target - main] INFO filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.*.* - schemac1.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.tablew.* - schemac2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemav.tablee.* - schemav1.tableee.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemab.tabler.* - -.tabler1.-| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablet.* - -.tablet1.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemaz.tableq.cola - -.-.colaa| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablew.colb - -.-.colbb| [kingbase8_target - main] INFO filter.RenameDefinitions *.*.colc - -.-.colcc| [kingbase8_target - main] INFO filter.RenameDefinitions schemax.*.cold - -.-.coldd| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.*.* - scheman2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.* - -.tabley2.-| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colj - -.-.colj2| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colk - -.-.colk2| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.*.* - schemam2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.* - schemam2.tableu2.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.colj - -.-.colj2| [kingbase8_target - main] INFO filter.RenameDefinitions exam.*.* - exam0017.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions exam.course.* - exam0017.course0017.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_commit_seqno.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.consistency.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.heartbeat.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard_channel.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO applier.JdbcApplier Connecting to data source| [kingbase8_target - main] INFO applier.JdbcApplier Suppressing logging on privileged slave| [kingbase8_target - main] WARN pipeline.StageTaskGroup [main-q-to-dbms-1] Last event data not available; Setting extractor to current position| [kingbase8_target - main] INFO filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.*.* - schemac1.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.tablew.* - schemac2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemav.tablee.* - schemav1.tableee.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemab.tabler.* - -.tabler1.-| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablet.* - -.tablet1.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemaz.tableq.cola - -.-.colaa| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablew.colb - -.-.colbb| [kingbase8_target - main] INFO filter.RenameDefinitions *.*.colc - -.-.colcc| [kingbase8_target - main] INFO filter.RenameDefinitions schemax.*.cold - -.-.coldd| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.*.* - scheman2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.* - -.tabley2.-| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colj - -.-.colj2| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colk - -.-.colk2| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.*.* - schemam2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.* - schemam2.tableu2.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.colj - -.-.colj2| [kingbase8_target - main] INFO filter.RenameDefinitions exam.*.* - exam0017.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions exam.course.* - exam0017.course0017.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_commit_seqno.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.consistency.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.heartbeat.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions flysync_kingbase8_source.trep_shard_channel.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO applier.JdbcApplier Connecting to data source| [kingbase8_target - main] INFO applier.JdbcApplier Suppressing logging on privileged slave| [kingbase8_target - main] WARN pipeline.StageTaskGroup [main-q-to-dbms-2] Last event data not available; Setting extractor to current position| [kingbase8_target - main] INFO pipeline.StageTaskGroup Preparing tasks for stage: loader-to-q| [kingbase8_target - main] INFO extractor.ExtractorWrapper Preparing raw extractor and heartbeat filter| [kingbase8_target - main] WARN pipeline.StageTaskGroup [main-loader-to-q-0] Last event data not available; Setting extractor to current position| [kingbase8_target - main] INFO pipeline.Pipeline Starting pipeline: slave| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Starting stage task thread| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Starting stage task thread| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Starting stage task thread| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Task has been cancelled| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno5 eventidNIL| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Task event count: 6| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Task has been cancelled| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Performing rollback of possible partial transaction: seqno(unavailable)| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno5 eventidNIL| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Task event count: 3| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Task has been cancelled| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Performing rollback of possible partial transaction: seqno(unavailable)| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno5 eventidNIL| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Task event count: 3| [kingbase8_target - main-q-to-dbms-2] INFO pipeline.SingleThreadStageTask Starting stage task thread| [kingbase8_target - main-q-to-dbms-2] INFO pipeline.SingleThreadStageTask Task has been cancelled| [kingbase8_target - main-q-to-dbms-2] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread| [kingbase8_target - main-q-to-dbms-2] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno5 eventidNIL| [kingbase8_target - main-q-to-dbms-2] INFO pipeline.SingleThreadStageTask Task event count: 2| [kingbase8_target - pool-15-thread-1] INFO pipeline.DeferredShutdownTask Pipeline has shut down, dispatching offline event: slave| [kingbase8_target - main] INFO pipeline.Pipeline Shutting down pipeline: slave| [kingbase8_target - main] INFO pipeline.Pipeline Releasing pipeline: slave| [kingbase8_target - main] INFO pipeline.StageTaskGroup Releasing tasks for stage: loader-to-q| [kingbase8_target - main] INFO extractor.ExtractorWrapper Releasing raw extractor and heartbeat filter| [kingbase8_target - main] INFO pipeline.StageTaskGroup Releasing tasks for stage: q-to-dbms| [kingbase8_target - main] INFO loader.LoaderCtrl Total time:2| [kingbase8_target - main] INFO loader.LoaderCtrl
--------------------------------------------------------------------------------
TOTAL: 5
SUCCESS: 5
FAILURE: 0
TOTAL TIME: 2s
FINISHED AT: 2023-11-16 23:07:02
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|NUM |SOURCE_TABLE |TARGET_TABLE |STATUS |TOTAL |
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|1 |exam.score |exam0017.score |Success |15 |
|2 |exam.student |exam0017.student |Success |8 |
|3 |exam.instructor_history |exam0017.instructor_history |Success |1 |
|4 |exam.course |exam0017.course0017 |Success |3 |
|5 |exam.instructor |exam0017.instructor |Success |7 |
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------[flysyncnode2 ~]$6.增量数据同步
(1).node1节点启动同步服务数据开始同步
[flysyncnode1 ~]$ fsrepctl -service kingbase8_source online
[flysyncnode1 ~]$ fsrepctl -service kingbase8_source services
Processing services command...
NAME VALUE
---- -----
appliedLastSeqno: 0
appliedLatency : 0.115
role : master
serviceName : kingbase8_source
serviceType : local
started : true
state : ONLINE
Finished services command...
[flysyncnode1 ~]$
(2).node2节点启动同步服务数据开始同步
[flysyncnode2 ~]$ fsrepctl -service kingbase8_target online
[flysyncnode2 ~]$ fsrepctl -service kingbase8_target services
Processing services command...
NAME VALUE
---- -----
appliedLastSeqno: 0
appliedLatency : 99.334
role : slave
serviceName : kingbase8_target
serviceType : local
started : true
state : ONLINE
Finished services command...
[flysyncnode2 ~]$ (3).同步进程状态查看同步是否正常。
a.查看node1同步状态
[flysyncnode1 ~]$ fsrepctl -service kingbase8_source status
Processing status command...
NAME VALUE
---- -----
appliedLastEventId : kb:31973688:31973840
appliedLastSeqno : 0
appliedLatency : 0.115
autoRecoveryEnabled : false
autoRecoveryTotal : 0
channels : 1
clusterName : kingbase8_source
currentEventId : kb:31973688:31974112
currentTimeMillis : 1700147650924
dataServerHost : node1
extensions :
host : node1
latestEpochNumber : 0
masterConnectUri : kufl://localhost:/
masterListenUri : kufl://node1:3112/
maximumStoredSeqNo : 0
minimumStoredSeqNo : 0
offlineRequests : NONE
pendingError : NONE
pendingErrorCode : NONE
pendingErrorEventId : NONE
pendingErrorSeqno : -1
pendingExceptionMessage: NONE
pipelineSource : UNKNOWN
relativeLatency : 234.457
resourcePrecedence : 99
rmiPort : 11000
role : master
seqnoType : java.lang.Long
serviceName : kingbase8_source
serviceType : local
simpleServiceName : kingbase8_source
siteName : default
sourceId : node1
state : ONLINE
timeInStateSeconds : 234.437
timezone : GMT08:00
transitioningTo :
uptimeSeconds : 3541.458
useSSLConnection : false
version : Kingbase FlySync V002R002C004B20230320
Finished status command...
[flysyncnode1 ~]$
b.查看node2同步状态
[flysyncnode2 ~]$ fsrepctl -service kingbase8_target status
Processing status command...
NAME VALUE
---- -----
appliedLastEventId : kb:31973688:31973840
appliedLastSeqno : 0
appliedLatency : 99.334
autoRecoveryEnabled : false
autoRecoveryTotal : 0
channels : 1
clusterName : kingbase8_target
currentEventId : NONE
currentTimeMillis : 1700147694828
dataServerHost : node2
extensions :
host : node2
latestEpochNumber : 0
masterConnectUri : kufl://node1:3112/
masterListenUri : kufl://node2:3112/
maximumStoredSeqNo : 0
minimumStoredSeqNo : 0
offlineRequests : NONE
pendingError : NONE
pendingErrorCode : NONE
pendingErrorEventId : NONE
pendingErrorSeqno : -1
pendingExceptionMessage: NONE
pipelineSource : kufl://node1:3112/
relativeLatency : 278.361
resourcePrecedence : 99
rmiPort : 11000
role : slave
seqnoType : java.lang.Long
serviceName : kingbase8_target
serviceType : local
simpleServiceName : kingbase8_target
siteName : default
sourceId : node2
state : ONLINE
timeInStateSeconds : 179.158
timezone : GMT08:00
transitioningTo :
uptimeSeconds : 1304.402
useSSLConnection : false
version : Kingbase FlySync V002R002C004B20230320
Finished status command...
[flysyncnode2 ~]$
(4).数据同步验证
a.登录node1的数据库生成数据
[flysyncnode1 ~]$ exit
exit
[rootnode1 install]# su - kingbase
上一次登录四 11月 16 18:09:03 CST 2023pts/1 上
[kingbasenode1 ~]$ cd /KingbaseES/V8/
ClientTools/ desktops/ DtsTools/ Help/ Interface/ jre/ Logs/ ManagerTools/ Scripts/ Server/ Uninstall/ xulrunner/
[kingbasenode1 ~]$ cd /KingbaseES/V8/Server/bin/
[kingbasenode1 bin]$ ./ksql -Usystem test
ksql (V8.0)
输入 help 来获取帮助信息.
test# \d exam.course;数据表 exam.course栏位 | 类型 | 校对规则 | 可空的 | 预设
-----------------------------------------------------------------cno | numeric(2,0) | | not null | cname | character varying(30 char) | | not null | clevel | character varying(13 char) | | | pass_mark | numeric(5,0) | | not null |
索引cno_pk PRIMARY KEY, btree (cno)
检查约束限制pass_mark_ck CHECK (pass_mark 0::numeric)
由引用TABLE exam.score CONSTRAINT cno_fk FOREIGN KEY (cno) REFERENCES exam.course(cno)
test# INSERT INTO exam.course VALUES(40,KFS,middle,80);
INSERT 0 1
test# INSERT INTO exam.instructor VALUES(108,fenxiaoyue,Intermediate Instructor);
INSERT 0 1
test# b.登录node2的数据库查询刚才在node1插入的数据
[flysyncnode2 ~]$ exit
登出
[rootnode2 ~]# su - kingbase
上一次登录四 11月 16 23:03:45 CST 2023pts/0 上
[kingbasenode2 ~]$ cd /KingbaseES/V8/Server/bin/
[kingbasenode2 bin]$ ./ksql -U system test
ksql (V8.0)
输入 help 来获取帮助信息.test# \d exam0017.course0017;数据表 exam0017.course0017栏位 | 类型 | 校对规则 | 可空的 | 预设
-----------------------------------------------------------------cno | numeric(2,0) | | not null | cname | character varying(30 char) | | not null | clevel | character varying(13 char) | | | pass_mark | numeric(5,0) | | not null |
索引course0017_pkey PRIMARY KEY, btree (cno)
test#SELECT * FROM exam0017.course0017 WHERE cno40;cno | cname | clevel | pass_mark
-------------------------------40 | KFS | middle | 80
(1 行记录)test# SELECT * FROM exam0017.instructor WHERE ino108;ino | iname | title
------------------------------------------108 | fenxiaoyue | Intermediate Instructor
(1 行记录)
test# 到这里我们的本次教程就结束了
报错处理
1.启动报错
请以下面命令方式查看报错信息
[flysyncnode1 KingbaseFlySync-V002R002C004B20230320-replicator]$ find /home/flysync/replicator/ -name *.log
/home/flysync/replicator/service_logs/replicator-user.log
/home/flysync/replicator/service_logs/xtrabackup.log
/home/flysync/replicator/service_logs/mysqldump.log
/home/flysync/replicator/releases/KingbaseFlySync-V002R002C004B20230320-replicator_pid9943/flysync-replicator/log/replicator-user.log
/home/flysync/replicator/releases/KingbaseFlySync-V002R002C004B20230320-replicator_pid9943/flysync-replicator/log/fsrepsvc.log
/home/flysync/replicator/releases/KingbaseFlySync-V002R002C004B20230320-replicator_pid9943/flysync-replicator/log/failure_record_file.log[flysyncnode1 KingbaseFlySync-V002R002C004B20230320-replicator]$ tail -100 /home/flysync/replicator/releases/KingbaseFlySync-V002R002C004B20230320-replicator_pid9943/flysync-replicator/log/fsrepsvc.log
