大朗网站制作,免费的一级域名申请,汕头建站网站模板,给朋友做的相册网站没有了--------------闲 扯------------------ 这里首先感谢酒哥的构建高可用的Linux服务器的这本书#xff0c;看了这本书上并参考里面的配置让自己对DRBDHeartBeatNFS思路清晰了许多。drbd简单来说就是一个网络raid-1,一般有2到多个node节点#xff0c;各个节点创建的磁盘块会… --------------闲 扯------------------ 这里首先感谢酒哥的构建高可用的Linux服务器的这本书看了这本书上并参考里面的配置让自己对DRBDHeartBeatNFS思路清晰了许多。 drbd简单来说就是一个网络raid-1,一般有2到多个node节点各个节点创建的磁盘块会映射到本地drbd块而后通过网络对各个节点drbd磁盘块进行互相同步更新。 heartbeat的作用就可以增加drbd的可用性它能在某节点故障后自动切换drbd块到备份节点并自动进行虚IP从新绑定DRBD块提权磁盘挂载以及启动NFS等脚本操作这一系列操作因为只在他后端节点间完成前端用户访问的是heartbeat的虚IP所以对用户来说无任何感知。 最后吐槽下yum安装真心坑爹以后如果非必须尽量源码包安装。---------------开 搞----------------系统版本 centos6.3 x64内核2.6.32DRBD DRBD-8.4.3HeartBeatepel更新源真坑NFS: 系统自带HeartBeat VIP: 192.168.7.90node1 DRBDHeartBeat: 192.168.7.88drbd1.example.comnode2 DRBDHeartBeat: 192.168.7.89 (drbd2.example.com)(node1)为仅主节点端配置(node2)为仅从节点端配置(node1,node2)为主从节点都需配置一.DRBD配置传送门http://showerlee.blog.51cto.com/2047005/1211963二.Hearbeat配置这里接着DRBD系统环境及安装配置1.安装heartbeatCentOS6.3中默认不带有Heartbeat包因此需要从第三方下载(node1,node2)# wget ftp://mirror.switch.ch/pool/1/mirror/scientificlinux/6rolling/i386/os/Packages/epel-release-6-5.noarch.rpm# rpm -ivUh epel-release-6-5.noarch.rpm# yum --enablerepoepel install heartbeat -y2.配置heartbeat(node1# vi /etc/ha.d/ha.cf---------------# 日志logfile /var/log/ha-loglogfacility local0# 心跳监测时间keepalive 2# 死亡时间deadtime 5# 指定对方IPucast eth0 192.168.7.89# 服务器正常后由主服务器接管资源另一台服务器放弃该资源auto_failback off#定义节点node drbd1.example.com drbd2.example.com---------------node2)# vi /etc/ha.d/ha.cf---------------# 日志logfile /var/log/ha-loglogfacility local0# 心跳监测时间keepalive 2# 死亡时间deadtime 5# 指定对方IPucast eth0 192.168.7.88# 服务器正常后由主服务器接管资源另一台服务器放弃该资源auto_failback off#定义节点node drbd1.example.com drbd2.example.com---------------编辑双机互联验证文件(node1,node2)# vi /etc/ha.d/authkeys--------------auth 11 crc--------------# chmod 600 /etc/ha.d/authkeys编辑集群资源文件(node1,node2)# vi /etc/ha.d/haresources--------------drbd1.example.com IPaddr::192.168.7.90/24/eth0 drbddisk::r0 Filesystem::/dev/drbd0::/data::ext4 killnfsd--------------注该文件内IPaddr,Filesystem等脚本存放路径在/etc/ha.d/resource.d/下,也可在该目录下存放服务启动脚本例如mysql,www,将相同脚本名称添加到/etc/ha.d/haresources内容中从而跟随heartbeat启动而启动该脚本。IPaddr::192.168.7.90/24/eth0用IPaddr脚本配置浮动VIPdrbddisk::r0用drbddisk脚本实现DRBD主从节点资源组的挂载和卸载Filesystem::/dev/drbd0::/data::ext4用Filesystem脚本实现磁盘挂载和卸载编辑脚本文件killnfsd,用来重启NFS服务注因为NFS服务切换后必须重新mount NFS共享出来的目录否则会报错待验证# vi /etc/ha.d/resource.d/killnfsd-----------------killall -9 nfsd; /etc/init.d/nfs restart;exit 0-----------------赋予执行权限# chmod 755 /etc/ha.d/resource.d/killnfsd创建DRBD脚本文件drbddisk:(node1,node2)注此处又是一个大坑如果不明白Heartbeat目录结构的朋友估计要在这里被卡到死因为默认yum安装Heartbeat不会在/etc/ha.d/resource.d/创建drbddisk脚本而且也无法在安装后从本地其他路径找到该文件。此处本人也是因为启动Heartbeat后无法PING通虚IP最后通过查看/var/log/ha-log日志找到一行ERROR: Cannot locate resource script drbddisk然后进而到/etc/ha.d/resource.d/路径下发现竟然没有drbddisk脚本最后在google上找到该代码创建该脚本终于测试通过# vi /etc/ha.d/resource.d/drbddisk-----------------------#!/bin/bash## This script is inteded to be used as resource script by heartbeat## Copright 2003-2008 LINBIT Information Technologies# Philipp Reisner, Lars Ellenberg####DEFAULTFILE/etc/default/drbdDRBDADM/sbin/drbdadmif [ -f $DEFAULTFILE ]; then . $DEFAULTFILEfiif [ $# -eq 2 ]; then RES$1 CMD$2else RESall CMD$1fi## EXIT CODES# since this is a legacy heartbeat R1 resource agent script,# exit codes actually do not matter that much as long as we conform to# http://wiki.linux-ha.org/HeartbeatResourceAgent# but it does not hurt to conform to lsb init-script exit codes,# where we can.# http://refspecs.linux-foundation.org/LSB_3.1.0/#LSB-Core-generic/LSB-Core-generic/iniscrptact.html####drbd_set_role_from_proc_drbd(){local outif ! test -e /proc/drbd; thenROLEUnconfiguredreturnfidev$( $DRBDADM sh-dev $RES )minor${dev#/dev/drbd}if [[ $minor *[!0-9]* ]] ; then# sh-minor is only supported since drbd 8.3.1minor$( $DRBDADM sh-minor $RES )fiif [[ -z $minor ]] || [[ $minor *[!0-9]* ]] ; thenROLEUnknownreturnfiif out$(sed -ne /^ *$minor: cs:/ { s/:/ /g; p; q; } /proc/drbd); thenset -- $outROLE${5%/**}: ${ROLE:Unconfigured} # if it does not show upelseROLEUnknownfi}case $CMD in start)# try several times, in case heartbeat deadtime# was smaller than drbd ping timetry6while true; do$DRBDADM primary $RES breaklet --try || exit 1 # LSB generic errorsleep 1done;; stop)# heartbeat (haresources mode) will retry failed stop# for a number of times in addition to this internal retry.try3while true; do$DRBDADM secondary $RES break# We used to lie here, and pretend success for anything ! 11,# to avoid the reboot on failed stop recovery for simple# config errors and such. But that is incorrect.# Dont lie to your cluster manager.# And dont do config errors...let --try || exit 1 # LSB generic errorsleep 1done;; status)if [ $RES all ]; then echo A resource name is required for status inquiries. exit 10fiST$( $DRBDADM role $RES )ROLE${ST%/**}case $ROLE inPrimary|Secondary|Unconfigured)# expected;;*)# unexpected. whatever...# If we are unsure about the state of a resource, we need to# report it as possibly running, so heartbeat can, after failed# stop, do a recovery by reboot.# drbdsetup may fail for obscure reasons, e.g. if /var/lock/ is# suddenly readonly. So we retry by parsing /proc/drbd.drbd_set_role_from_proc_drbdesaccase $ROLE inPrimary)echo running (Primary)exit 0 # LSB status service is OK;;Secondary|Unconfigured)echo stopped ($ROLE)exit 3 # LSB status service is not running;;*)# NOTE the running in below message.# this is a heartbeat resource script,# the exit code is _ignored_.echo cannot determine status, may be running ($ROLE)exit 4 # LSB status service status is unknown;;esac;; *)echo Usage: drbddisk [resource] {start|stop|status}exit 1;;esacexit 0-----------------------赋予执行权限# chmod 755 /etc/ha.d/resource.d/drbddisk在两个节点上启动HeartBeat服务先启动node1(node1,node2)# service heartbeat start# chkconfig heartbeat on这里能够PING通虚IP 192.168.7.90表示配置成功三.配置NFS:(node1,node2)# vi /etc/exports-----------------/data *(rw,no_root_squash)-----------------重启NFS服务# service rpcbind restart# service nfs restart# chkconfig rpcbind on# chkconfig nfs off这里设置NFS开机不要自动运行因为/etc/ha.d/resource.d/killnfsd 该脚本内容控制NFS的启动。四.最终测试在另外一台LINUX的客户端挂载虚IP192.168.7.90挂载成功表明NFSDRBDHeartBeat大功告成.# mount -t nfs 192.168.7.90:/data /tmp# df -h---------------......192.168.7.90:/data 1020M 34M 934M 4% /tmp---------------测试DRBDHeartBeatNFS可用性1.向挂载的/tmp目录传送文件忽然重新启动主端DRBD服务查看变化经本人测试能够实现断点续传2.正常状态重启Primary主机后观察主DRBD状态是否恢复Primary并能正常被客户端挂载并且之前写入的文件存在可以正常再写入文件。经本人测试可以正常恢复且客户端无需重新挂载NFS共享目录之前数据存在且可直接写入文件。3.当Primary主机因为硬件损坏或其他原因需要关机维修需要将Secondary提升为Primary主机如何手动操作如果设备能够正常启动则按照如下操作无法启动则强行提升Secondary为Primary待宕机设备能够正常启动若“脑裂”再做后续修复工作。首先先卸载客户端挂载的NFS主机目录# umount /tmp(node1)卸载DRBD设备# service nfs stop# umount /data降权# drbdadm secondary r0查看状态已降权# service drbd status-----------------drbd driver loaded OK; device status:version: 8.4.3 (api:1/proto:86-101)GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by rootdrbd1.example.com, 2013-05-27 20:45:19m:res cs ro ds p mounted fstype0:r0 Connected Secondary/Secondary UpToDate/UpToDate C-----------------(node2)提权# drbdadm primary r0查看状态已提权# service drbd status----------------drbd driver loaded OK; device status:version: 8.4.3 (api:1/proto:86-101)GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by rootdrbd2.example.com, 2013-05-27 20:49:06m:res cs ro ds p mounted fstype0:r0 Connected Primary/Secondary UpToDate/UpToDate C----------------这里还未挂载DRBD目录让Heartbeat帮忙挂载注若重启过程中发现Heartbeat日志报错ERROR: glib: ucast: error binding socket. Retrying: Permission denied请检查selinux是否关闭# service heartbeat restart# service drbd status-----------------------drbd driver loaded OK; device status:version: 8.4.3 (api:1/proto:86-101)GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by rootdrbd2.example.com, 2013-05-27 20:49:06m:res cs ro ds p mounted fstype0:r0 Connected Primary/Secondary UpToDate/UpToDate C /data ext4------------------------成功让HeartBeat挂载DRBD目录重新在客户端做NFS挂载测试# mount -t nfs 192.168.7.90:/data /tmp# ll /tmp------------------1 10 2 2222 3 4 5 6 7 8 9 lostfound orbit-root------------------重启刚刚被提权的主机待重启查看状态# service drbd status------------------------drbd driver loaded OK; device status:version: 8.4.3 (api:1/proto:86-101)GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by rootdrbd2.example.com, 2013-05-27 20:49:06m:res cs ro ds p mounted fstype0:r0 WFConnection Primary/Unknown UpToDate/DUnknown C /data ext4------------------------HeartBeat成功挂载DRBD目录drbd无缝连接到备份节点客户端使用NFS挂载点对故障无任何感知。4.测试最后刚才那台宕机重新恢复正常后他是否会从新夺取Primary资源重启后不会重新获取资源需手动切换主从权限方可。注vi /etc/ha.d/ha.cf配置文件内该参数--------------------auto_failback off--------------------表示服务器正常后由新的主服务器接管资源另一台旧服务器放弃该资源5.以上都未利用heartbeat实现故障自动转移当线上DRBD主节点宕机备份节点是否立即无缝接管heartbeatdrbd高可用性是否能够实现首先先在客户端挂载NFS共享目录# mount -t nfs 192.168.7.90:/data /tmpa.模拟将主节点node1 的heartbeat服务停止则备节点node2是否接管服务(node1)# service drbd status----------------------------drbd driver loaded OK; device status:version: 8.4.3 (api:1/proto:86-101)GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by rootdrbd1.example.com, 2013-05-27 20:45:19m:res cs ro ds p mounted fstype0:r0 Connected Primary/Secondary UpToDate/UpToDate C /data ext4----------------------------# service heartbeat stop(node2)# service drbd status----------------------------------------drbd driver loaded OK; device status:version: 8.4.3 (api:1/proto:86-101)GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by rootdrbd2.example.com, 2013-05-27 20:49:06m:res cs ro ds p mounted fstype0:r0 Connected Primary/Secondary UpToDate/UpToDate C /data ext4-----------------------------------------从机无缝接管测试客户端是否能够使用NFS共享目录# cd /tmp# touch test01 # ls test01------------------test01------------------测试通过。。。b.模拟将主节点宕机直接强行关机则备节点node2是否接管服务(node1)强制关机,直接关闭node1虚拟机电源(node2)# service drbd status-------------------------------drbd driver loaded OK; device status:version: 8.4.3 (api:1/proto:86-101)GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by rootdrbd2.example.com, 2013-05-27 20:49:06m:res cs ro ds p mounted fstype0:r0 WFConnection Primary/Unknown UpToDate/DUnknown C /data ext4-------------------------------从机无缝接管测试客户端是否能够使用NFS共享目录# cd /tmp# touch test02 # ls test02------------------test02------------------待node1恢复启动查看drbd状态信息# service drbd status------------------------------drbd driver loaded OK; device status:version: 8.4.3 (api:1/proto:86-101)GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by rootdrbd2.example.com, 2013-05-27 20:49:06m:res cs ro ds p mounted fstype0:r0 Connected Primary/Secondary UpToDate/UpToDate C /data ext4-------------------------------node1已连接上线处于UpToDate状态测试通过。。。注这里node1的heartbeat有几率在关闭服务时node2无法接管,所以有一定维护成本因为本人线上未跑该服务建议实际使用在上线前多做模拟故障演练再实际上线。-------大功告成----------参考酒哥的“构建高可用LINUX服务器”一书本文出自 “一路向北” 博客请务必保留此出处http://showerlee.blog.51cto.com/2047005/1212185 转载于:https://blog.51cto.com/lucifer119/1222533
