一、双主模式下Keepalived+LVS(DR)+nginx集群实验
拓扑图：
拓扑图配置
1、先做RS的VIP绑定：
RS1:
vim rs.sh
#!/bin/bash
#
vip=172.20.222.222
mask='255.255.255.255'
dev=lo:0
case $1 in
start)
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
ifconfig $dev $vip netmask $mask broadcast $vip up
#route add -host $vip dev $dev
;;
stop)
ifconfig $dev down
echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce
;;
*)
echo "Usage: $(basename $0) start|stop"
exit 1
;;
esac
安装nginx，并修改默认信息页面内容
RS2：
和RS1操作一致
完成上述步骤后可RS1/2可看到lo网卡上已经有了vip的地址
2、用ipvsadm脚本在keep1/keep2上测试LVS(DR)功能：
keep1:
vim vs.sh
#!/bin/bash
#
vip=172.20.222.222
iface='lo:0'
mask='255.255.255.255'
port='80'
rs1='172.20.222.4'
rs2='172.20.222.5'
scheduler='wrr'
type='-g'
case $1 in
start)
ifconfig $iface $vip netmask $mask broadcast $vip up
iptables -F
ipvsadm -A -t ${vip}:${port} -s $scheduler
ipvsadm -a -t ${vip}:${port} -r ${rs1} $type -w 1
ipvsadm -a -t ${vip}:${port} -r ${rs2} $type -w 1
;;
stop)
ipvsadm -C
ifconfig $iface down
;;
*)
echo "Usage $(basename $0) start|stop"
exit 1
esac
keep2:
同keep1一样执行bash vs.sh脚本
查看LVS的规则生成
[root@centos7b ~]#ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.20.222.222:80 wrr
-> 172.20.222.4:80 Route 1 0 17
-> 172.20.222.5:80 Route 1 0 16
client:
while true;do curl http://172.20.222.222 && sleep 1;done 可看到轮询效果
以上仅仅是对前面LVS-DR模型的实验的再次回顾，ipvsadm -C清除keep1/2LVS规则，并用Keepalived自动生成LVS规则
2、安装keepalived
3、修改主配置文件
keep1:
vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id centos7b.duanx.vip-----解析到本机主机名
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_iptables------
vrrp_garp_interval 0
vrrp_gna_interval 0
vrrp_mcast_group4 224.0.222.1
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 58
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 11111111
}
virtual_ipaddress {
172.20.222.222/16 brd 172.20.255.255 dev eth0
}
}
vrrp_instance VI_2 {
state BACKUP------一个虚拟路由中只能有一个MASTER和多个BACKUP，这里不能再为MASTER了
interface eth0
virtual_router_id 56
priority 98
advert_int 1
authentication {
auth_type PASS
auth_pass 11111111
}
virtual_ipaddress {
172.20.222.222/16 brd 172.20.255.255 dev eth0
}
}
virtual_server 172.20.222.222 80 {
delay_loop 2
lb_algo wrr
lb_kind DR
protocol TCP
sorry_server 127.0.0.1 80
#keep1宕机后显示的页面
real_server 172.20.222.4 80 {
weight 1
#健康状态检测：
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 1
nb_get_retry 3
delay_before_retry 1
}
}
real_server 172.20.222.5 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 1
nb_get_retry 3
delay_before_retry 1
}
}
}
keep2:-----------这里其实只有优先级和keep1不一样罢了
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id centos7b.duanx.vip
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_iptables
vrrp_garp_interval 0
vrrp_gna_interval 0
vrrp_mcast_group4 224.0.222.1
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 58
priority 98
advert_int 1
authentication {
auth_type PASS
auth_pass 11111111
}
virtual_ipaddress {
172.20.222.222/16 brd 172.20.255.255 dev eth0
}
}
vrrp_instance VI_2 {
state MASTER
interface eth0
virtual_router_id 56
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 11111111
}
virtual_ipaddress {
172.20.222.222/16 brd 172.20.255.255 dev eth0
}
}
virtual_server 172.20.222.222 80 {
delay_loop 2
lb_algo wrr
lb_kind DR
protocol TCP
sorry_server 127.0.0.1 80
#keep1宕机后显示的信息页面
real_server 172.20.222.4 80 {
weight 1
#健康状态检测：
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 1
nb_get_retry 3
delay_before_retry 1
}
}
real_server 172.20.222.5 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 1
nb_get_retry 3
delay_before_retry 1
}
}
}
4、keep1和keep2安装nginx并修改页面信息------做sorry server页面
5、client:
检查LVS规则和双主：同样用curl可检查到轮询效果；也可用tcpdump -i eth0 -nn host 224.0.222.1查看
03:21:05.860822 IP 172.20.222.3 > 224.0.222.1: VRRPv2, Advertisement, vrid 56, prio 100, authtype simple, intvl 1s, length 20
03:21:05.860848 IP 172.20.222.2 > 224.0.222.1: VRRPv2, Advertisement, vrid 58, prio 100, authtype simple, intvl 1s, length 20
健康状态检查：
分别停止RS1/RS2nginx服务和同时停止会看到client不同的反馈结果来验证keepalived的高可用性
反思：
尝试用到了两个vip,man keepalived.conf才发现只能给RS增加vip；和keepalived配置文件中再定义一次virtual_server{}
二、双主模式下Keepalived+Nginx高可用
拓扑图：
拓扑图配置
1、先做nginx反代：
docker1 ngx1:172.20.222.4
docker pull nginx:1.14-alpine
docker run --name ngx1 -d --network host nginx:1.14-alpine ---可以看到port:80暴露在外
docker exec -it ngx1 /bin/sh
vi /usr/share/nginx/html/index.html ngx1
docker ngx2:172.20.222.5
docker pull nginx:1.14-alpine
docker run --name ngx2 -d --network host nginx:1.14-alpine
docker exec -it ngx2 /bin/sh
vi /usr/share/nginx/html/index.html ngx2
注：为了不让port发生冲突，故两容器分开跑
keep1:172.20.222.2
yum install -y nginx
vim /etc/nginx/nginx.conf
http {
upstream websrvs {
server 172.20.222.4 weight1;
server 172.20.222.5 weight2;
}
server{
proce_pass http://websrvs'
}
}
systemctl start nginx
scp /etc/nginx/nginx.conf 172.20.222.3:/etc/nginx/nginx.conf
keep2:172.20.222.3
systemctl start nginx
client:
while true;do curl http://172.20.222.2 && sleep .5;done
while true;do curl http://172.20.222.2 && sleep .5;done
经过测试，client均能测试出两容器的nginx页面文件内容
2、Keepalived做高可用
keep1:
yum install -y keepalived
vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id centos7b.duanx.vip
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_iptables
vrrp_garp_interval 0
vrrp_gna_interval 0
vrrp_mcast_group4 224.0.222.2
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 58
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 11112222
}
virtual_ipaddress {
172.20.222.223/16 brd 172.20.255.255 dev eth0#-------外网地址
}
}
vrrp_instance VI_2 {
state BACKUP
interface eth0
virtual_router_id 56
priority 98
advert_int 1
authentication {
auth_type PASS
auth_pass 11111111
}
virtual_ipaddress {
172.20.222.224/16 brd 172.20.255.255 dev eth0
}
}
scp /etc/keepalived/keepalived.conf 172.20.222.3:/etc/keepalived/
keep2:
只需将权重和优先级更改了，其他不用再改，如下：
VI_1:state BACKUP;priority 98
VI_2:state MASTER;priority 100
反思：
1、尝试了在keepalived的两个主机上跑docker容器来运行nginx但是nginx.conf文件和yum源安装的内容不一致，配置起来较麻烦，但是可以行得通的；
2、keep1和keep2配置不同的组播域时出现本组播域宕机，交给它组播域来工作；
3、这里没有定义通知脚本以及资源监控，日后用到再回头翻看。
转载请注明：黑夜 » Keepalived系列(二)相关实验
