/ 中存储网

LVS+keepalived+Nginx实现负载均衡时前后端故障全自动化处理

2014-04-13 13:43:01 来源:ITJS.CN
最近帮朋友公司配了一下LVS,实现全自动化处理故障问题,基本环境如下

VIP : 192.168.101.100

LVS1: 192.168.101.103

LVS2: 192.168.101.104

RS1: 192.168.101.105

RS2: 192.168.101.108

后端还有一些服务器IP就不一一写出来了

首先各个服务器上配置好各种需要的环境,所需要依赖包,然后在master 和 backup上装好keepalived 和 LVS,直接yum安装 .不启动LVS,启动keepalived就好,下面直接说配置 全部在keepalived上

[[email protected] shell]# cat /etc/keepalived/keepalived.conf

! Configuration File for keepalived

global_defs {

   router_id LVS_DEVEL

}

vrrp_sync_group VGM {     ###定义一个vrrp组

group {

VI_1

}

notify_master "/data/shell/arp.sh"

notify_backup "/data/shell/arp.sh"

}

vrrp_script chk_nginx {    #### 定义脚本

script "/data/shell/check_nginx.sh"

interval 1

weight -2

}

vrrp_script chk_nginx2 {      #### 定义脚本

script "/data/shell/check_nginx2.sh"

interval 1

weight -2

}

vrrp_instance VI_1 {

state MASTER    ###backup上配置相同,只需要改下相关权重等

interface eth1

virtual_router_id 51

priority 101

advert_int 1

authentication {

auth_type PASS

auth_pass 1111

}

track_script {    #### 执行脚本检测

chk_nginx

chk_nginx2

}

virtual_ipaddress {

192.168.101.100

 }

}

virtual_server 192.168.101.100 80 {

delay_loop 2

lb_algo wrr

lb_kind DR

persistence_timeout 60

protocol TCP

real_server 192.168.101.105 80 {

weight 3

HTTP_GET {

url {

path /test.html

digest 9a8acfd3c07a45099bcde1a97a9fb7c5

}

connect_timeout 5

nb_get_retry 3

delay_before_retry 3

connect_port 80

}

}

 real_server 192.168.101.108 80 {

weight 3

HTTP_GET {

url {

path /test2.html

digest 9649358e69985102d0da5ff240e1f2a4

}

connect_timeout 5

nb_get_retry 3

delay_before_retry 3

connect_port 80

}

}

}

[[email protected] shell]# cat /etc/keepalived/keepalived.conf

! Configuration File for keepalived

global_defs {

   router_id LVS_DEVEL

}

vrrp_sync_group VGM {     ###定义一个vrrp组

group {

VI_1

}

notify_master "/data/shell/arp.sh"

notify_backup "/data/shell/arp.sh"

}

vrrp_script chk_nginx {    #### 定义脚本

script "/data/shell/check_nginx.sh"

interval 1

weight -2

}

vrrp_script chk_nginx2 {      #### 定义脚本

script "/data/shell/check_nginx2.sh"

interval 1

weight -2

}

vrrp_instance VI_1 {

state MASTER    ###backup上配置相同,只需要改下相关权重等

interface eth1

virtual_router_id 51

priority 101

advert_int 1

authentication {

auth_type PASS

auth_pass 1111

}

track_script {    #### 执行脚本检测

chk_nginx

chk_nginx2

}

virtual_ipaddress {

192.168.101.100  

}

}

virtual_server 192.168.101.100 80 {

delay_loop 2

lb_algo wrr

lb_kind DR

persistence_timeout 60

protocol TCP

real_server 192.168.101.105 80 {

weight 3

HTTP_GET {

url {

path /test.html

digest 9a8acfd3c07a45099bcde1a97a9fb7c5

}

connect_timeout 5

nb_get_retry 3

delay_before_retry 3

connect_port 80

}

}

real_server 192.168.101.108 80 {

weight 3

HTTP_GET {

url {

path /test2.html

digest 9649358e69985102d0da5ff240e1f2a4

}

connect_timeout 5

nb_get_retry 3

delay_before_retry 3

connect_port 80

}

}    

}

如果你前端开启了防火墙或者selinux,请关闭防火墙测试或者selinux测试,否则你会出现两个master,哈哈,两个vip地址同时出现,或者起不来服务,如果不关闭iptables,请加入以下策略

###允许80端口对外提供服务

-A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT

###DR模式,当用户请求LVS-DR VIP时,只有DR响应客户端的ARP广播包,允许vrrp虚拟路由器冗余协议-A   INPUT   -d   224.0.0.0/8   -j   ACCEPT

-A   INPUT    -p   vrrp   -j   ACCEPT

###允许80端口对外提供服务  

-A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT  

###DR模式,当用户请求LVS-DR VIP时,只有DR响应客户端的ARP广播包,允许vrrp虚拟路由器冗余协议-A   INPUT   -d   224.0.0.0/8   -j   ACCEPT  

-A   INPUT    -p   vrrp   -j   ACCEPT

下面是几个在master 和backup上执行的高效脚本,检测后端nginx的

[[email protected] shell]# cat arp.sh

#!/bin/bash

VIP=192.168.101.100

GATEWAY=192.168.101.1

/sbin/arping -I eth1 -c 5 -s $VIP $GATEWAY &>/dev/null

########################################################

[[email protected] shell]# cat check_nginx.sh

!/bin/bash

url="http://192.168.101.105"      ####realserver web地址检测

status=$(/usr/bin/curl -s --head "$url" | awk '/HTTP/ {print $2}')

if [ "$status" != "200" ]; then

/data/shell/arp.sh start

fi

[[email protected] shell]# cat check_nginx2.sh

!/bin/bash

url="http://192.168.101.108"      ####realserver web地址检测

status=$(/usr/bin/curl -s --head "$url" | awk '/HTTP/ {print $2}')

if [ "$status" != "200" ]; then

/data/shell/arp.sh start

fi

[[email protected] shell]# cat arp.sh

#!/bin/bash

VIP=192.168.101.100

GATEWAY=192.168.101.1

/sbin/arping -I eth1 -c 5 -s $VIP $GATEWAY &>/dev/null

########################################################

[[email protected] shell]# cat check_nginx.sh

!/bin/bash

url="http://192.168.101.105"      ####realserver web地址检测

status=$(/usr/bin/curl -s --head "$url" | awk '/HTTP/ {print $2}')

if [ "$status" != "200" ]; then

/data/shell/arp.sh start

fi

[[email protected] shell]# cat check_nginx2.sh

!/bin/bash

url="http://192.168.101.108"      ####realserver web地址检测

status=$(/usr/bin/curl -s --head "$url" | awk '/HTTP/ {print $2}')

if [ "$status" != "200" ]; then

/data/shell/arp.sh start

fi

 arp缓存清理shell  {arp.sh}

#!/bin/bash

VIP=192.168.101.100

GATEWAY=192.168.101.1

/sbin/arping -I eth1 -c 5 -s 192.168.101.100 192.168.101.1 &>/dev/null

#!/bin/bash

VIP=192.168.101.100

GATEWAY=192.168.101.1

/sbin/arping -I eth1 -c 5 -s 192.168.101.100 192.168.101.1 &>/dev/null

 这个配置会实现 无论前端调度或者后端nginx服务任意一台中断都会在最短的时间响应到正常的请求,经测试,lvs前端两台故障几乎是0秒切换,后端nginx的web服务故障也就是几秒钟的自动切换,如果是DNS解析,请做好解析定时缓存清理