lvs keepalived实现高可用负载均衡

发表于 更新于 分类于

今天要做的是通过LVS和keepalived实现的高可用的负载均衡, 通过LVS实现负载均衡能达到F5硬件负载均衡效果的60%,其他的负载均衡则无法达到10%. 而keepalived前面也讲过,是实现高可用的程序,可以在一台服务器崩溃后,另一台可以及时接管并继续提供服务. 保证服务器不会宕机. 好下面就开始配置:

系统环境:CentOS 5.5(定制安装) 组件:

1
2
3
4
5
Base
Development Libraries
Development Tools
Editors
Text-based Internet

服务器及网络参数:

1
2
3
4
5
6
7
lvs-master:192.168.9.201
lvs-backup:192.168.9.202
vip:192.168.9.200
web1:192.168.9.203
web2:192.168.9.204
netmask:255.255.255.0
gateway:192.168.9.1

安装ipvsadm、keepalived:

1
2
3
4
5
6
7
8
9
10
11
[root@master ~]# cd /usr/local/src
[root@master ~]# wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.24.tar.gz
[root@master ~]# wget http://www.keepalived.org/software/keepalived-1.1.17.tar.gz
[root@master ~]# ln -sv /usr/src/kernels/2.6.18-194.el5-i686/ /usr/src/linux
[root@master ~]# tar -zxvf ipvsadm-1.24.tar.gz
[root@master ~]# cd ipvsadm-1.24
[root@master ~]# make;make install
[root@master ~]# cd ..
[root@master ~]# tar -zxvf keepalived-1.1.17.tar.gz
[root@master ~]# cd keepalived-1.1.17
[root@master ~]# ./configure

出现编译错误:

1
2
!!! OpenSSL is not properly installed on your system. !!!
!!! Can not include OpenSSL headers files.

解决办法:

1
2
3
[root@master ~]# yum -y install openssl-devel
[root@master ~]# ./configure
[root@master ~]# make;make install

编译的时候出现这个提示,说明keepalived和内核结合了,如果不是这样的,需要加上这个参数

1
2
3
4
5
6
7
8
9
10
11
./configure --with-kernel-dir=/kernel/path
Keepalived configuration
Keepalived version : 1.1.17
Compiler : gcc
Compiler flags : -g -O2
Extra Lib : -lpopt -lssl -lcrypto
Use IPVS Framework : Yes
IPVS sync daemon support : Yes
Use VRRP Framework : Yes
Use LinkWatch : No
Use Debug flags : No

配置keepalived:

1
2
3
4
5
[root@master ~]# cp /usr/local/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/
[root@master ~]# cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/
[root@master ~]# mkdir /etc/keepalived
[root@master ~]# cp /usr/local/sbin/keepalived /usr/sbin/
[root@master ~]# vi /etc/keepalived/keepalived.conf

keepalived配置文件内容:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
global_defs {
    notification_email {
        king_819@163.com
    }
    notification_email_from king_819@163.com
    smtp_server smtp.163.com
    # smtp_connect_timeout 30
    router_id LVS_DEVEL
}
# VIP1
vrrp_instance VI_1 {
    state MASTER #备份服务器上将MASTER改为BACKUP
    interface eth0
    lvs_sync_daemon_inteface eth0
    virtual_router_id 51
    priority 100 # 备份服务上将100改为90
    advert_int 5
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.9.200
        #(如果有多个VIP,继续换行填写.)
    }
}
virtual_server 192.168.9.200 80 {
    delay_loop 6 #(每隔10秒查询realserver状态)
    lb_algo wlc #(lvs 算法)
    lb_kind DR #(Direct Route)
    persistence_timeout 60 #(同一IP的连接60秒内被分配到同一台realserver)
    protocol TCP #(用TCP协议检查realserver状态)
    real_server 192.168.9.203 80 {
        weight 100 #(权重)
        TCP_CHECK {
            connect_timeout 10 #(10秒无响应超时)
            nb_get_retry 3
            delay_before_retry 3
            connect_port 80
        }
    }
    real_server 192.168.9.204 80 {
        weight 100
        TCP_CHECK {
            connect_timeout 10
            nb_get_retry 3
            delay_before_retry 3
            connect_port 80
        }
    }
}
[root@master ~]# service keepalived start|stop
[root@master ~]# chkconfig –level 2345 keepalived on

配置realserver:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
[root@web_1 ~]# vi /root/lvs_real.sh
#!/bin/bash
# description: Config realserver
#Written by : http://kerry.blog.51cto.com
SNS_VIP=192.168.9.200
/etc/rc.d/init.d/functions
case "$1" in
start)
/sbin/ifconfig lo:0 $SNS_VIP netmask 255.255.255.255 broadcast $SNS_VIP
/sbin/route add -host $SNS_VIP dev lo:0
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
sysctl -p >/dev/null 2>&1
echo "RealServer Start OK"
;;
stop)
/sbin/ifconfig lo:0 down
/sbin/route del $SNS_VIP >/dev/null 2>&1
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
echo "RealServer Stoped"
;;
*)
echo "Usage: $0 {start|stop}"
exit 1
esac
exit 0
[root@web_1 ~]# chmod +x /roo/lvs_real.sh
[root@web_1 ~]# /root/lvs_real.sh start
[root@web_1 ~]# ifconfig
lo:0 Link encap:Local Loopback
inet addr:192.168.0.10 Mask:255.255.255.255
UP LOOPBACK RUNNING MTU:16436 Metric:1
[root@web_1 ~]# echo "/root/lvs_real.sh start" >> /etc/rc.local

测试LVS+keepalived:

LVS_master、LVS_backup上开启keepalived,LVS_master先绑定VIP,在LVS_master执行:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
ip a
1: lo: mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 08:00:27:8e:14:62 brd ff:ff:ff:ff:ff:ff
inet 192.168.0.254/24 brd 192.168.0.255 scope global eth0
inet 192.168.0.10/32 scope global eth0
inet6 fe80::a00:27ff:fe8e:1462/64 scope link
valid_lft forever preferred_lft forever
3: sit0: mtu 1480 qdisc noop
link/sit 0.0.0.0 brd 0.0.0.0

我们可以看到已经成功绑定了虚拟IP了. 在LVS_backup下则没有哪条加粗的虚拟ip地址. 在LVS_master执行ipvsadm,可以看到后端web服务器的负载情况.

1
2
3
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
TCP 192.168.0.10:http wlc persistent 60

现在我们就可以来测试一下了,首先我们访问虚拟IP,如果可以访问切多次刷新都没有问题的话,可以关闭一台LVS_master服务器,LVS_backup会很快接管工作. 我们如果停掉一台web服务器,则LVS服务器会将问题服务器剔除列表,执行ipvsadm会看到只剩一台web服务器. 但是虚拟ip的访问不受任何影响.这样一个高可用的负载均衡集群就搭建完成了.