LVS/TUN 实例部署

一.部署前说明:node

CIP:10.10.1.1/24
VIP:10.10.1.222/24nginx

DIR: Eth1:10.10.1.100/24 Eth0:192.168.1.2/24算法

RS server:(这里假设后端只有两台RS server)shell

RIP1( Eth0:192.168.1.12/24  &&  Eth1:10.10.1.10/24(提供nginx服务)vim

RIP2( Eth0:192.168.1.13/24  &&  Eth1:10.10.1.11/24(提供nginx服务)后端

 

二.部署操做:bash

 负载均衡器上配置操做服务器

(1)在DIR上安装ipvsadm软件包以及相关依赖包:网络

# yum install openssl-devel popt-devel libnl-devel ipvsadm  -y负载均衡

 [root@node~]# ipvsadm –help

                -A  添加虚拟服务器

                -t  设置群集地址(VIP,Virtual IP)

                -s  指定负载调度算法

                -a  添加真实服务器

                -d  删除真实服务器

                -r  指定真实服务器(Real Server)的地址

                -m  使用NAT模式;-g、-i分别对应DR、TUN模式

                -w  为节点服务器设置权重,默认为1

 

(2)在DIR上新建一个shell脚本文件,以下操做所示:

vim /etc/init.d/lvs-tun

#!/bin/sh
# Startup script handle the initialisation of LVS
# chkconfig: - 28 72
# description: Initialise the Linux Virtual Server for TUN
#
### BEGIN INIT INFO
# Provides: ipvsadm
# Required-Start: $local_fs $network $named
# Required-Stop: $local_fs $remote_fs $network
# Short-Description: Initialise the Linux Virtual Server
# Description: The Linux Virtual Server is a highly scalable and highly
#   available server built on a cluster of real servers, with the load
#   balancer running on Linux.
# description: start LVS of TUN
LOCK=/var/lock/lvs-tun.lock
VIP=10.10.1.222
RIP1=192.168.1.12
RIP2=192.168.1.13
ETH=ens33
. /etc/rc.d/init.d/functions

start()    {
     PID=`ipvsadm -Ln | grep ${VIP} | wc -l`
     if    [ $PID -gt 0 ];

     then
           echo "The LVS-TUN Server is already running !"
     else
           #Load the tun mod
           /sbin/modprobe tun
           /sbin/modprobe ipip
           #Set the tun Virtual IP Address
           /sbin/ifconfig tunl0 $VIP broadcast $VIP netmask 255.255.255.255 up
           /sbin/route add -host $VIP dev tunl0
           #Clear IPVS Table
           /sbin/ipvsadm -C
           #The icmp recruit setting
           echo "0" >/proc/sys/net/ipv4/ip_forward
           echo "0" >/proc/sys/net/ipv4/conf/all/send_redirects
           echo "0" >/proc/sys/net/ipv4/conf/default/send_redirects
           echo "0" >/proc/sys/net/ipv4/conf/$ETH/send_redirects
           echo "0" >/proc/sys/net/ipv4/conf/$ETH/send_redirects
           #Set Lvs
           /sbin/ipvsadm -At $VIP:80 -s rr
           /sbin/ipvsadm -at $VIP:80 -r $RIP1:80 -i  -w 1
           /sbin/ipvsadm -at $VIP:80 -r $RIP2:80 -i  -w 1
           /bin/touch $LOCK
           #Run Lvs
           echo "starting LVS-TUN-DIR Server is ok !"       
     fi
}

stop()    {
           #stop  Lvs server
           /sbin/ipvsadm -C
           /sbin/ifconfig tunl0 down >/dev/null
           #Remove the tun mod
           /sbin/modprobe -r tun
           /sbin/modprobe -r ipip
           rm -rf $LOCK
           echo "stopping LVS-TUN-DIR server is ok !"
}

status()  {
     if [ -e $LOCK ];
     then
         echo "The LVS-TUN Server is already running !"
     else
         echo "The LVS-TUN Server is not running !"
     fi
}

case "$1" in
  start)
        start
        ;;
  stop)
        stop
        ;;
  restart)
        stop
        sleep 1
        start
        ;;
  status)
        status
        ;;
  *)
        echo "Usage: $1 {start|stop|restart|status}"
        exit 1
esac
exit 0

受权并启动该脚本

# chmod 777 /etc/init.d/lvs-tun 

# /etc/init.d/lvs-tun  start 

注意: VIP也能够配置为serve物理网卡已配置的ip,好比上述Eth1的网卡ip,不过广播必须是本身!

RS server 上配置操做

 (1)分别在每一个RIP(RIP1,RIP2)上新建一个shell脚本文件,以下操做所示:

vim /etc/init.d/lvs-tun-dr

#!/bin/sh
#
# Startup script handle the initialisation of LVS
# chkconfig: - 28 72
# description: Initialise the Linux Virtual Server for TUN
#
### BEGIN INIT INFO
# Provides: ipvsadm
# Required-Start: $local_fs $network $named
# Required-Stop: $local_fs $remote_fs $network
# Short-Description: Initialise the Linux Virtual Server
# Description: The Linux Virtual Server is a highly scalable and highly
#   available server built on a cluster of real servers, with the load
#   balancer running on Linux.
# description: start LVS of TUN-RIP
LOCK=/var/lock/ipvsadm.lock
VIP=10.10.1.222
ETH=eno16777736
. /etc/rc.d/init.d/functions
start() {
     PID=`ifconfig | grep tunl0 | wc -l`
     if [ $PID -ne 0 ];
     then
         echo "The LVS-TUN-RIP Server is already running !"
     else
         #Load the tun mod
         /sbin/modprobe tun
         /sbin/modprobe ipip
         #Set the tun Virtual IP Address
         /sbin/ifconfig tunl0 $VIP netmask 255.255.255.255 broadcast $VIP up
         /sbin/route add -host $VIP dev tunl0
         echo "1" >/proc/sys/net/ipv4/conf/tunl0/arp_ignore
         echo "2" >/proc/sys/net/ipv4/conf/tunl0/arp_announce
         echo "1" >/proc/sys/net/ipv4/conf/$ETH/arp_ignore
         echo "2" >/proc/sys/net/ipv4/conf/$ETH/arp_announce
         echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
         echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
         echo "0" > /proc/sys/net/ipv4/conf/tunl0/rp_filter
         echo "0" > /proc/sys/net/ipv4/conf/all/rp_filter
         /bin/touch $LOCK
         echo "starting LVS-TUN-RIP server is ok !"
     fi
}

stop() {
         /sbin/ifconfig tunl0 down
         echo "0" >/proc/sys/net/ipv4/conf/tunl0/arp_ignore
         echo "0" >/proc/sys/net/ipv4/conf/tunl0/arp_announce
         echo "0" >/proc/sys/net/ipv4/conf/$ETH/arp_ignore
         echo "0" >/proc/sys/net/ipv4/conf/$ETH/arp_announce
         echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
         echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
         #Remove the tun mod
         /sbin/modprobe -r tun
         /sbin/modprobe -r ipip
         rm -rf $LOCK
         echo "stopping LVS-TUN-RIP server is ok !"
}

status() {
     if [ -e $LOCK ];
     then
        echo "The LVS-TUN-RIP Server is already running !"
     else
        echo "The LVS-TUN-RIP Server is not running !"
     fi
}

case "$1" in
  start)
        start
        ;;
  stop)
        stop
        ;;
  restart)
        stop
        start
        ;;
  status)
        status
        ;;
  *)
        echo "Usage: $1 {start|stop|restart|status}"
        exit 1
esac
exit 0

注:在LVS/TUN模式中,关于arp原则:

(1)若DIR和RIP在不一样lan网络中,好比不一样的网段,不一样的IDC机房,就不须要设置arp仰制,不一样网段中,arp会被屏蔽掉,因此只需设置 ip tunnel便可;

(2)若DIR和RIP在同一广播域中,须要和LVS/DR模式同样在全部的RIP上仰制arp,防止arp响应致使arp表混乱,这样lvs就不能正常工做!

受权并启动该脚本

# chmod 777 /etc/init.d/lvs-tun-dr

#/etc/init.d/lvs-tun-dr  start 

 ipvsadm -Ln  -c  

[root@node1 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.10.1.222:80 rr
  -> 192.168.1.12:80              Tunnel  1      0          0         
  -> 192.168.1.13:80              Tunnel  1      0          0

 

总 结 : LVS/TUN是全部模式中最最适用于跨网络跨地域地理位置的一种模式,须要注意的是:

(1)若DIR和RIP在不一样lan网络中,好比不一样的网段,不一样的IDC机房,就不须要设置arp仰制,不一样网段中,arp会被屏蔽掉,因此只需设置 ip tunne以及报文反向验证便可;

(2)若DIR和RIP在同一广播域中,须要和LVS/DR模式同样在全部的RIP上仰制arp,防止arp响应致使arp表混乱,这样lvs就不能正常工做!

 配置时除了配置DIR,还须要须要配置后端RS server,即在tunl上口配置vip地址(须要系统支持tunl才行),广播为为本身,此模式下无需开启路由转发功能!