• 1. 常用命令
    • 1.1. 查看当前VIP在哪个节点上
    • 1.2. 查看keepalived的日志
    • 1.3. 抓包命令
    • 1.4. VIP操作
    • 1.5. keepalived 切 VIP
      • 1.5.1. 停止keepalived服务
      • 1.5.2. 查看日志
  • 2. 指定keepalived的输出日志文件
    • 2.1. 修改 /etc/sysconfig/keepalived
    • 2.2. 修改rsyslog的配置 /etc/rsyslog.conf
    • 2.3. 重启rsyslog和keepalived
  • 3. Troubleshooting
    • 3.1. virtual_router_id 同网段重复

    1. 常用命令

    1.1. 查看当前VIP在哪个节点上

    1. # 查看VIP是否在筛选结果中
    2. ip addr show|grep "scope global"
    4. # 或者 
    5. ip addr show|grep {vip}

    1.2. 查看keepalived的日志

    1. tail /var/log/messages

    1.3. 抓包命令

    1. # 抓包
    2. tcpdump -nn vrrp
    3. # 可以用这条命令来查看该网络中所存在的vrid
    4. tcpdump -nn -i any net 224.0.0.0/8
    1. # tcpdump -nn -i any net 224.0.0.0/8
    2. # tcpdump -nn vrrp
    3. tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
    4. listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
    5. 14:40:00.576387 IP 192.168.98.57 > 224.0.0.18: VRRPv2, Advertisement, vrid 9, prio 99, authtype simple, intvl 1s, length 20
    6. 14:40:01.577605 IP 192.168.98.57 > 224.0.0.18: VRRPv2, Advertisement, vrid 9, prio 99, authtype simple, intvl 1s, length 20
    7. 14:40:02.578429 IP 192.168.98.57 > 224.0.0.18: VRRPv2, Advertisement, vrid 9, prio 99, authtype simple, intvl 1s, length 20
    8. 14:40:03.579605 IP 192.168.98.57 > 224.0.0.18: VRRPv2, Advertisement, vrid 9, prio 99, authtype simple, intvl 1s, length 20
    9. 14:40:04.580443 IP 192.168.98.57 > 224.0.0.18: VRRPv2, Advertisement, vrid 9, prio 99, authtype simple, intvl 1s, length 20

    1.4. VIP操作

    1. # 解绑VIP
    2. ip addr del  dev 
    3. # 绑定VIP
    4. ip addr add  dev

    1.5. keepalived 切 VIP

    例如将 A 机器上的 VIP 迁移到B 机器上。

    1.5.1. 停止keepalived服务

    停止被迁移的机器（A机器）的keepalived服务。

    1. systemctl stop keepalived

    1.5.2. 查看日志

    解绑 A机器 VIP的日志

    1. Sep 19 14:28:09 localhost systemd: Stopping LVS and VRRP High Availability Monitor...
    2. Sep 19 14:28:09 localhost Keepalived[45705]: Stopping
    3. Sep 19 14:28:09 localhost Keepalived_vrrp[45707]: VRRP_Instance(twemproxy) sent 0 priority
    4. Sep 19 14:28:09 localhost Keepalived_vrrp[45707]: VRRP_Instance(twemproxy) removing protocol VIPs.
    5. Sep 19 14:28:09 localhost Keepalived_healthcheckers[45706]: Stopped
    6. Sep 19 14:28:10 localhost Keepalived_vrrp[45707]: Stopped
    7. Sep 19 14:28:10 localhost Keepalived[45705]: Stopped Keepalived v1.3.5 (03/19,2017), git commit v1.3.5-6-g6fa32f2
    8. Sep 19 14:28:10 localhost systemd: Stopped LVS and VRRP High Availability Monitor.
    9. Sep 19 14:28:10 localhost ntpd[1186]: Deleting interface #10 bond0, 192.168.99.9#123, interface stats: received=0, sent=0, dropped=0, active_time=6755768 secs

    绑定 B 机器 VIP的日志

    1. Sep 17 17:20:25 localhost systemd: Starting LVS and VRRP High Availability Monitor...
    2. Sep 17 17:20:26 localhost Keepalived[34566]: Starting Keepalived v1.3.5 (03/19,2017), git commit v1.3.5-6-g6fa32f2
    3. Sep 17 17:20:26 localhost Keepalived[34566]: Opening file '/etc/keepalived/keepalived.conf'.
    4. Sep 17 17:20:26 localhost Keepalived[34568]: Starting Healthcheck child process, pid=34569
    5. Sep 17 17:20:26 localhost Keepalived[34568]: Starting VRRP child process, pid=34570
    6. Sep 17 17:20:26 localhost Keepalived_vrrp[34570]: Registering Kernel netlink reflector
    7. Sep 17 17:20:26 localhost Keepalived_vrrp[34570]: Registering Kernel netlink command channel
    8. Sep 17 17:20:26 localhost Keepalived_vrrp[34570]: Registering gratuitous ARP shared channel
    9. Sep 17 17:20:26 localhost Keepalived_vrrp[34570]: Opening file '/etc/keepalived/keepalived.conf'.
    10. Sep 17 17:20:26 localhost Keepalived_vrrp[34570]: Truncating auth_pass to 8 characters
    11. Sep 17 17:20:26 localhost Keepalived_vrrp[34570]: VRRP_Instance(twemproxy) removing protocol VIPs.
    12. Sep 17 17:20:26 localhost Keepalived_vrrp[34570]: Using LinkWatch kernel netlink reflector...
    13. Sep 17 17:20:26 localhost Keepalived_vrrp[34570]: VRRP_Instance(twemproxy) Entering BACKUP STATE
    14. Sep 17 17:20:26 localhost Keepalived_vrrp[34570]: VRRP sockpool: [ifindex(4), proto(112), unicast(0), fd(10,11)]
    15. Sep 17 17:20:26 localhost systemd: Started LVS and VRRP High Availability Monitor.
    16. Sep 17 17:20:26 localhost kernel: IPVS: Registered protocols (TCP, UDP, SCTP, AH, ESP)
    17. Sep 17 17:20:26 localhost kernel: IPVS: Connection hash table configured (size=4096, memory=64Kbytes)
    18. Sep 17 17:20:26 localhost kernel: IPVS: Creating netns size=2192 id=0
    19. Sep 17 17:20:26 localhost kernel: IPVS: Creating netns size=2192 id=1
    20. Sep 17 17:20:26 localhost kernel: IPVS: ipvs loaded.
    21. Sep 17 17:20:26 localhost Keepalived_healthcheckers[34569]: Opening file '/etc/keepalived/keepalived.conf'.

    2. 指定keepalived的输出日志文件

    2.1. 修改 /etc/sysconfig/keepalived

    将KEEPALIVED_OPTIONS="-D"改为KEEPALIVED_OPTIONS="-D -d -S 0"。

    1. # Options for keepalived. See `keepalived --help' output and keepalived(8) and
    2. # keepalived.conf(5) man pages for a list of all options. Here are the most
    3. # common ones :
    4. #
    5. # --vrrp               -P    Only run with VRRP subsystem.
    6. # --check              -C    Only run with Health-checker subsystem.
    7. # --dont-release-vrrp  -V    Dont remove VRRP VIPs & VROUTEs on daemon stop.
    8. # --dont-release-ipvs  -I    Dont remove IPVS topology on daemon stop.
    9. # --dump-conf          -d    Dump the configuration data.
    10. # --log-detail         -D    Detailed log messages.
    11. # --log-facility       -S    0-7 Set local syslog facility (default=LOG_DAEMON)
    12. #
    14. KEEPALIVED_OPTIONS="-D -d -S 0"

    2.2. 修改rsyslog的配置 /etc/rsyslog.conf

    在/etc/rsyslog.conf 添加 keepalived的日志路径

    1. vi /etc/rsyslog.conf
    2. ...
    3. # keepalived log
    4. local0.*                                                /etc/keepalived/keepalived.log

    2.3. 重启rsyslog和keepalived

    1. # 重启rsyslog
    2. systemctl restart rsyslog
    4. #  重启keepalived
    5. systemctl restart keepalived

    3. Troubleshooting

    3.1. virtual_router_id 同网段重复

    日志报错如下：

    1. Mar 09 21:28:28 k8s4 Keepalived_vrrp[8548]: bogus VRRP packet received on eth0 !!!
    2. Mar 09 21:28:28 k8s4 Keepalived_vrrp[8548]: VRRP_Instance(VI-kube-master) ignoring received advertisment...
    3. Mar 09 21:28:43 k8s4 Keepalived_vrrp[8548]: ip address associated with VRID not present in received packet : 192.168.1.10
    4. Mar 09 21:28:43 k8s4 Keepalived_vrrp[8548]: one or more VIP associated with VRID mismatch actual MASTER advert

    解决方法:

    同一网段内LB节点配置的 virtual_router_id 值有重复了，选择一个不重复的0~255之间的值，可以用以下命令查看已存在的vrid。

    1. tcpdump -nn -i any net 224.0.0.0/8