• kube-proxy via ipvs
    • ipvs示例
      • NAT mode
      • DR mode

    kube-proxy via ipvs

    Kubernetes v1.8已经支持ipvs负载均衡模式（alpha版），只需要配置kube-proxy --proxy-mode=ipvs即可启用。

    kube-proxy via ipvs - 图1

    ipvs示例

    NAT mode

    1. # prepare local kubernetes cluster
    2. $ sudo ./hack/local-up-cluster.sh
    3. $ sudo kill -9 $KUBE_PROXY_PID
    5. # run two nginx pods
    6. $ kubectl run --image nginx --replicas=2 nginx
    8. # expose deployment
    9. $ kubectl expose deployment nginx --port=80 --target-port=80
    11. $ kubectl get services
    12. NAME         CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
    13. kubernetes   10.0.0.1     <none>        443/TCP   3m
    14. nginx        10.0.0.185   <none>        80/TCP    4s
    16. $ kubectl get pods -o wide
    17. NAME                    READY     STATUS    RESTARTS   AGE       IP           NODE
    18. nginx-348975970-7x18g   1/1       Running   0          49s       172.17.0.3   127.0.0.1
    19. nginx-348975970-rtqrz   1/1       Running   0          49s       172.17.0.4   127.0.0.1
    21. # Add dummy link
    22. $ sudo ip link add type dummy
    23. $ sudo ip addr add 10.0.0.185 dev dummy0
    25. # Add ipvs rules; real server should use nat mode, since host is essentially
    26. # the gateway.
    27. $ sudo ipvsadm -A -t 10.0.0.185:80
    28. $ sudo ipvsadm -a -t 10.0.0.185:80 -r 172.17.0.3:80 -m
    29. $ sudo ipvsadm -a -t 10.0.0.185:80 -r 172.17.0.4:80 -m
    30. $ sudo ipvsadm -Ln
    31. IP Virtual Server version 1.2.1 (size=4096)
    32. Prot LocalAddress:Port Scheduler Flags
    33.   -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
    34. TCP  10.0.0.185:80 wlc
    35.   -> 172.17.0.3:80                Masq    1      0          1
    36.   -> 172.17.0.4:80                Masq    1      0          1
    38. # Works in container
    39. $ docker run -ti busybox wget -qO- 10.0.0.185:80
    40. <!DOCTYPE html>
    41. // truncated
    43. # Works in host
    44. $ curl 10.0.0.185:80
    45. <!DOCTYPE html>
    46. // truncated

    DR mode

    1. # continue above setup;
    2. $ PID=$(docker inspect -f '{{.State.Pid}}' k8s_nginx_nginx-348975970-rtqrz_default_b1661284-2eeb-11e7-924d-8825937fa049_0)
    3. $ sudo mkdir -p /var/run/netns
    4. $ sudo ln -s /proc/$PID/ns/net /var/run/netns/$PID
    5. $ sudo ip link add type dummy
    6. $ sudo ip link set dummy1 netns $PID
    7. $ sudo ip netns exec $PID ip addr add 10.0.0.185 dev dummy1
    8. $ sudo ip netns exec $PID ip link set dummy1 up
    9. # same for the other pod
    10. $ sudo ipvsadm -D -t 10.0.0.185:80
    11. $ sudo ipvsadm -A -t 10.0.0.185:80
    12. $ sudo ipvsadm -a -t 10.0.0.185:80 -r 172.17.0.3:80 -g
    13. $ sudo ipvsadm -a -t 10.0.0.185:80 -r 172.17.0.4:80 -g    
    14. $ docker run -ti busybox wget -qO- 10.0.0.185:80
    15. <!DOCTYPE html>
    16. // truncated
    18. // ignored seting arp_ignore/arp_announce