服务器负载均衡实服务组状态异常

L5030上配置了服务器负载均衡，想要实现将外网用户访问虚服务IP的流量分发给不同的实服务器，减轻服务器的业务负担。L5030旁挂在核心交换机上，现网的多台服务器在拓扑中用一台服务器体现

现场配置完服务器负载均衡后发现实服务组状态不可用，从图中可以看出状态不可用的实服务组中可用实服务器数为0。

健康性检测中有很多探测失败的系统日志，如果实服务器NQA探测失败会导致实服务器不可用。

1、于是根据不可用实服务组test-01进行排查，检查L5030和实服务组test-01中的两个服务器的连通性是否可达。查看实服务组状态显示两个实服务器状态均为探测失败。

[L5030]display server-farm

Server farm: test-01

  Description:

  Predictor: Hash address source IP

  Proximity: Disabled

  NAT: Enabled

  SNAT pool: test02

  Failed action: Keep

  Active threshold: Disabled

  Slow-online: Disabled

  Selected server: Disabled

  Busy action: Drop

  Probe information:

    Probe success criteria: At least 1

    Probe method:

    tcp-temp

  Total real server: 2

  Active real server: 0

  Real server list:

  Name          State        VPN-instance  Address         Port  Weight Priority

  test1         Probe-failed HLW           10.197.44.200   80    100    4      

  test2         Probe-failed HLW           10.197.44.201   80    100    4      

ping测试发现L5030无法ping通10.197.44.200，但是可以ping通10.197.44.201。

[GZ-HXY-G160201-V07-L5000S-L-01]ping -vpn-instance HLW 10.197.44.200

Ping 10.197.44.200 (10.197.44.200): 56 data bytes, press CTRL_C to break

Request time out

Request time out

Request time out

Request time out

Request time out

--- Ping statistics for 10.197.44.200 in VPN instance HLW ---

5 packet(s) transmitted, 0 packet(s) received, 100.0% packet loss

[GZ-HXY-G160201-V07-L5000S-L-01]ping -vpn-instance HLW 10.197.44.201

Ping 10.197.44.201 (10.197.44.201): 56 data bytes, press CTRL_C to break

56 bytes from 10.197.44.201: icmp_seq=0 ttl=63 time=0.993 ms

56 bytes from 10.197.44.201: icmp_seq=1 ttl=63 time=0.865 ms

56 bytes from 10.197.44.201: icmp_seq=2 ttl=63 time=0.814 ms

56 bytes from 10.197.44.201: icmp_seq=3 ttl=63 time=0.828 ms

56 bytes from 10.197.44.201: icmp_seq=4 ttl=63 time=0.864 ms

--- Ping statistics for 10.197.44.201 in VPN instance HLW ---

5 packet(s) transmitted, 5 packet(s) received, 0.0% packet loss

2、由于现场采用的探测方式是TCP探测，所以需要排查是否是服务器端口不可达导致10.197.44.201可以ping通，但是探测失败。首先查看会话，看报文是从哪个接口上出去的，如果看不到报文数量的话需要通过session statistics enable开启会话统计功能，可以看到报文是从冗余口2出去的。

[L5030]display session table ipv4 destination-ip 10.197.44.201 verbose

Slot 1:

Initiator:

  Source      IP/port: 10.197.0.122/51188

  Destination IP/port: 10.197.44.201/80

  DS-Lite tunnel peer: -

  VPN instance/VLAN ID/Inline ID: HLW/-/-

  Protocol: TCP(6)

  Inbound interface: InLoopBack0

Responder:

  Source      IP/port: 10.197.44.201/80

  Destination IP/port: 10.197.0.122/51188

  DS-Lite tunnel peer: -

  VPN instance/VLAN ID/Inline ID: HLW/-/-

  Protocol: TCP(6)

  Inbound interface: Reth2

State: TCP_SYN_SENT

Application: HTTP

Rule ID: -/-/-

Rule name:

Start time: 2023-06-25 13:53:05  TTL: 2s

Initiator->Responder:            0 packets          0 bytes

Responder->Initiator:            0 packets          0 bytes

查看冗余口2的配置，可以看到Route-Aggregation1.1973的优先级高，则报文会从Route-Aggregation1.1973转发。

interface Reth2

 description output-int

 ip binding vpn-instance HLW

 ip address 10.197.0.122 255.255.255.252

 member interface Route-Aggregation1.1973 priority 100

 member interface Route-Aggregation2.1973 priority 50

于是对Route-Aggregation1.1973进行抓包，发现反而和10.197.44.200之间有TCP报文交互，但是没有收到10.197.44.201的回应报文。3、由于抓包的结果比较奇怪，所以进一步对聚合口下的成员接口进行抓包。首先display link-aggregation verbose查看聚合口1中的成员接口，可以看到XGE1/1/0和XGE1/1/1为其成员接口。

Aggregate Interface: Route-Aggregation1

Aggregation Mode: Dynamic

Loadsharing Type: Shar

System ID: 0x8000, 9023-b46c-4400

Local:

  Port             Status  Priority Oper-Key  Flag

--------------------------------------------------------------------------------

  XGE1/1/0         S       32768    3         {ACDEFG}

  XGE1/1/1         U       32768    3         {ACG}

于是对两个接口进行抓包，发现XGE1/1/1接口上也抓取到了和实服务器10.197.44.200交互的报文。

1、排查交换机侧往本端未选中端口发包的原因。发现两端聚合模式不匹配，交换机上是静态聚合，L5030上的是动态聚合，将聚合模式改成一致后正常。

L5030配置

#

interface Route-Aggregation1

description input-int

link-aggregation mode dynamic

#

交换机配置

#

interface Bridge-Aggregation33

description GZ-HXY-G160201-V07-H5030

port link-type trunk

undo port trunk permit vlan 1

port trunk permit vlan 972 to 973 1972 to 1973

#

#

interface Ten-GigabitEthernet1/2/0/43

port link-mode bridge

description GZ-HXY-G160201-V07-H5030-01_PORT1

port link-type trunk

undo port trunk permit vlan 1

port trunk permit vlan 972 to 973 1972 to 1973

port link-aggregation group 33

#

interface Ten-GigabitEthernet1/2/0/44

port link-mode bridge

description GZ-HXY-G160201-V07-H5030-01_PORT2

port link-type trunk

undo port trunk permit vlan 1

port trunk permit vlan 972 to 973 1972 to 1973

port link-aggregation group 33