V7版本LB服务器负载SNAT导致业务异常问题分析

1、LB服务器负载组网拓扑

2、实现需求

按照上述拓扑实现LB旁路部署时的服务器负载，因为LB旁路部署，服务器原有网关均为核心交换机，在不改变服务器网关配置的前提下，为了保证来回的流量全部经过LB需要在LB上配置SNAT

3、故障现象

配置SNAT之后，通过LB上的虚服务无法访问后端服务器提供的web服务；如果不配置SNAT，同时在交换机上配置PBR保证来回流量都经过LB，此时可以通过LB的虚服务正常访问web业务

1、查看实服务和虚服务状态均为active

2、查看其中一个实服务组SNAT状态

[Sysname] display server-farm

Server farm: sf1

Description:

Predictor: Hash address source IP

NAT: Enabled

SNAT pool:snat

Failed action: Keep

Active threshold: Disabled

Slow-online: Disabled

Selected server: Disabled

Total real server: 1

Active real server: 1

Real server list:

Name State Address Port Weight Priority

rs1 Active 192.168.1.1 0 100 4

其中SNAT地址池配置为：
    loadbalance snat-pool snat
      ip range start 178.1.1.3 end 178.1.1.3

3、查看实服务和虚服务统计信息

[Sysname] display virtual-server statistics name vs

Slot 1:

Virtual server: vs

Total connections: 3

Active connections: 0

Max connections: 3

Connections per second: 0

Max connections per second: 1

Client input: 1717bytes

Client output: 2750bytes

Throughput: 0 bytes/s

Inbound throughput: 0 bytes/s

Outbound throughput: 0 bytes/s

Max throughput: 1423 bytes/s

Max inbound throughput: 673 bytes/s

Max outbound throughput: 750 bytes/s

Received packets:37

Sent packets:45

Dropped packets: 0

[Sysname] display real-server statistics name rs1

Slot 1:

Real server: rs1

Total connections:1

Active connections:0

Max connections: 1

Connections per second: 0

Max connections per second: 1

Server input: 0 bytes

Server output: 915 bytes

Throughput: 0 bytes/s

Inbound throughput: 0 bytes/s

Outbound throughput: 0 bytes/s

Max throughput: 915 bytes/s

Max inbound throughput: 0 bytes/s

Max outbound throughput: 915 bytes/s

Received packets:12

Sent packets:0

Dropped packets: 0

Received requests: 0

Dropped requests: 0

Sent responses: 0

Dropped responses: 0

[Sysname] display real-server statistics name rs2

Slot 1:

Real server: rs2

Total connections:1

Active connections:0

Max connections: 1

Connections per second: 0

Max connections per second: 1

Server input: 0 bytes

Server output: 559 bytes

Throughput: 0 bytes/s

Inbound throughput: 0 bytes/s

Outbound throughput: 0 bytes/s

Max throughput: 559 bytes/s

Max inbound throughput: 0 bytes/s

Max outbound throughput: 559 bytes/s

Received packets:12

Sent packets:0

Dropped packets: 0

Received requests: 0

Dropped requests: 0

Sent responses: 0

Dropped responses: 0

[Sysname] display real-server statistics name rs3

Slot 1:

Real server: rs3

Total connections:1

Active connections:0

Max connections: 1

Connections per second: 0

Max connections per second: 1

Server input: 0 bytes

Server output: 599 bytes

Throughput: 0 bytes/s

Inbound throughput: 0 bytes/s

Outbound throughput: 0 bytes/s

Max throughput: 559 bytes/s

Max inbound throughput: 0 bytes/s

Max outbound throughput: 599 bytes/s

Received packets:13

Sent packets:0

Dropped packets: 0

Received requests: 0

Dropped requests: 0

Sent responses: 0

Dropped responses: 0

从实服务的统计信息来看，server input和sent packet两个统计项均为0

1、根据LB的虚服务和实服务统计信息来看，终端和LB之间报文交互有收有发属于正常情况；但是LB和服务器之间，只有LB发送报文但是LB没有收到服务器的任何回应报文

2、经过在服务器上抓包发现，服务器收到了LB的TCP请求，也回应了LB的TCP请求

3、在交换机连接服务器和LB的端口上分别同时做端口镜像抓包，发现交换机收到了服务器的回应请求，但是交换机没有把服务器回应请求转发给LB

4、现场交换机是正常转发数据的，但是报文却丢在交换机上，查看交换机的ARP表项，发现没有LB上SNAT地址池中对应地址的转发表项

5、针对LB上服务器负载的SNAT地址池测试发现，如果SNAT地址池中的地址在对应接口上不存在时，LB不会响应该地址的任何ARP报文，所以导致直连核心交换机上没有对应的ARP表项，故LB和服务器无法正常转发数据

1、服务器负载SNAT地址池中的地址务必配置在对应的出接口上

2、不使用服务器负载SNAT方法来保证来回的流量都经过LB，可以在核心交换机上配置PBR保证来回流量都经过LB

1、在LB服务器负载组网时，一般旁路部署都是为了不改变客户处原有网络拓扑环境，包括不改变服务器原有网关地址等，此时需要保证来回流量都经过LB，最简单同时对客户网络改动最少的方法就是在LB上开启服务器负载SNAT

2、服务器负载SNAT地址池中的地址务必配置在对应的出接口，保证LB正常响应地址池中对应地址的ARP请求，可以通过在接口上配置sub地址实现，所以在配置SNAT地址池的时候需要注意