一、 组网:
XX集团广域网通过2台SR6616路由器与各矿业集团MSR50-60组建,采用mpls vpn技术组网,SR66与MSR50-60都为PE设备。为了实现链路备份,总部SR6616与各矿业集团的MSR50-60路由器之间分别通过联通、电信的10M MSTP链路连接,广域网设备通过两条链路采用逐流的方式实现负载分担。
客户反映其中一个矿业集团在召开视频会议时画面马赛克严重,基本无法正常召开会议,从视频会议系统中发现矿业集团的视频终端到总部的MCU丢包严重,有时达10%以上。
1、查看广域网链路是否拥塞?
首先查看2条广域网线路的流量,观察是否是因为流量过大网络拥塞导致,发现2条链路的流量峰值4M左右,而运营商提供的带宽为10M,不存在拥塞。
2、是否在广域网线上丢包?
在召开视频会议时,各矿业集团通过广域网线路与总部的MCU通信,除了出现故障的矿业集团反馈有问题外,其他矿业集团正常,据此判断此矿业集团丢包应该发生在广域网线路或矿业集团内部。
在MSR50-60上带vpn实例ping矿业集团的视频终端地址,用8100字节大包连续ping1000个报文未发现丢包现象。
在MSR50-60上分别ping联通线路的对端地址和电信线路的对端地址,发现ping联通线路的对端地址不丢包,ping电信线路的对端地址丢包。跟客户商量后,暂时关闭电信线路,丢包现象消失,视频会议正常;把电信线路恢复、联通线路关闭,丢包现象重现,至此说明丢包现象应该与电信线路有关。
那包是丢在链路上了,还是丢在SR6616或MSR50-60设备上了,还需要我们进一步定位。
3、确定数据包丢在哪儿?
组网设备是SR6616跟MSR50-60设备,所以我们采取在接口上下发包过滤规则,查看防火墙统计的方法来进行定位。
设备配置:
u SR6616路由器
firewall enable //开启防火墙功能
acl number 3900 //定义需统计的ACL规则
rule 0 permit icmp source 192.168.253.21 0 destination 192.168.253.22 0
rule 10 permit icmp source 192.168.253.22 0 destination 192.168.253.21 0
interface g0/1/5 //在电信线路的接口入方向上应用包过滤。
firewall packet-filter 3900 inbound
u MSR50-60路由器
firewall enable //开启防火墙功能
acl number 3900 //定义需统计的ACL规则
rule 0 permit icmp source 192.168.253.22 0 destination 192.168.253.21 0
rule 10 permit icmp source 192.168.253.21 0 destination 192.168.253.22 0
interface g0/2 //在电信线路的接口入方向上应用包过滤。
firewall packet-filter 3900 inbound
测试步骤:
在MSR50-60路由器上操作:
Interface: GigabitEthernet0/2
In-bound Policy: acl 3900
Fragments matched normally
From 2013-11-27 9:22:42 to 2013-11-27 9:24:22
0 packets, 0 bytes, 0% permitted,
0 packets, 0 bytes, 0% denied,
309 packets, 19673 bytes, 100% permitted default,
0 packets, 0 bytes, 0% denied default,
Totally 309 packets, 19673 bytes, 100% permitted,
Totally 0 packets, 0 bytes, 0% denied.
Interface: GigabitEthernet0/2
Out-bound Policy: acl 3900
Fragments matched normally
From 2013-11-27 9:22:42 to 2013-11-27 9:24:22
0 packets, 0 bytes, 0% permitted,
0 packets, 0 bytes, 0% denied,
160 packets, 9726 bytes, 100% permitted default,
0 packets, 0 bytes, 0% denied default,
Totally 160 packets, 9726 bytes, 100% permitted,
Totally 0 packets, 0 bytes, 0% denied.
PING 192.168.253.21: 56 data bytes, press CTRL_C to break
Reply from 192.168.253.21: bytes=56 Sequence=1 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=2 ttl=255 time=17 ms
Reply from 192.168.253.21: bytes=56 Sequence=3 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=4 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=5 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=6 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=7 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=8 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=9 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=10 ttl=255 time=10 ms
--- 192.168.253.21 ping statistics ---
10 packet(s) transmitted
10 packet(s) received
0.00% packet loss
round-trip min/avg/max = 10/10/17 ms
PING 192.168.253.21: 56 data bytes, press CTRL_C to break
Reply from 192.168.253.21: bytes=56 Sequence=1 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=2 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=3 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=4 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=5 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=6 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=7 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=8 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=9 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=10 ttl=255 time=11 ms
--- 192.168.253.21 ping statistics ---
10 packet(s) transmitted
10 packet(s) received
0.00% packet loss
round-trip min/avg/max = 10/10/11 ms
PING 192.168.253.21: 56 data bytes, press CTRL_C to break
Reply from 192.168.253.21: bytes=56 Sequence=1 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=2 ttl=255 time=25 ms
Reply from 192.168.253.21: bytes=56 Sequence=3 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=4 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=5 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=6 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=7 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=8 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=9 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=10 ttl=255 time=10 ms
--- 192.168.253.21 ping statistics ---
10 packet(s) transmitted
10 packet(s) received
0.00% packet loss
round-trip min/avg/max = 10/11/25 ms
PING 192.168.253.21: 56 data bytes, press CTRL_C to break
Reply from 192.168.253.21: bytes=56 Sequence=1 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=2 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=3 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=4 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=5 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=6 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=7 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=8 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=9 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=10 ttl=255 time=12 ms
--- 192.168.253.21 ping statistics ---
10 packet(s) transmitted
10 packet(s) received
0.00% packet loss
round-trip min/avg/max = 10/10/12 ms
PING 192.168.253.21: 56 data bytes, press CTRL_C to break
Reply from 192.168.253.21: bytes=56 Sequence=1 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=2 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=3 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=4 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=5 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=6 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=7 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=8 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=9 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=10 ttl=255 time=11 ms
--- 192.168.253.21 ping statistics ---
10 packet(s) transmitted
10 packet(s) received
0.00% packet loss
round-trip min/avg/max = 10/10/11 ms
PING 192.168.253.21: 56 data bytes, press CTRL_C to break
Reply from 192.168.253.21: bytes=56 Sequence=1 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=2 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=3 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=4 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=5 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=6 ttl=255 time=10 ms
Request time out
Reply from 192.168.253.21: bytes=56 Sequence=8 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=9 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=10 ttl=255 time=11 ms
--- 192.168.253.21 ping statistics ---
10 packet(s) transmitted
9 packet(s) received
10.00% packet loss
round-trip min/avg/max = 10/10/11 ms
PING 192.168.253.21: 56 data bytes, press CTRL_C to break
Reply from 192.168.253.21: bytes=56 Sequence=1 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=2 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=3 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=4 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=5 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=6 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=7 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=8 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=9 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=10 ttl=255 time=11 ms
--- 192.168.253.21 ping statistics ---
10 packet(s) transmitted
10 packet(s) received
0.00% packet loss
round-trip min/avg/max = 10/10/11 ms
PING 192.168.253.21: 56 data bytes, press CTRL_C to break
Reply from 192.168.253.21: bytes=56 Sequence=1 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=2 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=3 ttl=255 time=12 ms
Reply from 192.168.253.21: bytes=56 Sequence=4 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=5 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=6 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=7 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=8 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=9 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=10 ttl=255 time=11 ms
--- 192.168.253.21 ping statistics ---
10 packet(s) transmitted
10 packet(s) received
0.00% packet loss
round-trip min/avg/max = 10/10/12 ms
PING 192.168.253.21: 56 data bytes, press CTRL_C to break
Reply from 192.168.253.21: bytes=56 Sequence=1 ttl=255 time=10 ms
Request time out
Reply from 192.168.253.21: bytes=56 Sequence=3 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=4 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=5 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=6 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=7 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=8 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=9 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=10 ttl=255 time=10 ms
--- 192.168.253.21 ping statistics ---
10 packet(s) transmitted
9 packet(s) received
10.00% packet loss
round-trip min/avg/max = 10/10/11 ms
PING 192.168.253.21: 56 data bytes, press CTRL_C to break
Reply from 192.168.253.21: bytes=56 Sequence=1 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=2 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=3 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=4 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=5 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=6 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=7 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=8 ttl=255 time=10 ms
Reply from 192.168.253.21: bytes=56 Sequence=9 ttl=255 time=11 ms
Reply from 192.168.253.21: bytes=56 Sequence=10 ttl=255 time=12 ms
--- 192.168.253.21 ping statistics ---
10 packet(s) transmitted
10 packet(s) received
0.00% packet loss
round-trip min/avg/max = 10/10/12 ms
Interface: GigabitEthernet0/2
In-bound Policy: acl 3900
Fragments matched normally
From 2013-11-27 9:22:42 to 2013-11-27 9:25:19
98 packets, 8232 bytes, 14% permitted,
0 packets, 0 bytes, 0% denied,
558 packets, 33276 bytes, 86% permitted default,
0 packets, 0 bytes, 0% denied default,
Totally 656 packets, 41508 bytes, 100% permitted,
Totally 0 packets, 0 bytes, 0% denied.
Interface: GigabitEthernet0/2
Out-bound Policy: acl 3900
Fragments matched normally
From 2013-11-27 9:22:42 to 2013-11-27 9:25:19
100 packets, 8400 bytes, 28% permitted,
0 packets, 0 bytes, 0% denied,
251 packets, 15298 bytes, 72% permitted default,
0 packets, 0 bytes, 0% denied default,
Totally 351 packets, 23698 bytes, 100% permitted,
Totally 0 packets, 0 bytes, 0% denied.
以上数据说明,本端发出了100个报文,但是只接收到了98个,另外两个可能是对端SR66没有发出,或是在线路上丢了。
在SR66设备上操作:
Interface: GigabitEthernet0/1/5
In-bound Policy: acl 3900
From 2013-11-27 9:25:51 to 2013-11-27 9:28:03
98 packets, 8232 bytes, 16% permitted,
0 packets, 0 bytes, 0% denied,
503 packets, 27942 bytes, 84% permitted default,
0 packets, 0 bytes, 0% denied default,
Totally 601 packets, 36174 bytes, 100% permitted,
Totally 0 packets, 0 bytes, 0% denied.
Interface: GigabitEthernet0/1/5
Out-bound Policy: acl 3900
From 2013-11-27 9:25:51 to 2013-11-27 9:28:03
98 packets, 8232 bytes, 34% permitted,
0 packets, 0 bytes, 0% denied,
190 packets, 11828 bytes, 66% permitted default,
0 packets, 0 bytes, 0% denied default,
Totally 288 packets, 20060 bytes, 100% permitted,
Totally 0 packets, 0 bytes, 0% denied.
以上数据说明,从对端发出的100个报文,SR66上只收到了98个,在线路上丢了2个,同时发出了98个报文,对端都收到了。
SR66入方向线路有丢包。
PING 192.168.253.22: 56 data bytes, press CTRL_C to break
Reply from 192.168.253.22: bytes=56 Sequence=1 ttl=255 time=11 ms
Reply from 192.168.253.22: bytes=56 Sequence=2 ttl=255 time=11 ms
Reply from 192.168.253.22: bytes=56 Sequence=3 ttl=255 time=11 ms
Reply from 192.168.253.22: bytes=56 Sequence=4 ttl=255 time=10 ms
Request time out
--- 192.168.253.22 ping statistics ---
5 packet(s) transmitted
4 packet(s) received
20.00% packet loss
round-trip min/avg/max = 10/10/11 ms
PING 192.168.253.22: 56 data bytes, press CTRL_C to break
Reply from 192.168.253.22: bytes=56 Sequence=1 ttl=255 time=11 ms
Reply from 192.168.253.22: bytes=56 Sequence=2 ttl=255 time=10 ms
Reply from 192.168.253.22: bytes=56 Sequence=3 ttl=255 time=10 ms
Reply from 192.168.253.22: bytes=56 Sequence=4 ttl=255 time=11 ms
Request time out
--- 192.168.253.22 ping statistics ---
5 packet(s) transmitted
4 packet(s) received
20.00% packet loss
round-trip min/avg/max = 10/10/11 ms
我们ping了2次,共计10个报文,显示丢了2个。
Interface: GigabitEthernet0/1/5
In-bound Policy: acl 3900
From 2013-11-27 9:25:51 to 2013-11-27 9:28:49
106 packets, 8904 bytes, 13% permitted,
0 packets, 0 bytes, 0% denied,
668 packets, 36846 bytes, 87% permitted default,
0 packets, 0 bytes, 0% denied default,
Totally 774 packets, 45750 bytes, 100% permitted,
Totally 0 packets, 0 bytes, 0% denied.
Interface: GigabitEthernet0/1/5
Out-bound Policy: acl 3900
From 2013-11-27 9:25:51 to 2013-11-27 9:28:49
108 packets, 9072 bytes, 30% permitted,
0 packets, 0 bytes, 0% denied,
252 packets, 15426 bytes, 70% permitted default,
0 packets, 0 bytes, 0% denied default,
Totally 360 packets, 24498 bytes, 100% permitted,
Totally 0 packets, 0 bytes, 0% denied.
以上数据说明,SR66路由器发出了10个报文,而收到了8个。
Interface: GigabitEthernet0/2
In-bound Policy: acl 3900
Fragments matched normally
From 2013-11-27 9:22:42 to 2013-11-27 9:27:19
108 packets, 9072 bytes, 9% permitted,
0 packets, 0 bytes, 0% denied,
1051 packets, 59596 bytes, 91% permitted default,
0 packets, 0 bytes, 0% denied default,
Totally 1159 packets, 68668 bytes, 100% permitted,
Totally 0 packets, 0 bytes, 0% denied.
Interface: GigabitEthernet0/2
Out-bound Policy: acl 3900
Fragments matched normally
From 2013-11-27 9:22:42 to 2013-11-27 9:27:19
110 packets, 9240 bytes, 20% permitted,
0 packets, 0 bytes, 0% denied,
414 packets, 24622 bytes, 80% permitted default,
0 packets, 0 bytes, 0% denied default,
Totally 524 packets, 33862 bytes, 100% permitted,
Totally 0 packets, 0 bytes, 0% denied.
以上数据说明,MSR收到从SR66上发出的10个报文,自己又发出了10个回应报文,对比SR66上的数据,说明SR66入方向,线路又丢了2个包。
GigabitEthernet0/1/5 current state: UP
Line protocol current state: UP
Description: to SDNY-LKKY-R5060 GE0/1 10M
The Maximum Transmit Unit is 1500
Internet Address is 192.168.253.21/30 Primary
IP Packet Frame Type: PKTFMT_ETHNT_2, Hardware Address: 3822-d61e-71b5
IPv6 Packet Frame Type: PKTFMT_ETHNT_2, Hardware Address: 3822-d61e-71b5
Media type is twisted pair, loopback not set, promiscuous mode not set
100Mb/s, Full-duplex, link type is autonegotiation
Output flow-control is disabled, input flow-control is disabled
Output queue : (Urgent queuing : Size/Length/Discards) 0/100/0
Output queue : (Protocol queuing : Size/Length/Discards) 0/500/0
Output queue : (FIFO queuing : Size/Length/Discards) 0/1024/0
Last clearing of counters: Never
Last 300 seconds input rate 108926.19 bytes/sec, 871408 bits/sec, 102.83 packets/sec
Last 300 seconds output rate 2860.07 bytes/sec, 22880 bits/sec, 9.51 packets/sec
Input: 463363066 packets, 518090337252 bytes, 0 no buffers
21 broadcasts, 1069219 multicasts, 0 pauses
24203 errors, 0 runts, 0 giants
24203 crc, 0 align errors, 0 overruns
0 dribbles, 0 drops
Output:57125480 packets, 38163108493 bytes
152 broadcasts, 2299449 multicasts, 0 pauses
0 errors, 0 underruns, 0 collisions
0 deferred, 0 lost carriers
结论:通过前面的防火墙报文统计,可以看出报文丢在了SR66侧的入方向,并且SR66接口上也有CRC校验错误的报文,说明运营商线路存在问题。
。
四、 解决方法:
通过以上步聚分析,问题已经定位清楚,包丢在运营商线路上。让客户联系运营商排除线路,经过运营商调整线路后,网络正常,不再丢包,视频会议召开正常。
该案例暂时没有网友评论
✖
案例意见反馈
亲~登录后才可以操作哦!
确定你的邮箱还未认证,请认证邮箱或绑定手机后进行当前操作