H3C S6800系列交换机FCoE+FC对接典型配置

•        要求一：Server1、Server2均能够访问Storage1和Storage2；

•        要求二：Storage1和Storage2不能相互访问；

•        要求三：交换机之间通过聚合方式部署。

 

配置思路：

1、将Server1、Server2、Storage1、Storage2配置在同一个VSAN10中；

2、把Server1、Server2、Storage1划入Zone1；Server1、Server2、Storage2划入Zone2；

3、SW1与SW2间配置FC链路聚合，SW2与SW3间配置以太网链路聚合。

主要配置：

（一）SW1

开启高级模式，设置FCoE模式（必选）

<SW1> system-view

[SW1] system-working-mode advance //必选

[SW1] save

< SW1 > reboot

[SW1] fcoe-mode fcf     //必选

创建vsan与vlan关联（必选）

[SW1] vsan 10 //必选

[SW1-vsan10] domain-id 1 static //可选，建议手动配置

[SW1-vsan10] fspf enable //可选，默认使能

[SW1] vlan 10 //必选

[SW1-vlan10] fcoe enable vsan 10        //必选

[SW1-vlan10] quit

配置FC端口关联vsan（必选）

[SW1] interface Ten-GigabitEthernet1/2/1            

[SW1 -Ten-GigabitEthernet1/2/1] port-type fc  //必选

[SW1] interface fc 1/2/1

[SW1 -fc1/2/1] fc mode f //必选            

[SW1 -fc1/2/1] port access vsan 10      //必选 

[SW1 -fc1/2/1] qos trust dot1p     //必选

[SW1] interface Ten-GigabitEthernet1/2/2            

[SW1 -Ten-GigabitEthernet1/2/2] port-type fc  //必选

[SW1] interface fc 1/2/2

[SW1 -fc1/2/2] fc mode f //必选           

[SW1 -fc1/2/2] port access vsan 10      //必选 

[SW1 -fc1/2/2] qos trust dot1p     //必选 

[SW1] interface Ten-GigabitEthernet1/2/3

[SW1 -Ten-GigabitEthernet1/2/3] port-type fc  //必选

[SW1] interface Ten-GigabitEthernet1/2/4

[SW1 -Ten-GigabitEthernet1/2/4] port-type fc  //必选

[SW1] interface fc 1/2/3

[SW1 -fc1/2/3] fc mode e //必选           

[SW1 -fc1/2/3] port trunk vsan 10         //必选 

[SW1 -fc1/2/3] qos trust dot1p

[SW1] interface fc 1/2/4

[SW1 -fc1/2/4] fc mode e //必选           

[SW1 -fc1/2/4] port trunk vsan 10         //必选 

[SW1 -fc1/2/4] qos trust dot1p

配置FC链路聚合

[SW1] interface san-aggregation 1

[SW1-SAN-Aggregation1] fc mode e

[SW1-SAN-Aggregation1] quit

[SW1] interface fc 1/2/3

[SW1-Fc1/2/3] shutdown

[SW1-Fc1/2/3] san-aggregation group 1

[SW1-Fc1/2/3] undo shutdown

[SW1-Fc1/2/3] quit

[SW1] interface fc 1/2/4

[SW1-Fc1/2/4] shutdown

[SW1-Fc1/2/4] san-aggregation group 1

[SW1-Fc1/2/4] undo shutdown

[SW1-Fc1/2/4] quit

[SW1] interface san-aggregation 1

[SW1-SAN-Aggregation1] fc mode e //必选           

[SW1-SAN-Aggregation1] port trunk vsan 10       //必选 

ZONE配置（H3C设备必选）

[SW1] VSAN 10

[SW1 -vsan10] domain-id 1 static

[SW1 -vsan10] zone mode enhanced（可选，推荐-便于FCID管理 和 zone策略扩散）

[SW1 -vsan10] zone default-zone permit （必选，需要与对端设备配置保持一致）

[SW1 -vsan10] zone name zone1

[SW1 -vsan10-zone-zone1]pairwise-zoning enable

[SW1 -vsan10-zone-zone1] member pwwn 11:11:11:11:11:11:11:11 initiator

[SW1 -vsan10-zone-zone1] member pwwn 22:22:22:22:22:22:22:22 initiator

[SW1 -vsan10-zone-zone1] member pwwn 33:33:33:33:33:33:33:33 target

[SW1 -vsan10-zone-zone1] quit

[SW1 -vsan10] zone name zone2

[SW1 -vsan10-zone-zone2]pairwise-zoning enable

[SW1 -vsan10-zone-zone2] member pwwn 11:11:11:11:11:11:11:11 initiator

[SW1 -vsan10-zone-zone2] member pwwn 22:22:22:22:22:22:22:22 initiator

[SW1 -vsan10-zone-zone2] member pwwn 44:44:44:44:44:44:44:44 target

[SW1 -vsan10-zone-zone2] quit

[SW1 -vsan10] zoneset name zoneset1

[SW1 -vsan10-zoneset-zoneset1] member zone1

[SW1 -vsan10-zoneset-zoneset1] member zone2

[SW1 -vsan10-zoneset-zoneset1] quit

[SW1 -vsan10] zoneset activate name zoneset1  （zone策略激活并扩散同步）

（二）SW2

开启高级模式，设置FCoE模式（必选）

<SW2> system-view

[SW2] system-working-mode advance //必选

[SW2] save

< SW2 > reboot

[SW2] fcoe-mode fcf     //必选

创建vsan与vlan关联（必选）

[SW2] vsan 10 //必选

[SW2-vsan10] domain-id 2 static //可选，建议手动配置

[SW2-vsan10] fspf enable //可选，默认使能

[SW2] vlan 10 //必选

[SW2-vlan10] fcoe enable vsan 10        //必选

 [SW2-vlan10] quit

配置FC端口关联vsan（必选）

[SW2] interface Ten-GigabitEthernet1/2/3

[SW2 -Ten-GigabitEthernet1/2/3] port-type fc  //必选

[SW2] interface Ten-GigabitEthernet1/2/4

[SW2 -Ten-GigabitEthernet1/2/4] port-type fc  //必选

[SW2] interface fc 1/2/3

[SW2 -fc1/2/3] fc mode e //必选           

[SW2 -fc1/2/3] port trunk vsan 10         //必选 

[SW2 -fc1/2/3] qos trust dot1p

[SW2] interface fc 1/2/4

[SW2 -fc1/2/4] fc mode e //必选           

[SW2 -fc1/2/4] port trunk vsan 10         //必选 

[SW2 -fc1/2/4] qos trust dot1p

配置FC链路聚合

[SW2] interface san-aggregation 1

[SW2-SAN-Aggregation1] fc mode e

[SW2-SAN-Aggregation1] quit

[SW2] interface fc 1/2/3

[SW2-Fc1/2/3] shutdown

[SW2-Fc1/2/3] san-aggregation group 1

[SW2-Fc1/2/3] undo shutdown

[SW2-Fc1/2/3] quit

[SW2] interface fc 1/2/4

[SW2-Fc1/2/4] shutdown

[SW2-Fc1/2/4] san-aggregation group 1

[SW2-Fc1/2/4] undo shutdown

[SW2-Fc1/2/4] quit

[SW2] interface san-aggregation 1

[SW2-SAN-Aggregation1] fc mode e //必选            

[SW2-SAN-Aggregation1]port trunk vsan 10        //必选 

配置以太网链路聚合

[SW2] interface bridge-aggregation 1

[SW2-Bridge-Aggregation1] port trunk permit vlan 10

[SW2-Bridge-Aggregation1] quit

[SW2] interface ten-gigabitethernet 1/0/2

[SW2-Ten-GigabitEthernet1/0/2] port link-aggregation group 1

[SW2] interface ten-gigabitethernet 1/0/3

[SW2-Ten-GigabitEthernet1/0/3] port link-aggregation group 1 

配置VFC端口关联vsan（必选）

[SW2] interface vfc 102

[SW2-Vfc102] fc mode e

[SW2-Vfc102] bind interface interface bridge-aggregation 1

[SW2-Vfc102] port trunk vsan 10

[SW2-Vfc102] quit

配置物理端口（必选）

[SW2] interface Ten-GigabitEthernet1/0/2

[SW2-Ten-GigabitEthernet1/0/2] port link-mode bridge

[SW2-Ten-GigabitEthernet1/0/2] port link-type trunk    （必选，或hybrid模式）                                                               

[SW2-Ten-GigabitEthernet1/0/2] port trunk permit vlan 10（必选）

[SW2-Ten-GigabitEthernet1/0/2] ] qos trust dot1p（必选）

[SW2] interface Ten-GigabitEthernet1/0/3

[SW2-Ten-GigabitEthernet1/0/3] port link-mode bridge

[SW2-Ten-GigabitEthernet1/0/3] port link-type trunk    （必选，或hybrid模式）                                                               

[SW2-Ten-GigabitEthernet1/0/3] port trunk permit vlan 10（必选）

[SW2-Ten-GigabitEthernet1/0/3] ] qos trust dot1p（必选）

ZONE配置（H3C设备必选）

[SW2] VSAN 10

[SW2 -vsan10] domain-id 2 static

[SW2 -vsan10] zone mode enhanced（可选，推荐-便于FCID管理 和 zone策略扩散）

[SW2 -vsan10] zone default-zone permit （必选，需要与对端设备配置保持一致）

SW2无需进行ZONE相关配置

QoS配置- PFC （H3C设备必选）

[SW2] interface ten-gigabitethernet 1/0/2

[SW2 -Ten-GigabitEthernet1/0/2] priority-flow-control auto

[SW2 -Ten-GigabitEthernet1/0/2] priority-flow-control no-drop dot1p 3

[SW2 -Ten-GigabitEthernet1/0/2] qos trust dot1p

[SW2 -Ten-GigabitEthernet1/0/2] quit

 

 （三）SW3

开启高级模式，设置FCoE模式（必选）

<SW3> system-view

[SW3] system-working-mode advance //必选

[SW3] save

< SW3 > reboot

[SW3] fcoe-mode fcf     //必选

创建vsan与vlan关联（必选）

[SW3] vsan 10 //必选

[SW3-vsan10] domain-id 3 static //可选，建议手动配置

[SW3-vsan10] fspf enable //可选，默认使能

[SW3] vlan 10 //必选

[SW3-vlan10] fcoe enable vsan 10        //必选

[SW3-vlan10] quit

配置以太网链路聚合

[SW3] interface bridge-aggregation 1

[SW3-Bridge-Aggregation1] port trunk permit vlan 10

[SW3-Bridge-Aggregation1] quit

[SW3] interface ten-gigabitethernet 1/0/2

[SW3-Ten-GigabitEthernet1/0/2] port link-aggregation group 1

[SW3] interface ten-gigabitethernet 1/0/3

[SW3-Ten-GigabitEthernet1/0/3] port link-aggregation group 1

配置VFC端口关联vsan（必选）

[SW3] interface vfc 121 (必选)

[SW3-Vfc121] fc mode f (必选)

[SW3-Vfc121] bind interface ten-gigabitethernet 1/2/1 (必选)

[SW3-Vfc121] port trunk vsan 10 (必选)

[SW3-Vfc121] quit

[SW3] interface vfc 122 (必选)

[SW3-Vfc122] fc mode f (必选)

[SW3-Vfc122] bind interface ten-gigabitethernet 1/2/2 (必选)

[SW3-Vfc122] port trunk vsan 10 (必选)

[SW3-Vfc122] quit

[SW3] interface vfc 102

[SW3-Vfc102] fc mode e

[SW3-Vfc102] bind interface interface bridge-aggregation 1

[SW3-Vfc102] port trunk vsan 10

[SW3-Vfc102] quit

配置物理端口（必选）

[SW3] interface Ten-GigabitEthernet1/2/1

[SW3-Ten-GigabitEthernet1/2/1] port link-mode bridge

[SW3-Ten-GigabitEthernet1/2/1] port link-type trunk    （必选，或hybrid模式）                                                               

[SW3-Ten-GigabitEthernet1/2/1] port trunk permit vlan 10（必选）

[SW3-Ten-GigabitEthernet1/2/1] qos trust dot1p（必选）

[SW3] interface Ten-GigabitEthernet1/2/2

[SW3-Ten-GigabitEthernet1/2/2] port link-mode bridge

[SW3-Ten-GigabitEthernet1/2/2] port link-type trunk    （必选，或hybrid模式）                                                               

[SW3-Ten-GigabitEthernet1/2/2] port trunk permit vlan 10（必选）

[SW3-Ten-GigabitEthernet1/2/2] qos trust dot1p（必选）

[SW3] interface Ten-GigabitEthernet1/0/2

[SW3-Ten-GigabitEthernet1/0/2] port link-mode bridge

[SW3-Ten-GigabitEthernet1/0/2] port link-type trunk    （必选，或hybrid模式）                                                               

[SW3-Ten-GigabitEthernet1/0/2] port trunk permit vlan 10（必选）

[SW3-Ten-GigabitEthernet1/0/2] ] qos trust dot1p

[SW3] interface Ten-GigabitEthernet1/0/3

[SW3-Ten-GigabitEthernet1/0/3] port link-mode bridge

[SW3-Ten-GigabitEthernet1/0/3] port link-type trunk    （必选，或hybrid模式）                                                               

[SW3-Ten-GigabitEthernet1/0/3] port trunk permit vlan 10（必选）

[SW3-Ten-GigabitEthernet1/0/3] ] qos trust dot1p

ZONE配置（H3C设备必选）

[SW3] VSAN 10

[SW3 -vsan10] domain-id 2 static

[SW3 -vsan10] zone mode enhanced（可选，推荐-便于FCID管理 和 zone策略扩散）

[SW3 -vsan10] zone default-zone permit （必选，需要与对端设备配置保持一致）

QoS配置-DCBX （H3C设备必选）

[SW3] lldp global enable

[SW3] acl number 4000

[SW3-acl-ethernetframe-4000] rule 0 permit type 8906 ffff

[SW3-acl-ethernetframe-4000] rule 5 permit type 8914 ffff

[SW3-acl-ethernetframe-4000] quit

[SW3] traffic classifier DCBX operator or

[SW3-classifier-DCBX] if-match acl 4000

[SW3-classifier-DCBX] quit

[SW3] traffic behavior DCBX

[SW3-behavior-DCBX] remark dot1p 3

[SW3-behavior-DCBX] quit

[SW3] qos policy DCBX

[SW3-qospolicy-DCBX] classifier DCBX behavior DCBX mode dcbx

[SW3-qospolicy-DCBX] quit

[SW3] interface ten-gigabitethernet 1/2/1

[SW3-Ten-GigabitEthernet1/2/1] lldp enable

[SW3-Ten-GigabitEthernet1/2/1] lldp tlv-enable dot1-tlv dcbx

[SW3-Ten-GigabitEthernet1/2/1] qos apply policy DCBX outbound

[SW3-Ten-GigabitEthernet1/2/1] quit

[SW3] interface ten-gigabitethernet 1/2/2

[SW3-Ten-GigabitEthernet1/2/2] lldp enable

[SW3-Ten-GigabitEthernet1/2/2] lldp tlv-enable dot1-tlv dcbx

[SW3-Ten-GigabitEthernet1/2/2] qos apply policy DCBX outbound

[SW3-Ten-GigabitEthernet1/2/2] quit

QoS配置- PFC （H3C设备必选）

[SW3] interface ten-gigabitethernet 1/2/1

[SW3-Ten-GigabitEthernet1/2/1] priority-flow-control auto

[SW3 -Ten-GigabitEthernet1/2/1] priority-flow-control no-drop dot1p 3

[SW3 -Ten-GigabitEthernet1/2/1] qos trust dot1p

[SW3 -Ten-GigabitEthernet1/2/1] quit

[SW3] interface ten-gigabitethernet 1/2/2

[SW3-Ten-GigabitEthernet1/2/2] priority-flow-control auto

[SW3 -Ten-GigabitEthernet1/2/2] priority-flow-control no-drop dot1p 3

[SW3 -Ten-GigabitEthernet1/2/2] qos trust dot1p

[SW3 -Ten-GigabitEthernet1/2/2] quit

[SW3] interface ten-gigabitethernet 1/0/2

[SW3 -Ten-GigabitEthernet1/0/2] priority-flow-control auto

[SW3 -Ten-GigabitEthernet1/0/2] priority-flow-control no-drop dot1p 3

[SW3 -Ten-GigabitEthernet1/0/2] qos trust dot1p

[SW3 -Ten-GigabitEthernet1/0/2] quit

QoS配置- ETS （H3C设备必选）

[SW3] qos map-table dot1p-lp

[SW3-maptbl-dot1p-lp] import 3 export 1

[SW3-maptbl-dot1p-lp] import 0 export 0

[SW3-maptbl-dot1p-lp] import 1 export 0

[SW3-maptbl-dot1p-lp] import 2 export 0

[SW3-maptbl-dot1p-lp] import 4 export 0

[SW3-maptbl-dot1p-lp] import 5 export 0

[SW3-maptbl-dot1p-lp] import 6 export 0

[SW3-maptbl-dot1p-lp] import 7 export 0

[SW3-maptbl-dot1p-lp] quit

[SW3] interface ten-gigabitethernet 1/2/1

[SW3-Ten-GigabitEthernet1/2/1] qos wrr af1 group 1 byte-count 1

[SW3-Ten-GigabitEthernet1/2/1] qos wrr be group 1 byte-count 1

[SW3-Ten-GigabitEthernet1/2/1] qos wrr af2 group sp

[SW3-Ten-GigabitEthernet1/2/1] qos wrr af3 group sp

[SW3-Ten-GigabitEthernet1/2/1] qos wrr af4 group sp

[SW3-Ten-GigabitEthernet1/2/1] qos wrr ef group sp

[SW3-Ten-GigabitEthernet1/2/1] qos wrr cs6 group sp

[SW3-Ten-GigabitEthernet1/2/1] qos wrr cs7 group sp

[SW3-Ten-GigabitEthernet1/2/1] quit

[SW3] interface ten-gigabitethernet 1/2/2

[SW3-Ten-GigabitEthernet1/2/2] qos wrr af1 group 1 byte-count 1

[SW3-Ten-GigabitEthernet1/2/2] qos wrr be group 1 byte-count 1

[SW3-Ten-GigabitEthernet1/2/2] qos wrr af2 group sp

[SW3-Ten-GigabitEthernet1/2/2] qos wrr af3 group sp

[SW3-Ten-GigabitEthernet1/2/2] qos wrr af4 group sp

[SW3-Ten-GigabitEthernet1/2/2] qos wrr ef group sp

[SW3-Ten-GigabitEthernet1/2/2] qos wrr cs6 group sp

[SW3-Ten-GigabitEthernet1/2/2] qos wrr cs7 group sp

[SW3-Ten-GigabitEthernet1/2/2] quit

 表项信息：

（一）SW1

<SW1>dis fc name-service database //显示名称服务数据库信息

VSAN: 10

  FCID     Type               PWWN(vendor)                      FC4-type:feature

  0x010001 0x01(N)            11:11:11:11:11:11:11:11           SCSI-FCP:Initiator

  0x010002 0x01(N)            22:22:22:22:22:22:22:22           SCSI-FCP:Initiator

  0x030001 0x01(N)            33:33:33:33:33:33:33:33           SCSI-FCP:Target

  0x030002 0x01(N)            44:44:44:44:44:44:44:44           SCSI-FCP:Target

<SW1> display fc routing-table vsan 10

  Routing Table: VSAN 10

  Destinations : 3          Routes : 4

  Destination/mask   Protocol   Preference   Cost     Interface

  0x030000/8         FSPF       20           100      SAN-Aggregation1

（二）SW2

<SW1> display fc routing-table vsan 10

  Routing Table: VSAN 10

  Destinations : 3          Routes : 4

  Destination/mask   Protocol   Preference   Cost     Interface

  0x030000/8         FSPF       20           100      Vfc102

（三）SW3

<SW2> dis fc login vsan 10     //显示本地节点注册的相关信息

Interface VSAN FCID     Node WWN                Port WWN

Vfc121      10  0x030001 88:88:33:33:33:33:33:33 33:33:33:33:33:33:33:33

Vfc122      10  0x030002 88:88:44:44:44:44:44:44 44:44:44:44:44:44:44:44

 

 

 

 

1、 Qos trust dot1p配置，必须在SW1、SW2、SW3之间的12个物理接口上进行部署（San-aggregation接口上无法部署，仅能在成员FC物理接口上部署），以保障流量转发的可靠性；

2、 Zone视图下，若指定member角色（member pwwn xx:xx:xx:xx:xx:xx:xx:xx initiator/traget），则需要需要将zone设定为enhanced模式（zone mode enhanced），以便角色在zone策略扩散时，能够被其他设备识别并保证同步成功；

3、 Zone default-zone permit需要在SW1、SW2、SW3 三台交换机对应的VSAN中都进行配置，否则zone策略无法扩散成功；

4、 实验中两台服务器、两台存储设备对应的FC或VFC接口必须属于同一个VSAN_ID中（不同VSAN之间Enode设备无法进行通信）；

5、 FC接口在加入到San-aggregation前，必须要先shutdown，待FC接口加入到San-aggregation后，再undo shutdown；

6、 交换机上通过display fc login命令查看节点注册信息时，仅能够看到与本FCF交换机直连的服务器或存储设备的PWWN信息，对于远端交换机上的服务器或存储PWWN信息查看不到；

7、 实验中对于FCoE流量带宽均按照50%的形式进行配置部署，及FCoE与普通以太网流量比为1:1的关系。在实际的客户组网中，需要根据实际情况进行调整，请注意下调整方法。下面是给大家的一个举例，可参考：

a、 通过MQC将FCoE流量remark为dot1p 3

#

traffic classifier DCBX operator or

if-match acl 4000

#

traffic behavior DCBX

remark dot1p 3

#

b、通过dot1p-lp对应关系，将FCoE流量指定为AF1队列

#

qos map-table dot1p-lp

import 0 export 0

import 2 export 0

import 3 export 1        //1对应af1队列

import 4 export 0

import 5 export 0

import 6 export 0

import 7 export 0

#

[H3C-Ten-GigabitEthernet1/0/22]qos wrr ?

  INTEGER<0-7>  Queue number

  af1           Interior precedence is AF1 (1) 

c、 在接口上对FCoE流量与以太网流量带宽占比进行调整

#

interface Ten-GigabitEthernet1/0/27

……

priority-flow-control auto

 priority-flow-control no-drop dot1p 3

 lldp tlv-enable dot1-tlv dcbx

qos trust dot1p

qos wrr af1 group 1 byte-count 4      //将af1队列权值定义为4

qos wrr be group 1 byte-count 1（缺省配置，display cu时缺省不显示） //缺省所有流量都是在be 0号队列中的，权值为1,  因此Fcoe与普通以太网流量比就是4:1

……

#