License Server HA创建失败经验案例

某客户安装License Server之后，配置对应License Server 的HA高可用功能。填写对应参数，点击确认，License Server提示“创建集群失败”，如下图所示：

License Server在后台日志存放路径是/opt/LicServer/log

License Server日志licmcore.log日志提示如下信息：

2017-12-09 03:53:15.859 [INFO ] [team0035] Receive get team info message.

2017-12-09 03:53:15.859 [INFO ] [team0061]Begin to get team info from redis

2017-12-09 03:53:15.859 [ERROR] [team0039] Failed get team info.

2017-12-09 03:53:15.859 [INFO ] [team0008]Receive save team config info messgae.

2017-12-09 03:53:15.859 [INFO ] pcMaster 99.1.1.204 hello

2017-12-09 03:53:15.859 [INFO ] pcSlave 99.1.1.205 hello

2017-12-09 03:53:15.859 [INFO ] pcVirtual 99.1.1.212 hello

2017-12-09 03:53:15.859 [INFO ] team id 100 hello

2017-12-09 03:53:15.860 [INFO ] [team0054]master 99.1.1.204, slave 99.1.1.205, v 99.1.1.212, id 100

2017-12-09 03:53:15.860 [INFO ] [team0030]Add team info successfully

2017-12-09 03:53:25.354 [INFO ] [team0035] Receive get team info message.

2017-12-09 03:53:25.354 [INFO ] [team0061]Begin to get team info from redis

2017-12-09 03:53:25.354 [INFO ] [team0062]100

2017-12-09 03:53:25.361 [INFO ] [team0045]Receive delete team info messgae.

2017-12-09 03:53:28.799 [INFO ] [team0035] Receive get team info message.

2017-12-09 03:53:28.799 [INFO ] [team0061]Begin to get team info from redis

2017-12-09 03:53:28.799 [ERROR] [team0039] Failed get team info.

License Server日志licmwebj.log日志提示如下信息：

2017-12-09 03:53:15 [INFO ] [com.h3c.licm.controller.TeamController::createTeam] Enter add Team

2017-12-09 03:53:16 [ERROR] [com.h3c.licm.team.TeamConfigServiceImpl::execBuildFile] Redirecting to /bin/systemctl start  keepalived.service

2017-12-09 03:53:25 [ERROR] [com.h3c.licm.controller.TeamController::createTeam] can&＃39;t find role file

2017-12-09 03:53:25 [ERROR] [com.h3c.licm.controller.TeamController::createTeam] 创建集群失败

2017-12-09 03:53:28 [ERROR] [com.h3c.licm.team.TeamConfigServiceImpl::execBuildFile] Redirecting to /bin/systemctl stop  keepalived.service

2017-12-09 03:53:28 [ERROR] [com.h3c.licm.team.TeamConfigServiceImpl::execBuildFile] kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]

2017-12-09 03:53:28 [INFO ] [com.h3c.licm.controller.TeamController::createTeam] Exit add Team

2017-12-09 03:53:28 [INFO ] [com.h3c.licm.controller.TeamController::isTeamAlready] Enter addOrShow

 

从日志可以分析出集群创建失败的原因是keepalive进程异常导致。首先查看License Server集群配置文件是否已经正确生成。如下:

[root@localhost etc]# cat /opt/LicServer/keepalive/etc/keepalived.conf
! Configuration File for keepalived

global_defs {
    router_id 99.1.1.204
}

vrrp_instance VI_1_t {
    state BACKUP
    interface eth0
    mcast_src_ip 99.1.1.204
    virtual_router_id 100
    priority 60
    advert_int 1
    nopreempt
    authentication {
        auth_type xxxxxx
        auth_pass xxxxxx
    }

    virtual_ipaddress {
        99.1.1.212
    }
   
    notify_master "/opt/LicServer/keepalive/script/master.sh"
    notify_backup "/opt/LicServer/keepalive/script/standby.sh"
}

发现配置文件已经按照正确的配置进行生成。

继续查看当前keepalived进程是否正常，如下：

[root@localhost etc]# systemctl status keepalived
keepalived.service - keepalived
   Loaded: loaded (/etc/systemd/system/keepalived.service; disabled)
   Active: inactive (dead)

Dec 09 07:18:05 localhost.localdomain Keepalived_vrrp[3045]: bogus VRRP packet received on eth0 !!!
Dec 09 07:18:05 localhost.localdomain Keepalived_vrrp[3045]: VRRP_Instance(VI_1_t) ignoring received advertisment...
Dec 09 07:18:06 localhost.localdomain Keepalived_vrrp[3045]: ip address associated with VRID not present in received packet : 99.1.1.212
Dec 09 07:18:06 localhost.localdomain Keepalived_vrrp[3045]: one or more VIP associated with VRID mismatch actual MASTER advert
Dec 09 07:18:06 localhost.localdomain Keepalived_vrrp[3045]: bogus VRRP packet received on eth0 !!!
Dec 09 07:18:06 localhost.localdomain Keepalived_vrrp[3045]: VRRP_Instance(VI_1_t) ignoring received advertisment...
Dec 09 07:18:07 localhost.localdomain Keepalived_vrrp[3045]: ip address associated with VRID not present in received packet : 99.1.1.212
Dec 09 07:18:07 localhost.localdomain Keepalived_vrrp[3045]: one or more VIP associated with VRID mismatch actual MASTER advert
Dec 09 07:18:07 localhost.localdomain Keepalived_vrrp[3045]: bogus VRRP packet received on eth0 !!!
Dec 09 07:18:07 localhost.localdomain Keepalived_vrrp[3045]: VRRP_Instance(VI_1_t) ignoring received advertisment...
Dec 09 07:18:08 localhost.localdomain systemd[1]: Stopping keepalived...
Dec 09 07:18:10 localhost.localdomain systemd[1]: Stopped keepalived.

发现keepalived进程异常，并且提示VRID不一致。由于keepalived使用的是VRRP协议，VRRP协议通过向224.0.0.18地址发送对应协商组播报文，携带对应参数，其中数据包中包含VRID（虚拟路由标识），该ID取值范围为1-255（十进制）。

如果同一个网络中出现了相同VRID的组播报文，会出现VRRP协商失败的问题，需要更改VRID的数值重新进行协商。

更换License Server页面上的HA ID数值，重新进行HA配置。

需要注意的是，如果环境中不同版本的License Server如果配置成HA ID一致一样会出现某个集群无法正常建立的问题。

1、建议一个SDN网络环境，或者一个云网融合方案中仅部署一套具有HA功能的License Server，License Server能够同时满足各个客户端的同时接入，包含CloudOS，VCFC、VNF等。

2、在某些过度方案中，环境里留存两套具有HA功能的License Server时，请使用不同HA ID进行实施。