iMC CSM VCFC方案中下发VMware虚机失败经验案例

在iMC CSM VCFC方案中，底层虚拟化平台为VMware ESXi 5.5 U2平台。CSM组件版本为截至目前最新版本：CSM E0205H06。

保证下述配置正确：

1，      采用分布式虚拟交换机并且分布式虚拟交换机与ESXi主机关联，关联的操作是指ESXi主机有物理网卡与分布式虚拟交换相连；

2，      Openstack的补丁已安装，CSM与VMware配合时，无需安装联动插件，因为VMware自身有相应的接口。但是需要打上对应版本的Openstack补丁包，具体位置：在CSM的安装包中，CSM E0205H06版本对应的补丁位于CSM E0205P05安装包中: iMC_CSM_7.1-E0205P05\iMC CSM 7.1-E0205P05\openstack_upgrade\openstack-juno-sp05；安装补丁时需要看readme说明，控制器与计算节点分开时，各自安装对应的补丁；若为控制器与计算节点集中部署，compute-setup和controller-setup都需要安装；

3，      确认ESXi主机已正确配置共享存储；

4，      确认Openstack相关进程运行状态正常；

5，      确认虚机镜像模板制作符合要求；

保证上述配置无误条件下，遇到CSM下发VMware虚机依然失败的情况，具体如下图-1所示。

图-1

此时查看VCenter任务列表，发现无任何虚机创建相关的动作产生。

问题涉及的相关日志有iMC的前台日志\iMC\client\log\imcforegroud.log和Openstack日志，位于计算节点/var/log/nova/nova-compute.log。

前台日志中包含新建云主机的相关信息：

通过查看用户新建云主机的名称来确认新建的时间和相关的信息，比如使用到的镜像，此处为fe5a6e9d-289c-4039-8ad0-3886ffeb656a，以及虚机的配置信息。随后将对应的信息在nova-compute.log日志中搜索对应的报错信息来定位问题。

2015-12-23 11:22:15 [INFO ] [pool-42-thread-6] [com.imc.cloudss.service.client.ConnectionUtils::sendRequest] application/json,UTF-8,{

  "server" : {

    "name" : "testtest",

    "imageRef" : "http://10.16.100.37:8774/v2/9655fbd4795440e7985a7b7559c7a837/images/fe5a6e9d-289c-4039-8ad0-3886ffeb656a ",

    "flavorRef" : "http://10.16.100.37:8774/v2/9655fbd4795440e7985a7b7559c7a837/flavors/053cd523-bf0f-4e07-96e7-e63299347f0a ",

    "minCount" : 1,

    "maxCount" : 1,

    "networks" : [ {

      "name" : "ncdx-switch",

      "provider:segmentation_id" : "2",

      "port" : "5d86dea9-eaa8-42bb-ba14-f13945138058"

    } ],

    "availability_zone" : "zone1"

  }

查看nova-compute日志：

    2015-12-23 11:22:03.448 1786 ERROR oslo.vmware.common.loopingcall [-] in fixed duration looping call

2015-12-23 11:22:03.448 1786 TRACE oslo.vmware.common.loopingcall AttributeError: &＃39;NoneType&＃39; object has no attribute &＃39;state&＃39;

2015-12-23 11:22:03.450 1786 TRACE nova.compute.manager [instance: 3b4e5f0a-0aca-402b-832e-8950dad93eb6] AttributeError: &＃39;NoneType&＃39; object has no attribute &＃39;state&＃39;

2015-12-23 11:31:37.249 1786 ERROR oslo.vmware.api [-] Task: (returnval){

   value = "session[7c7fa1ec-3173-9197-2917-fd0bd989f735]5268450b-6f40-147d-5ed6-56de076f9d07"

   _type = "Task"

} failed with error: File [datastore1 (5)] 10.16.100.36_base/fe5a6e9d-289c-4039-8ad0-3886ffeb656a was not found.

虚机创建过程日志信息：

2015-12-23 11:20:59.172 1786 AUDIT nova.compute.resource_tracker [-] Free disk (GB): 4045

2015-12-23 11:20:59.173 1786 AUDIT nova.compute.resource_tracker [-] Total usable vcpus: 96, total allocated vcpus: 0

2015-12-23 11:20:59.173 1786 AUDIT nova.compute.resource_tracker [-] PCI stats: []

2015-12-23 11:20:59.174 1786 INFO nova.compute.resource_tracker [-] Compute_service record updated for OpenstackComputer:domain-c181(CSMSDN)

2015-12-23 11:21:57.190 1786 AUDIT nova.compute.resource_tracker [-] Auditing locally available compute resources

2015-12-23 11:21:59.173 1786 AUDIT nova.compute.resource_tracker [-] Total physical ram (MB): 760150, total allocated virtual ram (MB): 16896

2015-12-23 11:21:59.174 1786 AUDIT nova.compute.resource_tracker [-] Free disk (GB): 4045

2015-12-23 11:21:59.174 1786 AUDIT nova.compute.resource_tracker [-] Total usable vcpus: 96, total allocated vcpus: 0

2015-12-23 11:21:59.175 1786 AUDIT nova.compute.resource_tracker [-] PCI stats: []

2015-12-23 11:21:59.196 1786 INFO nova.scheduler.client.report [-] Compute_service record updated for (&＃39;OpenstackComputer&＃39;, domain-c181(CSMSDN))

2015-12-23 11:21:59.197 1786 INFO nova.compute.resource_tracker [-] Compute_service record updated for OpenstackComputer:domain-c181(CSMSDN)

2015-12-23 11:22:01.656 1786 AUDIT nova.compute.manager [req-971e2223-ec69-448e-8792-f78b80bed5ac None] [instance: 3b4e5f0a-0aca-402b-832e-8950dad93eb6] Starting instance...

2015-12-23 11:22:01.737 1786 AUDIT nova.compute.claims [-] [instance: 3b4e5f0a-0aca-402b-832e-8950dad93eb6] Attempting claim: memory 16384 MB, disk 60 GB

2015-12-23 11:22:01.738 1786 AUDIT nova.compute.claims [-] [instance: 3b4e5f0a-0aca-402b-832e-8950dad93eb6] Total memory: 760150 MB, used: 16896.00 MB

2015-12-23 11:22:01.738 1786 AUDIT nova.compute.claims [-] [instance: 3b4e5f0a-0aca-402b-832e-8950dad93eb6] memory limit: 1140225.00 MB, free: 1123329.00 MB

2015-12-23 11:22:01.738 1786 AUDIT nova.compute.claims [-] [instance: 3b4e5f0a-0aca-402b-832e-8950dad93eb6] Total disk: 4095 GB, used: 50.00 GB

2015-12-23 11:22:01.739 1786 AUDIT nova.compute.claims [-] [instance: 3b4e5f0a-0aca-402b-832e-8950dad93eb6] disk limit not specified, defaulting to unlimited

2015-12-23 11:22:01.754 1786 AUDIT nova.compute.claims [-] [instance: 3b4e5f0a-0aca-402b-832e-8950dad93eb6] Claim successful

2015-12-23 11:22:01.846 1786 INFO nova.scheduler.client.report [-] Compute_service record updated for (&＃39;OpenstackComputer&＃39;, domain-c181(CSMSDN))

2015-12-23 11:22:02.079 1786 INFO nova.scheduler.client.report [-] Compute_service record updated for (&＃39;OpenstackComputer&＃39;, domain-c181(CSMSDN))

2015-12-23 11:22:03.199 1786 INFO nova.scheduler.client.report [-] Compute_service record updated for (&＃39;OpenstackComputer&＃39;, domain-c181(CSMSDN))

2015-12-23 11:22:03.448 1786 ERROR oslo.vmware.common.loopingcall [-] in fixed duration looping call

2015-12-23 11:22:03.448 1786 TRACE oslo.vmware.common.loopingcall Traceback (most recent call last):

2015-12-23 11:22:03.448 1786 TRACE oslo.vmware.common.loopingcall   File "/usr/lib/python2.7/dist-packages/oslo/vmware/common/loopingcall.py", line 76, in _inner

2015-12-23 11:22:03.448 1786 TRACE oslo.vmware.common.loopingcall     self.f(*self.args, **self.kw)

2015-12-23 11:22:03.448 1786 TRACE oslo.vmware.common.loopingcall   File "/usr/lib/python2.7/dist-packages/oslo/vmware/api.py", line 402, in _poll_task

2015-12-23 11:22:03.448 1786 TRACE oslo.vmware.common.loopingcall     if task_info.state in [&＃39;queued&＃39;, &＃39;running&＃39;]:

2015-12-23 11:22:03.448 1786 TRACE oslo.vmware.common.loopingcall AttributeError: &＃39;NoneType&＃39; object has no attribute &＃39;state&＃39;

2015-12-23 11:22:03.448 1786 TRACE oslo.vmware.common.loopingcall

2015-12-23 11:22:03.450 1786 ERROR nova.compute.manager [-] [instance: 3b4e5f0a-0aca-402b-832e-8950dad93eb6] Instance failed to spawn

从日志中看主要报错为虚机创建使用的镜像查找失败，Openstack有虚机模板镜像老化机制，当虚机镜像在ESXi主机缓存中老化后，新建虚机时会从glance从新下载镜像，由于该机制始终能保证新建虚机有可用的镜像，因此该报错并不能反映出问题。

此时需要设置nova日志级别为debug级别。设置方法：修改计算节点的/etc/nova/nova.conf配置文件中[default]配置域中加上debug=True配置，随后运行service  nova-compute  restart 重启服务；企业增强版则运行 service  openstack-nova-compute  restart 重启服务。

重启服务后，收集计算节点的/var/log/nova/nova-compute.log日志文件，会有下述信息。

2015-12-24 09:24:42.486 22698 DEBUG nova.openstack.common.service [req-30acc03c-a2aa-4fb3-a375-191ad95d71e1 - - - - -] neutron_plugin = openvswitch log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2191

提示neutron_plugin值为openvswitch该值为vlan方案，在VCFC方案中应该为vcfc；因此判断出问题为计算节点的/etc/nova/nova.conf配置文件中[default]配置域少了neutron_plugin=vcfc配置。

在计算节点的/etc/nova/nova.conf配置文件中[default]配置域增加neutron_plugin=vcfc配置。目前给配置项已向资料部门反馈，添加到《20150702-H3Cloud云管理平台运营版部署指导》文档。