0720
现场采用自带的异构迁移,将vmware虚拟机迁移到UIS上,已经迁移了很多台了,就有两台虚拟机始终迁移不成功,每次迁移到中间都会中断,异构提示如下:
每次都是迁移过程中发生的故障,排查网络是没问题的,就怀疑虚拟机重启了,我们远程检查发现的确是虚拟机故障重启了,看系统日志是cgroup内存耗尽将虚拟机进程给kill掉了,但是主机内存是足够的
un 2 17:09:15 uisnode18 kernel: Task in /machine.slice/machine-qemu\x2d22\x2dssosqzxjkzx3.scope killed as a result of limit of /machine.slice
Jun 2 17:09:15 uisnode18 kernel: memory: usage 209715200kB, limit 209715200kB, failcnt 48720
Jun 2 17:09:15 uisnode18 kernel: memory+swap: usage 0kB, limit 9007199254740988kB, failcnt 0
Jun 2 17:09:15 uisnode18 kernel: kmem: usage 656716kB, limit 9007199254740988kB, failcnt 0
Jun 2 17:09:15 uisnode18 kernel: Memory cgroup stats for /machine.slice: cache:0KB rss:0KB rss_huge:0KB shmem:0KB mapped_file:0KB dirty:0KB writeback:0KB inactive_anon:0KB active_anon:0KB inactive_file:0KB active_file:0KB unevictable:0KB
Jun 2 17:09:15 uisnode18 kernel: [ pid ] uid tgid total_vm rss nr_ptes nr_pmds swapents oom_score_adj name
Jun 2 17:09:15 uisnode18 kernel: [3293835] 0 3293835 4602095 4210674 8617 20 0 0 kvm
Jun 2 17:09:15 uisnode18 kernel: [3325564] 0 3325564 4652838 4210192 8643 21 0 0 kvm
Jun 2 17:09:15 uisnode18 kernel: [4136915] 0 4136915 4570902 4208696 8616 20 0 0 kvm
Jun 2 17:09:15 uisnode18 kernel: [15304] 0 15304 4751312 4209961 8813 22 0 0 kvm
Jun 2 17:09:15 uisnode18 kernel: [218159] 0 218159 1448456 1061833 2456 9 0 0 kvm
Jun 2 17:09:15 uisnode18 kernel: [2085438] 0 2085438 4585412 3893894 8514 20 0 0 kvm
Jun 2 17:09:15 uisnode18 kernel: [2197548] 0 2197548 4605730 4152937 8533 21 0 0 kvm
Jun 2 17:09:15 uisnode18 kernel: [3972334] 0 3972334 1420936 1062303 2462 9 0 0 kvm
Jun 2 17:09:15 uisnode18 kernel: [1581765] 0 1581765 2481334 2111008 4543 12 0 0 kvm
Jun 2 17:09:15 uisnode18 kernel: [695985] 0 695985 8789397 8160590 16265 36 0 0 kvm
Jun 2 17:09:15 uisnode18 kernel: [695989] 0 695989 8730358 6612803 13162 32 0 0 kvm
Jun 2 17:09:15 uisnode18 kernel: Memory cgroup out of memory: Kill process 695985 (kvm) score 151 or sacrifice child
Jun 2 17:09:15 uisnode18 kernel: Killed process 695985 (kvm) total-vm:35157588kB, anon-rss:32628388kB, file-rss:13952kB, shmem-rss:20k
查看cgroup配置文件,发现machine.slice的大小只有200G,但是主机总内存500G左右
迁移过程中,主机内存超过200G,导致虚拟机被kill掉
执行bash /opt/bin/onestor_cgroup_config.sh重新配置cgroup,再去查看/etc/cgconfig.conf配置文件machine.slice的limit内存刷新到了493G.再次尝试迁移虚拟机,可以成功。
这种情况的出现可能是之前该cvk作为cvm节点部署过,导致cgroup配置不准确,主机重启时会自动刷新cgroup配置
该案例暂时没有网友评论
✖
案例意见反馈
亲~登录后才可以操作哦!
确定你的邮箱还未认证,请认证邮箱或绑定手机后进行当前操作