【3AR/Primera】growvv 后在 Linux 层面 online resizing 的实施过程

3PAR创建逻辑卷映射给服务器后，服务器也识别到存储映射过来的逻辑卷。但客户需要将此逻辑卷扩容，这时在操作系统及数据库侧需要做哪些操作。

可以参考以下的操作步骤

3PAR创建逻辑卷映射给服务器后，服务器也识别到存储映射过来的逻辑卷。但客户需要将此逻辑卷扩容，这时在操作系统及数据库侧需要做哪些操作。

可以参考以下的操作步骤

存储执行 growvv 后在 Linux 层面上的操作

首先我们对 3PAR/Primera 一个VV 扩容，我们的例子中从 300G 扩容到 350G 。

先确认一下当前要做调整的逻辑卷的大小： 300G

C670_CN701503MR cli% showvv

8213  ky670vv3         tdvv Yes   Yes   base  ---          8213 RW normal                0  179200   307200

在操作系统侧查看的磁盘容量

ky670vv3 (360002ac0000000000000201500025c03) dm-17 3PARdata,VV

size=300G features="2 queue_if_no_path retain_attached_hw_handler" hwhandler="1 alua" wp=rw

`-+- policy="round-robin 0" prio=0 status=active

  |- 4:0:0:2 sdak 66:64  active undef running

  |- 5:0:0:2 sdao 66:128 active undef running

  |- 4:0:2:2 sdam 66:96  active undef running

  `- 5:0:2:2 sdaq 66:160 active undef running

这个多路径对应的sdX 为 sdam/sdao/sdaq/sdak

我们记录下扩容前的fdisk显示

sdam --ky670v3

Disk /dev/sdam: 322.1 GB, 322122547200 bytes, 629145600 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 16384 bytes / 16777216 bytes

=========================================================================

sdao  --ky670vv3

Disk /dev/sdao: 322.1 GB, 322122547200 bytes, 629145600 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 16384 bytes / 16777216 bytes

==========================================================================

sdaq  --ky670vv3

Disk /dev/sdaq: 322.1 GB, 322122547200 bytes, 629145600 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 16384 bytes / 16777216 bytes

=========================================================================

sdak  --ky670vv3

Disk /dev/sdak: 322.1 GB, 322122547200 bytes, 629145600 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 16384 bytes / 16777216 bytes

growvv 将逻辑卷扩大到350G

C670_CN701503MR cli% growvv ky670vv3 50g

C670_CN701503MR cli% showvv -s ky670vv3

                                    ---------Snp---------- -----------Usr------------ -----------Total-----------

                                    --(MiB)-- -(% VSize)-- ----(MiB)---- -(% VSize)-- -----------(MiB)----------- ---Efficiency---

  Id Name     Prov Compr Dedup Type Rsvd Used Used Wrn Lim   Rsvd   Used Used Wrn Lim   Rsvd   Used HostWr  VSize Compact Compress

8213 ky670vv3 tdvv Yes   Yes   base    0    0  0.0  --  -- 179200 176026 49.1   0   0 179200 176026 176026 358400    2.04     1.00

----------------------------------------------------------------------------------------------------------------------------------

   1 total                             0    0              179200 176026              179200 176026 176026 358400

C670_CN701503MR cli%

在线扩盘需要将每个 sdX 都需要扫描

# echo 1 > /sys/block/sdak/device/rescan

# echo 1 > /sys/block/sdaq/device/rescan

# echo 1 > /sys/block/sdao/device/rescan

# echo 1 > /sys/block/sdam/device/rescan

扫描后发现这时的 sdX 的容量就扩上来了。

sdam --ky670v3

Disk /dev/sdam: 375.8 GB, 375809638400 bytes, 734003200 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 16384 bytes / 16777216 bytes

==================================================================

sdao  --ky670vv3

Disk /dev/sdao: 375.8 GB, 375809638400 bytes, 734003200 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 16384 bytes / 16777216 bytes

==================================================================

sdaq  --ky670vv3

Disk /dev/sdaq: 375.8 GB, 375809638400 bytes, 734003200 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 16384 bytes / 16777216 bytes

================================================================== 

sdak  --ky670vv3

Disk /dev/sdak: 375.8 GB, 375809638400 bytes, 734003200 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 16384 bytes / 16777216 bytes

但发现这时多路径聚合后的磁盘容量大小还是没有改过来

(base) [root@baremetal ~]# multipath -ll

ky670vv3 (360002ac0000000000000201500025c03) dm-17 3PARdata,VV

size=300G features="2 queue_if_no_path retain_attached_hw_handler" hwhandler="1 alua" wp=rw

`-+- policy="round-robin 0" prio=50 status=active

  |- 4:0:0:2 sdak 66:64  active ready running

  |- 5:0:0:2 sdao 66:128 active ready running

  |- 4:0:2:2 sdam 66:96  active ready running

  `- 5:0:2:2 sdaq 66:160 active ready running

多路径大小在线修改

(base) [root@baremetal ~]# multipathd -k "resize map ky670vv3"

multipathd> resize map ky670vv3

ok

multipathd>

这时再看多路径大小就修改过来了。

(base) [root@baremetal ~]# multipath -ll

ky670vv3 (360002ac0000000000000201500025c03) dm-17 3PARdata,VV

size=350G features="1 queue_if_no_path" hwhandler="1 alua" wp=rw

`-+- policy="round-robin 0" prio=50 status=active

  |- 4:0:0:2 sdak 66:64  active ready running

  |- 5:0:0:2 sdao 66:128 active ready running

  |- 4:0:2:2 sdam 66:96  active ready running

  `- 5:0:2:2 sdaq 66:160 active ready running

我们再用 grid 账号登录进去

[root@rac2 ~]# su - grid

Last login: Thu Jul 30 09:43:34 CST 2020

[grid@rac2:/home/grid]

我这里设置的环境变量可能有问题，只有进到 $ORACLE_HOME/bin 目录下才能执行 sqlplus

[grid@rac2:/home/grid]$cd $ORACLE_HOME

[grid@rac2:/home/ora12c/app/12.2.0.1/grid]$cd bin

[grid@rac2:/home/ora12c/app/12.2.0.1/grid/bin]$

[grid@rac2:/home/ora12c/app/12.2.0.1/grid/bin]./sqlplus / as sysasm

SQL*Plus: Release 12.2.0.1.0 Production on Thu Jul 30 09:50:11 2020

Copyright (c) 1982, 2016, Oracle.  All rights reserved.

Connected to:

Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

SQL>

查看现在的 asm disk group 的大小

SQL> select name,state,type,total_mb,free_mb from v$asm_diskgroup;

NAME

------------------------------------------------------------

STATE                  TYPE           TOTAL_MB    FREE_MB

---------------------- ------------ ---------- ----------

DATA

CONNECTED              EXTERN           409568     335892

OCR

MOUNTED                EXTERN           112608     112464

REDO

CONNECTED              EXTERN           102368      98500

NAME

------------------------------------------------------------

STATE                  TYPE           TOTAL_MB    FREE_MB

---------------------- ------------ ---------- ----------

THIN_DATA

CONNECTED              EXTERN           122864      68440

SQL>

可以看到 THIN_DATA 里面还是有数据的。

SQL> select name,state,type,total_mb,free_mb from v$asm_diskgroup where name="THIN_DATA";

NAME                           STATE       TYPE     TOTAL_MB    FREE_MB

------------------------------ ----------- ------ ---------- ----------

THIN_DATA                      MOUNTED     EXTERN     122864      68440

SQL>

这时我们执行 alter diskgroup THIN_DATA resize all

SQL> alter diskgroup THIN_DATA resize all;

Diskgroup altered.

SQL>

[grid@rac2:/home/grid]$cd $ORACLE_HOME

[grid@rac2:/home/ora12c/app/12.2.0.1/grid]$cd bin

[grid@rac2:/home/ora12c/app/12.2.0.1/grid/bin]$./sqlplus / as sysasm

SQL*Plus: Release 12.2.0.1.0 Production on Thu Jul 30 10:47:53 2020

Copyright (c) 1982, 2016, Oracle.  All rights reserved.

Connected to:

Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

SQL>

SQL> alter diskgroup THIN_DATA resize all;

Diskgroup altered.

SQL> select name,state,type,total_mb,free_mb from v$asm_diskgroup where name="THIN_DATA";

NAME                           STATE       TYPE     TOTAL_MB    FREE_MB

------------------------------ ----------- ------ ---------- ----------

THIN_DATA                      MOUNTED     EXTERN     430064     375640

SQL>

成功扩容。

这样就可以在线将3PAR/Primera存储在线扩容后，Linux 设备文件在线扩容，Oracle 的 asm disk group 实现在线扩容。