The ASM disks in Exadata are provisioned as griddisks from Exadata storage cells. The griddisks are created from the celldisks. Normally, there is no free space in celldisks, as all space is used for griddisks, as seen in this example from storage cell 1:
# cellcli -e "list celldisk where name like 'CD.*' attributes name, size, freespace"
CD_00_exacell01 528.734375G 0
CD_01_exacell01 528.734375G 0
CD_02_exacell01 557.859375G 0
...
This document shows how to free up some space from the griddisks in RECO disk group, and then reuse that space to increase the size of disk group DATA.
Free up space on celldisks
To free up some space, say 88 GB per disk in disk group RECO, we need to reduce the disk size in ASM, and then reduce the griddisk size in Exadata storage cells. Let's do that for disk group RECO.
We start with the RECO griddisks with the size of 316.6875 GB:
# cellcli -e "list griddisk where name like 'RECO.*' attributes name, size"
RECO_CD_00_exacell01 316.6875G
RECO_CD_01_exacell01 316.6875G
RECO_CD_02_exacell01 316.6875G
...
To free up 88 GB, the new griddisks size will be 316.6875 GB - 88 GB = 228.6875 GB = 234176 MB.
Reduce size of RECO disks in ASM
Resize all disks in disk group RECO in ASM:
$ sqlplus / as sysasm
SQL> alter diskgroup RECO resize all size 234176M rebalance power 32;
Diskgroup altered.
SQL>
The command will trigger the rebalance operation for disk group RECO.
Monitor the rebalance with the following command:
SQL> select * from gv$asm_operation;
Once the command returns "no rows selected", the rebalance would have completed and all disks in disk group RECO should show new size.
SQL> select name, total_mb from v$asm_disk_stat where name like 'RECO%';
NAME TOTAL_MB
---------------------- --------
RECO_CD_11_EXACELL01 234176
RECO_CD_10_EXACELL01 234176
RECO_CD_09_EXACELL01 234176
...
RECOC1_CD_03_EXACELL03 234176
36 rows selected.
SQL>
Reduce size of RECO disks in storage cells
Resize the RECO griddisks on all storage cells. On storage cell 1, the command would be:
# cellcli -e alter griddisk RECO_CD_00_exacell01, RECO_CD_01_exacell01, RECO_CD_02_exacell01, RECO_CD_03_exacell01, RECO_CD_04_exacell01, RECO_CD_05_exacell01, RECO_CD_06_exacell01, RECO_CD_07_exacell01, RECO_CD_08_exacell01, RECO_CD_09_exacell01, RECO_CD_10_exacell01, RECO_CD_11_exacell01 size=234176M;
GridDisk RECO_CD_00_exacell01 successfully altered
GridDisk RECO_CD_01_exacell01 successfully altered
GridDisk RECO_CD_02_exacell01 successfully altered
...
GridDisk RECO_CD_11_exacell01 successfully altered
#
Repeat the above resize on all storage cells.
Now we have some free space on celldisks (show here on storage cell 1):
# cellcli -e "list celldisk where name like 'CD.*' attributes name, size, freespace"
CD_00_exacell01 528.734375G 88G
CD_01_exacell01 528.734375G 88G
CD_02_exacell01 557.859375G 88G
...
#
Increase size of DATA disks in storage cells
We can now increase the size of DATA griddisks, and then increase all disks size of disk group DATA in ASM.
The current DATA griddisks size is 212 GB:
# cellcli -e "list griddisk where name like 'DATA.*' attributes name, size"
DATA_CD_00_exacell01 212G
DATA_CD_01_exacell01 212G
DATA_CD_02_exacell01 212G
...
The new griddisks size will be 212 GB + 88 GB = 300 GB.
Resize the DATA griddisks on all storage cells. On storage cell 1, the command would be:
# cellcli -e alter griddisk DATA_CD_00_exacell01, DATA_CD_01_exacell01, DATA_CD_02_exacell01, DATA_CD_03_exacell01, DATA_CD_04_exacell01, DATA_CD_05_exacell01, DATA_CD_06_exacell01, DATA_CD_07_exacell01, DATA_CD_08_exacell01, DATA_CD_09_exacell01, DATA_CD_10_exacell01, DATA_CD_11_exacell01 size=300G;
GridDisk DATA_CD_00_exacell01 successfully altered
GridDisk DATA_CD_01_exacell01 successfully altered
GridDisk DATA_CD_02_exacell01 successfully altered
...
GridDisk DATA_CD_11_exacell01 successfully altered
# cellcli -e "list griddisk where name like 'DATA.*' attributes name, size"
DATA_CD_00_exacell01 300G
DATA_CD_01_exacell01 300G
DATA_CD_02_exacell01 300G
...
#
Repeat the above resize on all storage cells.
Increase size of DATA disks in ASM
Increase the size of all disks in disk group DATA, with the following command:
$ sqlplus / as sysasm
SQL> alter diskgroup DATA resize all rebalance power 32;
Diskgroup altered.
SQL>
Note that there was no need to specify the new disks size, as ASM will get that from the griddisks. The rebalance clause is optional.
The command will trigger the rebalance operation for disk group DATA.
Monitor the rebalance with the following command:
SQL> select * from gv$asm_operation;
Once the command returns "no rows selected", the rebalance would have completed and all disks in disk group DATA should show new size:
SQL> select name, total_mb/1024 "GB" from v$asm_disk_stat where name like 'DATA%';NAME GB
------------------------------ ----------
DATA_CD_02_EXACELL01 300
DATA_CD_09_EXACELL01 300
DATA_CD_07_EXACELL01 300
...
DATA_CD_06_EXACELL03 300
36 rows selected.
SQL>
Conclusion
If there is free space in Exadata cell disks, increasing the disk group size can be accomplished in two steps - griddisk size increase on all storage cells followed by the disk size increase in ASM. This requires a single ASM rebalance operation. If there is no free space in celldisks, some space may be freed up from other disk group(s) as shown in the example.
Note that reducing starts with reducing the disks size in ASM, followed by reducing the griddisks size. The increase starts with the griddisks size increase, followed by increasing the disks size in ASM.
Document id- 1684112.1
