Increase GPFS Filesystem Size
1. On server03/04/05
To increase
dev/gpf_elm_general1 84040.90 15858.49 82% 613452 5% /elm/general/general1 - by 8TB
dev/gpf_general1 20026.54 3816.46 81% 355220 15% /edw/general/general1 - by 2TB
# root [server03] /: mmgetstate -a
Node number Node name GPFS state
------------------------------------------
1 server03-priv active
2 server04-priv active
3 server05-priv active
root [server03] /: df -g | grep -i general1
dev/gpf_elm_general1 84040.90 15858.49 82% 613452 5% /elm/general/general1
dev/gpf_general1 20026.54 3816.46 81% 355220 15% /edw/general/general1
#mmlsdisk gpf_elm_general1
Note the last few disks
# mmlsdisk gpf_elm_general1
disk driver sector failure holds holds storage
name type size group metadata data status availability pool
------------ -------- ------ ------- -------- ----- ------------- ------------ ------------
.
.
.
.
pelmcls1nsd_725 nsd 512 1 yes yes ready up system
pelmcls1nsd_726 nsd 512 1 yes yes ready up system
-
In this case - pelmcls1nsd_726
Add this to the end of the line, adjust it accordingly with the number of drives you will be adding
disk 1 - pelmcls1nsd_727
disk 2 - pelmcls1nsd_728
Update the hdiskxxx with the disk you see assigned by SAN team after running #cfgmgr on all 3 servers, the hdisk# may not be the same on all 3 servers, work on server03
In this example, these are the 2 new disks
hdisk622 none None
hdisk623 none None
Create the file: disks in /tmp directory
Asssume we are adding two new disks to extend the GPFS filesystem
# vi disks
# add the following 2 lines
hdisk622:server03-priv,server04-priv,server05-priv::dataAndMetadata:1:pelmcls1nsd_727
hdisk623:server03-priv,server04-priv,server05-priv::dataAndMetadata:1:pelmcls1nsd_728
save the file
run the following commands - mmcrnsd and mmadddisk
that will add the disk to gpfs format & they are ready to use
1. mmcrnsd -F /tmp/disks - to create nsd
2. mmlsnsd - to verify that disk has been created
(free disk) pelmcls1nsd_727 server03-priv.com,server04-priv.com,server05-priv.com
(free disk) pelmcls1nsd_728 server03-priv.com,server04-priv.com,server05-priv.com
3. mmadddisk gpf_elm_general1 -F /tmp/disks - to add the new nsd to the gpf_elm_general1 file system
this is an example of the output
server03 # mmadddisk gpf_general1 -F /tmp/disks
This is a private system. Unauthorized use is not permitted
This is a private system. Unauthorized use is not permitted
server04-priv.com: This is a private system. Unauthorized use is not permitted
The following disks of gpf_general1 will be formatted on node server03:
pedwcls1nsd_557: size 512179200 KB
pedwcls1nsd_558: size 512179200 KB
pedwcls1nsd_559: size 512179200 KB
pedwcls1nsd_560: size 512179200 KB
Extending Allocation Map
Checking Allocation Map for storage pool 'system'
Completed adding disks to file system gpf_general1.
server04-priv.com: This is a private system. Unauthorized use is not permitted
mmadddisk: Propagating the cluster configuration data to all
affected nodes. This is an asynchronous process.
4. mmdf gpf_elm_general1 to verify that nsd was added
root [server03] /: mmdf gpf_elm_general1
disk disk size failure holds holds free KB free KB
name in KB group metadata data in full blocks in fragments
--------------- ------------- -------- -------- ----- -------------------- -------------------
Disks in storage pool: system (Maximum disk size allowed is 16 TB)
.
.
.
pelmcls1nsd_743 512179200 1 yes yes 512021504 (100%) 496 ( 0%)
pelmcls1nsd_742 512179200 1 yes yes 512021504 (100%) 704 ( 0%)
pelmcls1nsd_741 512179200 1 yes yes 512021504 (100%) 496 ( 0%)
pelmcls1nsd_740 512179200 1 yes yes 512022016 (100%) 496 ( 0%)
pelmcls1nsd_739 512179200 1 yes yes 512023040 (100%) 496 ( 0%)
pelmcls1nsd_738 512179200 1 yes yes 512022016 (100%) 496 ( 0%)
pelmcls1nsd_737 512179200 1 yes yes 512023040 (100%) 496 ( 0%)
pelmcls1nsd_736 512179200 1 yes yes 512023040 (100%) 496 ( 0%)
pelmcls1nsd_735 512179200 1 yes yes 512023040 (100%) 496 ( 0%)
pelmcls1nsd_734 512179200 1 yes yes 512023552 (100%) 496 ( 0%)
pelmcls1nsd_733 512179200 1 yes yes 512023040 (100%) 496 ( 0%)
pelmcls1nsd_732 512179200 1 yes yes 512023552 (100%) 496 ( 0%)
pelmcls1nsd_731 512179200 1 yes yes 512023040 (100%) 544 ( 0%)
pelmcls1nsd_730 512179200 1 yes yes 512023552 (100%) 496 ( 0%)
pelmcls1nsd_729 512179200 1 yes yes 512024064 (100%) 496 ( 0%)
pelmcls1nsd_728 512179200 1 yes yes 512024064 (100%) 496 ( 0%)
.
.
pelmcls1nsd_727 512179200 1 yes yes 511833088 (100%) 1536 ( 0%)
after the 17 drives are added for additonal 8TB
root [server03] /: df -g /elm/general/general1
Filesystem GB blocks Free %Used Iused %Iused Mounted on
/dev/gpf_elm_general1 92344.59 20796.38 78% 628035 5% /elm/general/general1
1. On server03/04/05
To increase
dev/gpf_elm_general1 84040.90 15858.49 82% 613452 5% /elm/general/general1 - by 8TB
dev/gpf_general1 20026.54 3816.46 81% 355220 15% /edw/general/general1 - by 2TB
# root [server03] /: mmgetstate -a
Node number Node name GPFS state
------------------------------------------
1 server03-priv active
2 server04-priv active
3 server05-priv active
root [server03] /: df -g | grep -i general1
dev/gpf_elm_general1 84040.90 15858.49 82% 613452 5% /elm/general/general1
dev/gpf_general1 20026.54 3816.46 81% 355220 15% /edw/general/general1
#mmlsdisk gpf_elm_general1
Note the last few disks
# mmlsdisk gpf_elm_general1
disk driver sector failure holds holds storage
name type size group metadata data status availability pool
------------ -------- ------ ------- -------- ----- ------------- ------------ ------------
.
.
.
.
pelmcls1nsd_725 nsd 512 1 yes yes ready up system
pelmcls1nsd_726 nsd 512 1 yes yes ready up system
-
In this case - pelmcls1nsd_726
Add this to the end of the line, adjust it accordingly with the number of drives you will be adding
disk 1 - pelmcls1nsd_727
disk 2 - pelmcls1nsd_728
Update the hdiskxxx with the disk you see assigned by SAN team after running #cfgmgr on all 3 servers, the hdisk# may not be the same on all 3 servers, work on server03
In this example, these are the 2 new disks
hdisk622 none None
hdisk623 none None
Create the file: disks in /tmp directory
Asssume we are adding two new disks to extend the GPFS filesystem
# vi disks
# add the following 2 lines
hdisk622:server03-priv,server04-priv,server05-priv::dataAndMetadata:1:pelmcls1nsd_727
hdisk623:server03-priv,server04-priv,server05-priv::dataAndMetadata:1:pelmcls1nsd_728
save the file
run the following commands - mmcrnsd and mmadddisk
that will add the disk to gpfs format & they are ready to use
1. mmcrnsd -F /tmp/disks - to create nsd
2. mmlsnsd - to verify that disk has been created
(free disk) pelmcls1nsd_727 server03-priv.com,server04-priv.com,server05-priv.com
(free disk) pelmcls1nsd_728 server03-priv.com,server04-priv.com,server05-priv.com
3. mmadddisk gpf_elm_general1 -F /tmp/disks - to add the new nsd to the gpf_elm_general1 file system
this is an example of the output
server03 # mmadddisk gpf_general1 -F /tmp/disks
This is a private system. Unauthorized use is not permitted
This is a private system. Unauthorized use is not permitted
server04-priv.com: This is a private system. Unauthorized use is not permitted
The following disks of gpf_general1 will be formatted on node server03:
pedwcls1nsd_557: size 512179200 KB
pedwcls1nsd_558: size 512179200 KB
pedwcls1nsd_559: size 512179200 KB
pedwcls1nsd_560: size 512179200 KB
Extending Allocation Map
Checking Allocation Map for storage pool 'system'
Completed adding disks to file system gpf_general1.
server04-priv.com: This is a private system. Unauthorized use is not permitted
mmadddisk: Propagating the cluster configuration data to all
affected nodes. This is an asynchronous process.
4. mmdf gpf_elm_general1 to verify that nsd was added
root [server03] /: mmdf gpf_elm_general1
disk disk size failure holds holds free KB free KB
name in KB group metadata data in full blocks in fragments
--------------- ------------- -------- -------- ----- -------------------- -------------------
Disks in storage pool: system (Maximum disk size allowed is 16 TB)
.
.
.
pelmcls1nsd_743 512179200 1 yes yes 512021504 (100%) 496 ( 0%)
pelmcls1nsd_742 512179200 1 yes yes 512021504 (100%) 704 ( 0%)
pelmcls1nsd_741 512179200 1 yes yes 512021504 (100%) 496 ( 0%)
pelmcls1nsd_740 512179200 1 yes yes 512022016 (100%) 496 ( 0%)
pelmcls1nsd_739 512179200 1 yes yes 512023040 (100%) 496 ( 0%)
pelmcls1nsd_738 512179200 1 yes yes 512022016 (100%) 496 ( 0%)
pelmcls1nsd_737 512179200 1 yes yes 512023040 (100%) 496 ( 0%)
pelmcls1nsd_736 512179200 1 yes yes 512023040 (100%) 496 ( 0%)
pelmcls1nsd_735 512179200 1 yes yes 512023040 (100%) 496 ( 0%)
pelmcls1nsd_734 512179200 1 yes yes 512023552 (100%) 496 ( 0%)
pelmcls1nsd_733 512179200 1 yes yes 512023040 (100%) 496 ( 0%)
pelmcls1nsd_732 512179200 1 yes yes 512023552 (100%) 496 ( 0%)
pelmcls1nsd_731 512179200 1 yes yes 512023040 (100%) 544 ( 0%)
pelmcls1nsd_730 512179200 1 yes yes 512023552 (100%) 496 ( 0%)
pelmcls1nsd_729 512179200 1 yes yes 512024064 (100%) 496 ( 0%)
pelmcls1nsd_728 512179200 1 yes yes 512024064 (100%) 496 ( 0%)
.
.
pelmcls1nsd_727 512179200 1 yes yes 511833088 (100%) 1536 ( 0%)
after the 17 drives are added for additonal 8TB
root [server03] /: df -g /elm/general/general1
Filesystem GB blocks Free %Used Iused %Iused Mounted on
/dev/gpf_elm_general1 92344.59 20796.38 78% 628035 5% /elm/general/general1
No comments:
Post a Comment