Feeds:
Posts
Comments

Archive for the ‘SVM’ Category

By Joseph Gan

A metadevice consists of one or more devices (slices). It can be expanded by adding slices. Then, it can be grown to fill a larger space while the file system is in use.

However, not all UNIX file systems (UFS) can be expanded this way. The concatenation is good only for small random I/O and for even I/O distribution. On the other hand, striping is advantageous for large sequential I/O and for uneven I/O distribution, because striping will increase performance by accessing data in parallel.

Note: If you wish to expand a file system to be a single striped metadevice, you can’t do it on the fly. You have to dismount the file system, then copy or “move” over to a new partition.

How to Expand a File System With a Single Stripe, On the Fly

First, the file system has to be created and mounted as a one-way mirror metadevice; as in this example, with d80 mounted by /opt:

   # metastat d80
   d80: Mirror
       Submirror 0: d81
         State: Okay
       Pass: 1
       Read option: roundrobin (default)
       Write option: parallel (default)
       Size: 10261520 blocks

   d81: Submirror of d80
       State: Okay
       Size: 10261520 blocks
       Stripe 0: (interlace: 32 blocks)
           Device              Start Block  Dbase State        Hot Spare
           c1t12d0s0                  0     No    Okay
           c1t13d0s0               1520     No    Okay
           c1t14d0s0               1520     No    Okay
           c1t15d0s0               1520     No    Okay

Next, use the metattach command to dynamically concatenate a new slice, /dev/dsk/c0t1d0s1, to the end of the existing submirror of d80, d81:

   # metattach d81 c0t1d0s1

   # metastat d80
   d80: Mirror
       Submirror 0: d81
         State: Okay
       Pass: 1
       Read option: roundrobin (default)
       Write option: parallel (default)
       Size: 10261520 blocks

   d81: Submirror of d80
       State: Okay
       Size: 10261520 blocks
       Stripe 0: (interlace: 32 blocks)
           Device              Start Block  Dbase State        Hot Spare
           c1t12d0s0                  0     No    Okay
           c1t13d0s0               1520     No    Okay
           c1t14d0s0               1520     No    Okay
           c1t15d0s0               1520     No    Okay
       Stripe 1:
           Device              Start Block  Dbase State        Hot Spare
           c0t1d0s1                   0     No    Okay

Then, use the growfs command to expand the mounted file system ( /opt) onto the raw metadevice /dev/md/rdsk/d80:

# growfs -M /opt /dev/md/rdsk/d80

/dev/md/rdsk/d80: 12336320 sectors in 8116 cylinders of 19 tracks, 
   80 sectors
           6023.6MB in 129 cyl groups (63 c/g, 46.76MB/g, 5888 i/g)
super-block backups (for fsck -F ufs -o b=#) at:
 32, 95872, 191712, 287552, 383392, 479232, 575072, 670912, 766752, 862592,
 958432, 1054272, 1150112, 1245952, 1341792, 1437632, 1533472, 1629312,
 1725152, 1820992, 1916832, 2012672, 2108512, 2204352, 2300192, 2396032,
 2491872, 2587712, 2683552, 2779392, 2875232, 2971072, 3064352, 3160192,
 3256032, 3351872, 3447712, 3543552, 3639392, 3735232, 3831072, 3926912,
 4022752, 4118592, 4214432, 4310272, 4406112, 4501952, 4597792, 4693632,
 4789472, 4885312, 4981152, 5076992, 5172832, 5268672, 5364512, 5460352,
 5556192, 5652032, 5747872, 5843712, 5939552, 6035392, 6128672, 6224512,
 6320352, 6416192, 6512032, 6607872, 6703712, 6799552, 6895392, 6991232,
 7087072, 7182912, 7278752, 7374592, 7470432, 7566272, 7662112, 7757952,
 7853792, 7949632, 8045472, 8141312, 8237152, 8332992, 8428832, 8524672,
 8620512, 8716352, 8812192, 8908032, 9003872, 9099712, 9192992, 9288832,
 9384672, 9480512, 9576352, 9672192, 9768032, 9863872, 9959712, 10055552,
 10151392, 10247232, 10343072, 10438912, 10534752, 10630592, 10726432,
 10822272, 10918112, 11013952, 11109792, 11205632, 11301472, 11397312,
 11493152, 11588992, 11684832, 11780672, 11876512, 11972352, 12068192,
 12164032, 12257312,

Now the file system (/opt) has been expanded dynamically, but it contains two stripes: stripe 0, which is the original one, and stripe 1, which is the expanded one.

The next step is to create a single stripe metadevice d82, which is the same size as the submirror d81.

In the following example, we create a stripe with three 2.1-Gbyte disks:

   # metainit d82 1 3 c0t11d0s2 c0t12d0s2 c0t13d0s2
   d82: Concat/Stripe is setup

   # metastat d82
   d82: Concat/Stripe
       Size: 12457920 blocks
       Stripe 0: (interlace: 32 blocks)
           Device              Start Block  Dbase
           c0t11d0s2                  0     No
           c0t12d0s2               1520     No
           c0t13d0s2               1520     No

Then, add the metadevice d82 as the second submirror to d80, and resync will automatically take place:

   # metattach d80 d82
   d80: submirror d82 is attached

   # metastat d80
   d80: Mirror
       Submirror 0: d81
         State: Okay
       Submirror 1: d82
         State: Resyncing
       Resync in progress: 20 % done
       Pass: 1
       Read option: roundrobin (default)
       Write option: parallel (default)
       Size: 12336320 blocks
   ......

After the resync is complete, we have the following two-way mirrors:

   # metastat d80
   d80: Mirror
       Submirror 0: d81
         State: Okay
       Submirror 1: d82
         State: Okay
       Pass: 1
       Read option: roundrobin (default)
       Write option: parallel (default)
       Size: 12336320 blocks

   d81: Submirror of d80
       State: Okay
       Size: 12336320 blocks
       Stripe 0: (interlace: 32 blocks)
           Device              Start Block  Dbase State        Hot Spare
           c1t12d0s0                  0     No    Okay
           c1t13d0s0               1520     No    Okay
           c1t14d0s0               1520     No    Okay
           c1t15d0s0               1520     No    Okay
       Stripe 1:
           Device              Start Block  Dbase State        Hot Spare
           c0t1d0s1                   0     No    Okay

   d82: Submirror of d80
       State: Okay
       Size: 12336320 blocks
       Stripe 0: (interlace: 32 blocks)
           Device              Start Block  Dbase State        Hot Spare
           c0t11d0s2                  0     No    Okay
           c0t12d0s2               1520     No    Okay
           c0t13d0s2               1520     No    Okay

Finally, you can detach the submirror d81 from d80, and remove it completely:

   # metadetach d80 d81
   # metaclear d81

   # metastat d80
   d80: Mirror
       Submirror 1: d82
         State: Okay
       Pass: 1
       Read option: roundrobin (default)
       Write option: parallel (default)
       Size: 12336320 blocks

   d82: Submirror of d80
       State: Okay
       Size: 12336320 blocks
       Stripe 0: (interlace: 32 blocks)
           Device              Start Block  Dbase State        Hot Spare
           c0t11d0s2                  0         No        Okay
           c0t12d0s2               1520     No        Okay
           c0t13d0s2               1520     No        Okay

Now, you have dynamically expanded the file system (/opt) with a single stripe metadevice.


Please note: This procedure must be done during a quiet period, or the file system must be locked, in order to avoid possible changes to the file system during the sync. You can use the fuser -u command to check that no one is using the file system. If users are logged on overnight in their logging directory, the system admin could write-lock the file system if it is possible. In that case, users can still read files in the directory. As long as no one creates files during the sync, everything will be fine.

Source:

http://www.sun.com/bigadmin/content/submitted/expand_filesystem.html

Read Full Post »

Joseph Gan, August, 2004

What can you do if a file system needs to grow, but it cannot be mounted as a metadevice on the fly? What if the file system is not suitable to be mounted as a metadevice? What can you do when file systems like these run out the space?

Usually you have to copy the data to a larger partition, which means you have to dismount the file system or bring the machine down, and then copy the data over. Otherwise, you can restore the data from backup media. Here is a simple way to do all this using Solaris Volume Manager software: With this method, you can expand a UFS file system on the fly.

(Note: If you are able to mount the file system as a meta device instead, you can refer to my other Tech Tip, Expanding a File System on the Fly.)

In this case, I use the /test file system as an example.

The /test file system is 96 percent full, as indicated in the output by the df command:

  # df -k
  ......
  /dev/dsk/c7t5d7s0    8662733 8307768 268338    96%    /test
  ......

First, make a new partition, in this case device c9t1d3 on a Sun StorEdge T3 array, and make it 60 Gbyte in size:

# format
Searching for disks...done

AVAILABLE DISK SELECTIONS:
    ......
        1. c9t1d3
          /sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w50020f230000666e,3
Specify disk (enter its number): 1
selecting c9t1d3
[disk formatted]
Warning: Current Disk has mounted partitions.

partition> p
Current partition table (original):
Total disk cylinders available: 34901 + 2 (reserved cylinders)

Part      Tag    Flag     Cylinders         Size            Blocks
  0       home    wm       1 - 10240       60.00GB    (10240/0/0) 125829120
  1 unassigned    wm       0                0         (0/0/0)             0
  2     backup    wu       0 - 34900      204.50GB    (34901/0/0) 428863488
  3       home    wm   21000 - 22749       10.25GB    (1750/0/0)   21504000
  4       home    wm   26293 - 28172       11.02GB    (1880/0/0)   23101440
  5       home    wm   28173 - 33467       31.03GB    (5295/0/0)   65064960
  6 unassigned    wm       0                0         (0/0/0)             0
  7 unassigned    wm       0                0         (0/0/0)             0

partition> 1
Part      Tag    Flag     Cylinders         Size            Blocks
  1 unassigned    wm   0           0                (0/0/0)    0

Enter partition id tag[unassigned]:
Enter partition permission flags[wm]:
Enter new starting cyl[10241]: 10241
Enter partition size[132194304b, 10758c, 64548.00mb, 63.04gb]: 60g
partition> p
Current partition table (unnamed):
Total disk cylinders available: 34901 + 2 (reserved cylinders)

Part      Tag    Flag     Cylinders         Size            Blocks
  0       home    wm       1 - 10240       60.00GB    (10240/0/0) 125829120
  1 unassigned    wm   10241 - 20480       60.00GB    (10240/0/0) 125829120
  2     backup    wu       0 - 34900      204.50GB    (34901/0/0) 428863488
  3       home    wm   21000 - 22749       10.25GB    (1750/0/0)   21504000
  4       home    wm   26293 - 28172       11.02GB    (1880/0/0)   23101440
  5       home    wm   28173 - 33467       31.03GB    (5295/0/0)   65064960
  6 unassigned    wm       0                0         (0/0/0)             0
  7 unassigned    wm       0                0         (0/0/0)             0

partition> l
Ready to label disk, continue? y

partition> q

Second, use Solaris Volume Manager software as a “bridge” to create a one-way mirror named d124 on the original file system by using the -f (force) option:

# metainit -f d125 1 1 c7t5d7s0
d125: Concat/Stripe is setup

# metainit d124 -m d125
d124: Mirror is setup

Third, create a new sub-mirror named d126 on the new partition as formatted earlier:

# metainit d126 1 1 c9t1d3s1
d126: Concat/Stripe is setup

Now, attach d126 to d124, using the metattach command. Resynchronization will automatically take place:

# metattach d124 d126
d124: submirror d126 is attached

When this has been done, we have successfully “moved” the data over, and the best thing is that the data is always synchronized.

# metastat d124
d124: Mirror
    Submirror 0: d125
      State: Okay
    Submirror 1: d126
      State: Okay
    Resync in progress: 100 % done
    Pass: 1
    Read option: roundrobin (default)
    Write option: parallel (default)
    Size: 17596416 blocks

d125: Submirror of d124
    State: Okay
    Size: 17596416 blocks
    Stripe 0:
        Device              Start Block  Dbase State        Hot Spare
        c7t5d7s0                   0     No    Okay

d126: Submirror of d124
    State: Resyncing
    Size: 125829120 blocks
    Stripe 0:
        Device              Start Block  Dbase State        Hot Spare
        c9t1d3s1                   0     No    Okay

What you have to do now is dismount the file system from the old partition, detach the sub-mirror, and clear all the metadevices you have created for this case. Then remount the file system to the new partition:

# umount /test
# metadetach d124 d126
d124: submirror d126 is detached

# metaclear d124
d124: Mirror is cleared

# metaclear d125
d125: Concat/Stripe is cleared

# mount /dev/dsk/c9t1d3s1 /oracle

# df -k
......
/dev/dsk/c9t1d3s1    8662733  8316368 259738      96%    /test

At this point, the new file system is still the same size as the previous one, because it is a sub-mirror.

Finally, you’ll need to record the new size in blocks (from the output of metastat d126), as well as use some special arguments and the second hidden mkfs command, so you can expand the new file system on the fly:

# /usr/lib/fs/ufs/mkfs -G -M /test /dev/rdsk/c9t1d3s1 125829120

/dev/rdsk/c9t1d3s1:     125829120 sectors in 30720 cylinders of 64 tracks, 64
sectors
        61440.0MB in 1182 cyl groups (26 c/g, 52.00MB/g, 6400 i/g)
super-block backups (for fsck -F ufs -o b=#) at:
32, 106592, 213152, 319712, 426272, 532832, 639392, 745952, 852512, 959072,
......

125240864, 125347424, 125453984, 125560544, 125667104, 125773664,

You can check that the file system (/test) has been successfully expanded on the new partition:

# df -k
Filesystem            kbytes    used   avail capacity  Mounted on
......
/dev/dsk/c9t1d3s1    61950013  8316368 53633645      13%    /test
.....

Please note: This procedure must be done during a quiet period, or the file system must be locked, in order to avoid possible changes to the file system during the sync. You can use the fuser -u command to check that no one is using the file system. If users are logged on overnight in their logging directory, the system admin could write-lock the file system if it is possible. In that case, users can still read files in the directory. As long as no one creates files during the sync, everything will be fine.

Source:

http://www.sun.com/bigadmin/content/submitted/expand_ufs_svm.html

Read Full Post »

Joseph Gan, August, 2004

Let’s say you need to move the data of a file system from one partition to another on a disk unit such as a Sun RSM Array, a Sun StorEdge T3 array, or a RAID device from StorageTek. This generally involves dismounting the file system, backing it up to tapes, and restoring it to a new partition. You may also copy the file system from one place to another while it is unmounted. This process could take a significant amount of time for a large file system, and it will impact the use of the system.

To move a file system to a different partition on the fly, you can use Solaris Volume Manager software. If you don’t need to expand the file system, you don’t even have to mount it as a metadevice. In the end, you only need to dismount the file system, and mount it onto the new partition. All the processes can be executed in the background while the file system is still in use.

You can swap to the new partition at any time. This can reduce system downtime in many cases. Let’s use the /home file system as an example:

# df -k /home
Filesystem kbytes used avail capacity Mounted on
/dev/dsk/c0t2d1s3 50700783 39746386 10530839 80% /home

First, on top of this physical mount point of the /home file system, create a metadevice named d101. Remember, the /home file system is still mounted.

# metainit -f d101 1 1 c0t2d1s3

Next, you need to initialize a one-way mirror metadevice, d100, with the submirror d101 you have created.

# metainit d100 -m d101
# metastat d100
d100: Mirror
    Submirror 0: d101
      State: Okay
    Pass: 1
    Read option: roundrobin (default)
    Write option: parallel (default)
    Size: 102961152 blocks
d101: Submirror of d100
    State: Okay
    Size: 102961152 blocks
    Stripe 0:
        Device              Start Block  Dbase State        Hot Spare
	c0t2d1s3                   0     No    Okay

Now it’s time to create a single-stripe metadevice (in this example, d102 on c0t2d1s7), where the target file system would be. The size of the new partition should be the same as d101.

# metainit d102 1 1 c0t2d1s7
d102: Concat/Stripe is setup

# metastat d102
d102: Concat/Stripe
    Size: 102975488 blocks
    Stripe 0:
        Device              Start Block  Dbase
        c0t2d1s7                   0     No

After that, you add the metadevice d102 as the second submirror to d100. Here the resynchronization will automatically take place in the background.

# metattach d100 d102

After the resynchronization has been completed successfully, here’s an example of the two-way mirrors you would get:

# metastat d100
d100: Mirror
    Submirror 0: d101
      State: Okay
    Submirror 1: d102
      State: Okay
    Pass: 1
    Read option: roundrobin (default)
    Write option: parallel (default)
    Size: 102961152 blocks
d101: Submirror of d100
    State: Okay
    Size: 102961152 blocks
    Stripe 0:
        Device              Start Block  Dbase State        Hot Spare
	c0t2d1s3                   0     No    Okay

d102: Submirror of d100
    State: Okay
    Size: 102961152 blocks
    Stripe 0:
        Device              Start Block  Dbase State        Hot Spare
	c0t2d1s7                   0     No    Okay

Now, you have a new partition for the /home file system, ready to swap at any time while it’s still online.

With your user, you can schedule when to dismount the home file system. You can then mount the system to the new partition on c0t2d1s7, shown as follows:

# df -k /home
Filesystem kbytes used avail capacity Mounted on
/dev/dsk/c0t2d1s7 50700783 39746386 10530839 80% /home

Finally, you’ll detach and clean up all metadevices from your system:

# metadetach d100 d101
d100: submirror d101 is detached

# metaclear d101
d101: Concat/Stripe is cleared

# metaclear d100
d100: Concat/Stripe is cleared

# metaclear d102
d102: Concat/Stripe is cleared

Now you have performed fewer than 10 steps, and the /home file system has been moved dynamically.


Please note: This procedure must be done during a quiet period, or the file system must be locked, in order to avoid possible changes to the file system during the sync. You can use the fuser -u command to check that no one is using the file system. If users are logged on overnight in their logging directory, the system admin could write-lock the file system if it is possible. In that case, users can still read files in the directory. As long as no one creates files during the sync, everything will be fine.

Source:

http://www.sun.com/bigadmin/content/submitted/moving_filesystem.html

Read Full Post »