Veritas licenses

Veritas licenses
The following information pertains to Veritas Volume Manager 3.2 for Solaris.

Location of license keys:
/etc/vx/elm

The key is the fourth line of the license file, below:
!
# DO NOT EDIT/COPY/MOVE/TOUCH THIS FILE!
# DOING SO WILL INVALIDATE THE KEY!

Check validity of license keys:
vxliccheck -pv
vrts:vxliccheck: INFO: License 95 valid
vrts:vxliccheck: INFO: License 96 valid
vrts:vxliccheck: INFO: License 98 valid

Print license details:
vxlicense -p

Create a license key file:
vxlicense -c

Unencapsulating a root disk

Unencapsulating a root disk
If your system partitions (/, swap, /usr, /var) are located on more than one physical disk, you will have to manually "unencapsulate" your root disk instead of using Veritas’ vxunroot command below.

1. Modify /etc/vfstab to reference the cxtxdxsx devices instead of the VxVM devices.

2. Comment out the lines in /etc/system between:

* vxvm_START (do not remove)
* vxvm_END (do not remove)

3. Run the following command to prevent VxVM from starting up after reboot:

touch /etc/vx/reconfig.d/state.d/install-db

4. Reboot the system. After the reboot, you may uninstall VxVM if needed.

System partitions on boot disk
The Veritas vxunroot command is used to unencapsulate a root disk that contains all your system partitions. However, if the root disk is mirrored, you have to remove the mirror plexes.

Example:

# /etc/vx/bin/vxunroot

This operation will convert the following file systems from
volumes to regular partitions: root swap usr var opt home

ERROR: There are 2 plexes associated with volume rootvol
The vxunroot operation cannot proceed.

Listing of all volumes in rootdg:

# vxprint -v -g rootdg
TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0
v opt gen ENABLED 4198392 – ACTIVE – –
v rootvol root ENABLED 1050776 – ACTIVE – –
v swapvol swap ENABLED 4198392 – ACTIVE – –
v usr gen ENABLED 4198392 – ACTIVE – –
v var gen ENABLED 4198392 – ACTIVE – –

Here we see that rootdg contains volumes opt, rootvol, swapvol, usr, and var. Let’s see if the volumes consist of more than one plex.

# vxprint opt rootvol swapvol usr var
Disk group: rootdg

TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0
v opt gen ENABLED 4198392 – ACTIVE – –
pl opt-01 opt ENABLED 4198392 – ACTIVE – –
sd rootdisk-04 opt-01 ENABLED 4198392 0 – – –
pl opt-02 opt ENABLED 4198392 – ACTIVE – –
sd rootdisk-mirror-01 opt-02 ENABLED 4198392 0 – – –

v rootvol root ENABLED 1050776 – ACTIVE – –
pl rootvol-01 rootvol ENABLED 1050776 – ACTIVE – –
sd rootdisk-B0 rootvol-01 ENABLED 1 0 – – Block0
pl rootvol-02 rootvol ENABLED 1050776 – ACTIVE – –
sd rootdisk-02 rootvol-01 ENABLED 1050775 1 – – –

v swapvol swap ENABLED 4198392 – ACTIVE – –
pl swapvol-01 swapvol ENABLED 4198392 – ACTIVE – –
sd rootdisk-01 swapvol-01 ENABLED 4198392 0 – – –
pl swapvol-02 swapvol ENABLED 4198392 – ACTIVE – –
sd rootdisk-mirror-03 swapvol-02 ENABLED 4198392 0 – – –

v usr gen ENABLED 4198392 – ACTIVE – –
pl usr-01 usr ENABLED 4198392 – ACTIVE – –
sd rootdisk-03 usr-01 ENABLED 4198392 0 – – –
pl usr-02 usr ENABLED 4198392 – ACTIVE – –
sd rootdisk-mirror-04 usr-02 ENABLED 4198392 0 – – –

v var gen ENABLED 4198392 – ACTIVE – –
pl var-01 var ENABLED 4198392 – ACTIVE – –
sd rootdisk-05 var-01 ENABLED 4198392 0 – – –
pl var-02 var ENABLED 4198392 – ACTIVE – –
sd rootdisk-mirror-05 var-02 ENABLED 4198392 0 – – –

VM disk rootdisk-mirror contains mirror plexes for volumes opt,rootvol, swapvol, usr, and var. We have to remove the plexes before proceeding with vxunroot.

# vxplex -o rm dis opt-02 rootvol-02 swapvol-02 usr-02 var-02

# vxprint opt rootvol swapvol usr var
Disk group: rootdg

TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0
v opt gen ENABLED 4198392 – ACTIVE – –
pl opt-01 opt ENABLED 4198392 – ACTIVE – –
sd rootdisk-04 opt-01 ENABLED 4198392 0 – – –

v rootvol root ENABLED 1050776 – ACTIVE – –
pl rootvol-01 rootvol ENABLED 1050776 – ACTIVE – –
sd rootdisk-B0 rootvol-01 ENABLED 1 0 – – Block0
sd rootdisk-02 rootvol-01 ENABLED 1050775 1 – – –

v swapvol swap ENABLED 4198392 – ACTIVE – –
pl swapvol-01 swapvol ENABLED 4198392 – ACTIVE – –
sd rootdisk-01 swapvol-01 ENABLED 4198392 0 – – –

v usr gen ENABLED 4198392 – ACTIVE – –
pl usr-01 usr ENABLED 4198392 – ACTIVE – –
sd rootdisk-03 usr-01 ENABLED 4198392 0 – – –

v var gen ENABLED 4198392 – ACTIVE – –
pl var-01 var ENABLED 4198392 – ACTIVE – –
sd rootdisk-05 var-01 ENABLED 4198392 0 – – –

# /etc/vx/bin/vxunroot

This operation will convert the following file systems from
volumes to regular partitions: root swap usr var opt home

Replace volume rootvol with c0t0d0s0.

This operation will require a system reboot. If you choose to
continue with this operation, system configuration will be updated
to discontinue use of the volume manager for your root and swap
devices.

Do you wish to do this now [y,n,q,?] (default: y)

After a reboot, the root disk will be unencapsulated.

Resizing a Veritas file system

Resizing a file system
In this example, I will resize a UFS file system under VxVM control from 3GB to 4GB using vxresize.

Current capacity:

# df -k /dbfiles03
Filesystem kbytes used avail capacity Mounted on
/dev/vx/dsk/dg20/dbvol03
3079710 2709166 308950 90% /dbfiles03

File system type:

# mount -v | grep /dbfiles03
/dev/vx/dsk/dg20/dbvol03 on /dbfiles03 type ufs read/write/setuid/intr/largefiles/onerror=panic/dev=3d1349e on Sun Aug 3 16:21:54 2003

Volume information:

# vxprint dbvol03
Disk group: dg20

TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0
v dbvol03 fsgen ENABLED 6291456 – ACTIVE – –
pl dbvol03-01 dbvol03 ENABLED 6298619 – ACTIVE – –
sd dg2007-03 dbvol03-01 ENABLED 3149307 0 – – –
sd dg2006-03 dbvol03-01 ENABLED 3149307 0 – – –

Plex information:

# vxprint -l dbvol03-01
Disk group: dg20

Plex: dbvol03-01
info: len=6298619 contiglen=6298491
type: layout=STRIPE columns=2 width=128
state: state=ACTIVE kernel=ENABLED io=read-write
assoc: vol=dbvol03 sd=dg2007-03,dg2006-03
flags: busy complete

Increasing the volume to 4GB using vxresize:

# vxresize dbvol03 4g
/dev/vx/rdsk/dg20/dbvol03: 8388608 sectors in 4096 cylinders of 32 tracks, 64 sectors
4096.0MB in 88 cyl groups (47 c/g, 47.00MB/g, 7872 i/g)
super-block backups (for fsck -F ufs -o b=#) at:
32, 96352, 192672, 288992, 385312, 481632, 577952, 674272, 770592, 866912,
963232, 1059552, 1155872, 1252192, 1348512, 1444832, 1541152, 1637472,
1733792, 1830112, 1926432, 2022752, 2119072, 2215392, 2311712, 2408032,
2504352, 2600672, 2696992, 2793312, 2889632, 2985952, 3080224, 3176544,
3272864, 3369184, 3465504, 3561824, 3658144, 3754464, 3850784, 3947104,
4043424, 4139744, 4236064, 4332384, 4428704, 4525024, 4621344, 4717664,
4813984, 4910304, 5006624, 5102944, 5199264, 5295584, 5391904, 5488224,
5584544, 5680864, 5777184, 5873504, 5969824, 6066144, 6160416, 6256736,
6353056, 6449376, 6545696, 6642016, 6738336, 6834656, 6930976, 7027296,
7123616, 7219936, 7316256, 7412576, 7508896, 7605216, 7701536, 7797856,
7894176, 7990496, 8086816, 8183136, 8279456, 8375776,

New capacity:

# df -k /dbfiles03
Filesystem kbytes used avail capacity Mounted on
/dev/vx/dsk/dg20/dbvol03
4106286 2709166 1335526 67% /dbfiles03

New volume information (two new subdisks):

# vxprint dbvol03
Disk group: dg20

TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0
v dbvol03 fsgen ENABLED 8388608 – ACTIVE – –
pl dbvol03-01 dbvol03 ENABLED 8395767 – ACTIVE – –
sd dg2007-03 dbvol03-01 ENABLED 3149307 0 – – –
sd dg2007-05 dbvol03-01 ENABLED 1048572 3149307 – – –
sd dg2006-03 dbvol03-01 ENABLED 3149307 0 – – –
sd dg2006-05 dbvol03-01 ENABLED 1048572 3149307 – – –

New plex information:

# vxprint -l dbvol03-01
Disk group: dg20

Plex: dbvol03-01
info: len=8395767 contiglen=8395639
type: layout=STRIPE columns=2 width=128
state: state=ACTIVE kernel=ENABLED io=read-write
assoc: vol=dbvol03 sd=dg2007-03,dg2007-05,dg2006-03,dg2006-05
flags: busy complete

How to determine the number of inodes in use in a vxfs filesystem.

To determine the number of inodes currently in use in a particular
VxFS (Veritas) filesystem, run ‘df -g’ on the filesystem in question,
and then subtract the reported number of "free files" from the
reported number of "total files".

For example:

# df -g /ck
/ck (/dev/vx/dsk/vol04): 8192 block size 2048 frag size
17678336 total blocks 17325188 free blocks 16783892 available 1095168 total
files
1082820 free files 37086307 filesys id
vxfs fstype 0x00000004 flag 255 filename length

The number of inodes in use is 12348 (1095168 – 1082820).

Veritas Q&A

This is a supplement to the Veritas documentation. That documentation is accurate as far as it goes, but there are some things missing. In particular, a big chunk of how to really do RAID5 seems to have been elided in favor of the vxassist method, but vxassist is not capable of really creating a RAID5 volume out of scattered pieces of subdisk.

*

Q: How do I replace a failed drive?

A: To begin with, remove the failed plexes from their mirrors. Hey wait! I’ve got RAID5, not mirrors! Sorry, I haven’t encountered that yet myself. Let me know what you did. Also, I suspect that if you really knew what you were doing, this step could be skipped, or might happen automatically when you do some of the below, but I don’t know that, so I’m reporting what worked for me. The command to dissociate a plex from a volume is vxplex dis plex-name. Repeat for each plex containing part of the failed drive.

First, use vxprint to see which disk you’ll be replacing. Note the disk media (dm) name and the disk group name. Then invoke vxdiskadm. Select item 4 (Remove a disk for replacement), and enter the disk name when prompted. If you’re like me, you don’t have any hot spares to replace the disk with, so if you’re prompted to name the replacement drive, respond none.

Do what you need to do to replace the physical drive at this point.

Next, get the new drive into the system: run vxdiskadd. Enter the disk group name when prompted. You’ll be prompted to use the default disk name; respond n. You’ll be prompted to make it a spare disk; respond n. When prompted, confirm your choices. If prompted to encapsulate, respond n. About now, the disk will be initialized. Afterward, you’re prompted for the disk media name: instead just quit; we’ve accomplished initialization of the drive.

Finally, invoke vxdiskadm. Select 5 (Replace a failed or removed disk). You’ll be prompted for the disk media name; choose appropriately; you’ll be prompted for the access name (c#t#d#s#); choose appropriately; confirm your choice; it says it succeeded; tell it you don’t want to do another; and you’re finished.

Did I say “Finally”? There’s the small matter of recovering the mirrored or RAID 5 volumes that the failed disk sent to the edge. As noted above, I had dissociated the plexes first. To put them back, for each plex do vxplex att volume-name plex-name.
*

Q: How do I create a RAID5 volume?

A: First, identify the subdisks that are going to participate. Count the number of physical disks that will be participating. That’s the number of columns you’ll have in the plex. Find a common denominator for the sizes of the subdisks (the factor program is your friend). Create the plex, like this:

vxmake plex raid5plex-a stwidth=8 ncolumn=12

where stwidth is one of those common denominators in the size of your subdisks, and ncolumn is, you guessed it, the number of columns in the plex. Put all the subdisks from an actual physical disks into the same column of the plex, being careful to specify the offsets for the subdisks in the columns so that you start at 0 and so that subdisks beyond the first in the column have an offset that neither overlaps the previous subdisk sectors nor leaves a hole. The command to add a subdisk to your plex goes like this:

vxsd -l 10/1927208 assoc raid5plex-a disk1-2

Where 10 is the column number (counting from 1), 1927208 is the offset in secotors from the beginning of the column (other subdisks having completely covered the preceding sectors) raid5plex-a is the name of our plex, and disk1-2 is the name of our subdisk. Your shortest plex column is going to be the multiplier by (the number of columns minus one) to give the amount of space in your RAID5 volume, minus a little overhead. Since the space in the longer columns is wasted; you might contemplate redefining your subdisks so that you can use that space for something else. Now you have to attach the plex to a volume; that’s done like so:

vxmake -Uraid5 vol raid5vol-a plex=raid5plex-a

where raid5vol-a is your volume name and raid5plex-a is your plex name. Things get weird at this point: the manual says that your next step is to start the volume with vxrecover -s raid5vol-a, and that actually worked for me — once. But then I decided to reclaim some of that wasted space I just mentioned, so I stopped the volume, vxedit -rf rm raid5vol-a, and rebuilt things. But this time the vxrecover didn’t appear to do anything. Instead, the following seems to have worked:

#root# vxvol init active raid5vol-a
#root# vxvol resync raid5vol-a

The business of defining subdisks and attaching them to plexes is tedious and error prone. My solution is to use some simple perl scripts, with which I process the result of a vxprint free. Here’s some typical output, names changed to protect the guilty:

DISK DEVICE TAG OFFSET LENGTH FLAGS
disk101 c1t1d0s2 c1t1d0 2106264 702088 –
disk101 c1t1d0s2 c1t1d0 27381432 4212528 –
disk101 c1t1d0s2 c1t1d0 32291336 4712 –
disk102 c1t2d0s2 c1t2d0 2106264 702088 –
disk102 c1t2d0s2 c1t2d0 27381432 4212528 –
disk102 c1t2d0s2 c1t2d0 32291336 4712 –

So to find how much space I’ve got on each disk, I might use this:

#!/usr/local/bin/perl
# input line:
# disk1 c2t4d0s2 c2t4d0 25275168 3510440 –
$j=-1;
while(<>) {
next unless /^disk1/; # don’t process the title line
@F=split;
if ($F[0] eq $prevdsk) {
$s += $prevsiz;
} else {
print "$prevdsk size is: ", $s, "\n" if $prevdsk;
$s=0;
$prevdsk=$F[0];
}
$prevsiz = $F[4];
}
print "$prevdsk size is: ", $s, "\n" if $prevdsk;

This is gross overkill on this data, but if you’ve got lots of disks of different sizes with irregular chunks available, it can be quite a time saver.

Next, we want to turn those into subdisks. So how about something like this:

#!/usr/local/bin/perl
# input line:
# disk1 c2t4d0s2 c2t4d0 25275168 3510440 –
while(<>) {
next unless /^disk1/; # skip the title line
@F=split;
if ($F[0] eq $prev) {
$i++;
} else {
$i=1;
$prev=$F[0];
}
print "vxmake sd ", $F[0], "-", $i, " ", $F[0], ",", $F[3], ",", $F[4], "\n";
}

which, when fed the vxprint free output you saved, generates text like this:

vxmake sd disk1-1 disk1 2106264 702088

Pipe that to /bin/sh and you’re finished making subdisks! But be careful! You don’t want to set this broom to fetching water without eyeballing what it’s about to do!

The other tedious piece is getting all those subdisks into the right place in the plex. So here’s a possible time saver:

#!/usr/local/bin/perl
# input line:
# disk1 c2t4d0s2 c2t4d0 25275168 3510440 –
$j=-1;
while(<>) {
next unless /^disk1/;
@F=split;
if ($F[0] eq $prevdsk) {
$i++;
$s += $prevsiz;
} else {
$j++;
$s=0;
$i=1;
$prevdsk=$F[0];
}
$prevsiz = $F[4];
print "vxsd -l ", $j ,"/", $s, " assoc raid5plex-a ", $F[0], "-", $i, "\n";
}

When fed with that vxprint free output you’d tucked away, that should produce lines like

vxsd -l 1/0 assoc raid5plex-a disk1-1
vxsd -l 1/702088 assoc raid5plex-a disk1-2

Again, check that broom before you fly away on it!

Veritas commands

Veritas (VxVM)
============================
-Volumes
-Plexes
-Disk Groups
-Disks
-Subdisks

# vxdisk list
# vxprint -ht
# vxdg
# vxconfigd (must be running)
# vxdctl (enable,disable)
# rootdg (group that root is in)

==========================
volumes:
clean
active
empty
sync
need sync
==========================
Plexes:
Empty
clean
active
state
offline
temp
temprm
temprmsd
iofail
==========================
raid 0+1
Volume
plex plex
subdisk subdisk
subdisk subdisk
subdisk subdisk

raid 1+0
volume
Plex
subvolume subvolume subvolume
subdisk subdisk subdisk
subdisk subdisk subdisk
subdisk subdisk subdisk

================================

troubleshooting commands
# vxdctl [volume] (start up volume)
# vxdg [group]
# vxdisk
# vxdiskadd (add one or more disks)
# vxdiskadm** (menu driven admin)
# vdisk [setup|unsetup]
# vxassist (creates new volumes)
# vxconfigd (daemon – controlled by vxdctl)
# vxmake (creats vol mgr config files)
# vxmend (mends config records)
# vxplex (mgr operations on plexes)
# vxprint (display configs and layout) vxprint -he
# vxvol (volume manager on volumes)25

** = used the most

VERITAS Plex states and fixes

Any except) IOFAIL (Any) vxmend fix empty EMPTY DISABLED
(Disassociated) N/A vxplex att EMPTY DISABLED
ACTIVE (Any) vxmend fix stale STALE ENABLED
ACTIVE (Any) vxmend fix stale STALE DISABLED
ACTIVE DISABLED vxvol start ACTIVE ENABLED
ACTIVE ENABLED (system crash) ACTIVE DISABLED
ACTIVE ENABLED vxvol stop CLEAN DISABLED
ACTIVE ENABLED unrecoverable I/O error IOFAIL DETACHED
ACTIVE ENABLED vxmend off OFFLINE DISABLED
ACTIVE ENABLED vxplex det STALE ENABLED
CLEAN DISABLED vxvol start ACTIVE ENABLED
CLEAN DISABLED vxmend fix stale STALE DISABLED
EMPTY DISABLED vxvol init active ACTIVE ENABLED
EMPTY DISABLED vxvol init clean CLEAN DISABLED
IOFAIL DETACHED vxplex att ACTIVE ENABLED
OFFLINE DISABLED vxplex att STALE DETACHED
OFFLINE DISABLED vxmend on STALE DETACHED
RECOVER DETACHED vxrecover ACTIVE ENABLED
STALE DETACHED vxplex att
vxvol start ACTIVE ENABLED
STALE DISABLED vxmend fix active ACTIVE (volume goes to sync) ENABLED
STALE DISABLED vxmend fix clean CLEAN ENABLED
STALE ENABLED vxvol start ACTIVE ENABLED
TEMP DETACHED vxvol start (DIASSOCIATED)