VxVM System Administrator's Guide

Disk and Disk Group Administration

Chapter 3

Introduction

This chapter describes the Volume Manager operations for managing disks used by the Volume Manager. It also provides information on disk group operations.

Note: Most Volume Manager commands require appropriate privileges.

The following topics are covered in this chapter:

Standard Disk Devices

There are two classes of disk devices that can be used with the Volume Manager: standard devices and special devices. Special devices are described later in this chapter.

The Volume Manager supports up to sixteen partitions (also called slices) on a physical disk. These partitions are named, in order, 0 through 15. Partition 0 is reserved to indicate the entire disk.

When a partition is placed under Volume Manager control, a VM disk is assigned to that partition. A symbolic name (the disk name or disk media name) such as disk01 can be established to refer to a VM disk.

A partition is addressed through a physical address (generally referred to as the device name) of the form c#b#t#d#s#, which is comprised of the following elements:

c# --The number of the controller to which the disk drive is attached.
b#--The corresponding bus.
t# and d# --The target ID and device number that constitute the address of the disk drive on the controller.
s# --The partition number on the disk drive.

An example of a device name is c0b0t0d0s7. By convention, s7 is used to refer to the standard partitioning method used by VxVM. The physical disk is identified to the Volume Manager as c#b#t#d#s#. For many commands, the suffix s# is normally optional, though display commands usually report devices with an s# suffix. The Volume Manager vxdiskadm and vxdiskadd utilities take device names without the s7 suffix. For example, to specify the second disk attached to the first controller to vxdiskadd, use the name c0b0t1d0.

The boot disk (which contains the root file system and is used when booting the system) is often identified to the Volume Manager by the device name c0b0t0d0.

A VM disk is typically composed of two regions:

private region -- a small area where configuration information is stored. A disk label and configuration records are stored here.
public region -- a larger area that covers the remainder of the disk and is used to store subdisks (and allocate storage space).

Three basic disk types exist with VxVM:

sliced -- The public and private regions are on different disk partitions.
simple -- The public and private regions are on the same disk partition (with the public area following the private area).
nopriv -- There is no private region (only a public region for allocating subdisks).

The Volume Manager initializes each new disk with the fewest number of partitions possible. For disk access names ending in s7, the default type is sliced.

Disk Groups

Disks are organized by the Volume Manager into disk groups. A disk group is a named collection of disks that share a common configuration. Volumes are created within a disk group and are restricted to using disks within that disk group.

A system with the Volume Manager installed will always have the default disk group, rootdg. By default, operations are directed to the rootdg disk group. The system administrator can create additional disk groups, as necessary. Many systems do not need to use more than one disk group, unless they have a large number of disks. Disks do not need to be added to disk groups until the disks are needed to create VxVM objects. They can be initialized and reserved to be added to disk groups later. However, at least one disk (partition) must be added to rootdg in order to perform the VxVM installation procedures.

When a disk is added to a disk group, it is given a name (such as disk02). This name can be used to identify a disk for volume operations, such as volume creation or mirroring. This name relates directly to the physical disk. If a physical disk is moved to a different target address or to a different controller, the name disk02 will continue to refer to it. Disks can be replaced by first associating a different physical disk with the name of the disk to be replaced and then recovering any volume data that was stored on the original disk (from mirrors or backup copies).

Having excessively large disk groups may cause the private region to fill. In the case of larger disk groups, disks should be set up with larger private areas to log in. A major portion of a disk's private region consists of space for a disk group configuration database containing configuration records for each VxVM object in that disk group. Since each configuration record takes up 256 bytes (or half a block), the number of configuration records that can be created in a disk group is twice the configuration database copy size (which can be obtained from the output of the command vxdg list diskgroupname).

Disk and Disk Group Utilities

The Volume Manager provides three interfaces that can be used to manage disks with the Volume Manager:

the Visual Administrator graphical user interface
a set of command-line utilities
the vxdiskadm menu-based interface

Utilities discussed in this chapter include:

vxdiskadm --This is the Volume Manager Support Operations menu interface. This utility provides a menu of disk operations. Each entry in the main menu leads you through a particular operation by providing you with information and asking questions. Default answers are provided for many questions so that common answers can be selected easily.

: This chapter describes uses for the vxdiskadm utility, but does not describe its use in detail (refer to the VERITAS Volume Manager User's Guide for information on how to use vxdiskadm).

vxdiskadd --This utility is used to add standard disks to the Volume Manager. vxdiskadd leads you through the process of initializing a new disk by displaying information and asking questions.
vxdisk --This is the command-line utility for administering disk devices. vxdisk is used to define special disk devices, to initialize information stored on disks that the Volume Manager uses to identify and manage disks, and to perform additional special operations. See the vxdisk(1M) manual page for complete information on how to use vxdisk.
vxdg --This is the command-line utility for operating on disk groups. This can be used to create new disk groups, to add and remove disks from disk groups, and to enable (import) or disable (deport) access to disk groups. See the vxdg(1M) manual page for complete information on how to use vxdg.

Note: The vxdiskadd utility and most vxdiskadm operations can be used only with standard disk devices.

Using Disks

This section describes some of the basic disk administration options that are available with the Volume Manager.

Initializing and Adding Disks

There are two levels of initialization for disks in the Volume Manager:

: 1. Formatting of the disk media itself. This must be done outside of the Volume Manager.
: 2. Storing identification and configuration information on the disk for use by the Volume Manager. Volume Manager interfaces are provided to step through this level of disk initialization.

A fully initialized disk can be added to a disk group, used to replace a previously failed disk, or used to create a new disk group. These topics are discussed later in this chapter.

Formatting the Disk Media

To perform the first initialization phase, use the interactive diskadd command to do a media format of any disk.

Note: SCSI disks are usually preformatted. The diskadd command typically is needed only if the format becomes severely damaged.

VxVM Disk Installation

To perform the disk initialization phase, use either the vxdiskadm menus or vxdiskadd. This section describes how to use vxdiskadd. For information on how to use vxdiskadm to initialize a single disk or all disks on a controller, refer to Chapter 13, "Menu Interface Operations," of the VERITAS Volume Manager User's Guide.

You can use vxdiskadd to initialize a specific disk. For example, to initialize the second disk on the first controller, use the command:

vxdiskadd c0b0t1d0

This will examine your disk to determine whether it has been initialized already and will ask questions based on what it finds. vxdiskadd will check for disks that can be encapsulated (described in "Using vxdisk for Special Encapsulations"), for disks that have already been added to the Volume Manager, and for a number of less-likely conditions.

Note: If you are adding an uninitialized disk, warning and error messages may be displayed on the console during vxdiskadd. Ignore these messages. These messages should not appear after the disk has been fully initialized; vxdiskadd displays a success message when the initialization completes.

At the following prompt, enter y (or press Return) to continue:

Add or initialize disks
Menu: VolumeManager/Disk/AddDisks

 Here is the disk selected.  Output format: [Device_Name]
 
  c0b0t1d0
 
Continue operation? [y,n,q,?] (default: y) y

If the disk is uninitialized, or if you choose to reinitialize the disk, you will be prompted with the following:

  You can choose to add this disk to an existing disk group, a
  new disk group, or leave the disk available for use by future
  add or replacement operations.  To create a new disk group,
  select a disk group name that does not yet exist.  To leave
  the disk available for future use, specify a disk group name
  of "none".

Which disk group [<group>,none,list,q,?] (default: rootdg)

To add this disk to the default group rootdg, press Return. To leave the disk free as a replacement disk (not yet added to any disk group), enter none. After this, you will be asked to select a name for the disk in the disk group:

Use a default disk name for the disk? [y,n,q,?] (default: y) y

Normally, you should accept the default disk name (unless you prefer to enter a special disk name).

At the following prompt, enter n to indicate that this disk should not be used as a hot-relocation spare:

Add disk as a spare disk for rootdg? [y,n,q,?] (default: n) n

When the following screen appears, press Return to continue with the operation:

  The selected disks will be added to the disk group rootdg with
  default disk names.
 
  c0b0t1d0
 
Continue with operation? [y,n,q,?] (default: y) y

If you are certain that there is no data on this disk that needs to be preserved, enter n at the following prompt:

	The following disk device has a valid VTOC, but does not appear to
	have been initialized for the Volume Manager.  If there is data
 	on the disk that should NOT be destroyed you should encapsulate
 	the existing disk partitions as volumes instead of adding the disk
 	as a new disk. Output format: [Device_Name]

	c0b0t1d0

	Encapsulate this device? [y,n,q,?] (default: y)

When vxdiskadm asks if you want to initialize the disk instead, enter y:

  Instead of encapsulating, initialize? [y,n,q,?] (default: n) y

Messages similar to the following should now confirm that disk c1b0t0d1 is being placed under Volume Manager control. You are also given the option of performing surface analysis.

  Initializing device c0b0t1d0.
 
  Perform surface analysis (highly recommended)
  [y,n,q,?] (default: y) n
 
  Adding disk device c0b0t1d0 to disk group rootdg with disk
  name disk33.

Removing Disks

A disk that contains no subdisks can be removed from its disk group with the command:

vxdg [-g groupname] rmdisk diskname

where the disk group name need only be specified for a disk group other than the default, rootdg.

For example, to remove disk02 from rootdg, use:

vxdg rmdisk disk02

If the disk has subdisks on it when you try to remove it, the following error message is displayed:

vxdg:Disk diskname is used by one or more subdisks

Use the -k option to vxdg to remove device assignment. Using the -k option allows you to remove the disk in spite of the presence of subdisks. For more information, see the vxdg(1M) manual page.

Once the disk has been removed from its disk group, you can (optionally) remove it from Volume Manager control completely, as follows:

vxdisk rm devicename

For example, to remove c1b0t0d0 from Volume Manager control, enter:

vxdisk rm c1b0t0d0s7

In some cases, you may want to remove a disk on which some subdisks are defined. For example, you may have three disks on one system and you may want to consolidate all of the volumes onto one disk. If you use vxdiskadm to remove a disk, you can choose to move volumes off that disk. To do this, run vxdiskadm and select item 3 (Remove a disk) from the main menu.

If the disk is used by some subdisks, then a screen resembling the following is displayed:

The following subdisks currently use part of disk disk02: 

     home usrvol 

Subdisks must be moved from disk02 before it can be removed. 

Move subdisks to other disks? [y,n,q,?] (default: n)

If you choose y, then all subdisks will be moved off the disk, if possible. Some subdisks may not be movable. The most common reasons why a subdisk may not be movable are:

There is not enough space on the remaining disks.
Plexes or striped subdisks cannot be allocated on different disks from existing plexes or striped subdisks in the volume.

If vxdiskadm cannot move some subdisks, you may need to remove some plexes from some disks to free more space before proceeding with the disk removal operation. See Chapter 4 for information on how to remove volumes and plexes.

Moving Disks

If you want to reorganize by moving a disk between disk groups, remove the disk from one disk group and add it to the other. For example, to move the physical disk c0b0t3d0 (attached with the disk name disk04) from disk group rootdg and add it to disk group mktdg, you could use the commands:

vxdg rmdisk disk04 
vxdg -g mktdg adddisk mktdg02=c0b0t3d0

Note: This procedure does not save the configurations or data on the disks.

This can also be done using vxdiskadm by selecting item 3 (Remove a disk) from the main menu, and then selecting item 1 (Add or initialize a disk).

Detecting and Replacing Failed Disks

This section discusses how to detect disk failures and replace failed disks. It begins with a discussion of the Volume Manager's hot-relocation feature, which automatically attempts to restore redundant VxVM objects when a failure occurs.

Hot-Relocation

The hot-relocation feature enables a system to automatically react to I/O failures on redundant (mirrored or RAID-5) VxVM objects and restore redundancy and access to those objects. The Volume Manager detects I/O failures on VxVM objects and relocates the affected subdisks to disks designated as spare disks and/or free space within the disk group. VxVM then reconstructs the VxVM objects that existed before the failure and makes them redundant and accessible again. Refer to Chapter 1, "Description of the Volume Manager," for a more complete description of hot-relocation.

Note: Hot-relocation is only performed for redundant (mirrored or RAID-5) subdisks on a failed disk. Non-redundant subdisks on a failed disk are not relocated, but the system administrator is notified of their failure.

Hot-relocation is enabled by default and goes into effect without system administrator intervention when a failure occurs. The hot-relocation daemon, vxrelocd, detects and reacts to Volume Manager events that signify the following types of failures:

disk failure -- this is normally detected as a result of an I/O failure from a VxVM object. VxVM attempts to correct the error. If the error cannot be corrected, VxVM tries to access configuration information in the disk's private region. If it cannot access the disk's private region, it considers the disk failed.
plex failure -- this is normally detected as a result of an uncorrectable I/O error in the plex (which affects subdisks within the plex). For mirrored volumes, the plex is detached.
RAID-5 subdisk failure -- this is normally detected as a result of an uncorrectable I/O error. The subdisk is detached.

When such a failure is detected, vxrelocd informs the system administrator of the failure and which VxVM objects are affected (via electronic mail). vxrelocd then determines which subdisks (if any) can be relocated. If relocation is possible, vxrelocd finds suitable relocation space and relocates the subdisks. Hot-relocation space is chosen from the disks reserved for hot-relocation in the disk group where the failure occurred; if no spare disks are available or additional space is needed, free space in the same disk group is used. Once the subdisks are relocated, each relocated subdisk is reattached to its plex. Finally, vxrelocd initiates any appropriate recovery procedures (such as mirror resynchronization for mirrored volumes or data recovery for RAID-5 volumes). The system administrator is notified of the hot-relocation and recovery actions taken.

If relocation is not possible, the system administrator is notified and no further action is taken. Relocation is not possible in the following cases:

If subdisks are not redundant (i.e., they do not belong to mirrored or
RAID-5 volumes), they cannot be relocated.
If there is not enough space available (from spare disks and free space) in the disk group, failing subdisks cannot be relocated.
If the only available space is on a disk that already contains a mirror of the failing plex, the subdisks in that plex cannot be relocated.
If the only available space is on a disk that already contains the RAID-5 plex's log plex or one of its healthy subdisks, the failing subdisk in the RAID-5 plex cannot be relocated.
If a mirrored volume has a Dirty Region Logging log subdisk as part of its data plex, subdisks belonging to that plex cannot be relocated.
If a RAID-5 volume's log plex or a mirrored volume's DRL log plex fails, a new log plex is created elsewhere (so the log plex is not actually relocated).

It is a good idea to prepare for hot-relocation by designating one or more disks per disk group as hot-relocation spares. For information on how to designate a disk as a spare, see the VERITAS Volume Manager User's Guide. If no spares are available at the time of a failure or if there is not enough space on the spares, free space will automatically be used. By designating spare disks, you have some control over which space is used for relocation purposes in the event of a failure. If the combined free space and space on spare disks is not sufficient or does not meet the redundancy constraints, the subdisks are not relocated.

After a successful relocation occurs, you may need to remove and replace the failed disk (see "Replacing Disks"). Depending on the locations of the relocated subdisks, you may choose to move the relocated subdisks elsewhere after hot-relocation occurs (see "Moving Relocated Subdisks").

Modifying `vxrelocd`

As long as vxrelocd is running, hot-relocation is turned on. It is advisable to leave hot-relocation turned on so that you can take advantage of this feature if a failure occurs. However, if you choose to disable this feature for some reason (possibly because you do not want the free space on some of your disks used for relocation), you can do so by preventing vxrelocd from starting at system startup time. You can stop hot-relocation at any time by killing the vxrelocd process (this should not be done while a hot-relocation attempt is in progress).

You can make some minor changes to the way vxrelocd behaves by either editing the vxrelocd line in the startup file that invokes vxrelocd (/etc/rc2.d/S95vxvm-recover) or killing the existing vxrelocd process and restarting it with different options. After making changes to the way vxrelocd is invoked in the startup file, you need to reboot the system so that the changes will go into effect. If you choose to kill and restart the daemon instead, make sure that hot-relocation is not in progress when you kill the vxrelocd process. You should also restart the daemon immediately so that hot-relocation can take effect if a failure occurs.

You can alter vxrelocd's behavior in the following ways:

By default, vxrelocd sends electronic mail to root when failures are detected and relocation actions are performed. You can instruct vxrelocd to notify additional users by adding the appropriate user names and invoking vxrelocd as follows:

	vxrelocd root user_name1 user_name2 &

To reduce the impact of recovery on system performance, you can instruct vxrelocd to increase the delay between the recovery of each region of the volume, as follows:

	vxrelocd -o slow[=IOdelay] root &

: where the optional IOdelay indicates the desired delay (in milliseconds). The default value for the delay is 250 milliseconds.

Displaying Spare Disk Information

The command vxdg spare can be used to display information about all of the spare disks available for relocation. The output of this command lists the following type of information:

GROUP        DISK         DEVICE       TAG         OFFSET    LENGTH    FLAGS
rootdg       disk02       c0b0t2d0s2   c0b0t2d0    0         658007    s

In this case, disk02 is the only disk designated as a spare. The LENGTH field indicates how much spare space is currently available on this disk for relocation.

The following commands can also be used to display information about disks that are currently designated as spares:

vxdisk list -- lists disk information and displays spare disks with a spare flag.
vxprint -- lists disk and other information and displays spare disks with a SPARE flag.

Detecting Failed Disks

Note: VxVM's hot-relocation feature automatically detects disk failures and notifies the system administrator of the failures via electronic mail.
If hot-relocation is disabled or you miss the electronic mail, you may notice disk failures through the output of the vxprint command. or by using the Visual Administrator to look at the status of the disks. You may also see driver error messages on the console or in the system messages file.

If a volume encounters a disk I/O failure (for example, because the disk has an uncorrectable error), the Volume Manager may detach the plex involved in the failure. If a plex is detached, I/O stops on that plex but continues on the remaining plexes of the volume. If a disk fails completely, the Volume Manager may detach the disk from its disk group. If a disk is detached, all plexes on the disk are disabled. If there are any unmirrored volumes on a disk when it is detached, those volumes are disabled as well.

Partial Disk Failure

If hot-relocation is enabled when a plex or disk is detached by a failure, mail indicating the failed objects is sent to root. If a partial disk failure occurs, the mail should identify the failed plexes. For example, if a disk containing mirrored volumes experiences a failure, you might receive a mail message containing information such as:

To: root 
Subject: Volume Manager failures on host teal

Failures have been detected by the VERITAS Volume Manager: 

failed plexes: 
  home-02 
  src-02

See "Modifying vxrelocd," for information on how to have the mail sent to users other than root.

You can determine which disk is causing the failures in the above message by running the command:

vxstat -s -ff home-02 src-02

This produces output such as:

                            FAILED 
TYP NAME                READS   WRITES 
sd disk01-04                0        0 
sd disk01-06                0        0 
sd disk02-03                1        0 
sd disk02-04                1        0

This display indicates that the failures are on disk02 (and that subdisks disk02-03 and disk02-04 are affected).

Hot-relocation should automatically relocate the affected subdisks and initiate any necessary recovery procedures. However, if relocation is not possible or the hot-relocation feature is disabled, you may have to investigate the problem and attempt to recover the plexes. Sometimes these errors are caused by cabling failures, so you should check the cables connecting your disks to your system. If there are any obvious problems, correct them and recover the plexes with the command:

vxrecover -b home src

This command will start a recovery of the failed plexes in the background (the command will return before the operation is done). If an error message appears later, or if the plexes become detached again and there are no obvious cabling failures, the disk should be replaced (see "Replacing Disks").

Complete Disk Failure

If a disk fails completely and hot-relocation is enabled, the mail message will list the disk that has failed and all plexes that use the disk. For example, you may receive mail containing information similar to this:

To: root 
Subject: Volume Manager failures on host teal

Failures have been detected by the VERITAS Volume Manager: 

failed disks: 
  disk02 

failed plexes: 
  home-02 
  src-02 
  mkting-01 

failing disks: 
  disk02

This message indicates that disk02 was detached by a failure. When a disk is detached, I/O cannot get to that disk. The plexes home-02, src-02, and mkting-01 were also detached (probably because of the failure of the disk).

Again, the problem may be a cabling error. If the problem is not a cabling error, the disk should be replaced (see "Replacing Disks").

Replacing Disks

Disks that have failed completely (that have been detached by failure) can be replaced by running vxdiskadm and selecting item 5 (Replace a failed or removed disk) from the main menu. If there are any initialized but unadded disks, you will be able to select one of those disks as a replacement.

Note: Do not choose the old disk drive as a replacement even though it may appear in the selection list. If there are no suitable initialized disks, you can choose to initialize a new disk.

If a disk failure caused a volume to be disabled, the volume must be restored from backup after the disk is replaced. To identify volumes that wholly reside on disks that were disabled by a disk failure, use the command:

vxinfo

Any volumes that are listed as Unstartable must be restored from backup. For example, vxinfo might display:

home         fsgen     Started 
mkting       fsgen     Unstartable 
src          fsgen     Started 
standvol     gen       Started 
rootvol      root      Started 
swapvol      swap      Started

To restart volume mkting so that it can be restored from backup, use the command:

vxvol -o bg -f start mkting

The -o bg option combination causes any plexes to be resynchronized as a background task.

If failures are starting to occur on a disk, but the disk has not yet failed completely, you should replace the disk. This involves two steps:

: 1. Detaching the disk from its disk group.
: 2. Replacing the disk with a new one.

To detach the disk, run vxdiskadm and select item 4 (Remove a disk for replacement) from the main menu. If there are initialized disks available as replacements, you can specify the disk as part of this operation. Otherwise, you must specify the replacement disk later by selecting item 5 (Replace a failed or removed disk) from the main menu.

When you select a disk to remove for replacement, all volumes that will be affected by the operation are displayed. For example, the following output might be displayed:

The following volumes will lose mirrors as a result of this operation:

     home src

No data on these volumes will be lost.

The following volumes are in use, and will be disabled as a result of this operation:

     mkting

Any applications using these volumes will fail future accesses. These volumes will require restoration from backup.

Are you sure you want do this? [y,n,q,?] (default: n)

If any volumes are likely to be disabled, you should quit from vxdiskadm and save the volume. Either back up the volume or move the volume off of the disk. To move the volume mkting to a disk other than disk02, use the command:

vxassist move mkting !disk02

After the volume is backed up or moved, run vxdiskadm again and continue to remove the disk for replacement.

After the disk has been removed for replacement, a replacement disk can be specified by selecting item 5 (Replace a failed or removed disk) from vxdiskadm's main menu.

Moving Relocated Subdisks

This section describes how to move subdisks after hot-relocation occurs. When hot-relocation occurs, subdisks are relocated to spare disks and/or available free space within the disk group. The new subdisk locations may not provide the same performance or data layout that existed before hot-relocation took place. You may therefore wish to move the relocated subdisks (after hot-relocation is complete) to improve performance. Alternatively, you may want to move the relocated subdisks off the spare disk(s) to keep the spare disk space free for future hot-relocation needs. Another reason for moving subdisks is to recreate the configuration that existed before hot-relocation occurred.

During hot-relocation, one of the electronic mail messages sent to root should be similar to the following:

To: root
Subject: Volume Manager failures on host teal

Attempting to relocate subdisk disk02-03 from plex home-02.
Dev_offset 0 length 1164 dm_name disk02 da_name c0b0t5d0s2.
The available plex home-01 will be used to recover the data.

This message provides information about the characteristics of the subdisk before relocation and can be used to decide where to move the subdisk after relocation. In addition, a message similar to the following should indicate the new location for the relocated subdisk:

To: root
Subject: Attempting VxVM relocation on host teal

Volume home Subdisk disk02-03 relocated to disk05-01,
but not yet recovered.

Before moving any relocated subdisks, you should fix or replace the disk that experienced the failure (as described in previous sections). Once this is done, you can move a relocated subdisk back to the original disk. For example, you can move the relocated subdisk disk05-01 back to disk02 as follows:

vxassist -g rootdg move home !disk05 disk02

Note: During subdisk move operations, RAID-5 volumes are not redundant.

Working with Disk Groups

This section describes some of the disk group administrative options available with the Volume Manager.

Creating a Disk Group

A new disk group can be created on a physical disk using vxdiskadd. If using vxdiskadd, specify a new disk group name when prompted for a disk group. Disk groups can also be created using the operation vxdg init. To create a disk group with the vxdg utility, use the command:

vxdg init diskgroup diskname=devicename

For example, to create a disk group named mktdg on device c1b0t0d0s7, use the command:

vxdg init mktdg mktdg01=c1b0t0d0

The disk device given to vxdg must have been initialized already with vxdiskadd. The disk must not already be added to a disk group.

Using Disk Groups

Most Volume Manager commands allow a disk group to be specified using the -g option. For example, to create a volume in disk group mktdg, you could use the command:

vxassist -g mktdg make mktvol 50m

The (block) volume device for this volume is:

	/dev/vx/dsk/mktdg/mktvol

In many cases, the disk group does not have to be specified. Most Volume Manager commands use object names specified on the command line to determine the disk group for the operation. For example, a volume can be created on disk mktdg01 without specifying the disk group name:

vxassist make mktvol 50m mktdg01

This works for many commands as long as two disk groups do not have objects with the same name. For example, the Volume Manager allows you to create volumes named mktvol in both rootdg and in mktdg. If you do this, you must add -g mktdg to any command where you want to manipulate the volume in the mktdg disk group.

Removing a Disk Group

To remove a disk group, unmount and stop any volumes in the disk group and then run the command:

vxdg deport diskgroup

Deporting a disk group does not actually remove the disk group. It disables use of the disk group by the system. However, disks that are in a deported disk group can be reused, reinitialized, or added to other disk groups.

Moving Disk Groups Between Systems

An important feature of disk groups is that they can be moved between systems. If all disks in a disk group are moved from one system to another, then the disk group can be used by the second system without having to specify the configuration again.

The following steps are used to move a disk group between systems:

: 1. On the first system, stop all volumes in the disk group, then deport (disable local access to) the disk group with the command:

	vxdg deport diskgroup

: 2. Move all the disks to the second system and import (enable local access to) the disk group on the second system with the command:

	vxdg import diskgroup

3. Perform the necessary steps (system-dependent) to make the second system and VxVM recognize the new disks.

This may require a reboot, in which case the vxconfigd daemon will be restarted and will also recognize the new disks. If you do not reboot, use the command vxdctl enable to restart vxconfigd so that VxVM also recognizes the disks.

4. After the disk group is imported, start all volumes in the disk group with the command:

   vxrecover -g diskgroup -sb

You may want to move disks from a system that has crashed. In this case, you will not be able to deport the disk group from the first system. When a disk group is created or imported on a system, that system writes a lock on all disks in the disk group.

Note: The purpose of the lock is to ensure that dual-ported disks (disks that can be accessed simultaneously by two systems) will not be used by both systems at the same time. If two systems try to manage the same disks at the same time, configuration information stored on the disk will be corrupted and the disk and its data will become unusable.

If you move disks from a system that has crashed or failed to detect the group before the disk is moved, the locks stored on the disks will remain and must be cleared. The system returns the following error message:

vxdg:disk group groupname: import failed: Disk is in use by another host

To clear locks on a specific set of devices, use the command:

vxdisk clearimport devicename ...

It is possible to clear the locks during import by using the command:

vxdg -C import diskgroup

Note: Be careful when using the vxdisk clearimport or vxdg -C import command on systems that really do have dual-ported disks, as clearing the locks allows those disks to be accessed simultaneously and could result in corrupted data.

In some cases, you may want to import a disk group when some disks are not available. The import operation normally fails if some disks for the disk group cannot be found among the disk drives attached to the system. If the import operation fails, one of the following error messages will display:

vxdg: Disk group groupname: import failed: Disk group has no valid configuration copies

This message indicates a fatal situation, which requires hardware repair or the creation of a new disk group.

vxdg: Disk group groupname: import failed: Disk for disk group not found

This message indicates a recoverable situation.

If some of the disks in the disk group have failed, you can force the disk group to be imported with the command:

vxdg -f import diskgroup

Note: Be careful when using the -f option here, as it can cause the same disk group to be imported twice from disjoint sets of disks, causing the disk group to become inconsistent.

These operations can be performed using vxdiskadm. To deport a disk group via vxdiskadm, select menu item 9 (Remove access to (deport) a disk group). To import a disk group, select item 8 (Enable access to (import) a disk group). The vxdiskadm import operation checks for host import locks and asks if you want to clear any that are found. It also starts volumes in the disk group.

Renaming Disk Groups

Only one disk group of a given name can exist per system. It is therefore not possible to import or deport a disk group when the target system already has a disk group of the same name. To circumvent this problem, the Volume Manager allows you to temporarily or permanently rename a disk group during import or deport.

For instance, since every system running the Volume Manager must have a single rootdg default disk group, importing or deporting rootdg across systems presents a problem in that there cannot be two rootdg disk groups on the same system. This problem can be overcome by renaming the rootdg disk group during the import or deport.

To give a new name to the disk group during import, use the following command:

vxdg [-t] -n newdg_name import diskgroup

If the -t option is included, the import is temporary and will not persist across reboots. In this case, the stored name of the disk group remains unchanged on its original host, but the disk group will be known as newdg_name to the importing host. If the -t option is not used, the name change will be permanent.

As with importing, a disk group can be renamed on deport with the following command:

vxdg [-h hostname] -n newdg_name deport diskgroup

When renaming on deport, the -h hostname option can be specified to assign a lock to an alternate host. This ensures that the disk group is automatically imported when the alternate host reboots.

The following steps can be used to temporarily move the rootdg disk group from one host to another (for repair work on the root volume, for instance) and then move it back:

: 1. On the original host, identify the disk group ID of the rootdg disk group to be imported to the other host:

	vxdisk -s list

: This command results in output that includes disk group information similar to the following.

	dgname: rootdg
	dgid:   774226267.1025.tweety

: 2. On the importing host, import and rename the rootdg disk group as follows:

	vxdg -tC -n newdg_name import diskgroup

: where -t indicates a temporary import name; -C clears import locks;
-n specifies a temporary name for the rootdg to be imported (so that it does not conflict with the existing rootdg); and diskgroup is the disk group ID of the disk group being imported (774226267.1025.tweety, for example).
: If a reboot or crash occurs at this point, the temporarily-imported disk group will become unimported and will require a reimport.
: 3. After the necessary work has been done on the imported rootdg, deport it back to its original host as follows:

	vxdg -h hostname deport diskgroup

: where hostname is the name of the system whose rootdg is being returned (the system's name can be confirmed with the command uname -n).
: This command removes the imported rootdg from the importing host and returns locks to its original host. The original host will then autoimport its rootdg on its next reboot.

Reserving Minor Numbers for Disk Groups

Since disk groups can be moved between systems, it is desirable that volume device numbers be allocated in separate ranges for each disk group so that all disk groups in a group of machines can be moved around without causing device number collisions.

VxVM allows the administrator to select a range of minor numbers for a specified disk group. This range is used during the creation of a volume, guaranteeing that each volume will have the same minor number across reboots or reconfigurations. If two disk groups have overlapping ranges, an import collision will be detected and some avoidance or renumbering mechanism will then be needed.

A base volume device minor number can be set for a disk group with the command:

vxdg init diskgroup minor=base_minor devicename

Volume device numbers for a disk group will be chosen to have minor numbers starting at this base_minor number. Minor numbers can range up to 131071. A reasonably sized range should be left at the end for temporary device number remappings (in the event that two device numbers still conflict).

If no minor operand is specified on the vxdg init command line, the Volume Manager chooses a random number of at least 1000 that is a multiple of 1000, and yields a usable range of 1000 device numbers. This default number is chosen such that it does not overlap within a range of 1000 of any currently imported disk groups, and does not overlap any currently allocated volume device numbers.

Note: The default policy is likely to ensure that a small number of disk groups can be merged successfully between a set of machines. However, in cases where disk groups will be merged automatically using fail-over mechanisms, the administrator should select ranges that are known to avoid overlap.

For further information on minor number reservation, refer to the vxdg(1M) manual page.

Using Special Devices

This section discusses special devices used by the Volume Manager to perform administrative tasks.

Using `vxdisk` for Special Encapsulations

Encapsulation is a process that converts existing partitions on a specified disk to volumes. If any partitions contain file systems, /etc/vfstab entries are modified so that the file systems are mounted on volumes instead.

The normal disk encapsulation procedure requires that some free space be available on the disk for storing Volume Manager identification and configuration information. This free space cannot be included in any other partitions. (See the Online Data Manager overview and installation topic and the vxencap(1M) manual page for further information.)

In some cases, you may want to encapsulate a disk that does not have any space that can be used for the Volume Manager private region partition. The vxdisk utility can be used to encapsulate disks that do not have available space. This is done using special types of disk devices, called nopriv devices, that do not have private regions. To use this, create a partition on the disk device that maps all parts of the disk that you want to access, then add the partition device for that partition with the command:

vxdisk define partition-device type=nopriv

Here, partition-device is the basename of the device in the /dev/dsk directory. For example, to use partition 3 of disk device c0b0t4d0, use the command:

vxdisk define c0b0t4d0s3 type=nopriv

To create volumes for other partitions on the disk drive, add the device to a disk group, determine where those partitions reside within the encapsulation partition, then use vxassist to create a volume with that offset and length.

A major drawback with using these special encapsulation partition devices is that the Volume Manager cannot track changes in the address or controller of the disk. Normally, the Volume Manager uses identifying information stored in the private region on the physical disk to track changes in the location of a physical disk. Since nopriv devices do not have private regions and thus have no identifying information stored on the physical disk, this cannot occur.

The best use of special encapsulation partition devices is to encapsulate a disk so that the Volume Manager can be used to move space off the disk. When space is made available on the disk, the special partition device can be removed and the disk can then be encapsulated as a standard disk device.

A disk group cannot be formed entirely from nopriv devices. This is because nopriv devices do not provide space for storing disk group configuration information. Configuration information must be stored on at least one disk in the disk group.

Using `vxdisk` for RAM Disks

Note: This section only applies to systems with RAM disks.

Some systems support creation of RAM disks. A RAM disk is a device made from system RAM that looks like a small, simple, disk device. Often, the contents of a RAM disk are erased when the system is rebooted. RAM disks that are erased on reboot defeat the Volume Manager's means of identifying physical disks. This is because information stored on the physical disks is used to identify the disk.

nopriv devices have a special feature to support RAM disks: a volatile option which indicates to the Volume Manager that the device contents do not survive reboots. Volatile devices are handled specially on system startup. If a volume is mirrored, plexes made from volatile devices are always recovered by copying data from non-volatile plexes.

To use a RAM disk, a device node must be created for the disk in the /dev/dsk and /dev/rdsk directories (for example, /dev/dsk/ramd0 and /dev/rdsk/ramd0). To define the RAM disk device to the Volume Manager, use the command:

vxdisk define ramd0 type=nopriv volatile

Normally, the Volume Manager will not start volumes that are formed entirely from plexes with volatile subdisks. This is because there is no plex that is guaranteed to contain the most recent volume contents.

Sometimes, RAM disks are used in situations where all volume contents are recreated after reboot. In these situations, you can force volumes formed from RAM disks to be started at reboot using the command:

vxvol set startopts=norecov volume_name

This option can be used only with gen-type volumes. See vxvol(1M) for more information on the vxvol set operation and the norecov option.

[Next] [Previous] [Top] [Contents] [Index]