Remote Infrastructure Management: February 2012

Tuesday, February 14, 2012

Adding and Removing HBA in AIX

Background Information and Planning:

· Please note that these procedures are used to "hot-swap" an HBA adapter in IBM pSeries servers.

· Always plan for downtime in case the "hot-swap" fails and a reboot is required.

· HBA (Fibre Channel Adapter) name, Physical Location Code, AIX Location Code.

· PowerPath HBA number (from powermt display output).

· Example HBA: 'fcs3' is the parent of the HW Path 'fscsi3' you see in 'powermt display' output. Typical examples are fcs1, fcs2, fcs3, fcs4.

Find Physical Location Code and AIX Location Code of the FC Adapter:

# lscfg -vl fcs3

fcs3 U1.9-P2-I6/Q1 FC Adapter

Part Number.................00P4295

EC Level....................A

Serial Number...............1F41809326

Manufacturer................001F

Feature Code/Marketing ID...5704 Code printed on Card

FRU Number.............. 00P4297 Field Replaceable Unit

Device Specific.(ZM)........3

Network Address.............10000000C93F9EAB

ROS Level and ID............02E01871

Device Specific.(Z0)........2003806D

Device Specific.(Z1)........00000000

Device Specific.(Z2)........00000000

Device Specific.(Z3)........03000909

Device Specific.(Z4)........FF601231

Device Specific.(Z5)........02E01871

Device Specific.(Z6)........06631871

Device Specific.(Z7)........07631871

Device Specific.(Z8)........20000000C93F9EAB WWN

Device Specific.(Z9).....HS1.81X1 Adapter Microcode Version

Device Specific.(ZA)........H1D1.81X1

Device Specific.(ZB)........H2D1.81X1

Device Specific.(YL)..U1.9-P2-I6/Q1 Physical Location Code

# lsdev -H -l fcs3

name status location

fcs3 Available 3Y-08 AIX Location Code

Find HBA number for HW Path. In our example, the HBA # is "3" for fscsi3:

# powermt display

Symmetrix logical device count=16

CLARiiON logical device count=14

==============================================================================

----- Host Bus Adapters --------- ------ I/O Paths ----- ------ Stats ------

HBA Number

### HW Path Summary Total Dead IO/Sec Q-IOs Errors

==============================================================================

0 fscsi0 optimal 16 0 - 0 0

1 fscsi1 optimal 28 0 - 0 0

2 fscsi2 optimal 16 0 - 0 0

3 fscsi3 optimal 28 0 - 0 0

Find children assigned to the Parent which correlates to the Power Path HW Path in previous output:

# lsdev -p fcs3

fcnet3 Defined 3Y-08-02 Fibre Channel Network Protocol Device

fscsi3 Available 3Y-08-01 FC SCSI I/O Controller Protocol Device

HBA Replacement Procedure Begins Here

1. Remove HBA from PowerPath

powermt remove hba=[X]

Example: powermt remove hba=3

Special Note: If the HBA is in an intermittent failing condition this command may take a very long time or hang. This may require that the cable to the HBA be disconnected. If that fails, then the system may need to be shut down to replace the HBA.

2. Perform AIX Hot-Plug Tasks

WARNING: DO NOT RUN THESE SMITTY COMMANDS FROM THE CAMCS CONSOLE

# smitty devdrpci

PCI Hot Plug Manager

Move cursor to desired item and press Enter.

List PCI Hot Plug Slots

Add a PCI Hot Plug Adapter

Replace/Remove a PCI Hot Plug Adapter

Identify a PCI Hot Plug Slot

Unconfigure a Device

Configure a Defined Device

Install/Configure Devices Added After IPL

F1=Help F2=Refresh F3=Cancel Esc+8=Image

Esc+9=Shell Esc+0=Exit Enter=Do

· Select 'Unconfigure a Device' <enter>

Unconfigure a Device

Type or select values in entry fields.

Press Enter AFTER making all desired changes.

[Entry Fields]

* Device Name [fcs3]

Unconfigure any Child Devices yes

KEEP definition in database no

· Type either [F4] or [Esc 4] to bring up a menu with a list of devices. In our example it will be 'fcs3'. Use the Down and Up arrows to scroll to 'fcs3', then press <enter>.

· Then, arrow down to 'Unconfigure any Child Devices', press [F4] or [Esc 4] and select 'yes'.

· Arrow down to 'KEEP definition in database' and set to 'no'.

· Now, press <enter> to execute the command and screen will look similar to below:

Command: OK stdout: yes stderr: no

Before command completion, additional instructions may appear below.

A list of "hdisk###" devices will show Deleted

fscsi3 Deleted

fcnet3 Deleted

fcs3 Deleted

· Press [Esc 3] or [F3] to return to the PCI Hot-Plug Manager Menu.

· Select 'Replace/Remove a PCI Hot Plug Adapter'.

· Arrow down and select the Device to Remove/Replace. In our example it is 'fcs3'.

Replace/Remove a PCI Hot Plug Adapter

Move cursor to desired item and press Enter. Use arrow keys to scroll.

# Slot Description Device(s)

U1.9-P1-I1 PCI-X capable, 64 bit, 133MHz slot fcs0

U1.9-P1-I2 PCI-X capable, 64 bit, 133MHz slot fcs1

U1.9-P1-I3 PCI-X capable, 64 bit, 133MHz slot pci16 ent0 ent1 ent2 e

U1.9-P2-I5 PCI-X capable, 64 bit, 133MHz slot fcs2

U1.9-P2-I6 PCI-X capable, 64 bit, 133MHz slot fcs3

U1.9-P2-I8 PCI-X capable, 64 bit, 133MHz slot ent4

U1.9-P2-I9 PCI-X capable, 64 bit, 133MHz slot pci22 ent5 ent6 ent7 e

F1=Help F2=Refresh F3=Cancel

Esc+8=Image Esc+0=Exit Enter=Do

/=Find n=Find Next

· Press <enter> to execute the command and the next screen appears.

· Change the [Entry Fields] column to 'remove' from 'replace'.

Replace/Remove a PCI Hot Plug Adapter

Type or select a value for the entry field.

Press Enter AFTER making all desired changes.

[Entry Fields]

Operation remove

· Press <enter> to execute the command and the next screen appears. Tell Fujitsu the card is ready to be changed and a light should flash on the card. Once the card is removed, press <enter>.

Command: running stdout: yes stderr: no

Before command completion, additional instructions may appear below.

The visual indicator for the specified PCI slot has

been set to the identify state. Press Enter to continue

or enter x to exit.

The visual indicator for the specified PCI slot has

been set to the action state. Remove the PCI card

from the identified slot and press Enter to continue.

· The command will finish executing and will indicate the removal is successful.

Command: OK stdout: yes stderr: no

Before command completion, additional instructions may appear below.

The visual indicator for the specified PCI slot has

been set to the identify state. Press Enter to continue

or enter x to exit.

The visual indicator for the specified PCI slot has been set

to the action state. Remove the PCI card from the identified

slot and press Enter to continue.

Remove Operation Complete.

· Press [Esc 3] or [F3] to return to the PCI Hot-Plug Manager Menu.

· Select 'List PC Hot-Plug Slots' to verify slot is empty.

· Press [Esc 3] or [F3] to return to the PCI Hot-Plug Manager Menu.

· Select 'Add a PCI Hot-Plug Adapter'. The display screen indicates that the PCI slot is empty.

· On the display screen, select the indicated empty slot. Fujitsu should look at the adapter you intend to remove. A slow blinking amber LED located next to the adapter on the PCI riser card indicates that the slot has been identified. Press Enter. The adapter has been put into the action state.

· Fujitsu will disconnect any cables that are connected to the adapter being removed.

· Completely remove the hot-plug adapter from the slot. The LED goes off.

· Continue to follow the screen instructions until you receive a message that the replacement is successful. Successful replacement is indicated by the OK message displayed next to the Command field at the top of the screen.

Command: running stdout: yes stderr: no

Before command completion, additional instructions may appear below.

The visual indicator for the specified PCI slot has

been set to the identify state. Press Enter to continue

or enter x to exit.

The visual indicator for the specified PCI slot has

been set to the action state. Insert the PCI card

into the identified slot, connect any devices to be

configured and press Enter to continue. Enter x to exit.

Command: OK stdout: yes stderr: no

Before command completion, additional instructions may appear below.

The visual indicator for the specified PCI slot has

been set to the identify state. Press Enter to continue

or enter x to exit.

The visual indicator for the specified PCI slot has

been set to the action state. Insert the PCI card

into the identified slot, connect any devices to be

configured and press Enter to continue. Enter x to exit.

Add Operation Complete.

Press [Esc 0] to exit the Hot-Plug Manager.

3. Run AIX cfgmgr to detect the new HBA (fcs3 in our example)

# cfgmgr –l fcs3

After 'cfgmgr' finishes, verify that AIX shows the adapter with an 'Available' status:

# lsdev -H -l fcs3

name status location description

fcs3 Available 3Y-08 FC Adapter

4. Verify the version of the microcode installed on the adapter

# lscfg -vl fcs3

fcs3 U1.9-P2-I6/Q1 FC Adapter

Part Number.................00P4295

ROS Level and ID............02E01871

... other lines omitted for brevity …

Device Specific.(Z9)........HS1.81X1

IF THE VERSION IS H1.91X4, IT NEEDS TO BE FLASHED.

INFORM THE CSA SO IT CAN BE FLASHED AT A LATER DATE.

IF IT'S ANY OTHER VERSION, YOU ARE OK.

5. EDS Consolidated Storage can now zone and mask the new HBA's WWN so it can be seen.

6. EDS Consolidated Storage will run 'emc_cfgmgr'.

7. EDS Consolidated Storage will run 'powermt save'.

8. After EDS Consolidated Storage indicates that the work is done, confirm all paths are visible and optimal.

# powermt display

9. Confirm there are no disks in a 'Defined' state. If there are any, engage EDS Consolidated Storage.

# lsdev -Cc disk | grep -i define

10. This completes the IBM Hot-Swap HBA Replacement procedure.

AIX Interview Questions

Kernel

How would I know if I am running a 32-bit kernel or 64-bit kernel?

To display if the kernel is 32-bit enabled or 64-bit enabled, type:

bootinfo -K

How do I know if I am running a uniprocessor kernel or a multiprocessor kernel?

/unix is a symbolic link to the booted kernel. To find out what kernel mode is running, enter ls -l /unix and see what file /unix it links to. The following are the three possible outputs from the ls -l /unix command and their corresponding kernels:

/unix -> /usr/lib/boot/unix_up              # 32 bit uniprocessor kernel

/unix -> /usr/lib/boot/unix_mp              # 32 bit multiprocessor kernel

/unix -> /usr/lib/boot/unix_64              # 64 bit multiprocessor kernel

Note:
AIX 5L Version 5.3 does not support a uniprocessor kernel.

How can I change from one kernel mode to another?

During the installation process, one of the kernels, appropriate for the AIX version and the hardware in operation, is enabled by default. Let us use the method from the previous question and assume the 32-bit kernel is enabled. Let us also assume that you want to boot it up in the 64-bit kernel mode. This can be done by executing the following commands in sequence:

ln -sf /usr/lib/boot/unix_64    /unix

ln -sf /usr/lib/boot/unix_64    /usr/lib/boot/unix

bosboot -ad  /dev/hdiskxx

shutdown -r

The /dev/hdiskxx directory is where the boot logical volume /dev/hd5 is located. To find out what xx is in hdiskxx, run the following command:

 lslv -m hd5

Note:
In AIX 5.2, the 32-bit kernel is installed by default. In AIX 5.3, the 64-bit kernel is installed on 64-bit hardware and the 32-bit kernel is installed on 32-bit hardware by default.

Hardware

How would I know if my machine is capable of running AIX 5L Version 5.3?

AIX 5L Version 5.3 runs on all currently supported CHRP (Common Hardware Reference Platform)-based POWER hardware.

How would I know if my machine is CHRP-based?

Run the prtconf command. If it's a CHRP machine, the string chrp appears on the Model Architecture line.

How would I know if my System p machine (hardware) is 32-bit or 64-bit?

To display if the hardware is 32-bit or 64-bit, type:

bootinfo -y

How much real memory does my machine have?

To display real memory in kilobytes (KB), type one of the following:

bootinfo -r

lsattr -El sys0 -a realmem

Can my machine run the 64-bit kernel?

64-bit hardware is required to run the 64-bit kernel.

What are the values of attributes for devices in my system?

To list the current values of the attributes for the tape device, rmt0, type:

lsattr -l rmt0 -E

To list the default values of the attributes for the tape device, rmt0, type:

lsattr -l rmt0 -D

To list the possible values of the login attribute for the TTY device, tty0, type:

lsattr -l tty0 -a login -R

To display system level attributes, type:

lsattr -E -l sys0

How many processors does my system have?

To display the number of processors on your system, type:

lscfg | grep proc

How many hard disks does my system have and which ones are in use?

To display the number of hard disks on your system, type:

lspv

How do I list information about a specific physical volume?

To find details about hdisk1, for example, run the following command:

lspv hdisk1

How do I get a detailed configuration of my system?

Type the following:

lscfg

The following options provide specific information:

`-p`	Displays platform-specific device information. The flag is applicable to AIX 4.2.1 or later.
`-v`	Displays the VPD (Vital Product Database) found in the customized VPD object class.

For example, to display details about the tape drive, rmt0, type:

lscfg -vl rmt0

You can obtain very similar information by running the prtconf command.

How do I find out the chip type, system name, node name, model number, and so forth?

The uname command provides details about your system.

`uname -p`	Displays the chip type of the system. For example, PowerPC.
`uname -r`	Displays the release number of the operating system.
`uname -s`	Displays the system name. For example, AIX.
`uname -n`	Displays the name of the node.
`uname -a`	Displays the system name, nodename, version, machine ID.
`uname -M`	Displays the system model name. For example, IBM, 9114-275.
`uname -v`	Displays the operating system version.
`uname -m`	Displays the machine ID number of the hardware running the system.
`uname -u`	Displays the system ID number.

AIX

What version, release, and maintenance level of AIX is running on my system?

Type one of the following:

oslevel -r

lslpp -h bos.rte

How can I determine which fileset updates are missing from a particular AIX level?

To determine which fileset updates are missing from 5300-04, for example, run the following command:

oslevel -rl 5300-04

What SP (Service Pack) is installed on my system?

To see which SP is currently installed on the system, run the oslevel -s command. Sample output for an AIX 5L Version 5.3 system, with TL4, and SP2 installed would be:

oslevel –s

5300-04-02

Is a CSP (Concluding Service Pack) installed on my system?

To see if a CSP is currently installed on the system, run the oslevel -s command. Sample output for an AIX 5L Version 5.3 system, with TL3, and CSP installed would be:

oslevel –s

5300-03-CSP

How do I create a file system?

The following command will create, within volume group testvg, a jfs file system of 10MB with mounting point /fs1:

crfs -v jfs -g testvg -a size=10M -m /fs1

The following command will create, within volume group testvg, a jfs2 file system of 10MB with mounting point /fs2 and having read only permissions:

crfs -v jfs2 -g testvg -a size=10M -p ro -m /fs2

How do I change the size of a file system?

To increase the /usr file system size by 1000000 512-byte blocks, type:

chfs -a size=+1000000 /usr

Note:
In AIX 5.3, the size of a JFS2 file system can be shrunk as well.

How do I mount a CD?

Type the following:

mount -V cdrfs -o ro /dev/cd0  /cdrom

How do I mount a file system?

The following command will mount file system /dev/fslv02 on the /test directory:

mount /dev/fslv02 /test

How do I mount all default file systems (all standard file systems in the /etc/filesystems file marked by the mount=true attribute)?

The following command will mount all such file systems:

mount {-a|all}

How do I unmount a file system?

Type the following command to unmount /test file system:

umount /test

How do I display mounted file systems?

Type the following command to display information about all currently mounted file systems:

mount

How do I remove a file system?

Type the following command to remove the /test file system:

rmfs /test

How can I defragment a file system?

The defragfs command can be used to improve or report the status of contiguous space within a file system. For example, to defragment the file system /home, use the following command:

defragfs /home

Which fileset contains a particular binary?

To show bos.acct contains /usr/bin/vmstat, type:

lslpp -w /usr/bin/vmstat

Or to show bos.perf.tools contains /usr/bin/svmon, type:

which_fileset svmon

How do I display information about installed filesets on my system?

Type the following:

lslpp -l

How do I determine if all filesets of maintenance levels are installed on my system?

Type the following:

instfix -i | grep ML

How do I determine if a fix is installed on my system?

To determine if IY24043 is installed, type:

instfix -ik IY24043

How do I install an individual fix by APAR?

To install APAR IY73748 from /dev/cd0, for example, enter the command:

instfix -k IY73748 -d /dev/cd0

How do I verify if filesets have required prerequisites and are completely installed?

To show which filesets need to be installed or corrected, type:

lppchk -v

How do I get a dump of the header of the loader section and the symbol entries in symbolic representation?

Type the following:

dump -Htv

How do I determine the amount of paging space allocated and in use?

Type the following:

lsps -a

How do I increase a paging space?

You can use the chps -s command to dynamically increase the size of a paging space. For example, if you want to increase the size of hd6 with 3 logical partitions, you issue the following command:

chps -s 3 hd6

How do I reduce a paging space?

You can use the chps -d command to dynamically reduce the size of a paging space. For example, if you want to decrease the size of hd6 with four logical partitions, you issue the following command:

chps -d 4 hd6

How would I know if my system is capable of using Simultaneous Multi-threading (SMT)?

Your system is capable of SMT if it's a POWER5-based system running AIX 5L Version 5.3.

How would I know if SMT is enabled for my system?

If you run the smtctl command without any options, it tells you if it's enabled or not.

Is SMT supported for the 32-bit kernel?

Yes, SMT is supported for both 32-bit and 64-bit kernel.

How do I enable or disable SMT?

You can enable or disable SMT by running the smtctl command. The following is the syntax:

smtctl [ -m off | on [ -w boot | now]]

The following options are available:

`-m off`	Sets SMT mode to disabled.
`-m on`	Sets SMT mode to enabled.
`-w boot`	Makes the SMT mode change effective on next and subsequent reboots if you run the `bosboot` command before the next system reboot.
`-w now`	Makes the SMT mode change immediately but will not persist across reboot.

If neither the -w boot or the -w now options are specified, then the mode change is made immediately. It persists across subsequent reboots if you run the bosboot command before the next system reboot.

How do I get partition-specific information and statistics?

The lparstat command provides a report of partition information and utilization statistics. This command also provides a display of Hypervisor information.

Volume groups and logical volumes

How do I know if my volume group is normal, big, or scalable?

Run the lsvg command on the volume group and look at the value for MAX PVs. The value is 32 for normal, 128 for big, and 1024 for scalable volume group.

How to create a volume group?

Use the following command, where s partition_size sets the number of megabytes (MB) in each physical partition where the partition_size is expressed in units of MB from 1 through 1024. (It's 1 through 131072 for AIX 5.3.) The partition_size variable must be equal to a power of 2 (for example: 1, 2, 4, 8). The default value for standard and big volume groups is the lowest value to remain within the limitation of 1016 physical partitions per physical volume. The default value for scalable volume groups is the lowest value to accommodate 2040 physical partitions per physical volume.

mkvg -y name_of_volume_group -s partition_size list_of_hard_disks

How can I change the characteristics of a volume group?

You use the following command to change the characteristics of a volume group:

chvg

How do I create a logical volume?

Type the following:

mklv -y name_of_logical_volume name_of_volume_group number_of_partition

How do I increase the size of a logical volume?

To increase the size of the logical volume represented by the lv05 directory by three logical partitions, for example, type:

extendlv lv05 3

How do I display all logical volumes that are part of a volume group (for example, rootvg)?

You can display all logical volumes that are part of rootvg by typing the following command:

lsvg -l rootvg

How do I list information about logical volumes?

Run the following command to display information about the logical volume lv1:

lslv lv1

How do I remove a logical volume?

You can remove the logical volume lv7 by running the following command:

rmlv lv7

The rmlv command removes only the logical volume, but does not remove other entities, such as file systems or paging spaces that were using the logical volume.

How do I mirror a logical volume?

1. mklvcopy LogicalVolumeName Numberofcopies

2. syncvg VolumeGroupName

How do I remove a copy of a logical volume?

You can use the rmlvcopy command to remove copies of logical partitions of a logical volume. To reduce the number of copies of each logical partition belonging to logical volume testlv, enter:

rmlvcopy testlv 2

Each logical partition in the logical volume now has at most two physical partitions.

Queries about volume groups

To show volume groups in the system, type:

lsvg

To show all the characteristics of rootvg, type:

lsvg rootvg

To show disks used by rootvg, type:

lsvg -p rootvg

How to add a disk to a volume group?

Type the following:

extendvg   VolumeGroupName   hdisk0 hdisk1 ... hdiskn

How do I find out what the maximum supported logical track group (LTG) size of my hard disk?

You can use the lquerypv command with the -M flag. The output gives the LTG size in KB. For instance, the LTG size for hdisk0 in the following example is 256 KB.

/usr/sbin/lquerypv -M hdisk0

You can also run the lspv command on the hard disk and look at the value for MAX REQUEST.

What does syncvg command do?

The syncvg command is used to synchronize stale physical partitions. It accepts names of logical volumes, physical volumes, or volume groups as parameters.

For example, to synchronize the physical partitions located on physical volumes hdisk6 and hdisk7, use:

syncvg -p hdisk4 hdisk5

To synchronize all physical partitions from volume group testvg, use:

syncvg -v testvg

How do I replace a disk?

1. extendvg VolumeGroupName hdisk_new

2. migratepv hdisk_bad hdisk_new

3. reducevg -d VolumeGroupName hdisk_bad

How can I clone (make a copy of ) the rootvg?

You can run the alt_disk_copy command to copy the current rootvg to an alternate disk. The following example shows how to clone the rootvg to hdisk1.

alt_disk_copy -d  hdisk1

Network

How can I display or set values for network parameters?

The no command sets or displays current or next boot values for network tuning parameters.

How do I get the IP address of my machine?

Type one of the following:

ifconfig -a

host Fully_Qualified_Host_Name

For example, type host cyclop.austin.ibm.com.

How do I identify the network interfaces on my server?

Either of the following two commands will display the network interfaces:

lsdev -Cc if

ifconfig -a

To get information about one specific network interface, for example, tr0, run the command:

ifconfig tr0

How do I activate a network interface?

To activate the network interface tr0, run the command:

ifconfig tr0 up

How do I deactivate a network interface?

For example, to deactivate the network interface tr0, run the command:

ifconfig tr0 down