Patching with Live Upgrade
So you have a critical system that needs patching, but you can't risk killing the box. Solaris uses "Live Upgrade" as a tool for altering a system, while maintaining the ability to rollback the change. This is ideal for patching a system, especially kernel patches and the like, as it means that you can apply you patches, try them out and rollback if it's failed. The method I describe here, assumes that the boot disks are mirrored. You can still use Live Upgrade in a non-mirrored environment, but I'm not going in to that here.
SO in this example, the OS is partitioned as follows:
[root@testboxen]$ df -h Filesystem size used avail capacity Mounted on /dev/md/dsk/d0 3.9G 395M 3.5G 10% / /dev/md/dsk/d40 3.9G 1.4G 2.5G 36% /usr /dev/md/dsk/d20 7.9G 51M 7.8G 1% /var /dev/md/dsk/d30 3.9G 351M 3.6G 9% /opt
Obviously d0, d20, d30 and d40 are all mirrored meta devices. d10, is also a meta device, for swap. It is not relevant here. These meta devices are set up as follows with the sub-mirrors:
d0 : d1 and d2 d20 : d21 and d22 d30 : d31 and d32 d40 : d41 and d42
Preparation
So to use live upgrade, you first need to make sure that the necessary packages are installed and up to date.
1. Download and install the latest version of patch utilities patch 119254 (64 or higher) from sunsolve.sun.com
unzip 119254-66.zip patchadd -M . 119254-66
2. Download and install the latest version of pax patch 128330 from sunsolve.sun.com
unzip 128330-02.zip patchadd -M . 128330-02
3. Install the Live Upgrade packages if they don't already exist, from the Solaris install cd
cd PATH_TO_CDROM/Solaris_10/Tools/Installers/ ./liveupgrade20 -noconsole - nodisplay
4. Check the installation worked
pkgchk -v SUNWlucfg SUNWlur SUNWluu
Once, you've got the necessary packages installed, you can proceed with the Live Upgrade.
Live Upgrade
In this example, we assume that Live Upgrade has never been used before, so we will have to set up a base boot environment as well as a new boot environment that we will patch.
1. Create the base and the new boot environments. This will break the mirror!!!!
lucreate -c base -m /:/dev/md/dsk/d2:ufs,detach \ -m /var:/dev/md/dsk/d22:ufs,detach \ -m /opt:/dev/md/dsk/d32:ufs,detach \ -m /usr:/dev/md/dsk/d42:ufs,detach \ -n after_patch
The output will look like this:
Discovering physical storage devices Discovering logical storage devices Cross referencing storage devices with boot environment configurations Determining types of file systems supported Validating file system requests Preparing logical storage devices Preparing physical storage devices Configuring physical storage devices Configuring logical storage devices Analyzing system configuration. No name for current boot environment. Current boot environment is named <base>. Creating initial configuration for primary boot environment <base>. WARNING: The device </dev/md/dsk/d0> for the root file system mount point </> is not a physical device. WARNING: The system boot prom identifies the physical device </dev/dsk/c0t0d0s0> as the system boot device. Is the physical device </dev/dsk/c0t0d0s0> the boot device for the logical device </dev/md/dsk/d0>? (yes or no) yes INFORMATION: Assuming the boot device </dev/dsk/c0t0d0s0> obtained from the system boot prom is the physical boot device for logical device </dev/md/dsk/d0>. The device </dev/dsk/c0t0d0s0> is not a root device for any boot environment; cannot get BE ID. PBE configuration successful: PBE name <base> PBE Boot Device </dev/dsk/c0t0d0s0>. Comparing source boot environment <base> file systems with the file system(s) you specified for the new boot environment. Determining which file systems should be in the new boot environment. Updating boot environment description database on all BEs. Searching /dev for possible boot environment filesystem devices Updating system configuration files. The device </dev/dsk/c0t1d0s0> is not a root device for any boot environment; cannot get BE ID. Creating configuration for boot environment <after_patch>. Source boot environment is <base>. Creating boot environment <after_patch>. Creating file systems on boot environment <after_patch>. Creating <ufs> file system for </> in zone <global> on </dev/md/dsk/d2>. Creating <ufs> file system for </opt> in zone <global> on </dev/md/dsk/d32>. Creating <ufs> file system for </usr> in zone <global> on </dev/md/dsk/d42>. Creating <ufs> file system for </var> in zone <global> on </dev/md/dsk/d22>. Mounting file systems for boot environment <after_patch>. Calculating required sizes of file systems for boot environment <after_patch>. Populating file systems on boot environment <after_patch>. Checking selection integrity. Integrity check OK. Populating contents of mount point </>. Populating contents of mount point </opt>. Populating contents of mount point </usr>. Populating contents of mount point </var>. Copying. Creating compare databases for boot environment <after_patch>. Creating compare database for file system </var>. Creating compare database for file system </usr>. Creating compare database for file system </opt>. Creating compare database for file system </>. Updating compare databases on boot environment <after_patch>.
Check that it worked with the lustatus command:
lustatus Boot Environment Is Active Active Can Copy Name Complete Now On Reboot Delete Status -------------------------- -------- ------ --------- ------ ---------- base yes yes yes no - after_patch yes no no yes -
2. Activate the new boot environment
luactivate after_patch
A Live Upgrade Sync operation will be performed on startup of boot environment <after_patch>.
**********************************************************************
The target boot environment has been activated. It will be used when you
reboot. NOTE: You MUST NOT USE the reboot, halt, or uadmin commands. You
MUST USE either the init or the shutdown command when you reboot. If you
do not use either init or shutdown, the system will not boot using the
target BE.
**********************************************************************
In case of a failure while booting to the target BE, the following process
needs to be followed to fallback to the currently working boot environment:
1. Enter the PROM monitor (ok prompt).
2. Change the boot device back to the original boot environment by typing:
setenv boot-device /pci@1c,600000/scsi@2/disk@0,0:a
3. Boot to the original boot environment by typing:
boot
**********************************************************************
Modifying boot archive service
Activation of boot environment <after_patch> successful.
lustatus output should now look like this
lustatus Boot Environment Is Active Active Can Copy Name Complete Now On Reboot Delete Status -------------------------- -------- ------ --------- ------ ---------- base yes yes no no - after_patch yes no yes no -
3. Reboot the server using the 'shutdown' command. DO NOT use the 'reboot' command as it will then ignore the changes you've made and boot off the original boot environment
shutdown -g 0 -i 6 -y
After the reboot, you should see the following disk layout from the 'df' command:
df -h Filesystem size used avail capacity Mounted on /dev/md/dsk/d2 3.9G 412M 3.5G 11% / /dev/md/dsk/d42 3.9G 1.4G 2.5G 36% /usr /dev/md/dsk/d22 7.9G 51M 7.8G 1% /var /dev/md/dsk/d32 3.9G 351M 3.6G 9% /opt
4. Patch the box with the latest patch cluster and reboot. If all is OK, repeat steps 2 and 3 and then go to step 5.
5. If you're happy that the machine is happy, then delete the old boot environment
ludelete base
Output should look like:
Determining the devices to be marked free. Updating boot environment configuration database. Updating boot environment description database on all BEs. Updating all boot environment configuration databases. Boot environment <base> deleted.
6. Use metaclear to clear old mirrors, d0, d20, d30 and d40
metaclear d0 metaclear d20 metaclear d30 metaclear d40
7. Create a new boot environment re-using the the mirrors and the unused slices from the now deleted 'base' boot environment
lucreate -n patched_20101126 \ -m /:/dev/md/dsk/d0:ufs,mirror -m /:/dev/md/dsk/d1:attach \ -m /var:/dev/md/dsk/d20:ufs,mirror -m /var:/dev/md/dsk/d21:attach \ -m /opt:/dev/md/dsk/d30:ufs,mirror -m /opt:/dev/md/dsk/d31:attach \ -m /usr:/dev/md/dsk/d40:ufs,mirror -m /usr:/dev/md/dsk/d41:attach
This will re-create the meta devices d0, d20, d30 and d40 with sub-devices d1, d21, d31 and d41 respectively. It then copies the data from the current boot environment, 'after_patch' and create a new boot environment called 'patched_20101126'. Out put should look like:
Discovering physical storage devices Discovering logical storage devices Cross referencing storage devices with boot environment configurations Determining types of file systems supported Validating file system requests The device name </dev/md/dsk/d0> expands to device path </dev/md/dsk/d0> Preparing logical storage devices Preparing physical storage devices Configuring physical storage devices Configuring logical storage devices Analyzing system configuration. Comparing source boot environment <after_patch> file systems with the file system(s) you specified for the new boot environment. Determining which file systems should be in the new boot environment. Updating boot environment description database on all BEs. Searching /dev for possible boot environment filesystem devices Updating system configuration files. The device </dev/dsk/c0t0d0s0> is not a root device for any boot environment; cannot get BE ID. Creating configuration for boot environment <patched_20101126>. Source boot environment is <after_patch>. Creating boot environment <patched_20101126>. Creating file systems on boot environment <patched_20101126>. Creating <ufs> file system for </> in zone <global> on </dev/md/dsk/d0>. Creating <ufs> file system for </opt> in zone <global> on </dev/md/dsk/d30>. Creating <ufs> file system for </usr> in zone <global> on </dev/md/dsk/d40>. Creating <ufs> file system for </var> in zone <global> on </dev/md/dsk/d20>. Mounting file systems for boot environment <patched_20101126>. Calculating required sizes of file systems for boot environment <patched_20101126>. Populating file systems on boot environment <patched_20101126>. Checking selection integrity. Integrity check OK. Populating contents of mount point </>. Populating contents of mount point </opt>. Populating contents of mount point </usr>. Populating contents of mount point </var>. Copying. Creating shared file system mount points. Creating compare databases for boot environment <patched_20101126>. Creating compare database for file system </var>. Creating compare database for file system </usr>. Creating compare database for file system </opt>. Creating compare database for file system </>. Updating compare databases on boot environment <patched_20101126>. Making boot environment <patched_20101126> bootable. Setting root slice to Solaris Volume Manager metadevice </dev/md/dsk/d0>. Population of boot environment <patched_20101126> successful. Creation of boot environment <patched_20101126> successful.
8. Activate the new boot environment
luactivate patched_20101126
Output should look like:
A Live Upgrade Sync operation will be performed on startup of boot environment <patched_20101126>. ********************************************************************** The target boot environment has been activated. It will be used when you reboot. NOTE: You MUST NOT USE the reboot, halt, or uadmin commands. You MUST USE either the init or the shutdown command when you reboot. If you do not use either init or shutdown, the system will not boot using the target BE. ********************************************************************** In case of a failure while booting to the target BE, the following process needs to be followed to fallback to the currently working boot environment: 1. Enter the PROM monitor (ok prompt). 2. Change the boot device back to the original boot environment by typing:
setenv boot-device /pci@1c,600000/scsi@2/disk@1,0:a
3. Boot to the original boot environment by typing:
boot
**********************************************************************
Modifying boot archive service
Activation of boot environment <patched_20101126> successful.
9. Reboot the machine with the 'shutdown' command
shutdown -g 0 -i 6 -y
10. Use the 'lustatus' and df commands to check you've booted off the correct boot environment, 'patched_20101126'
lustatus Boot Environment Is Active Active Can Copy Name Complete Now On Reboot Delete Status -------------------------- -------- ------ --------- ------ ---------- after_patch yes no no yes - patched_20101126 yes yes yes no -
df -h Filesystem Size Used Avail Use% Mounted on /dev/md/dsk/d0 4.0G 412M 3.5G 11% / /dev/md/dsk/d40 4.0G 1.4G 2.6G 36% /usr /dev/md/dsk/d20 7.9G 52M 7.8G 1% /var /dev/md/dsk/d30 4.0G 352M 3.6G 9% /opt
11. Delete the old boot environment
ludelete after_patch
Check it worked
lustatus Boot Environment Is Active Active Can Copy Name Complete Now On Reboot Delete Status -------------------------- -------- ------ --------- ------ ---------- patched_20101126 yes yes yes no -
12. Re-attach sub-mirror devices that were used in the 'after_patch' boot environment to the new boot environment 'patched_20101126'
metattach d0 d2 metattach d20 d22 metattach d30 d32 metattach d40 d42
Wait for the mirrors to finish re-syncing.
13. Make sure the second drive is still bootable
installboot /usr/platform/`uname -i`/lib/fs/ufs/bootblk /dev/rdsk/c0t1d0s0
Now you're done!
