teh fear

Everytime I need to change something on a co-located server (ours are only about 2 miles from the main office, but it is still a pain if one has to go to the site because a server is not booting up) that requires restarting in a new configuration, I’m always worried that something might not work properly and I’ll be left stranded with a broken server.

A few years back linux booting was generally handled by a program called LILO, which involved placing entries in a configuration file (/etc/lilo.conf) and then calling the installer (/sbin/lilo)

And then… rebooting to see if it worked

And if you had a system that was a little out of the ordinary – you either had vast experience with lilo and it was easy (is there anyone who can say this?) or you had lots of failed reboots and a lot of head scratching.

has grub changed this?

Well, changing boot options is still a little scary, but I’m fairly confident I’ve got my head round most of this now – mostly because my system doesn’t appear to respond very well to the grub-install command (it just never finishes – or I’m too impatient)

There are three parts to grub’s boot procedure

• Grub’s ‘stage’ files
• OS boot information
• The Boot Record

Grub’s ‘stage’ files

If you are not using the grub-install script (I’m not as explained above), then grub’s stage files need to be copied into the relevant boot partition.

For me, as I generally use CentOS, I’ll find these files here ‘/usr/share/grub/’ – if you have installed grub from source you may find them here ‘/lib/grub’. If you are having trouble finding them, try one of these commands
locate stage1
find / -name stage1
Once you have found these files, copy ‘stage1′, ‘stage2′ and all the ‘*_stage1_5′ files into ‘/boot/grub/’
cp /lib/grub/i386-pc/*stage* /boot/grub/

OS boot information

Before continuing, I’ll need to clarify that I’ll be talking about booting when the partition I keep my boot files in is mounted as ‘/boot’. If you have your boot files in a normal directory called ‘/boot’ in a partition mounted as ‘/’ then you will need to prefix boot file paths with ‘/boot’. i.e: ‘/initrd’ will become ‘/boot/initrd’.

In order for grub to boot your OS successfully you need to tell it what you want, I’m only going to cover booting to linux/ xen here.

A typical grub configuration file ‘/boot/grub/grub.conf’ (often symlinked to from ‘/etc/grub.conf’) might look like this
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
 
default 1
timeout 5
 
title CentOS (2.6.9-55.0.9.EL)
root (hd0,0)
kernel /vmlinuz-2.6.9-55.0.9.EL ro root=/dev/md0
initrd /initrd-2.6.9-55.0.9.EL.img
title CentOS (2.6.18-xen)
root (hd0,0)
kernel /xen-3.1.0.gz
module /vmlinuz-2.6.18-xen_3.1.0 ro root=/dev/md0
module /initrd-2.6.18-xen_3.1.0.img
title CentOS (2.6.18)
root (hd0,0)
kernel /vmlinuz-2.6.18 ro root=/dev/md0
initrd /initrd-2.6.18.img
There are three boot entries in here, each starting with the keyword ‘title’
The middle entry boots xen, with linux kernel and initrd lines passed as modules.

default

Which entry in the list should be selected by default. Entries are numbered starting from 0, some versions of grub will also accept the value ‘saved’. See also savedefault

timeout

The number of seconds to wait before booting the default option

title

The string to be displayed in the grub boot menu

root

The disk partition where the boot files are stored, see Boot record below

kernel

The OS kernel file to boot (path relative to boot partition)

initrd

The path to a linux initial ramdisk image

module

a parameter to pass to the kernel image being loaded

savedefault

Change the saved default boot entry to be the number specified, only applies when default is set to ‘saved’. Also see version note below

The kernel line (or in the case of the xen entry, thie first module line) provides the path to the kernel to be loaded and also options to pass to that kernel. The above example has two options:
‘ro’ – mount the root partition read-only (this is so we can run a disk check on it during the boot process) and
‘root=/dev/md0′ – the root partition to be mounted is on raid device md0.
ext2/3 filesystems can be labelled to make this easier, I could label the filesystem in /dev/md0 to be called ‘foo’ by using the following command:
tune2fs -L foo /dev/md0
I could then change my kernel root option to read
root=LABEL=foo

The Boot Record

The last thing required is to install a small piece of code in a special sector of the disk which is used by the machine to boot, a hard coded place where every PC knows to look.
There is more than one hard coded place, a PC will look first at the Master Boot Record (MBR) of a disk – at the very start of the disk, then it will look at the start of any partition on the disk that is marked as bootable.

In order to install this record we use the program grub, just type ‘grub’ at the command line and you should enter the grub shell.

First, tell grub whre the stage files and boot information is stored.
If you only have one disk in your system, then this will probably be (hd0,0) – partition 1.
grub> root (hd0,0)
Filesystem type is ext2fs, partition type 0xfd
As you can see, grub will attempt to tell you the filesystem of the chosen partition – in this case, this tells me that the first partition on disk (hd0) contains an ext2/3 filesystem and its partition type is 0xfd – linux raid autodetect.
Similarily this
grub> root (hd0,1)
Filesystem type unknown, partition type 0x82
Tells me that the next partition, type 0×82, is one of my swap partitions. It is also enough to tell me that i’m looking at the right disk.

Next we run setup to install the record, telling the setup command where to install (for me, this is (hd0) the MBR of the first disk).
grub> setup (hd0)
Checking if "/boot/grub/stage1" exists... no
Checking if "/grub/stage1" exists... yes
Checking if "/grub/stage2" exists... yes
Checking if "/grub/e2fs_stage1_5" exists... yes
Running "embed /grub/e2fs_stage1_5 (hd0)"... 15 sectors are embedded.
succeeded
Running "install /grub/stage1 (hd0) (hd0)1+15 p (hd0,0)/grub/stage2 /grub/menu.lst"... succeeded
Done.

If you’re still not sure if that was the right partition – this is what happens if you try to use the wrong one
grub> root (hd1,0)
Filesystem type is ext2fs, partition type 0x83
 
grub> setup (hd1)
Checking if "/boot/grub/stage1" exists... no
Checking if "/grub/stage1" exists... no
 
Error 15: File not found
Also, if you really want to be sure, place a uniquely named file with the stage files – at the command line (not in the grub shell), type
touch /boot/find_this_file
then within the grub shell
grub> find /find_this_file
(hd0,0)
(hd2,0)
In my case, I get two responses – this is because ‘/boot’ on my system is a mirrored raid using the first partition of (hd0) and the firstpartition of (hd2).
I’ll also run grub with ‘root (hd2,0)’ and ‘setup (hd2)’ to ensure both mirrored disks can be used to boot my system.

Finally, to get out of the grub shell, type ‘quit’.

A note about grub versions

Most of my recent grub experience is with CentOS 4.
The version of grub that comes with centos 4 (0.95, I think) does not appear to support ‘default saved’ or ‘savedefault’, however, a more recent version (I used version 0.97 from here ftp://alpha.gnu.org/gnu/grub/) is available that does. I have found the savedefault option to be invaluable.

In order to build the downloaded version, first, unpack it
tar -xvzf grub-0.97.tar.gz
then go into the directory that has been created
cd grub-0.97
Next, configure the source for your system. I was trying to get as close to the configuration used by centos, so I used their configure options -
./configure --build=i686-redhat-linux-gnu --host=i686-redhat-linux-gnu --target=i386-redhat-linux-gnu --program-prefix= --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/usr/com --mandir=/usr/share/man --infodir=/usr/share/info --sbindir=/sbin --disable-auto-linux-mem-opt
You could just run ‘./configure’ but the files will end up installed in different locations.

Next, make the program
make
Then, if the program compiled successfully, install it (this needs to be done as user root)
make install

With all that out of the way, you will be able to use ‘default saved’ in your grub.conf file…
This reads the default entry number from a file called ‘default’ in the /boot/grub/ directory, don’t edit this file directly, set it in one of two ways

• By using an entry in the boot menu ‘savedefault n’ where n is the new default to set
• By using the command ‘grub-set-default n’ – can be placed in a start-up script like ‘/etc/rc.local’

We can utilise this to be able to try booting a new entry – as I mentioned at the start, I manage a number of co-located servers located in remote locations, if these machines fail to reboot I would need to travel to the data centre in order to work on the physical machine…

So, when I decided that xen was going to be installed on one of them, I needed to be able to try booting into xen, knowing that if it didn’t work I could reset the machine remotely and it would boot back into its old configuration.
I used a grub.conf file similar to this:
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
 
default saved
timeout 5
 
title CentOS (2.6.18-xen)
root (hd0,0)
kernel /xen-3.1.0.gz
module /vmlinuz-2.6.18-xen_3.1.0 ro root=/dev/md0
module /initrd-2.6.18-xen_3.1.0.img
savedefault 1
title CentOS (2.6.18)
root (hd0,0)
kernel /vmlinuz-2.6.18 ro root=/dev/md0
initrd /initrd-2.6.18.img
I set the default in both /etc/rc.local and initially at the command line by using
grub-set-default 0
On the initial boot, grub reads the saved value (0) from the default file, entry 0 is chosen.
The savedefault option changed the default file to contain the value 1.
Xen booted fine and so the /etc/rc.local script reset the default file to contain the value 0.

If I hadn’t been so lucky and the xen boot option had frozen the machine, when it was reset remotely the default value was now 1 – meaning the previous (known working) boot entry was chosen and the machine restarted.
Except, in this instance – next reboot would find default set to 0 again.