CentOS 5 - Build a Custom Xen Kernel with NFS Root Filesystem (CONFIG ROOT NFS) Support

In order to boot and run a Xen guest domain a key requirement (amongst other items) is a root filesystem populated with the files and directories necessary for the guest operating system to function. One of the most flexible Xen configurations involves the installation of the root filesystem on an NFS server such that it can be accessed and booted on a client system. This approach is of particular use when the need to migrate Xen domainU guests from one host to another arises. Since both the original host and the target host to which the guest is migrating need access to the domainU's root filesystem, NFS provides an ideal mechanism for achieving this by allowing the root filesystem located on the NFS server to be mounted on both client hosts.

In order to boot a Xen guest domain with an NFS mounted root filesystem a version of the Linux kernel with NFS root filesystem support is required. Unfortunately this functionality is not enabled by default in most standard kernels. It is necessary, therefore, to build a custom kernel with this capability enabled. In this tutorial we will work through an example of building a custom CentOS 5.1 kernel with NFS root filesystem support.

Obtaining the Kernel Sources

The first step in building a custom kernel involves the download and installation of the latest kernel sources. These can be obtained from the CentOS repository which is online at http://mirror.centos.org/centos/5/updates/SRPMS. The naming convention for the kernel source package takes the form of:

kernel-<version>.el5.src.rpm

where <version> represents the version of kernel running on your system. This information can be obtained by running:

uname -mr
2.6.18-53.1.14.el5xen i686

The correct kernel source package for the above release, therefore, would be kernel-2.6.18-53.1.14.el5.src.rpm.

Having identified the appropriate kernel source package for your Linux installation, download it to your home directory.


Preparing for the Custom Kernel Build

While logged in as yourself (do not perform these tasks as root) it is necessary to perform the following steps in preparation for installing the sources:

cd
mkdir -p rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
echo "%_topdir %(echo $HOME)/rpmbuild" > .rpmmacros

Next, you will need to install (as root) the rpm-build package if it is not already installed on your system:

su - 
yum install rpm-build

Be sure to exit from the su session to revert to your original user account.

Next, we can install the kernel sources from the rpm file downloaded previously:

rpm -Uvh kernel-<version>.el5.src.rpm

Once the packages are installed, the sources need to unpacked using the rpmbuild command:

cd ~/rpmbuild/SPECS
pmbuild -bp --target=`uname -m` kernel-2.6.spec

Once this task completes the sources will be located in ~/rpmbuild/BUILD/.

Customizing the Kernel for NFS Root Support

Now that we have the kernel sources installed we are ready to begin the process of customizing the kernel to support an NFS based root filesystem. The first step is to copy the configuration file from the running operating system to the appropriate directory of the kernel build tree:

cd ~/rpmbuild/BUILD/kernel-2.6.18/linux-2.6.18.i686
cp /boot/config-`uname -r` .config
make oldconfig

We are now ready to begin the customization process. For the purposes of this tutorial we will be using the make menuconfig mechanism which requires that the ncurses-devel package be installed. If this is not installed, install it now as follows:

su -
yum install ncurses-devel

Once again, be sure to exit from the su session before proceeding beyond this point.

Configuring the Kernel with make menuconfig

Whilst still in the ~/rpmbuild/BUILD/kernel-2.6.18/linux-2.6.18.i686 directory, start the kernel configuration tool:

make menuconfig

After some initialization, the following screen should appear:


The main menuconfig screen


In order to configure root NFS support it is first necessary to enable IP Autoconfiguration. This setting is accessed by selecting Network and then Networking Options in the menu system. On the Network options screen select IP: kernel level autoconfiguration and IP: DHCP support options so that an asterisk (*) appears within the square brackets. Also select the IP: BOOTP support and IP: RARP support options if required:


Configuring Kernel Networking Options


Press the Esc key twice to return to the main menu and then select File Systems->Network File Systems. On the Network File Systems page scroll down to the Root filesystem on NFS option and select it:


Configuring Kernel Root filesystem on NFS support


Press the Esc key three times to exit from menuconfig and select the option to save the new configuration.

Edit the .config file and add a line immediately at the top of the file reflecting your architecuture:

# i386

or

# x86_64

Finally, copy the new configuration file to the source directory ready for the kernel compilation:

cp .config ~/rpmbuild/SOURCES/kernel-2.6.18-<arch>-xen.config

Where <arch> represents your architecture, for example i686, ia64, x86_64, ppc64 etc.

Compiling the Kernel

The next step is to make some minor modifications to the ~/rpmbuild/SPECS/kernel-2.6.spec file. Load this file into an editor and change the build id definition such that it will be easy to distinguish the custom kernel from the original kernel. For example change:

#% define buildid

to:

%define buildid .mycustom

Also comment out the following lines:

%prep
#if a rhel kernel, apply the rhel config options
%if 0%{?rhel}
  for i in %{all_arch_configs}
  do
    mv $i $i.tmp
    $RPM_SOURCE_DIR/merge.pl $RPM_SOURCE_DIR/config-rhel-generic $i.tmp > $i
    rm $i.tmp
  done
  #CONFIG_FB_MATROX is disabled for rhel generic but needed for ppc64 rhel
  for i in $RPM_SOURCE_DIR/kernel-%{kversion}-ppc64.config
  do
    mv $i $i.tmp
    $RPM_SOURCE_DIR/merge.pl $RPM_SOURCE_DIR/config-rhel-ppc64-generic $i.tmp > $i
    rm $i.tmp
  done
%endif

Save the spec file and then initiate the build as follows:

rpmbuild -bb --target=`uname -m` kernel-2.6.spec

Once the build is complete rpm packages will be located in ~rpmbuild/RPMS/<arch>. Install the custom Xen kernel on the system:

rpm -ihv kernel-xen-<version>.el5.mycustom.i686.rpm

The /boot directory will contain the necessary files to boot from the custom kernel, either for the host system or for a Xen domainU system.