View source for CentOS 5 - Build a Custom Xen Kernel with NFS Root Filesystem (CONFIG ROOT NFS) Support

In order to boot and run a Xen guest domain a key requirement (amongst other items) is a root filesystem populated with the files and directories necessary for the guest operating system to function. One of the most flexible Xen configurations involves the installation of the root filesystem on an NFS server such that it can be accessed and booted on a client system. This approach is of particular use when the need to migrate Xen domainU guests from one host to another arises. Since both the original host and the target host to which the guest is migrating need access to the domainU's root filesystem, NFS provides an ideal mechanism for achieving this by allowing the root filesystem located on the NFS server to be mounted on both client hosts.

In order to boot a Xen guest domain with an NFS mounted root filesystem a version of the Linux kernel with NFS root filesystem support is required. Unfortunately this functionality is not enabled by default in most standard kernels. It is necessary, therefore, to build a custom kernel with this capability enabled. In this tutorial we will work through an example of building a custom CentOS 5.1 kernel with NFS root filesystem support.

== Obtaining the Kernel Sources ==

The first step in building a custom kernel involves the download and installation of the latest kernel sources. These can be obtained from the CentOS repository which is online at [http://mirror.centos.org/centos/5/updates/SRPMS http://mirror.centos.org/centos/5/updates/SRPMS]. The naming convention for the kernel source package takes the form of:

<tt>kernel-<version>.el5.src.rpm</tt>

where <version> represents the version of kernel running on your system. This information can be obtained by running:

<pre>
uname -mr
2.6.18-53.1.14.el5xen i686
</pre>

The correct kernel source package for the above release, therefore, would be kernel-2.6.18-53.1.14.el5.src.rpm.

Having identified the appropriate kernel source package for your Linux installation, download it to your home directory.

== Preparing for the Custom Kernel Build ==

While logged in as yourself (do not perform these tasks as root) it is necessary to perform the following steps in preparation for installing the sources:

<pre>
cd
mkdir -p rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
echo "%_topdir %(echo $HOME)/rpmbuild" > .rpmmacros
</pre>

Next, you will need to install (as root) the rpm-build package if it is not already installed on your system:

<pre>
su - 
yum install rpm-build
</pre>

Be sure to exit from the su session to revert to your original user account.

Next, we can install the kernel sources from the rpm file downloaded previously:

<pre>
rpm -Uvh kernel-<version>.el5.src.rpm
</pre>

Once the packages are installed, the sources need to unpacked using the ''rpmbuild'' command:

<pre>
cd ~/rpmbuild/SPECS
pmbuild -bp --target=`uname -m` kernel-2.6.spec
</pre>

Once this task completes the sources will be located in ~/rpmbuild/BUILD/.

== Customizing the Kernel for NFS Root Support ==

Now that we have the kernel sources installed we are ready to begin the process of customizing the kernel to support an NFS based root filesystem. The first step is to copy the configuration file from the running operating system to the appropriate directory of the kernel build tree:

<pre>
cd ~/rpmbuild/BUILD/kernel-2.6.18/linux-2.6.18.i686
cp /boot/config-`uname -r` .config
make oldconfig
</pre>

We are now ready to begin the customization process. For the purposes of this tutorial we will be using the ''make menuconfig'' mechanism which requires that the ''ncurses-devel'' package be installed. If this is not installed, install it now as follows:

<pre>
su -
yum install ncurses-devel
</pre>

Once again, be sure to exit from the su session before proceeding beyond this point.

== Configuring the Kernel with make menuconfig ==

Whilst still in the ~/rpmbuild/BUILD/kernel-2.6.18/linux-2.6.18.i686 directory, start the kernel configuration tool:

<pre>
make menuconfig
</pre>

After some initialization, the following screen should appear:


[[Image:make_menuconfig_main.jpg|The main menuconfig screen]]


In order to configure root NFS support it is first necessary to enable ''IP Autoconfiguration''. This setting is accessed by selecting ''Network'' and then ''Networking Options'' in the menu system. On the Network options screen select ''IP: kernel level autoconfiguration'' and ''IP: DHCP support'' options so that an asterisk (*) appears within the square brackets. Also select the ''IP: BOOTP support'' and ''IP: RARP support'' options if required:


[[Image:menu_config_network_options.jpg| Configuring Kernel Networking Options]]


Press the '''Esc''' key twice to return to the main menu and then select ''File Systems->Network File Systems''. On the Network File Systems page scroll down to the ''Root filesystem on NFS'' option and select it:


[[Image:menu_config_root_nfs.jpg| Configuring Kernel Root filesystem on NFS support]]


Press the '''Esc''' key three times to exit from menuconfig and select the option to save the new configuration.

Edit the .config file and add a line immediately at the top of the file reflecting your architecuture:

<pre>
# i386
</pre>

or 

<pre>
# x86_64
</pre>

Finally, copy the new configuration file to the source directory ready for the kernel compilation:

<pre>
cp .config ~/rpmbuild/SOURCES/kernel-2.6.18-<arch>-xen.config
</pre>

Where <arch> represents your architecture, for example ''i686'', ''ia64'', ''x86_64'', ''ppc64'' etc.

== Compiling the Kernel ==

The next step is to make some minor modifications to the ~/rpmbuild/SPECS/kernel-2.6.spec file. Load this file into an editor and change the ''build'' id definition such that it will be easy to distinguish the custom kernel from the original kernel. For example change:

<pre>
#% define buildid
</pre>

to:

<pre>
%define buildid .mycustom
</pre>

Also comment out the following lines:

<pre>
%prep
#if a rhel kernel, apply the rhel config options
%if 0%{?rhel}
  for i in %{all_arch_configs}
  do
    mv $i $i.tmp
    $RPM_SOURCE_DIR/merge.pl $RPM_SOURCE_DIR/config-rhel-generic $i.tmp > $i
    rm $i.tmp
  done
  #CONFIG_FB_MATROX is disabled for rhel generic but needed for ppc64 rhel
  for i in $RPM_SOURCE_DIR/kernel-%{kversion}-ppc64.config
  do
    mv $i $i.tmp
    $RPM_SOURCE_DIR/merge.pl $RPM_SOURCE_DIR/config-rhel-ppc64-generic $i.tmp > $i
    rm $i.tmp
  done
%endif
</pre>

Save the spec file and then initiate the build as follows:

<pre>
rpmbuild -bb --target=`uname -m` kernel-2.6.spec
</pre>

Once the build is complete rpm packages will be located in ~rpmbuild/RPMS/<arch>. Install the custom Xen kernel on the system:

<pre>
rpm -ihv kernel-xen-<version>.el5.mycustom.i686.rpm
</pre>

The /boot directory will contain the necessary files to boot from the custom kernel, either for the host system or for a Xen domainU system.