Ubuntu 10.04 LTS on XenServer 5.6

So, I've been trying out various virtualisation options on an old Dell 9150 tower I have. Firstly I tried VmWare ESXi which was ok but didn't support software RAID on the disks. So I got an Adaptec 3405 hardware RAID card off Ebay which was apparently supported by ESXi. Which it was except for the fact that there was no support for any monitoring at all. Essentially if you lost a disk the only way you'd know is by watching the console on a reboot. Much as I liked ESXi (mostly) and although this wasn't a critical server, this and the other general lack of support for removable storage etc meant that ESXi was not for me.

Next I got involved in a couple of boxes that happened to be running XenServer and I quite liked what I saw and also the XenCenter was pretty neat and not so complex as the VmWare vSphere Client. So I installed XenServer 5.6.1 on the box. As Xen is a bit more like a traditional Linux distro, and so is slightly less of a 'thin' layer, it is often easier to install third party packages on it. The first thing I did was install an Adaptec storage agent by following the instructions at http://forums.citrix.com/message.jspa?messageID=1467235 so now I could see when the RAID array was having problems. After that I created a couple of Centos VMs and sat back and thought how simple this all is. It was then I hit a couple of hurdles when I tried to create Ubuntu 10.04 LTS VMs (32 bit and 64 bit). There is no defined template for Ubuntu so I ended up using the 'other OS' option and sure enough managed to get them installed. However it turns out that Ubuntu is not really configured to run optimally on Xen and won't run in full Paravirtualised (PV) mode. Also you can't install XenTools so cant control the o/s via XenCenter. This is a royal PITA.

After a couple of days of googling and experimenting I have finally managed to aggregate the information require to actually do this, meaning that XenServer is now a realistic option for non-enterprise small scale virtualisation. I got the info from http://forums.virtualizationadmin.com/m_732/mpage_1/key_/tm.htm#732, http://forums.citrix.com/thread.jspa?messageID=1526184&#1526184 and http://www.bishopbarrington.net/wiki/howto/lucidonxenserver#xenserver_to... but in that form it is somewhat disjointed so here it is reproduced in the large.

==== snip ====
1. Create a VM in XenCenter. I used Windows 2008 R2 x64 as a template.
2. Install Ubuntu 10.04 in any way you like. Just remember that when you get to choosing your partitions that you select EXT3 for your /boot partition because pygrub does not support the EXT4 file system yet. All the other partitions can be made EXT4.
3. Install SSH SERVER just in case something goes wrong so that you can still access your VM once it is installed.
4. The console for the DomU in PV mode is HVC0. You will need to tell getty to this. Copy the existing tty1.conf and edit it to replace "tty1" with "hvc0".

sudo cp /etc/init/tty1.conf /etc/init/hvc0.conf
sudo nano -w /etc/init/hvc0.conf

5. Shutdown the VM
6. Retrieve the UUID for the VM by using : xe vm-list
7. Retireve the UUID for the VBD disk by using xe vm-disk-list uuid=xxxxxxxxxxxxxxxx
8. Clear out the HVM boot policy by using xe vm-param-set uuid=xxxxxxxxx HVM-boot-policy=
9. Set the PV bootloader to pygrub by using xe vm-param-set uuid=xxxxxxx PV-bootloader=pygrub
10. Set the VBD disk to bootable by using xe vbd-param-set uuid= bootable=true
11. Start the VM and login. If your console appears to hang, try connecting to the VM using SSH. If you missed step 4 the console will appear to hang. However it is actually not displaying the login prompt on hvc0
12. Mount the xs-tools.iso
sudo mount /dev/cdrom1 /mnt/
13. Replace dash shell with bash needed for installation, run this command. When asked to install dash as /bin/sh, choose No.
sudo apt-get -y install bash && sudo dpkg-reconfigure dash
14. Install the XenTools package (note the version and architecture may be different for you)
sudo dpkg -i /mnt/Linux/xe-guest-utilities_5.5.0-466_amd64.deb
15. Fix the service so it runs at boot:
sudo update-rc.d -f xe-linux-distribution remove
sudo update-rc.d xe-linux-distribution defaults
16. Unmount and eject the xs-tools.iso before restarting the VM.
==== snip ====

And bingo you have Ubuntu running in full paravirtualised mode with XenTools installed. It's much quicker and you have full control from XenCenter or the CLI.

A slightly annoying footnote to this exercise is that since installing the Adaptec agent, allowing me to query the state of the RAID drive the Adaptec Storage Manager, it turns out I have a 'bad stripe' which cannot be fixed without destroying the array. So a full array rebuild and reinstall of XenServer is on the cards before I can actually do anything useful with it. At least with Xen you can mount removable storage (unlike ESXi where copying over the network was the only option) so I can save the actual VMs by following the instructions at http://www.experts-exchange.com/Software/Virtualization/Xen/Q_24715870.html (the GUI does not seem to allow it) and not have to go through all that again. Fingers crossed anyway!