Installing OpenIndiana in a Xen DomU

Installing OpenIndiana in a Xen DomU

So I am on the ground in Dallas and trying to get my head around some of the technologies that we use in my new environment.  One of those being the Xen kernel.  I have never actually used Xen before, though I have used xenserver a bit.  Anyways so my first foray into the Xen world is trying to get OpenIndiana (successor to OpenSolaris) running on it, and see what sort of performance issues it might have.  We are hoping to be able to leverage its CIFS and vscan functionality to provide some security for a secured network file server.

I am by no means an expert in Xen, so you might notice some minor mistakes.  These instructions worked in my environment and should work in your environment as well, and the performance was actually quite good.  I am making the following assumptions.

  • You are using a Linux Dom0, this might not matter, but for the purposes of this exercise we will pretend that it does.
  • You are using LVM to back your DomU.
  • You are going to run OpenIndiana, if you are trying to run something different (Solaris, Nexenta, SmartOS, etc) then there will be other nuances to get those particular distributions working.
  • You are using XM configuration files directly and not using libvirt to try and simplify the configuration (I love libvirt, but could not get it working for these configurations).
  • You do have libvirt installed, so that we can take advantage of virsh console.

Prepare the File System

Make directories for your Xen DomU kernels, and for the OpenIndiana cd mount.

# mkdir /etc/xen/kernels/oi151a; mkdir /mnt/oi151a

Create the Logical Volume for your OpenIndiana DomU.

# lvcreate -L40G yourvgnamehere -n vm_openindiana_boot

Mount the OpenIndiana ISO.

# mount -o loop,ro /mnt/scratch/oi151a.iso /mnt./oi151a

Copy the Kernel and Boot Archive out of the CD.

# cp /mnt/oi151a/platform/i86pc/amd64/boot_archive /etc/xen/kernels/oi151a/
# cp /mnt/oi151a/platform/i86xpv/kernel/amd64/unix /etc/xen/kernels/oi151a/

Unmount the OpenIndiana CD.

# umount /mnt/oi151a

Create the DomU Configurations

You actually need 2 configurations, you have to boot your LiveCD installer differently then the installed system.

The first is the build configuration, in other words the one that you can boot the LiveCD and run the installer for OpenIndiana.  This configuration requires that the ISO image be the first disk, we also must use the pygrub boot loader and finally we configure the console and livemode with extra parameters.

# cat /etc/xen/vm/openindiana-build
disk = [ 'file:/mnt/scratch/oi151a.iso,xvdc:cdrom,r', 'phy:yourvgnamehere/vm_openindiana_boot,xvda,w' ]
vcpus = 1
memory = 1024
name = "openindiana"
#kernel = "/etc/xen/kernels/oi151a/unix"
#ramdisk = "/etc/xen/kernels/oi151a/boot_archive"
#extra = "/platform/i86xpv/kernel/amd64/unix -B console=ttya,livemode=text"
bootloader = "/usr/bin/pygrub"
on_shutdown = "destroy"
on_reboot = "restart"
on_crash = "destroy"
vif = [ 'mac=aa:00:0a:4c:00:31, bridge=br0' ]

The second is the running configuration, which will actually boot of the disk, and persist your changes across reboots.

This one no longer needs to pygrub bootloader, console and livemode, but instead we need to point to the rpool (system zfs filesystem) and the boot path, so that it can find the file systems.

# cat /etc/xen/vm/openindiana-run
disk = [ 'phy:yourvgnamehere/vm_openindiana_boot,xvda,w' ]
vcpus = 1
memory = 1024
name = "openindiana"
kernel = "/etc/xen/kernels/oi151a/unix"
ramdisk = "/etc/xen/kernels/oi151a/boot_archive"
extra = "/platform/i86xpv/kernel/amd64/unix -B zfs-bootfs=rpool/ROOT/openindiana,bootpath='/xpvd/xdf@51712:a'"
on_shutdown = "destroy"
on_reboot = "restart"
on_crash = "destroy"
vif = [ 'mac=aa:00:0a:4c:00:31, bridge=br0' ]

Boot to OpenIndiana LiveCD

Here is where everything starts to get a bit crazy.  So firstly we need to create the VM.

# xm create openindiana-build

Now we need to connect to the VM via console.  Keep in mind that when you need to close the console, you can use ^] to close it.  Default username is jack with a password of jack.

# virsh console openindiana

OpenIndiana Build oi_151a 64-bit (illumos f342d051b376)
SunOS Release 5.11 - Copyright 1983-2010 Oracle and/or its affiliates.
All rights reserved. Use is subject to license terms.
WARNING: No randomness provider enabled for /dev/random. Use cryptoadm(1M) to enable a provider.
Hostname: openindiana
Remounting root read/write
Probing for device nodes ...
Preparing live image for use
Done mounting Live image
USB keyboard
1. Albanian                      25. Latin-American
2. Arabic                        26. Lithuanian
3. Belarusian                    27. Latvian
4. Belgian                       28. Macedonian
5. Brazilian                     29. Malta_UK
6. Bulgarian                     30. Malta_US
7. Canadian-Bilingual            31. Norwegian
8. Croatian                      32. Polish
9. Czech                         33. Portuguese
10. Danish                        34. Romanian
11. Dutch                         35. Russian
12. Dvorak                        36. Serbia-And-Montenegro
13. Estonian                      37. Slovak
14. Finnish                       38. Slovenian
15. French                        39. Spanish
16. French-Canadian               40. Swedish
17. Hungarian                     41. Swiss-French
18. German                        42. Swiss-German
19. Greek                         43. Traditional-Chinese
20. Icelandic                     44. TurkishF
21. Italian                       45. TurkishQ
22. Japanese-type6                46. UK-English
23. Japanese                      47. US-English
24. Korean
To select the keyboard layout, enter a number [default 47]:

1. Arabic                        12. Hungarian
2. Catalan                       13. Indonesian
3. Chinese - Simplified          14. Italian
4. Chinese - Traditional         15. Japanese
5. Czech                         16. Korean
6. Dutch                         17. Polish
7. English                       18. Portuguese - Brazil
8. French                        19. Russian
9. German                        20. Slovak
10. Greek                         21. Spanish
11. Hebrew                        22. Swedish
To select the language you wish to use, enter a number [default is 7]:
User selected: English
Configuring devices.

openindiana console login: Jan 10 16:06:51 openindiana nwamd[66]: 10: start_dhcp: ipadm_create_addr failed for xnf0: Address object already exists
Jan 10 16:06:54 openindiana gdm-simple-slave[2150]: CRITICAL: file gdm-session-direct.c: line 2167: assertion `value != NULL' failed
Jan 10 16:06:56 openindiana gnome-keyring-daemon[2301]: couldn't allocate secure memory to keep passwords and or keys from being written to the disk

openindiana console login: jack
Password:
Last login: Tue Jan 10 16:06:54 on console
OpenIndiana (powered by illumos)    SunOS 5.11    oi_151a    September 2011

Configure VNC in the LiveCD Session

In my environment we do not have DHCP on this VLAN, so I have to reconfigure networking to use a static IP address.  I have a separate article here.  But we will need to configure VNC, so that we can launch the gui install program.

$ mkdir .vnc
$ cp .Xclients .vnc/startup
$ vncserver

You will require a password to access your desktops.

Password:
Verify:
xauth:    creating new authority file /jack/.Xauthority

New 'openindiana:1 ()' desktop is openindiana:1

Starting applications specified in /jack./.vnc/startup
Log file is /jack/.vnc/openindiana:1.log
$

Now simply use a VNC client on your workstation to connect to openindiana:1 via VNC.  Depending on your local environment you might need to use the IP address instead of the name.  Also please keep in mind that if you execute the VNC server as root (by doing a sudo -s) then you will be logged on as root.  Which means you won’t have an icon from which to install OpenIndiana, you can still execute it manually…  However at some point your screen could lock and you will be unable to unlock it.  So long story short execute it as jack.

Install OpenIndiana

Now everything should be setup in such a way that you are able to actually perform the installation of OpenIndiana.  So go ahead and do it.

Switch to the Running Configuration

So once we are done, just shutdown the machine.  Once it is safely shutdown we can proceed.

Destroy the DomU which is configured for the LiveCD.

# xm destroy openindiana

Create the DomU which is configured to boot from the disk.

# xm create openindiana-run
Using config file "./openindiana-run".
Started domain openindiana

Now once it is up and running you can go about your business of configuring your DomU services.