Xen installation » History » Revision 31
« Previous |
Revision 31/43
(diff)
| Next »
Lutz Suhrbier, 08/04/2009 05:22 PM
BGBM Xen server documentation¶
This document describes the general setup Xen servers hosting virtual machines (VMs) at BGBM. Currently, the following servers are configured as Xen servers:
wp5demo.e-taxonomy.eu (160.45.63.20)
bgbm45.bgbm.fu-berlin.de (160.45.63.135)
All Xen installations base on the Linux distribution Debian 5 (Lenny) for the 64 bit architecture (amd64). Debian Lenny provides packages for Xen version 3.2.1. That way, virtual machines may run on 64 bit or 32 bit architectures alternatively.
Regarding partition management, all Xen servers use Logical Volume Manager (LVM2) LVM permits flexible and individual storage attribution to virtual machines.
The setup of all Xen servers at BGBM should be as similar as possible. That way, improvements on one Xen server can easily be transmitted to any others. Synchronising these different hosts will be done through the version management tool Subversion For reasons of precaution, some adoptions to the usual file system must be done to enable version management with different repository locations.
The following sections shall serve as documentation and setup guide for the installation of similar Xen server hosts.
Debian System Installation¶
Please follow the instructions of the Debian Linux Installation Guide.
Xen Installation¶
Based on a core Debian System 64 bit (amd64) installation, use apt-get to install the required software packages as follows:
apt-get install xen-linux-system-2.6.26-2-xen-amd64 xen-utils-3.2-1 xenstore-utils xenwatch
All Xen servers are configured for Xen network bridging, which allows direct internet access and individual IP address assignment to all virtual machines hosted on those servers. To configure Xen network bridging, ensure the following settings in /etc/xen/xend-config.sxp:
(network-script network-bridge) ... (vif-script vif-bridge) ... #(network-script network-dummy)
Now we can reboot the system to use the xen kernel!
reboot
If the reboot was succesful, entering the following command should produce the subsequent output:
# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 3922 2 r----- 5.9
Xen-Shell Installation¶
Xen-Shell provides an easy to use xen administration shell for virtual machines managed by dedicated users. These users can access the Xen server host using SSH only. Therefore, beside xen-shell, also software package sudo is required to enable those users for running the corresponding xen commands with root user rights.
apt-get install xen-shell sudo
Next, we must use the command visudo to edit the configuration of sudo.
If you like to change root's system editor e.g. to vi, run the following command and select your favorite editor.
update-alternatives --config editor
Now, running visudo will open /etc/sudoers with your favorite editor. Adding the following lines would enable the users listed in the user alias XENUSERS (e.g. vmadmin1 and vmadmin2) to run the Xen commands specified with XEN and XENIMG without entering a password.
########################### #### Xen-Shell entries #### ########################### User_Alias XENUSERS = vmadmin1,vmadmin2 Cmnd_Alias XEN = /usr/sbin/xm Cmnd_Alias XENIMG = /usr/bin/xen-create-image XENUSERS ALL = NOPASSWD: XEN,XENIMG
Next, add any users listed in XENUSERS to your system, and make xen-shell their login shell
adduser vmadmin1 chsh -s /usr/bin/xen-login-shell vmadmin1
After that, ask the xen-shell users for their SSH public keys and repeat the following steps for each newly created user.
First, create a subdirectory .ssh and the file .ssh/authorized_keys. Edit authorized_keys and paste the content of the submitted SSH public key file into. Take care that the line with the public key is not splitted, i.e., the public key must be in a single line.
mkdir -p /home/vmadmin1/.ssh vi /home/vmadmin1/.ssh/authorized_keys Cut&Paste the public Key: ssh-rsa AAAAB3NzaC1yc2EAGwMwjHTaOpXBBT9FMMNTJtqYIULkxhthzxl1/f4Zo0K4sMOmtivUVUSYZPpnBtfHGX9AkGOs8My5f8/KdRCTqimw1lYz1QOPdb9nnmIC4NqfEA++OlQ92Jz9tft3CFncL+8v61BTygjulV/1a4Vx+KGD2LsKoBxLKxFVRq+Py8TqR5uvx6lN/KdRCTqimw1lYz1QOPdb9nVblQ5YDVKR0YCRuW2GO4xG2umFEQok7Pw8L8Ma/Zog4dWMeT41X0pzoOT5Ecf3YIFXyTw== vmadmin1
Then, set appropriate file and directory access rights to the home and .ssh directory of these users
chown -R vmadmin1:vmadmin1 /home/vmadmin1 chmod 700 /home/vmadmin1/.ssh chmod 600 /home/vmadmin1/.ssh/authorized_keys
Ok, having added the users, the last step is to restart the sudo daemon
/etc/init.d/sudo restart
Now, if a users connects to the Xen server host via SSH, xen-shell should attempt to start.
Note: xen-shell works in cooperation with Xen's virtual machines configuration files. That means, for each virtual machine running on your system, a line denoting the user names enabled for managing the given virtual machine must be added to the relating xxx.cfg file in the /etc/xen/ directory.
xen_shell = 'vmadmin1,vmadmin2'
If the connecting user had not been added to any existing virtual maching configuration file, xen-shell would simply abort and close the connection!
Xen-Tools Installation¶
Xen-tools is a script collection allowing to easily create new Xen virtual machines. It can be installed to Xen server hosts running the following command
apt-get install xen-tools
Xen-tools on BGBM Xen servers¶
For the BGBM Xen server hosts, xen-tools has been extended and adopted to our individual needs. For instance, more scripts and script libraries have been added to create, backup and manage virtual machines as well as for automatically setting up machines for specific roles (e.g. Shibboleth IdP and SP, Drupal hosts etc.).
Therefore, the BGBM Xen server hosts use a dedicated partitions for xen-tools, which is mounted to the file system at /xen. Here is how it was created.
First, a logical volume "xen" was created and formatted with an ext3 file system. Next, create the directory "/xen" and mount the logical volume "xen" to that directory
Note: Adopt the volume group name "vg" and the size of the logical volume ( "-L"parameter of lvcreate command ).
lvcreate -L 20G -n xen vg mkfs.ext3 /dev/vg/xen mkdir /xen mount /dev/vg/xen /xen
In order to mount the "xen" logical volume at boot time to the /xen directory add the following lines to the file /etc/fstab:
/dev/vg/xen /xen ext3 defaults 1 2
Next, the following directory hierarchie has been grown up below the "/xen" directory:
| backup | stores vm backups by vm hostnames |
| boot | stores Linux Kernels needed for booting by distribution and system architecture |
| etc/xen-tools | stores anything related to run xen-tools |
| etc/xen-tools/hooks | stores script libraries |
| etc/xen-tools/lib | stores script libraries |
| etc/xen-tools/partitions.d | stores script libraries |
| etc/xen-tools/resources | stores resources already downloaded from the web |
| etc/xen-tools/role.d | stores resources already downloaded from the web |
| etc/xen-tools/skel | stores skeletons for vm configurations |
| images | stores basic images of vm prototypes |
| mnt | provides mount points for individual vm file systems |
| sbin | stores extended scripts for xen management |
| skel | stores skeletons for the Xen server host |
Now, we touch the original xen-tools installation and redirected any necessary files to our xen file hierarchie on /xen.
First, xen-tools comes with some scripts called hooks, which will be executed whenever a new virtual machine is being created. As we want to control this ourselves, we need to copy some of them and create some links pointing to the file hierachie on "/xen".
For the Debian Lenny distribution, create a lenny.d directory in our hooks directory, copy the original file into and relink the original lenny.d to /xen/etc/xen-tools/hooks/lenny.d.
mkdir -p /xen/etc/xen-tools/hooks/lenny.d cp -av /usr/lib/xen-tools/debian.d/* /usr/lib/xen-tools/lenny.d/ unlink /usr/lib/xen-tools/lenny.d ln -s /xen/etc/xen-tools/hooks/lenny.d/ /usr/lib/xen-tools/lenny.d
Creation of a base Debian Etch Xen Domain with xen-tools¶
Now, we start with the domain creation. The aim is, to set up a configuration, where any domain can be recreated from the scratch using a single command. This is a little bit complicated, and we have to do some preparational work. So, let's start configuring the creation of a base Debian Etch system with an accessible OpenSSH access for root.
First, we need to edit the xen-tools configuration file. Create the directory /xen/etc/xen-tools and copy the original /etc/xen-tools/xen-tools.conf and /etc/xen-tools/xm.tmpl into it:
# mkdir /xen/etc/xen-tools # cp /etc/xen-tools/xen-tools.conf /xen/etc/xen-tools/base.conf # cp /etc/xen-tools/xm.tmpl /xen/etc/xen-tools/base.tmpl
We would like to
create LVM volumes for every domain (option lvm)
install Debian etch via network (option debootstrap)
set default sizes for the system and swap lvm volume (options size and swap )
set default size of memory (RAM) (option memory)
set the filesystem type (option fs)
set the Linux distribution to install (option dist)
configure the network parameters (options ip, netmask and gateway)
enable prompting for root password
set the boot kernel and initrd (options kernel and initrd)
set the default mirror for devootstrap
enable debug messages (option verbose)
enable harddisks be attached as ide drives (/dev/hdax, option ide)
specify the role of the domain (option role)
specify the template file for domain startup (option template)
specify the hostname (option hostname)
For that, we have to set the following default options within /xen/etc/xen-tools/base.conf:
lvm = vg1 debootstrap = 1 swap = 128Mb # Swap size size = 512Mb # Disk image size. memory = 64Mb # Memory size fs = ext3 # use the EXT3 filesystem for the disk image. dist = etch # Default distribution to install. image = full # Specify sparse vs. full disk images. ip = "10.0.0.10" netmask = 255.0.0.0 gateway = 10.0.0.254 passwd = 1 kernel = /boot/vmlinuz-2.6.18-4-xen-686 initrd = /boot/initrd.img-2.6.18-4-xen-686 mirror = http://ftp.de.debian.org/debian/ verbose = 1 ide = 1 role = base template = /xen/etc/xen-tools/base.tmpl hostname = base.wp5demo.org
For this basic installation, we can leave the template file /xen/etc/xen-tools/base.tmpl unchanged. This file will be used by the xen-create-image command to create a startup configuration file (in /etc/xen/) in order to be interpreted by the xend in order to create and start the relating virtual machine.
Further, the xen-tools configuration comprises a role.d directory, where templates for specific domain roles (e.g. base system, web-server, IdP) can be defined. These "roles" are executable shell scripts, where the existing role minimal provides a template for a minimal debian system installing and removing several packages. For the time being, we define a base role for our needs and store the relating file in the directory /xen/etc/xen-tools/role.d. Then, we can link this file within the regular xen-tools configuration directory
# mkdir /xen/etc/xen-tools/role.d # vi /xen/etc/xen-tools/role.d/base
Currently, we source the minimal role skript and extend it by installing the openssh-server package. Later on, we can extend this skript by removing or installing further packages, or doing other things.
#!/bin/sh
#
# Source the minimal role
#
prefix=$1
if [ -e ./minimal ]; then
. ./minimal
else
. /etc/xen-tools/role.d/minimal
fi
installDebianPackage ${prefix} openssh-server
Next, we create a skeleton directory for our base role.
# mkdir -p /xen/etc/xen-tools/skel/base
Within this directory, we can copy any files we would like to be copied into the file system of any virtual machine created upon our base role. To configure and get access to the virtual machine via ssh, we can simply copy the these files from our dom0 instance.
# mkdir -p /xen/etc/xen-tools/skel/base/root/.ssh # cp -av /root/.ssh/authorized_keys /xen/etc/xen-tools/skel/base/root/.ssh # mkdir -p /xen/etc/xen-tools/skel/base/etc/ssh # cp -av /etc/ssh/sshd_config /xen/etc/xen-tools/skel/base/etc/ssh
In the last step, we link all these files to the right place within the /etc/xen-tools directory and start to create our first base vm with the xen-tools command xen-create-image. Therefore, we use the following little shell-script and store it as /xen/create_base :
#!/bin/sh
ROLE=base
# link the role's xen-tools config to /etc/xen-tools
unlink /etc/xen-tools/xen-tools.conf
ln -s /xen/etc/xen-tools/${ROLE}.conf /etc/xen-tools/xen-tools.conf
# link the role's sxript to /etc/xen-tools/role.d
unlink /etc/xen-tools/role.d/${ROLE}
ln -s /xen/etc/xen-tools/role.d/${ROLE} /etc/xen-tools/role.d
# link the role's file system skeleton to /etc/xen-tools/skel
unlink /etc/xen-tools/skel
ln -s /xen/etc/xen-tools/skel/${ROLE} /etc/xen-tools/skel
xen-create-image $*
Before using this script for the first time, we need to delete the regular /etc/xen/skel directory
# rm -R /etc/xen-tools/skel
Make sure that this script as well as the role skript are executable.
# chmod 755 /xen/create_base # chmod 755 /xen/etc/xen-tools/role.d/base
Now, we are ready to create an instance of our first virtual machine executing our script:
# /xen/create_base
During the script execution, you should be prompted for a password for the root user! You can follow the installation process observing the corresponding log-file:
# tail -f /var/log/xen-tools/base.wp5demo.org.log
After the script has finished, you can start the vm with the following command:
# xm create /etc/xen/base.wp5demo.org.cfg -c
Firewall setup¶
Using Xen's NAT network configuration, connecting our virtual machines e.g. via the ssh protocol, we have to configure the firewall of our dom0 to forward e.g. port 110 to port 22 of our virtual machine. Therefore, store the following script in /etc/network/if-up.d/iptables and make it executable (chmod 755).
#!/bin/sh ### Clean up rules ### /sbin/iptables -F ### Nothing in ### /sbin/iptables -P INPUT DROP ### All out ### /sbin/iptables -P OUTPUT ACCEPT ### Accept all localhost traffic ### /sbin/iptables -A INPUT -j ACCEPT -i lo ### Accept all icmp traffic ### /sbin/iptables -A INPUT -j ACCEPT -p icmp ### Accept all responses ### /sbin/iptables -A INPUT -j ACCEPT -m conntrack --ctstate ESTABLISHED,RELATE ### Accept Incoming Ports ### /sbin/iptables -A INPUT -j ACCEPT -p tcp --dport ssh /sbin/iptables -A INPUT -j ACCEPT -p tcp --dport http ### Accept SSH-Port for base.wp5demo.org ### /sbin/iptables -A INPUT -j ACCEPT -p tcp --dport 110 ### Forwarding Ports ### ### Forward SSH-Port for base.wp5demo.org ### /sbin/iptables -A PREROUTING -t nat -p tcp -i eth0 --dport 110 -j DNAT --to 10.0.0.10:22
Now you should be able to login to the base virtual machine using ssh on port 110.
Creation of further virtual machine templates¶
Based on the previous xen-tools configuration, we can proceed setting up further configurations for other purposes.
On wp5demo, all xen-tools configuration files are located in the directory /xen. The latest revision can be retrieved source:/trunk/commtools/xen.
Shibboleth Identity Provider (IdP) Installation¶
According to the step-by-step description of the Shibboleth Identity Provider (IdP) Installation on Debian Etch, a Xen-Tools role script idp is prepared, a corresponding configuration file idp.conf the skeleton directory /etc/xen-tools/skel/idp and the setup, start and shutdown scripts /xen/create_idp, /xen/start_idp and /xen/shutdown_idp are availabe.
The setup script will automatically install a fresh copy of an Shibboleth Identity Provider. Nevertheless, you will have to follow the installation procedure in a second window, observing the log file produced:
# tail -f /var/log/xen-tools/idp.e-taxonomy.eu.log
During the shibboleth installation, you have to answer five questions just pushing the __ key to accept the prepared default values.
Before finishing the installation, you have to enter the root password for the newly created IdP instance.
*Please, make the required input in the terminal, where you started the ./create_idp skript !
Making input into the log windows will not have any effect !
*
The current instance is just prepared to be tested against the !TestShib service.
Further configurations will adept this installation to the final IdP instance for the EDIT Cyber Taxonomy Network.
Shibboleth Service Provider (SP) Installation¶
Regarding the installation procedure,please refer to the step-by-step description Shibboleth Service Provider (SP) Installation on Debian Etch. For the xen-tools, the role script sp, the configuration and template files sp.conf, the skeleton directory /etc/xen-tools/skel/sp and the setup, start and shutdown scripts /xen/create_sp, /xen/start_sp and /xen/shutdown_sp are prepared.
The setup script automatically installs a fresh copy of an Shibboleth Identity Provider. At the end of the installation script, you will be asked to enter the root password for the created sp instance.
The installation procedure can be observed via the installation log file produced:
# tail -f /var/log/xen-tools/sp.e-taxonomy.eu.log
The current instance is just prepared to be tested against the !TestShib service.
Further configurations will adept this installation to the final IdP instance for the EDIT Cyber Taxonomy Network.
Updated by Lutz Suhrbier over 14 years ago · 31 revisions