BGBM Xen server documentation¶
- Table of contents
- BGBM Xen server documentation
This document describes the general setup Xen servers hosting virtual machines (VMs) at BGBM. Currently, the following servers are configured as Xen servers:
wp5demo.e-taxonomy.eu (160.45.63.20)
bgbm45.bgbm.fu-berlin.de (160.45.63.135)
All Xen installations base on the Linux distribution Debian 5 (Lenny) for the 64 bit architecture (amd64). Debian Lenny provides packages for Xen version 3.2.1. That way, virtual machines may run on 64 bit or 32 bit architectures alternatively.
Regarding partition management, all Xen servers use Logical Volume Manager (LVM2) LVM permits flexible and individual storage attribution to virtual machines.
The setup of all Xen servers at BGBM should be as similar as possible. That way, improvements on one Xen server can easily be transmitted to any others. Synchronising these different hosts will be done through the version management tool Subversion For reasons of precaution, some adoptions to the usual file system must be done to enable version management with different repository locations.
The following sections shall serve as documentation and setup guide for the installation of similar Xen server hosts.
Debian System Installation¶
Please follow the instructions of the Debian Linux Installation Guide.
Xen Installation¶
Based on a core Debian System 64 bit (amd64) installation, use apt-get to install the required software packages as follows:
apt-get install xen-linux-system-2.6.26-2-xen-amd64 xen-utils-3.2-1 xenstore-utils xenwatch
All Xen servers are configured for Xen network bridging, which allows direct internet access and individual IP address assignment to all virtual machines hosted on those servers. To configure Xen network bridging, ensure the following settings in /etc/xen/xend-config.sxp:
(network-script network-bridge) ... (vif-script vif-bridge) ... #(network-script network-dummy)
Now we can reboot the system to use the xen kernel!
reboot
If the reboot was succesful, entering the following command should produce the subsequent output:
# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 3922 2 r----- 5.9
Xen-Shell Installation¶
Xen-Shell provides an easy to use xen administration shell for virtual machines managed by dedicated users. These users can access the Xen server host using SSH only. Therefore, beside xen-shell, also software package sudo is required to enable those users for running the corresponding xen commands with root user rights.
apt-get install xen-shell sudo
Next, we must use the command visudo to edit the configuration of sudo.
If you like to change root's system editor e.g. to vi, run the following command and select your favorite editor.
update-alternatives --config editor
Now, running visudo will open /etc/sudoers with your favorite editor. Adding the following lines would enable the users listed in the user alias XENUSERS (e.g. vmadmin1 and vmadmin2) to run the Xen commands specified with XEN and XENIMG without entering a password.
########################### #### Xen-Shell entries #### ########################### User_Alias XENUSERS = vmadmin1,vmadmin2 Cmnd_Alias XEN = /usr/sbin/xm Cmnd_Alias XENIMG = /usr/bin/xen-create-image XENUSERS ALL = NOPASSWD: XEN,XENIMG
Next, add any users listed in XENUSERS to your system, and make xen-shell their login shell
adduser vmadmin1 chsh -s /usr/bin/xen-login-shell vmadmin1
After that, ask the xen-shell users for their SSH public keys and repeat the following steps for each newly created user.
First, create a subdirectory .ssh and the file .ssh/authorized_keys. Edit authorized_keys and paste the content of the submitted SSH public key file into. Take care that the line with the public key is not splitted, i.e., the public key must be in a single line.
mkdir -p /home/vmadmin1/.ssh vi /home/vmadmin1/.ssh/authorized_keys Cut&Paste the public Key: ssh-rsa AAAAB3NzaC1yc2EAGwMwjHTaOpXBBT9FMMNTJtqYIULkxhthzxl1/f4Zo0K4sMOmtivUVUSYZPpnBtfHGX9AkGOs8My5f8/KdRCTqimw1lYz1QOPdb9nnmIC4NqfEA++OlQ92Jz9tft3CFncL+8v61BTygjulV/1a4Vx+KGD2LsKoBxLKxFVRq+Py8TqR5uvx6lN/KdRCTqimw1lYz1QOPdb9nVblQ5YDVKR0YCRuW2GO4xG2umFEQok7Pw8L8Ma/Zog4dWMeT41X0pzoOT5Ecf3YIFXyTw== vmadmin1
Then, set appropriate file and directory access rights to the home and .ssh directory of these users
chown -R vmadmin1:vmadmin1 /home/vmadmin1 chmod 700 /home/vmadmin1/.ssh chmod 600 /home/vmadmin1/.ssh/authorized_keys
Ok, having added the users, the last step is to restart the sudo daemon
/etc/init.d/sudo restart
Now, if a users connects to the Xen server host via SSH, xen-shell should attempt to start.
Note: xen-shell works in cooperation with Xen's virtual machines configuration files. That means, for each virtual machine running on your system, a line denoting the user names enabled for managing the given virtual machine must be added to the relating xxx.cfg file in the /etc/xen/ directory.
xen_shell = 'vmadmin1,vmadmin2'
If the connecting user had not been added to any existing virtual maching configuration file, xen-shell would simply abort and close the connection!
Xen-Tools Installation¶
Xen-tools is a script collection allowing to easily create new Xen virtual machines. It can be installed to Xen server hosts running the following command
apt-get install xen-tools
Extended Xen-tools installation¶
For the BGBM Xen server hosts, xen-tools has been extended and adopted to our individual needs. For instance, more scripts and script libraries have been added to create, backup and manage virtual machines as well as for automatically setting up machines for specific roles (e.g. Shibboleth IdP and SP, Drupal hosts etc.).
Therefore, the BGBM Xen server hosts use a dedicated partitions for xen-tools, which is mounted to the file system at /xen. Here is how it was created.
First, a logical volume "xen" was created and formatted with an ext3 file system. Next, create the directory "/xen" and mount the logical volume "xen" to that directory
Note: Adopt the volume group name "vg" and the size of the logical volume ( "-L"parameter of lvcreate command ).
lvcreate -L 20G -n xen vg mkfs.ext3 /dev/vg/xen mkdir /xen mount /dev/vg/xen /xen
In order to mount the "xen" logical volume at boot time to the /xen directory add the following lines to the file /etc/fstab:
/dev/vg/xen /xen ext3 defaults 1 2
The most recent revision of the extended xen-tools can be retrieved from the following subversion repository http://dev.e-taxonomy.eu/svn/trunk/security/xen.
You can install subversion with the following command to your host:
apt-get install subversion
In order to get a fresh copy of the system, the following command gets the latest revision from the subversion repository and copies it below the /xen directory
svn checkout http://dev.e-taxonomy.eu/svn/trunk/security/xen /xen
Now, the following directory hierarchie should have been created below the "/xen" directory:
| backup | stores vm backups by vm hostnames |
| boot | stores Linux Kernels needed for booting by distribution and system architecture |
| etc/xen-tools | stores anything related to run xen-tools |
| etc/xen-tools/hooks | stores script libraries |
| etc/xen-tools/lib | stores script libraries |
| etc/xen-tools/partitions.d | stores script libraries |
| etc/xen-tools/resources | stores resources already downloaded from the web |
| etc/xen-tools/role.d | stores resources already downloaded from the web |
| etc/xen-tools/skel | stores skeletons for vm configurations |
| images | stores basic images of vm prototypes |
| mnt | provides mount points for individual vm file systems |
| sbin | stores extended scripts for xen management |
| skel | stores skeletons for the Xen server host |
Next we touch the original xen-tools installation and redirected any necessary files to our xen file hierarchie on /xen.
First, xen-tools comes with some scripts called hooks, which will be executed whenever a new virtual machine is being created. As we want to control this ourselves, we need to copy some of them and create some links pointing to the file hierachie on "/xen".
For the Debian Lenny distribution, create a lenny.d directory in our hooks directory, copy the original file into and relink the original lenny.d to /xen/etc/xen-tools/hooks/lenny.d.
mkdir -p /xen/etc/xen-tools/hooks/lenny.d cp -av /usr/lib/xen-tools/debian.d/* /usr/lib/xen-tools/lenny.d/ unlink /usr/lib/xen-tools/lenny.d ln -s /xen/etc/xen-tools/hooks/lenny.d/ /usr/lib/xen-tools/
Since, we have some other requirements, the following modifications were made to the hook scripts.
| 35-setup-users | deactivated ( "chmod -x /xen/etc/xen-tools/hooks/lenny.d/35-setup-user" |
| 60-copy-host-files | modified, copying of Xen server sudo configuration deactivated |
| 65-copy-user-files | deactivated (chmod -x /xen/etc/xen-tools/hooks/lenny.d/65-copy-user-files") |
| 95-configure-locales | deactivated (chmod -x /xen/etc/xen-tools/hooks/lenny.d/95-configure-locales"), locale setup in minimal role |
Any of the newly introduced xen scripts can be adopted to individual system configurations (e.g. LVM volume group name) by environment variables. The usual place for those system wide configurations is /etc/profile. For better distinguishing system relevant configurations and global xen configurations, any global configurations necessary for our xen-tools setup will be located below /xen/skel/host. Symbolic links will be created in the system pointing at the corresponding locations within the xen host skeleton. System relevant configuration will be added or modified in the original system configuration files on the Xen server host (e.g. /etc).
The main intention of these configuration issues is for version management. That way, individual system configurations and global xen configurations can be strictly separated and managed within different repositories. Also, modifications to our xen-tools development can be easily upgraded using subversion and does not bother individual system configurations.
Here are the most relevant modifications:
/etc/profile¶
This file must be extended by the following lines:
. /etc/profile.lvm . /etc/profile.xen
/etc/profile.lvm
defines environment variables for system individual LVM settings. The LVM_VG variable must be set to the local volume group used for xen virtual machines.
export LVM_PROFILE=/etc/profile.lvm export LVM_VG='vg1'
/etc/profile.xen
includes any required environment variables for the extended xen-tools. Herein, only the following variables may be subject of modification
# base path of the xen script environment export XENBASE_PATH='/xen' # path to the original xen-tools distribution's configuration directory export DIST_XENTOOLS_PATH=/etc/xen-tools
Anyway, only /etc/profile and /etc/profile.xen should be linked to the Xen server host skeleton. Rename, copy or delete the file before linking!
ln -s /xen/skel/host/etc/profile /etc/ ln -s /xen/skel/host/etc/profile.xen /etc/
Also, the following links to the xen system should be created
ln -s /xen/skel/host/etc/hosts /etc/ ln -s /xen/skel/host/etc/ssh/sshd_config /etc/ssh ln -s /xen/skel/host/root/.bashrc /root
Please find more documentation on how to use the extended Xen-tools here.
Firewall setup (TODO)¶
Using Xen's NAT network configuration, connecting our virtual machines e.g. via the ssh protocol, we have to configure the firewall of our dom0 to forward e.g. port 110 to port 22 of our virtual machine. Therefore, store the following script in /etc/network/if-up.d/iptables and make it executable (chmod 755).
#!/bin/sh ### Clean up rules ### /sbin/iptables -F ### Nothing in ### /sbin/iptables -P INPUT DROP ### All out ### /sbin/iptables -P OUTPUT ACCEPT ### Accept all localhost traffic ### /sbin/iptables -A INPUT -j ACCEPT -i lo ### Accept all icmp traffic ### /sbin/iptables -A INPUT -j ACCEPT -p icmp ### Accept all responses ### /sbin/iptables -A INPUT -j ACCEPT -m conntrack --ctstate ESTABLISHED,RELATE ### Accept Incoming Ports ### /sbin/iptables -A INPUT -j ACCEPT -p tcp --dport ssh /sbin/iptables -A INPUT -j ACCEPT -p tcp --dport http ### Accept SSH-Port for base.wp5demo.org ### /sbin/iptables -A INPUT -j ACCEPT -p tcp --dport 110 ### Forwarding Ports ### ### Forward SSH-Port for base.wp5demo.org ### /sbin/iptables -A PREROUTING -t nat -p tcp -i eth0 --dport 110 -j DNAT --to 10.0.0.10:22
Now you should be able to login to the base virtual machine using ssh on port 110.
Updated by Andreas Müller almost 2 years ago · 43 revisions