XenServer Installation Guide

Release 4.1.0

Table of Contents

Introducing XenServer
About this document
How this Guide relates to other documentation
What's new in XenServer 4.1.0
What is XenServer?
Xen: the engine that powers XenServer
XenServer extends the power of Xen virtualization to server clusters
Powerful VM storage management and clustering
XenMotion™ delivers an agile virtual infrastructure
The XenServer product family
XenServer elements
System Requirements
XenServer Host system requirements
XenCenter requirements
VM support
Installing XenServer
Installing the XenServer Host
Installing XenCenter
Installation and deployment scenarios
XenServer Hosts with local storage
XenServer Hosts with shared NFS storage
XenServer Hosts with shared iSCSI storage
A. Troubleshooting
B. Maintenance Procedures
Preparing XenServer Hosts for maintenance operations
Applying updates
Applying updates using the CLI
Reinstalling the same version
Rolling upgrade from version 4.0.1 to the current version
Backup
Performing a rolling pool upgrade
Upgrading from version 3.2 to 4.0.1
Upgrading VHD files from version 4.0.1
Backing up and restoring XenServer Hosts and VMs
Backing up Virtual Machine metadata
Backing up XenServer Hosts
Backing up VMs
C. PXE installation of XenServer Host
Setting up the PXE boot environment
Creating an answerfile for unattended PXE installation
Installation media repository format
Presence of installation media repositories
Installation media repository metadata
Package metadata
Example files
Notes on best practice
D. Xen Memory Usage
Memory scaling algorithm
Increasing the reserved memory size

Thank you for choosing XenServer™ from Citrix Systems, Inc., the creators of the Xen® hypervisor and leaders of the open source Xen project.

XenServer is a server virtualization platform that offers near bare-metal virtualization performance for server and client operating systems. XenServer uses the Xen hypervisor to virtualize each server on which it is installed, enabling each to host multiple Virtual Machines simultaneously with guaranteed performance.

XenServer allows you to combine multiple Xen-enabled servers into a resource pool, using industry-standard shared storage architectures and leveraging resource clustering technology created by Citrix. In doing so, XenServer extends the basic single-server notion of virtualization to enable seamless virtualization of multiple servers as a resource pool, whose storage, memory, CPU and networking resources can be dynamically controlled to deliver optimal performance, increased resiliency and availability, and maximum utilization of data center resources.

XenCenter allows IT managers to create multiple clusters of resource pools, and to manage them and their resources from a single point of control, reducing complexity and cost, and dramatically simplifying the adoption and utility of a virtualized data center environment. With XenServer, a rack of servers can become a highly available compute cluster that protects key application workloads, leverages industry standard storage architectures, and offers no-downtime maintenance by allowing Virtual Machines to be moved while they are running between machines in the cluster.

XenServer extends the most powerful abstraction: virtualization across servers, storage and networking to enable users to realize the full potential of a dynamic, responsive, efficient data center environment for Windows and Linux workloads.

By providing a unified view of the resources of one or more clusters of servers, and through its use of a standardized abstraction for control of storage resources assigned to Virtual Machines, XenServer dramatically simplifies the job of the IT administrator seeking a painless solution for virtualization of demanding production workloads. XenServer is ideally suited to users seeking to maximize the benefits of server consolidation, automate test and development of software, or automate the assignment of resources and protection of performance-sensitive production workloads.

Xen provides fast, secure, open source virtualization that allows multiple operating system instances to run as Xen Virtual Machines or VMs on a single physical 64-bit x86 computer. Xen supports modified guest operating systems using a technique known as paravirtualization, which requires modifying the operating system to run on Xen, but offers near-native performance. Paravirtualized operating systems "know" that they are virtualized. Xen also supports unmodified operating systems using processor extensions from Intel (VT) and AMD (AMD-V).

Xen supports 32-bit processors with and without Physical Address Extension (PAE), 64-bit processors, and Symmetric Multiprocessing (SMP) guest operating systems.

Xen is exceptionally lean, which leads to extremely low overhead and near-native performance for VMs. Xen re-uses existing Linux device drivers for Linux VMs, and uses special paravirtualized device drivers for network and disk I/O on Windows VMs, making device management easy. Moreover, Xen is robust in the event of device driver failure and protects VMs, and also protects the hypervisor from faulty or malicious drivers.

Xen provides superb resource partitioning for CPU, memory, and block and network I/O. This resource protection model leads to improved security because VMs and drivers are not susceptible to denial of service attacks. Xen is fully open to scrutiny by the security community and its security is continuously tested. Xen is also the foundation for a Multi-Level Secure System architecture being developed by Citrix, IBM and Intel.

Xen was originally developed by the Systems Research Group at the University of Cambridge Computer Laboratory as part of the XenoServers project, funded by the Engineering and Physical Sciences Research Council (EPSRC), the main funding agency in the United Kingdom for research in engineering and the physical sciences as well as the managing agent on behalf of the other Research Councils for High Performance Computing.

XenServer allows IT administrators to flexibly assign up to 16 64-bit x86 servers into a single resource pool of server resources. Multiple pools can be managed from a single XenCenter management console. A resource pool is a tightly coupled collection of servers whose resources are virtualized to host a set of Virtual Machines. Servers in a resource pool monitor the configuration state and availability of their peers. XenServer management state is also replicated across all servers in a pool, with the benefit that failure of a pool master can be quickly remedied, since any node in the cluster can replace the failed node. Using the XenServer clustering architecture, the workload of a cluster can be protected from server failures, through a unique combination of shared storage, Xen virtualization, and replicated state management between servers in the cluster.

Virtual Machines assigned to a resource pool are automatically mapped onto the physical resources of the pool, but IT administrators retain full control of resource assignment, and full visibility into each system and each Virtual Machine, including the ability to manually place workload on specific servers, and drill down into each server within the pool to get a precise view of each server's resources and the Virtual Machines it hosts. At the simplest level, all the administrator needs to do is assign a Virtual Machine or a set of Virtual Machines to a resource pool. XenServer manages the rest, including the assignment of physical resources from servers in the pool to host the VMs, and ensuring that administrator policies for resilient restart of VMs are implemented. XenServer ensures that the overall utilization of the resources of the servers in the pool is maximized, to deliver lowest possible TCO.

Of course, if you want to assume full control, XenServer gives you the ability to manage each resource for each VM, but most users will appreciate the simplicity of the "drag and drop" interface for VM provisioning with guaranteed VM performance, automated VM storage and network management, and the use of policies for automatic restart on failure of physical components of the cluster.

In most datacenters, storage is managed as a shared, separately administered resource independent of the different server applications and OS types that make use of it. The rich set of choices for datacenter storage, and the emergence in its own right of storage virtualization as a powerful technology that reduces TCO for storage, leaves IT managers with a bewildering set of choices for storage and storage management. XenServer aims to simplify the management of diverse storage technologies for virtualized infrastructure. It does this by

XenServer, uniquely amongst all virtualization products on the market, offers an open API to integrate directly with the various kinds of storage infrastructures available. With built-in support for IDE, SATA, SCSI and SAS drives, XenServer can manage all forms of storage local to any server in a resource pool. Through NAS, iSCSI and SAN support, XenServer extends the available Virtual Machine storage options to the most common datacenter architectures in use today, while providing plug-in APIs for storage vendors to integrate any storage management technology, from clustered file systems, through clustered volume management. Storage repositories (or SRs) are elemental components of the XenServer architecture. All are managed via the XenServer API, and through this API XenServer can leverage the built-in features of the underlying storage infrastructure, including snapshotting, backup, and automated creation and assignment of LUNs for new Virtual Machines. Not all SRs support all primitives - but XenServer can accommodate this by adding software-level features that can be used if the storage infrastructure cannot support a particular primitive, such as snapshotting.

XenServer requires at least two separate physical x86 computers: one to be the XenServer Host, and the other to run the XenCenter application. The XenServer Host machine is dedicated entirely to the task of hosting VMs and is not used for other applications. The computer that runs XenCenter can be any general-purpose Windows computer that satisfies the hardware requirements, and can be used to run other applications simultaneously.

The XenServer Host is a 64-bit x86 server-class machine devoted to hosting multiple VMs. This machine runs a stripped-down Linux operating system with a Xen-enabled kernel which controls the interaction between the virtualized devices seen by VMs and the physical hardware.

The following are the system requirements for the XenServer Host:

Any XenServer network, from the simplest to the most complex deployment, is made up of one or more XenServer Hosts, each running some number of VMs, and one or more workstations running XenCenter to administer the XenServer Hosts.

In order to create resource pools and enable XenMotion (live migration of VMs), a means of shared storage also needs to be deployed on the network. This version of the XenServer product family supports Fibre Channel, NetApp filers, LVM over iSCSI, and NFS shared storage.

This chapter describes installing XenServer Host software on physical servers, installing XenCenter on Windows workstations, and connecting them to form the infrastructure for a network of Virtual Machines.

The first sections describe the installation of XenServer Host and XenCenter, which are common to all deployments. The following sections describe several common installation and deployment scenarios and provide information that is specific to each scenario.

Installers for both the XenServer Host and XenCenter are on the installation media. The installation media also includes:

The XenServer Host consists of a Xen-enabled Linux operating system, a management agent, VM templates, and a local storage repository reserved for VMs. The XenServer Host must be installed on a dedicated 64-bit x86 server. XenServer is not supported in a dual-boot configuration with any other operating system.

You can install the XenServer Host from the installation CDs or set up a network-accessible TFTP server to boot from via PXE. For details about setting up a TFTP server for PXE-booting the installer, see Appendix C, PXE installation of XenServer Host.

Note

Do not install any other operating system in a dual-boot configuration with the XenServer Host; this is an unsupported configuration.

The main installation CD contains the basic packages to set up the XenServer Host on a physical host, and to create Windows VMs by using the Windows installation CDs. The XenServer package also contains a separate CD containing support for creating Linux VMs (including complete built-in distributions of Debian Sarge and Etch), and six CDs containing source code for the included open-source software.

If you want to install Linux VMs, be sure to

  1. download the Linux Pack ISO
  2. burn it to a physical CD if installing from a DVD/CD drive, or set it up for PXE installation as described in Appendix C, PXE installation of XenServer Host
  3. insert the CD when prompted for the Linux Pack during XenServer Host installation

    Note

    If you decide later to add Linux support, mount the Linux Pack installation CD or ISO image on the XenServer Host and run the script install.sh, located in the root of the CD.

To install the XenServer Host

When booting the XenServer Host after installation, the message "Loading GRUB, please wait" is displayed, followed by several "Press any key to continue" messages. This allows you to press a key on either the physical console or a serial console and let GRUB know where to direct its output. If you wait, the GRUB menu is automatically directed to the physical console.

Note that on some machines, the BIOS causes the pause between these messages to be unusually long, such that it takes over a minute for it to automatically direct to the physical console.

  1. Boot the computer from the main installation CD, or PXE-boot from your TFTP server if applicable (see Appendix C, PXE installation of XenServer Host for details on how to set up the XenServer media for PXE installation).
  2. After the initial boot messages, the installer does some hardware detection and initialization, then presents a screen asking you to select which keyboard keymap you want to use for the installation. In this and the screens that follow, use Tab or Alt+Tab to move between elements, Space to select, and F12 to move to the next screen.Select the desired keymap and choose OK to proceed.
  3. Next, the "Welcome to XenServer" screen appears. Select Install or upgrade XenServer Host and choose OK to proceed.
  4. The next screen displays a message telling you that the setup program will install XenServer on the computer, and warns that it will overwrite data on any hard drives that you select to use for the installation. Choose OK to proceed.
  5. The XenServer End User License Agreement (EULA) is displayed. Use the up and down arrow keys to scroll through and read the agreement. Choose Accept EULA to proceed.
  6. At this point, if the computer on which you are installing the XenServer Host does not have a CPU which supports hardware virtualization, or if the support is disabled in the BIOS, a message appears to warn you that you will not be able to run Windows VMs. Choose OK to proceed.Note that some systems have bugs in their BIOS software which can result in the setting being incorrect. If you get a spurious warning about a lack of hardware virtualization (or do not see a warning when you expected one), then perform a hard power cycle of the host and restart the installation. You should also check the hardware manufacturer's support site for BIOS upgrades.
  7. If the installer detects a previously-installed version of XenServer Host, you are offered the choice to perform a clean installation, or to upgrade the existing version, which preserves any of the VMs present. Select an installation type and choose OK to proceed.If you selected to upgrade an existing version, you will get a message that the installer is going to create a backup of the existing installation. Select Continue to proceed.
  8. If you have multiple local hard disks, you are asked to choose the Primary Disk for the installation. Select the desired disk and choose OK to proceed. After selecting the primary one, you are also prompted to choose if you want any of the other drives to be formatted for use by XenServer for VM storage. Select and choose OK to proceed.If the computer has a single hard disk, these two screens do not appear.
  9. The next screen asks you to specify the source of the installation packages. If you are installing off the CD, you will most likely want to select Local media. If you are installing via PXE you will most likely want to select HTTP or FTP or NFS, as appropriate.If you select HTTP or FTP or NFS, you are next prompted to set up Networking so that the installation script can connect to the product repository.If the computer has multiple network interfaces, you are prompted to select one of them to be used to access the XenServer product repository. Select and choose OK to proceed.If the computer has a single network interface, that interface is used to access the XenServer product repository and no prompt is displayed.You can select Automatic configuration (DHCP) to configure the NIC using DHCP, or Static configuration, which prompts you to configure the NIC's properties manually. Following that, you are prompted to provide the URL or NFS server and path where the installation media are, as appropriate.

    Note

    To be part of a resource pool, XenServer Hosts need to have static IP addresses.

    If you select Local media, this networking setup appears later in the installation process.The next screen asks if you want to install the Linux Pack from a second CD. If you are planning to install VMs that will run Linux operating systems, choose Yes. If you are planning to install only Windows VMs, you can choose No.

    Important

    In a pooled setup, the Linux Pack must be installed either on all of the pool XenServer Hosts, or on none of them, so that they are matched.

  10. The next screen asks if you want to verify the integrity of the installation media. If you select Verify installation source, the MD5 checksum of the packages is calculated and checked against the known value. This may take some time. If you select Skip verification, this check is bypassed. Make your selection and choose OK to proceed.
  11. You are next prompted to set a root password. (This will be the password that the XenCenter application will use to connect to the XenServer Host.) Type the desired password and type it again to verify it.
  12. You are prompted to select the general geographical area for the Time Zone. Choose from the displayed list of geographical areas, then choose OK to proceed.
  13. You are prompted to select the specific locale for the Time Zone. (Note that this list is long, but if you type the first letter of the desired locale, the selection will jump to the first entry that begins with this letter.) Choose from the displayed list of locales, then choose OK to proceed.
  14. You are prompted to choose a method of setting the System Time. You can select Using NTP or Manual time entry. Make your selection and choose OK to proceed.
  15. If you selected Using NTP in the preceding step, you are prompted to identify the time server or servers you want to use. You can check NTP is configured by my DHCP server and the time server will be set by DHCP. Otherwise, enter at least one NTP server name or IP address in the fields below. Choose OK to proceed.Otherwise, the installation script moves to the next step; you will be prompted for the manually-entered time later, near the end of the installation.

    Warning

    Currently XenServer assumes that the time setting for the server’s BIOS is the current time in UTC, and that the time for the VMs reflects the local time based on the time zone offset specified.

  16. You are next prompted to set up Networking for the management NIC, which is the interface that will be used to connect to the XenCenter.If the computer has multiple network interfaces, you are prompted to select one of them to be used as the management NIC for the XenServer Host software. Select one and choose OK to proceed.If the computer has a single network interface, that interface is used as the management NIC and no prompt is displayed.Next you can select Automatic configuration (DHCP) to configure the NIC using DHCP, or Static configuration, which prompts you to configure the NIC's properties manually.

    Note

    To be part of a resource pool, XenServer Hosts need to have static IP addresses.

  17. You are next prompted to specify the hostname and the configuration for the name service.In the Hostname Configuration section, if you select Automatically set via DHCP, the DHCP server will provide the hostname along with the IP address. If you select Manually specify, enter the desired hostname for the server in the field below.In the DNS Configuration section, if you select Manually specify, enter the IP addresses of your primary (required), secondary (optional), and tertiary (optional) Nameservers in the fields below. Otherwise, select Automatically set up via DHCP to get name service configuration via DHCP.Select OK to proceed.
  18. A message is displayed that the installation is ready to proceed and that this will format the primary disk and any other disks selected for VM storage, destroying any data that is currently on them. Select Install XenServer Host to proceed.A progress bar is displayed as the installation commences. If you chose to set the system date and time manually, a dialog box appears when the progress bar has reached about 90%. Enter the correct numbers in the fields and select OK to proceed.
  19. If you are installing from CD and selected to include support for Linux VMs, you will be prompted to put in the Linux Pack disk. Eject the main disk, put in the Linux Pack disk, and close the CD drawer. Select OK. A screen appears, identifying that this disk contains the Linux Pack. Select Use media to proceed with installing it. Another progress bar is displayed, and when it reaches 100%, a completion message is displayed.If you selected not to install support for Linux VMs, a completion message is displayed.

    Note

    If you decide later to add Linux support, mount the Linux Pack installation CD or ISO image on the XenServer Host and run the script install.sh, located in the root of the CD.

  20. Select Reboot. Upon reaching the login prompt, the system should now be ready to be managed via XenCenter. To connect to it, you will need the IP address or hostname of the XenServer Host. This is displayed at the login prompt.

XenCenter is a Windows client application. XenCenter must be installed on a remote machine that can connect to the XenServer Host through the network; it cannot run on the same machine as the XenServer Host. It can be installed and run on Windows 2003, XP SP2, or Vista. The .NET framework version 2.0 or above must be installed as well.

Should you need to, XenCenter can be uninstalled from a system quite easily.

This section describes several common installation and deployment scenarios:

and details the steps that differ between scenarios.

Adding shared storage to the XenServer network enables grouping of XenServer Hosts into resource pools, enabling live relocation of VMs and sharing of server resources.

Requirements

  • two or more 64-bit x86 servers with local storage
  • one or more Windows workstations, on same network as the XenServer Hosts
  • a server exporting a shared directory via NFS

Note

To be part of a resource pool, the XenServer Hosts and the server or servers providing the shared NFS storage need to have static IP addresses.

For this procedure, a server running a typical Linux distribution is assumed as the NFS server. Consult your Linux distribution documentation for further information.

Adding shared storage to the XenServer network enables grouping of XenServer Hosts into resource pools, enabling live relocation of VMs and sharing of server resources.

Requirements

  • two or more 64-bit x86 servers with local storage
  • one or more Windows workstations, on same network as the XenServer Hosts
  • a server providing a shared directory via iSCSI

Note

To be part of a resource pool, the XenServer Hosts and the server or servers providing the shared iSCSI storage need to have static IP addresses.

The details of how to set up iSCSI storage differ between the various iSCSI solutions on the market. In general, though, you need to provide an iSCSI target on the SAN for the VM storage, and then configure XenServer Hosts to be able to see and connect to it. This is done by providing a valid iSCSI Qualified Name (IQN) to the iSCSI target and to the iSCSCI initiator on each XenServer Host.

You can use either XenCenter or the CLI to configure the IQN for each XenServer Host and to create the SR. The following describes using the CLI; see the XenServer Help for details on using XenCenter.

If you experience odd behavior, crashes, or have other issues during installation, this chapter is meant to help you solve the problem if possible and, failing that, describes where logs are located and other information that can help your Citrix Solution Provider and Citrix track and resolve the issue.

Note

We recommend that you follow the troubleshooting information in this chapter solely under the guidance of your Citrix Solution Provider or Citrix Support.

Citrix provides two forms of support: you can receive free self-help support via the Support site, or you may purchase our Support Services and directly submit requests by filing an online Support Case. Our free web-based resources include product documentation, a Knowledge Base, and discussion forums.

The XenServer Host installation CD runs Linux, so most standard Linux commands can be used to diagnose installation problems. There are three virtual terminals available during installation, which display the installation menu, an interactive console and an event log, respectively. Use the ALT + F1-F3 keys to toggle between the virtual terminals.

You can check some basic things in the interactive terminal:

  • fdisk lists all disks that can be seen as a result of the loaded storage device drivers. If a particular device driver did not load, for example,the driver for a RAID card, then the disks attached to that card will not appear in the output from the fdisk command.
  • ifconfig shows the network configuration of physical NICs, including their IP addresses, netmasks, and gateway.
  • ping can be used to verify network connectivity from the XenServer Host to a remote IP address and vice-versa.

You should use the two additional virtual terminals solely under the guidance of your Citrix Solution Provider.

Installation logs are written to /install/tmp/

This chapter documents some miscellaneous procedures for maintaining XenServer Hosts.

Between releases of XenServer software, Citrix occasionally releases updates to the software. These updates typically contain accumulated bug fixes and feature improvements. When an update is released, it is made accessible on the Internet and an email announcement is sent to all XenServer customers.

Once downloaded, updates can be applied most readily via XenCenter, but can also be applied using the CLI. Updates are applied through the Manage Updates dialog box, under the pool menu. See the XenCenter Help for details.

Updates may have differing after-apply guidance, such as requiring the XenAPI agent to be restarted. Where possible, updates will be such that they can be applied without interruption, but in some cases they may require system, or virtual machine restarts to be performed. In cases where a system restart is required, users can avoid downtime of virtual machines in a pooled environment by applying the update to each server in turn, migrating virtual machines away from each server in turn as the update is applied. XenCenter can take care of this update sequence automatically on your behalf via the Manage Updates option. If you are using the CLI, you will have to do this manually using the host-evacuate command.

If using the CLI to perform the update, XenServer Hosts to be updated should be prepared for this operation by the procedures in the section called “Preparing XenServer Hosts for maintenance operations”. If using XenCenter, this will be taken care of automatically where required.

First, the update must be uploaded to the pool or server to which it will be applied. This will cause a UUID (identifier) to be assigned to the update, and information about which servers it has been applied to will be tracked. Once an update has been uploaded to a pool or server, you can use the patch-list and patch-param-list commands to view information about the update. The second stage is to apply the update. We recommend that the patch-pool-apply command be used to do this; this will result in the update being applied on all servers in the pool. Alternatively, the patch-apply command may be used to apply the update to one server in a pool - this may be useful when applying the update and then restarting individual servers in the pool. Pools should not be left in an inconsistent update state (one where updates have been installed on some servers and not others).

Discussion of procedures using the CLI below assume a basic knowledge of the usage of the xe tool. For information about this, please see the Administrator's Guide.

The update procedure is essentially the same for both a single server and pool scenario, except that in a pooled scenario you must ensure that the update is applied to all servers in the pool. This will be achieved either by using the patch-pool-apply command, or by executing the patch-apply once for each host. These are described below.

To apply an update to a XenServer Host or XenServer Host pool using the CLI

  1. Download the update to a local directory. Note the path to the update file you have downloaded. (It is also possible to download the update directly to an appropriate location on the server, e.g. /root, using standard Linux commands, but it is usually best to download it to a remote client.)
  2. Upload the update to your server or pool. An example CLI command to do this might be:
    $ xe -s my_server -u root -pw root_password patch-upload \
    file-name=update_file b89249c7-feba-41c5-8838-911ded969add
    
    Here, the -s -u, and -pw options refer to the server, the username (which would usually be root), and the password, as usual - these would be omitted if running the command directly from the XenServer Host console.Once you have executed the above command, you will be given the UUID of the uploaded update. This UUID will be used to specify the update that is to be applied.
  3. Be sure to follow any guidance regarding the update before continuing, in particular any information provided about whether VMs should be moved away from the server or that the server should be restarted after applying the update. As always, we recommend taking appropriate backup measures before making modifications to system software. To automatically move VMs to other servers, you can make use of the host-evacuate CLI command.
  4. Apply the update to the pool. A command like the following may be used to do this:
    $ xe patch-pool-apply uuid=b89249c7-feba-41c5-8838-911ded969add
    
    This will apply the update to all servers in the pool. Alternatively, if you need to restart servers and perform the update in a rolling manner, you can apply the update to an individual server by running a command like the following:
    $ xe patch-apply host-uuid=ebf17583-d8c5-4835-999a-e0a29716207d \
    uuid=b89249c7-feba-41c5-8838-911ded969add
    						
  5. Verify that the update was applied by using the patch-list command again. Now the hosts field should contain the host UUID.

After an update is applied to a XenServer Host, a small file containing the same information stored on the master from the xe patch-upload command is written to a subdirectory of the machine's patch directory. This enables XenServer Hosts later ejected from the pool to repopulate their databases with information about updates already applied.

To save space on the master, large updates can be deleted from disk using the xe patch-clean command. (The update information stored in the master's database, though, is always retained.) These updates can be uploaded again using xe patch-upload if required.

XenServer allows you to upgrade a pool of XenServer Hosts running the 4.0.1 version, while keeping VMs on that pool running and thus avoiding downtime of your services. This is achieved by upgrading on a host-by-host basis, with only one XenServer Host offline at a time.

You can use XenCenter or the command line interface to migrate VMs running on a XenServer Host running an older version of the product to one running either the same version or higher. It is not possible to migrate VMs located on a XenServer Host with a newer XenServer version to one running an older version.

You should plan your upgrade path carefully, as we strongly advise against running a mixed-mode pool (one with multiple versions of XenServer co-existing) for longer than necessary. This is because the pool will be operating in a degraded state during the upgrade: all VMs will continue to function as normal, but control operations other than migration might not be available. In particular, it is not safe to perform storage-related operations such as adding, removing or resizing virtual disks in this mode.

The correct sequence for upgrading a pool of XenServer installations to a newer version is as follows:

  1. Eject any CDs from your Virtual Machines before starting the rolling upgrade. Having CDs inserted during rolling upgrade can prevent migrations from working correctly, and due to the mode of operation of the pool whilst the rolling upgrade is taking place, it is required that this be done before the rolling upgrade is started.
  2. Upgrade your XenCenter to the latest version. The newer version will continue to operate fine against older versions of XenServer hosts.
  3. Verify that there are no VMs in the Suspended state. This is indicated in XenCenter by a blue paused icon. Importantly, any suspended virtual machine with a CD drive attached (with the Tools ISO or a local physical drive, for example) will not be resumable after upgrade. To get the virtual machine back into a usable state, one would have to perform a "Force Shutdown" of the suspended VM.
  4. Migrate all VMs running on the pool master to other XenServer Hosts using XenMotion. The pool master is identified in XenCenter as being the topmost server in the pool, and shows Server type: Master in the General tab when selected.
  5. Shut down the pool master using XenCenter or the command line interface. This will cause your pool to enter emergency mode. VMs will continue to run, but you will be unable to perform control operations. This is expected behavior.
  6. Boot the pool master using your XenServer installation media or network and follow the instructions for doing a standard installation and upgrade (see Installing XenServer).
  7. On restarting your pool master, after a few minutes your pool will leave emergency mode and normal service will be restored.
  8. You are now ready to upgrade a second XenServer Host. You should select a XenServer Host still running an old version of XenServer and migrate the VMs running on this XenServer Host to the one you have just upgraded. Do not attempt to migrate a VM from an upgraded XenServer Host to one that has not yet been upgraded. You will see an error message if you attempt to do this, and your VM will continue running without being migrated.
  9. Upgrade the member XenServer Host you have just freed up following a similar procedure as for the master; shut down the member using XenCenter or the command line interface (your pool will not enter emergency mode this time), then upgrade the server software using your product media or remote installation repository.
  10. Repeat the previous two steps for each member XenServer Host in the pool.
  11. Now that you have upgraded the XenServer Host software on your pool, it is important to upgrade the XenServer Tools in each VM. This will enable new functionality and ensure the stability of your VMs. Running old versions of XenServer Tools on newer XenServer installations is not a supported configuration except for during the upgrade process. Please refer to the XenCenter Help, or the XenServer Virtual Machine Installation Guide for details on how to perform the upgrade of XenServer Tools for Windows and Linux VMs.

The following procedure describes how to install the current version of the XenServer Host over an existing installation of XenServer Host 3.2.0

When upgrading from version 3.2.0 to the current version, be aware of the following:

  • any custom RPMs which you might have installed on the XenServer Host control domain will not be preserved
  • existing Windows VMs will need to have the paravirtualized device drivers reinstalled

Note that there is no direct upgrade path to the current version from 3.2. You must first upgrade to 4.0.1, and then proceed to the next version. This will ensure that your VMs are preserved correctly.

VMs ext3 and nfs storage repositories, which are stored in the Microsoft Virtual Hard Disk (VHD) format under the 4.0.1 version of XenServer, have a free space bitmap written in with the wrong byte order. Version 4.1 provides an upgrade utility vhd-update, located in /usr/sbin/.

The command

vhd-update -f <filename>
			

copies all bitmaps in filename to filename.journal, then writes all bitmaps, transformed appropriately, from filename.journal to filename, and finally deletes filename.journal.

If the update is interrupted for any reason, it can be resumed by running:

vhd-update -f <filename> -j <filename>.journal
			

This will validate the specified journal file, then proceed to transform the bitmaps and rewrite filename.

There is also a rollback operation

vhd-update -f <filename> -j <filename>.journal -r

which will write the bitmaps from filename.journal to filename without transforming them (this is probably only useful for testing and debugging).

We recommend that, whenever possible, you leave the installed state of XenServer Hosts unaltered. That is, do not install any additional packages or start additional services on XenServer Hosts, and treat them as if they are appliances. The best way to restore, then, is to re-install XenServer Host software from the installation media. If you have multiple XenServer Hosts, the best approach is to configure a PXE boot server and appropriate answerfiles for this purpose (see Appendix C, PXE installation of XenServer Host).

For VMs, the best approach is to install backup agents on them, just as if they were standard physical servers. For Windows VMs, as of this release we have tested CA BrightStor ARCserve Backup, and Symantec NetBackup and Backup Exec.

For more information about backup tools tested, best practices, and backups in general, see the Citrix Knowledge Base.

XenServer Hosts use a per-host database to store metadata about VMs and associated resources such as storage and networking. When combined with storage repositories, this database forms the complete view of all VMs available across the pool. Thus, it is important to understand how to backup this database in order to recover from physical hardware failure and other disaster scenarios.

This section first describes how to backup metadata for single-host installations, and then for more complex pool setups.

The CLI must be used to backup the pool database. To obtain a consistent pool metadata backup file, run xe pool-dump-database against the XenServer Host and archive the resulting file. The backup file will contain sensitive authentication information about the pool, so ensure it is securely stored.

To restore the pool database, use the xe pool-restore-database from a previous dump file. If your XenServer Host has died completely, then you must first do a fresh install, and then run the xe pool-restore-database command against the freshly installed XenServer Host.

After a restoration of the pool database, some VMs may still be registered as being “suspended”, but if the storage repository with their suspended memory state (defined in the [suspend-VDI-uuid] field) was a local SR, it will no longer be available since the host has been reinstalled. To reset these VMs back to the halted state so that they can be started up again, use the xe vm-shutdown vm=vm_name -force command, or use the xe vm-reset-powerstate vm=vm_name -force command.

Note that XenServer Hosts restored using this method will have their UUIDs preserved. Thus, if you restore to a different physical machine while the original XenServer Host is still running, there will be a UUID clash. The main observable effect of this clash will be that XenCenter will refuse to connect to the second XenServer Host. Pool database backup is not the recommended mechanism for cloning physical hosts; you should use the automated installation support for that (see Appendix C, PXE installation of XenServer Host).

This section describes the XenServer Host control domain backup and restore procedures. These procedures do not back up the storage repositories that house the VMs, but only the privileged control domain that runs Xen and the XenServer agent.

Note that since the the privileged control domain is best left as installed, without customizing it with other packages, we recommend you set up a PXE boot environment to cleanly perform a fresh installation from the XenServer media as a recovery strategy. In many cases you will not need to backup the control domain at all, but just save the pool metadata (see the section called “Backing up Virtual Machine metadata”). This backup method should always be considered complementary to backing up the pool metadata.

Another approach is to run the XenServer installation twice, selecting to back up the existing installation when prompted. This will create a pristine copy of the freshly-installed control domain that can later be restored if necessary by using the installation CD and choosing the Restore option.

Using the xe commands host-backup and host-restore is another approach that you can take. The xe host-backup command archives the active partition to a file you specify, and the xe host-restore command extracts an archive created by xe host-backup over the host's currently inactive disk partition. This partition can then be made active by booting off the installation CD and choosing the Restore option.

After completing the above steps and rebooting the host, you must ensure that the VM meta-data is restored to a consistent state. This can be achived by running xe pool-restore-database on /var/backup/pool-database-${DATE}. This file is created by xe host-backup using xe pool-dump-database prior to archiving the running filesystem, in order to snapshot a consistent state of the VM metadata.

This appendix describes setting up a TFTP server to enable PXE booting of XenServer Host installations. It also describes the use of an XML answerfile, which allows you to perform unattended installations.

To create a PXE environment, you need:

These can all co-exist on the same server, or be distributed on different servers on the network.

Additionally, each system that you want to PXE boot and install the XenServer on needs a PXE boot-enabled Ethernet card.

The following steps assume that the Linux server or servers you will use have RPM support.

To set up a TFTP server for PXE booting

  1. TFTP requires SYSLINUX 3.11 or above. SYSLINUX is a collection of boot loaders for the Linux operating system which operates on Linux EXT2/EXT3 file systems, MS-DOS FAT file systems, network servers using PXE firmware, and CD-ROMs. Make sure you have SYSLINUX version 3.11 or above installed on your system with the command
    #rpm -q syslinux
    				
    If you have a earlier version, you can download an appropriate later version from ftp://ftp.kernel.org/pub/linux/utils/boot/syslinux/RPMS/i386/, then install it using the command
    #rpm -Uvh syslinux.-.rpm
  2. Check if the tftp server package is installed:
    #rpm -q tftp-server
    If not, use system-config-packages and install.
  3. Edit the file /etc/xinetd.d/tftp to change the line
    disable = yes
    to
    disable = no
  4. Restart the xinetd service, which manages tftp:
    # service xinetd restart
  5. Make a directory inside /tftpboot called xenserver.
  6. Copy the files mboot.c32 and pxelinux.0 from /usr/lib/syslinux to the /tftboot directory.
  7. Copy the files install.img, vmlinuz, and xen.gz from the Base Pack CD (found in the root of the Base Pack CD, and in its /boot directory respectively), and place them in /tftpboot/xenserver.
  8. Make a directory called pxelinux.cfg inside /tftboot and create a file named default. The file contents depend on how you want to configure your PXE boot environment. For example, you might have a configuration file like the following:

    Note

    The backslashes at the ends of lines in the example PXE configuration files shown below denote continuation of lines; do not actually include them in your PXE configuration file.

    Also note that the three hyphens in the examples are necessary parts of the mboot.c32 loader syntax, and not including them will cause PXE boot attempts to fail.

    default xenserver
    label xenserver
       kernel mboot.c32
       append path/to/boot/directory/xen.gz watchdog com1=115200,8n1i \
          console=com1,tty --- path/to/boot/directory/vmlinuz \
          console=tty0 console=ttyS0,115200n8 \
    	  --- path/to/boot/directory/install.img
    				
    (where path/to/boot/directory is the directory where you copied install.img, vmlinuz, and xen.gz files in the previous step). This will start an installation on any machine that boots from this server. Someone would then need to manually respond to the prompts to complete the installation. Alternatively, you might have a configuration file like the following:
    default xenserver-auto
    label xenserver-auto
       kernel mboot.c32
       append path/to/boot/directory/xen.gz watchdog com1=115200,8n1 \
          console=com1,tty --- path/to/boot/directory/vmlinuz \
          console=tty0 console=ttyS0,115200n8 \
          answerfile=http://pxehost.example.com/4.1.0-answerfile \
          install --- path/to/boot/directory/install.img
    				
    This will perform an unattended installation using the answerfile at the URL specified.

    Note

    Also, if you want to use the serial console to do an installation, be sure to include the argument output=ttyS0 on the kernel command-line (e.g. after "vmlinuz") in addition to any other appropriate console= values.

    For details on creating an answerfile for unattended installation, see
    the section called “Creating an answerfile for unattended PXE installation”. For more information on PXE configuration file syntax, see the SYSLINUX website.

Please refer to your server operating system manual for details for your specific operating system. The information here is a guide that can be used for Red Hat, Fedora, and some other RPM-based distributions.

To set up a DHCP server

  1. On the server that you will be using for DHCP, check if you have DHCP installed by issuing the command
    # rpm -qa dhcp
    If not, install using system-config-packages.
  2. Configure the dhcp server. Refer to article 4221 in the Red Hat Knowledge base for details.
  3. Add these lines to the end of the existing dhcpd.conf file where is your tftp server address:
    allow booting;
    allow bootp;
    class "pxeclients" {
        match if substring(option vendor-class-identifier, 0, 9) = "PXEClient";
        next-server ;
        filename "pxelinux.0";
    }
    			
  4. Restart the dhcpd service:
    # service dhcpd restart

For example, to install both packages from the webserver http://pxehost.example.com where the packages are in the directories mentioned above relative to the server's document root, the answerfile would contain this source element:

<source type="url">http://pxehost.example.com/XenServer_4.1.0</source>
		

or, to install just the basic pack and skip Linux support:

<source type="url">
http://pxehost.example.com/XenServer_4.1.0/packages.main
</source>