XenServer Administrator's Guide

Master nodes detect the failures of slaves by receiving regular heartbeat messages. If no heartbeat has been received for 30 seconds then the master assumes the slave is dead. There are two ways to recover from this problem:

When a slave host fails, there may be VMs still registered in the “running” state. If you are sure that the slave host is definitely down, and that the VMs have not been brought up on another host in the pool, use the xe vm-reset-powerstate command to forcibly halt the VMs. See Section 5.4.20.20, “vm-reset-powerstate” for more details.

Every member of a Resource Pool contains all the information necessary to take over the role of master if required. When a master node fails, the following sequence of events occurs:

If the master comes back up at this point, it will reestablish communication with its slaves, they will leave emergency mode, and operation will return to normal.

If the master is really dead, though, you should choose one of the slaves and issue to it the command xe pool-emergency-transition-to-master. Once it has become the master, issue the command xe pool-recover-slaves and the slaves will now point to the new master.

If you repair or replace the server that was the original master, you can simply bring it up, install the XenServer Host software, and add it to the pool. Since the XenServer Hosts in the pool are enforced to be homogeneous, there is no real need to make the replaced server the master.

When a slave host is transitioned to being a master, you should also check that the default pool storage repository is set to an appropriate value. This can be done using the xe pool-param-list command and verifying that the default-SR is pointing to a valid storage repository.

Edit the /etc/sysconfig/network file to change the hostname parameter to match your system hostname.

HOSTNAME=servername
				

DNS servers are defined in the /etc/resolv.conf file. To add additional name-servers or change existing name-server definitions, edit the file. Name-server entries in the file have the form

nameserver 192.168.1.34
nameserver 192.168.1.33
			

Network interface configurations are defined in files stored in the /etc/sysconfig/network-scripts directory. Each network interface has its own file with the name ifcfg-device_name. There is a device file for the physical card itself and another for the Linux bridge associated with this device. For example, the first network interface card is named eth0, and its associated bridge is xenbr0, so the network configuration information would be found in the files /etc/sysconfig/network-scripts/ifcfg-eth0 and /etc/sysconfig/network-scripts/ifcfg-xenbr0.

When XenServer Host is installed on a machine with multiple NICs, only the interface selected as the management NIC will be configured with an IP address. The NIC configurations for the additional interfaces will be present in /etc/sysconfig/network-scripts directory. If you wish to use other NICs, you need to add the line

	IPADDR=<ip address>
			

to the appropriate ifcfg-xenbrn file

If you add a new network interface to a XenServer Host, copy an existing pair of ifcfg-ethn and ifcfg-xenbrn files and edit them appropriately.

If a network interface is configured to get its network parameters from a DHCP server, its ifcfg files might be something like the following:

	DEVICE=eth1
	ONBOOT=yes
	TYPE=Ethernet
	HWADDR=00:15:60:95:4f:d0
	BRIDGE=xenbr1
	check_link_down() { return 1 ; }
			

	DEVICE=xenbr1
	ONBOOT=yes
	TYPE=Bridge
	DELAY=0
	STP=off
	BOOTPROTO=dhcp
	check_link_down() { return 1 ; }
			

If a network interface is configured with static network parameters, it needs to have IP address, gateway, and netmask specified explicitly in its ifcfg-xenbrn file:

	DEVICE=xenbr1
	BOOTPROTO=none
	ONBOOT=yes
	TYPE=Bridge
	NETMASK=255.255.255.0
	IPADDR=10.100.2.6
	GATEWAY=10.100.2.1
	PEERDNS=yes
	DELAY=0
	STP=off
	check_link_down() { return 1 ; }
			

The ONBOOT tells the system to enable the network device when the computer is booted up. To disable an existing network interface, edit its ifcfg file to that the ONBOOT parameter is set to no:

	ONBOOT=no
			

You can also disable an existing NIC by renaming or deleting its ifcfg file.

In order to activate the configuration file changes, you will need to restart the networking service by executing service network restart from the host console, and then restarting the host agent by running xe-toolstack-restart. See Section 3.3.4, “IP address changes in Resource Pools” for further required action if the management interface has been altered.

XenServer hosts in Resource Pools use a single management IP address to communicate with the other hosts in the same pool. Thus, extra care is required when changing the IP address of hosts in an existing pool.

Altering the IP address of the master host is slightly more difficult, since each of the slave hosts uses its advertised IP address to communicate with it, and will not know how to contact the master if its IP address changes. Where possible, ensure that you have a dedicated IP address for a pool master which is not likely to change in the lifetime of the pool.

By default, XenServer uses the local disk on the physical host on which it is installed. The Linux Logical Volume Manager (LVM) is used to manage VM storage. In this case a VDI is implemented as a LVM logical volume of the specified size.

Local LVM-based storage is high-performance and allows virtual disks to be dynamically resized. Virtual disks are fully allocated as an isolated volume on the underlying physical disk and so there is a minimum of storage virtualization overhead imposed. As such, this is a good option for high-performance storage, but lacks the flexibility of file-based storage options described below.

In addition to storing disks on an LVM-managed volume, local disks may be used to serve VDIs stored in the Microsoft VHD format. This may be configured through the XenServer CLI. VHD support is described in Section 4.1.2, “Shared Network Attached Storage - NFS”.

By definition, local disks are not shared across pools of XenServer hosts. As a consequence, VMs whose VDIs are stored in SRs on local disks are not agile; they may not be moved or migrated between hosts in a pool.

Supported device-config parameters for the local LVM SR type and the local VHD SR type are:

The NFS filer is a ubiquitous form of storage infrastructure that is available in many deployments. XenServer allows existing NFS servers to be immediately used as a storage repository for virtual disks (VDIs). VDIs are stored in the Microsoft VHD format, which is ideally suited to NFS environments. Moreover, as NFS SRs are shared, VDIs stored in them allow VMs to be started on any host in a Resource Pool and be live migrated between them using XenMotion.

Configuring an NFS SR is very easy. You just provide the hostname or IP address of the NFS server and the path to a directory that will be used to contain the SR. The NFS server must be configured to export the specified path to all hosts in the pool, otherwise the creation of the SR or the plugging of the PBD record will fail.

VDIs stored on NFS are sparse by default: the image file is allocated as the VM writes data into the disk. This has the considerable benefit that VM image files take up only as much space on the NFS filer as is required: If a 100GB VDI is allocated for a new VM and an OS is installed, the VDI file will only reflect the size of the OS data that has been written to the disk, typically only a few gigabytes.

VHD files may also be chained, allowing two VDIs to share common data. In cases where a NFS-based VM is cloned, the resulting VMs will share the common on-disk data at the time of cloning. Each will proceed to make its own changes in an isolated copy-on-write version of the VDI. This feature allows NFS-based VMs to be quickly cloned from templates -- facilitating very fast provisioning and deployment of new VMs.

As VHD-based images involve extra metadata to provide sparseness and chaining, the format is not as high-performance as LVM-based storage is. In cases where performance really matters, it is well worth forcibly allocating the sparse regions of an image file. This will improve performance at the cost of consuming addition disk space on the filer.

XenServer's NFS and VHD implementation assume that they have full control over the SR directory on the NFS server. Administrators are advised not to modify the contents of this directory as this may risk corrupting the contents of VDIs.

In considering best practice for NFS performance it is worth considering that XenServer has been tuned for enterprise filers that use non-volatile RAM to provide fast acknowledgments of write requests while maintaining a high degree of protection from failure. For reference, XenServer has been tested extensively against Network Appliance FAS270c and FAS3020c filers, using Data OnTap 7.2.2.

In situations where XenServer is used with lower-end filers, it will err on the side of caution by waiting for all writes to be acknowledged before passing acknowledgments on to guest VMs. This will incur a noticeable performance cost, and may be remedied by setting the filer to present the SR mount point as an asynchronous mode export. Asynchronous exports, however acknowledge writes that are not actually on disk, and so administrators should consider the risks of failure carefully in these situations.

Supported device-config parameters for the NFS SR type are:

In addition to NFS, XenServer provides support for shared SRs on iSCSI LUNs, using the open-iSCSI software iSCSI initiator. Shared iSCSI support is implemented based on the Linux Volume Manager (LVM) and provides the same performance benefits that were provided by LVM in the local disk case. iSCSI-based SRs enable VM agility -- VMs may be started on any host in a pool and migrated between them. However, iSCSI-based VDIs do not provide support for sparse provisioning or fast cloning.

Each XenServer is automatically configured with a single iSCSI software adapter and assigned a random adapter initiator IQN. The initiator IQN is an iSCSI id that uniquely identifies the host when connecting to an iSCSI target. Most targets provide access control via IQN lists and it is therefore important to ensure that all hosts always retain a unique IQN value. The host IQN value can be adjusted manually using XenCenter or via the CLI:

xe host-param-set uuid=[VALID_HOST_ID] other-config-iscsi_iqn=[NEW_INITIATOR_IQN]

The XenServer only supports a single iSCSI adapter to be configured. The adapter can connect to multiple different iSCSI targets and target IQNs in parallel, however it does not support configuration of more than one host initiator IQN, so all targets must be configured to allow access for the same initiator IQN.

iSCSI SRs may be created through both XenCenter and the CLI. XenServer hosts are limited to a single source initiator IQN, but may connect to multiple targets. Each individual iSCSI SR must be contained entirely on a single LUN, and may not span LUNs. CHAP support is provided for client authentication, both during the data path initialization and during the LUN discovery phase.

Supported device-config parameters for the shared LVM over ISCSI SR type are:

In order to aid in the LUN discovery and identification phase, certain scan facilities are provided during the sr-create phase:

If the targetIQN parameter is left blank, the command will fail and return an XML string listing the available targetIQNs on the specified target. Each target will also provide an index value that may be provided as the usediscoverynumber value.

If the LUNid parameter is left blank, the command will fail and return an XML string listing the available LUNids on the specified target and targetIQN.

XenServer hosts can also use Fibre Channel SANs using the Emulex or QLogic host bus adapter (HBA). Logical unit numbers (LUNs) are mapped to the XenServer Host as disk devices /dev/sdx just like physical disks would be.

The Command Line Interfaces (CLIs) for configuring and managing Emulex and QLogic Fibre Channel HBAs are included in the XenServer Host in the following locations:

If you are using a Fibre Channel SAN with an HBA that supports boot from LUN, you should do all your boot from LUN setup before installing the XenServer Host. During installation, just select the remote LUNs as if they were local disk drives. Once you complete the installation and reboot, the system will boot from the remote LUN.

Fibre Channel LUNs will appear on the host as scsi devices. Each scsi device is symlinked under the directory /dev/disk/by_id using its unique scsi_id. If you are unsure which scsi_id corresponds to which device, you can query a device with the sginfo command followed by the path. For example: sginfo /dev/disk/by_id/ {scsi_id}.

Fibre Channel disks should always be referenced by this path since it provides persistent device identification regardless of the core device name assigned by the host which may change, e.g. across host reboots.

If you add an Emulex or QLogic HBA to the XenServer Host after installation, you should edit /etc/modprobe.conf and add a line like this:

alias scsi_hostadaptern module_name
		

where n is the next available scsi_hostadapter number, and module_name is the appropriate module name for your HBA. Emulex cards are supported by the lpfcdfc module and qlogic cards by the qla**** modules, where **** will correspond to the version number. For full compatibility details, goto the online site Hardware Compatibility List for latest information.

For more information on Fibre Channel host adaptors, see the Emulex website and the QLogic website.

The following section provides guidelines on how to create, attach, detach and delete storage types to a XenServer Host. The examples provided pertain to storage configuration via the CLI which provides the greatest flexibility.

Creating a new SR requires the use of the sr-create command. It will create a new SR record in the database and a corresponding PBD record. Upon successful creation of the SR, the PBD will be automatically plugged. If the SR shared=true flag is set, a PBD entry will be created for every host in the pool and plugged on each host. The following steps illustrate how to create a local LVM SR and a shared LVM over iSCSI SR. In each case, a correct host-uuid is required:

Creating an LVM over iSCSI SR is very similar with the exception of being a shareable SR type, causing a PBD to be created for every host in the pool. Additionally, the device-config-* parameters are different, requiring a target, targetIQN and a LUNid parameter to be provided.

Introducing an existing SR to a host requires the manual generation of both an SR record and a PBD record. Furthermore the PBD must be manually plugged to activate the SR on the host:

Destroying an SR will actually delete the contents of the SR from the physical substrate. Alternatively, an SR record can be forgotten, which allows a user to re-attach the SR, e.g. to another host without removing any of the SR contents. In both cases, the SR's PBD must first be unplugged:

The following table summarizes the storage repository capabilities described above:

All storage repositories in XenServer are implemented as Python scripts and stored within the control domain's file system in /opt/xensource/sm. Advanced users may examine and even modify these scripts to adapt storage operations to their needs.

This is considered an advanced operation, and is not supported. However, visibility and customization of low-level storage management is of considerable value to some power users. Note also that new SR implementations may be placed in this directory and will be automatically detected by XenServer. The available SR types may be listed using the sm-list command (see Section 5.4.11, “Storage Manager commands”).

The objects that are addressed with the xe commands have sets of parameters that identify them and define their states.

Most parameters take a single value. For example, the name-label parameter of a VM contains a single string value. In the output from parameter list commands such as xe vm-param-list, such parameters have an indication in parentheses that defines whether they can be read and written to, or are read-only. For example, the output of xe vm-param-list on a specified VM might have the lines

                  user-version ( RW): 1
             is-control-domain ( RO): false
 		

The first parameter, user-version, is writeable and has the value 1. The second, is-control-domain, is read-only and has a value of false.

The two other types of parameters are multi-valued. A set parameter contains a list of values. A map parameter is a set of key/value pairs. As an example, look at the following excerpt of some sample output of the xe vm-param-list on a specified VM:

                      platform (MRW): acpi: true; apic: true; pae: true; nx: false
            allowed-operations (SRO): pause; clean_shutdown; clean_reboot; hard_shutdown; hard_reboot; suspend
		

The platform parameter has a list of items that represent key/value pairs. The key names are followed by a colon character (:). Each key/value pair is separated from the next by a semicolon character (;). The M preceding the RW indicates that this is a map parameter and is readable and writeable. The allowed-operations parameter has a list that makes up a set of items. The S preceding the RO indicates that this is a set parameter and is readable but not writeable.

In xe commands where you want to filter on a map parameter, or set a map parameter, use the separator : (colon) between the map parameter name and the key/value pair. For example, to set the value of the foo key of the other-config parameter of a VM to baa, the command would be

xe vm-param-set uuid=VM uuid other-config:foo=baa

There are several commands for operating on parameters of objects: class-param-get,class-param-set,class-param-add,class-param-remove,class-param-clear and class-param-list. Each of these takes a uuid parameter to specify the particular object.

The class-list command lists the objects of type class. By default it will list all objects, printing a subset of the parameters. This behavior can be modified in two ways: it can filter the objects so that it only outputs a subset, and the parameters that are printed can be modified.

To change the parameters that are printed, the argument params should be specified as a comma-separated list of the required parameters, e.g.:

 xe vm-list params=name-label,other-config
 		

Alternatively, to list all of the parameters, use the syntax:

 xe vm-list params=all
                

Note that some parameters that are expensive to calculate will not be shown by the list command. These will be shown as e.g.:

  allowed-VBD-devices (SRO): <expensive field>
                

In order to obtain these fields, use either the command class-param-list or class-param-get

To filter the list, the CLI will match parameter values with those specified on the command-line, only printing object that match all of the specified constraints. e.g.

 xe vm-list HVM-boot-policy="BIOS order" power-state=halted
                

will only list those VMs for which both the field 'power-state' has the value 'halted', and for which the field 'HVM-boot-policy' has the value 'BIOS order'.

It is also possible to filter the list based on the value of keys in maps, or on the existence of values in a set. The syntax for the first of these is map-name:key=value, and the second is set-name:contains=value

For scripting, a useful technique is passing '--minimal' on the command line, causing xe to print only the first field in a comma-separated list. For example, the command 'xe vm-list --minimal' on a XenServer Host with three VMs installed gives:

a85d6717-7264-d00e-069b-3b1d19d56ad9,aaa3eec5-9499-bcf3-4c03-af10baea96b7,42c044de-df69-4b30-89d9-2c199564581d
                

Commands for working with physical CD/DVD drives on XenServer Hosts.

CDs have the following parameters:

Commands for working with consoles.

The console objects can be listed with the standard object listing command (xe console-list), and the parameters manipulated with the standard parameter commands. See Section 5.3.2, “Low-level param commands” for details.

Consoles have the following parameters:

Commands for working with events.

Event classes are listed in the following table:

Commands for interacting with XenServer Hosts.

XenServer Hosts are the physical servers running XenServer software. They have VMs running on them under the control of a special privileged Virtual Machine, known as the control domain or domain 0.

The XenServer Host objects can be listed with the standard object listing command ( xe host-list, xe host-cpu-list, and xe host-crashdump-list), and the parameters manipulated with the standard parameter commands. See Section 5.3.2, “Low-level param commands” for details.

XenServer Hosts have the following parameters:

XenServer Hosts contain some other objects that also have parameter lists.

CPUs on XenServer Hosts have the following parameters:

Crash dumps on XenServer Hosts have the foll