What is a VM snapshot and why is a live acquisition typically required for VMs?
About Working with Virtual Machines¶CloudStack provides administrators with complete control over the lifecycle of all guest VMs executing in the cloud. CloudStack provides several guest management operations for end users and administrators. VMs may be stopped, started, rebooted, and destroyed. Show
Guest VMs have a name and group. VM names and groups are opaque to CloudStack and are available for end users to organize their VMs. Each VM can have three names for use in different contexts. Only two of these names can be controlled by the user:
Guest VMs can be configured to be Highly Available (HA). An HA-enabled VM is monitored by the system. If the system detects that the VM is down, it will attempt to restart the VM, possibly on a different host. For more information, see HA-Enabled Virtual Machines on Each new VM is allocated one public IP address. When the VM is started, CloudStack automatically creates a static NAT between this public IP address and the private IP address of the VM. If elastic IP is in use (with the NetScaler load balancer), the IP address initially allocated to the new VM is not marked as elastic. The user must replace the automatically configured IP with a specifically acquired elastic IP, and set up the static NAT mapping between this new IP and the guest VM’s private IP. The VM’s original IP address is then released and returned to the pool of available public IPs. Optionally, you can also decide not to allocate a public IP to a VM in an EIP-enabled Basic zone. For more information on Elastic IP, see “About Elastic IP”. CloudStack cannot distinguish a guest VM that was shut down by the user (such as with the “shutdown” command in Linux) from a VM that shut down unexpectedly. If an HA-enabled VM is shut down from inside the VM, CloudStack will restart it. To shut down an HA-enabled VM, you must go through the CloudStack UI or API. Best Practices for Virtual Machines¶For VMs to work as expected and provide excellent service, follow these guidelines. Monitor VMs for Max Capacity¶The CloudStack administrator should monitor the total number of VM instances in each cluster, and disable allocation to the cluster if the total is approaching the maximum that the hypervisor can handle. Be sure to leave a safety margin to allow for the possibility of one or more hosts failing, which would increase the VM load on the other hosts as the VMs are automatically redeployed. Consult the documentation for your chosen hypervisor to find the maximum permitted number of VMs per host, then use CloudStack global configuration settings to set this as the default limit. Monitor the VM activity in each cluster at all times. Keep the total number of VMs below a safe level that allows for the occasional host failure. For example, if there are N hosts in the cluster, and you want to allow for one host in the cluster to be down at any given time, the total number of VM instances you can permit in the cluster is at most (N-1) * (per-host-limit). Once a cluster reaches this number of VMs, use the CloudStack UI to disable allocation of more VMs to the cluster. VM Lifecycle¶Virtual machines can be in the following states: Once a virtual machine is destroyed, it cannot be recovered. All the resources used by the virtual machine will be reclaimed by the system. This includes the virtual machine’s IP address. A stop will attempt to gracefully shut down the operating system, which typically involves terminating all the running applications. If the operation system cannot be stopped, it will be forcefully terminated. This has the same effect as pulling the power cord to a physical machine. A reboot is a stop followed by a start. CloudStack preserves the state of the virtual machine hard disk until the machine is destroyed. A running virtual machine may fail because of hardware or network issues. A failed virtual machine is in the down state. The system places the virtual machine into the down state if it does not receive the heartbeat from the hypervisor for three minutes. The user can manually restart the virtual machine from the down state. The system will start the virtual machine from the down state automatically if the virtual machine is marked as HA-enabled. Creating VMs¶Virtual machines are usually created from a template. Users can also create blank virtual machines. A blank virtual machine is a virtual machine without an OS template. Users can attach an ISO file and install the OS from the CD/DVD-ROM. Note You can create a VM without starting it. You can determine whether the VM needs to be started as part of the VM deployment. A request parameter, startVM, in the deployVm API provides this feature. For more information, see the Developer’s Guide. To create a VM from a template:
To create a VM from an ISO: Note (XenServer) Windows VMs running on XenServer require PV drivers, which may be provided in the template or added after the VM is created. The PV drivers are necessary for essential management functions such as mounting additional volumes and ISO images, live migration, and graceful shutdown.
Accessing VMs¶Any user can access their own virtual machines. The administrator can access all VMs running in the cloud. To access a VM through the CloudStack UI:
To access a VM directly over the network:
Stopping and Starting VMs¶Once a VM instance is created, you can stop, restart, or delete it as needed. In the CloudStack UI, click Instances, select the VM, and use the Stop, Start, Reboot, and Destroy buttons. Assigning VMs to Hosts¶At any point in time, each virtual machine instance is running on a single host. How does CloudStack determine which host to place a VM on? There are several ways:
Affinity Groups¶By defining affinity groups and assigning VMs to them, the user or administrator can influence (but not dictate) which VMs should run on separate hosts. This feature is to let users specify that VMs with the same “host anti-affinity” type won’t be on the same host. This serves to increase fault tolerance. If a host fails, another VM offering the same service (for example, hosting the user’s website) is still up and running on another host. The scope of an affinity group is per user account. Creating a New Affinity Group¶To add an affinity group:
Assign a New VM to an Affinity Group¶To assign a new VM to an affinity group:
Change Affinity Group for an Existing VM¶To assign an existing VM to an affinity group:
View Members of an Affinity Group¶To see which VMs are currently assigned to a particular affinity group:
Delete an Affinity Group¶To delete an affinity group:
Virtual Machine Snapshots¶(Supported on VMware and XenServer) In addition to the existing CloudStack ability to snapshot individual VM volumes, you can take a VM snapshot to preserve all the VM’s data volumes as well as (optionally) its CPU/memory state. This is useful for quick restore of a VM. For example, you can snapshot a VM, then make changes such as software upgrades. If anything goes wrong, simply restore the VM to its previous state using the previously saved VM snapshot. The snapshot is created using the hypervisor’s native snapshot facility. The VM snapshot includes not only the data volumes, but optionally also whether the VM is running or turned off (CPU state) and the memory contents. The snapshot is stored in CloudStack’s primary storage. VM snapshots can have a parent/child relationship. Each successive snapshot of the same VM is the child of the snapshot that came before it. Each time you take an additional snapshot of the same VM, it saves only the differences between the current state of the VM and the state stored in the most recent previous snapshot. The previous snapshot becomes a parent, and the new snapshot is its child. It is possible to create a long chain of these parent/child snapshots, which amount to a “redo” record leading from the current state of the VM back to the original. If you need more information about VM snapshots on VMware, check out the VMware documentation and the VMware Knowledge Base, especially Understanding virtual machine snapshots. Limitations on VM Snapshots¶
Configuring VM Snapshots¶The cloud administrator can use global configuration variables to control the behavior of VM snapshots. To set these variables, go through the Global Settings area of the CloudStack UI. Configuration Setting Name Description vmsnapshots.max The maximum number of VM snapshots that can be saved for any given virtual machine in the cloud. The total possible number of VM snapshots in the cloud is (number of VMs) * vmsnapshots.max. If the number of snapshots for any VM ever hits the maximum, the older ones are removed by the snapshot expunge job. vmsnapshot.create.wait Number of seconds to wait for a snapshot job to succeed before declaring failure and issuing an error. Using VM Snapshots¶To create a VM snapshot using the CloudStack UI:
To delete a snapshot or restore a VM to the state saved in a particular snapshot:
Note VM snapshots are deleted automatically when a VM is destroyed. You don’t have to manually delete the snapshots in this case. Changing the VM Name, OS, or Group¶After a VM is created, you can modify the display name, operating system, and the group it belongs to. To access a VM through the CloudStack UI:
Appending a Display Name to the Guest VM’s Internal Name¶Every guest VM has an internal name. The host uses the internal name to identify the guest VMs. CloudStack gives you an option to provide a guest VM with a display name. You can set this display name as the internal name so that the vCenter can use it to identify the guest VM. A new global parameter, vm.instancename.flag, has now been added to achieve this functionality. The default format of the internal name is i- The following table explains how a VM name is displayed in different scenarios.
Changing the Service Offering for a VM¶To upgrade or downgrade the level of compute resources available to a virtual machine, you can change the VM’s compute offering.
CPU and Memory Scaling for Running VMs¶(Supported on VMware and XenServer) It is not always possible to accurately predict the CPU and RAM requirements when you first deploy a VM. You might need to increase these resources at any time during the life of a VM. You can dynamically modify CPU and RAM levels to scale up these resources for a running VM without incurring any downtime. Dynamic CPU and RAM scaling can be used in the following cases:
Updating Existing VMs¶If you are upgrading from a previous version of CloudStack, and you want your existing VMs created with previous versions to have the dynamic scaling capability, update the VMs using the following steps:
Configuring Dynamic CPU and RAM Scaling¶To configure this feature, use the following new global configuration variables:
How to Dynamically Scale CPU and RAM¶To modify the CPU and/or RAM capacity of a virtual machine, you need to change the compute offering of the VM to a new compute offering that has the desired CPU and RAM values. You can use the same steps described above in “Changing the Service Offering for a VM”, but skip the step where you stop the virtual machine. Of course, you might have to create a new compute offering first. When you submit a dynamic scaling request, the resources will be scaled up on the current host if possible. If the host does not have enough resources, the VM will be live migrated to another host in the same cluster. If there is no host in the cluster that can fulfill the requested level of CPU and RAM, the scaling operation will fail. The VM will continue to run as it was before. Limitations¶
Resetting the Virtual Machine Root Volume on Reboot¶For secure environments, and to ensure that VM state is not persisted across reboots, you can reset the root disk. For more information, see “Reset VM to New Root Disk on Reboot”. Moving VMs Between Hosts (Manual Live Migration)¶The CloudStack administrator can move a running VM from one host to another without interrupting service to users or going into maintenance mode. This is called manual live migration, and can be done under the following conditions:
To manually live migrate a virtual machine
Deleting VMs¶Users can delete their own virtual machines. A running virtual machine will be abruptly stopped before it is deleted. Administrators can delete any virtual machines. To delete a virtual machine:
Working with ISOs¶CloudStack supports ISOs and their attachment to guest VMs. An ISO is a read-only file that has an ISO/CD-ROM style file system. Users can upload their own ISOs and mount them on their guest VMs. ISOs are uploaded based on a URL. HTTP is the supported protocol. Once the ISO is available via HTTP specify an upload URL such as http://my.web.server/filename.iso. ISOs may be public or private, like templates.ISOs are not hypervisor-specific. That is, a guest on vSphere can mount the exact same image that a guest on KVM can mount. ISO images may be stored in the system and made available with a privacy level similar to templates. ISO images are classified as either bootable or not bootable. A bootable ISO image is one that contains an OS image. CloudStack allows a user to boot a guest VM off of an ISO image. Users can also attach ISO images to guest VMs. For example, this enables installing PV drivers into Windows. ISO images are not hypervisor-specific. Adding an ISO¶To make additional operating system or other software available for use with guest VMs, you can add an ISO. The ISO is typically thought of as an operating system image, but you can also add ISOs for other types of software, such as desktop applications that you want to be installed as part of a template.
Attaching an ISO to a VM¶
Changing a VM’s Base Image¶Every VM is created from a base image, which is a template or ISO which has been created and stored in CloudStack. Both cloud administrators and end users can create and modify templates, ISOs, and VMs. In CloudStack, you can change an existing VM’s base image from one template to another, or from one ISO to another. (You can not change from an ISO to a template, or from a template to an ISO). For example, suppose there is a template based on a particular operating system, and the OS vendor releases a software patch. The administrator or user naturally wants to apply the patch and then make sure existing VMs start using it. Whether a software update is involved or not, it’s also possible to simply switch a VM from its current template to any other desired template. To change a VM’s base image, call the restoreVirtualMachine API command and pass in the virtual machine ID and a new template ID. The template ID parameter may refer to either a template or an ISO, depending on which type of base image the VM was already using (it must match the previous type of image). When this call occurs, the VM’s root disk is first destroyed, then a new root disk is created from the source designated in the template ID parameter. The new root disk is attached to the VM, and now the VM is based on the new template. You can also omit the template ID parameter from the restoreVirtualMachine call. In this case, the VM’s root disk is destroyed and recreated, but from the same template or ISO that was already in use by the VM. Using SSH Keys for Authentication¶In addition to the username and password authentication, CloudStack supports using SSH keys to log in to the cloud infrastructure for additional security. You can use the createSSHKeyPair API to generate the SSH keys. Because each cloud user has their own SSH key, one cloud user cannot log in to another cloud user’s instances unless they share their SSH key files. Using a single SSH key pair, you can manage multiple instances. Creating an Instance Template that Supports SSH Keys¶Create an instance template that supports SSH Keys.
Creating the SSH Keypair¶You must make a call to the createSSHKeyPair api method. You can either use the CloudStack Python API library or the curl commands to make the call to the cloudstack api. For example, make a call from the cloudstack server to create a SSH keypair called “keypair-doc” for the admin account in the root domain: Note Ensure that you adjust these values to meet your needs. If you are making the API call from a different server, your URL/PORT will be different, and you will need to use the API keys.
Creating an Instance¶After you save the SSH keypair file, you must create an instance by using the template that you created at Section 5.2.1, “ Creating an Instance Template that Supports SSH Keys”. Ensure that you use the same SSH key name that you created at Section 5.2.2, “Creating the SSH Keypair”. Note You cannot create the instance by using the GUI at this time and associate the instance with the newly created SSH keypair. A sample curl command to create a new instance is: curl --globoff http://localhost: Substitute the template, service offering and security group IDs (if you are using the security group feature) that are in your cloud environment. Logging In Using the SSH Keypair¶To test your SSH key generation is successful, check whether you can log in to the cloud setup. For example, from a Linux OS, run: ssh -i ~/.ssh/keypair-doc The -i parameter tells the ssh client to use a ssh key found at ~/.ssh/keypair-doc. Resetting SSH Keys¶With the API command resetSSHKeyForVirtualMachine, a user can set or reset the SSH keypair assigned to a virtual machine. A lost or compromised SSH keypair can be changed, and the user can access the VM by using the new keypair. Just create or register a new keypair, then call resetSSHKeyForVirtualMachine. Assigning GPU/vGPU to Guest VMs¶CloudStack can deploy guest VMs with Graphics Processing Unit (GPU) or Virtual Graphics Processing Unit (vGPU) capabilities on XenServer hosts. At the time of VM deployment or at a later stage, you can assign a physical GPU ( known as GPU-passthrough) or a portion of a physical GPU card (vGPU) to a guest VM by changing the Service Offering. With this capability, the VMs running on CloudStack meet the intensive graphical processing requirement by means of the high computation power of GPU/vGPU, and CloudStack users can run multimedia rich applications, such as Auto-CAD, that they otherwise enjoy at their desk on a virtualized environment. CloudStack leverages the XenServer support for NVIDIA GRID Kepler 1 and 2 series to run GPU/vGPU enabled VMs. NVIDIA GRID cards allows sharing a single GPU cards among multiple VMs by creating vGPUs for each VM. With vGPU technology, the graphics commands from each VM are passed directly to the underlying dedicated GPU, without the intervention of the hypervisor. This allows the GPU hardware to be time-sliced and shared across multiple VMs. XenServer hosts use the GPU cards in following ways: GPU passthrough: GPU passthrough represents a physical GPU which can be directly assigned to a VM. GPU passthrough can be used on a hypervisor alongside GRID vGPU, with some restrictions: A GRID physical GPU can either host GRID vGPUs or be used as passthrough, but not both at the same time. GRID vGPU: GRID vGPU enables multiple VMs to share a single physical GPU. The VMs run an NVIDIA driver stack and get direct access to the GPU. GRID physical GPUs are capable of supporting multiple virtual GPU devices (vGPUs) that can be assigned directly to guest VMs. Guest VMs use GRID virtual GPUs in the same manner as a physical GPU that has been passed through by the hypervisor: an NVIDIA driver loaded in the guest VM provides direct access to the GPU for performance-critical fast paths, and a paravirtualized interface to the GRID Virtual GPU Manager, which is used for nonperformant management operations. NVIDIA GRID Virtual GPU Manager for XenServer runs in dom0. CloudStack provides you with the following capabilities:
Prerequisites and System Requirements¶Before proceeding, ensure that you have these prerequisites:
Before continuing with configuration, consider the following:
Supported GPU Devices¶
GPU/vGPU Assignment Workflow¶CloudStack follows the below sequence of operations to provide GPU/vGPU support for VMs:
|