Backing Up ESX Guest Images

One of the advantages of running VMware ESX Server is the ability to get a backup of the virtual guest image files while the server is running. There are several methods you can use to back up these image files:

1. VISBU. This is a free utility, which you can download at http://www.xtravirt.com/index.php?option=com_remository&Itemid=75&func=fileinfo&id=7. This script copies over the virtual guest images to a different folder on the ESX host hard drive or Windows or Linux file share. Then you can use a tape drive or other backup device to move the images offline.
2. VMware Consolidated Backup. This package is included with the Enterprise version of VMware’s Virtual Infrastructure and works best with the VirtualCenter Management Server. This is a good solution if you’re using a Storage Area Network (SAN) to store your guest images. You can move the virtual guest files off to a Just a Bunch of Disks (JBOD) array or directly attached SAN tape drive.
3. VRanger Pro. This is a third-party application that's available at http://www.vizioncore.com/vRangerPro.html. It is VMotion aware and has the ability to perform file-level restores using the virtual guest backup image. It also copies the virtual guest image files to any accessible Windows or Linux Share. This is also a good solution for virtual guest images stored on a SAN.
4. Linux Agent with Backup Exec. This is a good solution if you’re already running Backup Exec. The main issue with the first three options is that you must have adequate disk space to store the *.vmdk virtual guest files. As you know, with ESX Server you must pre-allocate all the disk space for your virtual guest disks when you create the virtual guest disks. This requires you to have double the disk space of your *.vmdk files available for backup, which can quickly add up if you have a significant number of virtual servers. With the Backup Exec Linux agent you can:
   • Create a Snapshot of the running guests. When you create a snapshot of a virtual server guest, all changes to the virtual guest are written to a snapshot .vmdk file, which frees up the “base” .vmdk file for backup.
   • Back up the “base” .vmdk virtual server disk files on the ESX host by using the Linux Agent and Backup Exec.
   • Delete the snapshot of the running guests. When a snapshot is deleted it merges any changes from the .vmdk snapshot file and then deletes the .vmdk snapshot file.

     The biggest advantage to using this method is you don’t need to copy the virtual guest files to a different folder or share before you back them up to a tape drive. This can significantly reduce the amount of the disk space required for virtual guest image backups. However this method is best suited for virtual guest files on locally attached storage and not on a SAN, because this solution isn't VMotion aware. This is the backup method I’m going to focus on for this article.

   Perform the following steps to back up virtual guest disk files using Backup Exec and the Backup Exec Linux Agent:

   1. Download WinSCP. WinSCP is a great GUI utility for moving files from Windows to ESX Host (and Linux) Servers. It can be downloaded from http://winscp.net/eng/index.php.
   2. Download Putty. Putty is a GUI Telnet/SSH client for Windows. It allows you to have console access to the ESX host. You can download it from http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html.
   3. Download Symantec’s Remote Agent for Linux and Unix Servers (Ralus). You must purchase the Ralus Agent for Linux from Symantec. You must also have a copy of Backup Exec for Windows. Ideally Backup Exec should run with a tape drive on a non-virtualized server. It's possible to run Backup Exec from a virtualized guest, but it complicates the restore process.
      1. Use WinSCP to copy the Ralus Agent to the ESX server. Using Putty, unzip the agent by using this command:

         tar –xvzf ralusagent.tar.gz 
         

         Then run the command

         run ./installralus 
         

         To install the Ralus agent on the ESX host.
      2. Configure the ESX firewall. Check the ESX firewall settings by running

         esxcfg-firewall –q 
         

         and note any open ports. By default, Backup Exec uses a random port for backup. In thisexample we will use ports TCP 8192 - 8199 to run the backup job. Configure the firewall on the ESX host to allow the following ports by using the associated commands:

         esxcfg-firewall –o 10000,tcp,out,BackupExec  (Backup Exec Agent) 
         
         esxcfg-firewall –o 10000,tcp,in,BackupExec (Backup Exec Agent ) 
         
         esxcfg-firewall –o 6101,tcp,out,BackupExec (Backup Exec Agent Browser) 
         
         esxcfg-firewall –o 6102,tcp,out,BackupExec (Backup Exec Agent Browser) 
         
         esxcfg-firewall –o 8192,tcp,in,BackupExec (Specified Port for Backup) 
         
          esxcfg-firewall –o 8193,tcp,in,BackupExec (Specified Port for Backup) 
         
          esxcfg-firewall –o 8194,tcp,in,BackupExec (Specified Port for Backup) 
         
          esxcfg-firewall –o 8195,tcp,in,BackupExec (Specified Port for Backup) 
         
          esxcfg-firewall –o 8196,tcp,in,BackupExec (Specified Port for Backup) 
         
          esxcfg-firewall –o 8197,tcp,in,BackupExec (Specified Port for Backup) 
         
          esxcfg-firewall –o 8198,tcp,in,BackupExec (Specified Port for Backup) 
         
          esxcfg-firewall –o 8199,tcp,in,BackupExec (Specified Port for Backup) 

         Reload the firewall configuration by running the command: esxcfg-firewall—load
   4. Start Backup Exec and select Tools, Options, Network and Security. Click the Enable tcp dynamic port range and enter the range of ports you just opened on the ESX host (8192 – 8199).
   5. Edit the /etc/bkupexec/agent.cfg file. This file should have the following entries at the bottom of the file:

      name 
      force_address 
      export / as root
      tell 
      tell_interval 30
      exclude_dir /dev
      exclude_dir /proc
      exclude_dir /tmp
      exclude_dir /var/run/vmware
      

   6. Restart the Linux Remote Agent. Run the commands: /etc/init.d/VRTSralus.init stop /etc/init.d/VRTSralus.init start
   7. Create an ESX host backup job and select the virtual guests you want to back up. Make sure to note the machine names (including case) of the *.vmx files.
   8. Create a prebackup.sh script in the root folder on the ESX host. The script file will create a snapshot for every virtual guest that you want to back up. In this example, the virtual guest servers are named vs1 and vs2. This file should contain the following commands:

      #!/bin/bash
      vmware-cmd /vmfs/volumes/storage1/vs1/vs1.vmx createsnapshot vs1_snap
      vmware-cmd /vmfs/volumes/storage1/vs2/vs2.vmx createsnapshot vs2_snap
      

      Note that the *.vmx files are case sensitive.
   9. Create a postbackup.sh script to delete the snapshot files that were created in the prebackup.sh script file. The postbackup.sh should contain the following commands:

      #!/bin/bash
      vmware-cmd /vmfs/volumes/storage1/vs1/vs1.vmx removesnapshots
      vmware-cmd /vmfs/volumes/storage1/vs2/vs2.vmx removesnapshots
      

   10. Test the scripts. Test each script by executing the command ./prebackup.sh Start the Virtual Infrastructure Client and verify that the snapshots were properly taken for every running virtual machine that you want to backup. Then execute the command: ./postbackup.sh Now verify that all of the snapshots were deleted on every running virtual machine.
   11. Edit the ESX backup job in Backup Exec. After you've tested the scripts, edit the ESX backup job and enter the pre and post commands by entering /root/./prebackup.sh and /root/./postbackup.sh respectively in the pre and post fields. Check the options: Run Job only if pre-command is successful.
       Run post-command only if pre-command is successful.
       Run post-command even if job fails.
   12. Run the ESX backup job and verify that the virtual guest files were properly backed up.

       I typically run this backup job over the weekend to get images of all of virtual guests running on the ESX host. During the week, I run a “traditional” backup using Backup Exec. In a worst case scenario, I would need to restore the guest image files on a new ESX host, start the guest and then restore the latest full backup from Backup Exec. You can also pre-stage the virtual guest image files on a remote ESX host, and leave them turned off until they're required.

   Restoring ESX Guest Files

   1. Create the Virtual Guest prior to the restore. If you have to restore the virtual guest files on a new ESX host, create the virtual guest on the new host with the Virtual Infrastructure Client before you perform the restore. Make sure to match the exact name and case of the restored machine. This makes it easier to manage the newly restored machine after the restore is completed.
   2. Edit the .vmx file before starting the restored virtual guest. After the restore is complete the .vmx configuration guest file will point to the snapshot image rather than the “base” image of the machine, because that’s the file that was active when the machine was backed up. Make sure to edit the .vmx file for the virtual guest and point it back to the “base” .vmdk rather than the snapshot .vmdk file, otherwise the guest will fail to start.
   A special thanks goes to my co-worker Blandy Allred for figuring out the details of this backup solution.