Short of adding special backup tools to Virtual Server or ESX Server, the recommended practice for backing up VMs is to install a traditional backup software agent on them and treat them as actual hardware-based systems. Of course, the virtual nature of the systems and the fact that the files comprising the systems are located on a SAN volume afford some flexibility. That said, if your requirements dictate offsite backup media storage or other similar considerations, you might still want to give VMs the traditional backup treatment.
Assuming you can afford a little downtime, the simplest way to back up a Virtual Server–based VM is to stop the system and copy the VHD and VMC files to a backup location. You can even script this type of operation. (See the Learning Path for a good sample script.) On the Web, you can find a few iterations of a script that uses Volume Shadow Copy Service (VSS) to perform a live backup of a VM. These scripts make use of Vshadow (vshadow.exe), available as part of the VSS SDK, which is freely downloadable from Microsoft. I didn’t attempt to use one of these scripts because of reported mixed results regarding consistency of the VMs and unexpected shutdown messages on the guest OSs. If your goal is to ensure consistent trouble-free backups of VMs, you should stick with a Virtual Server–compatible backup product, such as Microsoft System Center Data Protection Manager (DPM), which has built-in Virtual Server backup support.
For data protection on ESX Server VMs, assuming you can afford the licensing, the best choice is VMware Consolidated Backup. Consolidated Backup integrates with supported third-party backup software to create a seamless backup through a process whereby a proxy backup server creates a snapshot of the VM on the SAN. That snapshot is then mounted on the proxy backup server and backed up to disk or tape by the third-party software. The result is minimal performance impact on the actual ESX Server system and no downtime for the VM. Numerous other viable alternatives exist, as you’ll find in the Learning Path