10-01-2013 - Was just reminded of a low budget backup alternative I cobbled together while reading and responding to a post over on the IntelliAdmin website, so decided to post the mind-numbingly geeky details of the mechanism here in case I forget where else I might have put them.
All of our agency server instances run as guest OSes on an ESXi host. I have backup jobs scheduled from one of those servers to take care of nightly production data archives, but no automated mechanism for backing up the OS volumes from the host datastore itself. I briefly checked into Veeam's products, but we don't have an Essentials license for our hosts (or the cash for the automation upgrade in Veeam's full poduct), so it was a non-starter. Hot backups would have been nice for VM archiving, but if a window for offline archiving exists then this tool is good for that situation.
I figured out how to perform a command-line sync of folders on the ESXi host’s datastores using a batch script that will THEORETICALLY execute Windows scheduled task (run on scheduled or manually from any host on the same subnet) that:
- runs a batch file that connects PuTTY (free, portable install on server share) via SSH to the ESXi SSH server (which must be enabled and set to run on host startup)
- (will EVENTUALLY, after more testing) issue a shutdown command to the VM (no vSphere client or target VM console connection required)
- runs WinSCP (also free, and portable install on server share) via a command script and kicks off a synchronization between an ESXi datastore folder containing your (shut down/powered off) VM and a local or network folder
- (will EVENTUALLY, after more testing) issue a startup command to the VM
The script to connect the SSH session must be run first, and looks like this:
\\serverpath\putty.exe -ssh [ESXiuser]@[serverIP] -pw [ESXipw]
Running this the first time on a machine prompts for SSL cert confirmation.
The batch script then goes on to spawn a WinSCP sync session:
\\serverpath\WinSCPPE.exe /console /script=\\serverpath\[winscpbacup script].txt
In [winscpbackup script].txt we have:
cd /vmfs/volumes/[targetbackupdatastore ID]/[target folder]
lcd \\[backupserver path]\
option transfer binary
To find the value for [targetbackupdatastore ID], you will need to connect once to the ESXi host with the WinSCP GUI to browse to the datastore folders (from root >vmfs/volumes).
This backup will only run to completion if the target VM files are not locked, requiring powering off the guest OS. This can be done from command line in PuTTY as follows:
For command line shutdown (logged into host using ssh putty)...
To get the current state of a virtual machine:
vim-cmd vmsvc/power.getstate <vmid>
Shutdown the virtual machine using the VMID listed in the first column of output from Step 2 and run:
vim-cmd vmsvc/power.shutdown <vmid>
Note: If the virtual machine fails to shut down, use this command:
vim-cmd vmsvc/power.off <vmid>
Once backup has completed, powering on the machines from command line is as follows:
Check the power state of the virtual machine with the command:
vim-cmd vmsvc/power.getstate <vmid>
Power-on the virtual machine with the command:
vim-cmd vmsvc/power.on <vmid>
This is a wholesale sync of all data in the datastore folders of each VM. In order to reduce redundant data backup managed by other more configurable, finer grained backup tools, it is good practice to put the data drive instance of a server in a separate folder from the OS partition. If that is not possible, the WinSCP script might be enhanced using file type wildcards.
DISCLAIMER: While I have stepped through these procedures manually, I have not yet tried this mechanism as a scheduled task run without my eyeballs on it. I am not sure how the system will handle the putty and winscp instances in non-desktop mode. Will be testing this before the end of the month.
This post is referenced in the Contra Costa ARC Data Backup Summary [internal link].