When running a virtualized vCenter server it is very important to have a working backup. If the vCenter server crashes the virtual landscape cannot be managed or monitored.

Configuring backup is very easy when running VMware vCenter Server on a conventional Microsoft Windows servers because it is a fully-features operating system. In case you're using the VMware vCenter Server Appliance configuring backup might be more complex. By default this system doesn't come with pre-installed backup agents because it is assumed that a "agentless" backup solution for virtual machines is used.

Not every company uses backup solutions that are offering "agentless" backups. If this function is missing the backup is missing as well - which is very unfavorable for productive environments. For example - if you're using EMC NetWorker in an older version agentless backups aren't possible.

The vCSA is based on SUSE Linux Enterprise Server 11 SP2 which means that you can extend its software using RPM packages - if your backup software comes also in a adequate format. Unlike many other virtual appliances I saw recently it is still possible to gain root access to the system. So it is basically possible to installed additional needed software - of course this is not covered by the VMware support.

Installation of EMC NetWorker

EMC generates generic NetWorker RPMs for RPM-based Linux distros. Amongst others this software package is compatible with Red Hat Enterprise Linux and SUSE Linux Enterprise Server - which means that it is suitable for installation inside the vCSA.

It is necessary to resolve some dependencies for EMC NetWorker. The following software packages need to be installed (might differ depending on your NetWorker version):

• libcap1*.x86_64.rpm
• libstdc++33*.x86_64.rpm
• ksh-93u*.x86_64.rpm

And where do you get these packages from? Bascially you'll need a valid subscription for downloading SLES packages. SUSE also offers trial versions for their own enterprise products - including SLES: [klick mich!]

Owners of vSphere Standard or a higher edition also had the possibility to get product patches and updates (ended 07/25/2014, see here) at no fee in the "SUSE Linux Enterprise Server for VMware" programme. To use this it was needed to register the ESXi serial number at SUSE. Afterwards it was possible to activate installations by using activation codes: [click me!]

After a short registration it is possible to download a test version including 60 days free update support. Basically only the first DVD is needed - underneath the folder "suse/x86_64" the required RPM packages can be found. It recommended to download the SLES release SP2 the vCSA is based on instead of SP3.

For installing the EMC NetWorker agent the packages lgtoman and lgtoclient are sufficient. The RPM packages extracted from the DVD are copied to the vCSA using SSH/SCP and installed together with the agent:

1# zypper localinstall lib*.rpm ksh*.rpm lgtoclnt*.rpm lgtoman*.rpm

Service configuration

Before NetWorker is started for the first time it is important to create the required folder structure and a list of valid backup servers. If NetWorker is installed before those information were served the agent isn't working properly and might be reinstalled (because the erroneous information are cached).

1# mkdir -p /nsr/res
2# echo "backupserver.fqdn.loc" > /nsr/res/servers
3# chmod -R 755 /nsr/*
4# chkconfig rpcbind on
5# chkconfig networker on
6# service networker start
7starting NetWorker daemons:
8 nsrexecd

The services rpcbind and networker need to be active to make sure that backups can be created:

1# service rpcbind status
2Checking for service rpcbind running
3# service networker status
4+--o nsrexecd (PID)

Backup scripts

In many companies it is very common to create offline backups weekly, e.g. at the weekend. Machines are often backed up "online" daily which means that application services aren't stopped. As a result not all data can be copied consistently because some files (e.g. database files) are in use. These files are backed up during the offline backup.

For controlling offline backups using EMC NetWorker shell scripts are created. Those scripts are executed before and after runing the backup job.

For creating the backup scripts I was guided by the following VMware KB articles:

• KB 2034505: "Backing up and restoring the vCenter Server Appliance vPostgres database"
• KB 2063260: "Backing up and restoring the vCenter Single Sign-On 5.5 configuration for the vCenter Server Appliance"
• KB 2062682: "Backing up and restoring the vCenter Server Appliance Inventory Service database"

My scripts:

 1# vi /opt/start_offline_backup.sh
 2#!/bin/sh
 3service vmware-vpxd stop
 4service vmware-inventoryservice stop
 5/opt/vmware/vpostgres/1.0/bin/pg_dump INSTANCE -U USER -Fp -c > /tmp/VCDBackUp
 6/usr/lib/vmware-vpx/inventoryservice/scripts/backup.sh -file /tmp/InventoryServiceDB.DB
 7
 8ESC ZZ
 9
10# vi /opt/stop_offline_backup.sh
11#!/bin/sh
12service vmware-vpxd start
13service vmware-inventoryservice start
14
15ESC ZZ
16
17# chmod +x /opt/st*_offline_backup.sh

The variables INSTANCE and USER need to be customized - the appropriate values can be gathered from the configuration file /etc/vmware-vpx/embedded_db.cfg. It is also necessary to check whether the file system /tmp offers enough storage capacity for holding the backups.

Finally I'd like to mention that this procedure is working pretty fine but it is not covered by the VMware support. It is a good idea to revert those changes before opening a support case (or installing appliance updates!).