I was asked by a client for a robust backup solution for their Centos 7 server, that would allow a bare metal restore in the event of a server failure/repair. Having done a bit of research I decided on Mondorescue as this also provides individual directory/file restore as well.

This is what I was aiming to provide :

Mondorescue solution

Mondo install

We are going to assume here that the Centos server has a direct connection to the internet, or an indirect connection is available via a proxy server.

Configure the “proxy” entry in /etc/yum.conf as necessary.

As root, add these four repo files to /etc/yum.repos.d

afio.repo :

[afio]
name=rhel 7 x86_64 - afio Vanilla Packages
baseurl=ftp://ftp.project-builder.org//rhel/7/x86_64
enabled=1
gpgcheck=1
gpgkey=ftp://ftp.project-builder.org//rhel/7/x86_64/afio.pubkey

buffer.repo :

[buffer]
name=rhel 7 x86_64 - buffer Vanilla Packages
baseurl=ftp://ftp.mondorescue.org//rhel/7/x86_64
enabled=1
gpgcheck=1
gpgkey=ftp://ftp.mondorescue.org//rhel/7/x86_64/buffer.pubkey

mondorescue.repo :

[mondorescue]
name=rhel 7 x86_64 - mondorescue Vanilla Packages
baseurl=ftp://ftp.mondorescue.org//rhel/7/x86_64
enabled=1
gpgcheck=1
gpgkey=ftp://ftp.mondorescue.org//rhel/7/x86_64/mondorescue.pubkey

pb.repo :

[pb]
name=rhel 7 x86_64 - pb Vanilla Packages
baseurl=ftp://ftp.project-builder.org//rhel/7/x86_64
enabled=1
gpgcheck=1
gpgkey=ftp://ftp.project-builder.org//rhel/7/x86_64/pb.pubkey

These all actually came from ftp.mondorescue.org

Then to install use :

yum install -y mondo

Backups

The command mondoarchive can be used to interactively to backup your server, but we’d usually want an unattended scheduled backup to run using a command like:

mondoarchive -OViN -d /mnt/backups -9

/mnt/backups is actually a NFS share from elsewhere so we keep our backups off-server.

This would effectively be our level 0 (full) backup.

We could then do daily incrementals with :

mondoarchive -DOViN -d /mnt/backups -9

…(notice the ‘D’ option) which will only backup changes to the existing ISO files created previously.

NOTE : Using the same directory for incrementals will OVERWRITE the full backup – so it’s a good idea to create a wrapper script for mondo to be called by cron, specifying full or incremental. The script will then ensure the files go to specific directories.

A simple example :

# Script to call mondoarchive from cron
backupdir=/mnt/backups
remotebackupdir=/backups
remotenfshost=chef-workstation
datestamp=$(date +%Y%m%d)
mounted=$(df -h | grep -I $remotenfshost)
[[ "$mounted" = "" ]] && mount -o nolock ${remotenfshost}:${remotebackupdir} ${backupdir}
 
case $1 in 
   full) [[ ! -d ${backupdir}/$(hostname)/${datestamp}-full ]] && mkdir -p ${backupdir}/$(hostname)/${datestamp}-full
         mondoarchive -OViN -d ${backupdir}/$(hostname)/${datestamp}-full -9
         ;;
   inc) [[ ! -d ${backupdir}/$(hostname)/${datestamp}-inc ]] && mkdir -p ${backupdir}/$(hostname)/${datestamp}-inc
         mondoarchive -DOViN -d ${backupdir}/$(hostname)/${datestamp}-inc -9
         ;;
   *)    echo "Invalid option" && exit 1
         ;;
esac
 
exit 0

Restores

For bare metal restore, burn the ISO files to CD/DVD’s and boot the server from the first one, when at the prompt type “nuke” [enter] and it should prompt you through the restore process.

If you are using virtual Linux servers, you can of course simply mount the ISO virtually to the server you are aiming to restore.

Then once the full (level 0) restore is done, mount the backup directory and use mondorestore to restore any files from incremental backups that may not have been caught by the full backup.

For dir/file restores use “mondorestore” and you’ll get a text-based interface to help you select and restore the directories/files you need from the latest mondo ISO’s generated.

You could also kick off an interactive restore like this :

mondorestore -id /mnt/backups/centos7-minimal.troyski.co.uk/20160126-inc -Z interactive

Hit OK at the “Please insert tape/CD…” and then choose Hard Disk. The Storage dir will be populated for you, so just hit OK, and use the default Prefix (mondorescue) by hitting OK again – it will start “Thinking” and then present you with a curses-based list of directories which you can select for restore, or hit More to enter each one and select individual files if necessary by pressing [Enter].

Hitting OK here will begin the restore of whatever you have selected.

I’ve successfully backed up, performed bare metal restores, and file-level restores to virtual Linux servers using this method. Although it may be just as easy to use virtual snapshot backups, it was a good proof of concept.

Update : I recently repeated this for some physical servers, but found the mondoarchive command would fail during the mindi phase with a “Not enough space” issue related to the use of ramdisk it creates. I fixed this by adding the following to the /etc/mindi/mindi.conf file:

EXTRA_SPACE=360000
BOOT_SIZE=360000