#!/usr/bin/env sh
zpool import backups
/usr/local/bin/zap rep -Fv
/usr/local/bin/zap destroy
zpool export backups
camcontrol standby da0
A while back I tried to develop a script to handle ZFS backups from my server to my external hard drive. Well, in the end I really didn’t like what I came up with and thought it best to look at sysutils/zap to do my backups. The tool has worked great for me and reliably does snapshots and sends them to my destination. I have ran into some issues though. Maybe this will come by the developer or maybe I can learn to make the improvements I would like to see…or maybe there’s something better! But, I’d rather improve this tool than redo backups if it came down to it.
My Backup Process
I have a 3-2-1 process of doing backups
3 copies of data 2 on different media 1 offsite
I won’t go through any of the setting up because the GitHub repository and a blog linked in the developer’s website is plenty sufficient to get started.
With it set up, I rotate my drive out every month. Since this is just me and my family, I don’t have a more rapid rotation schedule. I have a cron job that will daily take a snapshot that has a TTL of 6 weeks, and once a month takes a snapshot with a 3 month TTL.
This imports the zpool, does the replication, deletes old snapshots, exports, and lastly does a standby on the drive so it powers down until next time.
A Problem with Rotations
Not too long ago, I found there was an issue with drive rotation. Namely,
there is an order to how to get the drive rotated in up to date. If I
just plugged in my drive and did a zap rep
, I would not get everything
to sync or get errors. That is because there is a mismatch in snapshots
between the source and destination. So, I found the best method is to
-
import the zpool
-
run
zap destroy
to delete old snapshots on the backup drive -
run
zap rep -Fv
to get everything up to date
The other way around does not work as you would expect.
Lazy Roation?
Just last night I came across an issue where I had everything automated and "set and forget" my backups for a couple months.
Important
|
Bad idea by the way |
I found out 2 datasets were not syncing. Somehow I missed them somewhere and I had all my ZAP snapshots get destroyed, likely due to no new ones getting sent over somehow and then getting destroyed because they expired.
With that incident, I was not getting any new ZAP snapshots syncing to my external hard drive. I wish I had the exact message, but the issue was that there was an older snapshot on my external drive that wasn’t taken by ZAP, and thus it would not send a snapshot.
My solution then was to delete all snapshots on the destination for that ZFS
dataset, then zfs send data/my/snapshot@ZAP_oldest_one | zfs receive backups
and follow that with a zap rep -Fv
which would sync from my oldest ZAP
snapshot to my newest one. I did that for the 2 datasets I had with this
issue and backups were properly underway.