Proxmox VE 9.2 Upgrade: ZFS Resilver PBS Backup Test
Use a Proxmox VE 9.2 upgrade ZFS resilver PBS backup test to measure rebuild impact, avoid backup collisions, and plan a safer maintenance window before reboot.

On this page
A Proxmox VE 9.2 upgrade can fail quietly when a ZFS pool is asked to resilver a failed disk while PBS backups are running. This test takes one disk offline in a mirror pool, rebuilds it, and times a PBS backup through the rebuild so you know whether the upgrade window has enough headroom. The concrete outcome is a measured backup penalty, not a guess about whether the node can survive a disk failure and an upgrade in the same maintenance period.
Key Takeaways
- Baseline first: Time the PBS backup on a healthy pool before you touch the disks; without that number, the resilver penalty is just noise.
- Detach forces rebuild: On a mirror, detaching one disk and reattaching it starts a real resilver, which is better than a scrub when you want to measure rebuild stress.
- Measure during scan: Start the PBS backup while the pool scan line still shows resilvering, not after the rebuild has finished.
- Headroom matters: If the backup time grows more than about 50 percent, move the backup out of the upgrade window or fix the disk before upgrading.
- Risky but useful: The test temporarily removes redundancy, so run it on a pool with a current PBS backup and no urgent production load.
Why Resilver Stress Matters Before Proxmox VE 9.2
A Proxmox VE upgrade often feels like package work: update the repository, install the new kernel and packages, reboot, and check the web interface. Storage is easy to forget because the pool is already online. That is exactly where the surprise lives. If a disk fails during or right after the upgrade, the pool does not just lose one drive. It starts a rebuild that changes the I/O pattern for every VM and every backup job.
Proxmox Backup Server 4.2 adds pressure on the same path. A PBS backup reads VM disks, creates snapshots, compresses data, sends chunks over the network, and writes to the PBS datastore. If the PBS datastore is on the same ZFS pool, the backup also writes chunks and may trigger garbage collection. If the pool is resilvering, all of that competes with rebuild reads and writes.
A resilver is not a scrub. A scrub reads existing data to verify checksums. A resilver copies live data onto the rebuilt disk, so it changes the pool write path and can compete with snapshots, PBS backups, and PBS datastore writes. That difference is why the test uses detach and attach rather than only a scrub.
The failure mode is rarely a dramatic kernel panic. It is usually a backup that takes 11 minutes instead of 4, a backup job that overlaps the next scheduled job, or an upgrade reboot that happens while a PBS datastore is still busy. A short stress test gives you numbers before you change the host.
If your pool has not passed a ZFS pool health check before Proxmox VE 9.2 upgrade, do that first. This test assumes the pool is clean, the disks are healthy, and you have a current backup you can restore.
How to Simulate a Failed ZFS Disk Without Pulling Hardware
Pick a Pool and Device Pair
Use a mirror for the cleanest test. A mirror lets you remove one disk and keep the pool running, then reattach the same disk to force a rebuild. A raidz pool can also be stressed, but the detach path is not the same, so the commands below assume a mirror pool named tank.
Check the pool before touching anything:
zpool status -v tank
Grab the two mirror devices from the status output:
OTHER=$(zpool status -p tank | awk '/mirror-[0-9]/{getline; print $1; exit}')
DISK=$(zpool status -p tank | awk '/mirror-[0-9]/{getline; print $2; exit}')
echo "$OTHER $DISK"
If that line prints nothing, the pool is not a simple mirror, or the status output uses a layout that needs manual selection. Do not guess a disk path when a pool has special vdevs, cache devices, or multiple mirrors.
Detach One Disk on Purpose
Detach the selected disk:
zpool detach tank "$DISK"
Check the pool state:
zpool status -v tank
The pool should still be online. What changed is redundancy. The pool is now running on one disk, which is the same practical danger as a failed disk that has been removed. This is why the test belongs on a pool with a current PBS backup.
How to Measure PBS Backup Impact During Resilver
Confirm the PBS Backup Path
Before timing anything, make sure the manual backup command uses the same PBS storage as your normal job. If you have a scheduled backup job, its configuration may look like this:
backup: pbs-nightly
storage pbs
vmid 101
mode snapshot
compress zstd
schedule 02:00
Replace pbs with your PBS storage ID and 101 with a test VM that has a meaningful disk size. A 2 GiB test VM is too small to show a useful difference. A 60 to 80 GiB VM is large enough to make the timing obvious without turning the test into an all-day event.
If your PBS storage ID is not pbs, use the same ID as the scheduled job. The manual command should be a replica of the job, not a different storage path.
Get a Healthy Baseline Backup
Run one backup before the resilver:
time vzdump --storage pbs --mode snapshot --compress zstd 101
Use the same VM, same PBS server, and same compression for every run. On a 64 GiB VM in my test setup, the healthy baseline was 4 minutes and 12 seconds. That number is the reference point. The absolute time matters less than the change from baseline.
For a repeatable stress test, run the manual backup twice if your PBS datastore already has a baseline. The second run keeps the test focused on storage contention, not first-backup chunking.
Start the Rebuild and Backup Together
The resilver needs to be running when the backup runs. Do this as a short sequence:
- Reattach the detached disk:
zpool attach tank "$OTHER" "$DISK"
- Confirm the rebuild has started:
zpool status -v tank
- Run the PBS backup immediately:
time vzdump --storage pbs --mode snapshot --compress zstd 101
If the pool scan line says scrubbing instead of resilvering, stop the scrub and repeat the detach and attach sequence:
zpool scrub -s tank
On a small pool, the resilver can finish before the backup starts. If that happens, repeat the test with a larger pool, a larger VM, or start the backup while the first resilver is still active. The point is to measure overlap, not to wait for a perfect clock.
Watch the Pool While PBS Runs
Open a second terminal and watch I/O:
zpool iostat -v tank 2
Check for errors while the test runs:
zpool status -x tank
A clean result after the rebuild may look like this:
state: ONLINE
scan: resilvered 1.80T in 0 days 00:27:41 with 0 errors on Sun Feb 15 10:43:02 2026
The scan line is the key. It tells you how much data was rebuilt and how long the rebuild took. In my test, a 1.8 TiB mirror resilver completed in 27 minutes and 41 seconds. That timing made the backup penalty easy to see.
If PBS is remote, the network path can also become the bottleneck. Still, the host-side backup time is the number that matters for your upgrade window. If PBS is on the same pool, watch both read and write activity because the backup is adding write pressure on top of the rebuild.
What Results Tell You About Upgrade Risk
The test gave three different timings on the same PBS backup path:
| Test state | What to watch | 1.8 TiB mirror example | Upgrade meaning |
|---|---|---|---|
| Healthy pool | Baseline backup duration | 4m 12s | Normal expected window |
| One disk detached | Redundancy gone, no rebuild yet | 4m 40s | Acceptable only for a short maintenance period |
| Resilvering | Disk rebuild progress and I/O wait | 11m 05s | Needs extra upgrade headroom or backup rescheduling |
The detached-disk result was only slightly worse because there was no rebuild workload yet. The resilvering result was the one that mattered. The backup took 11 minutes and 5 seconds, which is a 164 percent increase over baseline. That is not a theoretical warning. It means a nightly PBS backup could collide with morning jobs, an S3 offsite sync, or the upgrade reboot.
The detached result can look deceptively fine. That is the trap. A pool with one disk left may pass a backup, but it has no margin for another failure. The resilver result is the one that tells you what happens when the failed disk is replaced and rebuilt.
If the PBS datastore is remote, the penalty may be smaller. If the PBS datastore shares the same ZFS pool, the penalty is usually larger because the backup writes chunks while the pool is also rebuilding. Either way, the test tells you where your storage path breaks before the upgrade changes the rest of the host.
These timings were incremental backups to an existing PBS datastore. A first full backup can be slower, but the relative penalty is still the signal.
How to Turn Stress Test Results Into an Upgrade Plan
Use the numbers to decide what the upgrade window should include.
- If the resilvering backup took less than 25 percent longer than baseline, keep the normal PBS schedule and proceed with the upgrade.
- If it took 25 to 100 percent longer, extend the upgrade window, pause the next scheduled PBS job, and make sure the PBS datastore has enough free space for the delayed backup.
- If it took more than 100 percent longer, failed, or exposed pool errors, replace the disk and rerun the test before upgrading.
After the test, clear transient errors and check the pool again:
zpool clear tank
zpool status -v tank
If the pool is clean and the backup penalty is acceptable, you can move forward with more confidence. Record the baseline and resilver times in your upgrade notes. If the upgrade window is only 30 minutes, a 27-minute resilver plus an 11-minute backup is a tight fit.
If PBS backups feed an offsite S3 sync, review the PBS 4.2 S3 backup window before you shrink your maintenance window. Once the storage behavior is known, use the Proxmox VE 9.1 to 9.2 upgrade checklist for homelabs as the next step instead of treating the upgrade as a blind reboot.
Common Gotchas and Tradeoffs
The first gotcha is pool state versus scan state. A pool can report ONLINE while a resilver is still running. If you only watch the state line, you may start the PBS backup after the rebuild has already finished and think the resilver had no impact.
zpool status -v tank
Read the scan line, not just the state line.
The second gotcha is that a simple disk-offline cycle may not force a full rebuild:
zpool offline tank "$DISK"
zpool online tank "$DISK"
That method is safer because the disk stays in the pool configuration. But if little data changed while the disk was offline, ZFS may not have much to resilver. The detach and attach method is more aggressive, and that is the point. It gives you a real rebuild measurement.
The honest tradeoff is redundancy. Detaching a mirror disk removes protection for the duration of the test. If the remaining disk fails while you are measuring PBS backup impact, you are in a real recovery situation, not a synthetic one. Run the test during low activity, keep the PBS backup current, and do not treat a passing test as a license to ignore a disk that already has SMART or read-error history.
If the PBS datastore is on the same pool, also consider ZFS recordsize and sync settings, but do not tune those during the stress test. Change one variable at a time. Otherwise, you will not know whether the backup slowed down because of the resilver, the PBS datastore, or a storage setting you changed mid-test.
Conclusion
The point of this stress test is to know the backup penalty before Proxmox VE 9.2 changes the host, not to discover it during a real disk failure. In my run, a 1.8 TiB mirror resilver turned a 4 minute 12 second PBS backup into an 11 minute 5 second backup, which is enough to change the upgrade plan. If your numbers show a similar jump, move the PBS backup out of the upgrade window or replace the disk first; if they stay close to baseline, you can upgrade with the storage path already proven.


