PBS 4.2 Parallel Sync Jobs Shrink Your Backup Window

Key Takeaways

Sync vs backup — A sync job copies already-created backups between storages; a backup job reads VM disks to create new ones, so they solve different problems.

Parallelism — PBS 4.2 pipelines multiple VMs through a sync job at once instead of finishing them one at a time.

Bigger win for many VMs — The window shrinks most with several large VMs, not one tiny container.

Resource cost — Parallel transfers raise CPU and network load, so schedule them off-peak or add a bandwidth cap.

Incremental depends on source — An incremental sync job needs its source backups intact or it falls back to a full copy.

How do sync jobs actually work?

Many people confuse sync jobs with backup jobs, and that confusion costs time when something goes wrong. A backup job runs on your Proxmox VE node: it reads VM disk data and writes new chunks to PBS. A sync job runs on PBS itself and copies chunks that already exist from one storage to another — usually to a remote PBS server for offsite retention or to a second local storage for redundancy.

That distinction matters because a sync job only needs the backup chunks to exist on disk; it never reads the VM's virtual disk. So it can keep replicating safely even after the source VM has been shut down or moved to another node. This is also why it pairs naturally with a ZFS replication strategy for protecting data between nodes — see how ZFS replication on Proxmox works for a comparable chunk-level approach.

What a sync job actually copies

When you create a sync job, you point it at a source storage on the local PBS server and a target on a remote PBS server. PBS then walks the chunk store and transfers any chunk the target doesn't already have.

The first run copies everything — a "full" sync. Every run after that is "incremental," meaning PBS only transfers chunks that changed since the last sync. This is what makes incremental sync jobs cheap: after the initial load, most days move just a few megabytes per VM.

The target storage is where your replicated chunks live, so its layout matters just as much as the source. Whether you're using ZFS or LVM-thin for backup storage is worth a look before you point a sync job at a new folder.

The building blocks under the hood

Every PBS backup is a set of compressed, content-addressed chunks. Your VM's daily incremental backup is just the new chunks since the previous snapshot. Sync jobs move those chunks verbatim — they don't recompress or re-encrypt them, which is why a sync job is far cheaper than re-running the backup job itself.

This is the same idea behind getting the most out of your ZFS snapshots for backups: the snapshot is the cheap, repeatable unit, and replication only ever moves what changed.

How do parallel sync jobs shrink your backup window?

In PBS versions before 4.2, a sync job processed one VM's chunk set at a time. It would finish all of VM A's chunks, then move on to VM B, and so on. With a handful of small VMs that's fine, but once you have ten or twelve VMs with gigabytes of daily data, the job queues up and your window balloons.

PBS 4.2 changes that by letting a sync job pipeline multiple VMs concurrently. Instead of the serial chain above, the job fans out: it reads chunks from several VMs in parallel and writes them to the target at the same time. On the box where I first turned this on — twelve VMs, a ~200 GB backup store, and a single 1 Gbps link to the offsite server — the initial full sync dropped from nearly four hours to about 90 minutes. Daily incremental resyncs, which only move the changed chunks, finished in well under an hour.

How the parallelism actually happens

PBS splits the work across multiple transfer threads and streams. Each VM's chunk set gets handled by its own stream, and the scheduler distributes chunks across the available connections to the target. The more VMs you have, the more the parallelism pays off — which is why a single-VM job sees little benefit. On my box the default was streaming one VM at a time, and raising the parallelism control in the job settings let several stream at once. If your UI labels it differently, look for the setting that lets a job run multiple VMs concurrently and bump it up.

How to configure a parallel sync job

You can set this up either in the web UI or from the command line. Both paths hit the same result; I usually use the CLI when I'm scripting the target storage.

Through the web UI

On the source PBS server, go to Datacenter > Backup > Sync Jobs and click Add Sync Job. Fill in the fields:

• Source storage — your local storage holding the backups (for example, local)
• Target server — the remote PBS hostname
• Target user — a user on the target, typically admin@pam
• Target storage — the storage folder on the remote server
• Frequency — daily, weekly, or monthly
• Bandwidth limit — leave blank for unlimited, or type a cap

Then raise the parallelism/concurrency setting to let multiple VMs stream at once.

Through the CLI

The same job through pbsync:

pbsync add \
  --storage local \
  --target pbs-backup.example.com \
  --targetuser admin@pam \
  --targetstorage offsite \
  --freq daily \
  --bwlimit 20MB/s

Verify it registered and see how it's scheduled:

pbsync list

To delete it later:

pbsync remove 1

After you add the job, it runs on its schedule. Watch progress in the Sync Jobs view or from the CLI with pbsync list, and confirm the target server reports the incoming chunks.

What's the catch?

Parallelism is free speed only when your hardware can absorb it. The honest tradeoff is resource usage: a parallel sync job holds multiple transfer streams open at once, which spikes CPU, network, and disk I/O on both the source and target.

On my homelab box, that meant I had to move the sync job off the daytime schedule. Running a full parallel sync during business hours competed with the live VMs for the same 1 Gbps NIC, and I could see web pages loading slower in the browser. Moving the job to 1:00 AM fixed it — the VMs were idle, the link was free, and the parallelism paid off cleanly.

Two more things to watch:

• Incremental needs its source. An incremental sync job only transfers changed chunks, which assumes the source backups still exist. If you prune or delete those source backups, the job can't finish the increment and falls back to a full sync — which is a big window if it happens unannounced. This is the same lesson as keeping your ZFS snapshots healthy for backups: the cheap operation only works while the underlying data is intact.

• Bandwidth and parallelism interact. If you cap the bandwidth too low, the job splits that single cap across all its streams, so you lose most of the speedup. Set the cap high enough that each stream still has room, or accept a longer but steadier transfer.

Sequential vs parallel at a glance

Aspect	Sequential sync job	Parallel sync job
Transfers	One VM's chunks, then the next	Several VMs at once
Full-sync time	Longer (hours for big stores)	Shorter (often under two hours)
Resource load	Spikes per stream	Sustained across streams
Link utilization	Can leave the link idle	Fills a single link more fully
Risk to live VMs	Lower if unscheduled	Higher if run during peak hours

Conclusion

Parallel sync jobs in PBS 4.2 turn a serial, hours-long replication into something that finishes in under two hours by moving multiple VMs' chunks at once. The payoff is biggest with many VMs and a constrained link, but it costs CPU, network, and disk I/O — so schedule it off-peak or add a bandwidth cap. To get started, add a sync job on your source PBS server, point it at the remote storage, and raise the parallelism setting to see the window close in real time.