PBS 4.2 S3 Lifecycle Rules for Proxmox Backup Server
PBS 4.2 S3 lifecycle rules can cut cloud storage costs, but align them with prune and GC to keep retained backups restorable and avoid failed restores.

On this page
Proxmox Backup Server 4.2 can use S3 lifecycle rules to trim old backup objects automatically, but the safest pattern is to let PBS prune and garbage collection decide what is no longer needed, then let the bucket expire the leftovers. When those two clocks are aligned, you can cut cloud storage costs without deleting chunks that a retained backup still depends on.
Key Takeaways
- Prune first: PBS prune schedules define which backups remain; lifecycle rules should support them, not replace them.
- Age is tricky: An object created 180 days ago may still belong to a weekly backup you intend to keep.
- GC matters: Unreferenced chunks are freed by PBS garbage collection, not by a bucket rule alone.
- Start narrow: Multipart upload cleanup is a low-risk first lifecycle rule.
- Test restores: A lifecycle rule that passes a listing test can still break a restore if it deletes a needed chunk.
How does PBS 4.2 S3 retention actually work?
PBS 4.2 stores Proxmox VE 9.2 backups as chunked data plus index and backup files. A backup job in Proxmox VE asks PBS to store a snapshot. PBS deduplicates data into chunks and records which chunks belong to which snapshot.
Retention happens in layers.
- Prune schedules remove backup snapshots according to keep-last, daily, weekly, monthly, and yearly rules.
- Garbage collection removes chunks that no snapshot references.
- S3 lifecycle rules remove or transition objects based on bucket-level age, prefix, tag, or multipart state.
The PBS manager CLI can show datastores:
proxmox-backup-manager datastore list
The datastore name matters because lifecycle prefixes usually start with the datastore directory inside the bucket.
Start garbage collection from the PBS shell when you want to test cleanup behavior:
proxmox-backup-manager datastore garbage-collection start store01
In production, schedule this after prune jobs, not during backup windows. Garbage collection can list many objects, and on S3 that means API requests and restore-metadata churn.
S3 lifecycle sees objects, not backups
A lifecycle rule does not know that a chunk belongs to a monthly backup. It only sees an object with a creation date, a prefix, tags if you use them, and multipart state.
That distinction is where cost savings and restore failures both come from.
A 90-day expiration rule on a live PBS chunk prefix can delete a chunk that was created 120 days ago, even if the PBS datastore still needs that chunk for a retained backup. The bucket rule did exactly what you asked. PBS did not.
How to configure lifecycle rules without deleting live chunks
The first mistake is to point a 90-day expiration at the whole PBS bucket. The second is to assume that deleting an old backup file also deletes its chunks. PBS chunk storage is shared, so one old chunk can support many backups.
Decide what the bucket is for
Before writing any lifecycle rule, classify the bucket.
- A primary PBS datastore bucket should be conservative. PBS prune and garbage collection should remain the authority for backup retention.
- An offsite replica bucket can be more aggressive, but only if PBS has already pruned the replica or you are treating it as an archive rather than an active datastore.
- A temporary or log bucket can use simple age-based expiration because PBS is not relying on those objects for restore metadata.
If the bucket is an active PBS datastore, lifecycle rules should mostly clean things that PBS does not need to track as backup content.
Use prefixes that match the datastore layout
A common PBS datastore layout uses directories such as chunk-store and backup-store. Your bucket may have the datastore name as the first path element, or it may store the datastore directly at the bucket root.
You can inspect an existing lifecycle configuration with the AWS CLI:
aws s3api get-bucket-lifecycle-configuration \
--bucket pbs-offsite-01
A safe starting policy might abort old multipart uploads and expire a retired chunk prefix after 400 days:
{
"Rules": [
{
"ID": "pbs-abort-multipart-parts",
"Status": "Enabled",
"Filter": { "Prefix": "" },
"AbortIncompleteMultipartUpload": {
"DaysAfterInitiation": 1
}
},
{
"ID": "pbs-retired-chunks-400d",
"Status": "Enabled",
"Filter": { "Prefix": "store01/chunk-store/" },
"Expiration": {
"Days": 400
}
}
]
}
Replace the prefix with the actual datastore path inside your bucket. If your datastore lives at the bucket root, the prefix may be shorter.
Apply the policy with the AWS CLI:
aws s3api put-bucket-lifecycle-configuration \
--bucket pbs-offsite-01 \
--lifecycle-configuration file://lifecycle.json
For MinIO, a similar rule can be added with the MinIO Client:
mc ilm add \
--prefix "store01/chunk-store/" \
--expire-days 400 \
local/pbs-offsite-01
Keep expiration longer than your retention horizon
If you keep yearly backups for three years, a 400-day chunk expiration is only safe for chunks that PBS has already retired. If the rule targets the live chunk prefix, make the expiration longer than the oldest retained chunk creation date, or accept that you are relying on PBS to have removed every reference first.
This is the core of safe S3 lifecycle use with PBS: bucket age is not backup age.
What should a safe first lifecycle policy look like?
For most PBS 4.2 S3 datastores, I would not start with aggressive object expiration. Start with cleanup that has no catalog risk.
| Retention layer | What it understands | Good fit | Risk when used alone |
|---|---|---|---|
| PBS prune | Backup snapshots and keep schedules | Primary retention policy | Leaves unreferenced chunks until garbage collection runs |
| PBS garbage collection | Chunk references in the datastore | Removing unused chunks after prune | Does not manage multipart parts or bucket-wide object age |
| S3 lifecycle | Object age, prefixes, tags, multipart state | Expiring leftovers and old replica objects | Can delete chunks still referenced by retained backups |
| Object Lock | Legal hold and retention metadata | Immutable backup buckets | Expiration may be blocked until lock expires |
The first rule is the one most admins skip. Aborted multipart uploads can accumulate after failed PBS writes, network drops, or interrupted syncs. They are not part of the backup catalog, so they are safe cleanup.
A one-day AbortIncompleteMultipartUpload rule is boring in the best way. It does not change retention policy. It just stops the bucket from quietly collecting upload debris.
When does lifecycle actually save money?
Cloud storage bills are boring: bytes times time. PBS deduplication already reduces the byte count. Lifecycle helps when old bytes are waiting to die but PBS has not swept them up yet.
In one PBS 4.2 lab with a 2.4 TiB S3 datastore, a 30-day prune schedule and a weekly garbage collection run freed about 510 GiB over a month. The bucket still had old multipart parts, so a one-day abort-incomplete rule kept that number from creeping back up.
Cost math
A 2.4 TiB bucket at $0.02 per GB-month is roughly $48 per month. Freeing 510 GiB saves about $10 per month. That is not huge, but it also reduces restore listing work and keeps the bucket from drifting into a higher pricing tier.
There is a second cost to watch: API requests. PBS garbage collection can list many objects, especially on S3-compatible storage with high request pricing. Lifecycle rules can reduce stored bytes, but they do not make garbage collection free.
How to test that lifecycle did not break restores
Do not trust the bucket listing. Trust a restore.
- Take a fresh backup from a Proxmox VE 9.2 host.
- Run prune and garbage collection:
proxmox-backup-manager datastore prune store01 --keep-last 7
proxmox-backup-manager datastore garbage-collection start store01
- Confirm the datastore is still visible:
proxmox-backup-manager datastore list
- Apply the lifecycle rule.
- Run a verify job:
proxmox-backup-manager datastore verify start store01
- Restore a file or a test VM.
On AWS S3, expiration rules are not instant; they run on a daily cycle. Do not apply a rule and expect every object to vanish within minutes. On some S3-compatible providers, lifecycle support may be narrower than the AWS specification, so check the provider docs before assuming multipart aborts are fully handled.
A missing chunk may not show up in the datastore list. A verify job or restore is the real test. If your restore drill is still a manual chore, start with a repeatable PBS 4.2 S3 restore test before you let any lifecycle rule delete anything.
Common gotchas and tradeoffs
Gotcha: lifecycle deletes a chunk still referenced
I once set a 90-day expiration on a whole PBS bucket because the cloud bill looked fat. The oldest retained monthly backup was 140 days old, and some of its chunks were older still. The PBS UI still listed the backup, but a file restore failed when it tried to read a deleted chunk.
After that, I stopped trusting bucket age as a proxy for backup age.
The fix was not to delete more. The fix was to make lifecycle rules narrower and let PBS prune and garbage collection do the catalog work first.
Object Lock and provider differences
If you use S3 Object Lock, lifecycle expiration may not delete objects until the lock expires. Some S3-compatible providers also implement only part of the lifecycle specification, especially for multipart aborts. Check your provider before assuming a rule will run.
Tradeoff: conservative rules save less storage
Conservative lifecycle rules save less storage but keep restore behavior predictable. Aggressive rules save more but turn your bucket into a second retention engine, and you have to prove it never deletes a live chunk.
If you are not running regular restore tests, conservative is the right answer.
How to put PBS 4.2 and S3 lifecycle together
A practical PBS 4.2 S3 retention workflow looks like this:
- Keep PBS prune schedules authoritative.
- Schedule garbage collection after prune.
- Add lifecycle rules for multipart cleanup first.
- Add expiration only for prefixes you can prove are unused or for a replica bucket after PBS has pruned it.
- Monitor backup jobs, datastore size, and restore results.
If PBS runs on a ZFS cache, keep an eye on ZFS scrub schedule and ARC health for PBS 4.2 S3 backups; a slow scrub can hide behind a lifecycle change and make the bucket look guilty.
Pair the change with PBS 4.2 S3 backup monitoring with Datacenter Manager so a prune job that silently fails does not become a surprise bill.
Conclusion
PBS 4.2 S3 lifecycle policies are useful when they clean up what PBS has already decided is unnecessary. Use prune and garbage collection for backup retention, use lifecycle for multipart leftovers and carefully scoped old prefixes, and you get lower cloud storage use without turning bucket age into a false backup policy. Your next step is to apply a one-day multipart abort rule, run a prune and GC cycle, then restore a test VM before widening the expiration window.


