pgBackRest + ZFS Multi-Terabyte TimescaleDB Backup & PITR Architecture
TimescaleOps disaster-recovery dossier replacing multi-hour pg_dump locks on multi-terabyte TimescaleDB hypertables with parallel pgBackRest block-delta PITR and ZFS.
Logical pg_dump exports become impractical once a TimescaleDB sensor archive crosses 500 GB. Pairing pgBackRest block-level incremental backups with immutable compressed hypertable chunks and OpenZFS snapshots protects multi-terabyte databases with minimal daily backup transfer.
| Telemetry Scale | Chunk / Partition Interval | Server DDR5 RAM | Retained Disk Footprint | Matched Infrastructure Tier |
|---|---|---|---|---|
| 300 GB DB + 14d PITR | 1d Chunks |
16 GB DDR5 | 640 GB Total Storage | InterServer KVM 16 Slices |
| 1.5 TB DB + 30d PITR | 6h Chunks |
32–64 GB DDR5 | 4 TB ZFS RaidZ / Mirror | InterServer Dedicated Ryzen |
| 5 TB DB + 60d PITR | 6h / 1h Chunks |
64 GB DDR5 | 12 TB Enterprise Storage | InterServer Storage Server |
| 12 TB DB + Audit Vault | 1h Chunks |
128 GB DDR5 | 20 TB Price-Lock ZFS | InterServer Bare-Metal 2x10TB |
[global]
repo1-path=/zfs-backup-vault/pgbackrest
repo1-retention-full=2
repo1-retention-diff=14
repo1-block=y
compress-type=zst
compress-level=3
process-max=6
archive-async=y
spool-path=/var/spool/pgbackrest
[timescale_prod]
pg1-path=/var/lib/postgresql/16/main
1. Why Immutable Compressed TimescaleDB Chunks Keep Daily Incremental Backups Small
Once TimescaleDB compresses a historical chunk older than your 7-day hot window, its underlying PostgreSQL relation files remain read-only unless manually backfilled. Enabling repo1-block=y in pgBackRest skips unchanged historical blocks and copies only newly written hot chunk pages plus continuous WAL segments.
2. Eliminating Cloud Egress Fees on Multi-Terabyte TimescaleDB Backup Restores
Transferring a 6 TB PostgreSQL restore across metered cloud regions at $0.09/GB incurs $540 in outbound bandwidth charges per recovery drill. Hosting primary and standby ZFS vaults on unmetered 1Gbps/10Gbps InterServer storage nodes keeps restore drills predictable at $0 transfer fee.
3. Instant Writable TimescaleDB Staging Clones Using OpenZFS Copy-on-Write Snapshots
Executing zfs snapshot tank/pgdata@audit and zfs clone gives analytics engineers an isolated, zero-copy writable PostgreSQL test environment over terabytes of historical hypertables in seconds.
TimescaleOps Engineering Q&A: TS-07 // PGBACKREST-ZFS Hypertable Operations
Should OpenZFS compression be disabled on the pgBackRest repository dataset?
When pgBackRest already runs compress-type=zst, set ZFS compression=lz4 (lightweight metadata compression) on the backup dataset so CPU cores never waste cycles double-compressing ZSTD archives.
How frequently should teams verify full TimescaleDB Point-in-Time Recovery (PITR) archives?
Schedule an automated weekly restore job into an isolated container that replays WAL to a target timestamp and queries chunks_detailed_size() to verify chunk integrity.
What ZFS recordsize aligns best with PostgreSQL 16 and TimescaleDB hypertables?
Set recordsize=16K or 32K on hot uncompressed tablespaces (matching 8 KB PostgreSQL pages without read amplification) and 128K on cold compressed archive tablespaces.