How TimescaleOps Calculates Hypertable chunk_time_interval, Columnar Compression & NVMe Retention
Transparent PostgreSQL 16 and TimescaleDB mathematical equations powering the TimescaleOps IoT sensor ingestion, 25% RAM chunk interval, and NVMe retention calculator.
Every capacity figure and SQL policy emitted by TimescaleOps is derived deterministically from documented PostgreSQL 16 HeapTupleHeaderData byte geometry, B-tree page residency limits, and Gorilla XOR / Delta-of-Delta / ZSTD columnar encoding specifications — never from unverified synthetic claims.
// Core TimescaleOps Deterministic Equations (src/sizer-engine.mjs)
rows_per_sec = sensor_count * sampling_hz
rows_per_day = rows_per_sec * 86,400
heap_bytes_per_row = ceil((40 + metrics_per_row * 4) / 8) * 8
index_bytes_per_row = 18
daily_raw_gb = rows_per_day * (heap_bytes_per_row + index_bytes_per_row) / 1e9
// 1. 25% DDR5 RAM Active Hypertable Chunk B-Tree Rule
ram_target_25pct_gb = 0.25 * recommended_ram_gb
active_chunk_idx_gb = chunk_days * daily_index_gb * residency_factor
optimal_chunk = max(interval in ['7d','1d','6h','1h']
where interval <= hot_days
and active_chunk_idx_gb <= ram_target_25pct_gb)
// 2. Tiered Hot Uncompressed NVMe + Compressed Cold Archive Footprint
hot_uncompressed_gb = daily_raw_gb * hot_days * 1.12
compressed_cold_gb = (daily_heap_gb / compression_factor) * 1.28 * cold_days
total_retained_nvme = ceil((hot_uncompressed_gb + compressed_cold_gb) * 1.20 / 5) * 5| Model Constant | Deterministic Value | Official Documentation Basis |
|---|---|---|
HeapTupleHeaderData |
23 bytes + 1B null bitmap (8B MAXALIGN) | PostgreSQL 16 Storage Page Layout Documentation |
Composite B-Tree Index Row |
~18 bytes / row | PostgreSQL 16 B-Tree Deduplication on (device_id, ts DESC) |
Active Chunk RAM Ceiling |
25% of System DDR5 RAM | TimescaleDB timescaledb-tune shared_buffers Sizing Guide |
Columnar Batch Size |
1,000 rows per compressed segment | TimescaleDB Gorilla XOR + Delta-of-Delta Columnar Storage Docs |
-
PILLAR 01 // POSTGRESQL 16 TUPLE GEOMETRY PostgreSQL 16 23-Byte Heap Header + TimescaleDB B-Tree Key Deduplication
We model medium-wide Industrial IoT hypertable rows consisting of a 23-byte HeapTupleHeaderData prefix, 1-byte null bitmap, 8-byte TIMESTAMPTZ, 8-byte integer device/site identifiers, and 4-byte REAL sensor columns padded to 8-byte MAXALIGN boundaries, plus 18 bytes per row across composite (device_id, ts DESC) B-tree indexes using PostgreSQL 16 index deduplication.
-
PILLAR 02 // THE 25% DDR5 RAM CHUNK GUARDRAIL Keeping Active & Boundary-Rollover TimescaleDB Chunk Indexes Inside shared_buffers
Our TimescaleOps chunk evaluator tests candidate intervals from widest to narrowest (7d -> 1d -> 6h -> 1h). An interval qualifies only if it closes within your Uncompressed Hot Days window (allowing add_compression_policy to run cleanly) and its active B-tree index multiplied by a 1.25x–1.85x rollover residency factor remains strictly below 25% of recommended server RAM.
-
PILLAR 03 // GORILLA + DELTA-OF-DELTA + ZSTD CODECS Modeling TimescaleDB Cold Hypertable Compression Factors & Segment Metadata
When TimescaleDB converts a closed chunk into 1,000-row columnar batches segmented by entity key and ordered by ts DESC, row-level B-tree indexes are removed and replaced by lightweight segment metadata (modeled via a 1.28x metadata factor atop compressed heap bytes). Engineers can adjust the compression factor directly from 6x to 15x inside #tso-calc.
-
PILLAR 04 // WAL SORT BUFFER & FILESYSTEM HEADROOM Reserving 12% PostgreSQL WAL Workspace + 20% NVMe/ZFS Free-Space Headroom on TimescaleOps
Operating any PostgreSQL or OpenZFS volume above 85% capacity degrades NVMe garbage collection, fragments ZFS metaslabs, and starves background chunk compression sort buffers. Our Total Retained NVMe GB output reserves 12% WAL/sort space on hot data plus 20% overall free disk headroom.