05 · internal
Telemetry Ingest Pipeline
A database outage loses nothing. Proven rather than promised.
10,000
radios capacity-planned
80
tests, zero-loss proven
3
duplicate-prevention layers
10x
the transmit-power finding
/ interactive · take the database down, lose nothing
try it, everything responds
receiver
accepts every reading
holding queue
depth 120
database
storing
readings safely stored 184,320
readings lost 0
Take the database down and watch. Readings keep arriving and pile up in the holding queue instead of being lost. Bring it back and the backlog drains. The counter that matters never moves: nothing lost, and nothing counted twice, because a reading is only ticked off once it is genuinely saved. In the real system this is proven by tests, and the automatic watchdog is deliberately built so it can never restart the one service whose whole job is to hold on through exactly this moment.
A note on this demo. Every figure, name and data point here is invented for illustration. No client data, real economics or production configuration appears anywhere on this site. What is real is the machinery: the calculations, the rules and the logic running in your browser are the same ones the production system uses, so the demo behaves exactly as the real thing behaves.
What it is
/ in plain terms
Thousands of radios phone home with measurements every three minutes. This is the system that catches all of it, built so that even if the database goes down completely, not a single measurement is lost, and nothing is ever counted twice. Both properties are proven by tests, not just claimed.
The next generation of fleet telemetry. Hundreds of radios push measurements every 180 seconds into a receiving service, buffer through an in-memory queue and land in the analytics database in batches, capacity-planned for 10,000 devices. Duplicate protection is enforced at three independent layers, and the health checks are deliberately split so that the automated watchdog can never restart the one service whose entire job is to keep buffering through an outage.
Sole engineer. Contract, receiver, worker, schema, tests, capacity plan.
The load-bearing details
- 01A versioned, locked data contract between radios and server, designed so that older radios keep working as the format evolves, with a precise split between malformed requests and invalid data.
- 02Exactly-once delivery enforced at three layers: duplicates collapsed within each batch, a memory of recently seen readings marked only after the data is safely stored, and a database engine that deduplicates as a final backstop.
- 0380 tests mapped to a written coverage brief, including proofs that the system buffers through a full database outage without losing data.
- 04The motivating find: radios transmitting at roughly a tenth of their configured power, discovered from the telemetry itself. It redirected the company's firmware strategy, and the measurement tooling to quantify the fix came with it.