An NDJSON batch that ingests silently for four hours and then dies at row 3.7 million is a bad on-call experience. Monitoring is what turns that opaque failure into a graph, an alert, and an actionable metric. Every ingestion pipeline...

An uncompressed NDJSON export is a lot of JSON. Bulk exports for a mid-size health plan run in the tens of gigabytes uncompressed. Compression reduces both delivery bandwidth and storage, and the choice of compression algorithm matters...

FHIR $export usually delivers one NDJSON file per resource type — Patient.ndjson, Observation.ndjson, and so on. Sometimes the spec-compliant output mixes resource types in a single file. Handling that case is a specific dispatch pattern...

Corruption in an NDJSON stream is rarely announced. The file downloads cleanly, the checksum matches, and the pipeline chugs happily until row 4.2 million turns out to be malformed and everything past it silently misinterprets. Building...

A fresh $export lands in a bucket. Before running it through the ingestion pipeline, a five-second sanity check saves the hour of debugging you would otherwise do when the whole pipeline runs on a bad export. That check is a small NDJSON...

Streaming an NDJSON file into a database is the ingestion path for every real bulk-data pipeline. The naïve "load the file, insert row by row" approach hits memory ceilings on large exports. The naïve "insert every row in its own...

FHIR could have picked a Bundle-shaped export. It didn't. The bulk-data specification picked NDJSON — newline-delimited JSON — as the wire format for $export. That choice was deliberate, and understanding it is what makes bulk-data...

The first NDJSON file from a FHIR $export lands on a developer's laptop and the reflex is to open it in a text editor. Ten seconds later the editor locks up because the file is 800 MB. Reading NDJSON without loading the whole file is the...