FHIR could have picked a Bundle-shaped export. It didn't. The bulk-data specification picked NDJSON — newline-delimited JSON — as the wire format for $export. That choice was deliberate, and understanding it is what makes bulk-data engineering tractable. Every consequence of using NDJSON traces back to a small handful of design goals. The site's NDJSON export peeker is the tool for inspecting the format. For the wider FHIR framing, the EHR-payer integration hub has more.
The Format In Detail
- One JSON value per line
- Values are separated by
\n(LF), not by commas - The file has no outer wrapper — no
[, no], no root object - Each line is a complete, valid JSON value
- Encoding is UTF-8
A file with one thousand Patient resources has one thousand lines, each a JSON object.
Why Not A Single Bundle?
A single-Bundle export would produce one gigantic JSON document with an entry array containing every resource. Reading it requires either loading the whole thing into memory or writing a streaming parser that finds entry boundaries in a nested JSON document.
Streaming parsers for nested JSON exist but are complex. NDJSON is trivial to stream: read to newline, parse the line, repeat. That is the primary reason bulk data uses it.
For the read pattern, reading an NDJSON export without loading the whole file is the entry.
The Per-Line Property
Every line is independent. That has consequences:
- Corrupt lines do not corrupt the rest of the file
- Parallel processing works — split by lines and process in parallel
- Append is trivial — just add a new line at the end
- Extraction is trivial — grep for a pattern, get matching lines
Each is worth a lot in a production pipeline. Each is impossible or hard with a Bundle wrapper.
The Per-File Property
Bulk export produces one file per resource type. Patient.ndjson, Observation.ndjson, Condition.ndjson.
- Processing pipelines can parallelize across files
- Type-specific tools can consume only what they need
- Aggregation is per-type
For the mixed-file case, handling mixed-resource NDJSON in a $export covers when resources of different types share a file.
What NDJSON Is Not
- Not compressed by default (though gz is common)
- Not indexed
- Not columnar (unlike Parquet)
- Not queryable in place
Those are trade-offs. NDJSON optimizes for streaming write and streaming read. Anything analytical happens after ingestion into a more structured store.
For the ingestion story, streaming an NDJSON file into a database is the entry.
The Encoding Rule
UTF-8. Full stop. NDJSON files with UTF-16 BOMs or other encodings produce parse errors in most tools.
Producers should emit UTF-8 without a BOM. Consumers should reject anything else.
Line Ending Conventions
\n (LF) is the standard. Some Windows-generated files emit \r\n (CRLF). Most tools handle both; a few do not.
Producers should emit LF. Consumers should be tolerant of CRLF.
Compression Considerations
NDJSON compresses well. Each line has similar structure — repeating keys, similar types — which gzip and zstd exploit. Typical compression ratios are 5-10× depending on content.
For the deep dive, compression choices for NDJSON delivery is the entry.
Media Type
application/fhir+ndjson is the FHIR-specific media type. application/x-ndjson is the generic one. Servers should emit the FHIR-specific type on $export responses.
Consumers should accept either and not fail on the generic form.
The Short Version
NDJSON is one JSON value per line, no wrapper, UTF-8, LF endings. Streaming-friendly. Per-line independence enables parallel processing and simple append. Compression works well. It is a deliberate choice for bulk export because the alternatives are worse for streaming workloads.
Sources
- HL7 canonical Bulk Data Access IG - HL7 canonical Bulk Data Access IG