阅读中文版本

JSON Lines Statistics Script

A Python script that reads a JSON Lines file, counts records, reports the keys present, and computes basic statistics for numeric fields.

python --version

python jsonl_stats.py events.jsonl

python jsonl_stats.py events.jsonl

Prints the record count, the set of keys seen, and min, max, and mean for each numeric field.

What it does

Reads a JSON Lines file one record per line and summarises it, which is the fastest way to understand a stream of events before writing a full parser.

import json
import statistics
import sys

def read_records(path):
    records = []
    bad = 0
    with open(path, encoding="utf-8") as handle:
        for line in handle:
            line = line.strip()
            if not line:
                continue
            try:
                records.append(json.loads(line))
            except json.JSONDecodeError:
                bad += 1
    return records, bad

def summarise(records):
    keys = set()
    numeric = {}
    for record in records:
        keys.update(record.keys())
        for key, value in record.items():
            if isinstance(value, (int, float)):
                numeric.setdefault(key, []).append(value)
    return keys, numeric

def main():
    records, bad = read_records(sys.argv[1])
    keys, numeric = summarise(records)
    print(f"records: {len(records)} (bad lines: {bad})")
    print("keys: " + ", ".join(sorted(keys)))
    for key, values in sorted(numeric.items()):
        print(f"{key}: min={min(values)} max={max(values)} mean={statistics.mean(values):.2f}")

if __name__ == "__main__":
    main()

Numeric fields are detected automatically and summarised, while malformed lines are counted rather than crashing the run.

Run and verify

printf '{"latency":120}\n{"latency":80}\n' > events.jsonl
python jsonl_stats.py events.jsonl

Create the small sample file then run; malformed lines are counted instead of aborting.

Expected Output

For two records with a latency field the script should print the count, keys, and latency statistics. The output is:

records: 2 (bad lines: 0)
keys: latency
latency: min=80 max=120 mean=100.00

The key set is the union across records, each numeric field gets min, max, and mean, and a malformed line increments the bad-line count.

Limitations

The whole file is read into memory, so very large JSONL files should be summarised in a streaming pass instead.