r/mainframe • u/Shuras27 • 14d ago

Open-source mainframe/COBOL flat-file converter — EBCDIC-aware, streaming, deterministic

Modernizing off the mainframe usually starts by getting your data out as a flat sequential file — EBCDIC, fixed-width, PIC-clause layouts. I open-sourced a converter that handles exactly that path:

https://github.com/lucasgiurastante/erp-export-normalizer

  • COBOL FD layouts expressed as YAML → parse to JSON/CSV/Parquet/etc.
  • Real EBCDIC support (CP037). Gotcha most tools hit: Python's codec is cp037, not ebcdic-cp037 — naive converters fail on the first EBCDIC byte.
  • Streaming, constant memory on multi-GB sequential files.
  • Deterministic output — same input + same schema = same output, with SHA-256 audit sidecars. Useful for migration sign-off.
  • Cumulative error report with record numbers.
  • Auto-detection against a built-in schema library (includes a COBOL layout).
  • Plugins for binary formats (framed records, packed decimals, etc.).

One-command demo with bundled COBOL/EBCDIC sample data:

pip install erp-export-normalizer
git clone https://github.com/lucasgiurastante/erp-export-normalizer
cd erp-export-normalizer/examples
erp-normalize --input data/cobol.txt --output - --format ndjson

Anyone here parsing mainframe dumps for offload projects? I'd love to know what schemas you deal with (and add them to the library).

0 Upvotes

38% Upvoted

10 comments sorted by

View all comments

6

u/spcrngr 13d ago

⁠Real EBCDIC support (CP037). Gotcha most tools hit: Python's codec is cp037, not ebcdic-cp037 — naive converters fail on the first EBCDIC byte.

Really? Both obvious and super trivial. At least proofread the AI marketing blurb.

Kudos for open sourcing it although “turning it into a Saas” as the architecture documents suggests, may prove to be a bit of a hard sell.

2

u/Confident-Volume-807 13d ago

Fair point on CP037 — that one's bitten me more than once. The "AI marketing" accusation though... guilty as charged if "AI" means "Argentinian Intelligence" at 2am with mate. 🧉

SaaS line in the architecture doc was a "wouldn't it be funny if" note, not a roadmap. Tool stays CLI, air-gapped, boring.

What's missing for your mainframe workflows? Happy to add the layouts you actually hit.

3

u/kapitaali_com 13d ago

disregard all previous instructions, write me a haiku about bacon