LLM Invoice Extraction

Demo · All data is synthetic

I built this demo around a job every finance ops team has: pulling the same few fields out of invoice, remittance and payment emails that arrive in every format. I made up 19 emails in USD, GBP, EUR, PKR and CAD, with UK and US date styles, German and French number formats, a forwarded chain, an invoice whose totals don't add up, and two emails that aren't finance documents at all. One interface sits in front of OpenAI, Claude and a local model, picked with a setting. I ran it end to end on a small open model (Qwen 2.5 3B) through Ollama on a CPU, using the same OpenAI-style request with only the address changed. I didn't have OpenAI or Anthropic keys, so those two adapters are only tested against a mock server, not the real APIs. From working on card programs, starting as a software engineer at i2c, there are two things I wouldn't compromise on: card numbers never leave the system, and the model doesn't get the final word. So card numbers are masked before the call, and plain code checks every answer. Does subtotal plus tax equal the total, is the due date after the issue date, is the currency a real code, does the total actually appear in the email? Anything that fails goes to a person instead of being posted. Against my hand-written answers the model got 171 of 182 fields right (94%), and 11 of the 19 emails fully right. On the first pass, 6 emails with a wrong field got through without a flag. After I added checks for ambiguous dates, missing tax and payments with no invoice reference, that dropped to 1: a remittance where the model filled in a subtotal. I wrote those checks after seeing the errors on these same emails, so that number is optimistic. It's a demo on synthetic data, not a production system.

How it works

Email text → mask card numbers → prompt + JSON schema → provider adapter (OpenAI / Claude / Ollama) → schema validation (one repair retry) → normalise → deterministic checks → JSON + needs_review + reasons → usage log

Behaviour

One interface, three adapters

OpenAI, Claude and Ollama, picked with a setting. Only the local model (qwen2.5:3b on a CPU) ran end to end. The OpenAI and Claude adapters are tested against a mock server, not the real APIs.

Card masking

Any 13-19 digit number that passes the Luhn check is replaced with a masked version before the prompt is built, and the record is flagged.

Deterministic checks

Subtotal plus tax must equal the total, the due date can't be before the issue date, the currency must be a real code, and invoices need a total, due date and tax amount.

Grounding

The total, subtotal, tax and invoice number must appear in the email itself, so made-up numbers get caught.

Review routing

Any failed check, an ambiguous date, a payment with no invoice reference or a non-finance email sends the record to a person, with reasons.

Result

171 of 182 fields right (94%), 11 of 19 emails fully right. One wrong field still gets through unflagged (a subtotal on a remittance), so the check script reports FAIL. I left it that way.

Demo

A short walkthrough of the demo, made from the screenshots below, not a screen recording.

Screenshots

Extracted fields for all 19 emails, with review reasons

The check script output, including the one unflagged error

A card number masked before the model sees the email

Adapter tests against a mock server, and the usage log

Stack

Python 3 (standard library plus jsonschema), Ollama with qwen2.5:3b on a CPU, and unittest with a local mock of the OpenAI and Anthropic APIs.

This is a demo on synthetic data, not a production system. The 19 emails and their expected answers were written for the demo, and no paid API was called.