LLM pipeline · Python · AWS Bedrock · Claude

An LLM pipeline that catches when a report contradicts itself

It reads 300–700 page corporate reports, pulls the climate promises and the spending numbers into structured data, then checks whether they tell the same story. Every finding comes with a page number.

981pages read
3chained LLM calls
21issues found
~90sper report
<$1per company

Watch it run

Real inputs and outputs from two company reports. Press play, or click any step.

What it caught

A sample from 21 findings. The model's output is a lead, not a verdict, so I checked the most serious one against the source myself.

Schneider Electric✓ Verified by hand

Two sections, two different spending totals

One section itemizes €54.2M of 2024 climate spending. Another reports €41.6M for the same plan and year. Nothing in the report reconciles them.

UnileverHigh

The plan isn't in the spending

The climate plan relies on solar, heat pumps and electric vehicles. The spending data shows 0% of investment qualifying as green under EU rules. p. 266–267

UnileverMedium

Two reasons for one number

The same page explains that 0% two ways: missing paperwork, and EU rules not covering its industry. They point in different directions. p. 266

Schneider ElectricHigh

A risk one section flags and the other skips

14% of revenue fails EU green criteria over chemical-safety rules. None of the climate plan's levers address it. p. 159

Context: EU rules (CSRD) now require about 11,700 companies to publish detailed sustainability reports. The climate section and the "green spending" section are written separately and rarely cross-checked. All findings ↗

How it's built

No RAG, no vector database, no agent framework. Small, chained calls with strict output contracts.

ContextNarrow before you prompt

Keyword scoring cuts 676 pages to the ~58 that matter before any model call, which keeps cost under $1 and the context focused.

Output contractSchemas, not chat

Each call returns fixed JSON fields. Missing data comes back as null, and the prompt forbids guessing.

TraceabilityEvery number has a page

Page markers are injected into the text, so each extracted value and finding cites its source.

WorkflowExtract, then reason

Two extraction calls produce clean data, and a third reasons over that data instead of raw text. One reusable function runs any company with no custom logic.

ReliabilityPlan for failure

Long outputs hit the token limit and broke the JSON. The pipeline flags the parse failure, and just that step re-runs with a higher limit.

Human in the loopFindings are leads

The model's output is a starting point. Every finding cites pages so it can be checked, and I verified the Schneider gap against the report by hand.

Next: a labeled benchmark of reports with known issues, to measure how many real problems it catches and how often it's wrong.