An engineering story about extracting structured transactions from messy financial PDFs.
If you've ever tried extracting transactions from a PDF bank statement, you probably started with something simple like “just read the text and parse the rows.”
That works for exactly two PDFs — the ones you tested with.
After that, things start breaking.
Many early implementations attempt to parse bank statements using simple regular expressions applied to extracted text.
However this approach fails because PDF text order is not the same as visual order. The PDF format stores characters with coordinates rather than logical table structure.
This means text extraction libraries may return content in a sequence that does not match what a human sees on the page.
03/14 49.90 Amazon Marketplace 950.10
Even though visually the transaction row looks correct, the underlying extraction order can be scrambled.
This makes regex‑based parsing extremely fragile.
A bank statement appears to contain a clean table:
| Date | Description | Debit | Credit | Balance |
|---|---|---|---|---|
| 03/12 | Amazon Marketplace | 49.90 | 950.10 |
But internally the PDF looks closer to this:
| X | Y | Text |
|---|---|---|
| 42 | 510 | 03/12 |
| 120 | 510 | Amazon Marketplace |
| 460 | 510 | 49.90 |
| 540 | 510 | 950.10 |
PDFs don't contain tables — they contain positioned characters.
During development we tested the parser across more than 20 bank statement PDFs from multiple financial institutions.
We observed that naive parsing fails on roughly 60% of statements due to layout variations.
Common real‑world parsing failures include:
Example transaction that breaks naive parsers:
ATM Withdrawal Card Ending 0345
In this case the parser must recognize that both lines belong to the same transaction description rather than two separate rows.
03/14 AMAZON MARKETPLACE ORDER 29384 REFUND 49.90 950.10
A naive parser produced:
| Date | Description | Debit |
|---|---|---|
| 03/14 | AMAZON MARKETPLACE ORDER 29384 | REFUND |
Which is obviously wrong.
To extract text together with its exact coordinates we rely on the Apache PDFBox library. PDFBox allows developers to read PDF documents and intercept text rendering operations.
By extending PDFTextStripper we can access every text fragment
along with its position on the page using the TextPosition class.
This gives us access to X/Y coordinates, width and height for each fragment,
which is essential for rebuilding table layouts from PDF documents.
Example using a custom PDFTextStripper:
public class PositionAwareStripper extends PDFTextStripper {
@Override
protected void writeString(String text, List positions) throws IOException {
for (TextPosition pos : positions) {
float x = pos.getXDirAdj();
float y = pos.getYDirAdj();
float width = pos.getWidthDirAdj();
float height = pos.getHeightDir();
System.out.println(text + " -> (" + x + "," + y + ")");
}
}
}
Once extracted, each fragment can then be normalized into the structure used by the parsing pipeline below:
public record TextElement(
int page,
String text,
float x,
float y,
float width,
float height
) {}
public class LineBuilder {
private static final float Y_TOLERANCE = 3f;
public List> group(List elements){
List> lines = new ArrayList<>();
for(TextElement el : elements){
boolean added = false;
for(List line : lines){
TextElement ref = line.get(0);
if(Math.abs(ref.y() - el.y()) < Y_TOLERANCE){
line.add(el);
added = true;
break;
}
}
if(!added){
List newLine = new ArrayList<>();
newLine.add(el);
lines.add(newLine);
}
}
return lines;
}
}
Date,Description,Debit,Credit,Balance 2024-03-14,Amazon Marketplace,49.90,,950.10
Extracting transactions from PDFs is not a text parsing problem.
It is a document layout reconstruction problem.