Table of contents :

Scanned PDFs Are a Different Problem

100-days-of-chunking-with-llamaparse-and-undefined-018

Table of contents :

Scanned PDFs should not be treated like ordinary PDFs. A digitally generated PDF usually contains a text layer that can be extracted with reasonable fidelity. A scanned PDF is often a collection of page images. Before a retrieval or chunking pipeline can interpret it, the images must be converted into text through optical character recognition (OCR).

That distinction matters for WooCommerce agencies. Product catalogs, supplier price lists, invoices, installation manuals, compliance documents, and signed forms are frequently delivered as scans. If the ingestion pipeline assumes that every PDF already contains clean text, it may produce empty documents, incomplete chunks, or search results that appear to ignore relevant source material.

Why scanned PDFs require a separate ingestion path

For a text-based PDF, extraction can preserve words, paragraphs, and sometimes layout coordinates. A scanned PDF has no reliable words until an OCR system identifies them. The OCR result can contain errors such as:

  • Confusing a zero with the letter O.
  • Dropping decimal points from prices or measurements.
  • Merging adjacent columns into one sentence.
  • Reading page headers and footers as part of the body.
  • Splitting product codes at the wrong character.
  • Missing text in low-contrast, skewed, or handwritten regions.

These errors are especially costly in commerce workflows. A mistaken product code can point to the wrong variation, while a missing decimal point can change a price or specification. Chunking does not repair bad OCR. It can only organize the text that the OCR stage produces.

Inspect the document before choosing a chunking strategy

Start by classifying the PDF rather than assuming its format from the file extension. A PDF may be entirely scanned, entirely digital, or mixed. A mixed document can contain scanned forms followed by digitally generated reports.

Useful checks include:

  • Determine whether pages contain selectable text.
  • Measure extracted text per page. A page with almost no text may still contain a full-page image.
  • Check whether the document contains embedded images with dimensions close to the page size.
  • Compare the number of pages with the number of extracted text blocks.
  • Render representative pages to inspect orientation, contrast, tables, stamps, and handwritten annotations.

A simple diagnostic can prevent an expensive failure later. For example, if a 40-page supplier catalog produces only 120 extracted characters, it should be routed to OCR instead of being passed directly to a text splitter.

OCR is a document reconstruction step

OCR should be treated as more than a character-recognition operation. A useful OCR pipeline attempts to recover document structure, including page boundaries, reading order, headings, tables, lists, and sometimes bounding boxes or confidence values.

For example, these two OCR outputs are not equivalent:

SKU 1842    Cotton Shirt    Blue    £24.00    In stock

and:

SKU 1842 Cotton Shirt Blue 24.00 In stock

The second version may be adequate for a broad keyword search, but it loses currency formatting and column relationships. If the question is “Which blue cotton shirts are in stock for £24.00?”, the first representation gives a retrieval system more useful evidence.

When using LlamaParse or another document parsing service, preserve the structured output when available. Page numbers, detected headings, table boundaries, and source locations are valuable metadata for retrieval, citations, debugging, and human review.

Preprocessing improves OCR quality

Image preprocessing can substantially affect the resulting chunks. Depending on the source material, a pipeline may need to:

  • Rotate pages to the correct orientation.
  • Deskew images whose text baseline is tilted.
  • Increase resolution for small type.
  • Convert color pages to grayscale when color is not semantically important.
  • Increase contrast for faint scans.
  • Remove borders, speckles, and scanner noise.
  • Detect and preserve regions containing tables, stamps, or signatures.

Do not apply aggressive cleanup indiscriminately. Color can distinguish product variants, warning labels, or annotations. Removing it may improve recognition of ordinary text while destroying information required by the business process. Keep the original PDF and, where possible, the page image so that uncertain results can be audited.

Chunk by document structure, not only by character count

After OCR, use structural boundaries before applying a size limit. A product entry, warranty clause, or invoice line-item group is usually a better semantic unit than an arbitrary sequence of characters.

A practical hierarchy is:

  1. Document.
  2. Page.
  3. Detected section or heading.
  4. Table, list, paragraph, or record.
  5. Size-based subchunks only when a unit is too large for the target model.

For a scanned product catalog, a chunk might contain the product name, SKU, dimensions, materials, price, and availability together. Splitting the SKU into one chunk and the price into another makes retrieval less reliable because the answer depends on their relationship.

For a long scanned contract, preserve clause boundaries where possible. A chunk containing part of a clause without its exception or definition can produce a misleading answer even when every individual OCR token is correct.

Use metadata to compensate for imperfect text

Every OCR-derived chunk should carry enough metadata to locate and evaluate its source. Useful fields include:

  • source_file
  • page_number
  • section_title
  • document_type
  • ocr_required
  • ocr_confidence, when supplied by the OCR system
  • bounding_box, when available
  • parser_version

For example, a WooCommerce support assistant can return an answer together with “Supplier catalog, page 12” rather than presenting an unsupported statement. Page-level metadata also lets an agency create an internal review tool that opens the original scan beside the extracted text.

Handle tables as records when possible

Tables are one of the most common failure points in scanned documents. OCR may read cells in visual order, row order, or column order, and the result can vary by page layout. A table should not automatically be flattened into a paragraph.

For retrieval, represent each row with its column labels when the table structure is dependable:

Product: Cotton Shirt; SKU: 1842; Colour: Blue; Price: £24.00; Availability: In stock

Repeating the column names within each row increases chunk self-containment. If the parser cannot establish the table structure confidently, retain the raw extracted text and mark the table for review rather than silently inventing relationships between values.

Measure OCR and retrieval separately

An ingestion test should evaluate more than whether a parser returns non-empty text. Create a small labeled set of scanned documents and test:

  • Character accuracy for important fields such as SKUs, prices, dates, and dimensions.
  • Section and page preservation.
  • Table row and column integrity.
  • Retrieval recall for realistic agency questions.
  • Answer faithfulness against the original page image.

Keep OCR quality and retrieval quality as separate metrics. A retrieval failure may be caused by bad recognition, poor structural segmentation, an unsuitable embedding model, or an overly restrictive search filter. Without separate measurements, these problems are easy to confuse.

Build a fallback for high-risk fields

Some fields deserve stricter validation than ordinary prose. In a WooCommerce workflow, validate product IDs, variation codes, currency amounts, dates, and quantities against expected patterns or an authoritative catalog. A parser can flag a chunk for review when a product code contains an unexpected character or when a price has an unusual number of decimal places.

For high-risk automation, use OCR to locate candidate information, then require a second check before updating products, prices, inventory, or orders. A human review screen showing the extracted value and the original image region is often more reliable than trying to make the language model infer whether a questionable character is correct.

Recommended pipeline

  1. Classify each PDF as digital, scanned, or mixed.
  2. Extract the native text layer where it is trustworthy.
  3. Send image-only or low-text pages through OCR and document parsing.
  4. Preserve page, section, table, and confidence metadata.
  5. Normalize obvious formatting issues without changing uncertain values.
  6. Chunk by headings, records, clauses, and table rows before applying size limits.
  7. Attach source references to every chunk.
  8. Index the chunks for retrieval.
  9. Validate high-risk fields and sample answers against the original scans.
  10. Keep the original document and parser configuration for reproducibility.

The key design decision is to make scanned-document handling explicit. LlamaParse can be part of the parsing and structure-recovery stage, while the surrounding pipeline remains responsible for classification, validation, metadata, chunking, indexing, and review. Once scanned PDFs are treated as image-to-structure conversion problems rather than ordinary text files, agencies can make their document systems more dependable without hiding uncertainty in the final answers.

Trending posts
You might also like