Table of contents :

Preserving Page Boundaries

100-days-of-chunking-with-llamaparse-and-undefined-027

Table of contents :

Why Page Boundaries Matter in Document Chunking

Page boundaries are easy to lose when a parsed document is converted into one continuous string. For WooCommerce agencies, that loss can create practical retrieval problems: a product specification may become separated from its table heading, an invoice total may be associated with the wrong page, or a shipping policy may be retrieved without the exception listed on the following page.

When building a knowledge system with LlamaParse and Undefined, treat the page as meaningful document structure rather than incidental formatting. Preserve the page number, source location, and page-level content before applying any secondary chunking strategy.

What It Means to Preserve a Page Boundary

Preserving boundaries does not necessarily mean that every chunk must contain exactly one page. It means that the system retains enough information to identify where each page begins and ends, and that later chunking does not silently merge unrelated pages.

A useful intermediate representation contains one record per page:

{
  "document_id": "shipping-policy-2025",
  "page_number": 4,
  "text": "...text extracted from page four...",
  "source_file": "shipping-policy.pdf",
  "section": "International shipping"
}

The exact object shape depends on your pipeline, but the principle is consistent: page identity should be metadata, not information encoded only in a separator inside the text.

Preserve Boundaries Before Chunking

A common mistake is to concatenate all parsed pages and then split the result by character count or token count. Once concatenated, the chunker may place the final paragraph of one page beside the first paragraph of another without recording the transition.

A safer sequence is:

  1. Parse the source document.
  2. Extract page-level text and page metadata.
  3. Normalize whitespace without removing page identity.
  4. Chunk each page or controlled group of pages.
  5. Attach the original page number to every resulting chunk.
  6. Index the chunks with document and source metadata.

This approach allows a 1,000-token chunk to be traced back to page 4, even if the page required multiple chunks. It also allows a retrieval result to display the source page or generate a link to the relevant location in the original PDF.

Use Page Boundaries as Hard Limits for Some Content

For ordinary prose, allowing a chunk to cross a page boundary can be acceptable when the pages clearly form one continuous section. For structured or operational content, however, a page boundary should often be a hard limit.

Consider these WooCommerce examples:

  • A product catalog page containing a product name, dimensions, and shipping class.
  • An invoice page containing line items and a subtotal.
  • A returns policy page containing an eligibility rule and its exceptions.
  • A fulfillment guide containing a warehouse code table.
  • A legal agreement whose footer and page number are important for auditability.

For these documents, page-local retrieval reduces the chance that content from adjacent pages will be presented as if it belongs to the same table, product, or policy clause.

Handle Tables and Repeated Headers Carefully

PDF parsers may return tables as structured data, Markdown-like text, HTML, or plain text. A table may also continue across multiple pages. Preserving page boundaries alone does not guarantee that the table will be interpreted correctly.

Record whether a page contains the beginning, continuation, or end of a table. If a table continues on the next page, retain the table identity and repeat the column headers in the normalized representation when necessary. Do not assume that a repeated visual header is a new data section.

For example, a normalized record might include:

{
  "page_number": 7,
  "block_type": "table",
  "table_id": "shipping-rates-1",
  "continuation": "middle",
  "text": "Zone | Method | Rate ..."
}

When indexing this content, keep both the page number and table identifier. A WooCommerce shipping integration can then use the retrieved table rows while still showing the agency operator exactly where the values came from.

Page-Aware Chunking with Overlap

Chunk overlap is useful when a paragraph or section is split within a page. It should not be used to erase page boundaries. Apply overlap inside a page first, and only permit cross-page overlap when there is a deliberate reason to represent a continuous section.

For example, a page-aware strategy can follow these rules:

  • Keep each page as the initial unit.
  • Split oversized pages into token-based or semantic chunks.
  • Use a small overlap between chunks from the same page.
  • Do not combine the last chunk of one page with the first chunk of the next unless both belong to the same verified section.
  • Store page_start and page_end for every chunk.

This creates predictable retrieval behavior. A chunk spanning pages 8 and 9 should be distinguishable from a chunk created entirely from page 8.

Metadata for WooCommerce Knowledge Systems

Page metadata becomes especially valuable when the same content is used by support agents, administrators, and automation. A practical chunk record may include:

  • document_id for the canonical source document.
  • source_file for the original filename or URL.
  • page_start and page_end for source location.
  • section_title when a heading can be identified.
  • block_type, such as prose, table, list, or footer.
  • product_id, SKU, order type, or other business identifier when available.
  • parser and parser_version for troubleshooting.

For a customer-service assistant answering a question about a product’s delivery restrictions, this metadata supports a response that cites the exact catalog page. For an internal order workflow, it also makes it easier to investigate whether an answer came from the current policy version.

Do Not Treat Page Numbers as Trusted Business Data

Page numbers can be useful citations, but they are not always stable identifiers. A PDF may contain printed page labels such as “A-4,” while the parser reports the fourth physical page. Cover pages, appendices, scanned inserts, and rotated pages can also change the relationship between displayed labels and physical positions.

Store both values when available:

  • The physical page index reported by the parser.
  • The printed page label visible in the document.

Use the physical index for deterministic processing and the printed label for human-facing citations. If a document is replaced, create a new document version rather than silently reusing old page references.

Testing Boundary Preservation

Boundary handling should be tested with documents that expose failure modes, not only with short text PDFs. Include a multi-page product catalog, a scanned invoice, a table that continues across pages, a document with repeated headers, and a policy where an exception starts on a new page.

Useful assertions include:

  • Every chunk has a document identifier and page location.
  • No chunk contains text from two pages when hard page boundaries are enabled.
  • Page numbers increase in the expected order.
  • Tables retain their headers or continuation metadata.
  • Retrieval filters can restrict results to a document version or page range.
  • The displayed citation opens the correct source page.

Also inspect the parsed output manually. A technically valid page number does not help if a footer, column, or table row was assigned to the wrong page during extraction.

Operational Pattern

In a production agency workflow, keep the parsed page representation as an intermediate artifact. Store it alongside the final chunks and embeddings, preferably with a parser version and content hash. When chunking rules change, regenerate chunks from the page representation instead of reparsing every source document unnecessarily.

This separation makes it possible to compare chunking strategies, diagnose retrieval errors, and re-index a WooCommerce client’s catalog or policy library without losing the original page context. Page boundaries then serve as a stable control point between document parsing, chunk creation, retrieval, and source citation.

Trending posts
You might also like