Table of contents :

Parsing First, Chunking Second

100-days-of-chunking-with-llamaparse-and-undefined-033

Table of contents :

For WooCommerce search, support, and merchandising workflows, chunking should not be the first transformation applied to a document. The reliable order is: parse the source into meaningful structure, normalize that structure, and only then create retrieval chunks.

Why parsing must come first

A PDF, product export, invoice, or support document is usually organized for human reading rather than machine retrieval. A page may contain a heading, a two-column layout, a table, a footer, and a product note that visually belongs to the heading above it. Splitting the raw file by character count can separate those relationships before the chunker has a chance to use them.

Parsing converts the source into text and structural signals such as headings, paragraphs, lists, tables, page boundaries, and metadata. For a WooCommerce agency, those signals are important when indexing:

  • product specifications and compatibility notes;
  • shipping, refund, and warranty policies;
  • merchant onboarding guides;
  • customer-support playbooks;
  • tax or fulfilment documentation; and
  • catalogues containing tables of attributes and variants.

Use LlamaParse as the document boundary

LlamaParse can be used to turn difficult source files into a more consistent representation before a separate chunking step. In a production pipeline, request a format that preserves useful structure, commonly Markdown or a structured document representation, and retain source metadata alongside the parsed content.

from llama_parse import LlamaParse

parser = LlamaParse(
    api_key="LLAMA_CLOUD_API_KEY",
    result_type="markdown"
)

documents = parser.load_data("woocommerce-refund-policy.pdf")

for document in documents:
    print(document.metadata)
    print(document.text[:1000])

The important design decision is not the exact output format. It is the boundary between parsing and chunking. Treat the parser output as an intermediate document, not as the final set of embeddings.

Normalize before passing content to Undefined

Parser output still needs validation. Remove repeated headers and footers where appropriate, standardize whitespace, preserve heading hierarchy, and decide how tables should be represented. Do not blindly flatten every element into one string. A table containing variation names, SKU values, and stock rules may need a row-oriented representation so that the relationship between each value survives retrieval.

A useful intermediate record can look like this:

{
  "source": "woocommerce-refund-policy.pdf",
  "page": 4,
  "section": "Digital products",
  "content": "Digital products are eligible for a refund within 14 days if...",
  "content_type": "paragraph"
}

At this stage, the content is parsed and normalized but not yet chunked. This makes the pipeline easier to inspect and test. If search quality falls, the agency can determine whether the problem came from file parsing, cleanup, or chunk boundaries.

Chunk only after structure is available

Undefined, or any downstream chunking component, should receive normalized content with enough context to make informed boundaries. Prefer splitting at structural boundaries such as a section, subsection, paragraph group, or table row before applying a token or character limit.

For example, a refund policy might be processed conceptually like this:

parsed_sections = parse_with_llamaparse("woocommerce-refund-policy.pdf")
normalized_sections = normalize_sections(parsed_sections)

chunks = undefined_chunker.chunk(
    normalized_sections,
    max_tokens=450,
    overlap_tokens=60,
    preserve_metadata=True,
    split_on=["heading", "paragraph", "list_item"]
)

The exact Undefined API depends on the version and integration used by the project. The invariant is more important than the method name: the chunker must operate on parsed units, preserve metadata, and avoid cutting through a heading, list item, or table row unless a size limit makes that unavoidable.

WooCommerce example: product documentation

Consider a product manual with this structure:

  1. Installation
  2. Supported WooCommerce versions
  3. Subscription renewal rules
  4. Known compatibility issues

A raw character splitter may place the heading “Subscription renewal rules” at the end of one chunk and the actual renewal conditions at the beginning of the next. A parsed-first pipeline can keep the heading attached to its section and add metadata such as the product name, document version, and page number.

A resulting chunk might contain:

Product: Acme Subscriptions Extension
Section: Subscription renewal rules
Source page: 12

Renewals are attempted three days before the current term ends. If the payment fails, the order remains on hold and a second attempt is made after 24 hours.

This gives a retrieval system enough context to answer a support question without requiring the model to reconstruct the section title from another chunk.

Tables require special handling

WooCommerce documents often contain tables for shipping zones, tax classes, API fields, or product compatibility. Parsing first lets you decide whether a table should remain a table, become a series of labeled rows, or be summarized alongside the original values.

For retrieval, a row-oriented form is often clearer than a visually formatted table:

Shipping zone: United Kingdom
Method: Standard delivery
Minimum order: £50
Fee: £0
Condition: Excludes oversized products

Keep the original page and table identifiers in metadata. This allows an agency to display a citation or send an operator back to the source document when a merchant disputes an answer.

Validation checks before indexing

Before embeddings are generated, add automated checks for common parsing failures:

  • confirm that every non-empty chunk has a source identifier;
  • flag chunks containing repeated headers or footer text above a chosen threshold;
  • check that headings are not isolated from all following content;
  • verify that table rows retain their labels and values;
  • measure chunk size after normalization rather than before it; and
  • sample chunks from PDFs with columns, scans, and page breaks.

For scanned invoices or manuals, parsing may require OCR. OCR output should be treated as lower-confidence text and should not be mixed silently with clean digital text. Store the parser mode and source page in metadata so later evaluation can distinguish extraction errors from chunking errors.

Operational pattern for agency projects

Use separate pipeline stages with persisted intermediate results:

  1. Store the original file and a content hash.
  2. Parse the file with LlamaParse.
  3. Normalize and validate the parsed representation.
  4. Chunk the normalized units with Undefined.
  5. Attach tenant, store, product, document version, and source-location metadata.
  6. Generate embeddings and write the chunks to the search index.

Persisting the parsed stage is especially useful when a client changes chunk size or overlap. The agency can re-chunk the same normalized document without repeatedly paying for parsing or introducing a different extraction result.

Measure parsing and chunking separately

Evaluate the stages with different questions. Parsing quality asks whether the required text and relationships were extracted correctly. Chunking quality asks whether the extracted content was grouped into retrievable units with sufficient context and limited noise.

For a WooCommerce support dataset, create test questions such as “Which shipping method applies to oversized products?” or “What happens after the first subscription payment retry?” Verify that the expected source section is present in the parsed output before tuning chunk size. If the answer is missing or corrupted there, changing overlap will not fix the problem.

Parsing first and chunking second creates a controllable boundary between document understanding and retrieval preparation. That separation makes WooCommerce knowledge bases easier to debug, re-index, and adapt across different client document types.

Trending posts
You might also like