Extracting parcel tables from scanned municipal PDFs with pdfplumber

A planning department publishes its quarterly zoning schedule as a PDF: page 3 is a crisp ruled grid of parcel IDs, districts, and lot areas; page 4 is the same data with no cell borders at all; page 7 is a photocopy someone scanned at an angle. You run page.extract_table() across all of them and get one clean table, one table where three columns have collapsed into a single merged blob, and one page that returns None as if it were blank. The data is uniform to a human reader, but pdfplumber’s default heuristics see three different documents. This guide answers one narrow question: how do you pull reliable parcel records out of a mixed bag of ruled, borderless, and scanned municipal tables with pdfplumber inside the broader automated zoning change and municipal GIS tracking workflow? The fix is per-page table strategy selection, explicit line hints, a scan detector that routes raster pages to OCR, and a normalization pass that turns raw cell grids into validated parcel records.

Diagnosis: why the same call returns three different results jump to heading

extract_table() runs a geometry engine that infers cell boundaries from the vector primitives on the page. Its default table_settings use "lines" for both the vertical and horizontal strategy, meaning it looks for actual drawn rulings. That assumption is what fractures across a municipal document set, and the breakage takes three recognizable forms.

1. Borderless tables return merged or dropped columns. When a zoning schedule is laid out with whitespace instead of rules — common in documents exported from Word or a report writer — there are no lines for the "lines" strategy to find. pdfplumber falls back to treating large horizontal gaps as column edges, and two numeric columns that sit close together (say Lot Area and FAR) collapse into one cell. You get a table object, so a naive if table: check passes, but the column count is wrong and every downstream field is shifted.
2. Partial rulings split rows. Some templates draw only the outer border and a header underline. pdfplumber detects those few lines and builds a table with two or three giant cells instead of the intended grid, silently discarding the interior structure.
3. Scanned pages return None. A photocopied or faxed schedule has no text layer and no vector lines — it is a single raster image. extract_table() and extract_text() both return empty because there is nothing selectable to parse. No table_settings value can recover text that was never encoded; the page needs OCR.

Reproduce the failure deterministically and record which class each page falls into before touching any settings:

import pdfplumber

with pdfplumber.open("zoning_schedule.pdf") as pdf:
    for i, page in enumerate(pdf.pages):
        text = page.extract_text() or ""
        table = page.extract_table()
        n_cols = len(table[0]) if table else 0
        print(
            f"page={i} chars={len(text)} "
            f"lines={len(page.lines)} rects={len(page.rects)} "
            f"table_cols={n_cols} images={len(page.images)}"
        )

A page with chars=0 and a large images count is a scan bound for OCR. A page with many chars but lines=0 is borderless and needs a text-based strategy. A page with healthy lines but a wrong table_cols needs explicit vertical hints. That triage drives everything that follows.

Step-by-step implementation jump to heading

Each step handles one page class. The orchestrator in the final step dispatches on the profile.

Step 1 — Tune table_settings for ruled grids jump to heading

When the page has real rulings but the column count is wrong, the default snap tolerances are merging near-coincident lines or missing thin ones. Set the strategies explicitly and widen snap_tolerance so slightly misaligned rulings from a CAD export collapse to one edge, while keeping intersection_tolerance tight enough not to invent cells.

import pdfplumber

RULED_SETTINGS = {
    "vertical_strategy": "lines",
    "horizontal_strategy": "lines",
    "snap_tolerance": 4,          # merge rulings within 4pt of each other
    "join_tolerance": 4,          # bridge collinear segments split by anti-aliasing
    "intersection_tolerance": 3,  # how close lines must be to form a cell corner
    "min_words_vertical": 1,
}

def extract_ruled(page) -> list[list[str]]:
    """Extract a table from a page that has genuine drawn rulings."""
    table = page.extract_table(RULED_SETTINGS)
    return table or []

Step 2 — Add explicit vertical lines for borderless columns jump to heading

Borderless schedules have no rulings to snap to, so supply the column x-positions yourself. Derive them once from a known-good page by inspecting page.extract_words() x-coordinates, then feed them as explicit_vertical_lines. The "text" horizontal strategy infers rows from text baselines, which borderless tables always have.

def extract_borderless(page, column_x: list[float]) -> list[list[str]]:
    """Extract a table with no vertical rules by pinning column edges.

    column_x: page-space x-coordinates of each column boundary, including
    the left edge of the first column and the right edge of the last.
    """
    settings = {
        "vertical_strategy": "explicit",
        "explicit_vertical_lines": column_x,
        "horizontal_strategy": "text",
        "snap_tolerance": 3,
        "text_x_tolerance": 2,   # keep adjacent numeric columns from fusing
    }
    table = page.extract_table(settings)
    return table or []

The text_x_tolerance is the lever that stops Lot Area and FAR from fusing: lower it and pdfplumber requires a larger horizontal gap before it treats two words as one cell.

Step 3 — Detect scanned pages and route them to OCR jump to heading

A page with no text layer must be diverted before you waste cycles re-parsing it. Classify on the character count and image coverage, rasterize with page.to_image(), and hand the bitmap to Tesseract. Keep the OCR path isolated so a slow raster page never blocks the fast vector pages.

import logging
import pytesseract

logger = logging.getLogger("parcel_ingestion")

def is_scanned(page, char_floor: int = 20) -> bool:
    """A page with almost no characters but a full-page image is a scan."""
    text = page.extract_text() or ""
    has_full_image = any(
        img["width"] * img["height"] > 0.5 * (page.width * page.height)
        for img in page.images
    )
    return len(text.strip()) < char_floor and has_full_image

def ocr_page(page, dpi: int = 300) -> str:
    """Rasterize and OCR a scanned page; higher dpi helps small parcel digits."""
    try:
        image = page.to_image(resolution=dpi).original
        # --psm 6 assumes a single uniform block of text, good for tables
        return pytesseract.image_to_string(image, config="--psm 6")
    except Exception as exc:
        logger.error(f"OCR failed on page {page.page_number}: {exc}")
        return ""

Step 4 — Normalize raw rows into parcel records jump to heading

A raw cell grid is not a record set. Cells carry None, stray newlines, and header rows mixed with data. Map the header row to canonical field names, coerce types, and reject rows whose parcel ID does not match the expected format. This is the same discipline formalized under attribute normalization rules, applied at parse time.

import re

PARCEL_ID = re.compile(r"^\d{2,3}-\d{2,4}-\d{2,4}$")  # e.g. 12-3456-001

HEADER_MAP = {
    "parcel id": "parcel_id",
    "apn": "parcel_id",
    "zoning": "district",
    "district": "district",
    "lot area (sf)": "lot_area_sf",
    "far": "far",
}

def normalize_rows(table: list[list[str]]) -> list[dict]:
    """Turn a raw cell grid into validated parcel records."""
    if not table or len(table) < 2:
        return []
    header = [(_c or "").strip().lower() for _c in table[0]]
    fields = [HEADER_MAP.get(h) for h in header]
    records = []
    for raw in table[1:]:
        cells = [(c or "").replace("\n", " ").strip() for c in raw]
        record = {f: v for f, v in zip(fields, cells) if f}
        pid = record.get("parcel_id", "")
        if not PARCEL_ID.match(pid):
            logger.warning(f"Skipping row with bad parcel id: {pid!r}")
            continue
        if "lot_area_sf" in record:
            record["lot_area_sf"] = _to_float(record["lot_area_sf"])
        records.append(record)
    return records

def _to_float(value: str):
    try:
        return float(value.replace(",", ""))
    except ValueError:
        return None

A short reference for the table strategies you will dispatch between:

Step 5 — Orchestrate dispatch across the document jump to heading

Tie the profile to the extractor. Each page follows exactly one lane, and every produced record set is validated before it leaves the parser. Records that fail structural checks feed the same schema validation and data quality checks boundary the rest of the pipeline relies on.

def extract_document(pdf_path: str, column_x: list[float]) -> list[dict]:
    all_records: list[dict] = []
    with pdfplumber.open(pdf_path) as pdf:
        for page in pdf.pages:
            if is_scanned(page):
                text = ocr_page(page)
                table = [ln.split() for ln in text.splitlines() if ln.strip()]
            elif page.lines:
                table = extract_ruled(page)
            else:
                table = extract_borderless(page, column_x)
            all_records.extend(normalize_rows(table))
    logger.info(f"{pdf_path}: {len(all_records)} parcel records")
    return all_records

Verification & testing jump to heading

Confirm the extraction is correct, not merely non-empty.

• Column count matches the header. Assert every normalized record has the same key set. A row short one field means a borderless column fused — lower text_x_tolerance for that source.
• Row count matches a visible tally. Municipal schedules often print Total parcels: N in a footer. Extract it and assert len(records) == N; a mismatch flags dropped or duplicated rows.
• Parcel IDs are unique and well-formed. Every parcel_id must match PARCEL_ID and appear once. Duplicates usually mean a header row leaked into the data on a multi-page table with repeated headers.
• OCR pages round-trip a checksum digit. Many APN formats embed a check digit; validate it after OCR so a 5 misread as 6 is caught before it reaches the parcel fabric.
• Deterministic output. Re-running the same PDF must yield an identical record count and ordering. Non-determinism points to a page profiled inconsistently — pin its lane in the manifest.

Failure recovery jump to heading

When a page or row defeats the pipeline mid-batch, the run must continue and stay reproducible.

• Quarantine rows, never drop them. A row whose parcel ID fails validation is written to a failed_rows/ partition with the source hash, page index, and raw cells, then the loop continues. This mirrors the dead-letter discipline across the ingestion layer.
• Escalate OCR confidence failures. When Tesseract returns text but the check-digit assertion fails on more than a few rows, flag the whole page for manual review rather than trusting a partially misread grid.
• Re-derive column_x on layout change. If a borderless page suddenly yields the wrong column count across every row, the template’s column positions moved. Recompute explicit_vertical_lines from a fresh known-good page instead of forcing the stale hints.
• Resume from the last page index. Persist the last successfully parsed page per document so reruns skip completed pages and never double-write the target table.
• Alert on drift. Emit structured logs with pdf_hash, page_index, lane, and rows_quarantined, and trigger an alert when quarantine exceeds 2% of the daily batch.

Frequently asked questions jump to heading

Related jump to heading

• Parent topic: PDF & HTML Scraping Pipelines
• Section overview: Automated Feed Ingestion & GIS Data Parsing
• Attribute Normalization Rules — mapping raw cells to canonical parcel fields
• Schema Validation & Data Quality Checks — validating parcel records downstream
• Scraping zoning PDFs with Python and PyPDF2 — text and coordinate extraction from non-tabular PDFs