Change Detection & Geometry Diffing

Two exports of the same county zoning layer, pulled a month apart, will almost never be byte-identical even when nothing was rezoned. The GIS server re-serializes polygons in a different vertex order, a coordinate-precision setting rounds the seventh decimal differently, an ETL step re-closes rings the other way around, and suddenly a naive geom_a == geom_b comparison reports that every parcel in the jurisdiction changed. The opposite failure is quieter and worse: a genuine boundary adjustment or a district reclassification slips through because your comparison only looked at an attribute column and never touched the geometry. Change detection and geometry diffing is the discipline that sits between those two failure modes — deciding, per parcel and per overlay, whether the difference between two snapshots is a real municipal action worth alerting on or cosmetic noise that should be suppressed. This area is part of the Spatial Impact Analysis & Zoning Change Detection framework, and it is the stage that converts a pair of raw snapshots into a structured changeset that every downstream analysis depends on.

The core problem is that “did this parcel change?” is not one question but three. Did its geometry move (a lot-line adjustment, a split, a merge, an annexation)? Did its attributes change (a rezone from R-2 to C-1, a new overlay flag, a corrected owner)? Or did nothing meaningful change and the file merely looks different? A production diff has to answer all three independently, because the downstream response differs sharply — a geometry-only change re-triggers spatial joins, an attribute-only rezone fires a notification, and a cosmetic difference must be swallowed so it never reaches a human reviewer. Conflating them is how change-detection systems earn a reputation for crying wolf and get ignored.

Prerequisites and operational context jump to heading

Diffing is only meaningful when the two things you compare are actually comparable. Several conditions must hold before any of the code below produces trustworthy results:

• Both snapshots share a single CRS. A diff computed across mismatched projections reports enormous phantom movement because the same parcel occupies different coordinate values in EPSG:4326 versus a State Plane zone in feet. Reproject both inputs to one working CRS first, following the CRS alignment strategies you have standardized on. For area-based thresholds, that working CRS must be a projected one measured in meters or feet — never compute a symmetric-difference area in degrees.
• Both snapshots are immutable, timestamped captures. You cannot diff against a live endpoint that mutates mid-run. The prior and current states come from the temporal versioning and snapshots layer, which guarantees each snapshot is a frozen, addressable point in time with a known effective date.
• A stable join key exists. Change detection is fundamentally a keyed comparison: you match parcel T0 to parcel T1 by an identifier, then compare their contents. If parcel IDs are unstable across releases — reissued after a subdivision, re-formatted with leading zeros stripped — the diff will misread renumbered parcels as simultaneous add-and-remove events. Normalize identifiers before joining.
• Attributes are already normalized. The zoning-code column you diff must mean the same thing in both snapshots. If one export writes R2 and the next writes R-2 for the identical district, an attribute diff fires a false rezone. That normalization is upstream work.

If any of these is missing, fix it before diffing. A diff engine is an amplifier: feed it two snapshots in different projections with unstable keys and it will confidently manufacture thousands of changes that never happened.

Architecture: keyed comparison with a normalization gate jump to heading

Every robust diff follows the same three-phase shape, visible in the diagram above: partition by key, normalize the matched pairs, then measure and classify. The order matters, because normalization is what lets the measurement stage distinguish signal from noise.

The first phase is a full outer join between the two snapshots on the parcel identifier. This immediately partitions the universe into three sets. Keys present only in the current snapshot are added parcels — new subdivisions, newly annexed land, records that simply did not exist before. Keys present only in the prior snapshot are removed — merged lots, de-annexations, or records dropped from the source. Both of these classes are resolved by set membership alone and never need geometry comparison. Only the intersection — keys present in both snapshots — requires the expensive per-pair work, and in a typical monthly refresh that intersection is 99%+ of the parcels while the actual changes are a fraction of a percent. Structuring the join this way keeps the costly geometry math scoped to matched pairs.

The second phase normalizes each matched pair before comparison. This is the step that defeats cosmetic noise. Shapely’s normalize() rewrites a geometry into a canonical form — a deterministic ring orientation, a canonical starting vertex, and a consistent ordering of interior rings and multipart components — so that two polygons that describe the same boundary but were serialized differently become directly comparable. On top of normalization, a snap-to-grid pass with set_precision() collapses sub-tolerance coordinate jitter to a fixed grid, so that a difference in the sixth decimal place of longitude does not register as movement. After both transforms, a hash of the well-known binary (WKB) representation gives a cheap equality test: if the normalized, snapped WKB hashes match, the geometry is byte-for-byte identical and no further geometry math is needed for that pair.

The third phase measures what survives normalization. For pairs whose geometry hashes differ, two complementary metrics quantify the change. The symmetric-difference area — the area belonging to one polygon or the other but not both — measures how much the footprint moved in absolute terms, which is the right signal for detecting real boundary adjustments. The Hausdorff distance measures the largest gap between the two boundaries, which catches a single displaced vertex that barely changes area but signals a survey correction. Attribute comparison runs in parallel and is entirely independent: a set of watched columns (zoning code, overlay flags, land-use designation) is compared field by field to produce an attribute delta. The classifier then combines both signals — geometry moved, attributes changed, neither, or both — into a single change class per parcel.

Production implementation jump to heading

The engine below diffs two GeoDataFrames representing consecutive zoning snapshots. It normalizes and snaps geometry, hashes for fast equality, measures symmetric-difference area and Hausdorff distance on the survivors, computes an attribute delta over a watched column set, and emits a typed changeset. It is defensive about the failures that actually occur in municipal data: invalid geometry, empty geometry, CRS mismatch, and duplicate keys.

import hashlib
import logging
from dataclasses import dataclass, field
from typing import Dict, List, Optional

import geopandas as gpd
from shapely import set_precision
from shapely.geometry.base import BaseGeometry

logging.basicConfig(level=logging.INFO, format="%(asctime)s | %(levelname)s | %(message)s")
logger = logging.getLogger("zoning_diff")

# Grid size for snap-to-grid, in the units of the working CRS (meters here).
# 0.05 m collapses sub-5cm survey jitter without merging genuinely distinct lines.
SNAP_GRID = 0.05
# Below this symmetric-difference area (m^2) a geometry change is treated as noise.
AREA_NOISE_FLOOR = 1.0
# A single vertex displaced beyond this (m) is a real change even if area barely moves.
HAUSDORFF_FLOOR = 0.5
# Attribute columns whose change counts as a zoning action.
WATCHED_ATTRS = ("zoning_code", "overlay", "land_use")


@dataclass
class ParcelChange:
    parcel_id: str
    change_class: str                     # added | removed | geometry | attribute | both | unchanged
    symdiff_area: float = 0.0
    hausdorff: float = 0.0
    attr_delta: Dict[str, tuple] = field(default_factory=dict)


def _canonical_geom(geom: Optional[BaseGeometry]) -> Optional[BaseGeometry]:
    """Normalize ring order/start vertex and snap coordinates to a fixed grid.

    Returns None for missing/empty/irreparably-invalid geometry so the caller
    can route the parcel to quarantine instead of trusting a bad comparison.
    """
    if geom is None or geom.is_empty:
        return None
    try:
        snapped = set_precision(geom, SNAP_GRID)      # collapse sub-tolerance jitter
        if snapped.is_empty:
            return None
        if not snapped.is_valid:
            snapped = snapped.buffer(0)               # repair self-intersections
            if snapped.is_empty or not snapped.is_valid:
                return None
        return snapped.normalize()                    # canonical vertex ordering
    except Exception as exc:                          # GEOS topology error, etc.
        logger.warning("Geometry normalization failed: %s", exc)
        return None


def _geom_hash(geom: BaseGeometry) -> str:
    """Stable hash of canonical WKB; equal hashes mean identical footprints."""
    return hashlib.sha256(geom.wkb).hexdigest()


def _attr_delta(row_a, row_b, cols=WATCHED_ATTRS) -> Dict[str, tuple]:
    """Field-by-field comparison of watched attributes -> {col: (old, new)}."""
    delta = {}
    for col in cols:
        old, new = row_a.get(col), row_b.get(col)
        if old != new:
            delta[col] = (old, new)
    return delta


def diff_snapshots(
    prior: gpd.GeoDataFrame,
    current: gpd.GeoDataFrame,
    key: str = "parcel_id",
) -> List[ParcelChange]:
    """Classify every parcel as added, removed, geometry, attribute, both, or unchanged."""
    if prior.crs is None or current.crs is None:
        raise ValueError("Both snapshots must carry a CRS before diffing.")
    if prior.crs != current.crs:
        raise ValueError(f"CRS mismatch: {prior.crs} vs {current.crs}; reproject first.")
    if prior.crs.is_geographic:
        # Area/distance thresholds are metric; a geographic CRS makes them meaningless.
        raise ValueError("Diff requires a projected CRS (meters/feet), not degrees.")

    prior = prior.set_index(key, drop=False)
    current = current.set_index(key, drop=False)
    for name, df in (("prior", prior), ("current", current)):
        dupes = df.index[df.index.duplicated()].unique().tolist()
        if dupes:
            raise ValueError(f"Duplicate {key} in {name}: {dupes[:5]} ... resolve before diffing.")

    prior_keys, current_keys = set(prior.index), set(current.index)
    changes: List[ParcelChange] = []

    # Set-membership classes need no geometry math.
    for pid in current_keys - prior_keys:
        changes.append(ParcelChange(str(pid), "added"))
    for pid in prior_keys - current_keys:
        changes.append(ParcelChange(str(pid), "removed"))

    # Matched pairs: the only rows that pay for geometry comparison.
    for pid in prior_keys & current_keys:
        row_a, row_b = prior.loc[pid], current.loc[pid]
        attr_delta = _attr_delta(row_a, row_b)

        g_a = _canonical_geom(row_a.geometry)
        g_b = _canonical_geom(row_b.geometry)
        if g_a is None or g_b is None:
            logger.warning("Parcel %s has unusable geometry; quarantining.", pid)
            changes.append(ParcelChange(str(pid), "quarantine", attr_delta=attr_delta))
            continue

        geom_moved = False
        symdiff_area = hausdorff = 0.0
        if _geom_hash(g_a) != _geom_hash(g_b):
            # Hashes differ: measure whether the difference clears the noise floor.
            symdiff_area = g_a.symmetric_difference(g_b).area
            hausdorff = g_a.hausdorff_distance(g_b)
            geom_moved = symdiff_area >= AREA_NOISE_FLOOR or hausdorff >= HAUSDORFF_FLOOR

        attr_changed = bool(attr_delta)
        if geom_moved and attr_changed:
            klass = "both"
        elif geom_moved:
            klass = "geometry"
        elif attr_changed:
            klass = "attribute"
        else:
            klass = "unchanged"      # hashes differed only by sub-tolerance noise

        if klass != "unchanged":
            changes.append(ParcelChange(
                str(pid), klass,
                symdiff_area=round(symdiff_area, 3),
                hausdorff=round(hausdorff, 3),
                attr_delta=attr_delta,
            ))

    logger.info("Diff complete: %d changes over %d matched parcels",
                len(changes), len(prior_keys & current_keys))
    return changes

Several choices in this routine are deliberate. Normalization and snapping happen before the hash comparison, so the fast-path equality test already ignores vertex reordering and precision jitter — the expensive symmetric_difference and hausdorff_distance calls only run for pairs whose hashes genuinely differ. When they do run, a change is confirmed only if it clears an absolute area floor or a Hausdorff floor, so a re-rounded coordinate that survives snapping is still filtered as unchanged rather than leaking a false positive. Attribute and geometry signals are computed separately and combined at the end, which is what lets the classifier distinguish a pure rezone (attribute) from a lot-line adjustment (geometry) from an annexation-plus-rezone (both). Unusable geometry routes to a quarantine class instead of silently comparing a repaired polygon against a raw one.

Edge cases and gotchas jump to heading

Municipal snapshot pairs break the naive diff in specific, recurring ways:

• Ring-orientation flips masquerading as change. Some exporters emit polygons clockwise, others counter-clockwise, and a few flip orientation between releases of the same dataset. Without normalize(), every parcel reads as modified. Normalization fixes this, but only after set_precision/buffer(0) has produced a valid geometry — calling normalize() on a self-intersecting polygon can raise.
• Precision inflation from reprojection. Reprojecting on ingest introduces tiny coordinate perturbations that differ run to run. If your snap grid is finer than the reprojection’s residual error, those perturbations survive snapping and inflate the symmetric-difference area. Set SNAP_GRID coarser than your reprojection tolerance, not finer.
• Sliver polygons from symmetric difference. Two nearly identical polygons produce a symmetric difference made of hairline slivers with negligible area but occasionally large Hausdorff distance. Trusting Hausdorff alone flags these; requiring the area floor or a Hausdorff floor above realistic survey error keeps slivers from firing.
• Split and merge events read as add + remove. When one parcel splits into three, the old ID vanishes and three new IDs appear, so a keyed diff reports one removal and three additions with no link between them. Detecting that they are topologically related requires a spatial overlay of the removed footprint against the added ones — that reconciliation belongs downstream, not in the keyed diff.
• Attribute nulls that flip both ways. A source that intermittently omits the overlay column makes it read as None one month and "HPO" the next, firing a phantom rezone on the recovery. Treat a transition to/from null with suspicion and confirm against the source’s completeness before alerting.
• Geographic-CRS area thresholds. Computing symmetric_difference().area on EPSG:4326 geometry returns an area in square degrees, which is meaningless and varies with latitude. The guard clause rejecting a geographic CRS is not optional; it prevents the most common silent numerical error in this whole workflow.

Change classification reference jump to heading

Each detectable difference maps to a signal you can measure and a concrete action. Treating every difference as “changed” is what makes a diff useless; this table is the contract the classifier enforces.

Integration points jump to heading

Change detection is a midstream stage defined by what feeds it and what consumes it. Its inputs are two immutable captures from the temporal versioning and snapshots layer, which is responsible for freezing each source pull into an addressable, effective-dated state — without that guarantee, “the previous version” is ambiguous and the diff is not reproducible. Its output is the changeset that drives spatial overlay analysis: once a parcel is flagged as geometry-modified or added, the overlay engine intersects its new footprint against flood zones, school districts, and transit buffers to compute the actual impact of the change, and there is no reason to recompute those expensive intersections for the 99% of parcels the diff marked unchanged. The changeset is therefore both a result and a work list — it tells the overlay stage exactly which parcels to reprocess.

Upstream, the snapshots themselves arrive through the ingestion side of the platform. The current-state capture is frequently the product of GIS export sync workflows, which pull shapefiles and GeoJSON from municipal portals on a schedule; the diff runs each time a fresh export lands and is compared against the last-known-good snapshot. The whole comparison is only valid because both sides were forced into one projection by the CRS alignment strategies applied at ingest — the diff engine refuses to run otherwise, which surfaces a projection mistake as a hard error at diff time rather than as thousands of phantom changes.

Compliance and audit artifacts jump to heading

For PropTech underwriting and municipal-record defensibility, detecting a change is not enough — you must be able to prove what changed, when, and against which baseline. Every diff run should emit:

• A diff manifest recording the two snapshot identifiers and their effective dates, the working CRS, the exact SNAP_GRID, AREA_NOISE_FLOOR, and HAUSDORFF_FLOOR thresholds in force, and the library versions (shapely/GEOS, geopandas). Thresholds are part of the result: the same snapshot pair diffed at a different noise floor yields a different changeset, so the parameters must travel with the output.
• The immutable changeset ledger — one record per changed parcel with its class, symmetric-difference area, Hausdorff distance, and the before/after values of every changed attribute, retained so a reviewer can reconstruct the exact reason a parcel was flagged.
• A suppression log counting how many pairs were classified unchanged after normalization and snapping. This is your noise metric; a sudden spike in suppressions usually means an upstream exporter changed its serialization and warrants a look before the noise floor hides a real change.
• A content hash of each input snapshot, so a re-run against the same two captures reproduces byte-identical results and any silent drift in an upstream file is detectable.

These artifacts pair with the downstream schema validation and data quality checks to form a continuous, defensible chain from raw municipal publication to an alert a human can act on.

FAQ jump to heading

Related jump to heading

• Diffing parcel geometries to detect zoning boundary changes — a narrow, runnable walkthrough of separating a real rezone from serialization noise
• Spatial overlay analysis — the downstream consumer that turns the changeset into per-parcel impact
• Temporal versioning & snapshots — the immutable, effective-dated captures the diff compares
• CRS alignment strategies — the projection contract every diff enforces before it will run
• GIS export sync workflows — the ingestion stage that produces the current-state snapshot

Up: Spatial Impact Analysis & Zoning Change Detection