Fallback Routing Logic for Municipal GIS Pipelines

Municipal data feeds fail in ways that are routine, not exceptional: a county REST endpoint throttles to 429 under a Monday-morning batch, an FTP credential rotates without notice over a weekend, a planning portal restructures its DOM and silently breaks a scraper, or an authoritative GeoJSON service returns HTTP 200 with an empty FeatureCollection. In a naïve pipeline each of these is a single point of failure that halts ingestion, leaves the spatial store stale, and — worse — can let a partial or malformed payload through as if it were authoritative. Fallback routing logic is the part of Municipal Zoning Data Architecture & Compliance Frameworks that turns those inevitable source failures into graceful degradation rather than data loss: a deterministic hierarchy of sources, each with its own trust level, so the pipeline always returns the best available geometry while recording exactly which tier served it and why. It is not a retry wrapper bolted onto a request — it is an explicit, auditable state machine that decides where parcel and zoning data comes from when the primary source cannot be trusted.

Prerequisites and operational context jump to heading

Fallback routing only behaves deterministically when a few contracts are already in place. Without them, “fallback” degrades into an implicit try/except that swallows errors and hides which source actually produced production geometry:

• A ranked source registry. Every jurisdiction needs an ordered list of sources with an explicit trust level: primary live API or database replica, then secondary caches and aggregators, then tertiary recovery. The ranking is configuration, not code branches scattered through the pipeline.
• A canonical staging shape. Each source returns a different physical format, so all of them must normalize to the same record before routing decisions mean anything. That target shape is defined by the municipal data structures used across the architecture.
• A settled working projection. A cached shapefile in state plane feet and a live feed in EPSG:4326 cannot be compared, deduplicated, or bounds-checked until both are aligned. Routing assumes the CRS alignment strategies gate runs on every tier’s output before validation.
• Known jurisdictional bounding boxes. A secondary cache can be stale and geographically wrong — returning a neighboring county’s parcels. Routing needs a per-jurisdiction envelope to reject out-of-region geometry on fallback.
• A persistent last-known-good store. Tier 2 is only useful if a previous successful run was snapshotted with its checksum and timestamp. Without it the pipeline has nothing to fall back to.
• A quarantine / dead-letter path. When every tier fails or returns invalid geometry, the records must land somewhere reviewable — never be silently dropped or silently substituted.

If these are missing, fix them first. A fallback hierarchy with no trust ranking, no canonical shape, and no place to send rejects is just exception handling pretending to be resilience.

Architecture: a trust-ranked state machine jump to heading

The core design decision is to treat source selection as a state machine over a ranked registry, not as nested try/except blocks. Each tier is attempted in priority order, and each attempt passes through the same trust gate before its output is accepted. That gate is what distinguishes fallback routing from a plain retry loop: a source can return HTTP 200 and still fail the gate because the payload is empty, out of bounds, schema-drifted, or topologically invalid.

The hierarchy has three tiers, each with a confidence weight that travels downstream so consumers can reason about how much to trust the record:

Routing is fail-forward, not fail-fast: a failed tier does not abort the run, it advances the state machine to the next tier. Only exhausting all tiers raises a terminal failure and fires a compliance alert. Two properties make this safe:

1. Idempotent normalization per tier. Whatever a tier returns is coerced to the canonical staging shape and the working CRS before the trust gate runs, so the gate compares like with like regardless of which source fired.
2. Provenance on every record. Each accepted feature is tagged with source_tier, fetch_timestamp, validation_status, and a confidence weight. A downstream system never has to guess whether a parcel came from the live API or a three-day-old cache — the record says so.

The trust gate itself enforces, in order: HTTP success and latency under the SLA; a non-empty result set; spatial intersection with the jurisdiction envelope; and per-geometry topological validity. A tier only “succeeds” when its normalized output clears all four. This ordering is deliberate — cheap checks (status, emptiness) short-circuit before expensive ones (per-feature topology repair).

Production implementation jump to heading

The router below is a complete, runnable implementation. It configures a resilient HTTP session with error handling & retry logic at the transport layer (exponential backoff on transient 5xx/429), then layers the trust-ranked state machine on top. Transport retries handle flaky sources; the tier hierarchy handles failed ones. It assumes geopandas, requests, urllib3, and shapely are available.

import io
import logging
from dataclasses import dataclass, field
from datetime import datetime, timezone
from typing import Any, Optional

import geopandas as gpd
import requests
from requests.adapters import HTTPAdapter
from shapely.geometry import box
from shapely.validation import explain_validity
from urllib3.util.retry import Retry

logging.basicConfig(
    level=logging.INFO, format="%(asctime)s [%(levelname)s] %(message)s"
)
logger = logging.getLogger("fallback_router")


@dataclass
class SourceTier:
    """One ranked source in the fallback hierarchy."""
    name: str                 # "primary" | "secondary" | "tertiary"
    url: str
    confidence: float         # 1.0 primary, 0.7 secondary, 0.3 tertiary


@dataclass
class JurisdictionProfile:
    """Per-jurisdiction routing context."""
    name: str
    bbox_wgs84: tuple[float, float, float, float]  # minx, miny, maxx, maxy
    working_epsg: int = 4326
    latency_sla_seconds: float = 15.0
    tiers: list[SourceTier] = field(default_factory=list)


class ZoningFallbackRouter:
    """Trust-ranked fallback router with spatial validation and audit trail."""

    REQUIRED_COLS = ["parcel_id", "zoning_code", "effective_date", "geometry"]

    def __init__(self, profile: JurisdictionProfile):
        self.profile = profile
        self.envelope = box(*profile.bbox_wgs84)
        self.audit_log: list[dict[str, Any]] = []

        # Transport-layer resilience: retry transient failures before the
        # tier state machine ever decides a source has "failed".
        self.session = requests.Session()
        retry_strategy = Retry(
            total=3,
            backoff_factor=1.5,                       # 0s, 1.5s, 3s, 6s
            status_forcelist=[429, 500, 502, 503, 504],
            allowed_methods=["GET"],
            respect_retry_after_header=True,
        )
        adapter = HTTPAdapter(max_retries=retry_strategy)
        self.session.mount("https://", adapter)
        self.session.mount("http://", adapter)

    # --- fetch + normalize -------------------------------------------------
    def _fetch(self, tier: SourceTier) -> Optional[gpd.GeoDataFrame]:
        try:
            resp = self.session.get(tier.url, timeout=self.profile.latency_sla_seconds)
            resp.raise_for_status()
        except requests.exceptions.RequestException as exc:
            logger.error("Fetch failed for %s (%s): %s", tier.name, tier.url, exc)
            self._record(tier, "FETCH_ERROR", 0, getattr(exc.response, "status_code", None))
            return None

        try:
            gdf = gpd.read_file(io.BytesIO(resp.content))
        except Exception as exc:  # malformed payload, not valid spatial data
            logger.error("Unreadable payload from %s: %s", tier.name, exc)
            self._record(tier, "PARSE_ERROR", 0, resp.status_code)
            return None
        return self._normalize(gdf)

    def _normalize(self, gdf: gpd.GeoDataFrame) -> gpd.GeoDataFrame:
        """Coerce any tier's output to the canonical shape + working CRS."""
        gdf = gdf.copy()
        for col in self.REQUIRED_COLS:
            if col not in gdf.columns:
                gdf[col] = None

        # Align CRS so bounds + dedup are comparable across tiers.
        if gdf.crs is None:
            gdf = gdf.set_crs(epsg=4326, allow_override=True)
        if gdf.crs.to_epsg() != self.profile.working_epsg:
            gdf = gdf.to_crs(epsg=self.profile.working_epsg)

        extra = [c for c in gdf.columns if c not in self.REQUIRED_COLS]
        return gdf[self.REQUIRED_COLS + extra]

    # --- trust gate --------------------------------------------------------
    def _passes_trust_gate(self, gdf: gpd.GeoDataFrame, tier: SourceTier) -> bool:
        if gdf is None or gdf.empty:
            logger.warning("Empty result from %s.", tier.name)
            self._record(tier, "EMPTY", 0, 200)
            return False

        in_bounds = gdf[gdf.intersects(self.envelope)]
        if in_bounds.empty:
            logger.warning("No geometry intersects %s envelope from %s.",
                           self.profile.name, tier.name)
            self._record(tier, "OUT_OF_BOUNDS", 0, 200)
            return False
        if len(in_bounds) < len(gdf):
            logger.warning("%d of %d features dropped as out-of-region from %s.",
                           len(gdf) - len(in_bounds), len(gdf), tier.name)
        return True

    def _validate_and_tag(self, gdf: gpd.GeoDataFrame, tier: SourceTier) -> gpd.GeoDataFrame:
        gdf = gdf[gdf.intersects(self.envelope)].copy()

        valid = gdf.geometry.is_valid
        invalid_count = int((~valid).sum())
        gdf["validation_status"] = "VALID"
        gdf["validation_reason"] = None
        if invalid_count:
            logger.warning("%d invalid geometries from %s quarantined.",
                           invalid_count, tier.name)
            gdf.loc[~valid, "validation_status"] = "QUARANTINED"
            gdf.loc[~valid, "validation_reason"] = (
                gdf.loc[~valid, "geometry"].apply(explain_validity)
            )

        # Provenance every downstream consumer can trust-weight against.
        gdf["source_tier"] = tier.name
        gdf["confidence"] = tier.confidence
        gdf["fetch_timestamp"] = datetime.now(timezone.utc).isoformat()
        return gdf

    # --- audit + state machine --------------------------------------------
    def _record(self, tier: SourceTier, status: str, count: int,
                http_status: Optional[int]) -> None:
        self.audit_log.append({
            "tier": tier.name,
            "status": status,
            "record_count": count,
            "http_status": http_status,
            "confidence": tier.confidence,
            "timestamp": datetime.now(timezone.utc).isoformat(),
        })

    def execute(self) -> gpd.GeoDataFrame:
        logger.info("Routing %s through %d tiers.", self.profile.name,
                    len(self.profile.tiers))
        for tier in self.profile.tiers:           # fail-forward, in priority order
            gdf = self._fetch(tier)
            if not self._passes_trust_gate(gdf, tier):
                continue
            tagged = self._validate_and_tag(gdf, tier)
            self._record(tier, "SUCCESS", len(tagged), 200)
            logger.info("Served from %s tier (confidence %.1f, %d records).",
                        tier.name, tier.confidence, len(tagged))
            return tagged

        logger.error("All tiers exhausted for %s; firing compliance alert.",
                     self.profile.name)
        self._record(SourceTier("tertiary", "", 0.0), "EXHAUSTED", 0, None)
        return gpd.GeoDataFrame(columns=self.REQUIRED_COLS, geometry="geometry")

The two decisions worth calling out: the trust gate runs after normalization so an HTTP-200-but-empty or out-of-region payload is treated as a failure and advances the state machine, and every outcome — success, fetch error, empty, out-of-bounds, exhaustion — appends to audit_log so the run can be reconstructed exactly. The router never returns geometry without provenance, and never silently substitutes a tier without recording the substitution.

Edge cases and gotchas jump to heading

Municipal sources break fallback routing in specific, recurring ways:

• HTTP 200 with an empty FeatureCollection. A misconfigured WFS or a query past the layer’s date range returns success with zero features. Status-only checks pass; only the emptiness gate catches it, which is why emptiness is a gate condition and not just a log line.
• The “stale but wrong jurisdiction” cache. A secondary aggregator keyed on a fuzzy place name can return a neighboring county. Without the bounding-box intersection check the pipeline silently ingests the wrong parcels with a plausible record count.
• Schema drift between tiers. A primary feed ships effective_date as ISO-8601; a fallback cache flattens it to a shapefile DBF field truncated to 10 characters, or renames zoning_code to a legacy ZONE column. Normalization must reconcile these before the record reaches zoning taxonomy mapping, or the code-translation step fails on the fallback path only — a bug that never appears while the primary source is healthy.
• 429 storms cascading into fallback. When a county API rate-limits, naïve retries make it worse. Transport-layer backoff with respect_retry_after_header=True honors the server’s Retry-After; only after retries are exhausted should the state machine demote to the cache. Tuning this boundary belongs with municipal API rate limit management.
• CRS mismatch across tiers. A cached state-plane shapefile and a live EPSG:4326 feed will bounds-check against different numbers entirely. Normalization must reproject every tier to the working CRS before the envelope intersection, or fallbacks fail the bounds gate for the wrong reason.
• Topology exceptions on fallback only. Older cached snapshots often predate a make_valid pass and carry self-intersections. Quarantine invalid geometry per-feature rather than rejecting the whole tier — a cache with three bad rings is still better than tertiary interpolation.
• Silent confidence laundering. If the fallback record drops its source_tier/confidence tag, downstream systems treat three-day-old cache data as live. The provenance columns must be non-nullable and propagate through every join.

Integration points jump to heading

Fallback routing sits between ingestion and the spatial store, and is defined by what it consumes and what it guarantees downstream. Upstream, it is fed by the async batch processing engine that fans out fetches across many jurisdictions — the router is the per-jurisdiction policy that batch engine applies when a single source in the fan-out fails. Each tier’s output is run through the CRS alignment strategies gate so every fallback lands in the same metric-correct projection before the bounds check, and the dual relationship matters: when alignment quarantines a feature for an unresolvable CRS, it is fallback routing that substitutes a secondary source so the map has no hole.

Downstream, the trust-weighted, provenance-tagged stream feeds zoning taxonomy mapping, where a secondary source’s legacy zoning_code values may need a translation matrix to align with current district standards, and the confidence weight tells the mapper how aggressively to trust an ambiguous code. Records also flow to schema validation & data quality checks, which assert that a fallback payload still satisfies the canonical contract before it reaches the versioned store. When a tertiary tier writes a reconstructed snapshot back out, that export is handled by the GIS export sync workflows so the last-known-good store stays current for the next run’s secondary tier.

Compliance and audit artifacts jump to heading

For PropTech underwriting and regulatory review, the question is never just “what is this parcel’s zoning” but “which source said so, when, and how much do we trust it.” Fallback routing is where that lineage is captured, so every run must emit:

• A routing decision ledger. The serialized audit_log: every tier attempted, its outcome (SUCCESS, EMPTY, OUT_OF_BOUNDS, FETCH_ERROR, EXHAUSTED), the HTTP status, record count, and timestamp — the evidence of why a given tier served the data.
• Per-record provenance. source_tier, confidence, and fetch_timestamp persisted alongside every geometry, so an underwriter can filter out anything below a confidence threshold or flag parcels last sourced from a stale cache.
• A quarantine register. Invalid geometries with their explain_validity reason, retained rather than discarded, so a reviewer can see what was withheld and why.
• Tier-staleness metrics. How old the secondary snapshot was when it served, and how often the primary tier was unavailable — early warning that a municipal feed is degrading before it fails outright.

These artifacts dovetail with compliance framework integration to form a continuous chain of custody: any parcel’s zoning state can be traced from the exact source tier and timestamp that produced it, satisfying the data-lineage requirements that municipal record-keeping and PropTech underwriting audits demand. The full design for the recovery tiers — including synthetic interpolation from adjacent zoning polygons — is covered in designing fallback routing for missing municipal data feeds.

Implementation checklist jump to heading

• Express sources as a ranked registry with explicit confidence weights, not branching code.
• Normalize every tier to the canonical shape and working CRS before any trust check.
• Run the trust gate (status, emptiness, bounds, topology) after normalization on every tier.
• Use transport-layer backoff that honors Retry-After before demoting to the next tier.
• Reject out-of-region geometry against a per-jurisdiction bounding box.
• Quarantine invalid geometries per-feature instead of rejecting a whole tier.
• Tag every record with source_tier, confidence, and fetch_timestamp, non-nullable.
• Serialize the routing audit_log to a persistent compliance store on every run.

FAQ jump to heading

Related jump to heading

• Designing fallback routing for missing municipal data feeds — the recovery tiers and synthetic interpolation in depth
• CRS alignment strategies — the projection gate every tier’s output must clear before bounds checks
• Zoning taxonomy mapping — translates legacy codes from fallback sources to current district standards
• Schema validation & data quality checks — asserts a fallback payload still satisfies the canonical contract
• Error handling & retry logic — the transport-layer backoff that runs before a tier is demoted
• Municipal API rate limit management — tuning the 429 boundary that triggers fallback

Up: Municipal Zoning Data Architecture & Compliance Frameworks