Matching permit addresses to parcels with fuzzy and spatial joins

You have 14 000 building permits with addresses and 41 000 parcels with their own addresses, and you need them joined. A naive equality join on the raw strings matches about 60% of them. Lowering that to a fuzzy comparison lifts the number to 97% and quietly attaches several hundred permits to the wrong lot, because 123 N Main St and 123 S Main St differ by one character and are half a mile apart. This guide is the mechanics of doing it properly — the blocking, the scoring, and the spatial confirmation — as the hands-on companion to address geocoding & parcel matching, which sets out the two-phase design and the confidence tiers this code produces.

Diagnosis: why the naive join fails in both directions

Run the equality join first, on normalised strings, and look at what falls out. The residue splits into four groups and each needs different treatment.

Formatting-only differences — 123 NORTH MAIN STREET against 123 N MAIN ST — are the easy majority, and they are what normalisation exists for. If these dominate, fix normalisation rather than reaching for a similarity score.

Six address pairs, and what each scoring approach concludes A six-row by three-column matrix of address pairs. Rows are pairs that differ only in formatting, pairs differing by directional, by house number, by street suffix, by one letter in the street name, and a pair where one side omits the suffix. Columns are what whole-string token similarity concludes, what component scoring with hard constraints concludes, and which answer is correct. Token similarity matches all six, including the four that are different addresses. Component scoring matches only the two that should match. Six address pairs, and what each scoring approach concludes whole-string token similarity component scoring with constraints correct answer 123 NORTH MAIN STREET / 123 N Main St match (1.00) match match 123 N Main St / 123 S Main St match (0.96) reject — directional reject 123 Main St / 1123 Main St match (0.94) reject — number reject 123 Main St / 123 Main Ave match (0.93) reject — suffix reject 123 Main St / 123 Maine St match (0.95) reject — street reject 123 N Main / 123 N Main St match (0.90) match match correct verdict right by luck wrong verdict
Token similarity says yes to all six, and four of those are different addresses that happen to differ by one token. Component scoring with the number and directional as hard constraints reaches the correct verdict on every row — which is the whole reason the score is not computed over the raw string.

Genuinely different addresses that score highly. 123 N MAIN ST against 123 S MAIN ST scores 0.96 on token similarity. So does 1123 MAIN ST against 123 MAIN ST. These are the reason a fuzzy join alone is unsafe: the pairs most likely to be confused are the pairs whose difference is one token or one digit.

Addresses absent from the parcel layer — new plats, private drives, addresses assigned but not yet published. No amount of scoring finds a match that does not exist, and forcing one is worse than reporting none.

One address, many parcels — condominium plats and address ranges. This is not a scoring problem at all.

def triage(permits, parcels):
    """Count the residue by cause before choosing a strategy."""
    pkeys = set(parcels["addr_key"])
    exact = sum(1 for p in permits if p.addr_key in pkeys)
    same_number_diff_dir = sum(
        1 for p in permits
        if p.addr_key not in pkeys
        and any(k for k in pkeys if _same_but_directional(p.addr_key, k)))
    return {"exact": exact,
            "directional_risk": same_number_diff_dir,
            "residue": len(permits) - exact}

If directional_risk is non-trivial — and in any grid-planned city it is — the scoring function has to treat the directional as structure rather than as characters.

Step-by-step implementation

1. Parse into components, do not score whole strings 

import re
from dataclasses import dataclass

SUFFIXES = {"STREET": "ST", "AVENUE": "AVE", "ROAD": "RD", "DRIVE": "DR",
            "BOULEVARD": "BLVD", "LANE": "LN", "COURT": "CT", "PLACE": "PL",
            "TERRACE": "TER", "CIRCLE": "CIR", "PARKWAY": "PKWY", "HIGHWAY": "HWY"}
DIRECTIONALS = {"NORTH": "N", "SOUTH": "S", "EAST": "E", "WEST": "W",
                "NORTHEAST": "NE", "NORTHWEST": "NW", "SOUTHEAST": "SE",
                "SOUTHWEST": "SW"}
UNIT_TOKENS = ("APT", "UNIT", "STE", "SUITE", "#", "BLDG", "FL", "RM")


@dataclass(frozen=True)
class Address:
    number: str          # "123"; kept as text because "123B" is a real house number
    predir: str          # "N"
    street: str          # "MAIN"
    suffix: str          # "ST"
    postdir: str         # rare, but real in some grids
    unit: str            # "4B", kept OUT of the street key

    @property
    def street_key(self) -> str:
        """The key a parcel can be matched on. Unit deliberately excluded so a
        140-unit tower normalises to one address rather than 140."""
        return " ".join(t for t in (self.number, self.predir, self.street,
                                    self.suffix, self.postdir) if t)


def parse(raw: str) -> Address:
    s = " ".join(raw.upper().replace(".", " ").replace(",", " ").split())

    unit = ""
    for token in UNIT_TOKENS:
        m = re.search(rf"(?:^|\s){re.escape(token)}\s*([A-Z0-9-]+)", s)
        if m:
            unit = m.group(1)
            s = (s[:m.start()] + " " + s[m.end():]).strip()
            break

    parts = s.split()
    number = parts.pop(0) if parts and re.match(r"^\d+[A-Z]?(-\d+)?$", parts[0]) else ""

    predir = ""
    if parts and parts[0] in DIRECTIONALS:
        predir = DIRECTIONALS[parts.pop(0)]
    elif parts and parts[0] in DIRECTIONALS.values():
        predir = parts.pop(0)

    postdir = ""
    if parts and (parts[-1] in DIRECTIONALS or parts[-1] in DIRECTIONALS.values()):
        last = parts.pop()
        postdir = DIRECTIONALS.get(last, last)

    suffix = ""
    if parts:
        last = parts[-1]
        if last in SUFFIXES or last in SUFFIXES.values():
            suffix = SUFFIXES.get(last, last)
            parts.pop()

    return Address(number, predir, " ".join(parts), suffix, postdir, unit)

2. Block before you score

Comparing 14 000 permits against 41 000 parcels is 574 million pairs. Blocking on the house number plus the street’s first character reduces that to a few hundred candidates per permit, and it costs nothing in recall because a match must share the house number anyway.

Candidate pairs to score, by blocking key Lollipop chart of the number of candidate pairs that must be scored for 14 000 permits against 41 000 parcels under three blocking strategies. With no blocking there are 574 million pairs. Blocking on the house number leaves 168 thousand. Blocking on the house number plus the first letter of the street name leaves 42 thousand, a reduction of four orders of magnitude, with no loss of recall because a genuine match agrees on both components by definition. Candidate pairs to score, by blocking key 0 200000000 400000000 600000000 800000000 1 M pairs — above this, scoring dominates the run no blocking 574000000 house number 168000 house number + street initial 42000 Candidate pairs to score
Four orders of magnitude, at no cost in recall: a genuine match agrees on the house number and the street's first letter by definition, so nothing correct is excluded. Blocking on the full street name would be faster still and unsafe, because that is the component the fuzzy score exists to compare.
from collections import defaultdict


def build_blocks(parcels):
    blocks = defaultdict(list)
    for pid, addr in parcels:
        blocks[(addr.number, addr.street[:1])].append((pid, addr))
    return blocks

3. Score components, with hard constraints on the structural ones 

from rapidfuzz import fuzz

HARD_MISMATCH = -1.0


def score(a: Address, b: Address) -> float:
    """Component scoring. The number and directional are CONSTRAINTS, not
    contributions: differing on either means these are different addresses,
    however similar the strings look."""
    if a.number != b.number:
        return HARD_MISMATCH
    if a.predir and b.predir and a.predir != b.predir:
        return HARD_MISMATCH          # 123 N Main vs 123 S Main — never a match
    if a.postdir and b.postdir and a.postdir != b.postdir:
        return HARD_MISMATCH

    street = fuzz.token_sort_ratio(a.street, b.street) / 100.0
    if street < 0.80:
        return HARD_MISMATCH          # a different street entirely

    # Suffix disagreement is a real signal (MAIN ST and MAIN AVE can both exist)
    # but a missing suffix on one side is common and should not be punished.
    if a.suffix and b.suffix:
        suffix = 1.0 if a.suffix == b.suffix else 0.4
    else:
        suffix = 0.9

    # A missing directional on one side is ambiguity, not disagreement.
    directional = 1.0 if a.predir == b.predir else 0.75

    return round(0.65 * street + 0.20 * suffix + 0.15 * directional, 4)

Making the number and directional hard constraints rather than weighted terms is the difference between a join that is 97% right and one that is defensible. A weighted score can always be dragged over a threshold by strong agreement elsewhere; a constraint cannot.

4. Confirm spatially, then tie-break deliberately 

from shapely.geometry import Point

ACCEPT = 0.92
MAX_CENTROID_M = 400.0


def match(permit, blocks, parcels_gdf, geocoder):
    a = parse(permit.address)
    candidates = [(pid, score(a, b)) for pid, b in blocks.get((a.number, a.street[:1]), [])]
    candidates = [(pid, s) for pid, s in candidates if s >= ACCEPT]

    if not candidates:
        return Match(None, "unmatched", evidence="no candidate above threshold")

    if len(candidates) > 1:
        best = max(s for _p, s in candidates)
        tied = [pid for pid, s in candidates if s >= best - 1e-9]
        if len(tied) > 1:
            # Do NOT pick one. Several parcels legitimately share this address —
            # a condominium plat, a range, a split parcel — and the consuming
            # question decides what to do, not the matcher.
            return Match(None, "ambiguous", evidence=f"{len(tied)} tied candidates",
                         candidates=tied)
        candidates = [(tied[0], best)]

    pid, s = candidates[0]
    geo = geocoder.geocode(permit.address)
    if geo is None:
        return Match(pid, "probable", evidence="text only, no geocode", text_score=s)

    parcel = parcels_gdf.loc[pid]
    pt = Point(geo.x, geo.y)
    distance = float(parcel.geometry.centroid.distance(pt))

    if distance > MAX_CENTROID_M:
        return Match(pid, "ambiguous", evidence=f"geocode {distance:.0f} m away",
                     text_score=s, distance_m=distance)

    inside = bool(parcel.geometry.contains(pt))
    if inside and geo.kind in ("address_point", "rooftop"):
        return Match(pid, "confirmed", evidence=f"text+{geo.kind}|inside",
                     text_score=s, distance_m=distance)
    return Match(pid, "probable", evidence=f"text+{geo.kind}|{'inside' if inside else 'near'}",
                 text_score=s, distance_m=distance)

Verification & testing

The tests that matter are the near-miss pairs, because those are the ones a similarity score gets wrong.

@pytest.mark.parametrize("left,right", [
    ("123 N Main St", "123 S Main St"),      # directional differs
    ("123 Main St",   "1123 Main St"),       # number differs
    ("123 Main St",   "123 Main Ave"),       # suffix differs, both streets exist
    ("123 Main St",   "123 Maine St"),       # street differs by one letter
])
def test_near_misses_never_match(left, right):
    assert score(parse(left), parse(right)) < ACCEPT


@pytest.mark.parametrize("left,right", [
    ("123 NORTH MAIN STREET", "123 N Main St"),
    ("123 n. main st.",       "123 N MAIN ST"),
    ("123 N Main St Apt 4B",  "123 N Main St"),
    ("123 N Main",            "123 N Main St"),     # suffix missing on one side
])
def test_formatting_variants_match(left, right):
    assert score(parse(left), parse(right)) >= ACCEPT


def test_tied_candidates_are_ambiguous_not_arbitrary():
    m = match(permit("500 Tower Plaza"), blocks_with_140_units(), gdf, geocoder)
    assert m.tier == "ambiguous"
    assert len(m.candidates) == 140

The parametrised near-miss test is the one to keep permanently. Every future change to normalisation or scoring will be motivated by raising the match rate, and this test is what stops that change from raising it by matching N to S.

Failure recovery

Permits already attached to the wrong parcel. Re-run the matcher with the constraints in place, diff the new assignments against the old, and treat every changed assignment as a correction that needs re-deriving downstream. The changed set is usually small and concentrated on one street pattern, which is itself diagnostic.

Tier mix before and after the hard constraints Stacked bar chart of match outcomes for 14 000 permits under two scoring approaches. Fuzzy scoring alone reports 13 580 matched and 420 unmatched, of which 610 of the matches are later found to be wrong. Component scoring with hard constraints reports 11 940 confirmed, 1 190 probable, 520 ambiguous and 350 unmatched, with 12 wrong. The headline match rate falls while the number of wrong matches falls by fifty times. Tier mix before and after the hard constraints 0 5000 10000 15000 20000 12970 fuzzy only 1710 11928 constraints + spatial Permits (14 000) wrong (found later) unmatched ambiguous / probable confirmed Wrong-match counts come from a 500-record hand-checked reconciliation sample.
The match rate falls from 97% to 94% and the number of wrong matches falls from 610 to 12. That is the trade this page argues for, and it is why the report has to show tiers rather than a single rate — otherwise the correct change reads as a regression.

A drop in match rate after adding constraints. Expected, and correct. The lost matches were the wrong ones. Report the four tiers rather than the rate so the change reads as evidence replacing guesses rather than as a regression.

Blocking dropped genuine matches. If a source writes house numbers with a suffix on one side only (123 against 123B), the block key differs and the pair is never compared. Widen the block to the numeric prefix of the house number rather than lowering the score threshold — the fix belongs in candidate generation, not in scoring.

Frequently asked questions

Why treat the house number and directional as constraints instead of scoring them?

Because the pairs most likely to be confused differ only in those components, and a weighted score can always be dragged over the threshold by strong agreement on the street name. 123 N Main St and 123 S Main St agree on everything except the one token that puts them half a mile apart. A constraint cannot be outvoted; a weight can.

Should the unit number be part of the match key?

No. Keeping the unit in the key turns a 140-unit tower into 140 distinct addresses, none of which match the single parcel address, so the whole building goes unmatched. Strip the unit into its own field, match on the street key, and let the consuming question decide whether unit-level detail matters — it does for counting permits and does not for a zoning determination.

Is blocking safe, or does it lose matches?

Safe if the block key uses only components a match must share. Blocking on the house number plus the street’s first letter is safe because a genuine match agrees on both by definition. Blocking on the full street name is not safe, because that is exactly the component the fuzzy score exists to compare.

What threshold should the accept score be?

High — 0.92 in the code above — because the score only runs after the hard constraints have already eliminated the dangerous pairs. Once the number, directional and street similarity are constrained, the remaining score is measuring formatting noise, and formatting noise scores well above 0.92. A low threshold is a symptom of doing the constraints as weights.