nameparser 2.0 implementation by derek73 · Pull Request #288 · derek73/python-nameparser

derek73 · 2026-07-12T21:22:44Z

Implementation branch for the 2.0 design — see the RFC in #285 and the umbrella issue #284 for the design discussion and feedback questions. Implementation learnings that change the design will land as amendment commits on #285, per the plan described there.

⚠️ Merges only at 2.0.0. This branch raises the Python floor to >=3.11 (#257) and drops the typing_extensions conditional dependency (nameparser now has zero runtime dependencies). Master must remain able to cut 1.4.x patch releases for Python 3.10 users, so this PR stays a draft until the 2.0.0 release.

Progress

Notes on what's here so far

Exception taxonomy (recorded in AGENTS.md): wrong type → TypeError, well-typed bad value → ValueError, enum lookups stay ValueError per stdlib precedent.
Import layering is test-enforced (tests/v2/test_layering.py), and the seven-field canonical order derives from Role declaration order everywhere — no ordering is ever stated twice.
Vocabulary is single-sourced from the nameparser/config data modules through 2.x, including the new maiden_markers.py (Recognize maiden-name markers like "née" and "geb." (Jane Smith née Jones) #274 — marker words like "née"/"geb."; decisions recorded on the issue).
Progresses Raise the Python floor to 3.11 in 2.0; drop the last dependency #257, Support family-first name order via an order-spec config (name_order) #270 (order-spec constants), Recognize maiden-name markers like "née" and "geb." (Jane Smith née Jones) #274 (vocabulary; consuming rule comes with the pipeline). Filed from this work: Recognize multi-word credential suffixes after a comma ("Smith, LEED AP") #291 (multi-word credential suffixes on the comma path — 2.0-or-2.1 decision pending).
Full suite: 2,395 tests plus 177 Sphinx doctests, mypy strict-profile clean, all green under filterwarnings = ["error"].
Commit history is themed and bisectable; each stage of the work went through multi-agent code review with fixes applied before landing.

🤖 Generated with Claude Code

codecov-commenter · 2026-07-12T21:24:02Z

Codecov Report

❌ Patch coverage is 98.20759% with 43 lines in your changes missing coverage. Please review.
✅ Project coverage is 98.19%. Comparing base (7c4f9ac) to head (ac59a5f).
⚠️ Report is 320 commits behind head on master.

Files with missing lines	Patch %	Lines
nameparser/__main__.py	0.00%	28 Missing ⚠️
nameparser/_config_shim.py	97.89%	9 Missing ⚠️
nameparser/_facade.py	99.43%	2 Missing ⚠️
nameparser/_lexicon.py	98.92%	2 Missing ⚠️
nameparser/_parser.py	98.82%	1 Missing ⚠️
nameparser/_pipeline/_tokenize.py	98.24%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #288      +/-   ##
==========================================
+ Coverage   97.67%   98.19%   +0.51%     
==========================================
  Files          13       37      +24     
  Lines         947     2434    +1487     
==========================================
+ Hits          925     2390    +1465     
- Misses         22       44      +22

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Every matrix job was silently testing CPython 3.11: uv sync honors .python-version (pinned 3.11 by the floor bump) over setup-python's interpreter, so 'build (3.14)' ran the suite on 3.11.15 (verified in the PR #288 run logs). UV_PYTHON makes the matrix real. With that fixed, add 3.15 (final 2026-10-01) as a non-blocking pre-release job so interpreter breakage surfaces while 2.0 is being built; flip it to blocking when 3.15 goes final. Locally green today on 3.15.0b3: pytest, mypy, ruff. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Ports the emoji/bidi character classes verbatim from config/regexes.py per the layering contract's duplication-by-design note; whitespace, emoji, bidi marks, and comma chars all act as separators without ever entering a token, so token spans always index original exactly. Wires tokenize into STAGES and registers the new module in test_layering's _MUST_EXIST. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Adds _vocab.py (is_initial/is_suffix_strict/is_suffix_lenient, shared across stages) and _segment.py (the family-comma/suffix-comma/no-comma decision). segment reads Lexicon suffix vocabulary directly rather than waiting for classify (recorded plan deviation #3: the suffix-comma vs family-comma call is definitionally vocabulary-dependent, mirroring v1's are_suffixes_after_comma). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Ports v1's join_on_conjunctions, prefix chains, and _join_bound_first_name as pure index operations over WorkToken runs (the anti-#100 invariant: no value-based lookup, no string joins). Adds two v2 additions from the plan: the "Ph. D."-split merge (plan deviation #1 -- lands in group since assign needs the merged piece to exist first) and the maiden-marker consuming rule (#274). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Completes the seven-stage STAGES tuple. Patronymic regexes verified byte-for-byte against nameparser/config/regexes.py's east_slavic/turkic entries -- no mismatch found. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Not a pipeline stage but the runner's tail: converts the seven-stage fold's ParseState into a public ParsedName, dropping structural marker tokens (e.g. maiden markers) and materializing PendingAmbiguity indices into real Ambiguity objects over the final token tuple. Also adds _pipeline/_assemble.py to test_layering's _MUST_EXIST now that the file exists. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Adds tests/v2/cases.py (Case dataclass + CASES tuple) and tests/v2/test_cases.py (the core runner), seeded with the v1 parity battery pinned in the plan header plus three carried-forward rows verified live against v1.4.0: - suffix_stays_suffix / suffix_stays_suffix_title: v1 routes a lone trailing recognized suffix (PhD) to the family/last field; v2 keeps it in suffix. Classified fix(suffix-routing). - family_comma_lone_title: v1 puts the pre-comma piece in first when it's followed only by a title; v2 treats pre-comma as definitionally family. Classified fix(comma-family). All 35 case rows pass against the pipeline as built through Task 11 with no corrections needed to the plan's predicted values; the East Slavic/Turkic policy rows were cross-checked against v1's equivalent patronymic_name_order opt-in and confirmed as true parity. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The ParseState ownership map said roles belong to assign/post_rules exclusively, but group writes maiden roles -- an ownership contract wrong in one place teaches readers to distrust it everywhere. Fixed the map, noted post-group segments staleness and the sole-producer text invariant on WorkToken, documented PendingAmbiguity's dangling- index tolerance and Parser's None-default resolution, made assemble's role fallback say what it means, and pinned the whole ownership map mechanically: a new test folds the case corpus stage by stage and asserts each stage changes only the fields it owns. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

In the real data suffix_acronyms_ambiguous is a SUBSET of suffix_acronyms (v1 shape), so the plain membership test in classify and is_suffix_strict fired before the period gate could withhold vocab:suffix -- the gate was dead code for the overlap members ({'ed', 'jd'}), and 'Smith, Ed' silently dropped the given name into suffix. Both sites now exclude the ambiguous subset, Lexicon enforces the subset invariant like particles_ambiguous, the classify/vocab fixtures mirror the real subset shape instead of hiding the bug with disjoint sets, and three v1-verified case rows pin the parity. Found by cross-session PR review (C1/I1). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The group-stage merge reunited the two tokens as a piece, but the suffix view joins SUFFIX tokens with ', ' -- 'John Ph. D.' rendered as suffix 'Ph., D.' -- and a mid-name credential never reached the trailing suffix peel, landing in MIDDLE. Continuation tokens of a suffix-merged piece now carry a new stable 'joined' tag (spec's provenance-tag sketch anticipated it) which the suffix view uses to attach with a space; assign peels group-flagged suffix pieces at ANY position, matching v1's fix_phd which extracted the credential before parsing. Contract-test trigger added for the new stable tag; two v1-verified case rows pin the behavior. Found by cross-session PR review (I2/I3). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

v1's handle_non_first_name_prefix was unported: 'de la Vega' produced given='de' with no ambiguity ('de' is not particles_ambiguous, so PARTICLE_OR_GIVEN never fires -- a silent misassignment). Port it as post_rules rule 1b keyed on the parsed given exactly like v1: a lone given token tagged particle-but-not-ambiguous with middles or family present folds into FAMILY; 'Jean de Mesnil' (non-leading) and bare 'de' (degenerate guard) stay untouched, and ambiguous 'van Gogh' keeps the given reading. v1-verified case row added. Found by cross-session PR review (I4). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

middle_as_family was validated but consumed by no stage -- ported v1's handle_middle_name_as_last as post_rules rule 4 (runs after the patronymic rotations, comma or not, matching v1's post_process order; span order reproduces v1's prepend). lenient_comma_suffixes now gates segment's post-comma predicate (strict mode vetoes initial-shaped suffix words). extra_suffix_delimiters cannot be honored until token re-splitting lands with the migration work, so setting it warns instead of silently doing nothing. Found by cross-session PR review (I5). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

PR review I6/I7/I8/I9: - assemble omits an ambiguity whose referent tokens were ALL dropped (born-empty ambiguities like UNBALANCED_DELIMITER are kept -- they are token-independent by design) - Policy.name_order now accepts only GIVEN_FIRST / FAMILY_FIRST / FAMILY_FIRST_GIVEN_LAST: the unnamed permutations had no implemented assignment semantics and would silently misassign. Non-Role elements get the taxonomy's TypeError. - Span.__add__ comment no longer cites a nonexistent join stage; the real rationale is that no covering-span operation exists (anti-#100) - the stage ownership test now also pins token-level ownership: texts and spans fixed at tokenize, classify changes only tags, the role-assigning stages only roles (group also tags, for 'joined') - docstring polish: ParseState names the three ambiguity-recording stages; _types names the matches() call-time import; _assign's PARTICLE_OR_GIVEN wording is order-neutral Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

- T1: three-piece FAMILY_FIRST_GIVEN_LAST pins given-from-the-END semantics (not a rotation of FAMILY_FIRST) - T2: reverse-coverage property -- every input char lies in a token span, a masked delimited span, or is individually ignorable; no character silently vanishes - T3: case row for post-comma non-suffix extras ('Smith, John, Extra, Jr.' -> suffix 'Extra, Jr.' + COMMA_STRUCTURE; v1 parity pinned live) - T4: multiple unbalanced delimiters are each reported; the scan does not stop at the first unmatched opener - T6: digits join the Hypothesis stress alphabet Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

/simplify (all four reviewers converged on the first): the trickiest membership expression in the vocabulary layer -- unambiguous acronym OR suffix word, ambiguous acronym only with periods -- was spelled in full in both classify and is_suffix_strict, aligned only by comments. Drift would make the comma-structure decision disagree with piece classification for the same token. Now _vocab.suffix_as_written is the single source; classify and the strict/lenient predicates compose it (and normalize each token once instead of twice on the lenient path). The three-comma set was likewise defined in tokenize and extract under different names with no sync note; it moves to _state.COMMA_CHARS. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

/simplify: the piece predicates were typed for group's internal mutable lists, so every cross-stage call paid a triple list/set/tuple conversion (including O(n) token-tuple copies inside assign's peel loops and group's maiden scan). They now take Sequence/Set and the call-site churn is gone. Also: the leading-title peel was spelled twice (assign's main path and the FAMILY_COMMA branch, the latter with a dead 'not given_done' guard) -- extracted as _peel_leading_titles; the trailing all-suffix segment loops collapse into one loop over the structure's tail; the five hand-rolled pieces/ptags parallel-slice merges in _group_segment become one merge() closure that keeps the arrays in lockstep; post_rules' 'others' list was only ever tested for truthiness (any()); the patronymic comma gate is a named rotations_apply boolean instead of falsifying the policy value; and the module docstring records the GIVEN_FIRST positional assumption the rules inherit from v1 (#270). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

/simplify efficiency findings: with the default strip flags every non-space character paid two failing regex calls in _ignorable; both strip classes are entirely non-ASCII, so an isascii() fast path skips them for virtually all real input. Span is a NamedTuple and already sorts as a tuple, so the tuple(...) sort keys in tokenize/extract were pure allocation. The case runner's seven-field list now derives from Role, whose declaration order the type declares canonical -- a new Role member fails the runner instead of being silently skipped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Every Lexicon vocabulary field except given_name_titles is matched one word at a time, so a multi-word entry can never match -- the library itself shipped eight such dead entries for years (repaired in the previous commit). Per the inert-not-harmful convention a raise is forbidden (see the given_name_titles Gotcha in AGENTS.md); warn instead, from _normset (all per-word vocab fields) and _normpairs (capitalization_exceptions keys), through every entry path -- constructor, add(), union, unpickle re-validation, and the v1 shim's snapshot. Also fixes tests/v2/test_properties.py's shared _VOCAB hypothesis pool, which drew a multi-word phrase ("grand duke") into every Lexicon field even though its own comment says it's "legal only in given_name_titles" -- the fuzzer applied it everywhere, so it now tripped the new warning in every per-word field. Split it into a separate _TITLE_VOCAB used only for given_name_titles. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

… sweep Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…ribution Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…ntness Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Post-rc1 API polish: StrEnum Role, Parser conveniences, dead-vocabulary repair

rc2 is rc1 plus the post-rc1 API-polish bundle (PR #290): Role as a StrEnum, the Parser conveniences (matches/capitalized/revise and Policy.patched), the dead-vocabulary repair, the multi-word-entry warning, and the legacy-pickle guard. The release log's "Changed since 2.0.0rc1" section lists what an rc1 tester would notice. Verified: __version__ == "2.0.0rc2" (packaging confirms is_prerelease), the wheel/sdist carry it, and twine check passes on both. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Make unexpected warnings test failures

The override neutralized the parent suite's dual-run fixture by name; the parent was renamed to _isolate_constants when the dual-run was dropped for #255, so the override has matched nothing since -- a dead autouse no-op under a docstring claiming the parent's behavior was skipped. Deleted rather than reconnected: the docstring's premise (v2 code never reads shared CONSTANTS) stopped being true when the facade/shim tests arrived -- test_config_shim.py mutates the shared singleton, so the parent's snapshot/restore is wanted protection here. Suite counts identical before and after (2393 passed), confirming the fixture changed nothing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

setattr(hn, 'given', ...) has always succeeded silently while changing nothing about the parse -- the facade spells those attributes first/last, so the assignment just creates a stray instance attribute and the name silently forks. Role becoming a StrEnum added setattr(hn, Role.GIVEN, ...) as a second spelling of the same trap. Warn on exactly these two names and still perform the assignment: ad-hoc attribute stashing is a legal v1 pattern, the other five 2.0 field names are real properties whose setters work, and any code that worked keeps working. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The maiden-marker bullet listed 'born', which is not in the shipped vocabulary ('Jane Smith born Jones' does not route to maiden); the two #270 bullets each closed a 'half' without saying the halves add up; the #256 bullet claimed both mapping managers list known keys when only capitalization_exceptions does (the regexes proxy names just the miss). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

# Conflicts: # README.rst

derek73 self-assigned this Jul 12, 2026

derek73 added the enhancement label Jul 12, 2026

derek73 added this to the v2.0 milestone Jul 12, 2026

derek73 mentioned this pull request Jul 12, 2026

Add Python 3.15 to the test workflow #259

Open

derek73 and others added 12 commits July 12, 2026 21:35

Add extract_delimited: delimiter pairs to nickname/maiden spans

24c5191

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Add classify: vocabulary tags from the lexicon

a91e465

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Add assign: positional roles per name_order over pieces

5e22e78

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Add Parser and parse(): the 2.0 pipeline goes end to end

9bcd007

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Add parser_for() with locale-identified errors and matches()

8ad0247

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Add property, contract, and benchmark test layers

9347848

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

github-advanced-security AI found potential problems Jul 13, 2026

View reviewed changes

Comment thread nameparser/_pipeline/_tokenize.py Fixed

derek73 and others added 11 commits July 13, 2026 02:28

Add the shim SetManager (migration spec §3)

12298df

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

derek73 and others added 20 commits July 26, 2026 02:43

Export STABLE_TAGS and write its docstring for users

278adbd

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Make ParsedName.as_dict's include_empty keyword-only

c12c795

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Add Policy.patched() so a PolicyPatch is applicable without a Locale

4b80a10

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Add Parser.matches and Parser.capitalized for custom-config workflows

468561b

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Add Parser.revise, the tag-preserving field replacement

4a1dfbb

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Repair the eight dead multi-word vocabulary entries

a11d51e

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Accept Role members as HumanName subscript keys

90afe20

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Sweep AGENTS.md and docs for the API-critique changes

fbc81af

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Drop the eight legacy dead entries when restoring a pre-2.0 pickle

86157d6

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Polish from the final bundle review: rc1-delta notes, doc links, repr…

d023579

… sweep Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Ship the unaccented 'charge' title spelling, like attaché/attache

d556dae

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Harden the review seams: repr smuggle-proofing, tag docs, warning att…

76cc09b

…ribution Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Pin the review's test gaps: warning counts, role forcing, enum disjoi…

3bf981f

…ntness Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Show the replace() degradation and the revise() fix as runnable examples

20ca3bf

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Cross-reference the Parser convenience trio from the revise() section

fd43eac

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Teach the particle ambiguity with the Van Johnson / Van Buren pair

f5c74e7

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Say what replace() actually carries over, and what it deliberately drops

9d42aaa

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Merge pull request #290 from derek73/claude/v2-core-api-critique-91a960

17cc254

Post-rc1 API polish: StrEnum Role, Parser conveniences, dead-vocabulary repair

derek73 temporarily deployed to pypi July 26, 2026 20:41 — with GitHub Actions Inactive

derek73 and others added 6 commits July 26, 2026 14:03

Make unexpected warnings test failures

c12093b

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Merge pull request #292 from derek73/fix/pytest-filterwarnings-error

5ad5aa1

Make unexpected warnings test failures

Merge remote-tracking branch 'origin/master' into v2/core-foundation

ac59a5f

# Conflicts: # README.rst

derek73 marked this pull request as ready for review July 28, 2026 05:44

derek73 merged commit 6dc5261 into master Jul 28, 2026
7 checks passed

derek73 deleted the v2/core-foundation branch July 28, 2026 05:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

nameparser 2.0 implementation - #288

nameparser 2.0 implementation#288
derek73 merged 319 commits into
masterfrom
v2/core-foundation

derek73 commented Jul 12, 2026 •

edited

Loading

Uh oh!

codecov-commenter commented Jul 12, 2026 •

edited by codecov Bot

Loading

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

derek73 commented Jul 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Progress

Notes on what's here so far

Uh oh!

codecov-commenter commented Jul 12, 2026 • edited by codecov Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

derek73 commented Jul 12, 2026 •

edited

Loading

codecov-commenter commented Jul 12, 2026 •

edited by codecov Bot

Loading