[dexhunter]

TL;DR

• This PR makes the lexer faster when it builds name tokens, for example in parse(query, no_location=True).
• read_name now uses one compiled regex match instead of a per-character loop.
• create_token and Token.__init__ store their values directly, without temporary locals or tuple packing.
• test_parse_large_query workload, median wall time on one isolated core: 46.19 ms -> 41.43 ms, 10.3% faster.
• Tests: pytest tests gives 3821 passed, 18 skipped.

Refs #306

What & Why

Before this change, Lexer.read_name checked every character of a name in a Python loop. Each step called is_name_continue() and compared the position with the body length. Names are the most common tokens, so this loop used a large part of the parse time.

Now the lexer compiles the regex [0-9A-Z_a-z]* once per lexer and uses its match to find where each name ends. This set is exactly what is_name_continue() accepts: ASCII letters, digits and underscore. Thus every token keeps the same kind, start, end, line, column and value.

create_token and Token.__init__ compute the same values as before. They only skip temporary locals and tuple packing.

Results

Median of several runs with CPython 3.12 on one isolated CPU core:

Situation	Before	After	Change
large_query.graphql, no_location=True (as in test_parse_large_query)	46.19 ms	41.43 ms	10.3% faster
github_schema.graphql SDL, no_location=True	97.11 ms	91.48 ms	5.8% faster
kitchen_sink.graphql with locations	0.759 ms	0.723 ms	4.7% faster
Query with 300 long string values (few names)	16.44 ms	16.22 ms	no real change
Tiny query { hello }	22.4 µs	22.9 µs	within run-to-run spread
Introspection query on a built schema (no lexing in the loop)	102.80 ms	102.85 ms	no change

A wider lexer workload measured 271.1 ms before and 245.1 ms after (about 10% less time, three repeats). It parses the large query, the kitchen sink with locations and full token streams.

Why this is safe and easy to maintain

The change is small: about 20 lines in two files, and each changed part is easy to read. The regex accepts the same characters as is_name_continue(), so tokens and errors do not change. The lexer, parser and AST tests in tests/language cover this code.

Validation

• pytest tests: 3821 passed, 18 skipped.
• pytest tests/language: 409 passed, 1 skipped.
• ruff check and ruff format --check pass on the two changed files.
• Benchmark: pytest tests/benchmarks/test_serialization.py -k parse_large_query.

AI assistance

The code change was found with automated tooling: OpenAI Codex (gpt-6.1-sol) proposed the candidates, and Weco Observe tracked them. An automated agent picked the candidate, ran the tests and did the timing. No human review has happened yet.

Weco share link: https://dashboard.weco.ai/share/FRl_naVP6T86RC8_gUDY6lGUHpnbABkm

[codspeed]

Merging this PR will improve performance by 12.52%

⚡ 1 improved benchmark
✅ 19 untouched benchmarks

Performance Changes

	Benchmark	BASE	HEAD	Efficiency
⚡	test_parse_large_query	543 ms	482.6 ms	+12.52%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing dexhunter:perf/lexer-regex-name-scan (57404b9) with main (1d505c1)}