A fresh Layout1 is already ISO A3 landscape (ezdxf R2018 default: 420 x 297,
plot_rotation 0), so an A3-only /MediaBox read-back passed with
page_setup_apply stubbed to a no-op: the A/B gate row and the v4 matrix task
could not fail on the setter. Both now apply ANSI B (432 x 279, a size no
fresh document has; spec section 9 names it) and read that PDF first, then
apply ISO A3, read its PDF, and require the setter's changed.size_mm to have
moved from B to A3. The stub fails at the first PDF.
The moved-sheet comparison is +/-0.5 mm, the spec's tolerance, not exact:
measured live on AutoCAD 2026 (scratch Documents.Add(), closed unsaved), the
COM engine reports ANSI B as 431.8 x 279.4 and the DWG To PDF.pc3 file's
MediaBox as 431.8 x 279.4 then 420.158 x 297.039 for A3; the task passes
there with changed.size_mm [[431.8, 279.4], [420.0, 297.0]]. Headless it is
[[432.0, 279.0], [420.0, 297.0]].
tests/test_benchmark_v4.py gains the mutation test the finding describes
(setter stubbed -> both rows must fail; both passed on the old code) and a
run of the gate row itself; the headless task test asserts the B stage.
README rows and the tasks_v4 docstring say what the task now does; the README
collected-tests figure follows the two new tests (3282 -> 3284). The
regenerated A/B report against v1.5.1 is byte-identical (26 -> 32/32).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd
scripts/smoke_pid_com.py runs the whole track against the live AutoCAD in
a new document. ComBackend.connect() is lazy by design, so the script reads
reachability from system_status()["connected"] rather than from connect()
as the plan sketched, and closes its document without saving unless --keep
is passed. On this machine (AutoCAD 2026, "25.1s (LMS Tech)") it exits 0:
5 nodes, 4 edges, 0 dangling, confidence_min 1.0, 0 critique issues,
2 XDATA chunks — identical to the headless engine on the same spec. The
first attempt exited 2 on a stale win32com gen_py cache (a package folder
holding only __pycache__); an environment fault, no backend change needed.
A/B against v1.5.1 published as ab-v1.5.1-vs-v1.6.0-dev.json: 26/29 ->
29/29, three miss -> pass, zero regressed. check_doc_numbers.py now names
the report and the two row labels in one place (AB_REPORT, AB_*_LABEL),
and README.md / benchmarks/README.md carry the new rows; the v1.5.0 and
v1.4.0 tables stay as history.
README: P&ID row, the showcase and catalogue images, the token table
re-measured (161 / 44,930 by default, 152 / 41,420 with TOOL_PACKS=core,
lean 50 / 14,007), every stale count reconciled (154 -> 166 registered,
149 -> 161 advertised, corpus 158/672 -> 162/731 by the same count, 19 ->
20 groups, 6 -> 8 resources, six-task matrix with pid_roundtrip), a
"Known limitations of the P&ID track" subsection, and the snapshot line at
this machine's 2423 collected tests (CI has not run on this tree; noted in
the changelog with the pre-existing 1369 vs 1363 drift).
CLAUDE.md: section 17, sections 5 and 7 counts, TOOL_PACKS, the two 1.6
contracts and the reader rules. Spec status -> implemented. The plan and
spec markdown were reformatted by ruff (fenced Python only) because
`ruff format --check .` was already red on them at HEAD and the gate
requires the whole tree clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd
tasks_v4.py extends v3 with one task the reader can fail on its own: the
example spec is drawn through pid_from_spec's code and read back by the
graph builder, which never sees the spec. The runner, the chart and the
doc-number gate now default to v4; --matrix v3 / v2 reproduce the earlier
sets, and tests/test_benchmark_v3.py labels its full-matrix run "v3"
explicitly now that the default has moved.
The plan's adapter check expected the two unnumbered lines to come back
with no line number; draw_line builds one from the default number_format
(bare sequence when size/service/spec/insulation are empty), so the task
asserts the two authored numbers on the two process lines and sequence-
built XDATA numbers on the other two instead.
The published A/B report still covers 26 checks; Task 21 re-runs it
against v1.5.1, which is what clears the remaining doc-number drift.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd
Every one of these was reported ok/clean to the caller while being wrong.
Silently wrong numbers
- selection_filter(min_area) read `entity.dxf.get("area", 0.0)` for SOLID,
TRACE and HATCH, none of which carry that attribute, so every hatch
measured 0.0. A 1600 mm^2 hatch was excluded by min_area=1.0 while
`filtered_by` asserted the filter had run, and COM (which reads ActiveX
.Area) returned a different selection for the same call. It now answers
from the measurement `entity_measure` uses, so the two cannot disagree.
- Every headless dimension rendered through ezdxf's bare `Standard` style:
1.0 mm text with a comma decimal marker, against ISO 3098's 2.5 mm minimum
and ISO 129's point -- while the document header itself said $DIMTXT 2.5.
`ezdxf.new(setup=True)` is not the fix; measured, it renders a 100 mm
dimension as "10000". The attributes are now stored explicitly.
- Saving writes `dimrnd = 0.0` onto the Standard style, and ezdxf applies
`xround(value, dimrnd)` whenever the attribute is present at all --
`xround(33.333, 0.0)` is 33. So on any reopened file, including one this
server saved, every dimension was rounded to a whole unit before DIMDEC
formatted it: a 12.75 mm feature dimensioned 13. AutoCAD means DIMRND 0 as
*no* rounding; the sentinel is now discarded at every load site, and a real
0.5 mm rounding is left alone.
- The DIM* header fold gave out-of-range values teeth they did not have when
they died in the header: dim_decimals=-1 returned ok and then killed the
next dimension with `ValueError: Format specifier missing precision` from
inside ezdxf, and 40 printed forty digits of float noise onto the sheet.
Ranges are now AutoCAD's own (DIMDEC 0-8, DIMZIN 0-15, lengths positive).
Security
- security.py recorded the policy as "sanitize_command and sanitize_lisp each
have exactly ONE caller ... the policy is scoped to the CHANNEL". The
premise was false. `_ensure_linetype_loaded` and `ComBackend.linetype_load`
build `_-LINETYPE _LOAD {name} {file}` and hand it to SendCommand, which
treats a newline and ';' as segment separators -- so a `linetype=` argument
carrying either injected further AutoCAD commands, and the 36-verb denylist
never saw that channel. Guarded with the DXF symbol-name rule (the
injection characters fall out of it rather than being enumerated) plus
validate_path for the file argument, which was also the one path-taking
tool reaching the filesystem unchecked. The policy comment is corrected.
Gates that could not see
- drawing_critique returned zero issues on a quarantined document: all seven
checks wrapped their backend call in a bare `except Exception`. CLAUDE.md
rule 7 makes that call the gate an agent is told to trust. Only the
quarantine now escapes; an ordinary broken check still degrades.
`_check_dimension_legibility` also read `_doc` directly from the event loop
thread, missing the gate and performing exactly the unsynchronised access
the quarantine exists to prevent. `construction_clear` answered
"deleted: 0" -- there was nothing to clear -- for the same reason.
- The iso128 focus checked layer lineweights only, so a sheet lettered at
1.0 mm critiqued clean. It now reads the effective dimension height and
separator, header-then-style, in the order the renderer resolves.
- check_doc_numbers.py derived five values it never checked, and those were
the ones that had drifted. The collected-test count (README said 1274, the
tree had 1369) and the A/B pass-rate and Fixed columns are now gated.
- benchmarks/README.md's v1.4.0 table was impossible on its face -- 21
passing plus 5 fixed is 26, not 24 -- and contradicted the report it
linked, because `ab-v1.4.0-vs-v1.5.0.json` was regenerated from a tree that
already carried the dimension fixes and so never measured the v1.5.0 tag.
Replaced with a real run: ab-v1.4.0-vs-v1.5.1.json, 21/26 -> 26/26.
Release gate
- release.yml ran one ubuntu/3.11 lane, so a Windows-only defect could reach
PyPI with no Windows job having executed. That is not hypothetical: v1.5.0
shipped `except pywintypes.com_error` raising NameError wherever pywin32 is
absent. It now runs the same three lanes as ci.yml, Windows included.
1368 passed, 1 xfailed. ruff clean. Tool inventory and all 16 gated document
numbers match the tree.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The triage over the 43 unaddressed verdicts returned 4 code and 35 doc items.
The code ones shipped in e24b46d; these are the documents.
Twenty-one applied mechanically after a validation pass confirmed each locator
still matched exactly once and each replacement was supported by the tree. Ten
were held back because a re-verify pass found them mutually exclusive -- three
patches rewriting the same benchmarks/README block, three rewriting the same
OCS paragraph -- and applying them in sequence would have corrupted the files.
Those were resolved by hand into one coherent edit each. One was rejected.
The corrections worth naming:
- **The A/B table contradicted itself in three files.** benchmarks/README.md
and CLAUDE.md said 24 checks / 21-24 / 87.5% / "fixed 3" while the suite has
26 and the published JSON says 21/26, 80.8%, five fixed. All three now agree,
and the five are split into the three new-capability `miss -> pass` and the
two repaired-defect `fail -> pass`, which is the distinction the lane exists
to make.
- **"Every number below is produced by a script"** was false twice over:
section 5's nine scores are literals a human assigned, and section 4 now
quotes hand-collected timings. The claim is narrowed to the charts, which
really are all rendered from committed data, and both exceptions are named.
- **The OCS paragraph over-quantified.** TEXT rotation was given as the single
exception; z components and HATCH boundaries are also entity-frame, and the
live backend records `2d_polyline_coordinates_not_verified_against_live_autocad`
in the same reason string. It now points at `ocs_normalized` as the live list.
- **Transactions were shown as bare parity.** The server itself grades them
differently -- `FeatureCapability(True, "undo_mark")` on COM against
`(True, "snapshot")` headless -- and COM rollback sends `_UNDO B` without
confirming it landed. The row says so.
- **"Raw commands: Opt-in"** named a switch that does not exist. Both tools are
registered unconditionally; `DANGEROUS_COMMANDS_ENABLED` only disables the
sanitizer.
- **"Wave A shipped 13 of a planned 39"** hardened an approximation: the only
source writes `~39`.
- **CAD_PROGID** said nothing beyond the connection was tested against
BricsCAD/ZWCAD/GstarCAD, implying the connection was. It was not.
Perf lane reworked against a fresh paired measurement. The old text claimed
"28% slower than v1.4.0, median of three runs" and that
`EZDXF_CALL_TIMEOUT=0` restored v1.4.0's numbers "exactly". Three problems:
`perf_suite` runs each workload once per invocation and cannot produce a
median, "exactly" was refuted by the figures in its own sentence, and the
number moved when the timeout wrapper was rebuilt. It now reports the guard's
cost -- +20.8% and +19.9%, three paired runs, median taken by hand -- says the
figures have no published artifact, and says the wrapper rework recovered about
a third of the v1.5.0 overhead rather than closing it.
perf-ezdxf.json and the perf chart regenerated against the shipped code, and
the README table rebuilt from that artifact so the two cannot drift.
1267 passed, 1 xfailed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Tool count 131 -> 154; 474 -> 1242 collected tests.
The release exists for two measured reasons. A 149-tool catalog costs a client
~40,300 tokens before it has asked for anything, and a drafter searching the way
drafters think -- FILLET, BPOLY, QSELECT, WBLOCK -- scored df=0 against the stock
index, because those words were simply not in the data.
Discovery and batching
- DISCOVERY_MODE=search replaces the catalog with search_tools + call_tool,
ranked over an AutoCAD command and synonym corpus (discovery/aliases.py) on
top of fastmcp's BM25 index. Measured: 40,305 -> 356 tokens advertised.
- cad_batch runs a step list in one round trip; fields= projects result-heavy
responses. The `core` profile is gone -- discovery replaced it.
Honesty debt (Tier 0)
- OCS/ECS: `extrusion` appeared nowhere in this codebase, so mirrored and
rotated geometry reported coordinates that were quietly wrong. backends/ocs.py
translates at the backend boundary; every tool coordinate is WCS on both
engines, and a tilted plane refuses rather than answering with a number that
looks fine.
- drawing_save_as(fmt="dwg") wrote DXF bytes to a .dwg path. drawing_audit never
repaired. analysis_measure_area shoelaced points the caller typed in.
- ezdxf calls gained a per-call timeout, so one hung call can no longer wedge a
server whose document lock is a single asyncio.Lock.
Architecture
- The 1854-line backend ABC is 19 contract modules; @capability(key, reason=)
makes optional methods declarative and a gate fails on undeclared holes.
- 8 layout/viewport tools, CHSPACE, CAD_PROGID escape hatch.
- Wave A: selection filters, boundary tracing, hatch depth, annotation objects,
measurement -- 13 tools of a planned 39. The REGION and 2D-boolean family was
cut on evidence: add_region() yields zero ACIS bytes and the greiner_hormann
substitute loses 28.2% of the area on a square with one semicircular edge.
Verified against a live AutoCAD 2026, which found four COM defects no amount of
reading would have caught -- AcadPViewport has no ViewCenter, and the line that
set it had been swallowed by a bare except since v1.4.
Benchmark v3 (15 tasks) and what it caught
- v2 scored this server 10/10, which carried no information: every task in it
exercised something the server was built around. Five tasks were added because
they can fail, and three did while being written -- a hatch could not be
measured though the refusal listed HATCH as measurable, the boundary tools and
the measure tool disagreed about the same shape (139.2177 vs 139.2699), and a
circle stored as two bulged vertices measured 0.0.
- Competitor reports are NOT re-scored. They are pinned v1.4 runs against v2 and
were never asked these five questions; the chart shows those rows `not run`.
- Correctness A/B: 21/24 -> 24/24, zero regressions.
Performance, stated plainly
The published v1.4 perf report was recorded on CPython 3.14 and the new one on
3.11.15, so the apparent 2-6x speedup is the interpreter, not this release.
Re-measured on one interpreter, v1.5.0 is 28% slower creating entities and 22%
slower on the 10k roundtrip. Fully attributed: EZDXF_CALL_TIMEOUT=0 restores
v1.4.0's exact numbers. That is the deadlock fix being paid for, the knob is
documented, and winning it back is on the 1.6 roadmap.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
source_review.json extended with three projects reviewed 2026-07-23 (LokmenoWer/best-cad-mcp B/74, daobataotie/CAD-MCP B/35, zh19980811/Easy-MCP-AutoCad C/30) and our entry updated to the released v1.4.0 (95: CI matrix, PyPI packaging, A/B gate, paper space, ISO 286, opt-in solids). Chart regenerated via render_chart; README benchmark section now opens with the leaderboard + 9-row table, source-review lane merged into it.
Co-authored-by: Cursor <cursoragent@cursor.com>
Ran benchmarks/compare_versions.py against the v1.3.0 tag: 21/21 on both versions, 0 fixed / 0 regressed - the v1.4.0 feature+infra work introduced no correctness regressions. Main README gains lane 4 (version A/B, with the historical v1.0.0 -> v1.1.0 contrast); benchmarks/README records the release-gate result and drops the stale 'performance coming' stub.
Co-authored-by: Cursor <cursoragent@cursor.com>
- benchmarks/correctness_suite.py: 21 deterministic headless (ezdxf) checks
across core ops, dimensions, arrays, geometry, query/parity, render, the
quality gate, and engineering.
- benchmarks/compare_versions.py: one-command A/B runner — checks out an older
ref into a throwaway worktree and runs the same suite against both, each check
in its own subprocess so a crash is a miss, not a run-killer.
- benchmarks/README.md: methodology, results table, honesty caveats.
- README: Benchmarks section now reports the measured result —
v1.0.0 8/21 (38.1%) -> v1.1.0 21/21 (100%); 13 fixed, 0 regressions.
