The evidence for track F, in the shape tracks A, E and B + G used.
- benchmarks/tasks_v6.py: 19 tasks, DEFAULT_MATRIX = "v6"; v5 to v2 stay
addressable. arch_roundtrip draws the example plan with
arch_plan_from_spec and reads it back without the spec: rooms_detect must
find each labelled room within 0.1 % of its label and of the net floor
computed by hand (4825 x 5750 and 3825 x 5750), the schedules must list
D1, D2, W1, W2, and the plan is gated on a zero-issue critique and a
finalize score of at least 90. A test removes one room label and the gate
fails naming arch_room_unlabelled.
- Four correctness checks, every number worked out by hand in the docstring:
L / T / X junction corners and wall areas, a 4 x 5 m room labelled 20.00 m2
rather than its 21.84 m2 axis area, a door interrupting both faces, and a
plan of plain lines read as 20 and 15 m2 at confidence 0.6. The published
A/B report is re-run over 43 checks: 26 / 43 -> 43 / 43, zero regressed.
- scripts/smoke_arch_com.py runs the whole plan once on a live seat and
closes every document it opens; the CHANGELOG records what it returned.
- The token suite measures a TOOL_PACKS=core,arch row; check_doc_numbers
derives the task count from the v6 matrix; CLAUDE.md, both READMEs and
the spec status line say what shipped.
- fix(com): the first live smoke exited 1 - the room reader found 0 wall
segments. Measured on AutoCAD 2026 (2026-09-23): with a makepy cache,
ModelSpace.Item(i) returns the gen_py IAcadEntity wrapper
(type(item).__name__ == "IAcadEntity") and item.StartPoint on an
AcDbLine raises AttributeError, so every entity_list row lost its
type-specific properties (28 wall LINEs, 0 with start/end). _entity_info
now re-dispatches exactly that wrapper through _com_unnarrow; after the
fix 38 of 53 rows carry start/end and the smoke exits 0 with both rooms
at 27 743 750 / 21 993 750 mm2 and 0 critique issues. Re-dispatch
measured at ~1.7 ms per entity (dynamic vs gencache alike).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd
The evidence for tracks B and G, in the shape tracks A and E used.
- benchmarks/tasks_v5.py: 18 tasks, DEFAULT_MATRIX = "v5"; v4, v3 and v2 stay
addressable so an earlier report reproduces exactly rather than being only
described. mech_assembly is roadmap criterion 3 - a flange-coupling sheet
built from the part model and the sheet standard, through the code the
tools call (apply_titleblock, draw_part/add_view/dimension_part,
insert_std_part, extract_records -> draw_bom_table, add_balloon), gated on
two numbers that cannot be negotiated: drawing_critique(focus=None) returns
zero issues and the finalize score is at least 90 (it scores 100, 18/18 on
the v5 matrix). The adapter returns the focuses that fired, so a
regression names itself, and a test drops one balloon to prove the gate
fails and names mech_bom_balloon_mismatch.
- Seven correctness checks (32 -> 39). mech_section_hatch_area is the one
that cannot be faked: a sleeve of length 60, outside diameter 40 and bore
20 cuts two faces totalling 1200 mm^2 exactly, read back out of the drawing
through entity_measure. sheet_frame_iso5457_a3 pins the A3 frame (20,10 to
410,287 inside the 420 x 297 trimmed sheet); mech_iso286_on_dimension pins
40 H7 at +0.025/0 through the layout; std_part_iso4014_m12 pins s = 18 and
k = 7.5 by value; mech_thread_unrepresented_is_caught is the seventh: a
quality gate that never fires is not a gate.
- scripts/smoke_mech_com.py executed every new COM path once on AutoCAD 2026
(25.1s): frame, title block, a sectioned shaft with 12 dimensions, an ISO
4014 block with 1 ACADMCP_MECH chunk, a weld symbol, the parts list read off
the drawing, a linked balloon, an xref attached / listed / detached, a
31687-byte .dwg (AC1032) through ActiveX SaveAs, critique 0 issues, exit 0,
every scratch document closed. It retries RPC_E_CALL_REJECTED (measured:
the first call after Documents.Add is refused) and tracks its documents by
the document list, since SaveAs renames them.
- Defects the evidence found, each fixed with a test that fails on the old
code:
* ezdxf dimension_diameter put the text beyond the SECOND chord point;
ActiveX AddDimDiametric puts it beyond the first (measured live: a d50
from (x,170) to (x,220), leader 10, text at (x,160)). Headlessly every
front-view diameter stood on its own chain dimension. ezdxf now follows
ActiveX.
* draw_part's anchor goes on MECH; ActiveX refuses entity.Layer on a
missing layer (measured -0x7ffdfff7 'Key not found'). draw.py ensures
every target layer before the first entity.
* bom.extract_records filtered on INSERT; ActiveX names it BLOCKREFERENCE
(ObjectName AcDbBlockReference) - the live parts list was empty.
* COM _entity_info reported no dimension text_position: ModelSpace.Item
narrows to IAcadEntity, which has no TextPosition (measured
AttributeError); read through _com_unnarrow (TextPosition measured on
AcDbDiametricDimension, WCS). dim_overlap compared bbox centres and a
diameter collided with its own bore.
* pid graph raised KeyError 'start' mid-critique on a LINE whose point read
AutoCAD refused while busy; such a line is left out.
* the validator read "ISO 4014 - M12x60" as fake dimension text.
- The published A/B report is re-run over the grown suite rather than
re-typed (26/39 -> 39/39, 13 fixed, 0 regressed), check_doc_numbers derives
the task count from the v5 matrix, the token table is remeasured with a
TOOL_PACKS=core,mech row (193 tools / 61,673 tokens), and the README's
collected-tests figure is retaken (3990).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd
A fresh Layout1 is already ISO A3 landscape (ezdxf R2018 default: 420 x 297,
plot_rotation 0), so an A3-only /MediaBox read-back passed with
page_setup_apply stubbed to a no-op: the A/B gate row and the v4 matrix task
could not fail on the setter. Both now apply ANSI B (432 x 279, a size no
fresh document has; spec section 9 names it) and read that PDF first, then
apply ISO A3, read its PDF, and require the setter's changed.size_mm to have
moved from B to A3. The stub fails at the first PDF.
The moved-sheet comparison is +/-0.5 mm, the spec's tolerance, not exact:
measured live on AutoCAD 2026 (scratch Documents.Add(), closed unsaved), the
COM engine reports ANSI B as 431.8 x 279.4 and the DWG To PDF.pc3 file's
MediaBox as 431.8 x 279.4 then 420.158 x 297.039 for A3; the task passes
there with changed.size_mm [[431.8, 279.4], [420.0, 297.0]]. Headless it is
[[432.0, 279.0], [420.0, 297.0]].
tests/test_benchmark_v4.py gains the mutation test the finding describes
(setter stubbed -> both rows must fail; both passed on the old code) and a
run of the gate row itself; the headless task test asserts the B stage.
README rows and the tasks_v4 docstring say what the task now does; the README
collected-tests figure follows the two new tests (3282 -> 3284). The
regenerated A/B report against v1.5.1 is byte-identical (26 -> 32/32).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd
tasks_v4.py extends v3 with one task the reader can fail on its own: the
example spec is drawn through pid_from_spec's code and read back by the
graph builder, which never sees the spec. The runner, the chart and the
doc-number gate now default to v4; --matrix v3 / v2 reproduce the earlier
sets, and tests/test_benchmark_v3.py labels its full-matrix run "v3"
explicitly now that the default has moved.
The plan's adapter check expected the two unnumbered lines to come back
with no line number; draw_line builds one from the default number_format
(bare sequence when size/service/spec/insulation are empty), so the task
asserts the two authored numbers on the two process lines and sequence-
built XDATA numbers on the other two instead.
The published A/B report still covers 26 checks; Task 21 re-runs it
against v1.5.1, which is what clears the remaining doc-number drift.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd
`add_diameter_dim` / `add_radius_dim` measure to the point they are handed;
`leader_length` is a text placement. The ezdxf backend passed
`centre + (radius + leader)` as that point, so the leader became part of the
measurement: a true 40 mm bore was dimensioned 60 at the default
`leader_length=10`, and 108 at 34. Radius callouts were `leader_length` too
large the same way.
This is the worst class of defect for this project -- a wrong number, at
default settings, on the one part of a drawing that exists to carry numbers --
and it shipped in v1.4.0 and earlier. The size now comes from `radius` and
`location` only places the text.
The live COM backend was never affected: ActiveX's AddDimDiametric takes the
two chord points and the leader length as separate arguments. The two engines
had been disagreeing by 2 x leader_length on the same call.
Found while building the drawing for the new README, whose own bore callout was
wrong. Two checks added to correctness_suite.py; both report fail -> pass
against v1.4.0, not miss -> pass, which is what separates a repaired defect
from new capability in that lane. A/B is now 21/26 -> 26/26, five fixed, zero
regressed.
README rewritten from scratch
- Hero is a real ISO A3 sheet the server drew: m6 z24 involute spur gear, DIN
6885 keyed bore, section A-A, ISO 129 dimensions, ISO 286 H7 fit, ISO 7200
title block, 56 entities, drawing_critique clean. New
scripts/render_readme_showcase.py rebuilds it, and the tool-call sequence
that produces it is printed in the page.
- Star history chart, badge row, capability matrix, five evidence lanes, and a
"what this release is bad at" section carrying the 28% throughput regression.
- Every image and repo link is now an absolute URL: README.md is the PyPI long
description, where relative paths 404.
Corrections this work forced, all measured
- The df=0 example was wrong. FILLET is already in entity_fillet's own text;
the commands that genuinely scored zero are WBLOCK, PEDIT, OVERKILL,
LAYTRANS and MATCHPROP.
- 40,305 tokens is the tools/list wire payload; ~25,143 is the part forwarded
to the model. The old wording said "of schema", which is neither.
- `--tokenizer anthropic` needs a package and credentials this repo does not
lock, and refuses rather than falling back. Said so.
- Discovery ranks over the 149-tool advertised catalog, not the 154 registered.
- Five middlewares, not four; the fifth is what makes refusals machine-readable.
- Five of the 27 capability keys are environment-dependent, so the matrix
carries footnotes instead of bare ticks.
- "Every coordinate is WCS" has one deliberate exception: TEXT rotation.
- render_live_chart's axis still said "fixed 10-task matrix" while our bar came
from v3's fifteen. Each bar now carries its own matrix.
- .env.example said MAX_UNDO_STACK=50 (code: 5) and omitted four real
variables, two of them the remote-HTTP controls.
- server.json described an empty ALLOWED_PATHS as blocking access. It disables
the allowlist.
- server.py called the lean profile ~46 tools; it is 47.
- backends/capability.py cited a test file that does not exist.
1249 passed, 1 xfailed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Tool count 131 -> 154; 474 -> 1242 collected tests.
The release exists for two measured reasons. A 149-tool catalog costs a client
~40,300 tokens before it has asked for anything, and a drafter searching the way
drafters think -- FILLET, BPOLY, QSELECT, WBLOCK -- scored df=0 against the stock
index, because those words were simply not in the data.
Discovery and batching
- DISCOVERY_MODE=search replaces the catalog with search_tools + call_tool,
ranked over an AutoCAD command and synonym corpus (discovery/aliases.py) on
top of fastmcp's BM25 index. Measured: 40,305 -> 356 tokens advertised.
- cad_batch runs a step list in one round trip; fields= projects result-heavy
responses. The `core` profile is gone -- discovery replaced it.
Honesty debt (Tier 0)
- OCS/ECS: `extrusion` appeared nowhere in this codebase, so mirrored and
rotated geometry reported coordinates that were quietly wrong. backends/ocs.py
translates at the backend boundary; every tool coordinate is WCS on both
engines, and a tilted plane refuses rather than answering with a number that
looks fine.
- drawing_save_as(fmt="dwg") wrote DXF bytes to a .dwg path. drawing_audit never
repaired. analysis_measure_area shoelaced points the caller typed in.
- ezdxf calls gained a per-call timeout, so one hung call can no longer wedge a
server whose document lock is a single asyncio.Lock.
Architecture
- The 1854-line backend ABC is 19 contract modules; @capability(key, reason=)
makes optional methods declarative and a gate fails on undeclared holes.
- 8 layout/viewport tools, CHSPACE, CAD_PROGID escape hatch.
- Wave A: selection filters, boundary tracing, hatch depth, annotation objects,
measurement -- 13 tools of a planned 39. The REGION and 2D-boolean family was
cut on evidence: add_region() yields zero ACIS bytes and the greiner_hormann
substitute loses 28.2% of the area on a square with one semicircular edge.
Verified against a live AutoCAD 2026, which found four COM defects no amount of
reading would have caught -- AcadPViewport has no ViewCenter, and the line that
set it had been swallowed by a bare except since v1.4.
Benchmark v3 (15 tasks) and what it caught
- v2 scored this server 10/10, which carried no information: every task in it
exercised something the server was built around. Five tasks were added because
they can fail, and three did while being written -- a hatch could not be
measured though the refusal listed HATCH as measurable, the boundary tools and
the measure tool disagreed about the same shape (139.2177 vs 139.2699), and a
circle stored as two bulged vertices measured 0.0.
- Competitor reports are NOT re-scored. They are pinned v1.4 runs against v2 and
were never asked these five questions; the chart shows those rows `not run`.
- Correctness A/B: 21/24 -> 24/24, zero regressions.
Performance, stated plainly
The published v1.4 perf report was recorded on CPython 3.14 and the new one on
3.11.15, so the apparent 2-6x speedup is the interpreter, not this release.
Re-measured on one interpreter, v1.5.0 is 28% slower creating entities and 22%
slower on the 10k roundtrip. Fully attributed: EZDXF_CALL_TIMEOUT=0 restores
v1.4.0's exact numbers. That is the deadlock fix being paid for, the knob is
documented, and winning it back is on the 1.6 roadmap.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ruff format was never run repo-wide; CI now enforces 'ruff format --check', so normalize every module once. No behavior changes.
Co-authored-by: Cursor <cursoragent@cursor.com>
- benchmarks/correctness_suite.py: 21 deterministic headless (ezdxf) checks
across core ops, dimensions, arrays, geometry, query/parity, render, the
quality gate, and engineering.
- benchmarks/compare_versions.py: one-command A/B runner — checks out an older
ref into a throwaway worktree and runs the same suite against both, each check
in its own subprocess so a crash is a miss, not a run-killer.
- benchmarks/README.md: methodology, results table, honesty caveats.
- README: Benchmarks section now reports the measured result —
v1.0.0 8/21 (38.1%) -> v1.1.0 21/21 (100%); 13 fixed, 0 regressions.
