diff --git a/CHANGELOG.md b/CHANGELOG.md index 6141997..32a7d49 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [1.6.0] — 2026-09-25 + +The v1.6 roadmap, shipped as five tracks: 154 → 247 registered tools in 28 +groups, the correctness suite at 48 / 48 against v1.5.1's 26 / 48 with zero +regressions, the 21-task benchmark matrix at 21 / 21, and a live AutoCAD 2026 +run of every track's COM paths. Two roadmap criteria are not green, and +**Not in 1.6** below says which and why: track D (Plant 3D) was dropped, and +headless creation throughput is still below v1.4.0's. Roadmap: +`docs/superpowers/specs/2026-09-15-v1.6-roadmap-design.md`. + Track A of 1.6: the server becomes a P&ID expert and a P&ID reader. Nine `pid_*` tools in a new SECTION 17, two generic contracts every engine implements, six critique focuses that join `drawing_finalize`, and a @@ -255,6 +265,34 @@ member. Spec: `docs/superpowers/specs/2026-09-24-v1.6-understand-design.md`. ### Fixed +- **Found by the 1.6.0 README renders** (each with a test that fails on the + old code): + - The exterior dimension chains of `arch_dimension_chains` and + `arch_plan_from_spec` were drawn at the dimension style's DIMSCALE 1, so a + 1:50 chain's 2.5 text was 2.5 drawing units high — 0.05 mm on the sheet — + and the plan render showed empty boxes where the numbers belong. Every + other plan annotation already followed the plot scale, and now the chains + do too: DIMSCALE is the plan's scale while they are drawn (reported as + `dimscale`, replacing the old advice to set it by hand), and the drawing's + own DIMSCALE is put back afterwards. Headlessly each chain dimension + carries the scale as its override. The live engine reaches the same + DIMSCALE through `SetVariable`; that combination has not been re-run on a + live seat since this change. + - A headless TABLE drew its title as the first column's cell, so a title + such as `DOOR SCHEDULE` wrapped inside that column and ran over the header + row. The title row is now merged across the table, as AutoCAD's TABLE + draws it. + - A headless TABLE's cell margin was a flat 1.0 drawing unit — 0.02 mm on a + 1:50 schedule, where the text sat on the cell line and `LEFT` read as + `FT`. The margin now scales with the lettering (0.4 × text height, at most + 0.15 × row height), which leaves every table at the default 2.5 text as + it was. + - `drawing_new` left the Standard text style on ezdxf's default `txt`, + which has no superscript two: a room label's `m²` read `m?` when AutoCAD + opened the DXF and rendered as a box headlessly. Standard is now + `arial.ttf`, as AutoCAD 2026's `acadiso.dwt` ships it, so both engines + begin from the same Standard; the five bundled templates are rebuilt on it + and `scripts/build_templates.py --check` reproduces them. - **Hatch islands on a live seat.** `hatch_add_boundary` was refused on COM (`hatch_edge_paths`), so a transverse section of any hollow part - a bushing, a flange bore - could not be drawn in AutoCAD. Each typed edge now @@ -426,6 +464,20 @@ member. Spec: `docs/superpowers/specs/2026-09-24-v1.6-understand-design.md`. ### Changed +- **Release evidence re-measured for 1.6.0.** The A/B gate against v1.5.1 + (`ab-v1.5.1-vs-v1.6.0.json`: 26 / 48 → 48 / 48, 22 fixed, 0 regressed); the + reference adapter on matrix v7 (`autocad-mcp-pro.json`: 21 / 21, replacing + the v3 run of 1.5); the 100-point rubric re-scored per category with the + evidence for every point withheld (93; the competitor rows are their July + snapshots, not re-reviewed); the perf lane as medians of five alternating + runs, with the v1.4.0 comparison published beside it + (`perf-v1.4.0-vs-v1.6.0.json`). The four charts are regenerated from those + files, and the README is rewritten around them, with the mechanical and + architectural sheets rendered by `scripts/render_readme_mech.py` and + `scripts/render_readme_arch.py`. +- `ruff` skips `docs/superpowers`: ruff 0.16 formats Markdown code blocks, + and reflowing a plan's quoted code would change what the document says was + written. - **`MAX_DXF_BYTES` defaults to 512 MB** (was 50 MB): a real plant layout read for track H was a 188 MB DXF. The snapshot's refusal names the file's size, the limit and the variable; `0` still disables the check. @@ -458,6 +510,27 @@ member. Spec: `docs/superpowers/specs/2026-09-24-v1.6-understand-design.md`. the README and Linux collection predates both tracks and is superseded by the new figure. +### Not in 1.6 + +- **Track D — Plant 3D, read-only — is dropped**, and roadmap criterion 5 is + withdrawn, not met. The maintainer does not use Plant 3D, and a reader + tested against nobody's real projects would be a guess. `TOOL_PACKS` keeps + no `plant3d` pack; the `plant` pack is track H's takeoffs. +- **Roadmap criterion 7 is not met: headless creation is slower than + v1.4.0.** Medians of five alternating runs, one CPython 3.11.15 and one + ezdxf over each tree's own code: 2,000 line creates 244.5 → 306.1 ms + (1.25×), the 10,000-line build / export / reopen 1,847.9 → 2,553.5 ms + (1.38×); with `EZDXF_CALL_TIMEOUT=0` still 1.17× and 1.20×. The premium + quality pass went the other way, 150.6 → 65.3 ms. A cProfile of the + event-loop thread over the 2,000 creates counts 491,071 function calls on + 1.6.0 against 319,038 on v1.4.0. The timeout is part of the cost, not all + of it, and earlier notes that switching it off restored v1.4.0's numbers + were wrong. Carried to 1.7. +- **Four mechanical feature tables ship without rows**: DIN 471/472, DIN 509, + DIN 332 and ISO 3601-2. Their drawing code and provenance tests are in; + every size is refused by name until rows verified against the standard's + own table are transcribed. + ### Live COM smoke `scripts/smoke_pid_com.py`, AutoCAD 2026 (`25.1s (LMS Tech)`), Windows 11, diff --git a/CLAUDE.md b/CLAUDE.md index a7c0d17..84eba29 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co ## Project Overview -AutoCAD MCP Pro is a FastMCP 3.0 server that exposes ~166 tools, 8 resources, and 5 prompt templates for AutoCAD automation. (The exact tool count is reported dynamically by `system_status` / `system_about` — never hardcode it.) It runs with a dual-engine architecture: a live COM backend (Windows/AutoCAD required) and a headless ezdxf backend (works anywhere). +AutoCAD MCP Pro is a FastMCP 3.0 server that exposes ~247 tools, 8 resources, and 5 prompt templates for AutoCAD automation. (The exact tool count is reported dynamically by `system_status` / `system_about` — never hardcode it.) It runs with a dual-engine architecture: a live COM backend (Windows/AutoCAD required) and a headless ezdxf backend (works anywhere). ## Running the Server @@ -96,7 +96,7 @@ Tools are organized into sections (counts are indicative — `system_about` is a 19. Page setup & templates (6 tools, pack `core`): `page_setup_list/apply` (ISO 216 / ANSI Y14.1 sizes from `PAPER_SIZES`; an unknown ctb is written and reported `plot_style_known: false`), `plot_style_list` (catalog + the files actually installed on a live seat), `batch_plot` (every paper layout through `drawing_export_pdf(layout=)`; `mediabox_mm` is **read back from the PDF's own `/MediaBox`**, so the sheet size is verified, not assumed), `drawing_template_list` (the five bundled templates: `iso_a3_mech`, `iso_a1_arch`, `iso_a3_pid`, `ansi_b_mech`, `ansi_d_arch` — DXF headless, `.dwt` twin on COM, `scripts/build_templates.py --check` proves the DXFs reproducible), `drawing_template_save` (a `.dwt` request headlessly is refused with capability `dwt_write` naming the DXF route). `drawing_new(template=)` resolves a catalog name to the bundled file and reports `template: {name, source}`. 20. Environment (22 tools, pack `settings`): documents `document_list/activate/close` (both engines — the headless backend keeps a document registry; `document_close` refuses to discard unsaved work unless `discard=True`), layer states `layer_state_save/restore/list/delete` (portable: JSON chunks in an `ACADMCP_LAYERSTATES` XRECORD that travels with the file and **does not appear in AutoCAD's Layer States Manager** — `system_capabilities.layer_states` says so), named views `view_named_save/restore/list` (headless restore writes `$VIEWCTR/$VIEWSIZE` and reports `applied: "header_only"`), UCS `ucs_list/set/restore` (non-orthogonal axes refused with the measured angle; **tool coordinates stay WCS** — a UCS is stored for the operator, no tool interprets input in it), `system_variable_describe` (the 90-entry `SYSVAR_CATALOG`; an unknown name answers `known: false` plus nearest names, never an error; `system_set_variable` now refuses an out-of-range value for a catalogued name before writing), `drawing_properties_get/set` (custom properties on both engines through `$CUSTOMPROPERTYTAG`; the five summary fields are COM-only, capability `dwgprops`), and the live-only group `system_launch` (`live_application`), `system_preferences_get/set` (`preferences`; a whitelist of `Preferences.*` keys with ranges, read-only keys refused by name, writes report `old`/`new`), `user_pick_point/select`, `system_prompt_message` (`interactive_prompt`; a cancelled pick returns `{cancelled: true}`, a `COM_CALL_TIMEOUT` overrun reports `timed_out`). The COM paths were executed live once by `scripts/smoke_settings_com.py` (AutoCAD 2026), which also built the `.dwt` twins. 21. Mechanical parts (6 tools, pack `mech`): `mech_part_draw`, `mech_view_add`, `mech_dimension_part`, `mech_hole_pattern`, `mech_part_from_spec`, `mech_part_inspect`. One part model — a profile (`RevolvedPart` segments or a `PrismaticPart` outline) plus typed features — and one view engine that turns it into front / side / top / section / detail views with hidden lines, cut faces and ISO 129 dimensions. The model is written onto the drawing as `ACADMCP_MECH` XDATA, so `mech_view_add` can add a view months later without the caller re-describing the part. A feature that cannot be projected into a view is listed in `omitted: [{feature, reason}]` — never dropped. -22. Standard parts (3 tools, pack `mech`): `std_part_list`, `std_part_insert`, `std_feature_draw`. ISO 4014 / 4017 / 4032 / 7089 / 4762 fasteners, ISO 15 bearings with ISO 8826-1 representation, DIN 471/472 retaining-ring grooves, DIN 509 undercuts, DIN 332 centre holes and ISO 3601-2 O-ring grooves — authored tables, inserted as real blocks with attributes and `ACADMCP_MECH` XDATA, which is what makes the parts list a *read* of the drawing. A size outside a table's transcribed coverage is refused by name, never interpolated. +22. Standard parts (3 tools, pack `mech`): `std_part_list`, `std_part_insert`, `std_feature_draw`. ISO 4014 / 4017 / 4032 / 7089 / 4762 fasteners, ISO 15 bearings with ISO 8826-1 representation, DIN 471/472 retaining-ring grooves, DIN 509 undercuts, DIN 332 centre holes and ISO 3601-2 O-ring grooves — authored tables, inserted as real blocks with attributes and `ACADMCP_MECH` XDATA, which is what makes the parts list a *read* of the drawing. A size outside a table's transcribed coverage is refused by name, never interpolated. The four feature tables (DIN 471/472, DIN 509, DIN 332, ISO 3601-2) ship with **no rows** in 1.6: their drawing code and provenance tests exist, every size is refused by name, and filling one is a data edit described in its module docstring. 23. Mechanical annotation (5 tools, pack `mech`): `surface_texture` (ISO 21920-1, ISO 1302's older Ra form accepted), `weld_symbol` (ISO 2553), `centre_marks` (ISO 128-23), `section_line` (ISO 128-40 — it returns the plane `mech_view_add(kind="section")` consumes, so the label and the view cannot disagree), `hatch_material` (the ISO 128-50 map; an unknown material is refused with the list, never hatched as steel). 24. Sheet and delivery (11 tools, pack `core`): `sheet_frame` (ISO 5457 A4-A0 — 20 mm filing margin, 10 mm elsewhere on A3, zone grid, centring and trimming marks), `titleblock_apply` (ISO 7200 for every size; `titleblock_apply_iso_a3` is now a thin alias), `revision_add`, `bom_extract` (never modifies the drawing), `bom_table` (ISO 7573), `balloon_add` (ISO 6433, linked by XDATA so a rerun renumbers instead of duplicating), `data_extract` (CSV always; XLSX refused with `xlsx_write` when `openpyxl` is absent), `xref_attach`, `xref_manage`, `image_attach`, `drawing_export_dwg` (COM `SaveAs` natively; headless only with the ODA File Converter, otherwise `dwg_write` refuses and names the install route). The COM paths of SECTIONS 21-24 were executed live once by `scripts/smoke_mech_com.py` (AutoCAD 2026): part, section, dimensions, an ISO 4014 block with its XDATA, the parts list read off the drawing, a linked balloon, an xref attached and detached, a real `.dwg`, and `drawing_critique(focus=None)` clean. 25. Architecture (12 tools, pack `arch`): `arch_wall`, `arch_opening`, `arch_stair`, `arch_room`, `arch_rooms_detect`, `arch_schedule`, `arch_grid`, `arch_symbol`, `arch_catalogue_list`, `arch_catalogue_insert`, `arch_dimension_chains`, `arch_plan_from_spec`. One plan model — a wall network (axis polyline, thickness, justification, material) with hosted doors and windows — and one engine: L / T / X junctions resolved, openings cut, cut faces hatched by material (`engineering/arch/materials.py`: ISO 128-50 general hatching for brick, the JIS A 0150 RC / stone / wood symbols, AutoCAD's AR-CONC / AR-SAND, AAC as a declared convention), and anything it cannot place listed in `omitted: [{element, reason}]`, never dropped. Rooms are *computed*: a label's area is the planar face its point lies in, and `arch_rooms_detect` reads rooms from any plan (confidence 1.0 for our walls, 0.6 for plain lines) without modifying it. Schedules are a read of the `ACADMCP_ARCH` records, drawn as real TABLEs. Labels and schedules speak English, or Turkish with `lang="tr"` (decimal comma). The COM paths were executed live once by `scripts/smoke_arch_com.py` (AutoCAD 2026). @@ -111,13 +111,13 @@ Tools are organized into sections (counts are indicative — `system_about` is a **ISO 129 tolerances:** `dimension_linear` / `dimension_radius` / `dimension_diameter` take `tol_upper` / `tol_lower` / `tol_mode` (`symmetric` ± / `deviation` +a/-b / `limit` / `basic`) and `text_override` (e.g. `⌀20 H7`). Prefer these over hand-drawn tolerance text. -**ISO 286 fits:** the same dimension tools take `fit="H7"` (mutually exclusive with `tol_*`) — deviations are resolved from authored ISO 286 tables (`engineering/fits.py`; shafts d/e/f/g/h/js/k/m/n/p, holes D/E/F/G/H/JS, sizes 1–500 mm) against the measured nominal and the fit code is appended to the dimension text. +**ISO 286 fits:** the same dimension tools take `fit="H7"` (mutually exclusive with `tol_*`) — deviations are resolved from authored ISO 286 tables (`engineering/fits.py`; shafts d/e/f/g/h/js/k/m/n/p and interference r/s/t/u, holes D/E/F/G/H/JS and K/M/N/P by the ISO 286 delta rule, sizes 1–500 mm) against the measured nominal and the fit code is appended to the dimension text. **Drawing score:** `drawing_finalize` returns `payload["score"]` — a 0-100 scalar + `invalidity_ratio` + A-F `grade` over the validator + critique union. Use it as the objective quality metric. **Measuring:** never read vertices back and shoelace them — that loses 28.2% of the area on a semicircular edge, silently. `analysis_measure_entity(handle)` reads the real geometry, and its payload states its own accuracy: `exact`, `flatten_tolerance`, `assumed_closed`, `self_intersecting`. A HATCH reports the area it *fills* (outer loops minus islands) plus `hatch_style`; on a curved hatch edge `flatten_tolerance` is not the accuracy knob, because ezdxf hands boundaries over as cubic Beziers whose ~0.028% circle error is already baked in. `boundary_trace` / `boundary_from_entities` report the same number `analysis_measure_entity` gives for the polyline they just drew — if those ever diverge, one of them is approximating and neither says so. -**Benchmark matrix:** `benchmarks/tasks_v7.py` is the current set (21 tasks; `--matrix v6` / `v5` / `v4` / `v3` / `v2` reproduce the earlier sets). The published competitor reports are pinned v1.4 runs against v2 and are **never** back-filled for the five v3 tasks, the v4 `pid_roundtrip` / `page_setup_truth`, the v5 `mech_assembly` (a flange-coupling sheet gated on a zero-issue `drawing_critique(focus=None)` and a finalize score of at least 90) or the v6 `arch_roundtrip` (a two-room plan drawn by `arch_plan_from_spec`, its rooms read back by `arch_rooms_detect` within 0.1 % of the label and of the hand-computed net floor, the schedules listing the drawn tags, gated like `mech_assembly`) or the v7 `takeoff_roundtrip` / `understand_foreign` (the synthetic plant pair of `tests/fixtures/plant_pair.py` read without its truth: the P&ID called schematic, every routable run's layout length equal to the rectilinear MST of its tags, the cable metres rounded up after the allowance; units, the outlier, both plan copies and the service layers found by one `describe`) — the chart shows them `not run`, not zero. `benchmarks/correctness_suite.py` (48 checks) is the release A/B gate: `python benchmarks/compare_versions.py v1.5.1` (the published report is `benchmarks/results/published/ab-v1.5.1-vs-v1.6.0-dev.json`; `scripts/check_doc_numbers.py` names it in `AB_REPORT`). +**Benchmark matrix:** `benchmarks/tasks_v7.py` is the current set (21 tasks; `--matrix v6` / `v5` / `v4` / `v3` / `v2` reproduce the earlier sets). The published competitor reports are pinned v1.4 runs against v2 and are **never** back-filled for the five v3 tasks, the v4 `pid_roundtrip` / `page_setup_truth`, the v5 `mech_assembly` (a flange-coupling sheet gated on a zero-issue `drawing_critique(focus=None)` and a finalize score of at least 90) or the v6 `arch_roundtrip` (a two-room plan drawn by `arch_plan_from_spec`, its rooms read back by `arch_rooms_detect` within 0.1 % of the label and of the hand-computed net floor, the schedules listing the drawn tags, gated like `mech_assembly`) or the v7 `takeoff_roundtrip` / `understand_foreign` (the synthetic plant pair of `tests/fixtures/plant_pair.py` read without its truth: the P&ID called schematic, every routable run's layout length equal to the rectilinear MST of its tags, the cable metres rounded up after the allowance; units, the outlier, both plan copies and the service layers found by one `describe`) — the chart shows them `not run`, not zero. `benchmarks/correctness_suite.py` (48 checks) is the release A/B gate: `python benchmarks/compare_versions.py v1.5.1` (the published report is `benchmarks/results/published/ab-v1.5.1-vs-v1.6.0.json`; `scripts/check_doc_numbers.py` names it in `AB_REPORT`). ### Adding a New Tool diff --git a/README.md b/README.md index 86a1ef5..14680bc 100644 --- a/README.md +++ b/README.md @@ -11,44 +11,72 @@ Live through COM on Windows, or headless through ezdxf anywhere — one typed co [![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-1f6feb?logo=python&logoColor=white)](https://github.com/U-C4N/Autocad-MCP/blob/main/pyproject.toml) [![License: MIT](https://img.shields.io/badge/license-MIT-3fb950)](https://github.com/U-C4N/Autocad-MCP/blob/main/LICENSE) -[Install](#install) · [Tools](#what-you-get) · [Engines](#the-two-engines) · [Evidence](#evidence) · [Limits](#what-this-release-is-bad-at) · [Config](#configuration) · [Changelog](https://github.com/U-C4N/Autocad-MCP/blob/main/CHANGELOG.md) +[Install](#install) · [What's new](#whats-new-in-16) · [Tools](#what-you-get) · [Engines](#the-two-engines) · [Evidence](#evidence) · [Limits](#what-this-release-is-bad-at) · [Config](#configuration) · [Changelog](https://github.com/U-C4N/Autocad-MCP/blob/main/CHANGELOG.md) ISO A3 sheet: a module-6, 24-tooth spur gear with a keyed bore, its section A-A, ISO 129 dimensions with an H7 bore fit, and an ISO 7200 title block Not a mockup. Every line on this sheet was drawn by the tools this server exposes — ISO layers, involute gear geometry, DIN 6885 keyway, section A-A, ISO 129 dimensions, an ISO 286 H7 bore fit, ISO 7200 title block — then rendered headlessly by view_screenshot. drawing_critique returns 0 issues on it. Rebuild it with python scripts/render_readme_showcase.py. -P&ID sheet: a centrifugal pump, a diaphragm-actuated globe control valve, a vertical vessel with two nozzles, a flow controller bubble and an off-page connector, joined by numbered process lines and an electric signal line - -The same server as a P&ID drafter. pid_from_spec placed five catalogue symbols as real blocks with TAG attributes and named ports, routed four port-to-port lines with ISA-5.1 classes and line numbers in one transaction, then pid_graph read the sheet back: 5 nodes, 4 edges, 0 dangling ends, confidence_min 1.0. Rebuild it with python scripts/render_readme_pid.py. - -Contact sheet of the 43 P&ID catalogue symbols: valve bodies with actuators, pumps and compressors, heat exchangers, vessels, instrument bubbles, line markers and connectors - -The catalogue, authored in code. 43 ISO 10628-2 / ISA-5.1 symbols — 11 valve bodies × 6 actuators, 6 rotating machines, 4 heat exchangers, 6 parametric vessels, 7 inline items, the instrument bubble in 4 types × 5 locations, 5 line markers, 3 connectors — every one placed through pid_symbol_insert's own path. Rebuild it with python scripts/render_pid_catalog.py. - -> **v1.5 release snapshot:** 247 tools · 8 resources · 5 prompt templates · 4987 collected tests. +> **v1.6 release snapshot:** 247 tools · 8 resources · 5 prompt templates · 4991 collected tests. > 247 is the **registered** count; a default install advertises 242 over `tools/list`, > because `ENABLE_3D` is unset. `system_about` is the runtime authority. +## What's new in 1.6 + +Five tracks of one [roadmap](https://github.com/U-C4N/Autocad-MCP/blob/main/docs/superpowers/specs/2026-09-15-v1.6-roadmap-design.md), 154 → 247 registered tools. Each track has its own design spec, correctness checks that can fail, and a live AutoCAD 2026 run of its COM paths recorded in the [changelog](https://github.com/U-C4N/Autocad-MCP/blob/main/CHANGELOG.md). + +| Track | What it adds | Pack | +|---|---|---| +| **P&ID — drafter and reader** | 43 ISO 10628-2 / ISA-5.1 symbols as real blocks with ports and `TAG` attributes, port-to-port lines with ISA-5.1 classes and line numbers, `pid_graph` reading any P&ID back with a confidence per node, instrument index, line list, equipment list, `pid_from_spec` for a whole sheet in one transaction | `pid` | +| **The AutoCAD environment** | ISO-25 / ANSI dimension styles, text and leader styles, `drawing_apply_standard`, ISO 216 / ANSI page setups checked against the plotted PDF's own `/MediaBox`, five bundled templates, several open documents headlessly, portable layer states, named views, UCS, a 90-entry system-variable catalogue; on a live seat, preferences, launch and operator prompts | `core`, `settings` | +| **Mechanical parts and the sheet** | one part model and one view engine (front, side, top, section and detail views with hidden lines, cut faces and ISO 129 dimensions), ISO fasteners and bearings as blocks with attributes, ISO 21920-1 surface texture, ISO 2553 welds, ISO 5457 frames A4–A0, ISO 7200 title blocks, an ISO 7573 parts list read off the drawing with linked ISO 6433 balloons, xrefs, images, real `.dwg` | `mech`, `core` | +| **Architecture** | walls with hosted doors and windows: one engine resolves L / T / X junctions and cuts the openings. Room areas are measured from the faces the walls enclose, and rooms are read back from any plan. Door / window / room schedules are real tables; labels in English or Turkish | `arch` | +| **Understanding and takeoffs** | `drawing_understand` reads a drawing somebody else made in one call. Also `drawing_scale_check`, `drawing_diff` and `drawing_topology_check`. `pipe_takeoff` / `cable_takeoff` take the topology from the P&ID and the lengths from the layout into a Turkish, English or Russian workbook | `core`, `plant` | + +`TOOL_PACKS` keeps a client from paying for the tracks it does not use ([idle cost below](#why-this-exists)). + +
+ +ISO A3 flange-coupling sheet: a hub with a full section and chain dimensions, a flange with its bolt-circle end view, two ISO 4014 bolts with ISO 4032 nuts, a parts list with linked balloons, an ISO 5457 frame with zones and an ISO 7200 title block + +A mechanical sheet, gated. The mech_assembly benchmark sheet: a hub drawn from its part model with a full section and chain dimensions, a flange with its bolt-circle end view, two ISO 4014 M12×60 bolts and ISO 4032 nuts inserted as blocks, the ISO 7573 parts list read off the drawing, balloons linked to its rows, an ISO 5457 frame and an ISO 7200 title block. drawing_critique(focus=None) returns 0 issues and the finalize score is 100. Rebuild it with python scripts/render_readme_mech.py. + +Two-room floor plan: hatched brick walls with an aerated-concrete partition, two doors, two windows, a straight stair, room labels with measured areas, exterior dimension chains, and door, window and room schedules + +A floor plan whose areas nobody typed. arch_plan_from_spec's example, drawn in one transaction: a 250 mm brick ring split by a 100 mm AAC wall, two doors, two windows, a straight stair and exterior dimension chains at 1:50. The rooms measure 27.74 and 21.99 m², read from the faces the walls enclose, and the door, window and room schedules are read off the drawing. drawing_critique(focus=None) returns 0 issues. Rebuild it with python scripts/render_readme_arch.py. + +P&ID sheet: a centrifugal pump, a diaphragm-actuated globe control valve, a vertical vessel with two nozzles, a flow controller bubble and an off-page connector, joined by numbered process lines and an electric signal line + +The same server as a P&ID drafter. pid_from_spec placed five catalogue symbols as real blocks with TAG attributes and named ports and routed four port-to-port lines with ISA-5.1 classes and line numbers, all in one transaction. Then pid_graph read the sheet back: 5 nodes, 4 edges, 0 dangling ends, confidence_min 1.0. Rebuild it with python scripts/render_readme_pid.py. + +Contact sheet of the 43 P&ID catalogue symbols: valve bodies with actuators, pumps and compressors, heat exchangers, vessels, instrument bubbles, line markers and connectors + +The catalogue, authored in code. 43 ISO 10628-2 / ISA-5.1 symbols: 11 valve bodies × 6 actuators, 6 rotating machines, 4 heat exchangers, 6 parametric vessels, 7 inline items, the instrument bubble in 4 types × 5 locations, 5 line markers and 3 connectors. Every one was placed through pid_symbol_insert's own path. Rebuild it with python scripts/render_pid_catalog.py. + +
+ ## Why this exists -**A big MCP server is expensive to be connected to.** The full catalog costs a client **83,758 tokens** before it has asked for anything. Discovery mode replaces it with two tools and costs **356**. +**A big MCP server is expensive to be connected to.** The full catalog costs a client **83,921 tokens** before it has asked for anything. Discovery mode replaces it with two tools and costs **356**. -**A drafter searches for `FILLET`, not `entity_fillet`.** Those command names appeared in no tool name or description — `df = 0` against a stock index, not badly ranked but *absent*. The fix was data: an authored corpus of **206 AutoCAD command names and 1439 synonym phrases** covering all 247 tools. A test refuses to let a tool exist without one. +**A drafter searches for `FILLET`, not `entity_fillet`.** Before the corpus, those command names appeared in no tool name or description. That was `df = 0` against a stock index: the tools were not ranked badly, they were *absent*. The fix was data: an authored corpus of **206 AutoCAD command names and 1439 synonym phrases** covering all 247 tools. A test refuses to let a tool exist without one. | Advertised surface | Tools seen | Idle cost | |---|---:|---:| -| `TOOL_PROFILE=full` (default) | 242 | 83,758 tokens | +| `TOOL_PROFILE=full` (default) | 242 | 83,921 tokens | | `TOOL_PACKS=core,settings` (full profile) | 205 | 63,890 tokens | | `TOOL_PACKS=core,mech` (full profile) | 197 | 64,836 tokens | -| `TOOL_PACKS=core,arch` (full profile) | 195 | 63,532 tokens | +| `TOOL_PACKS=core,arch` (full profile) | 195 | 63,549 tokens | +| `TOOL_PACKS=core,pid` (full profile) | 192 | 60,844 tokens | +| `TOOL_PACKS=core,plant` (full profile) | 185 | 59,290 tokens | | `TOOL_PACKS=core` (full profile) | 183 | 57,122 tokens | | `TOOL_PROFILE=lean` | 65 | 22,661 tokens | | `DISCOVERY_MODE=search` | 2 | **356 tokens** | > [!NOTE] -> Offline ratio estimates, not tokenizer counts, and the *uncached* ratio — prompt caching amortises the idle term. `benchmarks/token_suite.py --tokenizer anthropic` counts for real. +> These are offline ratio estimates, not tokenizer counts, and they are the *uncached* cost: prompt caching amortises the idle term. `benchmarks/token_suite.py --tokenizer anthropic` counts for real. ## Install @@ -67,6 +95,7 @@ AUTOCAD_MCP_BACKEND=com autocad-mcp # live AutoCAD (needs the [com] extra) | *(none)* | `fastmcp`, `ezdxf`, `pydantic` | Headless DXF on any OS — **no rendering** | | `[com]` | `pywin32`, `Pillow` | Live AutoCAD control + window capture | | `[pdf]` | `matplotlib` | PDF export and headless PNG, any platform | +| `[office]` | `openpyxl` | XLSX workbooks (takeoffs, `data_extract`); CSV is always written | | `[full]` | everything above | Development and CI | > [!NOTE] @@ -84,6 +113,7 @@ AUTOCAD_MCP_BACKEND=com autocad-mcp # live AutoCAD (needs the [com] extra) "AUTOCAD_MCP_BACKEND": "auto", "ALLOWED_PATHS": "C:\\Users\\you\\Documents\\AutoCAD", "TOOL_PROFILE": "full", + "TOOL_PACKS": "all", "DISCOVERY_MODE": "off" } } @@ -91,7 +121,7 @@ AUTOCAD_MCP_BACKEND=com autocad-mcp # live AutoCAD (needs the [com] extra) } ``` -Claude Desktop, Cursor, or any stdio MCP host. For HTTP: `autocad-mcp --transport http --port 8000` — loopback only unless remote HTTP is explicitly enabled **and** a bearer token is set. +It works with Claude Desktop, Cursor, or any stdio MCP host. For HTTP, run `autocad-mcp --transport http --port 8000`. It binds loopback only, unless remote HTTP is explicitly enabled **and** a bearer token is set. @@ -101,35 +131,38 @@ Claude Desktop, Cursor, or any stdio MCP host. For HTTP: `autocad-mcp --transpor |---|---| | Drawing lifecycle | create, open, save, export DXF/PDF, audit *(repairs)*, purge, undo/redo | | Geometry | lines, arcs, polylines, splines, hatches, trim/extend/fillet/chamfer, handle-preserving edits | -| Annotation | ISO 129 toleranced dimensions, ISO 286 fits (`fit="H7"`), TABLE, MLEADER, GD&T frames and datums (ISO 1101) | +| Annotation | ISO 129 toleranced dimensions, ISO 286 fits (`fit="H7"`, interference shafts r/s/t/u included), TABLE, MLEADER, GD&T frames and datums (ISO 1101) | | Engineering generators | involute gears (front + section A-A), DIN 6885 keyed bores, ISO A3 title block | -| Mechanical parts | one part model (segments or outline + typed features) and one view engine: front / side / top / section / detail with hidden lines, ISO 128-50 cut faces and ISO 129 dimensions; ISO 4014/4017/4032/7089/4762 fasteners, ISO 15 bearings, DIN 471/472, DIN 509, DIN 332 and ISO 3601-2 as real blocks with attributes; ISO 21920-1 surface texture, ISO 2553 welds, ISO 128-40 section lines | +| Mechanical parts | one part model (segments or outline + typed features) and one view engine: front / side / top / section / detail with hidden lines, ISO 128-50 cut faces and ISO 129 dimensions. ISO 4014/4017/4032/7089/4762 fasteners and ISO 15 bearings insert as real blocks with attributes. DIN 471/472, DIN 509, DIN 332 and ISO 3601-2 features have their drawing code, but *their tables ship empty in 1.6* (see [limits](#what-this-release-is-bad-at)). Also ISO 21920-1 surface texture, ISO 2553 welds and ISO 128-40 section lines | | Sheet & delivery | ISO 5457 frames A4-A0 with zones and trim marks, ISO 7200 title blocks for every size, revision blocks with clouds, ISO 7573 parts lists with ISO 6433 balloons linked by XDATA, CSV/XLSX extraction, xrefs and images on both engines, real `.dwg` on a live seat | -| Architecture | one plan model — walls with hosted doors and windows — and one engine: L / T / X junctions resolved, openings cut, poché by material; rooms measured from the faces the walls enclose (never typed) and read back from any plan with a confidence; door / window / room schedules as real TABLEs; a furniture and sanitary catalogue at nominal sizes; structural grid, north arrow, section / level / elevation marks; exterior dimension chains; English or Turkish labels (`lang="tr"`, decimal comma) | -| Understanding & takeoffs | `drawing_understand` reads a drawing somebody else made in one call — declared vs inferred units, robust extents and the outlier behind them, plan copies, each layer's discipline and service in five languages, tags, rooms; `drawing_scale_check` says whether a P&ID is to scale against its layout; `pipe_takeoff` / `cable_takeoff` take the topology from the P&ID and the lengths from the layout (rectilinear MST, statuses A/B/C, power never invented) into a Turkish / English / Russian workbook; `drawing_diff` between revisions; `drawing_topology_check` for dangling ends, near misses and crossings | +| Architecture | one plan model (walls with hosted doors and windows) and one engine that resolves L / T / X junctions, cuts openings and draws poché by material. Room areas are measured from the faces the walls enclose, never typed, and rooms are read back from any plan with a confidence. Door / window / room schedules are real TABLEs. Also a furniture and sanitary catalogue at nominal sizes, a structural grid, north arrow, section / level / elevation marks, exterior dimension chains, and English or Turkish labels (`lang="tr"`, decimal comma) | +| Understanding & takeoffs | `drawing_understand` reads a drawing somebody else made in one call: declared vs inferred units, robust extents and the outlier behind them, plan copies, each layer's discipline and service in five languages, tags, rooms. `drawing_scale_check` says whether a P&ID is to scale against its layout. `pipe_takeoff` / `cable_takeoff` take the topology from the P&ID and the lengths from the layout (rectilinear MST, statuses A/B/C, power never invented) into a Turkish / English / Russian workbook. `drawing_diff` compares revisions; `drawing_topology_check` finds dangling ends, near misses and crossings | | P&ID | catalogue blocks with ports and tags (ISO 10628-2 / ISA-5.1), port-to-port lines with ISA-5.1 classes and line numbers, `pid_graph` reads any P&ID back with confidence, instrument index / line list / equipment list, `pid_from_spec` one-call sheets | | Styles & standards | ISO-25 / ANSI dimension styles, ISOCP / ROMANS text styles, ISO / ANSI leader styles from authored presets; `drawing_apply_standard("iso")` sets all of it plus units and the `mech` layers in one call; `changed` reports only what moved | | Discovery | `search_tools` ranked over an AutoCAD command and synonym corpus — `FILLET`, `BPOLY`, `QSELECT`, `WBLOCK`, `OVERKILL`, `CHSPACE` each rank **#1** of the 242-tool advertised catalog | | Batching | `cad_batch` runs a step list in one round trip; `fields=` projects 11 result-heavy tools | -| Paper space | tab lifecycle, viewports, `entity_change_space` (CHSPACE), `page_setup_apply` (ISO 216 / ANSI Y14.1 paper, ctb, scale, device — on both engines), `batch_plot` with every sheet size read back from its PDF's `/MediaBox`, `drawing_export_pdf(layout=…)` | -| Page setup & templates | ISO 216 / ANSI Y14.1 sheets, ctb catalog, `batch_plot` verified by the PDF's own `/MediaBox`, five bundled templates built by the server's own tools and pinned reproducible (`drawing_new(template="iso_a3_mech")`, `iso_a1_arch`, `iso_a3_pid`, `ansi_b_mech`, `ansi_d_arch`), save any drawing as a template (`.dwt` on live AutoCAD, `dwt_write` refused headlessly) | -| Environment | several open documents headlessly, portable layer states (`ACADMCP_LAYERSTATES` XRECORDs — in the file, not in AutoCAD's Layer States Manager), named views, UCS *(tool coordinates stay WCS)*, a system-variable catalog with ranges, document properties, and on a live seat: launch/attach, preferences, an operator prompt / pick / select — where ESC is `cancelled`, not an error | +| Paper space | tab lifecycle, viewports, `entity_change_space` (CHSPACE), `drawing_export_pdf(layout=…)` | +| Page setup & templates | ISO 216 / ANSI Y14.1 sheets, ctb catalog, `page_setup_apply` on both engines, and `batch_plot` with every sheet size read back from its PDF's `/MediaBox`. Five bundled templates, built by the server's own tools and pinned reproducible: `drawing_new(template="iso_a3_mech")`, `iso_a1_arch`, `iso_a3_pid`, `ansi_b_mech`, `ansi_d_arch`. Save any drawing as a template (`.dwt` on live AutoCAD; `dwt_write` refused headlessly) | +| Environment | several open documents headlessly, portable layer states (`ACADMCP_LAYERSTATES` XRECORDs — in the file, not in AutoCAD's Layer States Manager), named views, UCS *(tool coordinates stay WCS)*, a system-variable catalog with ranges, document properties. On a live seat: launch/attach, preferences, and an operator prompt / pick / select, where ESC is `cancelled`, not an error | | Selection | window vs crossing stated back to the caller; a polygon tested against its own shape, not its bounding box | | Boundaries | `boundary_trace` (BOUNDARY/BPOLY) chains loose edges into one closed polyline, arcs kept as bulges *(headless)* | | Measurement | `analysis_measure_entity` measures what is *in* the drawing, by handle | | Hatch depth | gradients, in-place edits, typed edge boundaries *(headless)*, island styles | | Annotation objects | WIPEOUT, REVCLOUD, MTEXT background masks *(headless)*, text find/replace | | 3D solids | `solid_box/cylinder/extrude/revolve/boolean` on live AutoCAD (`ENABLE_3D=true`) | -| Quality loop | `drawing_preflight` → `drawing_plan` → `drawing_critique` → `drawing_refine` → `drawing_finalize` (0–100 score) | +| Quality loop | `drawing_preflight` → `drawing_plan` → `drawing_critique` → `drawing_refine` → `drawing_finalize` (0–100 score); 18 domain critique focuses (P&ID 6, mechanical 6, architecture 3, topology 3), each silent on a drawing outside its domain | | Delivery | `drawing_deliver`: DXF/PDF/PNG + SHA-256 manifest + reopen-parity checks | -247 tools in 28 groups; TOOL_PACKS=core hides the nine pid_* tools, the 22 environment tools, the 14 mechanical tools, the 12 architectural tools and the two plant takeoffs (TOOL_PACKS=core,mech keeps the mechanical ones, TOOL_PACKS=core,arch the architectural ones, TOOL_PACKS=core,plant the takeoffs) from a client that needs none of them. Plus 8 resources that cost nothing in the tool budget (autocad://drawing/info, layers, blocks, entities/stats, entities/{layer_name}, system/status, pid/symbols, standards/isa51) and 5 prompt templates. +247 tools in 28 groups. TOOL_PACKS=core hides these from a client that needs none of them: the nine pid_* tools, the 22 environment tools, the 14 mechanical tools, the 12 architectural tools and the two plant takeoffs. TOOL_PACKS=core,mech keeps the mechanical ones, TOOL_PACKS=core,arch the architectural ones, and TOOL_PACKS=core,plant the takeoffs. There are also 8 resources that cost nothing in the tool budget (autocad://drawing/info, layers, blocks, entities/stats, entities/{layer_name}, system/status, pid/symbols, standards/isa51) and 5 prompt templates. -**Two rules worth knowing.** Every coordinate in and out of a tool is WCS on both engines — the one exception is TEXT `rotation`, which stays in the entity frame because a mirrored TEXT is mirror-imaged and no scalar angle expresses that. And never read vertices back and shoelace them: that loses **28.2%** of the area on a semicircular edge, silently. `analysis_measure_entity(handle)` reads the real geometry and states its own accuracy. +**Two rules worth knowing.** + +- Every coordinate in and out of a tool is WCS on both engines. The one exception is TEXT `rotation`, which stays in the entity frame: a mirrored TEXT is mirror-imaged, and no scalar angle expresses that. +- Never read vertices back and shoelace them. That silently loses **28.2%** of the area on a semicircular edge. `analysis_measure_entity(handle)` reads the real geometry and states its own accuracy. ## The two engines -One contract across 19 modules in `backends/contracts/`. `@capability(key, reason=…)` supplies a default that *refuses*, and a test holds both backends' capability key sets equal. There are **27 capability keys**; read `system_capabilities` at runtime rather than trusting the table — five of them depend on how the machine is set up. +One contract across 23 modules in `backends/contracts/`. `@capability(key, reason=…)` supplies a default that *refuses*, and a test holds both backends' capability key sets equal. There are **41 capability keys**. Read `system_capabilities` at runtime rather than trusting the table: several depend on what the machine has installed (matplotlib, openpyxl, the ODA File Converter). | Capability | COM (live AutoCAD) | ezdxf (headless) | |---|:---:|:---:| @@ -143,7 +176,11 @@ One contract across 19 modules in `backends/contracts/`. `@capability(key, reaso | HATCH filled area (islands subtracted) | AutoCAD's own number | Loops walked, `hatch_style` reported | | REGION / 3DSOLID area | ✅ | — *ACIS is opaque to ezdxf* | | 3D solids | ✅ *with `ENABLE_3D=true`* | — | -| DWG write | ✅ | — | +| DWG write | ✅ `SaveAs` | via the ODA File Converter, if installed | +| `.dwt` templates | ✅ | DXF only — `dwt_write` refuses | +| Reading a foreign drawing (`drawing_understand`, takeoffs) | ✅ one `Document.Export` snapshot; the document is never renamed | ✅ | +| XLSX workbooks | ✅ ᵒ | ✅ ᵒ | +| Preferences · launch · operator prompts | ✅ | — *refused by name* | | WIPEOUT · MTEXT background colour | — *verified absent, AutoCAD 2026* | ✅ | | REVCLOUD · BPOLY · typed hatch edges | — *no ActiveX member* | ✅ | | CHSPACE | — *unverified on a live seat* | ✅ ᶜ | @@ -151,80 +188,176 @@ One contract across 19 modules in `backends/contracts/`. `@capability(key, reaso | TABLE and MLEADER | Native | Portable composite | | Screenshots and PDF | Window capture | Matplotlib ᵐ | -ᵐ Needs matplotlib ([pdf]/[full]); without it png, pdf, viewport_render and handle_overlay report unsupported headlessly.
-ᶜ Headless CHSPACE carries four named restrictions in its capability reason: top-view untwisted viewports only, dimensions refused unless frozen, ACIS/proxy/table refused, viewport clipping reported rather than applied. +ᵐ Needs matplotlib ([pdf]/[full]). Without it, png, pdf, viewport_render and handle_overlay report unsupported headlessly.
+ᵒ Needs openpyxl ([office]/[full]). Without it, XLSX is refused with xlsx_write and CSV is still written.
+ᶜ Headless CHSPACE has four restrictions, named in its capability reason: top-view untwisted viewports only; dimensions refused unless frozen; ACIS, proxy and table entities refused; viewport clipping reported rather than applied. > [!IMPORTANT] -> Every COM path added in v1.5 was executed against a **live AutoCAD 2026**. That run found four defects reading could not — the largest being that `AcadPViewport` has no `ViewCenter` member, so the line setting it had been silently swallowed by a bare `except` since v1.4. +> Every COM path added in v1.6 was executed against a **live AutoCAD 2026**, one smoke script per track (`scripts/smoke_*_com.py`), each run recorded in the changelog. Those runs found defects no headless test could. ActiveX refuses `entity.Layer` on a missing layer, where ezdxf quietly creates it, so `mech_part_draw` died after drawing every view. ActiveX calls a block reference `BLOCKREFERENCE`, not `INSERT`, so the parts list read nothing. And once pywin32's type-library cache existed, every listed entity arrived as a bare `IAcadEntity` with no line endpoints, so the room reader saw no walls and `arch_plan_from_spec` refused its first room label. ## Evidence -Self-measurement, produced by scripts in [`benchmarks/`](https://github.com/U-C4N/Autocad-MCP/blob/main/benchmarks/). +Everything below is produced by scripts in [`benchmarks/`](https://github.com/U-C4N/Autocad-MCP/blob/main/benchmarks/) and published as JSON under [`benchmarks/results/published/`](https://github.com/U-C4N/Autocad-MCP/tree/main/benchmarks/results/published). + +### Against the other AutoCAD MCP servers + +Source-reviewed capability leaderboard: U-C4N/Autocad-MCP 93, varavista 84, beiming183-cloud 83, best-cad-mcp 74, multiCAD-mcp 73, puran-water 70, NCO-1986 68, daobataotie 35, Easy-MCP-AutoCad 30 + +A fixed 100-point rubric applied to nine public AutoCAD MCP servers ([`source_review.json`](https://github.com/U-C4N/Autocad-MCP/blob/main/benchmarks/source_review.json)). Stars and raw tool counts score nothing. 1.6.0 is scored **per category, with the evidence for every point withheld**: + +| Category | Weight | 1.6.0 | Points withheld for | +|---|---:|---:|---| +| Functional CAD coverage | 25 | 24 | no electrical schematics, image-to-CAD tracing, Plant 3D, obstacle-avoiding routing or headless REGION / 3DSOLID | +| Correctness and delivery | 20 | 19 | a headless DIMENSION does not re-render; a takeoff on a real P&ID still needs the engineer's scope decisions | +| Backends and platforms | 15 | 14 | BricsCAD, ZWCAD and GstarCAD unverified | +| Engineering production | 15 | 14 | four DIN / ISO feature tables ship empty | +| Tests and maintenance | 15 | 14 | headless entity creation slower than v1.4.0 | +| Security and operations | 10 | 8 | `ALLOWED_PATHS` unscoped by default; command filtering is a denylist | +| **Total** | **100** | **93** | | + +> [!NOTE] +> Read this as a documented self-assessment, not an independent review: this repository applies the rubric to itself and to the others. The competitor rows are their July 2026 snapshots and were not re-reviewed for this release, and the best of them, varavista/autocad-mcp, scored 84. 1.4.0 was given one overall 95. This review is stricter and scores each category apart. + +A second lane runs tasks instead of reading source. Every task in the fixed-task matrix can fail, and its artifacts are verified by re-opening the DXF, never by trusting a response. The reference adapter passes **21 / 21** of matrix v7. The two competitors with adapters, pinned at their July 2026 commits, passed 5 / 10 and 4 / 10 of the ten v2 tasks they were run on. The eleven tasks added since then were never put to them, so they show as *not run*, not as zero: + +Task matrix: autocad-mcp-pro passes all 21 tasks; beiming183 and puran-water were run on the first ten, with the later eleven not run ### Correctness — every release re-proves itself -48 deterministic headless checks against the previous tag and the current tree, each in its own subprocess so a hard crash counts as a miss rather than killing the run. +48 deterministic headless checks against the previous tag and the current tree. Each runs in its own subprocess, so a hard crash counts as a miss rather than killing the run. | Version | Checks passing | Pass rate | Fixed | Regressed | |---|---:|---:|---:|---:| | v1.5.1 *(baseline)* | 26 / 48 | 54.2 % | — | — | -| **v1.6.0-dev** *(this branch)* | **48 / 48** | **100 %** | 22 | **0** | +| **v1.6.0** *(this release)* | **48 / 48** | **100 %** | 22 | **0** | -The twenty-two fixed rows are new capability (`miss → pass`). Track H adds five: `scale_check_detects_schematic` (the synthetic P&ID reads **schematic** against its layout, and a uniform 2× copy of six tags reads **to scale** with factor 2), `pipe_takeoff_rmst_exact` (all eight routable runs of the pair measure exactly the rectilinear MST of their tags), `cable_takeoff_roundup` (9, 17, 19 and 10 m after the 20 % allowance, and the heater's unstated power stays empty), `diff_detects_known_edits` (a move, a text change, an attribute change, an add and a delete — and nothing else) and `topology_known_defects` (a 5 mm near miss, one interior crossing, and a clean T that is neither). Track F adds four: `arch_junction_l_t_x` (an L corner mitres at (4100, −100) / (3900, 100), a T stem stops on the near face at (2950, 100) / (3050, 100), an X cuts all four faces, and the wall areas are 1 400 000, 1 490 000 and 2 360 000 mm² — computed by hand), `arch_room_area_net` (a 4 × 5 m room between 200 mm walls measures 20.00 m² in its label and its record, not the 21.84 m² its axes enclose), `arch_opening_cuts_wall` (both faces are interrupted across a 900 door and two jambs close it) and `arch_rooms_detect_foreign` (a plan of plain lines on a `WALLS` layer yields 20 and 15 m² at confidence 0.6, and nothing is written). Tracks B and G add seven: `mech_part_roundtrip` (a part read back out of its own `ACADMCP_MECH` XDATA equals the part drawn), `mech_section_hatch_area` (a 60 x ⌀40 sleeve with a ⌀20 bore cuts 1200 mm² exactly, measured out of the drawing by `analysis_measure_entity`), `mech_iso286_on_dimension` (40 H7 is +0.025 / 0 and survives the layout), `mech_thread_unrepresented_is_caught` (the ISO 6410 focus fires on a thread drawn as a plain circle — a gate that never fires is not a gate), `std_part_iso4014_m12` (s = 18, k = 7.5, read by value), `sheet_frame_iso5457_a3` (the frame runs 20,10 to 410,287 inside the 420 × 297 sheet) and `bom_balloon_link` (two identical bolts are one row of quantity 2, and its balloon carries the row's item number). The other six are the three P&ID checks of track A and, from track E, `settings_dimstyle_iso25_values` (the ISO-25 preset lands in the DIMSTYLE table with ISO 129-1's numbers), `settings_layer_state_roundtrip` (save → change → restore puts the layer table back, and the state survives save/reopen because it is an XRECORD in the file) and `settings_pdf_mediabox_a3` (the plotted PDF's own `/MediaBox` reads 420 × 297). The 26 checks v1.5.1 was gated on all still pass. Against the older `v1.4.0` baseline the same 26 reported **21 / 26 → 26 / 26, five fixed, zero regressed** — two of those were repaired defects (`fail → pass`), the diameter and radius callouts, which measured the leader as geometry and dimensioned a 40 mm bore as 60 at default settings. +The 26 checks v1.5.1 was released on all still pass. The twenty-two fixed rows are new capability (`miss → pass`), and each one pins a number computed by hand, not read back from the code that drew it: + +
+The twenty-two checks 1.6 added + +| Track | Check | Verified against | +|---|---|---| +| Understanding | `scale_check_detects_schematic` | the synthetic P&ID reads **schematic** against its layout; a uniform 2× copy of six tags reads **to scale** with factor 2 | +| | `pipe_takeoff_rmst_exact` | all eight routable runs measure exactly the rectilinear MST of their tags | +| | `cable_takeoff_roundup` | 9, 17, 19 and 10 m after the 20 % allowance; the heater's unstated power stays empty | +| | `diff_detects_known_edits` | a move, a text change, an attribute change, an add and a delete, and nothing else | +| | `topology_known_defects` | a 5 mm near miss, one interior crossing, and a clean T that is neither | +| Architecture | `arch_junction_l_t_x` | an L corner mitres at (4100, −100) / (3900, 100), a T stem stops on the near face, an X cuts all four faces; wall areas 1 400 000, 1 490 000 and 2 360 000 mm² | +| | `arch_room_area_net` | a 4 × 5 m room between 200 mm walls is labelled 20.00 m², not the 21.84 m² its axes enclose | +| | `arch_opening_cuts_wall` | both faces are interrupted across a 900 door and two jambs close it | +| | `arch_rooms_detect_foreign` | plain lines on a `WALLS` layer yield 20 and 15 m² at confidence 0.6, and nothing is written | +| Mechanical and sheet | `mech_part_roundtrip` | a part read back out of its own `ACADMCP_MECH` XDATA equals the part drawn | +| | `mech_section_hatch_area` | a 60 × ⌀40 sleeve with a ⌀20 bore cuts 1200 mm², measured out of the drawing by `analysis_measure_entity` | +| | `mech_iso286_on_dimension` | 40 H7 is +0.025 / 0 and survives the layout | +| | `mech_thread_unrepresented_is_caught` | the ISO 6410 focus fires on a thread drawn as a plain circle. A gate that never fires is not a gate | +| | `std_part_iso4014_m12` | s = 18, k = 7.5, read by value | +| | `sheet_frame_iso5457_a3` | the frame runs 20,10 to 410,287 inside the 420 × 297 sheet | +| | `bom_balloon_link` | two identical bolts are one row of quantity 2, and its balloon carries the row's item number | +| Environment | `settings_dimstyle_iso25_values` | the ISO-25 preset lands in the DIMSTYLE table with ISO 129-1's numbers | +| | `settings_layer_state_roundtrip` | save → change → restore puts the layer table back, and the state survives save/reopen | +| | `settings_pdf_mediabox_a3` | the plotted PDF's own `/MediaBox` reads 420 × 297 | +| P&ID | `pid_block_define_attdef_roundtrip`, `pid_tag_parse_fic`, `pid_graph_edge_count` | a block with ATTDEFs defined and read back, an ISA-5.1 tag parsed, a drawn P&ID's edge count | + +
### The task matrix — tasks that can fail -An earlier matrix scored this server 10/10, which carried no information: every task in it exercised something the server was built around. Five were added in 1.5 because they *can* fail, and three did while being written; 1.6 adds six more — one the P&ID reader can fail on its own, one where the PDF file, not the setter, is the witness, one where a whole mechanical sheet has to come out clean, one where a floor plan's rooms are read back off the drawing, and two where a foreign plant's P&ID and layout are read: the takeoffs against lengths computed in advance, and the one-call report against the defects planted in it. +An earlier matrix scored this server 10/10, which carried no information: every task in it exercised something the server was built around. Five tasks were added in 1.5 because they *can* fail, and three of them did while being written. 1.6 adds six more: + +- one the P&ID reader can fail on its own; +- one where the PDF file, not the setter, is the witness; +- one where a whole mechanical sheet has to come out clean; +- one where a floor plan's rooms are read back off the drawing; +- two where a foreign plant's P&ID and layout are read: the takeoffs against lengths computed in advance, and the one-call report against the defects planted in it. | Task | Verified against | |---|---| | `tool_discovery` | six AutoCAD command names, each ranking #1 | -| `token_budget` | 83,758 → 356 tokens, against a ceiling fixed in advance | +| `token_budget` | 83,921 → 356 tokens, against a ceiling fixed in advance | | `hatch_islands` | 300 filled with the island, 400 ignoring it | | `selection_filter` | window 1, crossing 2, bounding box 3, polygon 1 | | `measure_from_handle` | 139.2699 against the 100.0 a vertex shoelace gives | | `pid_roundtrip` | the example sheet drawn by `pid_from_spec`, read back by `pid_graph`, which never sees the spec: 5 nodes, 4 edges, 0 dangling, `confidence_min` 1.0, `FIC-101` wired to `FCV-101` | -| `page_setup_truth` | ANSI B then ISO A3 landscape applied to Layout1, each plotted through `batch_plot` and read back from its PDF's `/MediaBox`: 432 × 279 then 420 × 297 mm, not the setter's return value (a fresh layout is already A3, so the B sheet is what a no-op setter cannot fake) | -| `mech_assembly` | a flange-coupling A3 sheet — hub with a full section, flange with its bolt-circle end view, two ISO 4014 bolts with ISO 4032 nuts, an ISO 7573 parts list read off the drawing, linked balloons, ISO 5457 frame, ISO 7200 title block — gated on `drawing_critique(focus=None)` returning **zero** issues and a finalize score of at least **90**; a test drops one balloon and watches the gate fail | -| `arch_roundtrip` | a two-room plan — an entrance door, an interior door, two windows, a stair — drawn by `arch_plan_from_spec` and read back by `arch_rooms_detect`, which never sees the spec: each room within 0.1 % of its label and of the net floor computed by hand (4825 × 5750 and 3825 × 5750 mm), the door and window schedules listing D1, D2, W1, W2, `drawing_critique(focus=None)` at **zero** and a finalize score of at least **90**; a test removes one room label and watches the gate fail | -| `takeoff_roundtrip` | the synthetic plant pair — a P&ID stretched 1.3× in one room and rearranged in the other, Russian supply / return words, a return drawn on the supply layer, a reducer, a segment drawn twice — read by the scale check (**schematic**: 21.8 % of 55 pairs within ±10 %), `pipe_takeoff` (every routable run's layout length equal to the rectilinear MST of its tags, sizes and services as drawn, the run to a tag the layout lacks status C) and `cable_takeoff` (9 / 17 / 19 / 10 m after the 20 % allowance, the heater's power left empty); a test removes the wiring callouts and watches the cable half fail by name | -| `understand_foreign` | the pair's layout — inches declared over millimetres, a stray line 5,000 km out, two plan copies — comes back as millimetres with an `INSUNITS` warning, the stray line's handle and two clusters; the P&ID's `P_product piping` / `P_cipsupplyline` / `P_cipreturnline` / `P_ijswater` / `E_power` classify as product / CIP supply / CIP return / ice water / electrical; a test makes the layout declare millimetres and watches the units gate fail | +| `page_setup_truth` | ANSI B, then ISO A3 landscape, applied to Layout1. Each is plotted through `batch_plot` and read back from its PDF's `/MediaBox`: 432 × 279, then 420 × 297 mm, not the setter's return value. A fresh layout is already A3, so the B sheet is what a no-op setter cannot fake | +| `mech_assembly` | a flange-coupling A3 sheet: a hub with a full section, a flange with its bolt-circle end view, two ISO 4014 bolts with ISO 4032 nuts, an ISO 7573 parts list read off the drawing, linked balloons, an ISO 5457 frame and an ISO 7200 title block. Gated on `drawing_critique(focus=None)` returning **zero** issues and a finalize score of at least **90**; a test drops one balloon and watches the gate fail | +| `arch_roundtrip` | a two-room plan (an entrance door, an interior door, two windows, a stair) drawn by `arch_plan_from_spec` and read back by `arch_rooms_detect`, which never sees the spec. Each room must be within 0.1 % of its label and of the net floor computed by hand (4825 × 5750 and 3825 × 5750 mm), and the schedules must list D1, D2, W1, W2. Gated like `mech_assembly`; a test removes one room label and watches the gate fail | +| `takeoff_roundtrip` | the synthetic plant pair: a P&ID stretched 1.3× in one room and rearranged in the other, Russian supply / return words, a return drawn on the supply layer, a reducer, a segment drawn twice. The scale check must call it **schematic** (21.8 % of 55 pairs within ±10 %). `pipe_takeoff` must give every routable run's layout length equal to the rectilinear MST of its tags, sizes and services as drawn, and the run to a tag the layout lacks status C. `cable_takeoff` must give 9 / 17 / 19 / 10 m after the 20 % allowance and leave the heater's power empty. A test removes the wiring callouts and watches the cable half fail by name | +| `understand_foreign` | the pair's layout (inches declared over millimetres, a stray line 5,000 km out, two plan copies) comes back as millimetres with an `INSUNITS` warning, the stray line's handle and two clusters. The P&ID's `P_product piping` / `P_cipsupplyline` / `P_cipreturnline` / `P_ijswater` / `E_power` classify as product / CIP supply / CIP return / ice water / electrical. A test makes the layout declare millimetres and watches the units gate fail | ### Headless performance Performance across four headless workloads -Workloads call the same backend methods the MCP tools call, so server-side overhead is included. Numbers move with hardware; the report records the machine fingerprint. **Read the next section before quoting them** — this release is slower than v1.4.0 at creating entities, on purpose. +The workloads call the same backend methods the MCP tools call, so server-side overhead is included. Numbers move with hardware; the report records the machine fingerprint. **Read the next section before quoting them.** This release is slower than v1.4.0 at creating entities. ## What this release is bad at A page that only lists strengths is a page that has not been measured. -**Entity creation is slower than v1.4.0.** Median of three runs on one machine and interpreter: 2,000 lines **1.4× slower**, the 10,000-line roundtrip **2.3× slower**. Attributed — setting `EZDXF_CALL_TIMEOUT=0` returns creation to v1.4.0's numbers. The cost is the per-call `asyncio.wait_for` wrapped around every headless call so one hung call can no longer wedge a server whose document lock is a single `asyncio.Lock`. Deliberate trade, documented knob, on the 1.6 roadmap. The premium quality pass went the other way — **3.7× faster**. +**Headless entity creation is slower than v1.4.0, and 1.6 did not win it back.** These are medians of five alternating runs on one machine, with one interpreter (CPython 3.11.15) and one ezdxf, v1.4.0's code against 1.6.0's ([`perf-v1.4.0-vs-v1.6.0.json`](https://github.com/U-C4N/Autocad-MCP/blob/main/benchmarks/results/published/perf-v1.4.0-vs-v1.6.0.json)): -**Wave A shipped 13 tools of a planned 39.** Every cut is backed by a measurement. The REGION and 2D-boolean family went because `add_region()` produces a REGION with **zero ACIS bytes**, and the `greiner_hormann` substitute loses **28.2%** of the area on a square with one semicircular edge. +| Workload | v1.4.0 | v1.6.0 | v1.6.0, `EZDXF_CALL_TIMEOUT=0` | +|---|---:|---:|---:| +| 2,000 lines, one call each | 244.5 ms | 306.1 ms (1.25×) | 284.9 ms (1.17×) | +| 10,000 lines: build, DXF export, reopen | 1,847.9 ms | 2,553.5 ms (1.38×) | 2,216.4 ms (1.20×) | +| Region query over 10,000 entities | 665.2 ms | 712.2 ms (1.07×) | 783.9 ms (1.18×) | +| Premium quality pass | 150.6 ms | 65.3 ms (**2.3× faster**) | 69.9 ms | -**`system_run_command` / `system_run_lisp` are a guardrail, not a security boundary** — the rejection message says so in those words. A 36-verb denylist refuses the obvious cases, but AutoCAD accepts hundreds of commands and `DANGEROUS_COMMANDS_ENABLED=true` switches it off entirely. +The per-call timeout stops one hung call from wedging a server whose document lock is a single `asyncio.Lock`. It explains only part of the slowdown: switched off, creation is still 1.17× and 1.20× slower. The rest is extra work per call that 1.6 has not removed. A cProfile of the event-loop thread counts 491,071 function calls for the 2,000 creates, against v1.4.0's 319,038. Earlier READMEs said that switching the timeout off returned creation to v1.4.0's numbers. Measured this way, it does not. The region-query row, which moves the wrong way, shows this machine's noise floor (about ±10 %). Roadmap criterion 7 is not met and moves to 1.7. -**Path validation is per-tool, and unscoped until you scope it.** With `ALLOWED_PATHS` empty — the default — the only positive bound is a ten-entry system-directory denylist that does not include `C:/Users`, `/home`, `/root` or `/var`. **Set `ALLOWED_PATHS`.** +**Four mechanical feature tables ship empty.** DIN 471/472 retaining-ring grooves, DIN 509 undercuts, DIN 332 centre holes and ISO 3601-2 O-ring grooves have their drawing code and their provenance tests, but no rows. A groove diameter written from memory is a wrong workshop drawing, while a narrow table is merely narrow. So every size is refused by name until rows verified against the standard's own table are transcribed. That is a data edit; each module's docstring says how. + +**Takeoffs are estimates, and they say so.** + +- A layout length is the rectilinear minimum spanning tree of the tags a run connects, plus a stated vertical allowance, and it is labelled an estimate. +- A P&ID length is used only when `drawing_scale_check` calls the P&ID to scale. +- A power the P&ID does not state stays empty and becomes an open item, and no cable-section table is authored: a section comes only from your `section_rules`. +- Where no callout names a panel, the nearest panel is inferred and flagged. +- Detail copies of equipment are flagged, and excluded only when you ask. + +The readers were tuned on the synthetic plant pair and on one real P&ID / layout pair that is not in this repository. Another drafting office's layer and text conventions may need vocabulary. + +**No Plant 3D.** The roadmap's track D (read-only queries of a Plant 3D project) was dropped: the maintainer does not use Plant 3D, and a reader tested against nobody's real projects would be a guess. Roadmap criterion 5 is withdrawn, not met. + +**No REGION or 2D booleans headlessly.** `add_region()` produces a REGION with **zero ACIS bytes**, and the `greiner_hormann` substitute loses **28.2%** of the area on a square with one semicircular edge. + +**`system_run_command` / `system_run_lisp` are a guardrail, not a security boundary.** The rejection message says so in those words. A 36-verb denylist refuses the obvious cases, but AutoCAD accepts hundreds of commands, and `DANGEROUS_COMMANDS_ENABLED=true` switches the list off entirely. + +**Path validation is per-tool, and unscoped until you scope it.** With `ALLOWED_PATHS` empty (the default), the only positive bound is a ten-entry system-directory denylist that does not include `C:/Users`, `/home`, `/root` or `/var`. **Set `ALLOWED_PATHS`.** **Non-AutoCAD ProgIDs are unverified.** `CAD_PROGID` changes which COM application the backend attaches to; nothing beyond the connection has been tested against BricsCAD, ZWCAD or GstarCAD. ### Known limitations of the P&ID track -**The router is orthogonal and blind.** `pid_line_draw` picks `auto` from seven orthogonal candidates — straight, one bend, two bends through the midline, or a stub out of each port joined by three bends — by fewest bends then length (or takes `direct`, or the waypoints you pass), and *counts* the lines it crosses — `crossings` comes back in every result and `pid_from_spec` sums it — but it does not route around them. Obstacle-avoiding routing is out of 1.6; move the symbol or pass waypoints. +**The router is orthogonal and blind.** `pid_line_draw` picks `auto` from seven orthogonal candidates, by fewest bends then length: -**A foreign drawing is read at the confidence it earns, and no higher.** A block this server did not place classifies by its `TAG`-style attribute or a keyword in its name at 0.6 with inferred ports, or at 0.3 when a line merely touches it — and a 0.3 guess can neither raise nor suppress a critique finding. On the live engine an INSERT turned off a right angle whose members ActiveX cannot measure reports an *approximate* box; `pid_graph` takes the tighter wall of that and the attribute-inclusive box on every side, but a line ending on such a body's true edge can still be reported `dangling` with the nearest port as the hint. Read `stats.confidence_min` first. +- straight; +- one bend; +- two bends through the midline; +- a stub out of each port joined by three bends. -**Same-file only, DXF headlessly.** Off-page connectors link within one drawing; links across files are not resolved. The headless engine reads DXF; a DWG P&ID is read through the live backend. No DEXPI/Proteus export and no ISA-5.2 binary-logic symbols in this release, recorded in the spec so nobody re-derives the cut. +You can also take `direct`, or pass waypoints. It *counts* the lines it crosses (`crossings` comes back in every result, and `pid_from_spec` sums it), but it does not route around them. Obstacle-avoiding routing is not in 1.6; move the symbol or pass waypoints. + +**A foreign drawing is read at the confidence it earns, and no higher.** A block this server did not place classifies at 0.6, with inferred ports, by its `TAG`-style attribute or a keyword in its name. It classifies at 0.3 when a line merely touches it, and a 0.3 guess can neither raise nor suppress a critique finding. + +On the live engine, an INSERT turned off a right angle has members ActiveX cannot measure, so it reports an *approximate* box. `pid_graph` takes the tighter wall of that box and the attribute-inclusive box on every side. Even so, a line ending on such a body's true edge can still be reported `dangling`, with the nearest port as the hint. Read `stats.confidence_min` first. + +**Same-file only, DXF headlessly.** Off-page connectors link within one drawing; links across files are not resolved. The headless engine reads DXF; a DWG P&ID is read through the live backend. There is no DEXPI/Proteus export and there are no ISA-5.2 binary-logic symbols in this release. Both cuts are recorded in the spec so nobody re-derives them. ### Known limitations of the settings track -**Layer states are ours.** `layer_state_save` writes a portable snapshot into an `ACADMCP_LAYERSTATES` XRECORD. It survives save/reopen on both engines and travels with the DWG/DXF; it is **not** an AutoCAD `LAYERSTATE` and does not appear in the Layer States Manager. `system_capabilities` reports `layer_states` as `mode: "xrecord"` for that reason. +**Layer states are ours.** `layer_state_save` writes a portable snapshot into an `ACADMCP_LAYERSTATES` XRECORD. It survives save/reopen on both engines and travels with the DWG/DXF. It is **not** an AutoCAD `LAYERSTATE` and does not appear in the Layer States Manager, which is why `system_capabilities` reports `layer_states` as `mode: "xrecord"`. -**Six of the 22 environment tools need a live seat.** Preferences, launching, the operator prompts, and the five summary fields of document properties are refused headlessly with `preferences` / `dwgprops` / `live_application` / `interactive_prompt` — a refusal, not a stub. A `.dwt` template is the same case (`dwt_write`): a `.dwt` is a DWG container, so headlessly the template is saved as DXF. `mleaderstyle_create` is *not* such a case — ActiveX has no MLeaderStyle collection, but the `ACAD_MLEADERSTYLE` dictionary holds full `IAcadMLeaderStyle` objects, so both engines create leader styles. +**Six of the 22 environment tools need a live seat.** Preferences, launching, the operator prompts and the five summary fields of document properties are refused headlessly with `preferences` / `dwgprops` / `live_application` / `interactive_prompt`. Each is a refusal, not a stub. -**A headless dimension does not re-render.** `dimstyle_modify` reports `dimensions_using_style` and `rerender_required: true` on the ezdxf engine; the DIMENSION keeps its rendered block until it is redrawn. AutoCAD regenerates on the next regen. +A `.dwt` template is the same case (`dwt_write`): a `.dwt` is a DWG container, so headlessly the template is saved as DXF. `mleaderstyle_create` is *not* such a case. ActiveX has no MLeaderStyle collection, but the `ACAD_MLEADERSTYLE` dictionary holds full `IAcadMLeaderStyle` objects, so both engines create leader styles. + +**A headless dimension does not re-render.** On the ezdxf engine, `dimstyle_modify` reports `dimensions_using_style` and `rerender_required: true`, and the DIMENSION keeps its rendered block until it is redrawn. AutoCAD regenerates on the next regen. + +### Known limitations of the architecture track + +**Nominal, not standard.** The furniture and sanitary blocks carry nominal catalogue sizes and say so. The stair tool reports the Blondel relation (2R + G within 600–650 mm) as a design rule, never as a building code. The `arch` layer colours are a repository convention: ISO 13567 fixes the names, not the colours. Labels speak English or Turkish; any other `lang` is refused with the list. ## Configuration @@ -261,7 +394,7 @@ Nothing loads a `.env` file — export these, or set them in your MCP client's ` flowchart LR A[MCP host / AI agent] --> B[FastMCP 3 server] B --> C[Error · audit · timing · logging · capability-refusal middleware] - C --> D[Typed contract · 19 modules · 27 capabilities] + C --> D[Typed contract · 23 modules · 41 capability keys] D --> E[COM backend
single-STA thread] D --> F[ezdxf backend
asyncio.to_thread] E --> G[Live AutoCAD] @@ -305,11 +438,16 @@ uv run pytest uv run ruff check . && uv run ruff format --check . ``` -`uv.lock` pins the whole transitive graph, so this reproduces CI exactly. CI runs Linux (3.11 / 3.12), Windows (mocked-COM), plus package, Docker and MCP-registry jobs; the release gate runs the same three test lanes before anything reaches PyPI. Releases are tag-driven: `git tag vX.Y.Z && git push origin vX.Y.Z`. +`uv.lock` pins the whole transitive graph, so this reproduces CI exactly. CI runs Linux (3.11 / 3.12) and Windows (mocked COM), plus package, Docker and MCP-registry jobs. The release gate runs the same three test lanes before anything reaches PyPI. Releases are tag-driven: `git tag vX.Y.Z && git push origin vX.Y.Z`. -## Roadmap (1.6) +## Roadmap (1.7) -Give back the creation throughput the per-call timeout costs — arm it only for calls that can actually block. ISO 286 interference **shafts** r/s/t/u. HATCH boundary *geometry* in `entity_get` (1.5 measures a hatch's filled area but does not hand back its loops). REGION/3DSOLID area headlessly, which needs a modelling kernel ezdxf does not have. An *allowlist* of permitted AutoLISP heads, replacing the denylist — enumerating dangerous symbols does not terminate. `ezdxf.recover` as a fallback on `drawing_open`. +- **Speed.** Win back the headless creation throughput that 1.6 lost against v1.4.0 (criterion 7): arm the per-call timeout only where a call can block, and remove the per-call work that came with it. +- **IEC 60617 electrical schematics.** They reuse the P&ID track's symbol, port and graph machinery. +- **Transcribed rows** for DIN 471/472, DIN 509, DIN 332 and ISO 3601-2, each verified against the standard's own table. +- **HATCH boundary geometry in `entity_get`.** 1.5 and 1.6 measure a hatch's filled area but do not hand back its loops. +- **`ezdxf.recover` as a fallback** on `drawing_open`. +- **An allowlist of permitted AutoLISP heads**, replacing the denylist. Enumerating dangerous symbols does not terminate. Features ship when their contracts and limitations are testable — not when they make a longer checklist. diff --git a/benchmarks/README.md b/benchmarks/README.md index 62a3fe7..a134354 100644 --- a/benchmarks/README.md +++ b/benchmarks/README.md @@ -41,13 +41,14 @@ Unsupported tasks remain in the fixed-matrix denominator with score zero, preventing partial implementations from receiving an inflated score. ```bash -python -m benchmarks.run_competitors --list --matrix v5 -python -m benchmarks.run_competitors --server autocad-mcp-pro --backend ezdxf --matrix v5 --json +python -m benchmarks.run_competitors --list --matrix v7 +python -m benchmarks.run_competitors --server autocad-mcp-pro --backend ezdxf --matrix v7 --json python -m benchmarks.run_competitors --task table_mleader --task hatch_islands --json ``` -The release-machine ezdxf self-check is **18/18 (100.0)** on the v5 matrix -(`--matrix v4` / `--matrix v3` / `--matrix v2` reproduce the earlier sets). +The release-machine ezdxf self-check is **21/21 (100.0)** on the v7 matrix +(`--matrix v6` … `--matrix v2` reproduce the earlier sets); the published +report is [`results/published/autocad-mcp-pro.json`](results/published/autocad-mcp-pro.json). Repository stars and raw tool counts do not contribute to the score. Adapter registration lives in `competitors.yaml`. @@ -145,13 +146,13 @@ MCP stdio, against commits pinned in `competitors.yaml`: | Server | Matrix | Pinned | Score | Pass | Coverage | |---|---|---|---:|---:|---:| -| autocad-mcp-pro (reference) | v3 (15) | working tree | 100.0 | 15/15 | 100% | +| autocad-mcp-pro (reference) | v7 (21) | 1.6.0 tree | 100.0 | 21/21 | 100% | | beiming183-cloud/AutoCAD-MCP | v2 (10) | `11f7c47e` | 50.0 | 5/10 | 50% | | puran-water/autocad-mcp | v2 (10) | `95476a33` | 45.0 | 4/10 | 50% | The competitor rows are the v1.4 runs, unchanged. Read the matrix column -before the score column: the denominators differ because the five v3 tasks -were added after those runs and have not been put to them. +before the score column: the denominators differ because the eleven tasks +added since v2 came after those runs and have not been put to them. Method and boundaries: @@ -205,38 +206,79 @@ python -m benchmarks.perf_suite --out benchmarks/results/published/perf-ezdxf.js python -m benchmarks.render_perf_chart ``` -### What the call-timeout guard costs +### v1.4.0 against v1.6.0, and what the call-timeout guard costs The published v1.4 report was recorded on CPython 3.14 and the later ones on -3.11.15, so comparing those two files directly shows an apparent 2–6× speedup -that is almost entirely the interpreter. The comparison that means something is -the guard against no guard, on one interpreter, back to back. Three runs of -each configuration, median taken by hand — `perf_suite` executes each workload -once per invocation and cannot produce a median itself: +3.11.15, so comparing those files directly shows an apparent speedup that is +almost entirely the interpreter. The comparison that means something runs one +interpreter and one ezdxf over each tree's own code: 1.6.0's `perf_suite.py` +(v1.4.0 had none) against a v1.4.0 worktree, against 1.6.0, and against 1.6.0 +with the guard off — five rounds, the three configurations alternating within +a round. Medians, from +[`results/published/perf-v1.4.0-vs-v1.6.0.json`](results/published/perf-v1.4.0-vs-v1.6.0.json), +which keeps every run: -| Workload | `EZDXF_CALL_TIMEOUT=0` | default (120 s) | Cost | +| Workload | v1.4.0 | v1.6.0 | v1.6.0, `EZDXF_CALL_TIMEOUT=0` | |---|---:|---:|---:| -| `create_lines_2k` | 243.7 ms | 294.3 ms | **+20.8%** | -| `roundtrip_10k` | 1,709.6 ms | 2,049.2 ms | **+19.9%** | +| `create_lines_2k` | 244.5 ms | 306.1 ms (1.25×) | 284.9 ms (1.17×) | +| `roundtrip_10k` | 1,847.9 ms | 2,553.5 ms (1.38×) | 2,216.4 ms (1.20×) | +| `region_query_10k` | 665.2 ms | 712.2 ms (1.07×) | 783.9 ms (1.18×) | +| `premium_pass` | 150.6 ms | 65.3 ms (0.43×) | 69.9 ms (0.46×) | -v1.5.0 paid 28.6% on the first workload; v1.5.1 rebuilt the wrapper around a -single `asyncio.timeout_at` instead of two `asyncio.wait_for`s — `wait_for` -wraps its awaitable in a Task, so the old shape created two extra Tasks and two -timer handles per call — which recovered about a third of the overhead. The -rest is the Task the abandon-on-timeout design genuinely needs. The guard is -what stops one hung ezdxf call wedging a server whose document lock is a single -`asyncio.Lock`, so it stays; the knob is documented rather than hidden. +The guard explains part of the creation cost, not all of it. v1.5.1 rebuilt +the wrapper around one `asyncio.timeout_at` instead of two `asyncio.wait_for`s +(`wait_for` wraps its awaitable in a Task, so the old shape created two extra +Tasks and two timer handles per call), which recovered about a third of its +overhead; the rest is the Task the abandon-on-timeout design needs. With the +guard off, creation is still 1.17× and 1.20× v1.4.0's time. A cProfile of the +event-loop thread over the 2,000 creates counts **491,071** function calls on +1.6.0 against **319,038** on v1.4.0: per-call work added since 1.4 that 1.6 +has not removed. The region-query row, whose guard-off median is the slower +one, is this machine's noise floor (about ±10 %). -These figures are hand-collected and have **no published artifact** — -`perf-ezdxf.json` records one default-configuration run only. Reproduce: +Roadmap criterion 7 — headless creation throughput back to v1.4.0's numbers — +is therefore **not met** and moves to 1.7. Earlier versions of this page said +that `EZDXF_CALL_TIMEOUT=0` returned creation to v1.4.0's numbers; measured +this way, it does not. The guard stays: it is what stops one hung ezdxf call +wedging a server whose document lock is a single `asyncio.Lock`. + +`perf-ezdxf.json`, the chart's input, is the median of the five +default-configuration runs, with every run kept in `runs_ms`. + +Reproduce (the v1.4.0 tree needs 1.6.0's `benchmarks/perf_suite.py` copied in): ```bash -for i in 1 2 3; do - python -m benchmarks.perf_suite --out /tmp/on-$i.json - EZDXF_CALL_TIMEOUT=0 python -m benchmarks.perf_suite --out /tmp/off-$i.json +git worktree add ../v140 v1.4.0 && cp benchmarks/perf_suite.py ../v140/benchmarks/ +for i in 1 2 3 4 5; do + python -m benchmarks.perf_suite --json > main-$i.json + EZDXF_CALL_TIMEOUT=0 python -m benchmarks.perf_suite --json > off-$i.json + (cd ../v140 && python -m benchmarks.perf_suite --json) > v140-$i.json done ``` +The profile, run from each tree's root: + +```python +import asyncio, cProfile, pstats +from backends.ezdxf_backend import EzdxfBackend + + +async def work(): + b = EzdxfBackend() + await b.connect() + await b.drawing_new() + for i in range(2000): + await b.entity_create_line(0, i, 100, i) + await b.disconnect() + + +pr = cProfile.Profile() +pr.enable() +asyncio.run(work()) +pr.disable() +print(pstats.Stats(pr).total_calls) +``` + ## Correctness A/B suite `correctness_suite.py` is a set of deterministic, headless (ezdxf-backend) checks @@ -253,15 +295,15 @@ python benchmarks/compare_versions.py v1.0.0 # vs a tag/ref python benchmarks/compare_versions.py --json results.json ``` -### Result — this branch (1.6.0-dev) vs v1.5.1 (release gate) +### Result — v1.6.0 vs v1.5.1 (release gate) 48 checks, ezdxf backend, one subprocess per check. Machine-readable report: -[`results/published/ab-v1.5.1-vs-v1.6.0-dev.json`](results/published/ab-v1.5.1-vs-v1.6.0-dev.json). +[`results/published/ab-v1.5.1-vs-v1.6.0.json`](results/published/ab-v1.5.1-vs-v1.6.0.json). | Version | Checks passing | Pass rate | Fixed | Regressed | |---------|----------------|-----------|-------|-----------| | **v1.5.1** (baseline) | 26 / 48 | 54.2 % | — | — | -| **v1.6.0-dev** (this branch) | 48 / 48 | 100 % | 22 | **0** | +| **v1.6.0** (this release) | 48 / 48 | 100 % | 22 | **0** | The twenty-two are all `miss → pass` — v1.5.1 has none of the methods. Five are track H: `scale_check_detects_schematic` (the synthetic P&ID is schematic against its layout; a uniform 2× copy is to scale with factor 2), `pipe_takeoff_rmst_exact` (eight routable runs, each exactly the rectilinear MST of its tags), `cable_takeoff_roundup` (9 / 17 / 19 / 10 m, the unstated power left empty), `diff_detects_known_edits` (five known edits and nothing else) and `topology_known_defects` (one near miss, one crossing, a clean T). Four are track F: `arch_junction_l_t_x` (L, T and X junction outlines against hand-computed corners and wall areas), `arch_room_area_net` (a 4 × 5 m room between 200 mm walls is labelled 20.00 m², not its 21.84 m² axis area), `arch_opening_cuts_wall` (both faces interrupted across a door, two jambs) and `arch_rooms_detect_foreign` (plain lines on a WALLS layer read as 20 and 15 m² at confidence 0.6, nothing written). Seven are tracks B and G: `mech_part_roundtrip` (a part read back from its own @@ -286,7 +328,7 @@ through the lister; `settings_layer_state_roundtrip`: save → change → restor puts the layer table back and the state survives save/reopen as an XRECORD in the file; `settings_pdf_mediabox_a3`: the plotted PDF's own `/MediaBox` reads 420 × 297 mm). Every one of the 26 checks it was released on still passes, so -the four tracks added capability without moving a number it had already earned. +the five tracks added capability without moving a number it had already earned. ### Result — v1.5.1 vs v1.5.0 diff --git a/benchmarks/render_chart.py b/benchmarks/render_chart.py index 09b54f0..21025a8 100644 --- a/benchmarks/render_chart.py +++ b/benchmarks/render_chart.py @@ -95,7 +95,7 @@ def render_chart(data: dict[str, Any], output: Path) -> None: ax.set_title( "Public AutoCAD MCP leaderboard — Source-reviewed capability benchmark", loc="left", - fontsize=17, + fontsize=15, fontweight="bold", color="#e6edf3", pad=28, diff --git a/benchmarks/render_live_chart.py b/benchmarks/render_live_chart.py index 3fd2da1..82d352e 100644 --- a/benchmarks/render_live_chart.py +++ b/benchmarks/render_live_chart.py @@ -103,8 +103,8 @@ def render_chart(reports: list[dict[str, Any]], output: Path) -> None: ax.text( 0, -0.22, - "Unsupported tasks score 0 within a server's own matrix. The five v3 tasks were " - "added after the competitor runs and were never put to them; per-task statuses are " + "Unsupported tasks score 0 within a server's own matrix. The tasks added after v2 " + "were never put to the pinned competitor runs; per-task statuses are " "in benchmarks/results/published/.", transform=ax.transAxes, color="#8b949e", diff --git a/benchmarks/render_matrix_chart.py b/benchmarks/render_matrix_chart.py index 226befb..1c67e4f 100644 --- a/benchmarks/render_matrix_chart.py +++ b/benchmarks/render_matrix_chart.py @@ -133,8 +133,8 @@ def render_chart(reports: list[dict[str, Any]], output: Path) -> None: Patch(facecolor=STATUS_COLORS["fail"], label="fail"), Patch(facecolor=STATUS_COLORS["unsupported"], label="unsupported (no equivalent)"), # Real rows now, not a theoretical state: the competitor reports were - # pinned and run against the v2 matrix, so the five v3 tasks were - # never put to them. An invented zero would look identical to a + # pinned and run against the v2 matrix, so every task added since v2 + # was never put to them. An invented zero would look identical to a # measured one, so they stay blank and say why. Patch(facecolor=STATUS_COLORS["not_run"], edgecolor="#30363d", label="not run"), ] diff --git a/benchmarks/results/published/ab-v1.5.1-vs-v1.6.0.json b/benchmarks/results/published/ab-v1.5.1-vs-v1.6.0.json new file mode 100644 index 0000000..a22e996 --- /dev/null +++ b/benchmarks/results/published/ab-v1.5.1-vs-v1.6.0.json @@ -0,0 +1,302 @@ +{ + "summary": { + "baseline_ref": "v1.5.1", + "total": 48, + "old_pass": 26, + "new_pass": 48, + "old_pct": 54.2, + "new_pct": 100.0, + "fixed": 22, + "regressed": 0 + }, + "rows": [ + { + "check": "core_line_length", + "category": "Core", + "old": "pass", + "new": "pass" + }, + { + "check": "core_circle_radius", + "category": "Core", + "old": "pass", + "new": "pass" + }, + { + "check": "core_layer_create", + "category": "Core", + "old": "pass", + "new": "pass" + }, + { + "check": "core_polyline_closed", + "category": "Core", + "old": "pass", + "new": "pass" + }, + { + "check": "core_linear_dim", + "category": "Core", + "old": "pass", + "new": "pass" + }, + { + "check": "core_save_dxf_roundtrip", + "category": "Core", + "old": "pass", + "new": "pass" + }, + { + "check": "dim_aligned_no_error", + "category": "Dimensions", + "old": "pass", + "new": "pass" + }, + { + "check": "dim_angular_no_error", + "category": "Dimensions", + "old": "pass", + "new": "pass" + }, + { + "check": "array_polar_360_distinct", + "category": "Modify", + "old": "pass", + "new": "pass" + }, + { + "check": "point_intersection_line_line", + "category": "Geometry", + "old": "pass", + "new": "pass" + }, + { + "check": "point_tangent_external", + "category": "Geometry", + "old": "pass", + "new": "pass" + }, + { + "check": "arc_has_length", + "category": "Query", + "old": "pass", + "new": "pass" + }, + { + "check": "arc_select_by_length", + "category": "Query", + "old": "pass", + "new": "pass" + }, + { + "check": "ezdxf_bounding_box", + "category": "Query", + "old": "pass", + "new": "pass" + }, + { + "check": "mtext_rotation_roundtrip", + "category": "Entities", + "old": "pass", + "new": "pass" + }, + { + "check": "screenshot_png", + "category": "Render", + "old": "pass", + "new": "pass" + }, + { + "check": "dim_overlap_critique_fires", + "category": "Quality gate", + "old": "pass", + "new": "pass" + }, + { + "check": "iso13567_dim_layer", + "category": "Quality gate", + "old": "pass", + "new": "pass" + }, + { + "check": "construction_left_iso_caught", + "category": "Quality gate", + "old": "pass", + "new": "pass" + }, + { + "check": "gear_no_self_overlap", + "category": "Engineering", + "old": "pass", + "new": "pass" + }, + { + "check": "center_linetype_applied", + "category": "Entities", + "old": "pass", + "new": "pass" + }, + { + "check": "hatch_area_subtracts_its_island", + "category": "Measurement", + "old": "pass", + "new": "pass" + }, + { + "check": "boundary_area_agrees_with_measure", + "category": "Measurement", + "old": "pass", + "new": "pass" + }, + { + "check": "two_vertex_circle_has_area", + "category": "Measurement", + "old": "pass", + "new": "pass" + }, + { + "check": "diameter_dim_measures_the_diameter", + "category": "Dimensions", + "old": "pass", + "new": "pass" + }, + { + "check": "radius_dim_ignores_the_leader_length", + "category": "Dimensions", + "old": "pass", + "new": "pass" + }, + { + "check": "pid_block_define_attdef_roundtrip", + "category": "P&ID", + "old": "miss", + "new": "pass" + }, + { + "check": "pid_tag_parse_fic", + "category": "P&ID", + "old": "miss", + "new": "pass" + }, + { + "check": "pid_graph_edge_count", + "category": "P&ID", + "old": "miss", + "new": "pass" + }, + { + "check": "settings_dimstyle_iso25_values", + "category": "Settings", + "old": "miss", + "new": "pass" + }, + { + "check": "settings_layer_state_roundtrip", + "category": "Settings", + "old": "miss", + "new": "pass" + }, + { + "check": "settings_pdf_mediabox_a3", + "category": "Settings", + "old": "miss", + "new": "pass" + }, + { + "check": "mech_part_roundtrip", + "category": "Mechanical", + "old": "miss", + "new": "pass" + }, + { + "check": "mech_section_hatch_area", + "category": "Mechanical", + "old": "miss", + "new": "pass" + }, + { + "check": "mech_iso286_on_dimension", + "category": "Mechanical", + "old": "miss", + "new": "pass" + }, + { + "check": "mech_thread_unrepresented_is_caught", + "category": "Mechanical", + "old": "miss", + "new": "pass" + }, + { + "check": "std_part_iso4014_m12", + "category": "Mechanical", + "old": "miss", + "new": "pass" + }, + { + "check": "sheet_frame_iso5457_a3", + "category": "Sheet", + "old": "miss", + "new": "pass" + }, + { + "check": "bom_balloon_link", + "category": "Sheet", + "old": "miss", + "new": "pass" + }, + { + "check": "arch_junction_l_t_x", + "category": "Architecture", + "old": "miss", + "new": "pass" + }, + { + "check": "arch_room_area_net", + "category": "Architecture", + "old": "miss", + "new": "pass" + }, + { + "check": "arch_opening_cuts_wall", + "category": "Architecture", + "old": "miss", + "new": "pass" + }, + { + "check": "arch_rooms_detect_foreign", + "category": "Architecture", + "old": "miss", + "new": "pass" + }, + { + "check": "scale_check_detects_schematic", + "category": "Understanding", + "old": "miss", + "new": "pass" + }, + { + "check": "pipe_takeoff_rmst_exact", + "category": "Understanding", + "old": "miss", + "new": "pass" + }, + { + "check": "cable_takeoff_roundup", + "category": "Understanding", + "old": "miss", + "new": "pass" + }, + { + "check": "diff_detects_known_edits", + "category": "Understanding", + "old": "miss", + "new": "pass" + }, + { + "check": "topology_known_defects", + "category": "Understanding", + "old": "miss", + "new": "pass" + } + ] +} \ No newline at end of file diff --git a/benchmarks/results/published/autocad-mcp-pro.json b/benchmarks/results/published/autocad-mcp-pro.json index e94e570..1099d32 100644 --- a/benchmarks/results/published/autocad-mcp-pro.json +++ b/benchmarks/results/published/autocad-mcp-pro.json @@ -25,6 +25,11 @@ "reason": null, "supported": true }, + "documents": { + "mode": "registry", + "reason": null, + "supported": true + }, "drawing_2d": { "mode": "native", "reason": null, @@ -35,11 +40,31 @@ "reason": "no_dwg_writer_requires_live_autocad", "supported": false }, + "dwg_write": { + "mode": null, + "reason": "requires_oda_file_converter:https://www.opendesign.com/guestfiles/oda_file_converter", + "supported": false + }, + "dwgprops": { + "mode": null, + "reason": "summary_fields_need_live_autocad;custom_properties_need_r2004_or_newer", + "supported": false + }, + "dwt_write": { + "mode": null, + "reason": "dwt_is_a_dwg_container_requires_live_autocad", + "supported": false + }, "dxf": { "mode": "native", "reason": null, "supported": true }, + "explode_opaque_members": { + "mode": null, + "reason": "ezdxf_cannot_transform:OLE2FRAME,VIEWPORT,ACAD_PROXY_ENTITY_without_proxy_graphic;refused_before_writing", + "supported": false + }, "handle_overlay": { "mode": "rendered", "reason": null, @@ -50,11 +75,26 @@ "reason": null, "supported": true }, + "interactive_prompt": { + "mode": null, + "reason": "no_operator_to_prompt_headlessly", + "supported": false + }, + "layer_states": { + "mode": "xrecord", + "reason": "portable_acadmcp_xrecord;not_listed_in_autocad_layer_states_manager", + "supported": true + }, "lisp": { "mode": null, "reason": "live_com_only", "supported": false }, + "live_application": { + "mode": null, + "reason": "headless_engine_has_no_application_to_launch_or_attach", + "supported": false + }, "measure_area_acis": { "mode": null, "reason": "acis_evaluation_requires_live_autocad", @@ -70,6 +110,11 @@ "reason": null, "supported": true }, + "named_views": { + "mode": "vport_active", + "reason": "restore_sets_the_view_the_file_opens_on;no_live_display", + "supported": true + }, "ocs_normalized": { "mode": "wcs", "reason": "reported_xy_is_wcs;z_component_not_reported;text_rotation_stays_in_entity_frame;hatch_boundaries_not_reported", @@ -95,6 +140,11 @@ "reason": null, "supported": true }, + "preferences": { + "mode": null, + "reason": "preferences_live_in_the_running_application_not_in_a_file", + "supported": false + }, "preflight": { "mode": "shared", "reason": null, @@ -105,6 +155,11 @@ "reason": null, "supported": true }, + "registry_sysvar": { + "mode": null, + "reason": "registry_saved_variables_have_no_home_in_a_file", + "supported": false + }, "revcloud": { "mode": "native", "reason": "ezdxf_revcloud_helper", @@ -125,6 +180,11 @@ "reason": null, "supported": true }, + "ucs": { + "mode": "stored_not_interpreted", + "reason": "tool_coordinates_stay_wcs", + "supported": true + }, "undo_history": { "mode": null, "reason": "set_EZDXF_UNDO_DEPTH_to_enable", @@ -139,6 +199,16 @@ "mode": "native", "reason": null, "supported": true + }, + "xlsx_write": { + "mode": "openpyxl", + "reason": null, + "supported": true + }, + "xref_live": { + "mode": null, + "reason": "reload_and_bind_need_a_live_seat;list_detach_path_are_headless", + "supported": false } } } @@ -149,13 +219,13 @@ "platform": "Windows-10-10.0.26200-SP0", "python": "3.11.15" }, - "generated_at": "2026-08-06T06:27:59.080169+00:00", - "git_sha": "d5a0ff5109ddbd37d88371c10a1ed087201de4b5", - "matrix": "v3", + "generated_at": "2026-09-24T20:58:06.608758+00:00", + "git_sha": "554daff8d08c7d891974e55db046e7c70c15e185", + "matrix": "v7", "results": [ { "artifacts": [], - "duration_ms": 2.49, + "duration_ms": 2.71, "message": "", "metrics": { "line_length": 5.0 @@ -168,7 +238,7 @@ }, { "artifacts": [], - "duration_ms": 1.41, + "duration_ms": 1.92, "message": "", "metrics": {}, "score": 100.0, @@ -179,7 +249,7 @@ }, { "artifacts": [], - "duration_ms": 3.31, + "duration_ms": 3.38, "message": "", "metrics": {}, "score": 100.0, @@ -190,7 +260,7 @@ }, { "artifacts": [], - "duration_ms": 17.55, + "duration_ms": 24.95, "message": "", "metrics": { "entity_type": "DIMENSION" @@ -203,7 +273,7 @@ }, { "artifacts": [], - "duration_ms": 3.7, + "duration_ms": 3.64, "message": "", "metrics": { "representations": [ @@ -219,7 +289,7 @@ }, { "artifacts": [], - "duration_ms": 29.09, + "duration_ms": 8.37, "message": "", "metrics": { "entity_count": 0 @@ -232,7 +302,7 @@ }, { "artifacts": [], - "duration_ms": 1.2, + "duration_ms": 0.85, "message": "", "metrics": {}, "score": 100.0, @@ -243,7 +313,7 @@ }, { "artifacts": [], - "duration_ms": 8.42, + "duration_ms": 17.45, "message": "", "metrics": { "final_score": 100.0, @@ -286,11 +356,11 @@ "artifacts": [ { "filename": "roundtrip.dxf", - "sha256": "d11dfce5eddaf849075c62854a8526efaaa3bbd779d374e4c287de77f361caf7", - "size_bytes": 18440 + "sha256": "28a18050aea00646f9ff7a5fdf8fe3b8783cd4cb77d71108b3834c746e7259c5", + "size_bytes": 18446 } ], - "duration_ms": 32.29, + "duration_ms": 7.21, "message": "", "metrics": { "entity_count": 1 @@ -305,11 +375,11 @@ "artifacts": [ { "filename": "manifest.json", - "sha256": "a74b2e96dbe7ea373717f26a8d26e58fe079a61370eb5d646f04a09f1b7cca2f", - "size_bytes": 5662 + "sha256": "7178055a030b533e83c4a914466c7474350e7728657cd624c9ebc01c753d9e13", + "size_bytes": 7929 } ], - "duration_ms": 305.81, + "duration_ms": 274.87, "message": "", "metrics": { "score": { @@ -329,7 +399,7 @@ }, { "artifacts": [], - "duration_ms": 1149.77, + "duration_ms": 699.92, "message": "", "metrics": { "ranks": { @@ -350,15 +420,15 @@ }, { "artifacts": [], - "duration_ms": 53.97, + "duration_ms": 86.79, "message": "", "metrics": { "advertised_tools": 2, - "catalog_tokens": 40305, - "catalog_tools": 149, + "catalog_tokens": 83921, + "catalog_tools": 242, "ceiling": 2000, "discovery_tokens": 356, - "reduction_x": 113.2, + "reduction_x": 235.7, "tokenizer": "ratio/v1 (estimate)" }, "score": 100.0, @@ -369,7 +439,7 @@ }, { "artifacts": [], - "duration_ms": 10.6, + "duration_ms": 7.47, "message": "", "metrics": { "area_ignoring_islands": 400.0, @@ -386,7 +456,7 @@ }, { "artifacts": [], - "duration_ms": 8.0, + "duration_ms": 26.1, "message": "", "metrics": { "bounding_box": 3, @@ -403,7 +473,7 @@ }, { "artifacts": [], - "duration_ms": 5.36, + "duration_ms": 5.83, "message": "", "metrics": { "boundary_agrees": true, @@ -417,16 +487,330 @@ "stderr_summary": "", "stdout_summary": "", "task_id": "measure_from_handle" + }, + { + "artifacts": [], + "duration_ms": 58.19, + "message": "", + "metrics": { + "authored_line_numbers_kept": true, + "confidence_min": 1.0, + "critique_issues": 0, + "dangling": 0, + "edges": 4, + "instrument_index_rows": 1, + "line_list_rows": 4, + "nodes": 5, + "unnumbered_lines_sequenced": true + }, + "score": 100.0, + "status": "pass", + "stderr_summary": "", + "stdout_summary": "", + "task_id": "pid_roundtrip" + }, + { + "artifacts": [ + { + "filename": "untitled-Layout1-ansi_b.pdf", + "sha256": "af3c19d4d8b14fbc116ab807bb6feee9fc900fc114e57cd6feb658a6c0dabffb", + "size_bytes": 1287 + }, + { + "filename": "untitled-Layout1-iso_a3.pdf", + "sha256": "8d1d1ed505903b6f692ed6e91e9f52577c2c78db3bc2942b949ea4a3d73c9b01", + "size_bytes": 1287 + } + ], + "duration_ms": 65.46, + "message": "", + "metrics": { + "mediabox_mm": [ + 420.0, + 297.0 + ], + "mediabox_mm_before": [ + 432.0, + 279.0 + ], + "orientation": "landscape", + "paper": "ISO_A3", + "pdf_bytes": 1287, + "plot_style_known": true, + "size_mm_changed": [ + [ + 432.0, + 279.0 + ], + [ + 420.0, + 297.0 + ] + ], + "size_mm_listed": [ + 420.0, + 297.0 + ] + }, + "score": 100.0, + "status": "pass", + "stderr_summary": "", + "stdout_summary": "", + "task_id": "page_setup_truth" + }, + { + "artifacts": [ + { + "filename": "mech_assembly.dxf", + "sha256": "407dcbed3eb17be7f9d72dda2c1951e8f97fa3fb6090713440404a3f1497d1b2", + "size_bytes": 101158 + } + ], + "duration_ms": 487.19, + "message": "", + "metrics": { + "critique_focuses": [], + "critique_issues": 0, + "fasteners": 4, + "grade": "A", + "invalidity_ratio": 0.0, + "parts_list_rows": 2, + "score": 100.0, + "validator_findings": [] + }, + "score": 100.0, + "status": "pass", + "stderr_summary": "", + "stdout_summary": "", + "task_id": "mech_assembly" + }, + { + "artifacts": [ + { + "filename": "arch_roundtrip.dxf", + "sha256": "ebec378ef28670e091e651b4130a6eac5a0d2fb480d60f9335b37de28ea713c7", + "size_bytes": 82829 + } + ], + "duration_ms": 982.07, + "message": "", + "metrics": { + "confidence_min": 1.0, + "critique_focuses": [], + "critique_issues": 0, + "door_tags": [ + "D1", + "D2" + ], + "faces_detected": 2, + "grade": "A", + "invalidity_ratio": 0.0, + "omitted": [], + "rooms": { + "01": { + "agrees": true, + "detected_mm2": 27743750.0, + "hand_mm2": 27743750.0, + "label_mm2": 27743750.0 + }, + "02": { + "agrees": true, + "detected_mm2": 21993750.0, + "hand_mm2": 21993750.0, + "label_mm2": 21993750.0 + } + }, + "score": 100.0, + "validator_findings": [], + "window_tags": [ + "W1", + "W2" + ] + }, + "score": 100.0, + "status": "pass", + "stderr_summary": "", + "stdout_summary": "", + "task_id": "arch_roundtrip" + }, + { + "artifacts": [ + { + "filename": "pid.dxf", + "sha256": "94dbdc026878927560a6c81e2283a7d2a26fbe6a398475d415c6383728c46672", + "size_bytes": 33795 + }, + { + "filename": "layout.dxf", + "sha256": "37963e237f27f9f495b30d7c0b3bf7f4eff54ff4fec9665f34ab50ddc3f3c7a7", + "size_bytes": 40503 + } + ], + "duration_ms": 614.2, + "message": "", + "metrics": { + "cable_mismatch": [], + "length_source": "layout", + "loads_checked": 4, + "pipe_mismatch": [], + "runs_checked": 9, + "scale_verdict": "schematic", + "within_10pct": 0.21818181818181817 + }, + "score": 100.0, + "status": "pass", + "stderr_summary": "", + "stdout_summary": "", + "task_id": "takeoff_roundtrip" + }, + { + "artifacts": [ + { + "filename": "layout.dxf", + "sha256": "ff728965a5698cf9ddf9d9a01cfa244102d183065a149488d4d29dbcee404e85", + "size_bytes": 40503 + }, + { + "filename": "pid.dxf", + "sha256": "de4f1ab4102d220efa7167b192b8fdaf9d57cbf300241e945c16967770401cd8", + "size_bytes": 33795 + } + ], + "duration_ms": 311.97, + "message": "", + "metrics": { + "clusters": 2, + "copies_separated": true, + "misread_layers": {}, + "outlier_found": true, + "units": { + "agree": false, + "confidence": 0.76, + "declared": { + "code": 1, + "label": "Inches", + "name": "in" + }, + "evidence": [ + { + "as": { + "cm": 2.5, + "ft": 76.2, + "in": 6.35, + "m": 250.0, + "mm": 0.25 + }, + "band": [ + 0.0018, + 4.0 + ], + "band_unit": "m", + "count": 26, + "informative": true, + "kind": "text_height", + "plausible": [ + "mm", + "cm" + ], + "value": 250.0, + "weight": 0.5 + }, + { + "as": { + "cm": 2.0, + "ft": 60.96, + "in": 5.08, + "m": 200.0, + "mm": 0.2 + }, + "band": [ + 0.05, + 1.0 + ], + "band_unit": "m", + "count": 24, + "informative": true, + "kind": "wall_thickness", + "plausible": [ + "mm" + ], + "value": 200.0, + "weight": 1.0 + }, + { + "as": { + "cm": 21480.0, + "ft": 19955572.992000002, + "in": 138580.368, + "m": 214800000.0, + "mm": 214.79999999999998 + }, + "band": [ + 1.0, + 5000.0 + ], + "band_unit": "m2", + "count": 4, + "informative": true, + "kind": "room_area", + "plausible": [ + "mm" + ], + "value": 214800000.0, + "weight": 1.0 + }, + { + "as": { + "cm": 1364.0, + "ft": 41574.72, + "in": 3464.56, + "m": 136400.0, + "mm": 136.4 + }, + "band": [ + 0.01, + 2000.0 + ], + "band_unit": "m", + "count": 80, + "informative": true, + "kind": "overall_size", + "plausible": [ + "mm", + "cm" + ], + "value": 136400.0, + "weight": 0.25 + } + ], + "inferred": "mm", + "mm_per_unit": 1.0, + "scores": { + "cm": 0.75, + "ft": 0, + "in": 0.5, + "m": 0, + "mm": 2.75 + }, + "warning": "INSUNITS declares Inches (1) but the geometry reads as mm: text_height 250 = 0.25 m as mm, 6.35 m as in; wall_thickness 200 = 0.2 m as mm, 5.08 m as in; room_area 2.148e+08 = 214.8 m2 as mm, 1.386e+05 m2 as in; overall_size 136400 = 136.4 m as mm, 3465 m as in. Lengths are measured in mm; the header is not changed." + }, + "units_flagged": true + }, + "score": 100.0, + "status": "pass", + "stderr_summary": "", + "stdout_summary": "", + "task_id": "understand_foreign" } ], "schema_version": "2.0", "summary": { - "attempted": 15, + "attempted": 21, "coverage_percent": 100.0, - "passed": 15, + "passed": 21, "score": 100.0, - "supported": 15, + "supported": 21, "unsupported": 0 }, - "timeout_seconds": 180.0 + "timeout_seconds": 30.0 } diff --git a/benchmarks/results/published/perf-ezdxf.json b/benchmarks/results/published/perf-ezdxf.json index 28f1732..6d05ba1 100644 --- a/benchmarks/results/published/perf-ezdxf.json +++ b/benchmarks/results/published/perf-ezdxf.json @@ -14,23 +14,37 @@ "entity_count": 2000 }, "metric": "entities_per_second", - "metric_value": 5715.8, - "wall_ms": 349.9, - "workload_id": "create_lines_2k" + "wall_ms": 306.1, + "workload_id": "create_lines_2k", + "runs_ms": [ + 287.6, + 373.6, + 298.2, + 425.1, + 306.1 + ], + "metric_value": 6533.8 }, { "description": "Create 10,000 lines, export DXF, reopen and verify", "detail": { - "build_ms": 1633.1, + "build_ms": 2149.4, "entity_count": 10000, - "export_ms": 273.0, - "file_bytes": 1369663, - "reopen_ms": 272.1 + "export_ms": 516.6, + "file_bytes": 1369662, + "reopen_ms": 535.2 }, "metric": "entities_per_second", - "metric_value": 4590.8, - "wall_ms": 2178.3, - "workload_id": "roundtrip_10k" + "wall_ms": 2553.5, + "workload_id": "roundtrip_10k", + "runs_ms": [ + 3201.3, + 2550.0, + 3118.3, + 2553.5, + 2289.4 + ], + "metric_value": 3916.2 }, { "description": "Crossing-selection region query over 10,000 entities", @@ -39,8 +53,15 @@ }, "metric": "entities_matched", "metric_value": 2500.0, - "wall_ms": 2466.9, - "workload_id": "region_query_10k" + "wall_ms": 712.2, + "workload_id": "region_query_10k", + "runs_ms": [ + 712.2, + 806.7, + 295.1, + 836.4, + 300.6 + ] }, { "description": "Plate with hole: ISO layers, geometry, dimensions and full critique", @@ -49,8 +70,16 @@ }, "metric": "critique_issues", "metric_value": 0.0, - "wall_ms": 255.7, - "workload_id": "premium_pass" + "wall_ms": 65.3, + "workload_id": "premium_pass", + "runs_ms": [ + 76.3, + 65.3, + 36.9, + 83.8, + 41.3 + ] } - ] + ], + "aggregation": "median of 5 default-configuration runs, one machine, one interpreter" } diff --git a/benchmarks/results/published/perf-v1.4.0-vs-v1.6.0.json b/benchmarks/results/published/perf-v1.4.0-vs-v1.6.0.json new file mode 100644 index 0000000..19153c7 --- /dev/null +++ b/benchmarks/results/published/perf-v1.4.0-vs-v1.6.0.json @@ -0,0 +1,142 @@ +{ + "schema_version": "perf-ab/1.0", + "method": "benchmarks/perf_suite.py of 1.6.0 run against each tree's own code with one CPython 3.11.15 and one ezdxf, five rounds, the three configurations alternating within a round; medians of wall_ms. v1.4.0 had no perf suite; the 1.6.0 script drives the same backend methods.", + "environment": { + "implementation": "CPython", + "machine": "AMD64", + "platform": "Windows-10-10.0.26200-SP0", + "python": "3.11.15" + }, + "medians_ms": { + "v1.4.0": { + "create_lines_2k": 244.5, + "roundtrip_10k": 1847.9, + "region_query_10k": 665.2, + "premium_pass": 150.6 + }, + "v1.6.0": { + "create_lines_2k": 306.1, + "roundtrip_10k": 2553.5, + "region_query_10k": 712.2, + "premium_pass": 65.3 + }, + "v1.6.0 EZDXF_CALL_TIMEOUT=0": { + "create_lines_2k": 284.9, + "roundtrip_10k": 2216.4, + "region_query_10k": 783.9, + "premium_pass": 69.9 + } + }, + "ratios_to_v1_4_0": { + "v1.6.0": { + "create_lines_2k": 1.25, + "roundtrip_10k": 1.38, + "region_query_10k": 1.07, + "premium_pass": 0.43 + }, + "v1.6.0 EZDXF_CALL_TIMEOUT=0": { + "create_lines_2k": 1.17, + "roundtrip_10k": 1.2, + "region_query_10k": 1.18, + "premium_pass": 0.46 + } + }, + "runs": { + "v1.4.0": [ + { + "create_lines_2k": 244.5, + "roundtrip_10k": 1936.8, + "region_query_10k": 670.3, + "premium_pass": 160.3 + }, + { + "create_lines_2k": 221.9, + "roundtrip_10k": 1592.5, + "region_query_10k": 638.2, + "premium_pass": 150.6 + }, + { + "create_lines_2k": 236.7, + "roundtrip_10k": 1706.6, + "region_query_10k": 927.2, + "premium_pass": 140.2 + }, + { + "create_lines_2k": 261.0, + "roundtrip_10k": 1979.2, + "region_query_10k": 665.2, + "premium_pass": 191.4 + }, + { + "create_lines_2k": 308.4, + "roundtrip_10k": 1847.9, + "region_query_10k": 296.5, + "premium_pass": 70.3 + } + ], + "v1.6.0": [ + { + "create_lines_2k": 287.6, + "roundtrip_10k": 3201.3, + "region_query_10k": 712.2, + "premium_pass": 76.3 + }, + { + "create_lines_2k": 373.6, + "roundtrip_10k": 2550.0, + "region_query_10k": 806.7, + "premium_pass": 65.3 + }, + { + "create_lines_2k": 298.2, + "roundtrip_10k": 3118.3, + "region_query_10k": 295.1, + "premium_pass": 36.9 + }, + { + "create_lines_2k": 425.1, + "roundtrip_10k": 2553.5, + "region_query_10k": 836.4, + "premium_pass": 83.8 + }, + { + "create_lines_2k": 306.1, + "roundtrip_10k": 2289.4, + "region_query_10k": 300.6, + "premium_pass": 41.3 + } + ], + "v1.6.0 EZDXF_CALL_TIMEOUT=0": [ + { + "create_lines_2k": 285.3, + "roundtrip_10k": 1817.2, + "region_query_10k": 926.5, + "premium_pass": 61.4 + }, + { + "create_lines_2k": 256.1, + "roundtrip_10k": 2520.9, + "region_query_10k": 750.3, + "premium_pass": 80.2 + }, + { + "create_lines_2k": 260.2, + "roundtrip_10k": 1714.1, + "region_query_10k": 783.9, + "premium_pass": 76.7 + }, + { + "create_lines_2k": 351.7, + "roundtrip_10k": 3057.1, + "region_query_10k": 940.8, + "premium_pass": 69.9 + }, + { + "create_lines_2k": 284.9, + "roundtrip_10k": 2216.4, + "region_query_10k": 311.2, + "premium_pass": 41.7 + } + ] + } +} diff --git a/benchmarks/source_review.json b/benchmarks/source_review.json index 7e2951a..d1cafc8 100644 --- a/benchmarks/source_review.json +++ b/benchmarks/source_review.json @@ -1,25 +1,88 @@ { "schema_version": "1.0", - "reviewed_at": "2026-07-23", + "reviewed_at": "2026-09-24", "title": "AutoCAD MCP source-reviewed capability benchmark", "evidence_boundary": "source_review_not_shared_live_run", - "methodology": "Public source, documented behavior, and locally recorded test evidence scored with a fixed 100-point rubric. Repository stars and raw tool counts do not score points.", + "methodology": "Public source, documented behavior, and locally recorded test evidence scored with a fixed 100-point rubric. Repository stars and raw tool counts do not score points. The U-C4N row was re-scored for 1.6.0 on 2026-09-24 with per-category evidence; competitor rows are their July snapshots and were not re-reviewed.", "rubric": [ - {"category": "Functional CAD coverage", "weight": 25}, - {"category": "Correctness and delivery", "weight": 20}, - {"category": "Backends and platforms", "weight": 15}, - {"category": "Engineering production", "weight": 15}, - {"category": "Tests and maintenance", "weight": 15}, - {"category": "Security and operations", "weight": 10} + { + "category": "Functional CAD coverage", + "weight": 25 + }, + { + "category": "Correctness and delivery", + "weight": 20 + }, + { + "category": "Backends and platforms", + "weight": 15 + }, + { + "category": "Engineering production", + "weight": 15 + }, + { + "category": "Tests and maintenance", + "weight": 15 + }, + { + "category": "Security and operations", + "weight": 10 + } ], "projects": [ { "repository": "U-C4N/Autocad-MCP", "url": "https://github.com/U-C4N/Autocad-MCP", - "reviewed_version": "1.4.0 (PyPI release)", - "score": 95, + "reviewed_version": "1.6.0 (release)", + "score": 93, "evidence_grade": "A", - "note": "Dual COM/ezdxf backends, paper space, ISO 129/286/1101 annotation, closed-loop quality gates, auditable delivery; v1.4 adds CI matrix incl. Windows, PyPI packaging, and a version A/B correctness gate (21/21, 0 regressions)." + "note": "Scored per category, each with its evidence (roadmap criterion 8). 1.4.0 received a single overall 95; this review is stricter, and every point withheld is named. New in 1.5-1.6: P&ID, environment, mechanical, sheet, architecture, understanding and takeoffs; A/B 48/48 with 0 regressions; live COM smokes on every track.", + "category_scores": [ + { + "category": "Functional CAD coverage", + "weight": 25, + "score": 24, + "evidence": "247 tools: drafting, ISO 129/286/1101 annotation, paper space, styles and page setup, environment, P&ID (ISO 10628-2 / ISA-5.1, read back with confidence), a mechanical part model with standard parts, ISO 5457/7200/7573/6433 sheets, an architectural plan engine, drawing understanding, diff, topology and plant takeoffs. Not covered: electrical schematics, image-to-CAD tracing, Plant 3D, obstacle-avoiding routing, headless REGION/3DSOLID." + }, + { + "category": "Correctness and delivery", + "weight": 20, + "score": 19, + "evidence": "A/B gate against v1.5.1: 48/48 checks, 0 regressions (ab-v1.5.1-vs-v1.6.0.json). Matrix v7: 21 tasks whose gates can fail (zero critique issues and a finalize score of at least 90, PDF /MediaBox, rooms within 0.1 %, takeoff lengths exact). drawing_deliver writes a manifest and checks reopen parity. A field test on a real P&ID pair drove nine fixes. Minus: a headless DIMENSION does not re-render, and a takeoff on a real P&ID still needs the engineer's scope decisions." + }, + { + "category": "Backends and platforms", + "weight": 15, + "score": 14, + "evidence": "COM and ezdxf behind one typed contract, with capability-map parity enforced by a test. Every 1.6 track's COM path ran live on AutoCAD 2026. CI runs Linux 3.11/3.12 and Windows. Minus: BricsCAD, ZWCAD and GstarCAD are unverified." + }, + { + "category": "Engineering production", + "weight": 15, + "score": 14, + "evidence": "Transcribed standards tables (ISO 4014/4017/4032/7089/4762, ISO 15, ISO 286, ISO 5457, ISO 7200, ISO 7573) and one part model and view engine. 18 domain critique focuses (P&ID 6, mechanical 6, architecture 3, topology 3) feed drawing_finalize's 0-100 score. Minus: four DIN/ISO feature tables ship empty until verified rows exist." + }, + { + "category": "Tests and maintenance", + "weight": 15, + "score": 14, + "evidence": "About 4,990 tests on a locked dependency graph. The release gate re-runs three OS/Python lanes; a document-number gate and the per-release A/B suite also run. The perf lane is published (perf-v1.4.0-vs-v1.6.0.json). Minus: entity creation is 1.25x (2,000 lines) and 1.38x (10,000-line roundtrip) slower than v1.4.0, so roadmap criterion 7 is not met." + }, + { + "category": "Security and operations", + "weight": 10, + "score": 8, + "evidence": "Per-tool path validation, a loopback-only HTTP bind unless a bearer token is set, the audit middleware, and a field-test script that refuses to write inside the repository. Minus: ALLOWED_PATHS is empty, and so unscoped, by default; command and LISP filtering is a denylist, which is a guardrail rather than a security boundary." + } + ], + "history": [ + { + "version": "1.4.0", + "score": 95, + "reviewed_at": "2026-07-23" + } + ] }, { "repository": "varavista/autocad-mcp", @@ -85,5 +148,6 @@ "evidence_grade": "C", "note": "Basic COM drawing tool set with SQLite persistence; reviewed from the documented tool surface only, execution path not exercised locally." } - ] + ], + "competitors_reviewed_at": "2026-07-23" } diff --git a/docs/assets/arch-showcase.png b/docs/assets/arch-showcase.png new file mode 100644 index 0000000..439e452 Binary files /dev/null and b/docs/assets/arch-showcase.png differ diff --git a/docs/assets/autocad-mcp-benchmark.svg b/docs/assets/autocad-mcp-benchmark.svg index dc8413b..af54395 100644 --- a/docs/assets/autocad-mcp-benchmark.svg +++ b/docs/assets/autocad-mcp-benchmark.svg @@ -29,19 +29,19 @@ z - - + @@ -50,14 +50,14 @@ L 0 3.5 " style="stroke: #8b949e; stroke-width: 0.8"/> - + - + - - + - + - + - + - - - + + - + - + - + - - - + + - + - + - + - - - + + - + - + - + - - - + + - + - + - + - - - - + + + - + - - - - - - - - - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -720,9 +720,9 @@ z - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -928,9 +928,9 @@ z - + - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + @@ -979,9 +979,9 @@ z - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + @@ -1093,9 +1093,9 @@ z - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -1164,9 +1164,9 @@ z - + - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + @@ -1224,9 +1224,9 @@ z - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -1283,9 +1283,9 @@ z - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -1356,9 +1356,9 @@ z - + - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + @@ -1400,9 +1400,9 @@ z - + - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + - +" clip-path="url(#paa82582780)" style="fill: #8957e5"/> - +" clip-path="url(#paa82582780)" style="fill: #f0876e"/> - +" clip-path="url(#paa82582780)" style="fill: #bf4b8a"/> - +" clip-path="url(#paa82582780)" style="fill: #e3b341"/> - +" clip-path="url(#paa82582780)" style="fill: #c4622d"/> - +" clip-path="url(#paa82582780)" style="fill: #d29922"/> - +" clip-path="url(#paa82582780)" style="fill: #6e7ce0"/> - +" clip-path="url(#paa82582780)" style="fill: #58a6ff"/> - +" clip-path="url(#paa82582780)" style="fill: #3fb950"/> - - + + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - + - - - - + - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + - + - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + - + - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + - + - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + - + - - - - - - - - - - - - - - - + + + + + + + + + + + + + + - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + - - + + - - - - - - - - - - - - - - - + + + + + + + + + + + + + + - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - - + + diff --git a/docs/assets/autocad-mcp-livebench.svg b/docs/assets/autocad-mcp-livebench.svg index f1c665f..40012af 100644 --- a/docs/assets/autocad-mcp-livebench.svg +++ b/docs/assets/autocad-mcp-livebench.svg @@ -30,8 +30,8 @@ z @@ -41,7 +41,7 @@ z +" clip-path="url(#p09509dacb4)" style="fill: none; stroke: #30363d; stroke-opacity: 0.8; stroke-width: 0.8; stroke-linecap: square"/> @@ -85,18 +85,18 @@ z - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + +" clip-path="url(#p09509dacb4)" style="fill: #484f58"/> +" clip-path="url(#p09509dacb4)" style="fill: #484f58"/> +" clip-path="url(#p09509dacb4)" style="fill: #58a6ff"/> @@ -1378,7 +1378,7 @@ z - + - + @@ -1959,39 +1959,17 @@ z - - + + - @@ -2001,11 +1979,11 @@ z - - + + - - + + @@ -2036,12 +2014,12 @@ z - + - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -2508,8 +2475,8 @@ z - - + + diff --git a/docs/assets/autocad-mcp-perf.svg b/docs/assets/autocad-mcp-perf.svg index 30033ab..e6f3ccb 100644 --- a/docs/assets/autocad-mcp-perf.svg +++ b/docs/assets/autocad-mcp-perf.svg @@ -39,8 +39,8 @@ z - @@ -50,12 +50,12 @@ L 0 3.5 " style="stroke: #8b949e; stroke-width: 0.8"/> - + - - + + + + + + + + + + + + + + + + + + + + + + + - - + + - + - + - + - + - + + - - - - - - - - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -701,8 +789,8 @@ z - - + + @@ -948,8 +1036,8 @@ z - - + + @@ -973,30 +1061,6 @@ Q 1578 4750 2638 4750 Q 3056 4750 3426 4639 Q 3797 4528 4122 4306 z -" transform="scale(0.015625)"/> - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + @@ -1546,9 +1610,9 @@ z - - - + + + - - @@ -1733,8 +1755,8 @@ z - - + + @@ -1752,9 +1774,9 @@ z - - - + + + + + + - - - - + @@ -1908,133 +1969,11 @@ z - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + - - + + @@ -2049,9 +1988,31 @@ z - - + + + - + - - + + @@ -2214,6 +2175,70 @@ z + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -2389,7 +2414,7 @@ z - + diff --git a/docs/assets/autocad-mcp-taskmatrix.svg b/docs/assets/autocad-mcp-taskmatrix.svg index d5eca7e..2d7c074 100644 --- a/docs/assets/autocad-mcp-taskmatrix.svg +++ b/docs/assets/autocad-mcp-taskmatrix.svg @@ -1,7 +1,7 @@ - + @@ -20,8 +20,8 @@ - - - +" clip-path="url(#p8297dbecf1)" style="fill: #238636; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #238636; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #238636; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #238636; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #238636; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #238636; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #238636; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #238636; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #d29922; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #238636; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #238636; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #238636; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #238636; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #30363d; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #30363d; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #238636; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #30363d; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #30363d; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #238636; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #30363d; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #30363d; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #238636; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #30363d; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #30363d; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #238636; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #238636; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #238636; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #238636; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #30363d; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #30363d; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #238636; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #161b22; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #161b22; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #238636; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #161b22; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #161b22; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #238636; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #161b22; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #161b22; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #238636; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #161b22; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #161b22; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #238636; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #161b22; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> - +" clip-path="url(#p8297dbecf1)" style="fill: #161b22; stroke: #30363d; stroke-width: 0.8; stroke-linejoin: miter"/> + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -941,7 +1085,7 @@ z - + - + - + - + @@ -1524,7 +1668,7 @@ z - + - + @@ -1622,7 +1766,7 @@ z - + - + - + - + - + @@ -1901,7 +2045,7 @@ z - + - + @@ -2023,7 +2167,7 @@ z - + @@ -2059,7 +2203,7 @@ z - + @@ -2097,10 +2241,244 @@ z + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - + - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + - - - + + + - - - + + + - + - + - + - + - + - + - + - - - + + + - - - + + + - + - + - + - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + - + - + - + - + - + - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -2605,34 +3091,34 @@ z - - + - + - + - - + - + - + @@ -2642,34 +3128,34 @@ z - - + - + - + - - + - + - + @@ -2699,17 +3185,17 @@ z - - + - + - + @@ -2723,8 +3209,8 @@ z - - + + diff --git a/docs/assets/mech-showcase.png b/docs/assets/mech-showcase.png new file mode 100644 index 0000000..3226f68 Binary files /dev/null and b/docs/assets/mech-showcase.png differ diff --git a/docs/superpowers/specs/2026-09-15-v1.6-roadmap-design.md b/docs/superpowers/specs/2026-09-15-v1.6-roadmap-design.md index 716cddc..5435d69 100644 --- a/docs/superpowers/specs/2026-09-15-v1.6-roadmap-design.md +++ b/docs/superpowers/specs/2026-09-15-v1.6-roadmap-design.md @@ -4,6 +4,19 @@ own design spec and implementation plan before code · **Baseline:** v1.5.1 (154 registered tools, 19 groups, 1369 tests) +> **Status at release (1.6.0, 2026-09-25).** Tracks A, E, B, G, F and H shipped: +> 247 registered tools in 28 groups. Criteria 1–4, 6 and 8 are met. 1 and 3 are +> the `pid_roundtrip` and `mech_assembly` benchmark tasks; 2 is the foreign-P&ID +> fixture test; 4 is the three topology focuses plus the correctness check +> `diff_detects_known_edits`, on synthetic revisions rather than the README hero +> sheet; 6 is `ab-v1.5.1-vs-v1.6.0.json` (48 / 48, 0 regressions); 8 is the +> per-category rubric in `benchmarks/source_review.json` (93). **Criterion 5 is +> withdrawn**: track D was dropped because the maintainer does not use Plant 3D. +> **Criterion 7 is not met**: creation is 1.25× and 1.38× v1.4.0's time +> (`perf-v1.4.0-vs-v1.6.0.json`), still 1.17× and 1.20× with the timeout off, and +> it moves to 1.7. §4's rule that 1.6.0 waits for every criterion was set aside +> for these two by the maintainer's decision to release. + ## 1. Goal An LLM driving this server can — without guessing coordinates, to a named diff --git a/docs/tool-inventory.json b/docs/tool-inventory.json index 566685c..065b7a4 100644 --- a/docs/tool-inventory.json +++ b/docs/tool-inventory.json @@ -1,7 +1,7 @@ { "_comment": "GENERATED FILE - do not edit by hand. Derived from the live FastMCP component registry; regenerate with `python scripts/generate_tool_inventory.py`.", "schema_version": 1, - "package_version": "1.5.1", + "package_version": "1.6.0", "totals": { "tools": 247, "resources": 8, diff --git a/pyproject.toml b/pyproject.toml index 125db9f..a67655b 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "autocad-mcp-pro" -version = "1.5.1" +version = "1.6.0" description = "Production-grade AutoCAD MCP server — COM API + ezdxf dual engine" readme = "README.md" requires-python = ">=3.11" @@ -151,6 +151,10 @@ only-include = [ [tool.ruff] line-length = 100 target-version = "py311" +# Design documents and implementation plans quote code as illustration; ruff +# 0.16 formats Markdown code blocks, and reflowing a plan's quoted code would +# change what the document says was written. +extend-exclude = ["docs/superpowers"] [tool.ruff.lint] select = ["E", "F", "I", "W", "UP", "B"] diff --git a/scripts/check_doc_numbers.py b/scripts/check_doc_numbers.py index ab8a881..030f843 100644 --- a/scripts/check_doc_numbers.py +++ b/scripts/check_doc_numbers.py @@ -29,9 +29,9 @@ ROOT = Path(__file__).resolve().parents[1] #: The published A/B report the README and benchmarks/README quote. Bump this, #: the two row labels below and the prose together, or the gate reports drift. -AB_REPORT = "benchmarks/results/published/ab-v1.5.1-vs-v1.6.0-dev.json" +AB_REPORT = "benchmarks/results/published/ab-v1.5.1-vs-v1.6.0.json" AB_BASELINE_LABEL = r"v1\.5\.1" -AB_CURRENT_LABEL = r"v1\.6\.0-dev" +AB_CURRENT_LABEL = r"v1\.6\.0" def _read(relative: str) -> str: @@ -160,7 +160,7 @@ CHECKS: list[tuple[str, str, str, str]] = [ "README.md", "A/B fixed count", AB_CURRENT_LABEL - + r"\*\* \*\(this branch\)\* \| \*\*\d+ / \d+\*\* \| \*\*\d+ %\*\* \| (\d+)", + + r"\*\* \*\(this release\)\* \| \*\*\d+ / \d+\*\* \| \*\*\d+ %\*\* \| (\d+)", "ab_fixed", ), ( @@ -172,7 +172,7 @@ CHECKS: list[tuple[str, str, str, str]] = [ ( "benchmarks/README.md", "A/B fixed count", - r"\*\*" + AB_CURRENT_LABEL + r"\*\* \(this branch\) \| \d+ / \d+ \| \d+ % \| (\d+)", + r"\*\*" + AB_CURRENT_LABEL + r"\*\* \(this release\) \| \d+ / \d+ \| \d+ % \| (\d+)", "ab_fixed", ), ("CLAUDE.md", "A/B suite size", r"correctness_suite\.py` \((\d+) checks\)", "ab_total"), diff --git a/scripts/render_readme_arch.py b/scripts/render_readme_arch.py new file mode 100644 index 0000000..ca64387 --- /dev/null +++ b/scripts/render_readme_arch.py @@ -0,0 +1,63 @@ +"""Render the floor plan the README shows, using only the server's own tools. + +The plan is ``EXAMPLE_SPEC`` from ``engineering/arch/spec.py`` - the document +``arch_plan_from_spec`` draws in one call: a 9 x 6 m brick ring split by an AAC +wall, an entrance door, an interior door, two windows, a straight stair, two +rooms whose areas are measured from the faces the walls enclose, the exterior +dimension chains and the door / window / room schedules. The ``arch_roundtrip`` +benchmark task gates on the same spec, so a regression changes both. + +Reproduce: + + AUTOCAD_MCP_BACKEND=ezdxf uv run --frozen python scripts/render_readme_arch.py +""" + +from __future__ import annotations + +import argparse +import asyncio +import sys +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(ROOT)) + +DEFAULT_OUTPUT = ROOT / "docs" / "assets" / "arch-showcase.png" + + +async def render(output: Path) -> dict: + from backends.ezdxf_backend import EzdxfBackend + from engineering.arch.spec import EXAMPLE_SPEC, draw_plan_from_spec + from engineering.critique import run_critique + + backend = EzdxfBackend() + await backend.connect() + try: + await backend.drawing_new() + result = await draw_plan_from_spec(backend, EXAMPLE_SPEC) + issues = await run_critique(backend, None) + png = await backend.view_screenshot() + if not png: + raise RuntimeError("view_screenshot returned no PNG (is matplotlib installed?)") + output.parent.mkdir(parents=True, exist_ok=True) + output.write_bytes(png) + return { + "output": str(output), + "bytes": len(png), + "omitted": len(result.get("omitted", [])), + "critique": len(issues), + } + finally: + await backend.disconnect() + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--output", type=Path, default=DEFAULT_OUTPUT) + args = parser.parse_args() + print(asyncio.run(render(args.output))) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/render_readme_mech.py b/scripts/render_readme_mech.py new file mode 100644 index 0000000..3f9c868 --- /dev/null +++ b/scripts/render_readme_mech.py @@ -0,0 +1,63 @@ +"""Render the mechanical sheet the README shows, using only the server's own tools. + +The sheet is the one the ``mech_assembly`` benchmark task gates on - a flange +coupling: a hub with a full section, a flange with its bolt-circle end view, two +ISO 4014 bolts with ISO 4032 nuts, an ISO 7573 parts list read off the drawing, +linked ISO 6433 balloons, an ISO 5457 frame and an ISO 7200 title block. It is +built by the benchmark's own task, so the picture and the gate cannot drift +apart: if the view engine or the sheet standard regresses, both change. + +Reproduce: + + AUTOCAD_MCP_BACKEND=ezdxf uv run --frozen python scripts/render_readme_mech.py +""" + +from __future__ import annotations + +import argparse +import asyncio +import sys +import tempfile +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(ROOT)) + +DEFAULT_OUTPUT = ROOT / "docs" / "assets" / "mech-showcase.png" + + +async def render(output: Path) -> dict: + from benchmarks.adapters.autocad_mcp_pro import AutoCADMCPProAdapter + + adapter = AutoCADMCPProAdapter(backend="ezdxf") + with tempfile.TemporaryDirectory(prefix="readme_mech_") as folder: + await adapter.setup(Path(folder)) + try: + await adapter.backend.drawing_new() # the harness opens one before every task + passed, metrics = (await adapter._task_mech_assembly())[:2] + png = await adapter.backend.view_screenshot() + finally: + await adapter.cleanup() + if not png: + raise RuntimeError("view_screenshot returned no PNG (is matplotlib installed?)") + output.parent.mkdir(parents=True, exist_ok=True) + output.write_bytes(png) + return { + "output": str(output), + "bytes": len(png), + "gate_passed": passed, + "critique_issues": metrics["critique_issues"], + "score": metrics["score"], + } + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--output", type=Path, default=DEFAULT_OUTPUT) + args = parser.parse_args() + print(asyncio.run(render(args.output))) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/server.json b/server.json index 0fb614e..97e29da 100644 --- a/server.json +++ b/server.json @@ -2,17 +2,17 @@ "$schema": "https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json", "name": "io.github.u-c4n/autocad-mcp", "title": "AutoCAD MCP Pro", - "description": "AutoCAD automation for AI agents: live COM or headless ezdxf, ISO dims, GD&T, quality gate.", + "description": "AutoCAD for AI agents, live or headless: drafting, P&ID, mechanical parts, architecture, takeoffs.", "repository": { "url": "https://github.com/U-C4N/Autocad-MCP", "source": "github" }, - "version": "1.5.1", + "version": "1.6.0", "packages": [ { "registryType": "pypi", "identifier": "autocad-mcp-pro", - "version": "1.5.1", + "version": "1.6.0", "transport": { "type": "stdio" }, @@ -38,6 +38,13 @@ "format": "string", "isSecret": false }, + { + "name": "TOOL_PACKS", + "description": "Vertical packs to advertise: all (default) or a comma list of core, pid, settings, mech, arch, plant; core is always on.", + "isRequired": false, + "format": "string", + "isSecret": false + }, { "name": "DISCOVERY_MODE", "description": "Tool discovery: off (default) advertises the full catalog; search replaces it with search_tools plus a call_tool proxy.", diff --git a/tests/test_benchmark_v3.py b/tests/test_benchmark_v3.py index 370d79b..58fbdf2 100644 --- a/tests/test_benchmark_v3.py +++ b/tests/test_benchmark_v3.py @@ -158,12 +158,3 @@ def test_competitor_reports_carry_no_result_for_the_new_tasks(filename): "result here would be fabricated" ) assert report.get("matrix", "v2") == "v2" - - -def test_the_published_reference_report_covers_the_whole_v3_matrix(): - report = json.loads((PUBLISHED / "autocad-mcp-pro.json").read_text(encoding="utf-8")) - scored = {item["task_id"] for item in report["results"]} - - assert scored == {task.task_id for task in TASKS_V3} - assert report["matrix"] == "v3" - assert report["summary"]["score"] == 100.0 diff --git a/tests/test_benchmark_v7.py b/tests/test_benchmark_v7.py index 3d1832d..72001bd 100644 --- a/tests/test_benchmark_v7.py +++ b/tests/test_benchmark_v7.py @@ -10,6 +10,9 @@ A/B report. from __future__ import annotations +import json +from pathlib import Path + import ezdxf import pytest @@ -36,6 +39,19 @@ def test_v7_extends_v6_without_touching_it(): assert task_by_id("understand_foreign").category == "understand" +def test_the_published_reference_report_covers_the_whole_current_matrix(): + # The release's reference run: every task of the current matrix, all passed. + # It replaces 1.5's v3 run; the competitor reports stay the v2 runs. + published = Path(__file__).resolve().parents[1] / "benchmarks" / "results" / "published" + report = json.loads((published / "autocad-mcp-pro.json").read_text(encoding="utf-8")) + scored = {item["task_id"] for item in report["results"]} + + assert scored == {task.task_id for task in TASKS_V7} + assert report["matrix"] == DEFAULT_MATRIX == "v7" + assert report["summary"]["score"] == 100.0 + assert {item["status"] for item in report["results"]} == {"pass"} + + def test_the_adapter_implements_every_v7_task(): from benchmarks.adapters.autocad_mcp_pro import AutoCADMCPProAdapter diff --git a/uv.lock b/uv.lock index fd0f27c..db4d398 100644 --- a/uv.lock +++ b/uv.lock @@ -67,7 +67,7 @@ wheels = [ [[package]] name = "autocad-mcp-pro" -version = "1.5.1" +version = "1.6.0" source = { editable = "." } dependencies = [ { name = "ezdxf" },