fix(dimensions): diameter and radius callouts measured the leader as geometry
`add_diameter_dim` / `add_radius_dim` measure to the point they are handed; `leader_length` is a text placement. The ezdxf backend passed `centre + (radius + leader)` as that point, so the leader became part of the measurement: a true 40 mm bore was dimensioned 60 at the default `leader_length=10`, and 108 at 34. Radius callouts were `leader_length` too large the same way. This is the worst class of defect for this project -- a wrong number, at default settings, on the one part of a drawing that exists to carry numbers -- and it shipped in v1.4.0 and earlier. The size now comes from `radius` and `location` only places the text. The live COM backend was never affected: ActiveX's AddDimDiametric takes the two chord points and the leader length as separate arguments. The two engines had been disagreeing by 2 x leader_length on the same call. Found while building the drawing for the new README, whose own bore callout was wrong. Two checks added to correctness_suite.py; both report fail -> pass against v1.4.0, not miss -> pass, which is what separates a repaired defect from new capability in that lane. A/B is now 21/26 -> 26/26, five fixed, zero regressed. README rewritten from scratch - Hero is a real ISO A3 sheet the server drew: m6 z24 involute spur gear, DIN 6885 keyed bore, section A-A, ISO 129 dimensions, ISO 286 H7 fit, ISO 7200 title block, 56 entities, drawing_critique clean. New scripts/render_readme_showcase.py rebuilds it, and the tool-call sequence that produces it is printed in the page. - Star history chart, badge row, capability matrix, five evidence lanes, and a "what this release is bad at" section carrying the 28% throughput regression. - Every image and repo link is now an absolute URL: README.md is the PyPI long description, where relative paths 404. Corrections this work forced, all measured - The df=0 example was wrong. FILLET is already in entity_fillet's own text; the commands that genuinely scored zero are WBLOCK, PEDIT, OVERKILL, LAYTRANS and MATCHPROP. - 40,305 tokens is the tools/list wire payload; ~25,143 is the part forwarded to the model. The old wording said "of schema", which is neither. - `--tokenizer anthropic` needs a package and credentials this repo does not lock, and refuses rather than falling back. Said so. - Discovery ranks over the 149-tool advertised catalog, not the 154 registered. - Five middlewares, not four; the fifth is what makes refusals machine-readable. - Five of the 27 capability keys are environment-dependent, so the matrix carries footnotes instead of bare ticks. - "Every coordinate is WCS" has one deliberate exception: TEXT rotation. - render_live_chart's axis still said "fixed 10-task matrix" while our bar came from v3's fifteen. Each bar now carries its own matrix. - .env.example said MAX_UNDO_STACK=50 (code: 5) and omitted four real variables, two of them the remote-HTTP controls. - server.json described an empty ALLOWED_PATHS as blocking access. It disables the allowlist. - server.py called the lean profile ~46 tools; it is 47. - backends/capability.py cited a test file that does not exist. 1249 passed, 1 xfailed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Umut CAN
co-authored by
Claude Opus 5
Claude Opus 5
parent
e27071ab48
commit
8f405bddf3
+19
-2
@@ -37,8 +37,9 @@ CAD_PROGID=AutoCAD.Application
|
||||
# Example: ALLOWED_PATHS=C:\Drawings,D:\Projects\CAD
|
||||
ALLOWED_PATHS=
|
||||
|
||||
# Maximum undo stack size (number of snapshots to keep)
|
||||
MAX_UNDO_STACK=50
|
||||
# Maximum undo stack size (number of snapshots to keep). The code's default is
|
||||
# 5; this file used to say 50, which was a documentation-only disagreement.
|
||||
MAX_UNDO_STACK=5
|
||||
|
||||
# Logging level: DEBUG | INFO | WARNING | ERROR
|
||||
LOG_LEVEL=INFO
|
||||
@@ -75,3 +76,19 @@ EZDXF_UNDO_DEPTH=0
|
||||
|
||||
# Opt-in 3D solids (COM backend only; ezdxf reports them as unsupported)
|
||||
ENABLE_3D=false
|
||||
|
||||
# --- Limits and remote access -------------------------------------------------
|
||||
# These four are read by config.py but were missing from this file, which made
|
||||
# it look like the complete list. Two of them are the remote-HTTP controls.
|
||||
|
||||
# Reject DXF input larger than this many bytes. 0 disables the check.
|
||||
MAX_DXF_BYTES=52428800
|
||||
|
||||
# Upper bound on list/selection response sizes.
|
||||
MAX_LIST_LIMIT=5000
|
||||
|
||||
# Permit a non-loopback HTTP bind. Requires MCP_AUTH_TOKEN as well.
|
||||
ALLOW_REMOTE_HTTP=false
|
||||
|
||||
# Bearer token required for remote HTTP. No default; remote binds refuse without it.
|
||||
MCP_AUTH_TOKEN=
|
||||
|
||||
+24
-4
@@ -355,10 +355,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
exactly this, but a refusal is still an exception and was being caught by the
|
||||
generic handler — "your engine cannot reach this" and "your server got this
|
||||
wrong" are different findings about a competitor.
|
||||
- Three measurement checks added to `correctness_suite.py` (24, was 21). They
|
||||
are new capability rather than repaired regressions — v1.4.0 misses them by
|
||||
not having the method — and they are there because all three broke during
|
||||
this release and nothing outside their own unit tests would have noticed.
|
||||
- Five checks added to `correctness_suite.py` (26, was 21), and the two kinds
|
||||
read differently in the A/B table. Three measurement checks are new
|
||||
capability — v1.4.0 misses them by not having the method (`miss → pass`) —
|
||||
and are there because all three broke during this release. Two dimension
|
||||
checks are repaired defects: v1.4.0 has the methods and gets them wrong
|
||||
(`fail → pass`). Result against v1.4.0: 21/26 → 26/26, five fixed, zero
|
||||
regressed.
|
||||
|
||||
### Changed
|
||||
|
||||
@@ -704,6 +707,23 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
- `release.yml` now runs `ruff format --check .` alongside `ruff check .`,
|
||||
matching the CI lint job — the release gate previously skipped formatting.
|
||||
|
||||
- **Every headless diameter and radius dimension reported the wrong number, at
|
||||
default settings.** `add_diameter_dim` / `add_radius_dim` measure to the point
|
||||
they are given; `leader_length` is a *text placement*. Passing
|
||||
`centre + (radius + leader)` as that point made the leader part of the
|
||||
measurement, so a true ⌀40 bore was dimensioned **60** with the default
|
||||
`leader_length=10`, and **108** at 34. Radius callouts were `leader_length`
|
||||
too large the same way. This is the worst class of defect for this project —
|
||||
a wrong number, silently, on the one part of a drawing that exists to carry
|
||||
numbers — and it shipped in v1.4.0 and earlier. Now the size comes from
|
||||
`radius` and `location` only places the text.
|
||||
The live COM backend was never affected: ActiveX's `AddDimDiametric` takes
|
||||
the two chord points and the leader length as separate arguments, so the two
|
||||
engines had been disagreeing by `2 × leader_length` on the same call.
|
||||
Found while building the sheet in the README's hero image, whose own bore
|
||||
callout was wrong. Two checks added to `correctness_suite.py` (26, was 24);
|
||||
both report `fail → pass` against v1.4.0 rather than `miss → pass`, which is
|
||||
what distinguishes a repaired defect from new capability in that lane.
|
||||
- **A HATCH could not be measured, and the refusal listed HATCH as
|
||||
measurable.** `entity_measure` had no HATCH branch, so a hatch fell through
|
||||
to the generic error — whose own text named HATCH among the measurable types.
|
||||
|
||||
@@ -1,22 +1,54 @@
|
||||
<div align="center">
|
||||
|
||||
# AutoCAD MCP Pro
|
||||
|
||||
Production-grade AutoCAD automation for AI agents — live through COM on
|
||||
Windows, or headless through ezdxf on any platform.
|
||||
**Production-grade AutoCAD automation for AI agents.**
|
||||
Live through COM on Windows, or headless through ezdxf on any platform — one typed contract, two engines.
|
||||
|
||||
[](https://github.com/U-C4N/Autocad-MCP/actions/workflows/ci.yml)
|
||||
[](https://pypi.org/project/autocad-mcp-pro/)
|
||||
[](LICENSE)
|
||||
[](pyproject.toml)
|
||||
[](https://pypi.org/project/autocad-mcp-pro/)
|
||||
[](https://pypi.org/project/autocad-mcp-pro/)
|
||||
[](https://github.com/U-C4N/Autocad-MCP/blob/main/pyproject.toml)
|
||||
[](https://github.com/jlowin/fastmcp)
|
||||
[](https://github.com/U-C4N/Autocad-MCP/blob/main/LICENSE)
|
||||
[](https://github.com/U-C4N/Autocad-MCP/stargazers)
|
||||
|
||||
One typed MCP contract controls two execution engines. Build and edit drawings,
|
||||
query exact geometry, apply engineering standards (ISO 128/129/286/1101),
|
||||
refine quality issues inside transactions, and deliver hashed artifacts with
|
||||
validation evidence.
|
||||
[Install](#install-in-10-seconds) · [Evidence](#evidence) · [Two engines](#the-two-engines) · [Limits](#what-this-release-is-bad-at) · [Config](#configuration) · [Changelog](https://github.com/U-C4N/Autocad-MCP/blob/main/CHANGELOG.md)
|
||||
|
||||
> **v1.5 release snapshot:** 154 tools · 6 resources · 5 prompt templates ·
|
||||
> 1245 collected tests. Runtime discovery through `system_about` is authoritative.
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
<div align="center">
|
||||
|
||||
<img src="https://raw.githubusercontent.com/U-C4N/Autocad-MCP/main/docs/assets/autocad-mcp-showcase.png" alt="ISO A3 sheet: a module-6, 24-tooth spur gear with a keyed bore, its section A-A, ISO 129 dimensions with an H7 bore fit, and an ISO 7200 title block" width="820">
|
||||
|
||||
<sub><b>Not a mockup.</b> Every line on this sheet was drawn by the tools this server exposes — ISO layers, involute gear geometry, DIN 6885 keyway, section A-A, ISO 129 dimensions, an ISO 286 <code>H7</code> bore fit, ISO 7200 title block — then rendered headlessly by <code>view_screenshot</code>. <code>drawing_critique</code> returns <b>0 issues</b> on it. Rebuild it with <code>python scripts/render_readme_showcase.py</code>.</sub>
|
||||
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
> **v1.5 release snapshot:** 154 tools · 6 resources · 5 prompt templates · 1250 collected tests.
|
||||
> 154 is the **registered** count. A default install **advertises 149** over `tools/list`, because
|
||||
> `ENABLE_3D` is unset and the five `solid_*` tools stay hidden. `system_about` is the runtime authority.
|
||||
|
||||
## Why this exists
|
||||
|
||||
Two problems, both measured before a line was written.
|
||||
|
||||
**A big MCP server is expensive to be connected to.** The full catalog puts an estimated **40,305 tokens** on the wire before the client has asked for anything, of which about **25,143** are the part a client actually forwards to the model — 149 advertised tools at ~169 tokens of name, description and input schema each. Turning on discovery replaces that catalog with two tools and **356** wire tokens.
|
||||
|
||||
**A drafter searches for `WBLOCK`, not for `block_create_from_entities`.** `WBLOCK`, `PEDIT`, `OVERKILL`, `LAYTRANS` and `MATCHPROP` appeared in *no* tool name or description, so they scored `df = 0` against a stock text index — not badly ranked, *absent*. No amount of tuning finds a word that is not in the data, so the fix was data: an authored corpus of **158 distinct AutoCAD command names and 672 synonym phrases** covering all 154 tools, layered onto the index. A test holds that df=0 list honest in both directions — it fails if one of those commands becomes findable without the corpus, and it fails if the corpus stops covering one.
|
||||
|
||||
| Advertised surface | Tools a client sees | `tools/list` on the wire |
|
||||
|---|---:|---:|
|
||||
| `TOOL_PROFILE=full` (default) | 149 | 40,305 tokens |
|
||||
| `TOOL_PROFILE=lean` | 47 | 12,131 tokens |
|
||||
| `DISCOVERY_MODE=search` | 2 | **356 tokens** |
|
||||
|
||||
> [!NOTE]
|
||||
> Token figures are **estimates** from an offline ratio counter (`ratio/v1`), not tokenizer counts — characters are the measured unit and the report says so. Under prompt caching the idle term is largely amortised, so read **113×** as the *uncached* wire ratio, not a bill. `python -m benchmarks.token_suite --tokenizer anthropic` counts for real, but it needs the `anthropic` package and credentials, neither of which is in this repo's lock — it refuses rather than silently falling back.
|
||||
|
||||
## Install in 10 seconds
|
||||
|
||||
@@ -25,310 +57,26 @@ pip install autocad-mcp-pro # or: uvx autocad-mcp-pro
|
||||
autocad-mcp # stdio MCP server, backend auto-selected
|
||||
```
|
||||
|
||||
Headless (no AutoCAD required, any OS) or live AutoCAD on Windows:
|
||||
Headless anywhere, or live AutoCAD on Windows:
|
||||
|
||||
```bash
|
||||
AUTOCAD_MCP_BACKEND=ezdxf autocad-mcp # portable DXF engine
|
||||
AUTOCAD_MCP_BACKEND=ezdxf autocad-mcp # portable DXF engine, no AutoCAD needed
|
||||
AUTOCAD_MCP_BACKEND=com autocad-mcp # live AutoCAD (pip install "autocad-mcp-pro[com]")
|
||||
```
|
||||
|
||||
Extras: `[com]` live AutoCAD (pywin32 + Pillow), `[pdf]` PDF/screenshot
|
||||
rendering (matplotlib), `[full]` everything. Working from a checkout is the
|
||||
same commands with `pip install -e ".[full]"` and `python server.py`.
|
||||
|
||||
## Benchmarks
|
||||
|
||||
Every number below is generated by a script in [`benchmarks/`](benchmarks/)
|
||||
and rendered to SVG with Python (matplotlib). No hand-edited charts, no
|
||||
adjectives.
|
||||
|
||||
### The leaderboard — nine public AutoCAD MCP servers, one rubric
|
||||
|
||||
|
||||
|
||||
| # | Project | Score | Evidence |
|
||||
|--:|---|---:|:---:|
|
||||
| 1 | **[U-C4N/Autocad-MCP](https://github.com/U-C4N/Autocad-MCP)** (this repo, v1.4.0) | **95** | A |
|
||||
| 2 | [varavista/autocad-mcp](https://github.com/varavista/autocad-mcp) | 84 | A |
|
||||
| 3 | [beiming183-cloud/AutoCAD-MCP](https://github.com/beiming183-cloud/AutoCAD-MCP) | 83 | A |
|
||||
| 4 | [LokmenoWer/best-cad-mcp](https://github.com/LokmenoWer/best-cad-mcp) | 74 | B |
|
||||
| 5 | [AnCode666/multiCAD-mcp](https://github.com/AnCode666/multiCAD-mcp) | 73 | A |
|
||||
| 6 | [puran-water/autocad-mcp](https://github.com/puran-water/autocad-mcp) | 70 | A |
|
||||
| 7 | [NCO-1986/AutoCAD_mcp](https://github.com/NCO-1986/AutoCAD_mcp) | 68 | B |
|
||||
| 8 | [daobataotie/CAD-MCP](https://github.com/daobataotie/CAD-MCP) | 35 | B |
|
||||
| 9 | [zh19980811/Easy-MCP-AutoCad](https://github.com/zh19980811/Easy-MCP-AutoCad) | 30 | C |
|
||||
|
||||
Fixed public 100-point rubric — CAD breadth (25), correctness and delivery
|
||||
(20), backends/platforms (15), engineering production (15), tests and
|
||||
maintenance (15), security and operations (10) — applied to the public source
|
||||
and documentation of each project
|
||||
([`benchmarks/source_review.json`](benchmarks/source_review.json), dated).
|
||||
Stars and raw tool counts score nothing. Evidence grade **A** = source plus
|
||||
locally executed evidence, **B** = deep source review without a full live run,
|
||||
**C** = documented tool surface only. This is a source review, not a shared
|
||||
live run — the lanes below are the runtime evidence.
|
||||
|
||||
**The table above is not re-scored per release.** It reads the dated review,
|
||||
and this repository's row is still the one written against v1.4.0. Raising our
|
||||
own number for v1.5.0 would mean re-reviewing eight other projects on the same
|
||||
day to keep the comparison fair, and that has not been done — so the score
|
||||
stays where the evidence is.
|
||||
|
||||
```bash
|
||||
python -m benchmarks.render_chart
|
||||
```
|
||||
|
||||
### 1 · Live-run scores — same harness, pinned competitors
|
||||
|
||||
|
||||
|
||||
| Server | Matrix | Pinned commit | Score | Pass | Coverage |
|
||||
|---|---|---|---:|---:|---:|
|
||||
| **autocad-mcp-pro** (this repo) | v3 (15 tasks) | working tree | **100.0** | 15/15 | 100% |
|
||||
| [beiming183-cloud/AutoCAD-MCP](https://github.com/beiming183-cloud/AutoCAD-MCP) | v2 (10 tasks) | `11f7c47` | 50.0 | 5/10 | 50% |
|
||||
| [puran-water/autocad-mcp](https://github.com/puran-water/autocad-mcp) | v2 (10 tasks) | `95476a3` | 45.0 | 4/10 | 50% |
|
||||
|
||||
**Read the matrix column before the score column.** The five v3 tasks are new
|
||||
in v1.5.0; the competitor runs are pinned at v1.4 and were never put to them.
|
||||
Their scores are over the ten tasks they actually ran, and the five new rows
|
||||
are left blank in the chart rather than scored zero — an invented zero is
|
||||
indistinguishable from a measured one.
|
||||
|
||||
One runner, headless ezdxf lane. Competitors are driven **black-box over MCP
|
||||
stdio** using their own documented tool contracts at the pinned commit, and
|
||||
every geometry claim is verified by re-opening the exported DXF with ezdxf
|
||||
inside the harness — no self-reporting. `unsupported` scores zero within a
|
||||
server's own matrix; a capability refusal is recorded as `unsupported` rather
|
||||
than `fail`, because "cannot reach this" and "got this wrong" are different
|
||||
findings. Per-task reasons are committed under
|
||||
[`benchmarks/results/published/`](benchmarks/results/published/). File-IPC
|
||||
(live AutoCAD) lanes run locally, not in CI.
|
||||
|
||||
```bash
|
||||
python -m benchmarks.run_competitors --list --matrix v3
|
||||
python -m benchmarks.run_competitors --server autocad-mcp-pro --matrix v3 \
|
||||
--publish benchmarks/results/published/autocad-mcp-pro.json
|
||||
python -m benchmarks.render_live_chart
|
||||
```
|
||||
|
||||
### 2 · Task matrix — where the score comes from
|
||||
|
||||
|
||||
|
||||
The single score hides *which* capabilities exist, so the matrix shows every
|
||||
task against every server: green = verified pass, amber = partial, gray = the
|
||||
server documents no equivalent (planning, refinement, TABLE/MLEADER, hashed
|
||||
delivery), red = attempted but failed verification, blank = never run against
|
||||
that server.
|
||||
|
||||
The v2 matrix scored this repository 10/10, and that was not a good sign —
|
||||
every task in it exercised something the server was built around, so full
|
||||
marks were the only possible outcome. **v3 adds five tasks chosen because they
|
||||
can fail**, and three of them did fail while being written: a hatch could not
|
||||
be measured at all, the boundary tools disagreed with the measurement tool
|
||||
about the same shape, and a circle stored as two bulged vertices measured zero.
|
||||
|
||||
| v3 task | What it pins | Verified against |
|
||||
| Extra | Pulls in | For |
|
||||
|---|---|---|
|
||||
| `tool_discovery` | six AutoCAD command names resolve to the right tool | all six rank #1 of a 154-tool catalog |
|
||||
| `token_budget` | idle catalog cost in discovery mode | 40,305 → 356 tokens, ceiling 2,000 |
|
||||
| `hatch_islands` | a hatch reports the area it *fills* | 300 with the island, 400 ignoring it |
|
||||
| `selection_filter` | window ≠ crossing, polygon ≠ bounding box | 1 / 2 / 3 / 1 on placed geometry |
|
||||
| `measure_from_handle` | curved boundaries measured by handle | 139.2699 against a 100.0 vertex shoelace |
|
||||
| *(none)* | `fastmcp`, `ezdxf`, `pydantic` | Headless DXF on any OS — **no rendering** |
|
||||
| `[com]` | `pywin32`, `Pillow` | Live AutoCAD control + window capture |
|
||||
| `[pdf]` | `matplotlib` | PDF export and headless PNG rendering, any platform |
|
||||
| `[full]` | everything above | Development and CI |
|
||||
|
||||
Token figures come from the offline ratio estimator, not a tokenizer — they are
|
||||
estimates and the report labels them so. `benchmarks/token_suite.py
|
||||
--tokenizer anthropic` counts for real.
|
||||
> [!NOTE]
|
||||
> The bare install **cannot draw pixels**. `ezdxf.addons.drawing.frontend` does an unconditional top-level `import PIL.Image`, so Pillow is required by *every* ezdxf render path — PNG screenshot and PDF export alike — not just by the COM window capture it is listed under. Add `[pdf]` (or `[full]`) if you want images on Linux or macOS.
|
||||
|
||||
Two tasks are renamed from the release plan, and the renames are the point.
|
||||
`tool_discovery_bilingual` lost its "bilingual": a Turkish query normalizer was
|
||||
written and then removed, because this repository is English throughout and the
|
||||
Turkish was inferred from the author's chat language rather than required by
|
||||
the product. `measurement_massprops` became `measure_from_handle`: no centroid
|
||||
or moment-of-inertia tool ships.
|
||||
From a checkout: `pip install -e ".[full]"` then `python server.py`.
|
||||
|
||||
```bash
|
||||
python -m benchmarks.render_matrix_chart
|
||||
```
|
||||
|
||||
### 3 · Headless performance — scale evidence
|
||||
|
||||
|
||||
|
||||
| Workload | Wall time | Throughput / result |
|
||||
|---|---:|---|
|
||||
| Create 2,000 lines (individual calls) | 0.32 s | ~6,300 entities/s |
|
||||
| 10,000 lines: build + DXF export + reopen | 2.16 s | ~4,600 entities/s end-to-end |
|
||||
| Region query over 10,000 entities | 0.89 s | 2,500 matched |
|
||||
| Premium pass (layers + part + dims + full critique) | 0.18 s | 0 issues |
|
||||
|
||||
These are roughly 2–6× the numbers the v1.4 README printed, and **almost none
|
||||
of that is this release**: the old report was recorded on CPython 3.14 and this
|
||||
one on 3.11.15. Interpreter, not code. Re-measuring v1.4.0 in a worktree on the
|
||||
*same* interpreter with the *same* suite file gives the honest comparison, and
|
||||
it does not flatter us — median of three runs each:
|
||||
|
||||
| Workload | v1.4.0 | v1.5.0 | Change |
|
||||
|---|---:|---:|---|
|
||||
| Create 2,000 lines | 245 ms | 315 ms | **28% slower** |
|
||||
| 10,000-line roundtrip | 1,765 ms | 2,146 ms | **22% slower** |
|
||||
| Region query over 10,000 | 1,090 ms | 1,039 ms | within run-to-run spread |
|
||||
| Premium pass | 375 ms | 44 ms | faster, but the v1.4 spread (279–772 ms) is too wide to put a number on |
|
||||
|
||||
The regression is fully attributed: setting `EZDXF_CALL_TIMEOUT=0` returns
|
||||
v1.5.0 to **244.8 ms** and **1,679 ms** — v1.4.0's numbers exactly. The whole
|
||||
cost is the per-call `asyncio.wait_for` this release wrapped around every
|
||||
headless call, so that one hung ezdxf call can no longer wedge a server whose
|
||||
document lock is a single `asyncio.Lock`. That is a deliberate trade of ~25% of
|
||||
creation throughput for a deadlock that used to be unrecoverable, and the knob
|
||||
to take it back is documented rather than hidden.
|
||||
|
||||
Workloads call the same backend methods the MCP tools call, so server-side
|
||||
overhead is included. Self-measurement only — competitor servers would pay an
|
||||
extra stdio serialization cost that in-process runs do not, so no cross-server
|
||||
timing claims are made. Numbers move with hardware; the machine fingerprint is
|
||||
recorded in the report.
|
||||
|
||||
```bash
|
||||
python -m benchmarks.perf_suite --out benchmarks/results/published/perf-ezdxf.json
|
||||
python -m benchmarks.render_perf_chart
|
||||
```
|
||||
|
||||
### 4 · Version A/B — every release re-proves correctness
|
||||
|
||||
`compare_versions.py` runs the same 24 deterministic, headless correctness
|
||||
checks against the previous release tag and the current tree — each check in
|
||||
its own subprocess, so a hard crash counts as a miss instead of killing the
|
||||
run.
|
||||
|
||||
**v1.5.0 release gate** (baseline `v1.4.0`, report:
|
||||
[`ab-v1.4.0-vs-v1.5.0.json`](benchmarks/results/published/ab-v1.4.0-vs-v1.5.0.json)):
|
||||
|
||||
| Version | Checks passing | Pass rate | Fixed | Regressed |
|
||||
|---|---:|---:|---:|---:|
|
||||
| v1.4.0 (baseline) | 21 / 24 | 87.5 % | — | — |
|
||||
| **v1.5.0** (this release) | **24 / 24** | **100 %** | 3 | **0** |
|
||||
|
||||
The three the baseline misses are *new capability, not repaired regressions*:
|
||||
v1.4.0 has no `entity_measure` and no boundary tracing, so it misses them by
|
||||
not having the method. They are in the suite because all three broke during
|
||||
v1.5.0 development and nothing outside their own unit tests would have caught
|
||||
it. The 21 pre-existing checks are **unchanged and all still pass** — the whole
|
||||
of the discovery layer, the contract split, the layout/viewport family and Wave
|
||||
A landed with **zero correctness regressions**.
|
||||
|
||||
For contrast, the same suite caught the v1.0.0 → v1.1.0 jump:
|
||||
|
||||
| Version | Checks passing | Pass rate |
|
||||
|---|---:|---:|
|
||||
| v1.0.0 (first public release) | 8 / 21 | 38.1 % |
|
||||
| v1.1.0 | 21 / 21 | 100 % |
|
||||
|
||||
```bash
|
||||
python benchmarks/compare_versions.py v1.4.0 --json ab.json
|
||||
```
|
||||
|
||||
Full rubric, caveats, method and boundaries for every lane:
|
||||
[`benchmarks/README.md`](benchmarks/README.md).
|
||||
|
||||
## What ships in v1.5
|
||||
|
||||
| Area | Tools |
|
||||
|---|---|
|
||||
| Drawing lifecycle | create, open, save, export DXF/PDF, audit **(now repairs)**, purge, undo/redo |
|
||||
| Geometry | lines, arcs, polylines, splines, hatches, trim/extend/fillet/chamfer, handle-preserving edits |
|
||||
| Annotation | ISO 129 toleranced dimensions, ISO 286 fits (`fit="H7"`), TABLE, MLEADER, GD&T frames + datums (ISO 1101) |
|
||||
| Engineering generators | involute gears (front + section A-A), DIN 6885 keyed bores, ISO A3 titleblock |
|
||||
| **Discovery** *(new)* | `DISCOVERY_MODE=search` replaces the catalog with `search_tools` + `call_tool`, ranked over an AutoCAD command and synonym corpus — `FILLET`, `BPOLY`, `QSELECT`, `WBLOCK` score nothing against a stock index because the words are simply not in the data |
|
||||
| **Batching** *(new)* | `cad_batch` runs a whole step list in one round trip; `fields=` projection trims result-heavy responses |
|
||||
| **Paper space** *(new)* | full tab lifecycle (`layout_list/create/set_current/delete/rename/copy`), viewports (`viewport_create/list/set_scale/lock/delete`), `entity_change_space` (CHSPACE), `drawing_export_pdf(layout=...)` |
|
||||
| **Selection** *(new)* | `selection_window` / `selection_polygon` / `selection_filter` — window vs crossing stated back to the caller, and a polygon tested against its own shape rather than its bounding box |
|
||||
| **Boundaries** *(new)* | `boundary_trace` (BOUNDARY/BPOLY) and `boundary_from_entities` chain loose edges into one closed polyline, arcs kept as bulges |
|
||||
| **Measurement** *(new)* | `analysis_measure_entity` measures what is *in* the drawing by handle — reading vertices back and shoelacing them loses 28.2% of the area on a semicircular edge |
|
||||
| **Hatch depth** *(new)* | gradients, in-place edits, typed edge boundaries, island styles |
|
||||
| **Annotation objects** *(new)* | WIPEOUT, REVCLOUD, MTEXT background masks, text find/replace |
|
||||
| **3D solids** *(opt-in)* | `solid_box/cylinder/extrude/revolve/boolean` on live AutoCAD (`ENABLE_3D=true`) |
|
||||
| Quality loop | `drawing_preflight` → `drawing_plan` → `drawing_critique` → `drawing_refine` → `drawing_finalize` (0-100 score) |
|
||||
| Delivery | `drawing_deliver`: DXF/PDF/PNG + SHA-256 manifest + reopen-parity checks |
|
||||
| Profiles | `TOOL_PROFILE=lean` (~47 curated tools) for clients with tight tool caps; `core` was removed — the discovery layer replaced it |
|
||||
|
||||
**Every coordinate in and out of a tool is WCS, on both engines.** DXF stores
|
||||
CIRCLE / ARC / LWPOLYLINE / TEXT / INSERT geometry in a per-entity frame, and
|
||||
before v1.5.0 the word `extrusion` did not appear anywhere in this codebase —
|
||||
so a mirrored or rotated entity reported coordinates that were quietly wrong.
|
||||
`backends/ocs.py` now translates at the backend boundary; an entity in a plane
|
||||
tilted out of WCS XY reports `plane_normal` and omits the fields xy cannot
|
||||
express, rather than answering with a number that looks fine.
|
||||
|
||||
The tool surface evolves — ask `system_about` for the live grouped inventory
|
||||
and `system_capabilities` for per-backend support modes instead of copying
|
||||
lists from this README.
|
||||
|
||||
## Architecture
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
A[MCP host / AI agent] --> B[FastMCP server]
|
||||
B --> C[Error - audit - timing - logging middleware]
|
||||
C --> D[Typed tool contract + tool profiles]
|
||||
D --> E[COM backend]
|
||||
D --> F[ezdxf backend]
|
||||
E --> G[Live AutoCAD]
|
||||
F --> H[Headless DXF]
|
||||
E --> I[Engineering quality + delivery]
|
||||
F --> I
|
||||
```
|
||||
|
||||
- `server.py` — FastMCP surface, lifespan, middleware, resources, prompts.
|
||||
- `backends/contracts/` — the typed contract, one module per domain.
|
||||
- `backends/capability.py` — capability model, typed refusals, `@capability`.
|
||||
- `backends/base.py` — shared data models + the contract composition.
|
||||
- `backends/com_backend.py` — single-STA-thread COM executor with per-call
|
||||
timeouts.
|
||||
- `backends/ezdxf_backend.py` — `asyncio.to_thread`-wrapped DXF engine with
|
||||
snapshot transactions.
|
||||
- `engineering/` — standards-aware planning, generators, critique, scoring,
|
||||
fits, refinement, validation, delivery.
|
||||
- `security.py` — path validation and command/AutoLISP sanitization.
|
||||
|
||||
## Backend capabilities
|
||||
|
||||
| Capability | COM backend | ezdxf backend |
|
||||
|---|:---:|:---:|
|
||||
| Live AutoCAD document control | ✓ | — |
|
||||
| Headless DXF creation and editing | — | ✓ |
|
||||
| Cross-platform execution | — | ✓ |
|
||||
| Transactions and rollback | ✓ | ✓ |
|
||||
| Paper-space layouts + viewports | ✓ | ✓ |
|
||||
| Viewport model-content rendering | ✓ | ✓ (no viewport borders) |
|
||||
| CHSPACE (`entity_change_space`) | — (no ActiveX member) | ✓ |
|
||||
| WIPEOUT / REVCLOUD | — (no ActiveX member) | ✓ |
|
||||
| Boundary tracing (BPOLY) | — (no ActiveX member) | ✓ |
|
||||
| Selection window / crossing / polygon | ✓ | ✓ |
|
||||
| Entity area by handle | ActiveX `.Area` | Analytic, bulges included |
|
||||
| HATCH filled area (islands subtracted) | AutoCAD's own number | Loops walked, `hatch_style` reported |
|
||||
| REGION / 3DSOLID area | ✓ | — (ACIS is opaque to ezdxf) |
|
||||
| TABLE and MLEADER semantics | Native | Portable composite |
|
||||
| 3D solids (opt-in) | ✓ | — (no headless ACIS) |
|
||||
| Screenshots | AutoCAD window | Matplotlib render |
|
||||
| Raw AutoCAD commands / AutoLISP | Opt-in | — |
|
||||
|
||||
`system_capabilities` returns this machine-readably (`native`, `composite`,
|
||||
`rendered`, `snapshot`, `shared`, `unsupported`) — use it at runtime instead of
|
||||
assuming.
|
||||
|
||||
## A production drawing workflow
|
||||
|
||||
```text
|
||||
drawing_preflight → drawing_plan → drawing_apply_iso_layers
|
||||
→ deterministic create/edit tools → dimension_auto (+ fit="H7" callouts)
|
||||
→ drawing_critique → drawing_refine
|
||||
→ layout_create + viewport_create → drawing_finalize → drawing_deliver
|
||||
```
|
||||
|
||||
## Connect an MCP client
|
||||
|
||||
### Claude Desktop / Cursor / any stdio host
|
||||
### Wire it to a client
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -338,88 +86,256 @@ drawing_preflight → drawing_plan → drawing_apply_iso_layers
|
||||
"env": {
|
||||
"AUTOCAD_MCP_BACKEND": "auto",
|
||||
"ALLOWED_PATHS": "C:\\Users\\you\\Documents\\AutoCAD",
|
||||
"TOOL_PROFILE": "full"
|
||||
"TOOL_PROFILE": "full",
|
||||
"DISCOVERY_MODE": "off"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
After `pip install autocad-mcp-pro` the `autocad-mcp` command is on PATH; from
|
||||
a checkout use `"command": "python"`, `"args": ["path/to/server.py"]`. The
|
||||
model vendor is not part of the server contract.
|
||||
Works with Claude Desktop, Cursor, or any stdio MCP host. For HTTP: `autocad-mcp --transport http --port 8000` — loopback only unless remote HTTP is explicitly enabled **and** a bearer token is set. The model vendor is not part of the server contract.
|
||||
|
||||
### Local HTTP
|
||||
## What you get
|
||||
|
||||
```bash
|
||||
autocad-mcp --transport http --port 8000
|
||||
```
|
||||
<table>
|
||||
<tr><th align="left">Area</th><th align="left">What it does</th></tr>
|
||||
<tr><td><b>Drawing lifecycle</b></td><td>create, open, save, export DXF/PDF, audit <i>(now repairs)</i>, purge, undo/redo</td></tr>
|
||||
<tr><td><b>Geometry</b></td><td>lines, arcs, polylines, splines, hatches, trim/extend/fillet/chamfer, handle-preserving edits</td></tr>
|
||||
<tr><td><b>Annotation</b></td><td>ISO 129 toleranced dimensions, ISO 286 fits (<code>fit="H7"</code>), TABLE, MLEADER, GD&T frames and datums (ISO 1101)</td></tr>
|
||||
<tr><td><b>Engineering generators</b></td><td>involute gears (front + section A-A), DIN 6885 keyed bores, ISO A3 title block</td></tr>
|
||||
<tr><td><b>Discovery</b> ✨</td><td><code>search_tools</code> ranked over an AutoCAD command and synonym corpus — <code>FILLET</code>, <code>BPOLY</code>, <code>QSELECT</code>, <code>WBLOCK</code>, <code>OVERKILL</code>, <code>CHSPACE</code> each rank <b>#1</b> of the 149-tool advertised catalog</td></tr>
|
||||
<tr><td><b>Batching</b> ✨</td><td><code>cad_batch</code> runs a whole step list in one round trip; <code>fields=</code> projects the response on 11 result-heavy tools (a closed, gated list)</td></tr>
|
||||
<tr><td><b>Paper space</b> ✨</td><td>tab lifecycle, viewports, <code>entity_change_space</code> (CHSPACE), <code>drawing_export_pdf(layout=...)</code></td></tr>
|
||||
<tr><td><b>Selection</b> ✨</td><td>window vs crossing stated back to the caller; a polygon tested against its own shape, not its bounding box</td></tr>
|
||||
<tr><td><b>Boundaries</b> ✨</td><td><code>boundary_trace</code> (BOUNDARY/BPOLY) chains loose edges into one closed polyline, arcs kept as bulges</td></tr>
|
||||
<tr><td><b>Measurement</b> ✨</td><td><code>analysis_measure_entity</code> measures what is <i>in</i> the drawing, by handle</td></tr>
|
||||
<tr><td><b>Hatch depth</b> ✨</td><td>gradients, in-place edits, typed edge boundaries, island styles</td></tr>
|
||||
<tr><td><b>Annotation objects</b> ✨</td><td>WIPEOUT, REVCLOUD, MTEXT background masks, text find/replace</td></tr>
|
||||
<tr><td><b>3D solids</b></td><td><code>solid_box/cylinder/extrude/revolve/boolean</code> on live AutoCAD (<code>ENABLE_3D=true</code>)</td></tr>
|
||||
<tr><td><b>Quality loop</b></td><td><code>drawing_preflight</code> → <code>drawing_plan</code> → <code>drawing_critique</code> → <code>drawing_refine</code> → <code>drawing_finalize</code> (0–100 score)</td></tr>
|
||||
<tr><td><b>Delivery</b></td><td><code>drawing_deliver</code>: DXF/PDF/PNG + SHA-256 manifest + reopen-parity checks</td></tr>
|
||||
</table>
|
||||
|
||||
Loopback only by default; a non-loopback bind is refused unless remote HTTP is
|
||||
explicitly enabled **and** a bearer token is configured.
|
||||
<sub>✨ = new in v1.5.0. 154 tools across 19 groups — <code>entity_creation</code> 18, <code>entity_modification</code> 15, <code>layers</code> 14, <code>analysis</code> 12, <code>layouts</code> 12, <code>premium</code> 12, <code>drawing</code> 11, <code>engineering</code> 10, <code>blocks</code> 8, <code>entity_query</code> 8, <code>system</code> 7, <code>dimensions</code> 5, <code>solids</code> 5, <code>corner_ops</code> 4, <code>view</code> 4, <code>batch</code> 3, <code>transactions</code> 3, <code>templates</code> 2, <code>validation</code> 1.</sub>
|
||||
|
||||
Alongside the tools, **6 MCP resources** cost nothing in the tool budget and are browsable at any time — `autocad://drawing/info`, `autocad://layers`, `autocad://blocks`, `autocad://entities/stats`, `autocad://entities/{layer_name}`, `autocad://system/status` — plus **5 prompt templates** (`prompt_mechanical_drawing`, `prompt_floor_plan`, `prompt_electrical_schematic`, `prompt_pid_diagram`, `prompt_quick_drawing`).
|
||||
|
||||
### Two rules worth knowing before you draw
|
||||
|
||||
**Every coordinate in and out of a tool is WCS, on both engines** — with one deliberate exception. DXF stores CIRCLE / ARC / LWPOLYLINE / TEXT / INSERT geometry in a per-entity frame, and before v1.5.0 the word `extrusion` appeared nowhere in this codebase, so a mirrored or rotated entity reported coordinates that were quietly wrong. An entity in a plane tilted out of WCS XY now reports `plane_normal` and omits the fields xy cannot express, rather than answering with a number that looks fine. The exception is **TEXT `rotation`**, which stays in the entity frame on purpose: a mirrored TEXT is mirror-imaged, and no single scalar angle expresses that.
|
||||
|
||||
**Never read vertices back and shoelace them.** That loses **28.2%** of the area on a semicircular edge, silently. `analysis_measure_entity(handle)` reads the real geometry and states its own accuracy: `exact`, `flatten_tolerance`, `assumed_closed`, `self_intersecting`.
|
||||
|
||||
## The two engines
|
||||
|
||||
One contract, split across 19 modules in `backends/contracts/`. Most are ABCs; `gdt.py` and `settings.py` are concrete shared mixins rather than interfaces. Optional methods are declarative — `@capability(key, reason=...)` supplies a default that *refuses* — and a test holds the two backends' capability key sets equal, so a key cannot exist on one engine and be forgotten on the other.
|
||||
|
||||
There are **27 capability keys**. `system_capabilities` returns the live map, and you should read it at runtime rather than trusting the table below, because five of the 27 depend on how the machine is set up:
|
||||
|
||||
| Capability | COM (live AutoCAD) | ezdxf (headless) |
|
||||
|---|:---:|:---:|
|
||||
| Live document control | ✅ | — |
|
||||
| Cross-platform, no AutoCAD | — | ✅ |
|
||||
| Transactions and rollback | ✅ | ✅ |
|
||||
| Paper-space layouts + viewports | ✅ | ✅ |
|
||||
| Viewport model-content rendering | ✅ | ✅ ᵐ *(no viewport borders)* |
|
||||
| Selection window / crossing / polygon | ✅ | ✅ |
|
||||
| Entity area by handle | ActiveX `.Area` | Analytic, bulges included |
|
||||
| HATCH filled area (islands subtracted) | AutoCAD's own number | Loops walked, `hatch_style` reported |
|
||||
| REGION / 3DSOLID area | ✅ | — *ACIS is opaque to ezdxf* |
|
||||
| 3D solids | ✅ *only with `ENABLE_3D=true`* | — *no headless ACIS* |
|
||||
| DWG write | ✅ | — *no DWG writer* |
|
||||
| WIPEOUT | — *`AddWipeout` verified absent, AutoCAD 2026* | ✅ |
|
||||
| MTEXT background colour | — *`BackgroundFillColor` verified absent, AutoCAD 2026* | ✅ |
|
||||
| REVCLOUD, BPOLY / boundary tracing | — *no ActiveX member; command-line only* | ✅ |
|
||||
| CHSPACE (`entity_change_space`) | — *unverified against a live seat* | ✅ ᶜ |
|
||||
| Undo history | ✅ | opt-in — `EZDXF_UNDO_DEPTH` defaults to `0` |
|
||||
| TABLE and MLEADER | Native | Portable composite |
|
||||
| Raw commands / AutoLISP | Opt-in | — |
|
||||
| Screenshots and PDF export | AutoCAD window capture | Matplotlib render ᵐ |
|
||||
|
||||
<sub>ᵐ Needs matplotlib — the `[pdf]` or `[full]` extra. Without it `png`, `pdf`, `viewport_render` and `handle_overlay` all report unsupported on the headless engine.<br>
|
||||
ᶜ The headless CHSPACE carries four named restrictions in its own capability reason: top-view untwisted viewports only, dimensions refused unless frozen, ACIS/proxy/table refused, and viewport clipping reported rather than applied.</sub>
|
||||
|
||||
> [!IMPORTANT]
|
||||
> Every COM path added in v1.5.0 was executed against a **live AutoCAD 2026** before release. That run found four defects no amount of reading would have caught — the largest being that `AcadPViewport` has no `ViewCenter` member, so the line setting it had been silently swallowed by a bare `except` since v1.4.
|
||||
|
||||
## Evidence
|
||||
|
||||
Every number below is produced by a script in [`benchmarks/`](https://github.com/U-C4N/Autocad-MCP/blob/main/benchmarks/) and rendered to SVG by Python. No hand-edited charts.
|
||||
|
||||
### 1 · Live-run scores
|
||||
|
||||
<img src="https://raw.githubusercontent.com/U-C4N/Autocad-MCP/main/docs/assets/autocad-mcp-livebench.svg" alt="Live-run score bars: autocad-mcp-pro 100 on the v3 matrix, beiming183 50 and puran-water 45 on v2" width="820">
|
||||
|
||||
| Server | Matrix | Pinned | Score | Pass |
|
||||
|---|---|---|---:|---:|
|
||||
| **autocad-mcp-pro** *(this repo)* | v3 · 15 tasks | working tree | **100.0** | 15/15 |
|
||||
| [beiming183-cloud/AutoCAD-MCP](https://github.com/beiming183-cloud/AutoCAD-MCP) | v2 · 10 tasks | `11f7c47` | 50.0 | 5/10 |
|
||||
| [puran-water/autocad-mcp](https://github.com/puran-water/autocad-mcp) | v2 · 10 tasks | `95476a3` | 45.0 | 4/10 |
|
||||
|
||||
> [!WARNING]
|
||||
> **Read the matrix column before the score column.** The five v3 tasks are new in v1.5.0; the competitor runs are from 2026-07-23 against pinned competitor commits, and were never put to them. Their scores are over the ten tasks they actually ran, and the five new rows render blank rather than as zeros — an invented zero is indistinguishable from a measured one. These bars do not share a denominator, which is why each one carries its matrix.
|
||||
|
||||
Competitors are driven **black-box over MCP stdio** using their own documented tool contracts at the pinned commit, and every geometry claim is verified by re-opening the exported DXF with ezdxf inside the harness. No self-reporting. A capability refusal is recorded `unsupported`, not `fail` — "cannot reach this" and "got this wrong" are different findings.
|
||||
|
||||
### 2 · Task matrix — where the score comes from
|
||||
|
||||
<img src="https://raw.githubusercontent.com/U-C4N/Autocad-MCP/main/docs/assets/autocad-mcp-taskmatrix.svg" alt="Task matrix heatmap: 15 tasks against three servers" width="820">
|
||||
|
||||
**v2 scored this repo 10/10, and that number carried no information**: every task in it exercised something the server was built around. v3 adds five tasks chosen because they *can* fail — and three of them did, while being written.
|
||||
|
||||
| v3 task | Verified against |
|
||||
|---|---|
|
||||
| `tool_discovery` | six AutoCAD command names, each ranking **#1** of the 149-tool advertised catalog |
|
||||
| `token_budget` | 40,305 → 356 tokens advertised, against a 2,000 ceiling fixed in advance |
|
||||
| `hatch_islands` | **300** filled with the island, 400 ignoring it |
|
||||
| `selection_filter` | window 1, crossing 2, bounding box 3, polygon 1 |
|
||||
| `measure_from_handle` | **139.2699** against the 100.0 a vertex shoelace gives |
|
||||
|
||||
### 3 · Correctness A/B — every release re-proves itself
|
||||
|
||||
26 deterministic headless checks against the previous tag and the current tree, each in its own subprocess so a hard crash counts as a miss rather than killing the run.
|
||||
|
||||
| Version | Checks passing | Pass rate | Fixed | Regressed |
|
||||
|---|---:|---:|---:|---:|
|
||||
| v1.4.0 *(baseline)* | 21 / 26 | 80.8 % ᵃ | — | — |
|
||||
| **v1.5.0** *(this release)* | **26 / 26** | **100 %** | 5 | **0** |
|
||||
|
||||
<sub>ᵃ Against the 26-check v1.5.0 suite. v1.4.0 passed 21/21 of the checks that existed when it shipped.</sub>
|
||||
|
||||
The five splits into two kinds, and the difference matters:
|
||||
|
||||
- **Three are new capability, not repaired regressions.** v1.4.0 has no `entity_measure` and no boundary tracing, so it misses `hatch_area_subtracts_its_island`, `boundary_area_agrees_with_measure` and `two_vertex_circle_has_area` by not having the method.
|
||||
- **Two are real defects that v1.4.0 shipped.** It has `dimension_diameter` and `dimension_radius`, and both returned the wrong number: `leader_length` — a *text placement* — was being measured as geometry, so at default settings every diameter callout came out **2 × leader_length too large** (60 on a true ⌀40) and every radius `leader_length` too large. Found while building the drawing at the top of this page. The live COM backend was never affected.
|
||||
|
||||
The 21 pre-existing checks are unchanged and all still pass: the discovery layer, `cad_batch`, the 19-module contract split, layouts/viewports and Wave A landed with **zero correctness regressions**.
|
||||
|
||||
### 4 · Headless performance
|
||||
|
||||
<img src="https://raw.githubusercontent.com/U-C4N/Autocad-MCP/main/docs/assets/autocad-mcp-perf.svg" alt="Performance chart across four headless workloads" width="820">
|
||||
|
||||
| Workload | Wall time | Result |
|
||||
|---|---:|---|
|
||||
| Create 2,000 lines (individual calls) | 0.32 s | ~6,300 entities/s |
|
||||
| 10,000 lines: build + DXF export + reopen | 2.16 s | ~4,600 entities/s end-to-end |
|
||||
| Region query over 10,000 entities | 0.89 s | 2,500 matched |
|
||||
| Premium pass (layers + part + dims + full critique) | 0.18 s | 0 issues |
|
||||
|
||||
Workloads call the same backend methods the MCP tools call, so server-side overhead is included. **Self-measurement only** — competitor servers pay an extra stdio serialization cost that in-process runs do not, so no cross-server timing claims are made.
|
||||
|
||||
> [!CAUTION]
|
||||
> These are **not** an improvement over v1.4.0. Measured against v1.4.0 on the same interpreter, this release is **28% slower** at creating entities — see [what this release is bad at](#what-this-release-is-bad-at) for the number and its full attribution.
|
||||
|
||||
### 5 · Source-review leaderboard
|
||||
|
||||
<img src="https://raw.githubusercontent.com/U-C4N/Autocad-MCP/main/docs/assets/autocad-mcp-benchmark.svg" alt="Capability leaderboard across nine public AutoCAD MCP servers" width="820">
|
||||
|
||||
Nine public AutoCAD MCP projects against a fixed 100-point rubric — CAD breadth (25), correctness and delivery (20), backends/platforms (15), engineering production (15), tests and maintenance (15), security and operations (10). Stars and raw tool counts score nothing. Data and dates in [`benchmarks/source_review.json`](https://github.com/U-C4N/Autocad-MCP/blob/main/benchmarks/source_review.json).
|
||||
|
||||
**This table is not re-scored per release.** This repository's row is still the one written against **v1.4.0**. Raising our own number for v1.5.0 would mean re-reviewing eight other projects the same day to keep it fair, and that has not been done — so the score stays where the evidence is.
|
||||
|
||||
Method, caveats and boundaries for every lane: [`benchmarks/README.md`](https://github.com/U-C4N/Autocad-MCP/blob/main/benchmarks/README.md).
|
||||
|
||||
## What this release is bad at
|
||||
|
||||
A page that only lists strengths is a page that has not been measured.
|
||||
|
||||
**Entity creation is 28% slower than v1.4.0.** Measured on one interpreter, median of three runs: 2,000 lines went 245 ms → 315 ms, and the 10,000-line roundtrip 1,765 ms → 2,146 ms. Fully attributed — setting `EZDXF_CALL_TIMEOUT=0` returns v1.5.0 to **244.8 ms** and **1,679 ms**, which are v1.4.0's numbers exactly. The cost is the per-call `asyncio.wait_for` this release wrapped around every headless call so that one hung call can no longer wedge a server whose document lock is a single `asyncio.Lock`. A deliberate trade, with a documented knob, on the 1.6 roadmap to win back.
|
||||
|
||||
> The v1.4 README printed *slower* numbers than the table above — 0.64 s and 13.9 s for the same two workloads. That looks like a 2–6× win and is not one: the old report was recorded on CPython 3.14 and this one on 3.11.15. Interpreter, not code. Comparing the two published files directly is the mistake this paragraph exists to stop.
|
||||
|
||||
**Wave A shipped 13 tools of a planned 39.** Every cut is backed by a measurement. The REGION and 2D-boolean family went because `add_region()` produces a REGION with **zero ACIS bytes**, and the `greiner_hormann` substitute loses **28.2%** of the area on a square with one semicircular edge — shipping it would have re-created the exact class of silent error this release spent four milestones deleting. Most of the planned measurement family was already covered by the eight `analysis_*` tools that existed.
|
||||
|
||||
**`system_run_command` / `system_run_lisp` are a guardrail, not a security boundary** — the rejection message clients receive says so in those words. A 36-verb command denylist and a 24-pattern AutoLISP denylist refuse the obviously destructive cases, but AutoCAD accepts hundreds of commands and any loaded ARX/LISP adds more, so the list cannot be complete, and `DANGEROUS_COMMANDS_ENABLED=true` switches both off entirely.
|
||||
|
||||
**Path validation is per-tool, and unscoped until you scope it.** Nine tools call `validate_path`; nothing in the middleware enforces it server-wide. With `ALLOWED_PATHS` empty — the default — the only positive bound is a ten-entry system-directory denylist that does not include `C:/Users`, `/home`, `/root` or `/var`. **Set `ALLOWED_PATHS`.** It is the setting that actually scopes the filesystem.
|
||||
|
||||
**Non-AutoCAD ProgIDs are unverified.** `CAD_PROGID` changes which COM application the backend attaches to, and nothing beyond the connection has been tested against BricsCAD, ZWCAD or GstarCAD. They are listed in `.env.example` commented out and marked UNVERIFIED.
|
||||
|
||||
## Configuration
|
||||
|
||||
Nothing here loads a `.env` file — there is no dotenv dependency. Export these, or set them in your MCP client's `env` block. `.env.example` is documentation, and it is not the full list.
|
||||
|
||||
<details>
|
||||
<summary><b>All 16 environment variables</b></summary>
|
||||
|
||||
| Variable | Default | Purpose |
|
||||
|---|---|---|
|
||||
| `AUTOCAD_MCP_BACKEND` | `auto` | `auto`, `com`, or `ezdxf` |
|
||||
| `CAD_PROGID` | `AutoCAD.Application` | COM ProgID the live backend attaches to (see below) |
|
||||
| `TOOL_PROFILE` | `full` | `lean` (~47 curated tools) or `full` |
|
||||
| `CAD_PROGID` | `AutoCAD.Application` | COM ProgID the live backend attaches to (unverified for clones) |
|
||||
| `TOOL_PROFILE` | `full` | `lean` (47 curated tools) or `full` |
|
||||
| `DISCOVERY_MODE` | `off` | `search` replaces the catalog with `search_tools` + `call_tool` |
|
||||
| `ENABLE_3D` | `false` | Expose the opt-in `solid_*` tools (COM backend) |
|
||||
| `LOG_LEVEL` | `INFO` | Python logging level |
|
||||
| `ALLOWED_PATHS` | empty | Comma-separated absolute paths the server may access |
|
||||
| `ALLOWED_PATHS` | *(empty)* | Comma-separated absolute paths the server may access |
|
||||
| `MAX_UNDO_STACK` | `5` | Maximum retained undo snapshots |
|
||||
| `EZDXF_UNDO_DEPTH` | `0` | Headless undo history depth; `0` disables it |
|
||||
| `MAX_DXF_BYTES` | `52428800` | Reject larger DXF input; `0` disables |
|
||||
| `MAX_LIST_LIMIT` | `5000` | Bound list/selection response sizes |
|
||||
| `COM_CALL_TIMEOUT` | `60` | Per-call live AutoCAD timeout (seconds); `0` disables |
|
||||
| `EZDXF_CALL_TIMEOUT` | `120` | Per-call headless ezdxf timeout (seconds); `0` disables |
|
||||
| `EZDXF_CALL_TIMEOUT` | `120` | Per-call headless timeout (seconds); `0` disables |
|
||||
| `DANGEROUS_COMMANDS_ENABLED` | `false` | Allow blocked commands/LISP; reported as unsafe mode |
|
||||
| `ALLOW_REMOTE_HTTP` | `false` | Permit a non-loopback HTTP bind |
|
||||
| `MCP_AUTH_TOKEN` | empty | Bearer token required for remote HTTP |
|
||||
| `MCP_AUTH_TOKEN` | *(empty)* | Bearer token required for remote HTTP |
|
||||
|
||||
`CAD_PROGID` only changes which COM application the backend attaches to — every
|
||||
tool beyond the connection itself is developed and tested against AutoCAD alone,
|
||||
and AutoCAD-clone compatibility is unverified, so treat a non-default ProgID as
|
||||
experimental. Candidates such as `BricscadApp.AcadApplication`,
|
||||
`ZWCAD.Application` and `GstarCAD.Application` are listed in `.env.example`
|
||||
commented out and marked UNVERIFIED: none has been tested against a seat. An
|
||||
unrecognised ProgID fails loudly rather than falling back, because the fallback
|
||||
path COM-launches the application and would start the very AutoCAD you said you
|
||||
were not using.
|
||||
</details>
|
||||
|
||||
## Security model
|
||||
## Architecture
|
||||
|
||||
Tool input is treated as untrusted:
|
||||
```mermaid
|
||||
flowchart LR
|
||||
A[MCP host / AI agent] --> B[FastMCP 3 server]
|
||||
B --> C[Error · audit · timing · logging middleware]
|
||||
C --> D[Typed contract · 19 modules · 27 capabilities]
|
||||
D --> E[COM backend<br/>single-STA thread]
|
||||
D --> F[ezdxf backend<br/>asyncio.to_thread]
|
||||
E --> G[Live AutoCAD]
|
||||
F --> H[Headless DXF]
|
||||
E --> I[Engineering · critique · scoring · delivery]
|
||||
F --> I
|
||||
```
|
||||
|
||||
- every path passes traversal + allowed-root validation, and non-local anchors
|
||||
(`\\?\`, UNC, `\\localhost\C$`) are rejected in every spelling — the
|
||||
forward-slash forms used to normalise past the pattern check and then compare
|
||||
unequal to the system-directory list;
|
||||
- `system_run_command` / `system_run_lisp` are free-text escape hatches into the
|
||||
live AutoCAD command line. A **verb** denylist (the first token of each macro
|
||||
segment, decorators stripped) refuses the obviously destructive ones by
|
||||
default, and quoted AutoLISP string literals are treated as data so drawing
|
||||
text is not mistaken for code. Treat this as a guardrail against an agent
|
||||
accidentally issuing `ERASE ALL` — **not** as a boundary against a hostile
|
||||
client. AutoCAD accepts hundreds of commands and any loaded ARX/LISP adds
|
||||
more, so the list cannot be complete. The real boundaries are `ALLOWED_PATHS`,
|
||||
not exposing the server over the network, and preferring the typed tools
|
||||
(which validate every argument) over raw commands;
|
||||
- remote HTTP requires explicit opt-in plus a bearer token;
|
||||
- DXF size, response sizes, undo history and COM call duration are bounded;
|
||||
- audit middleware records tool timing and failures;
|
||||
- `system_status` / `system_about` expose `unsafe_mode` when dangerous
|
||||
operations are enabled.
|
||||
| Path | Role |
|
||||
|---|---|
|
||||
| `server.py` | FastMCP surface, lifespan, middleware, resources, prompts |
|
||||
| `backends/contracts/` | The typed contract, one module per domain |
|
||||
| `backends/capability.py` | Capability model, typed refusals, `@capability` |
|
||||
| `backends/ocs.py` | OCS ↔ WCS translation at the backend boundary |
|
||||
| `backends/com_backend.py` | Single-STA-thread COM executor with per-call timeouts |
|
||||
| `backends/ezdxf_backend.py` | `asyncio.to_thread`-wrapped DXF engine, snapshot transactions |
|
||||
| `discovery/` | Alias corpus, ranking transform, compact serializer |
|
||||
| `engineering/` | Standards, generators, critique, fits, scoring, delivery |
|
||||
| `security.py` | Path validation, command and AutoLISP sanitization |
|
||||
|
||||
Please report vulnerabilities privately via [GitHub](https://github.com/U-C4N)
|
||||
rather than public issues.
|
||||
### The drawing at the top of this page, as tool calls
|
||||
|
||||
Not pseudocode — this is the sequence in [`scripts/render_readme_showcase.py`](https://github.com/U-C4N/Autocad-MCP/blob/main/scripts/render_readme_showcase.py), which produced the sheet in the hero image:
|
||||
|
||||
```text
|
||||
drawing_new() fresh document, ISO linetypes + layers bootstrapped
|
||||
drawing_apply_iso_layers("mech") ISO 128 lineweights per layer
|
||||
drawing_settings({units: "mm", linear_precision: 2})
|
||||
|
||||
gear_draw_spur_front_view( involute flanks, not a decorated circle
|
||||
module=6, teeth=24, center=[135, 172],
|
||||
bore_diameter=40, keyway_width=12, keyway_depth=3.3)
|
||||
gear_draw_section_aa(x_offset=300, face_width=46)
|
||||
|
||||
dimension_linear(...) ISO 129
|
||||
dimension_diameter(..., fit="H7") deviations from authored ISO 286 tables
|
||||
titleblock_apply_iso_a3(title="SPUR GEAR m6 z24", material="C45E", ...)
|
||||
|
||||
drawing_critique(focus=None) -> [] must be empty before finalize
|
||||
view_screenshot() -> PNG
|
||||
```
|
||||
|
||||
The full production loop adds `drawing_preflight` and `drawing_plan` at the front (the plan is held by the backend and replayed by `drawing_critique` at finalize time), `drawing_refine` after the critique, and `layout_create` + `viewport_create` + `drawing_deliver` at the end for a hashed, reopen-verified handoff.
|
||||
|
||||
## Development
|
||||
|
||||
`uv.lock` pins the whole transitive graph — runtime *and* dev tools — to the
|
||||
set the suite was last run against, so this reproduces the CI environment
|
||||
exactly rather than approximately:
|
||||
`uv.lock` pins the whole transitive graph — runtime *and* dev tools — so this reproduces CI exactly rather than approximately:
|
||||
|
||||
```bash
|
||||
uv sync --locked --all-extras
|
||||
@@ -427,82 +343,39 @@ uv run pytest
|
||||
uv run ruff check . && uv run ruff format --check .
|
||||
```
|
||||
|
||||
Without uv, from the declared dependency ranges (needs pip ≥ 25.1 for
|
||||
`--group`; resolves fresh, so it can differ from CI):
|
||||
|
||||
```bash
|
||||
pip install -e ".[full]"
|
||||
pip install --group dev build
|
||||
python -m pytest && python -m build
|
||||
```
|
||||
|
||||
CI installs with `uv sync --locked`, which fails rather than re-resolving, so a
|
||||
fresh run cannot silently pick up an untested upstream — run `uv lock` after
|
||||
changing dependencies or the lint job will reject the drift. The gates run on
|
||||
Linux (3.11/3.12), Windows (mocked-COM suite), plus package, Docker and
|
||||
MCP-registry-schema jobs; the package job deliberately stays *unlocked*,
|
||||
because it is the one that proves a plain `pip install autocad-mcp-pro` still
|
||||
resolves. A scheduled `deps-floating` workflow resolves the ranges fresh and
|
||||
re-runs the suite, so a locked CI does not become blindness to upstream drift.
|
||||
Releases are tag-driven: `git tag vX.Y.Z && git push origin vX.Y.Z` builds,
|
||||
publishes to PyPI and cuts the GitHub Release. See
|
||||
[`docs/RELEASE-DISTRIBUTION.md`](docs/RELEASE-DISTRIBUTION.md).
|
||||
|
||||
Repository map:
|
||||
|
||||
```text
|
||||
server.py FastMCP surface and orchestration
|
||||
config.py Environment-driven settings
|
||||
security.py Path, command, and AutoLISP guards
|
||||
version.py Canonical package version
|
||||
uv.lock Exact dependency graph CI installs from
|
||||
backends/ COM and ezdxf implementations
|
||||
engineering/ Standards, generators, critique, fits, delivery
|
||||
benchmarks/ Runners, adapters, published reports, chart renderers
|
||||
tests/ Headless, mocked-COM, contract, and security tests
|
||||
```
|
||||
CI runs on Linux (3.11 / 3.12), Windows (mocked-COM suite), plus package, Docker and MCP-registry-schema jobs. The package job deliberately stays *unlocked* — it is the one that proves a plain `pip install autocad-mcp-pro` still resolves. A scheduled `deps-floating` workflow resolves the ranges fresh and re-runs the suite, so a locked CI does not become blindness to upstream drift. Releases are tag-driven: `git tag vX.Y.Z && git push origin vX.Y.Z`.
|
||||
|
||||
## Roadmap (1.6)
|
||||
|
||||
**Give back the 25% creation throughput** the per-call ezdxf timeout costs
|
||||
(measured and attributed above) — arm it only for calls that can actually block,
|
||||
or find a deadline cheaper than an `asyncio.wait_for` per call. ISO 286
|
||||
interference **shafts** r/s/t/u (they need sub-stepped table data nothing in the
|
||||
module can derive, and guessing them would put wrong tolerances on a production
|
||||
drawing — so 1.5 refuses them by name instead). HATCH boundary *geometry* in
|
||||
`entity_get` — 1.5 measures a hatch's filled area but still does not hand back
|
||||
its loops. REGION/3DSOLID area on the headless engine, which needs a modelling
|
||||
kernel ezdxf does not have. An *allowlist* of permitted AutoLISP heads,
|
||||
replacing the denylist — enumerating dangerous symbols does not terminate.
|
||||
`ezdxf.recover` as a fallback on `drawing_open`. Features are published when
|
||||
their contracts and limitations are testable — not when they make a longer
|
||||
checklist.
|
||||
Give back the 25% creation throughput the per-call timeout costs — arm it only for calls that can actually block. ISO 286 interference **shafts** r/s/t/u (they need sub-stepped table data nothing in the module can derive; 1.5 refuses them by name instead of guessing). HATCH boundary *geometry* in `entity_get` — 1.5 measures a hatch's filled area but still does not hand back its loops. REGION/3DSOLID area headlessly, which needs a modelling kernel ezdxf does not have. An *allowlist* of permitted AutoLISP heads, replacing the denylist — enumerating dangerous symbols does not terminate. `ezdxf.recover` as a fallback on `drawing_open`.
|
||||
|
||||
Cut from 1.5 on evidence, not on time: Wave A shipped 13 of a planned 39 tools.
|
||||
The 2D-boolean and REGION family went because `add_region()` produces a REGION
|
||||
with zero ACIS bytes and the `greiner_hormann` substitute loses **28.2% of the
|
||||
area** on a square with one semicircular edge; most of the planned measurement
|
||||
family was already covered by the eight `analysis_*` tools that existed. The
|
||||
full cut list and its measurements are in
|
||||
[`CHANGELOG.md`](CHANGELOG.md).
|
||||
Features are published when their contracts and limitations are testable — not when they make a longer checklist.
|
||||
|
||||
## Star history
|
||||
|
||||
<div align="center">
|
||||
|
||||
<a href="https://star-history.com/#U-C4N/Autocad-MCP&Date">
|
||||
<img src="https://api.star-history.com/svg?repos=U-C4N/Autocad-MCP&type=Date&theme=dark" alt="Star history chart for U-C4N/Autocad-MCP" width="720">
|
||||
</a>
|
||||
|
||||
If this saved you a day of drafting, a star helps other engineers find it.
|
||||
|
||||
</div>
|
||||
|
||||
## Contributing
|
||||
|
||||
Pull requests are welcome — especially reproducible competitor adapters,
|
||||
mocked/live COM coverage, engineering standards, and backend parity. For a
|
||||
large change, open an issue first so the tool contract and evidence plan can
|
||||
be agreed before implementation.
|
||||
Pull requests are welcome — especially reproducible competitor adapters, mocked/live COM coverage, engineering standards, and backend parity. For a large change, open an issue first so the tool contract and evidence plan can be agreed before implementation.
|
||||
|
||||
## Author
|
||||
|
||||
**Umutcan Edizsalan** · Mechanical engineering work at **Anka-Makine** ·
|
||||
GitHub [@U-C4N](https://github.com/U-C4N)
|
||||
**Umutcan Edizsalan** · Mechanical engineering work at **Anka-Makine** · GitHub [@U-C4N](https://github.com/U-C4N)
|
||||
|
||||
Built from production drawing work, then made model-agnostic through MCP.
|
||||
|
||||
## License
|
||||
|
||||
[MIT](LICENSE)
|
||||
[MIT](https://github.com/U-C4N/Autocad-MCP/blob/main/LICENSE)
|
||||
|
||||
<!-- MCP registry ownership marker; must equal the "name" in server.json. -->
|
||||
mcp-name: io.github.u-c4n/autocad-mcp
|
||||
|
||||
@@ -21,7 +21,7 @@ hand-written refusal. Three properties follow:
|
||||
|
||||
* An unimplemented method refuses instead of pretending. It cannot silently
|
||||
succeed and it cannot silently do nothing.
|
||||
* The refusal is *declared*: `test_capability.py` fails if a key used here is
|
||||
* The refusal is *declared*: `test_capability_contract.py` fails if a key used here is
|
||||
missing from either backend's `CapabilityMap`, so `system_capabilities` can
|
||||
never disagree with what a call actually does. This is the part that keeps
|
||||
the escape hatch honest — without it, `@capability` would just be a way to
|
||||
|
||||
+17
-10
@@ -2968,13 +2968,15 @@ class EzdxfBackend(AutoCADBackend):
|
||||
radius = math.sqrt((chord_x - cxf) ** 2 + (chord_y - cyf) ** 2)
|
||||
angle_rad = math.atan2(chord_y - cyf, chord_x - cxf)
|
||||
leader = float(leader_length)
|
||||
mpoint = (
|
||||
cxf + (radius + leader) * math.cos(angle_rad),
|
||||
cyf + (radius + leader) * math.sin(angle_rad),
|
||||
)
|
||||
# Same correction as dimension_diameter: the size comes from
|
||||
# `radius`, and `location` only places the text.
|
||||
dim = msp.add_radius_dim(
|
||||
center=(cxf, cyf),
|
||||
mpoint=mpoint,
|
||||
radius=radius,
|
||||
location=(
|
||||
cxf + (radius + leader) * math.cos(angle_rad),
|
||||
cyf + (radius + leader) * math.sin(angle_rad),
|
||||
),
|
||||
text=text if text is not None else "<>",
|
||||
override=override or None,
|
||||
)
|
||||
@@ -3011,13 +3013,18 @@ class EzdxfBackend(AutoCADBackend):
|
||||
radius = math.sqrt((x2f - x1f) ** 2 + (y2f - y1f) ** 2) / 2
|
||||
angle_rad = math.atan2(y2f - y1f, x2f - x1f)
|
||||
leader = float(leader_length)
|
||||
mpoint = (
|
||||
cx + (radius + leader) * math.cos(angle_rad),
|
||||
cy + (radius + leader) * math.sin(angle_rad),
|
||||
)
|
||||
# The measured size comes from `radius`; `location` only places the
|
||||
# text. This used to pass centre + (radius + leader) as `mpoint`,
|
||||
# which ezdxf measures to — so every diameter came out
|
||||
# 2 x leader_length too large. 60 on a true 40, at default
|
||||
# settings, with nothing on the drawing to show for it.
|
||||
dim = msp.add_diameter_dim(
|
||||
center=(cx, cy),
|
||||
mpoint=mpoint,
|
||||
radius=radius,
|
||||
location=(
|
||||
cx + (radius + leader) * math.cos(angle_rad),
|
||||
cy + (radius + leader) * math.sin(angle_rad),
|
||||
),
|
||||
text=text if text is not None else "<>",
|
||||
override=override or None,
|
||||
)
|
||||
|
||||
@@ -269,6 +269,30 @@ async def two_vertex_circle_has_area():
|
||||
return abs(area - math.pi * 100.0) < 1e-9
|
||||
|
||||
|
||||
async def diameter_dim_measures_the_diameter():
|
||||
"""The number on the callout is the whole point of the callout.
|
||||
|
||||
`add_diameter_dim` measures to `mpoint`; the leader is a text placement.
|
||||
Feeding it centre + (radius + leader) made every diameter come back
|
||||
2 x leader_length too large at DEFAULT settings - 60 on a true 40.
|
||||
"""
|
||||
b = await _b()
|
||||
info = await b.dimension_diameter(-20, 0, 20, 0)
|
||||
dim = b._doc.entitydb.get(info.handle)
|
||||
return abs(dim.get_measurement() - 40.0) < 1e-6
|
||||
|
||||
|
||||
async def radius_dim_ignores_the_leader_length():
|
||||
b = await _b()
|
||||
near = await b.dimension_radius(0, 0, 20, 0, leader_length=5)
|
||||
far = await b.dimension_radius(0, 0, 20, 0, leader_length=40)
|
||||
values = [
|
||||
b._doc.entitydb.get(near.handle).get_measurement(),
|
||||
b._doc.entitydb.get(far.handle).get_measurement(),
|
||||
]
|
||||
return all(abs(v - 20.0) < 1e-6 for v in values)
|
||||
|
||||
|
||||
CHECKS = {
|
||||
"core_line_length": (core_line_length, "Core"),
|
||||
"core_circle_radius": (core_circle_radius, "Core"),
|
||||
@@ -294,6 +318,8 @@ CHECKS = {
|
||||
"hatch_area_subtracts_its_island": (hatch_area_subtracts_its_island, "Measurement"),
|
||||
"boundary_area_agrees_with_measure": (boundary_area_agrees_with_measure, "Measurement"),
|
||||
"two_vertex_circle_has_area": (two_vertex_circle_has_area, "Measurement"),
|
||||
"diameter_dim_measures_the_diameter": (diameter_dim_measures_the_diameter, "Dimensions"),
|
||||
"radius_dim_ignores_the_leader_length": (radius_dim_ignores_the_leader_length, "Dimensions"),
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -51,6 +51,12 @@ def render_chart(reports: list[dict[str, Any]], output: Path) -> None:
|
||||
coverage = [item["summary"]["coverage_percent"] for item in reports]
|
||||
passed = [item["summary"]["passed"] for item in reports]
|
||||
attempted = [item["summary"]["attempted"] for item in reports]
|
||||
# Which task set each server actually ran. Not decoration: since v1.5.0
|
||||
# the reference adapter runs the 15-task v3 matrix while the pinned
|
||||
# competitor reports are v2 runs from v1.4, so the bars do not share a
|
||||
# denominator and a chart that implies they do is the lie this lane
|
||||
# exists to avoid.
|
||||
matrices = [item.get("matrix", "v2") for item in reports]
|
||||
colors = ["#58a6ff" if label == "autocad-mcp-pro" else "#484f58" for label in labels]
|
||||
|
||||
fig, ax = plt.subplots(figsize=(12, 4.8), constrained_layout=True)
|
||||
@@ -60,7 +66,9 @@ def render_chart(reports: list[dict[str, Any]], output: Path) -> None:
|
||||
|
||||
ax.set_xlim(0, 100)
|
||||
ax.set_xlabel(
|
||||
"Weighted live-run score / 100 (fixed 10-task matrix)", color="#e6edf3", labelpad=10
|
||||
"Weighted live-run score / 100, within each server's own task matrix",
|
||||
color="#e6edf3",
|
||||
labelpad=10,
|
||||
)
|
||||
ax.set_title(
|
||||
"Live-run benchmark - headless ezdxf lane",
|
||||
@@ -73,17 +81,20 @@ def render_chart(reports: list[dict[str, Any]], output: Path) -> None:
|
||||
ax.text(
|
||||
0,
|
||||
1.02,
|
||||
"Same 10 tasks, same harness, pinned competitor commits; artifacts verified by re-opening the DXF",
|
||||
"Same harness, pinned competitor commits; artifacts verified by re-opening the DXF. "
|
||||
"Matrices differ - read the per-bar label.",
|
||||
transform=ax.transAxes,
|
||||
color="#8b949e",
|
||||
fontsize=10,
|
||||
)
|
||||
|
||||
for bar, score, cov, ok, total in zip(bars, scores, coverage, passed, attempted, strict=True):
|
||||
for bar, score, cov, ok, total, matrix in zip(
|
||||
bars, scores, coverage, passed, attempted, matrices, strict=True
|
||||
):
|
||||
ax.text(
|
||||
min(score + 1.2, 82),
|
||||
min(score + 1.2, 76),
|
||||
bar.get_y() + bar.get_height() / 2,
|
||||
f"{score:g} ({ok}/{total} pass, {cov:g}% coverage)",
|
||||
f"{score:g} ({ok}/{total} pass, {cov:g}% coverage, matrix {matrix})",
|
||||
va="center",
|
||||
color="#e6edf3",
|
||||
fontweight="bold",
|
||||
@@ -92,7 +103,9 @@ def render_chart(reports: list[dict[str, Any]], output: Path) -> None:
|
||||
ax.text(
|
||||
0,
|
||||
-0.22,
|
||||
"Unsupported tasks score 0 in the fixed matrix; per-task statuses and reasons are in benchmarks/results/published/.",
|
||||
"Unsupported tasks score 0 within a server's own matrix. The five v3 tasks were "
|
||||
"added after the competitor runs and were never put to them; per-task statuses are "
|
||||
"in benchmarks/results/published/.",
|
||||
transform=ax.transAxes,
|
||||
color="#8b949e",
|
||||
fontsize=9,
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"summary": {
|
||||
"baseline_ref": "v1.4.0",
|
||||
"total": 24,
|
||||
"total": 26,
|
||||
"old_pass": 21,
|
||||
"new_pass": 24,
|
||||
"old_pct": 87.5,
|
||||
"new_pass": 26,
|
||||
"old_pct": 80.8,
|
||||
"new_pct": 100.0,
|
||||
"fixed": 3,
|
||||
"fixed": 5,
|
||||
"regressed": 0
|
||||
},
|
||||
"rows": [
|
||||
@@ -153,6 +153,18 @@
|
||||
"category": "Measurement",
|
||||
"old": "miss",
|
||||
"new": "pass"
|
||||
},
|
||||
{
|
||||
"check": "diameter_dim_measures_the_diameter",
|
||||
"category": "Dimensions",
|
||||
"old": "fail",
|
||||
"new": "pass"
|
||||
},
|
||||
{
|
||||
"check": "radius_dim_ignores_the_leader_length",
|
||||
"category": "Dimensions",
|
||||
"old": "fail",
|
||||
"new": "pass"
|
||||
}
|
||||
]
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
|
Before Width: | Height: | Size: 69 KiB After Width: | Height: | Size: 82 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 44 KiB |
@@ -0,0 +1,164 @@
|
||||
"""Render the drawing the README shows, using only the server's own tools.
|
||||
|
||||
A CAD README that shows nothing but bar charts is asking to be taken on faith.
|
||||
This produces the one asset that is not a claim: an actual sheet, built through
|
||||
the same backend methods the MCP tools call, in the order CLAUDE.md's premium
|
||||
workflow prescribes -- ISO layers, deterministic gear geometry, a keyed bore,
|
||||
ISO 129 dimensions, an ISO 7200 title block -- and rendered headlessly.
|
||||
|
||||
Reproduce:
|
||||
|
||||
AUTOCAD_MCP_BACKEND=ezdxf python scripts/render_readme_showcase.py
|
||||
|
||||
Nothing here is decoration: every line on the sheet comes from a tool the
|
||||
server exposes, so if a tool regresses the picture changes with it.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT))
|
||||
|
||||
DEFAULT_OUTPUT = ROOT / "docs" / "assets" / "autocad-mcp-showcase.png"
|
||||
|
||||
#: A 24-tooth, module-6 spur gear. Sized to fill the A3 drawing area, because
|
||||
#: the render crops to the sheet border: geometry that fills the border gets
|
||||
#: the pixels, and involute flanks that do not resolve prove nothing.
|
||||
GEAR_MODULE = 6.0
|
||||
GEAR_TEETH = 24
|
||||
BORE_DIAMETER = 40.0
|
||||
KEYWAY_WIDTH = 12.0
|
||||
KEYWAY_DEPTH = 3.3
|
||||
FACE_WIDTH = 46.0
|
||||
GEAR_CENTER = (135.0, 172.0)
|
||||
SECTION_X = 300.0
|
||||
|
||||
#: No DIMSCALE here on purpose. `drawing_settings({"dimscale": ...})` sets the
|
||||
#: header variable and reports success, but ezdxf renders dimensions from the
|
||||
#: dimstyle table entry, so headlessly it changes nothing (measured). Setting it
|
||||
#: in this script would put a call in the README's own showcase that does not do
|
||||
#: what it looks like it does.
|
||||
|
||||
|
||||
async def build(backend) -> dict:
|
||||
from engineering.gear import draw_gear_section_aa, draw_spur_gear_front_view
|
||||
from engineering.titleblock import TitleBlockMetadata, apply_iso_a3_titleblock
|
||||
|
||||
await backend.drawing_new()
|
||||
await backend.drawing_apply_iso_layers("mech")
|
||||
await backend.drawing_settings({"units": "mm", "linear_precision": 2})
|
||||
|
||||
gear = await draw_spur_gear_front_view(
|
||||
backend,
|
||||
module=GEAR_MODULE,
|
||||
teeth=GEAR_TEETH,
|
||||
center=GEAR_CENTER,
|
||||
bore_diameter=BORE_DIAMETER,
|
||||
keyway_width=KEYWAY_WIDTH,
|
||||
keyway_depth=KEYWAY_DEPTH,
|
||||
)
|
||||
|
||||
# `gear_metadata` is the geometry record, not the handle map the front view
|
||||
# returns -- the section needs radii, not handles.
|
||||
await draw_gear_section_aa(
|
||||
backend,
|
||||
gear_metadata=gear["metadata"],
|
||||
x_offset=SECTION_X,
|
||||
face_width=FACE_WIDTH,
|
||||
)
|
||||
|
||||
# ISO 129 dimensions, and a real ISO 286 fit on the bore -- "H7" is the
|
||||
# whole reason a drafter trusts a generated hole. `fit=` is resolved by the
|
||||
# tool layer, so the deviations are looked up here the same way it does
|
||||
# rather than typed in as text.
|
||||
from engineering.fits import fit_lookup
|
||||
|
||||
cx, cy = GEAR_CENTER
|
||||
pitch_radius = GEAR_MODULE * GEAR_TEETH / 2.0
|
||||
outer_radius = pitch_radius + GEAR_MODULE
|
||||
await backend.dimension_linear(
|
||||
cx - pitch_radius,
|
||||
cy,
|
||||
cx + pitch_radius,
|
||||
cy,
|
||||
cx,
|
||||
cy - outer_radius - 22.0,
|
||||
layer="DIM",
|
||||
)
|
||||
# dimension_diameter takes the two ENDS OF A DIAMETER, not the centre and a
|
||||
# point on the circle. Passing the latter halves the measured size, which is
|
||||
# how the first draft of this sheet ended up calling a 40 mm bore 20.
|
||||
bore_fit = fit_lookup("H7", BORE_DIAMETER)
|
||||
await backend.dimension_diameter(
|
||||
cx - BORE_DIAMETER / 2.0,
|
||||
cy,
|
||||
cx + BORE_DIAMETER / 2.0,
|
||||
cy,
|
||||
leader_length=34.0,
|
||||
layer="DIM",
|
||||
tol_upper=bore_fit.upper_mm,
|
||||
tol_lower=-bore_fit.lower_mm,
|
||||
tol_mode="deviation",
|
||||
text_override=f"<> {bore_fit.code}",
|
||||
)
|
||||
|
||||
await apply_iso_a3_titleblock(
|
||||
backend,
|
||||
metadata=TitleBlockMetadata(
|
||||
title=f"SPUR GEAR m{GEAR_MODULE:g} z{GEAR_TEETH}",
|
||||
drawing_no="AMC-1500-001",
|
||||
part_no="1500-001",
|
||||
material="C45E",
|
||||
scale="1:1",
|
||||
drawn_by="autocad-mcp-pro",
|
||||
checked_by="drawing_critique",
|
||||
date="2026-08-06",
|
||||
revision="A",
|
||||
),
|
||||
origin=(0.0, 0.0),
|
||||
)
|
||||
|
||||
issues = await backend.drawing_critique(focus=None)
|
||||
info = await backend.drawing_info()
|
||||
return {"issues": len(issues), "entities": info.entity_count, "gear": gear}
|
||||
|
||||
|
||||
async def main(output: Path) -> int:
|
||||
os.environ.setdefault("AUTOCAD_MCP_BACKEND", "ezdxf")
|
||||
from backends.ezdxf_backend import EzdxfBackend
|
||||
|
||||
backend = EzdxfBackend()
|
||||
await backend.connect()
|
||||
try:
|
||||
summary = await build(backend)
|
||||
png = await backend.view_screenshot(overlay_handles=False)
|
||||
finally:
|
||||
await backend.disconnect()
|
||||
|
||||
if not png:
|
||||
print("render produced no image", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
output.parent.mkdir(parents=True, exist_ok=True)
|
||||
output.write_bytes(png)
|
||||
print(
|
||||
f"wrote {output} ({len(png)} bytes) - "
|
||||
f"{summary['entities']} entities, {summary['issues']} critique issues"
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
def build_parser() -> argparse.ArgumentParser:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--output", type=Path, default=DEFAULT_OUTPUT)
|
||||
return parser
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(asyncio.run(main(build_parser().parse_args().output)))
|
||||
+1
-1
@@ -26,7 +26,7 @@
|
||||
},
|
||||
{
|
||||
"name": "ALLOWED_PATHS",
|
||||
"description": "Comma-separated absolute paths the server may read and write. Empty blocks file access outside temp.",
|
||||
"description": "Comma-separated absolute paths the server may read and write. Empty means no allowlist is enforced at all - traversal and system-directory checks still run, but nothing scopes the filesystem. Set this.",
|
||||
"isRequired": false,
|
||||
"format": "string",
|
||||
"isSecret": false
|
||||
|
||||
@@ -832,7 +832,7 @@ def cad_tool(*, summary: str, cost: str):
|
||||
# Some MCP clients degrade (or hard-cap) when a server exposes 100+ tools.
|
||||
# TOOL_PROFILE selects how much of the surface is advertised:
|
||||
# full — everything (default, backwards compatible)
|
||||
# lean — a curated ~46-tool drafting/inspection core
|
||||
# lean — a curated 47-tool drafting/inspection core
|
||||
# Disabled tools stay registered (system_about still reports the full
|
||||
# inventory) but are hidden from MCP list_tools and rejected if called.
|
||||
|
||||
|
||||
@@ -36,3 +36,60 @@ async def test_dimension_diameter(backend):
|
||||
info = await backend.dimension_diameter(20, 50, 80, 50, leader_length=10)
|
||||
assert info.handle
|
||||
assert info.type == "DIMENSION"
|
||||
|
||||
|
||||
# ── the number on the dimension must be the number on the drawing ───────────
|
||||
#
|
||||
# `add_diameter_dim`/`add_radius_dim` take `mpoint` as a point ON the circle;
|
||||
# `leader_length` is where the *text* sits. Passing `centre + (radius + leader)`
|
||||
# as `mpoint` makes ezdxf measure to that point, so every diameter came out
|
||||
# `2 x leader_length` too big and every radius `leader_length` too big -- at
|
||||
# DEFAULT settings, silently, on a drawing whose whole purpose is the number.
|
||||
# The live COM backend was never wrong here: AddDimDiametric takes the two
|
||||
# chord points and the leader length as separate arguments.
|
||||
|
||||
|
||||
async def test_a_diameter_dimension_measures_the_diameter(backend):
|
||||
"""Two ends of a diameter of a true 40 mm circle."""
|
||||
info = await backend.dimension_diameter(-20, 0, 20, 0)
|
||||
|
||||
dim = backend._doc.entitydb.get(info.handle)
|
||||
assert dim.get_measurement() == pytest.approx(40.0)
|
||||
|
||||
|
||||
async def test_the_leader_length_moves_the_text_not_the_measurement(backend):
|
||||
"""The regression in one assertion: leader_length must not be measurable."""
|
||||
near = await backend.dimension_diameter(-20, 0, 20, 0, leader_length=5.0)
|
||||
far = await backend.dimension_diameter(-20, 0, 20, 0, leader_length=40.0)
|
||||
|
||||
measurements = [
|
||||
backend._doc.entitydb.get(near.handle).get_measurement(),
|
||||
backend._doc.entitydb.get(far.handle).get_measurement(),
|
||||
]
|
||||
assert measurements == pytest.approx([40.0, 40.0])
|
||||
|
||||
|
||||
async def test_a_radius_dimension_measures_the_radius(backend):
|
||||
info = await backend.dimension_radius(0, 0, 20, 0)
|
||||
|
||||
dim = backend._doc.entitydb.get(info.handle)
|
||||
assert dim.get_measurement() == pytest.approx(20.0)
|
||||
|
||||
|
||||
async def test_a_radius_measurement_ignores_the_leader_too(backend):
|
||||
near = await backend.dimension_radius(0, 0, 20, 0, leader_length=5.0)
|
||||
far = await backend.dimension_radius(0, 0, 20, 0, leader_length=40.0)
|
||||
|
||||
measurements = [
|
||||
backend._doc.entitydb.get(near.handle).get_measurement(),
|
||||
backend._doc.entitydb.get(far.handle).get_measurement(),
|
||||
]
|
||||
assert measurements == pytest.approx([20.0, 20.0])
|
||||
|
||||
|
||||
async def test_an_off_axis_diameter_is_measured_from_its_own_chord(backend):
|
||||
"""The two points are a diameter at any angle, not just horizontal."""
|
||||
info = await backend.dimension_diameter(0, 0, 30, 40, leader_length=12.0)
|
||||
|
||||
dim = backend._doc.entitydb.get(info.handle)
|
||||
assert dim.get_measurement() == pytest.approx(50.0)
|
||||
|
||||
Reference in New Issue
Block a user