diff --git a/CHANGELOG.md b/CHANGELOG.md index 15927db..5849cab 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,7 +7,86 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] -_Next: capability-group schema routing, paper-space/layout production depth, and ISO 286 fit-table lookup._ +_Next (1.5): screenshot overlay + handle grounding v1, ezdxf redo (forward +snapshots), COM `block_create_from_entities`, ISO 286 transition/interference +hole letters (delta rule), titleblock on paper-space layouts._ + +## [1.4.0] — 2026-07-23 + +Release infrastructure + the roadmap features shipped together. **474 tests, +Ruff lint- and format-clean. Tool count: 122 → 131.** + +### Added + +- **CI restored and extended** (`.github/workflows/ci.yml`): ruff lint/format, + Linux test matrix (3.11/3.12) with coverage, a **Windows leg** running the + mocked-COM suite on the platform the COM backend targets, a package job + (build + twine check + clean-venv wheel smoke incl. `autocad-mcp --help`), + a Docker build/smoke job, and MCP `server.json` schema validation. +- **Release pipeline** (`.github/workflows/release.yml`): tag `v*` → test gate + → tag/version match check → build → **PyPI trusted publishing** (OIDC) → + GitHub Release with artifacts. +- **MCP registry manifest** `server.json` (schema 2025-12-11, name + `io.github.u-c4n/autocad-mcp`, PyPI package) plus the `mcp-name` ownership + marker in the README and an `autocad-mcp-pro` console alias so + `uvx autocad-mcp-pro` works directly. +- **Live competitor benchmark lane**: a generic black-box MCP-stdio driver + (`benchmarks/adapters/mcp_stdio.py`), reproducible pinned checkouts + (`benchmarks/competitors_env.py`), and adapters for + **puran-water/autocad-mcp** and **beiming183-cloud/AutoCAD-MCP** that run + the same 10-task matrix over stdio with harness-side DXF verification. + Published results (`benchmarks/results/published/`): autocad-mcp-pro + **100.0**, beiming183 **50.0**, puran-water **45.0**; rendered to + `docs/assets/autocad-mcp-livebench.svg`. Weekly/dispatch CI workflow. +- **Tool profiles** (`TOOL_PROFILE=lean|core|full`): capability-aware + discovery applied in the server lifespan — `lean` is a curated ~46-tool + drafting core, `core` hides raw escape hatches and long-tail tools, `full` + (default) exposes everything. Reported by `system_about`. +- **Paper space / layouts**: `layout_list`, `layout_create`, + `layout_set_current`, `viewport_create` (scaled model viewports) on both + backends, and `drawing_export_pdf(layout=...)` for plotting a layout. + Headless limitation is explicit: viewport model-content projection is + COM-only (`viewport_render` capability). +- **ISO 286 limits and fits** (`engineering/fits.py`): authored table data + (IT4–IT11, sizes 1–500 mm; shafts d/e/f/g/h/js/k/m/n/p, holes D/E/F/G/H/JS) + with `fit_lookup("H7", 20.0)`; `dimension_linear/radius/diameter` gained a + `fit` parameter that resolves deviations from the measured nominal and + appends the fit code to the dimension text. Out-of-scope letters raise a + clear error naming the supported set. +- **Opt-in native 3D solids** (`ENABLE_3D=true`, COM only): `solid_box`, + `solid_cylinder`, `solid_extrude`, `solid_revolve`, `solid_boolean`. + Hidden from discovery and rejected while disabled; ezdxf reports the honest + `solid_3d` capability boundary (ACIS cannot be generated headlessly). +- **Release-consistency test suite** (`tests/test_release_consistency.py`): + pyproject ↔ version.py ↔ CHANGELOG ↔ README snapshot ↔ server.json versions, + README tool/resource/prompt counts vs live registrations, per-section header + counts vs decorators, Dockerfile COPY vs wheel `only-include`, and the + README `mcp-name` marker. + +### Changed + +- **pyproject metadata completed** for PyPI: SPDX license + license file, + authors, keywords, classifiers, project URLs. +- **`build_dim_override`**: `deviation`/`limit` modes now pass DIMTM signed + (`-lower`) instead of `abs()` — double-positive ISO 286 fits (e.g. p6 + +0.035/+0.022) render correctly; the legacy "positive magnitude = minus + deviation" contract is unchanged. +- **Capability maps**: `paper_space` is now native on both backends; + new `viewport_render` feature (COM native / ezdxf unsupported); + `solid_3d` reflects the ENABLE_3D gate on COM. +- README rebuilt around the live-run benchmark lane; CI/PyPI badges added; + `ruff format` applied repo-wide and enforced in CI. + +### Fixed + +- **Dockerfile shipped a broken image**: `engineering/` and `version.py` were + never copied, so engineering tools failed in containers. Both are now + copied (plus README/LICENSE for metadata) and the healthcheck imports the + engineering package. +- `server.py` section-header tool counts and the CLAUDE.md inventory were + stale (six sections drifted); both now match the live surface and are + locked by tests. Removed the reference to the non-existent + `.claude/skills/` directory. ## [1.3.0] — 2026-07-17 diff --git a/CLAUDE.md b/CLAUDE.md index 63cf17d..47ba365 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co ## Project Overview -AutoCAD MCP Pro is a FastMCP 3.0 server that exposes ~110 tools, 5 resources, and 5 prompt templates for AutoCAD automation. (The exact tool count is reported dynamically by `system_status` / `system_about` — never hardcode it.) It runs with a dual-engine architecture: a live COM backend (Windows/AutoCAD required) and a headless ezdxf backend (works anywhere). +AutoCAD MCP Pro is a FastMCP 3.0 server that exposes ~131 tools, 6 resources, and 5 prompt templates for AutoCAD automation. (The exact tool count is reported dynamically by `system_status` / `system_about` — never hardcode it.) It runs with a dual-engine architecture: a live COM backend (Windows/AutoCAD required) and a headless ezdxf backend (works anywhere). ## Running the Server @@ -69,24 +69,30 @@ The `mcp` FastMCP instance is configured with: - **Middleware stack**: `ErrorHandlingMiddleware` → `AuditMiddleware` (custom timing/audit log) → `TimingMiddleware` → `LoggingMiddleware` - **`_backend(ctx)`** helper: retrieves the backend from lifespan context, raises `ToolError` if not ready -Tools are organized into 12 sections: -1. Drawing Management (11 tools): `drawing_*` -2. Entity Creation (13 tools): `entity_create_*` +Tools are organized into sections (counts are indicative — `system_about` is authoritative): +1. Drawing Management (12 tools): `drawing_*` (includes `drawing_redo`) +2. Entity Creation (14 tools): `entity_create_*` (includes `entity_create_table`, `leader_create_mleader`) 3. Dimensions (5 tools): `dimension_*` -4. Entity Modification (12 tools): `entity_move/copy/rotate/scale/mirror/offset/delete/array_*`, plus in-place `entity_edit_text` (TEXT/MTEXT content/height/rotation) and `entity_edit_geometry` (CIRCLE/LINE/ARC — center/radius/endpoints/angles), both handle-preserving -5. Entity Query (3 tools): `entity_get`, `entity_list`, `entity_delete_many` -6. Layer Management (12 tools + 2 linetype_*): `layer_*`, `linetype_list`, `linetype_load` +4. Entity Modification (16 tools): `entity_move/copy/rotate/scale/mirror/offset/delete/array_*`, corner ops (`entity_trim/extend/fillet/chamfer`), plus in-place `entity_edit_text` (TEXT/MTEXT content/height/rotation) and `entity_edit_geometry` (CIRCLE/LINE/ARC — center/radius/endpoints/angles), both handle-preserving +5. Entity Query (4 tools): `entity_get`, `entity_list`, `entity_delete_many`, `selection_get` +6. Layer Management (14 tools incl. 2 linetype_*): `layer_*`, `linetype_list`, `linetype_load` 7. Block Operations (7 tools): `block_*` -8. Analysis & Query (8 tools): `analysis_*` -9. View & Screenshot (5 tools — includes `view_zoom_and_screenshot`): `view_*` +8. Analysis & Query (8 tools): `analysis_*` — plus Batch (2), Templates (2), Validation (1) +9. View & Screenshot (4 tools — includes `view_zoom_and_screenshot`): `view_*` 10. Transactions (3 tools): `transaction_begin/commit/rollback` -11. System (7 tools): `system_status/get_variable/set_variable/run_command/run_lisp/about`, plus `drawing_settings` (friendly units/scale/precision/osnap facade over system variables) -12. Engineering / Deterministic CAD (7 tools): `gear_draw_*`, `keyway_draw_*`, `titleblock_apply_iso_a3`, `drawing_finalize` -13. Premium meta-tools: `drawing_plan`, `drawing_critique`, `point_from_snap/intersection/tangent`, `construction_*`, `drawing_apply_iso_layers`, `dimension_auto`, `entity_select_smart` +11. System (8 tools): `system_status/get_variable/set_variable/run_command/run_lisp/about/capabilities`, plus `drawing_settings` (friendly units/scale/precision/osnap facade over system variables) +12. Engineering / Deterministic CAD (8 tools): `gear_draw_*`, `keyway_draw_*`, `titleblock_apply_iso_a3`, `drawing_finalize` +13. Premium meta-tools (12): `drawing_preflight`, `drawing_plan`, `drawing_critique`, `drawing_refine`, `drawing_deliver`, `point_from_snap/intersection/tangent`, `construction_*`, `drawing_apply_iso_layers`, `dimension_auto`, `entity_select_smart` 14. GD&T (ISO 1101 / ASME Y14.5): `gd_frame` (feature control frames), `datum_feature` — enforced by the `gdt` critique focus +15. Layouts & Paper Space (4 tools): `layout_list/create/set_current`, `viewport_create` (scaled model viewports); `drawing_export_pdf` takes an optional `layout` param. Viewport model-content projection is COM-only (`viewport_render` capability). +16. 3D Solids (5 tools, opt-in via `ENABLE_3D=true`, COM only): `solid_box/cylinder/extrude/revolve/boolean` — hidden from discovery and rejected while disabled; ezdxf reports `solid_3d` as unsupported (no headless ACIS). + +**Tool profiles:** `TOOL_PROFILE=lean|core|full` (default `full`) controls the advertised surface — `lean` ≈ 46 curated drafting tools, `core` hides raw escape hatches (`system_run_command/lisp`, low-level variables, long-tail tools). Applied in the lifespan; reported by `system_about`. **ISO 129 tolerances:** `dimension_linear` / `dimension_radius` / `dimension_diameter` take `tol_upper` / `tol_lower` / `tol_mode` (`symmetric` ± / `deviation` +a/-b / `limit` / `basic`) and `text_override` (e.g. `⌀20 H7`). Prefer these over hand-drawn tolerance text. +**ISO 286 fits:** the same dimension tools take `fit="H7"` (mutually exclusive with `tol_*`) — deviations are resolved from authored ISO 286 tables (`engineering/fits.py`; shafts d/e/f/g/h/js/k/m/n/p, holes D/E/F/G/H/JS, sizes 1–500 mm) against the measured nominal and the fit code is appended to the dimension text. + **Drawing score:** `drawing_finalize` returns `payload["score"]` — a 0-100 scalar + `invalidity_ratio` + A-F `grade` over the validator + critique union. Use it as the objective quality metric. ### Adding a New Tool @@ -110,8 +116,7 @@ Tools are organized into 12 sections: ### Premium Drawing Rules -These rules are non-negotiable for production engineering output. The full -discipline + workflow templates live in `.claude/skills/autocad-mcp-premium/`. +These rules are non-negotiable for production engineering output. 1. **Plan before draw**: Call `drawing_plan(intent, scale, sheet)` *before* any `entity_create_*`. The returned PlanSpec is held by the backend and replayed diff --git a/README.md b/README.md index 24423f6..226f402 100644 --- a/README.md +++ b/README.md @@ -3,6 +3,8 @@ Production-grade AutoCAD automation for AI agents—live through COM on Windows, or headless through ezdxf on any platform. +[![CI](https://github.com/U-C4N/Autocad-MCP/actions/workflows/ci.yml/badge.svg)](https://github.com/U-C4N/Autocad-MCP/actions/workflows/ci.yml) +[![PyPI](https://img.shields.io/pypi/v/autocad-mcp-pro?color=1f6feb)](https://pypi.org/project/autocad-mcp-pro/) [![License: MIT](https://img.shields.io/badge/license-MIT-3fb950)](LICENSE) [![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-1f6feb?logo=python&logoColor=white)](pyproject.toml) [![FastMCP 3](https://img.shields.io/badge/MCP-FastMCP%203-8957e5)](https://github.com/jlowin/fastmcp) @@ -12,8 +14,8 @@ One typed MCP contract controls two execution engines. Build and edit drawings, query exact geometry, apply engineering standards, refine quality issues inside transactions, and deliver hashed artifacts with validation evidence. -> **v1.3 release snapshot:** 122 tools · 6 resources · 5 prompt templates · -> 415 collected tests. Runtime discovery through `system_about` is authoritative. +> **v1.4 release snapshot:** 131 tools · 6 resources · 5 prompt templates · +> 474 collected tests. Runtime discovery through `system_about` is authoritative. ## Start in 60 seconds @@ -86,12 +88,32 @@ python -m benchmarks.render_chart python -m benchmarks.run_competitors --server autocad-mcp-pro --backend ezdxf --json ``` -The v1.3 ezdxf reference run records **10/10 tasks (100.0)**. It is a -self-check, not a competitor ranking. +### Live-run lane — same harness, pinned competitors -Cross-project runtime scores will be published only after equivalent adapters -execute the same task contract. The complete rubric, caveats, version A/B suite, -and reproduction commands live in [`benchmarks/`](benchmarks/README.md). +![Live benchmark](docs/assets/autocad-mcp-livebench.svg) + +| Server | Pinned commit | Score | Pass | Coverage | +|---|---|---:|---:|---:| +| **autocad-mcp-pro** (this repo) | working tree | **100.0** | 10/10 | 100% | +| [beiming183-cloud/AutoCAD-MCP](https://github.com/beiming183-cloud/AutoCAD-MCP) | `11f7c47` | 50.0 | 5/10 | 50% | +| [puran-water/autocad-mcp](https://github.com/puran-water/autocad-mcp) | `95476a3` | 45.0 | 4/10 | 50% | + +Same ten fixed tasks, same runner, headless ezdxf lane. Competitors are driven +**black-box over MCP stdio** using their own documented tool contracts at the +pinned commit, and every geometry claim is verified by re-opening the exported +DXF with ezdxf in the harness — no competitor self-reporting. `unsupported` +scores zero in the fixed matrix; per-task statuses with exact reasons are +committed under [`benchmarks/results/published/`](benchmarks/results/published/). +File-IPC (live AutoCAD) lanes run locally, not in CI. + +```bash +python -m benchmarks.run_competitors --server puran-water-autocad-mcp --backend ezdxf +python -m benchmarks.run_competitors --server beiming183-autocad-mcp --backend ezdxf +python -m benchmarks.render_live_chart +``` + +The complete rubric, caveats, version A/B suite, and reproduction commands live +in [`benchmarks/`](benchmarks/README.md). ## Why this architecture holds up @@ -346,3 +368,6 @@ Built from production drawing work, then made model-agnostic through MCP. ## License [MIT](LICENSE) + + +mcp-name: io.github.u-c4n/autocad-mcp diff --git a/tests/test_release_consistency.py b/tests/test_release_consistency.py new file mode 100644 index 0000000..a39ffe6 --- /dev/null +++ b/tests/test_release_consistency.py @@ -0,0 +1,160 @@ +"""Release-consistency gates (v1.4.0 Faz 0). + +Locks the version string, the README release snapshot, the server.py section +headers, and the Dockerfile file selection to the actual state of the tree so +none of them can drift again: + +- pyproject version == CHANGELOG top release == README snapshot major.minor +- version.py fallback parser returns the pyproject version +- README snapshot tool/resource/prompt counts == live @mcp.* decorator counts +- every ``SECTION n: ... (N tools)`` header matches the decorators below it +- every wheel ``only-include`` entry is COPY'd into the Docker image +""" + +from __future__ import annotations + +import re +import tomllib +from pathlib import Path + +import pytest + +ROOT = Path(__file__).resolve().parents[1] + + +def _pyproject() -> dict: + with (ROOT / "pyproject.toml").open("rb") as stream: + return tomllib.load(stream) + + +def _pyproject_version() -> str: + return str(_pyproject()["project"]["version"]) + + +def _server_source() -> str: + return (ROOT / "server.py").read_text(encoding="utf-8") + + +SEMVER_RE = re.compile(r"^\d+\.\d+\.\d+$") + + +def test_pyproject_version_is_semver() -> None: + assert SEMVER_RE.match(_pyproject_version()), ( + f"pyproject version {_pyproject_version()!r} is not MAJOR.MINOR.PATCH" + ) + + +def test_version_py_fallback_matches_pyproject(monkeypatch: pytest.MonkeyPatch) -> None: + """The pyproject fallback in version.py must resolve the same version. + + The installed-metadata short circuit is forced to miss so the test checks + our parser (the code path a source checkout without an install uses). + """ + import version as version_module + + def _raise(_name: str) -> str: + raise version_module.PackageNotFoundError + + monkeypatch.setattr(version_module, "version", _raise) + assert version_module.package_version() == _pyproject_version() + + +def test_changelog_top_release_matches_pyproject() -> None: + changelog = (ROOT / "CHANGELOG.md").read_text(encoding="utf-8") + match = re.search(r"^## \[(\d+\.\d+\.\d+)\]", changelog, flags=re.MULTILINE) + assert match, "CHANGELOG.md has no '## [X.Y.Z]' release heading" + assert match.group(1) == _pyproject_version(), ( + f"CHANGELOG top release {match.group(1)} != pyproject {_pyproject_version()}" + ) + + +def _readme_snapshot() -> tuple[str, int, int, int]: + readme = (ROOT / "README.md").read_text(encoding="utf-8") + # The snapshot may wrap across blockquote lines; join continuations first. + flat = re.sub(r"\n>\s*", " ", readme) + match = re.search( + r"\*\*v(\d+\.\d+) release snapshot:\*\*\s*(\d+) tools\D+?(\d+) resources" + r"\D+?(\d+) prompt templates", + flat, + ) + assert match, "README.md release-snapshot line not found or malformed" + return match.group(1), int(match.group(2)), int(match.group(3)), int(match.group(4)) + + +def test_readme_snapshot_matches_pyproject_minor() -> None: + snapshot_minor, _, _, _ = _readme_snapshot() + expected = ".".join(_pyproject_version().split(".")[:2]) + assert snapshot_minor == expected, ( + f"README snapshot v{snapshot_minor} != pyproject major.minor v{expected}" + ) + + +def test_readme_snapshot_counts_match_registrations() -> None: + src = _server_source() + tools = len(re.findall(r"@mcp\.tool\(", src)) + resources = len(re.findall(r"@mcp\.resource\(", src)) + prompts = len(re.findall(r"@mcp\.prompt\(", src)) + _, readme_tools, readme_resources, readme_prompts = _readme_snapshot() + assert (readme_tools, readme_resources, readme_prompts) == (tools, resources, prompts), ( + f"README snapshot says {readme_tools}/{readme_resources}/{readme_prompts} " + f"(tools/resources/prompts) but server.py registers {tools}/{resources}/{prompts}" + ) + + +def test_section_header_counts_match_decorators() -> None: + src = _server_source() + headers = [ + (m.start(), m.group(1), m.group(2)) + for m in re.finditer(r"#[^\n]*SECTION ([\w]+):[^\n]*?\((\d+) tools?\)", src) + ] + assert headers, "no counted SECTION headers found in server.py" + # Segment boundaries: every SECTION header (counted or not) ends a segment. + boundaries = sorted(m.start() for m in re.finditer(r"#[^\n]*SECTION [\w]+:", src)) + boundaries.append(len(src)) + mismatches: list[str] = [] + for start, section_id, declared in headers: + end = min(b for b in boundaries if b > start) + actual = len(re.findall(r"@mcp\.tool\(", src[start:end])) + if actual != int(declared): + mismatches.append(f"SECTION {section_id}: header says {declared}, actual {actual}") + assert not mismatches, "; ".join(mismatches) + + +def test_server_json_versions_match_pyproject() -> None: + """MCP registry manifest must ship the same version as the package.""" + import json + + manifest = json.loads((ROOT / "server.json").read_text(encoding="utf-8")) + expected = _pyproject_version() + assert manifest["version"] == expected, ( + f"server.json version {manifest['version']} != pyproject {expected}" + ) + for package in manifest["packages"]: + assert package["version"] == expected, ( + f"server.json package {package['identifier']} version " + f"{package['version']} != pyproject {expected}" + ) + if package["registryType"] == "pypi": + assert package["identifier"] == _pyproject()["project"]["name"] + + +def test_readme_contains_mcp_name_marker() -> None: + """PyPI ownership validation for the MCP registry needs this marker in the + README (which ships to PyPI as the long description).""" + import json + + manifest = json.loads((ROOT / "server.json").read_text(encoding="utf-8")) + readme = (ROOT / "README.md").read_text(encoding="utf-8") + assert f"mcp-name: {manifest['name']}" in readme + + +def test_dockerfile_copies_every_wheel_include() -> None: + """The Docker image must contain everything the wheel ships (GH bug: the + image previously missed engineering/ and version.py).""" + dockerfile = (ROOT / "Dockerfile").read_text(encoding="utf-8") + copy_lines = "\n".join( + line for line in dockerfile.splitlines() if line.strip().upper().startswith("COPY") + ) + includes = _pyproject()["tool"]["hatch"]["build"]["targets"]["wheel"]["only-include"] + missing = [name for name in includes if name not in copy_lines] + assert not missing, f"Dockerfile COPY misses wheel-shipped paths: {missing}"