release: v1.5.0 — discovery layer, honesty debt closed, Wave A

Tool count 131 -> 154; 474 -> 1242 collected tests.

The release exists for two measured reasons. A 149-tool catalog costs a client
~40,300 tokens before it has asked for anything, and a drafter searching the way
drafters think -- FILLET, BPOLY, QSELECT, WBLOCK -- scored df=0 against the stock
index, because those words were simply not in the data.

Discovery and batching
- DISCOVERY_MODE=search replaces the catalog with search_tools + call_tool,
  ranked over an AutoCAD command and synonym corpus (discovery/aliases.py) on
  top of fastmcp's BM25 index. Measured: 40,305 -> 356 tokens advertised.
- cad_batch runs a step list in one round trip; fields= projects result-heavy
  responses. The `core` profile is gone -- discovery replaced it.

Honesty debt (Tier 0)
- OCS/ECS: `extrusion` appeared nowhere in this codebase, so mirrored and
  rotated geometry reported coordinates that were quietly wrong. backends/ocs.py
  translates at the backend boundary; every tool coordinate is WCS on both
  engines, and a tilted plane refuses rather than answering with a number that
  looks fine.
- drawing_save_as(fmt="dwg") wrote DXF bytes to a .dwg path. drawing_audit never
  repaired. analysis_measure_area shoelaced points the caller typed in.
- ezdxf calls gained a per-call timeout, so one hung call can no longer wedge a
  server whose document lock is a single asyncio.Lock.

Architecture
- The 1854-line backend ABC is 19 contract modules; @capability(key, reason=)
  makes optional methods declarative and a gate fails on undeclared holes.
- 8 layout/viewport tools, CHSPACE, CAD_PROGID escape hatch.
- Wave A: selection filters, boundary tracing, hatch depth, annotation objects,
  measurement -- 13 tools of a planned 39. The REGION and 2D-boolean family was
  cut on evidence: add_region() yields zero ACIS bytes and the greiner_hormann
  substitute loses 28.2% of the area on a square with one semicircular edge.

Verified against a live AutoCAD 2026, which found four COM defects no amount of
reading would have caught -- AcadPViewport has no ViewCenter, and the line that
set it had been swallowed by a bare except since v1.4.

Benchmark v3 (15 tasks) and what it caught
- v2 scored this server 10/10, which carried no information: every task in it
  exercised something the server was built around. Five tasks were added because
  they can fail, and three did while being written -- a hatch could not be
  measured though the refusal listed HATCH as measurable, the boundary tools and
  the measure tool disagreed about the same shape (139.2177 vs 139.2699), and a
  circle stored as two bulged vertices measured 0.0.
- Competitor reports are NOT re-scored. They are pinned v1.4 runs against v2 and
  were never asked these five questions; the chart shows those rows `not run`.
- Correctness A/B: 21/24 -> 24/24, zero regressions.

Performance, stated plainly
The published v1.4 perf report was recorded on CPython 3.14 and the new one on
3.11.15, so the apparent 2-6x speedup is the interpreter, not this release.
Re-measured on one interpreter, v1.5.0 is 28% slower creating entities and 22%
slower on the 10k roundtrip. Fully attributed: EZDXF_CALL_TIMEOUT=0 restores
v1.4.0's exact numbers. That is the deadlock fix being paid for, the knob is
documented, and winning it back is on the 1.6 roadmap.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Umut CAN
2026-08-06 09:45:20 +03:00
co-authored by Claude Opus 5
parent d5a0ff5109
commit b03a7615b1
107 changed files with 32640 additions and 4810 deletions
+9 -4
View File
@@ -13,7 +13,7 @@ import json
from pathlib import Path
from typing import Any
from benchmarks.tasks_v2 import TASKS_V2
from benchmarks.tasks_v3 import TASKS_V3
ROOT = Path(__file__).resolve().parents[1]
DEFAULT_INPUT_DIR = ROOT / "benchmarks" / "results" / "published"
@@ -66,8 +66,8 @@ def render_chart(reports: list[dict[str, Any]], output: Path) -> None:
matplotlib.rcParams["svg.hashsalt"] = "autocad-mcp-taskmatrix-v1"
task_ids = [task.task_id for task in TASKS_V2]
task_labels = [f"{task.task_id} ({task.category})" for task in TASKS_V2]
task_ids = [task.task_id for task in TASKS_V3]
task_labels = [f"{task.task_id} ({task.category})" for task in TASKS_V3]
servers = [report["adapter"] for report in reports]
status_by_server = {
report["adapter"]: {item["task_id"]: item["status"] for item in report["results"]}
@@ -132,12 +132,17 @@ def render_chart(reports: list[dict[str, Any]], output: Path) -> None:
Patch(facecolor=STATUS_COLORS["partial"], label="partial"),
Patch(facecolor=STATUS_COLORS["fail"], label="fail"),
Patch(facecolor=STATUS_COLORS["unsupported"], label="unsupported (no equivalent)"),
# Real rows now, not a theoretical state: the competitor reports were
# pinned and run against the v2 matrix, so the five v3 tasks were
# never put to them. An invented zero would look identical to a
# measured one, so they stay blank and say why.
Patch(facecolor=STATUS_COLORS["not_run"], edgecolor="#30363d", label="not run"),
]
ax.legend(
handles=legend_handles,
loc="upper left",
bbox_to_anchor=(0, -0.02),
ncol=4,
ncol=5,
frameon=False,
fontsize=9,
labelcolor="#e6edf3",