Tool count 131 -> 154; 474 -> 1242 collected tests. The release exists for two measured reasons. A 149-tool catalog costs a client ~40,300 tokens before it has asked for anything, and a drafter searching the way drafters think -- FILLET, BPOLY, QSELECT, WBLOCK -- scored df=0 against the stock index, because those words were simply not in the data. Discovery and batching - DISCOVERY_MODE=search replaces the catalog with search_tools + call_tool, ranked over an AutoCAD command and synonym corpus (discovery/aliases.py) on top of fastmcp's BM25 index. Measured: 40,305 -> 356 tokens advertised. - cad_batch runs a step list in one round trip; fields= projects result-heavy responses. The `core` profile is gone -- discovery replaced it. Honesty debt (Tier 0) - OCS/ECS: `extrusion` appeared nowhere in this codebase, so mirrored and rotated geometry reported coordinates that were quietly wrong. backends/ocs.py translates at the backend boundary; every tool coordinate is WCS on both engines, and a tilted plane refuses rather than answering with a number that looks fine. - drawing_save_as(fmt="dwg") wrote DXF bytes to a .dwg path. drawing_audit never repaired. analysis_measure_area shoelaced points the caller typed in. - ezdxf calls gained a per-call timeout, so one hung call can no longer wedge a server whose document lock is a single asyncio.Lock. Architecture - The 1854-line backend ABC is 19 contract modules; @capability(key, reason=) makes optional methods declarative and a gate fails on undeclared holes. - 8 layout/viewport tools, CHSPACE, CAD_PROGID escape hatch. - Wave A: selection filters, boundary tracing, hatch depth, annotation objects, measurement -- 13 tools of a planned 39. The REGION and 2D-boolean family was cut on evidence: add_region() yields zero ACIS bytes and the greiner_hormann substitute loses 28.2% of the area on a square with one semicircular edge. Verified against a live AutoCAD 2026, which found four COM defects no amount of reading would have caught -- AcadPViewport has no ViewCenter, and the line that set it had been swallowed by a bare except since v1.4. Benchmark v3 (15 tasks) and what it caught - v2 scored this server 10/10, which carried no information: every task in it exercised something the server was built around. Five tasks were added because they can fail, and three did while being written -- a hatch could not be measured though the refusal listed HATCH as measurable, the boundary tools and the measure tool disagreed about the same shape (139.2177 vs 139.2699), and a circle stored as two bulged vertices measured 0.0. - Competitor reports are NOT re-scored. They are pinned v1.4 runs against v2 and were never asked these five questions; the chart shows those rows `not run`. - Correctness A/B: 21/24 -> 24/24, zero regressions. Performance, stated plainly The published v1.4 perf report was recorded on CPython 3.14 and the new one on 3.11.15, so the apparent 2-6x speedup is the interpreter, not this release. Re-measured on one interpreter, v1.5.0 is 28% slower creating entities and 22% slower on the 10k roundtrip. Fully attributed: EZDXF_CALL_TIMEOUT=0 restores v1.4.0's exact numbers. That is the deadlock fix being paid for, the knob is documented, and winning it back is on the 1.6 roadmap. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
123 lines
4.7 KiB
Python
123 lines
4.7 KiB
Python
"""Tool-profile (TOOL_PROFILE=lean/full) behavior.
|
|
|
|
v1.5.0 removed the `core` profile: the discovery layer (tool search + the
|
|
AutoCAD command aliases) is the answer to a crowded surface, so a third
|
|
hand-maintained deny-list had no reason to exist. `core` must still *start*
|
|
the server — it falls back to `full` with a deprecation warning that names it.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import logging
|
|
|
|
import pytest
|
|
import pytest_asyncio
|
|
from fastmcp import Client
|
|
|
|
import config
|
|
import server
|
|
|
|
pytestmark = pytest.mark.asyncio
|
|
|
|
|
|
@pytest_asyncio.fixture(autouse=True)
|
|
async def _restore_full_profile():
|
|
"""Profiles mutate global server enablement; always restore full."""
|
|
yield
|
|
await server._apply_tool_profile("full")
|
|
|
|
|
|
async def _registered_names() -> set[str]:
|
|
return {tool.name for tool in await server._registered_tools() if getattr(tool, "name", None)}
|
|
|
|
|
|
async def test_lean_names_all_exist_in_registry():
|
|
registered = await _registered_names()
|
|
missing = server.LEAN_TOOL_NAMES - registered
|
|
assert not missing, f"LEAN_TOOL_NAMES references unknown tools: {sorted(missing)}"
|
|
|
|
|
|
async def test_only_lean_and_full_profiles_exist():
|
|
"""M1.1 — `core` is gone, deny-list and all."""
|
|
assert server.TOOL_PROFILES == ("lean", "full")
|
|
assert not hasattr(server, "CORE_EXCLUDED_TOOL_NAMES")
|
|
|
|
|
|
async def test_full_profile_hides_only_gated_solids():
|
|
"""Full hides nothing except the opt-in 3D tools while ENABLE_3D=false."""
|
|
info = await server._apply_tool_profile("full")
|
|
assert info["profile"] == "full"
|
|
assert set(info["disabled_tools"]) == set(server.SOLID_TOOL_NAMES)
|
|
assert info["enabled_count"] == info["registered_count"] - len(server.SOLID_TOOL_NAMES)
|
|
# An accepted profile is applied as requested, so there is nothing to report.
|
|
assert "requested" not in info
|
|
|
|
|
|
async def test_invalid_profile_falls_back_to_full(caplog):
|
|
with caplog.at_level(logging.WARNING, logger="autocad_mcp"):
|
|
info = await server._apply_tool_profile("does-not-exist")
|
|
assert info["profile"] == "full"
|
|
assert info["requested"] == "does-not-exist"
|
|
assert set(info["disabled_tools"]) == set(server.SOLID_TOOL_NAMES)
|
|
text = caplog.text
|
|
assert "does-not-exist" in text
|
|
# An unknown value is not a removed one; don't tell the user we deleted it.
|
|
assert "removed" not in text.lower()
|
|
|
|
|
|
async def test_removed_core_profile_falls_back_to_full_with_a_deprecation(caplog):
|
|
"""TOOL_PROFILE=core must keep the server bootable, loudly."""
|
|
with caplog.at_level(logging.WARNING, logger="autocad_mcp"):
|
|
info = await server._apply_tool_profile("core")
|
|
assert info["profile"] == "full"
|
|
assert set(info["disabled_tools"]) == set(server.SOLID_TOOL_NAMES)
|
|
text = caplog.text
|
|
assert "core" in text
|
|
assert "removed" in text.lower()
|
|
# The warning has to say what to use instead.
|
|
assert "lean" in text and "full" in text
|
|
|
|
|
|
async def test_removed_core_profile_is_reported_by_system_about():
|
|
"""A log line on a STDIO server is easy to miss; system_about must show it."""
|
|
await server._apply_tool_profile("core")
|
|
ctx = _AboutCtx()
|
|
about = await server.system_about(ctx=ctx)
|
|
assert about["tool_profile"]["profile"] == "full"
|
|
assert about["tool_profile"]["requested"] == "core"
|
|
|
|
|
|
class _AboutCtx:
|
|
"""system_about only reads lifespan_context; no backend needed."""
|
|
|
|
def __init__(self):
|
|
self.lifespan_context = {"backend": None}
|
|
|
|
|
|
async def test_core_env_value_still_serves_the_full_surface(monkeypatch):
|
|
"""The removal must not brick an existing TOOL_PROFILE=core deployment."""
|
|
monkeypatch.setenv("AUTOCAD_MCP_BACKEND", "ezdxf") # setattr here was a no-op (T0.8-B)
|
|
monkeypatch.setattr(config.settings, "tool_profile", "core")
|
|
async with Client(server.mcp) as client:
|
|
visible = {tool.name for tool in await client.list_tools()}
|
|
registered = await _registered_names()
|
|
assert visible == registered - server.SOLID_TOOL_NAMES
|
|
# The escape hatches core used to hide are back on the wire.
|
|
assert "system_run_command" in visible
|
|
|
|
|
|
async def test_lean_profile_filters_client_view(monkeypatch):
|
|
monkeypatch.setenv("AUTOCAD_MCP_BACKEND", "ezdxf") # setattr here was a no-op (T0.8-B)
|
|
monkeypatch.setattr(config.settings, "tool_profile", "lean")
|
|
async with Client(server.mcp) as client:
|
|
visible = {tool.name for tool in await client.list_tools()}
|
|
assert visible == set(server.LEAN_TOOL_NAMES)
|
|
|
|
|
|
async def test_profile_switch_is_idempotent():
|
|
await server._apply_tool_profile("lean")
|
|
info = await server._apply_tool_profile("full")
|
|
assert set(info["disabled_tools"]) == set(server.SOLID_TOOL_NAMES)
|
|
lean_again = await server._apply_tool_profile("lean")
|
|
assert lean_again["enabled_count"] == len(server.LEAN_TOOL_NAMES)
|