Files
autocad-mcp-pro/scripts/smoke_mech_com.py
T
Umut CANandClaude Opus 5.5 898e775f93 bench: matrix v5 adds mech_assembly; the correctness suite grows with the mechanical and sheet checks
The evidence for tracks B and G, in the shape tracks A and E used.

- benchmarks/tasks_v5.py: 18 tasks, DEFAULT_MATRIX = "v5"; v4, v3 and v2 stay
  addressable so an earlier report reproduces exactly rather than being only
  described. mech_assembly is roadmap criterion 3 - a flange-coupling sheet
  built from the part model and the sheet standard, through the code the
  tools call (apply_titleblock, draw_part/add_view/dimension_part,
  insert_std_part, extract_records -> draw_bom_table, add_balloon), gated on
  two numbers that cannot be negotiated: drawing_critique(focus=None) returns
  zero issues and the finalize score is at least 90 (it scores 100, 18/18 on
  the v5 matrix). The adapter returns the focuses that fired, so a
  regression names itself, and a test drops one balloon to prove the gate
  fails and names mech_bom_balloon_mismatch.
- Seven correctness checks (32 -> 39). mech_section_hatch_area is the one
  that cannot be faked: a sleeve of length 60, outside diameter 40 and bore
  20 cuts two faces totalling 1200 mm^2 exactly, read back out of the drawing
  through entity_measure. sheet_frame_iso5457_a3 pins the A3 frame (20,10 to
  410,287 inside the 420 x 297 trimmed sheet); mech_iso286_on_dimension pins
  40 H7 at +0.025/0 through the layout; std_part_iso4014_m12 pins s = 18 and
  k = 7.5 by value; mech_thread_unrepresented_is_caught is the seventh: a
  quality gate that never fires is not a gate.
- scripts/smoke_mech_com.py executed every new COM path once on AutoCAD 2026
  (25.1s): frame, title block, a sectioned shaft with 12 dimensions, an ISO
  4014 block with 1 ACADMCP_MECH chunk, a weld symbol, the parts list read off
  the drawing, a linked balloon, an xref attached / listed / detached, a
  31687-byte .dwg (AC1032) through ActiveX SaveAs, critique 0 issues, exit 0,
  every scratch document closed. It retries RPC_E_CALL_REJECTED (measured:
  the first call after Documents.Add is refused) and tracks its documents by
  the document list, since SaveAs renames them.
- Defects the evidence found, each fixed with a test that fails on the old
  code:
  * ezdxf dimension_diameter put the text beyond the SECOND chord point;
    ActiveX AddDimDiametric puts it beyond the first (measured live: a d50
    from (x,170) to (x,220), leader 10, text at (x,160)). Headlessly every
    front-view diameter stood on its own chain dimension. ezdxf now follows
    ActiveX.
  * draw_part's anchor goes on MECH; ActiveX refuses entity.Layer on a
    missing layer (measured -0x7ffdfff7 'Key not found'). draw.py ensures
    every target layer before the first entity.
  * bom.extract_records filtered on INSERT; ActiveX names it BLOCKREFERENCE
    (ObjectName AcDbBlockReference) - the live parts list was empty.
  * COM _entity_info reported no dimension text_position: ModelSpace.Item
    narrows to IAcadEntity, which has no TextPosition (measured
    AttributeError); read through _com_unnarrow (TextPosition measured on
    AcDbDiametricDimension, WCS). dim_overlap compared bbox centres and a
    diameter collided with its own bore.
  * pid graph raised KeyError 'start' mid-critique on a LINE whose point read
    AutoCAD refused while busy; such a line is left out.
  * the validator read "ISO 4014 - M12x60" as fake dimension text.
- The published A/B report is re-run over the grown suite rather than
  re-typed (26/39 -> 39/39, 13 fixed, 0 regressed), check_doc_numbers derives
  the task count from the v5 matrix, the token table is remeasured with a
  TOOL_PACKS=core,mech row (193 tools / 61,673 tokens), and the README's
  collected-tests figure is retaken (3990).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd
2026-09-23 13:30:02 +03:00

291 lines
11 KiB
Python

"""Exercise tracks B and G once against the live AutoCAD on this machine.
Creates NEW documents (never touches the operator's open drawings), then runs
the whole track through the same code the tools call:
xref source: drawing_new -> a line -> drawing_export_dwg -> close
apply_layer_set("mech") -> apply_titleblock (ISO 5457 A3 + ISO 7200)
-> build_part -> draw_part -> add_view(section) -> dimension_part
-> insert_std_part (ISO 4014 block + ACADMCP_MECH xdata)
-> weld_symbol_prims -> extract_records -> rows_from_records
-> draw_bom_table -> add_balloon
-> xref_attach / xref_manage("list") / xref_manage("detach")
-> run_critique(focus=None) -> drawing_export_dwg
and prints what came back as JSON. Exit codes:
0 the sheet drew, the critique returned nothing and the .dwg exists
1 it drew but the critique found something or the DWG is missing
2 no live CAD application reachable (pywin32 missing, or the ProgID
cannot be created) -- the COM paths then remain fake-tested only
The backend's ``connect()`` is lazy on purpose (the server must start before
AutoCAD is open), so reachability is read from ``system_status()`` rather than
from ``connect()``. Every document this run opens is closed on the way out,
also when a step fails, unless ``--keep`` is given.
AUTOCAD_MCP_BACKEND=com uv run --frozen python scripts/smoke_mech_com.py
uv run --frozen python scripts/smoke_mech_com.py --keep # leave it open
"""
from __future__ import annotations
import asyncio
import json
import sys
import tempfile
import time
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT))
XDATA_APP = "ACADMCP_MECH"
#: Every DWG since R13 opens with "AC10xx" (AC1032 for the 2018 format).
DWG_MAGIC = b"AC10"
#: ``RPC_E_CALL_REJECTED``: AutoCAD's message filter refused the call because
#: it was busy (a document still initialising after ``Documents.Add`` is the
#: case measured on this seat). A refused call was not executed.
RPC_E_CALL_REJECTED = -2147418111
#: How long a refused call is retried before the smoke gives up on it.
REJECTED_PATIENCE_S = 90.0
#: Names of the documents this run opened and has not closed yet.
_OPENED: list[str] = []
def _patient(backend) -> None:
"""Retry any backend call AutoCAD refused with ``RPC_E_CALL_REJECTED``.
Measured on AutoCAD 2026: the first call after ``Documents.Add`` was
refused, the smoke died, and its cleanup - refused as well - left the
scratch document open. Refused calls are retried with backoff for up to
``REJECTED_PATIENCE_S``; a document created twice by a retried
``drawing_new`` is still closed, because ``_new_document`` tracks the
document list, not a name it assumes.
"""
run = backend._run
async def patient(func, *args, **kwargs):
deadline = time.monotonic() + REJECTED_PATIENCE_S
pause = 0.5
while True:
try:
return await run(func, *args, **kwargs)
except RuntimeError as exc:
cause = exc.__cause__
hr = cause.args[0] if cause is not None and cause.args else None
if hr != RPC_E_CALL_REJECTED or time.monotonic() + pause > deadline:
raise
await asyncio.sleep(pause)
pause = min(pause * 2.0, 5.0)
backend._run = patient
async def _names(backend) -> set[str]:
return {row["name"] for row in await backend.document_list()}
async def _new_document(backend) -> str:
"""``drawing_new``, recording every document it added (normally one)."""
before = await _names(backend)
await backend.drawing_new()
added = sorted((await _names(backend)) - before)
_OPENED.extend(name for name in added if name not in _OPENED)
return added[-1] if added else ""
async def _close_leftovers(backend) -> None:
open_names = {row["name"] for row in await backend.document_list()}
for name in list(_OPENED):
if name in open_names:
try:
await backend.document_close(name, discard=True)
except Exception as exc: # best effort on the way out
print(f"could not close {name}: {exc}", file=sys.stderr)
_OPENED.remove(name)
async def _xref_source(backend, workspace: Path) -> Path:
"""A one-line DWG in its own scratch document, saved and closed.
An xref has to be a DWG, and it cannot be the host document itself -
``SaveAs`` rebinds the host to the file it wrote.
"""
created = await _new_document(backend)
await backend.entity_create_line(0.0, 0.0, 50.0, 0.0, layer="0")
path = workspace / "xref_source.dwg"
before = await _names(backend)
await backend.drawing_export_dwg(str(path), "R2018")
rebound = await _rebound(backend, created, before)
await backend.document_close(rebound, discard=True)
_OPENED.remove(rebound)
return path
async def _rebound(backend, created: str, before: set[str]) -> str:
"""``SaveAs`` renames the active document to the file it wrote; track that name.
Read from the document list rather than ``system_status``: the latter
swallows a refused call and answers without ``active_document``.
"""
after = await _names(backend)
renamed = sorted(after - before)
rebound = renamed[0] if created not in after and len(renamed) == 1 else created
if created in _OPENED:
_OPENED.remove(created)
if rebound not in _OPENED:
_OPENED.append(rebound)
return rebound
async def _sheet(backend, workspace: Path, xref_path: Path) -> dict:
from engineering.critique import run_critique
from engineering.layers import apply_layer_set
from engineering.mech.annotate import weld_symbol_prims
from engineering.mech.draw import add_view, dimension_part, draw_part, draw_prims
from engineering.mech.part import build_part
from engineering.mech.stdparts import insert_std_part
from engineering.sheet.bom import (
add_balloon,
draw_bom_table,
extract_records,
rows_from_records,
)
from engineering.sheet.titleblock import (
TB_HEIGHT,
TitleBlockMetadata,
apply_titleblock,
titleblock_origin,
)
created = await _new_document(backend)
await apply_layer_set(backend, "mech")
await apply_titleblock(
backend,
size="A3",
metadata=TitleBlockMetadata(title="COM SMOKE", drawing_no="SMOKE-001", scale="1:1"),
projection="first",
frame=True,
zones=True,
marks=True,
)
shaft = build_part(
{
"kind": "revolved",
"name": "SHAFT",
"material": "steel",
"segments": [
{"length": 40.0, "d_outer": 30.0},
{"length": 20.0, "d_outer": 50.0, "d_inner": 20.0},
],
}
)
drawn = await draw_part(backend, shaft, at=(125.0, 170.0), views=("front",), dimension=False)
section = await add_view(
backend,
drawn["part_id"],
"section",
plane={"p1": [0.0, 0.0], "p2": [60.0, 0.0], "label": "A"},
style="full",
)
dims = await dimension_part(backend, drawn["part_id"], style="chain")
bolt = await insert_std_part(backend, "ISO 4014 - M12x60", at=(250.0, 120.0), view="side")
bolt_handle = bolt.get("handle") or (bolt.get("handles") or [None])[0]
got = await backend.entity_get_xdata(bolt_handle, XDATA_APP)
await draw_prims(backend, weld_symbol_prims(kind="fillet", size=5.0), at=(60.0, 100.0))
records = await extract_records(backend)
rows = rows_from_records(records)
tb_x, tb_y = titleblock_origin("A3")
table = await draw_bom_table(backend, rows, at=(tb_x, tb_y + TB_HEIGHT))
balloon = await add_balloon(
backend,
item=int(rows[0]["item"]),
at=(250.0, 150.0),
leader_to=(250.0, 125.0),
targets=[str(h) for h in rows[0]["handles"]],
)
attached = await backend.xref_attach(str(xref_path), (30.0, 30.0), 1.0, 0.0, "attach")
listed = await backend.xref_manage(attached["name"], "list", None)
detached = await backend.xref_manage(attached["name"], "detach", None)
after = await backend.xref_manage(attached["name"], "list", None)
issues = await run_critique(backend, None)
dwg_path = workspace / "mech_smoke.dwg"
before = await _names(backend)
exported = await backend.drawing_export_dwg(str(dwg_path), "R2018")
await _rebound(backend, created, before)
head = dwg_path.read_bytes()[:6] if dwg_path.exists() else b""
return {
"part_id": drawn["part_id"],
"views": len(drawn.get("views") or []) + 1,
"section_omitted": section.get("omitted"),
"dimensions": int(dims.get("count") or len(dims.get("handles") or [])),
"bolt_handle": bolt_handle,
"xdata_chunks": len(got["xdata"].get(XDATA_APP, [])),
"parts_list_rows": len(rows),
"parts_list_representation": table.get("representation"),
"balloon": balloon.get("handle"),
"xref_attached": attached.get("name"),
"xref_listed": len(listed.get("xrefs") or []),
"xref_detached": bool(detached.get("ok")),
"xrefs_after_detach": len(after.get("xrefs") or []),
"dwg": {
"path": exported.get("path", str(dwg_path)),
"exists": dwg_path.exists(),
"bytes": dwg_path.stat().st_size if dwg_path.exists() else 0,
"magic": head.decode("ascii", "replace"),
},
"critique_issues": len(issues),
"critique": [issue.to_dict() for issue in issues],
}
async def main(keep_document: bool = False) -> int:
try:
from backends.com_backend import ComBackend
except ImportError as exc:
print(f"pywin32 not available: {exc}")
return 2
backend = ComBackend()
_patient(backend)
try:
await backend.connect()
except Exception as exc:
print(f"no live AutoCAD reachable: {exc}")
return 2
try:
status = await backend.system_status()
if not status.get("connected"):
print(f"no live AutoCAD reachable: {status.get('error', status)}")
return 2
workspace = Path(tempfile.mkdtemp(prefix="mech_smoke_"))
xref_path = await _xref_source(backend, workspace)
report = {
"autocad": status.get("autocad_version") or status.get("cad_progid"),
**await _sheet(backend, workspace, xref_path),
}
print(json.dumps(report, indent=2))
ok = (
report["critique_issues"] == 0
and report["dwg"]["exists"]
and report["dwg"]["magic"].startswith(DWG_MAGIC.decode())
and report["xrefs_after_detach"] == 0
)
return 0 if ok else 1
finally:
if not keep_document:
await _close_leftovers(backend)
await backend.disconnect()
if __name__ == "__main__":
raise SystemExit(asyncio.run(main(keep_document="--keep" in sys.argv[1:])))