Files
autocad-mcp-pro/tests/test_com_timeout.py
T
Umut CANandClaude Opus 5 b03a7615b1 release: v1.5.0 — discovery layer, honesty debt closed, Wave A
Tool count 131 -> 154; 474 -> 1242 collected tests.

The release exists for two measured reasons. A 149-tool catalog costs a client
~40,300 tokens before it has asked for anything, and a drafter searching the way
drafters think -- FILLET, BPOLY, QSELECT, WBLOCK -- scored df=0 against the stock
index, because those words were simply not in the data.

Discovery and batching
- DISCOVERY_MODE=search replaces the catalog with search_tools + call_tool,
  ranked over an AutoCAD command and synonym corpus (discovery/aliases.py) on
  top of fastmcp's BM25 index. Measured: 40,305 -> 356 tokens advertised.
- cad_batch runs a step list in one round trip; fields= projects result-heavy
  responses. The `core` profile is gone -- discovery replaced it.

Honesty debt (Tier 0)
- OCS/ECS: `extrusion` appeared nowhere in this codebase, so mirrored and
  rotated geometry reported coordinates that were quietly wrong. backends/ocs.py
  translates at the backend boundary; every tool coordinate is WCS on both
  engines, and a tilted plane refuses rather than answering with a number that
  looks fine.
- drawing_save_as(fmt="dwg") wrote DXF bytes to a .dwg path. drawing_audit never
  repaired. analysis_measure_area shoelaced points the caller typed in.
- ezdxf calls gained a per-call timeout, so one hung call can no longer wedge a
  server whose document lock is a single asyncio.Lock.

Architecture
- The 1854-line backend ABC is 19 contract modules; @capability(key, reason=)
  makes optional methods declarative and a gate fails on undeclared holes.
- 8 layout/viewport tools, CHSPACE, CAD_PROGID escape hatch.
- Wave A: selection filters, boundary tracing, hatch depth, annotation objects,
  measurement -- 13 tools of a planned 39. The REGION and 2D-boolean family was
  cut on evidence: add_region() yields zero ACIS bytes and the greiner_hormann
  substitute loses 28.2% of the area on a square with one semicircular edge.

Verified against a live AutoCAD 2026, which found four COM defects no amount of
reading would have caught -- AcadPViewport has no ViewCenter, and the line that
set it had been swallowed by a bare except since v1.4.

Benchmark v3 (15 tasks) and what it caught
- v2 scored this server 10/10, which carried no information: every task in it
  exercised something the server was built around. Five tasks were added because
  they can fail, and three did while being written -- a hatch could not be
  measured though the refusal listed HATCH as measurable, the boundary tools and
  the measure tool disagreed about the same shape (139.2177 vs 139.2699), and a
  circle stored as two bulged vertices measured 0.0.
- Competitor reports are NOT re-scored. They are pinned v1.4 runs against v2 and
  were never asked these five questions; the chart shows those rows `not run`.
- Correctness A/B: 21/24 -> 24/24, zero regressions.

Performance, stated plainly
The published v1.4 perf report was recorded on CPython 3.14 and the new one on
3.11.15, so the apparent 2-6x speedup is the interpreter, not this release.
Re-measured on one interpreter, v1.5.0 is 28% slower creating entities and 22%
slower on the 10k roundtrip. Fully attributed: EZDXF_CALL_TIMEOUT=0 restores
v1.4.0's exact numbers. That is the deadlock fix being paid for, the knob is
documented, and winning it back is on the 1.6 roadmap.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 09:45:20 +03:00

133 lines
5.2 KiB
Python

"""``ComBackend._run`` deadline honesty — the STA executor must only be rebuilt
when *our* deadline actually expired.
On Python 3.11+ ``TimeoutError`` **is** ``asyncio.TimeoutError`` (and an
``OSError`` subclass), so a wrapped COM call that raises ``TimeoutError`` itself
— a socket read against a dead network share, a helper that re-raises one —
arrives at ``except TimeoutError`` looking exactly like a blown deadline. Taking
it at face value told the user AutoCAD "did not respond within 60s" for a call
that failed instantly, and (far worse than the ezdxf equivalent) threw away the
live single-thread STA executor and the cached AutoCAD connection with it.
These tests exercise the real ``_run`` — a plain ``ThreadPoolExecutor`` stands in
for the COM apartment, no AutoCAD and no win32 calls involved.
"""
from __future__ import annotations
import asyncio
import threading
import time
from concurrent.futures import ThreadPoolExecutor
import pytest
import backends.com_backend as cb
import config
from backends.com_backend import ComBackend
pytestmark = pytest.mark.asyncio
@pytest.fixture
def com_backend():
"""A ComBackend wired to a real (AutoCAD-free) single-thread executor."""
b = ComBackend()
# No initializer: _com_init would CoInitialize a real STA apartment.
b._executor = ThreadPoolExecutor(max_workers=1)
b._connected = True
yield b
if b._executor is not None:
b._executor.shutdown(wait=False)
b._executor = None
def _dead_share():
"""Stand-in for a COM/file call that raises TimeoutError of its own accord."""
raise TimeoutError("the network share went away")
async def test_call_raising_TimeoutError_is_not_blamed_on_the_deadline(com_backend, monkeypatch):
"""A TimeoutError *from the call* is an ordinary failure, not our deadline."""
monkeypatch.setattr(config.settings, "com_call_timeout", 30)
started = time.monotonic()
with pytest.raises(TimeoutError, match="network share") as excinfo:
await com_backend._run(_dead_share)
assert time.monotonic() - started < 5, "it failed instantly; nothing waited 30s"
assert "did not respond" not in str(excinfo.value)
assert not isinstance(excinfo.value, RuntimeError), "the original error must survive"
async def test_call_raising_TimeoutError_keeps_the_sta_executor(com_backend, monkeypatch):
"""Rebuilding the apartment on an ordinary error drops a live AutoCAD session."""
monkeypatch.setattr(config.settings, "com_call_timeout", 30)
monkeypatch.setitem(cb._COM_STATE, "app", object())
live = com_backend._executor
with pytest.raises(TimeoutError):
await com_backend._run(_dead_share)
assert com_backend._executor is live, "the STA executor must not be rebuilt"
assert com_backend._connected is True, "the AutoCAD connection must not be dropped"
assert "app" in cb._COM_STATE, "the cached COM app must survive an ordinary error"
assert await asyncio.wait_for(com_backend._run(lambda: "next"), timeout=5) == "next"
async def test_disabled_timeout_still_surfaces_a_callable_TimeoutError(com_backend, monkeypatch):
"""With COM_CALL_TIMEOUT=0 there is no deadline at all to blame."""
monkeypatch.setattr(config.settings, "com_call_timeout", 0)
live = com_backend._executor
with pytest.raises(TimeoutError, match="network share"):
await com_backend._run(_dead_share)
assert com_backend._executor is live
assert com_backend._connected is True
async def test_genuine_deadline_still_rebuilds_the_executor(com_backend, monkeypatch):
"""The real overrun path is deliberate and must keep working: the worker is
stuck inside an uncancellable COM call, so it is abandoned and replaced."""
monkeypatch.setattr(config.settings, "com_call_timeout", 0.2)
monkeypatch.setitem(cb._COM_STATE, "app", object())
stuck = com_backend._executor
entered = threading.Event()
def _hang():
entered.set()
time.sleep(2.0)
return "too late"
started = time.monotonic()
with pytest.raises(RuntimeError, match="did not respond") as excinfo:
await com_backend._run(_hang)
assert entered.is_set()
assert time.monotonic() - started < 1.5, "must abandon the call, not wait it out"
assert isinstance(excinfo.value.__cause__, TimeoutError)
assert com_backend._executor is not stuck, "the stuck apartment must be replaced"
assert com_backend._connected is False
assert "app" not in cb._COM_STATE, "the cached app belongs to the abandoned thread"
# The anti-deadlock guarantee: the next call runs on the fresh executor
# instead of queueing behind the still-sleeping worker.
monkeypatch.setattr(config.settings, "com_call_timeout", 10)
assert await asyncio.wait_for(com_backend._run(lambda: "fresh"), timeout=3) == "fresh"
async def test_ordinary_errors_are_unchanged(com_backend, monkeypatch):
"""Sanity: non-timeout failures never touched the executor and still don't."""
monkeypatch.setattr(config.settings, "com_call_timeout", 5)
live = com_backend._executor
def _boom():
raise ValueError("nope")
with pytest.raises(ValueError, match="nope"):
await com_backend._run(_boom)
assert com_backend._executor is live
assert await com_backend._run(lambda x: x * 2, 21) == 42