Files
autocad-mcp-pro/tests/test_quarantine_gate.py
T
Umut CANandClaude Opus 5 ca1c79fd01
Release / test (full, windows-latest, 3.11) (push) Canceled after 0s
Release / test (pdf, ubuntu-latest, 3.11) (push) Canceled after 0s
Release / test (pdf, ubuntu-latest, 3.12) (push) Canceled after 0s
Release / build (push) Canceled after 0s
Release / publish-pypi (push) Canceled after 0s
Release / github-release (push) Canceled after 0s
fix: seven defects an adversarial audit found, and the gates that missed them
Every one of these was reported ok/clean to the caller while being wrong.

Silently wrong numbers
- selection_filter(min_area) read `entity.dxf.get("area", 0.0)` for SOLID,
  TRACE and HATCH, none of which carry that attribute, so every hatch
  measured 0.0. A 1600 mm^2 hatch was excluded by min_area=1.0 while
  `filtered_by` asserted the filter had run, and COM (which reads ActiveX
  .Area) returned a different selection for the same call. It now answers
  from the measurement `entity_measure` uses, so the two cannot disagree.
- Every headless dimension rendered through ezdxf's bare `Standard` style:
  1.0 mm text with a comma decimal marker, against ISO 3098's 2.5 mm minimum
  and ISO 129's point -- while the document header itself said $DIMTXT 2.5.
  `ezdxf.new(setup=True)` is not the fix; measured, it renders a 100 mm
  dimension as "10000". The attributes are now stored explicitly.
- Saving writes `dimrnd = 0.0` onto the Standard style, and ezdxf applies
  `xround(value, dimrnd)` whenever the attribute is present at all --
  `xround(33.333, 0.0)` is 33. So on any reopened file, including one this
  server saved, every dimension was rounded to a whole unit before DIMDEC
  formatted it: a 12.75 mm feature dimensioned 13. AutoCAD means DIMRND 0 as
  *no* rounding; the sentinel is now discarded at every load site, and a real
  0.5 mm rounding is left alone.
- The DIM* header fold gave out-of-range values teeth they did not have when
  they died in the header: dim_decimals=-1 returned ok and then killed the
  next dimension with `ValueError: Format specifier missing precision` from
  inside ezdxf, and 40 printed forty digits of float noise onto the sheet.
  Ranges are now AutoCAD's own (DIMDEC 0-8, DIMZIN 0-15, lengths positive).

Security
- security.py recorded the policy as "sanitize_command and sanitize_lisp each
  have exactly ONE caller ... the policy is scoped to the CHANNEL". The
  premise was false. `_ensure_linetype_loaded` and `ComBackend.linetype_load`
  build `_-LINETYPE _LOAD {name} {file}` and hand it to SendCommand, which
  treats a newline and ';' as segment separators -- so a `linetype=` argument
  carrying either injected further AutoCAD commands, and the 36-verb denylist
  never saw that channel. Guarded with the DXF symbol-name rule (the
  injection characters fall out of it rather than being enumerated) plus
  validate_path for the file argument, which was also the one path-taking
  tool reaching the filesystem unchecked. The policy comment is corrected.

Gates that could not see
- drawing_critique returned zero issues on a quarantined document: all seven
  checks wrapped their backend call in a bare `except Exception`. CLAUDE.md
  rule 7 makes that call the gate an agent is told to trust. Only the
  quarantine now escapes; an ordinary broken check still degrades.
  `_check_dimension_legibility` also read `_doc` directly from the event loop
  thread, missing the gate and performing exactly the unsynchronised access
  the quarantine exists to prevent. `construction_clear` answered
  "deleted: 0" -- there was nothing to clear -- for the same reason.
- The iso128 focus checked layer lineweights only, so a sheet lettered at
  1.0 mm critiqued clean. It now reads the effective dimension height and
  separator, header-then-style, in the order the renderer resolves.
- check_doc_numbers.py derived five values it never checked, and those were
  the ones that had drifted. The collected-test count (README said 1274, the
  tree had 1369) and the A/B pass-rate and Fixed columns are now gated.
- benchmarks/README.md's v1.4.0 table was impossible on its face -- 21
  passing plus 5 fixed is 26, not 24 -- and contradicted the report it
  linked, because `ab-v1.4.0-vs-v1.5.0.json` was regenerated from a tree that
  already carried the dimension fixes and so never measured the v1.5.0 tag.
  Replaced with a real run: ab-v1.4.0-vs-v1.5.1.json, 21/26 -> 26/26.

Release gate
- release.yml ran one ubuntu/3.11 lane, so a Windows-only defect could reach
  PyPI with no Windows job having executed. That is not hypothetical: v1.5.0
  shipped `except pywintypes.com_error` raising NameError wherever pywin32 is
  absent. It now runs the same three lanes as ci.yml, Windows included.

1368 passed, 1 xfailed. ruff clean. Tool inventory and all 16 gated document
numbers match the tree.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 23:07:50 +03:00

121 lines
3.9 KiB
Python

"""A quarantined document must not be able to pass the quality gate.
`backends/quarantine.py` refuses every call routed through `_async` once a
timed-out write has been abandoned on the document. But `engineering/critique.py`
wraps each check in a bare ``except Exception`` and degrades to ``[]`` — so
`DocumentQuarantineError` was swallowed and `drawing_critique` returned *zero
issues* on a document the backend had declared untrusted.
CLAUDE.md premium rule 7 makes "drawing_critique must return zero issues before
drawing_finalize" the gate an agent is told to trust. A gate that reports clean
because it could not look is worse than no gate: it is the same silent-success
failure the whole quarantine exists to stop, one layer up.
The same swallow sat in `construction_clear` (step 9 of the standard workflow),
which answered ``{"ok": true, "deleted": 0}`` — "there was nothing to clear" —
on a document it could not read.
"""
from __future__ import annotations
import threading
import pytest
import config
from backends.quarantine import DocumentQuarantineError
pytestmark = pytest.mark.asyncio
async def _quarantine(backend, monkeypatch):
"""Abandon a real write, the way a real timeout does.
Hand-building a QuarantineRecord would test the gate against a fixture
rather than against the state the backend actually reaches.
"""
monkeypatch.setattr(config.settings, "ezdxf_call_timeout", 0.2)
entered = threading.Event()
release = threading.Event()
def _writer():
entered.set()
release.wait(5.0)
return "too late"
with pytest.raises(RuntimeError):
await backend._async(_writer)
assert entered.is_set()
assert backend._quarantine is not None
monkeypatch.setattr(config.settings, "ezdxf_call_timeout", 10)
return release
async def test_a_geometric_focus_refuses_rather_than_reporting_clean(backend, monkeypatch):
from engineering.critique import run_critique
await backend.entity_create_line(0, 0, 10, 0, layer="GEOMETRY")
release = await _quarantine(backend, monkeypatch)
try:
with pytest.raises(DocumentQuarantineError):
await run_critique(backend, focus=["duplicate_entities"])
finally:
release.set()
async def test_the_full_critique_refuses_too(backend, monkeypatch):
from engineering.critique import run_critique
release = await _quarantine(backend, monkeypatch)
try:
with pytest.raises(DocumentQuarantineError):
await run_critique(backend, focus=None)
finally:
release.set()
async def test_the_legibility_check_does_not_read_the_document_under_quarantine(
backend, monkeypatch
):
"""It reads `backend._doc` directly, off the event loop thread, with no
lock — the exact unsynchronised access the quarantine exists to prevent."""
from engineering.critique import _check_dimension_legibility
release = await _quarantine(backend, monkeypatch)
try:
with pytest.raises(DocumentQuarantineError):
_check_dimension_legibility(backend)
finally:
release.set()
async def test_construction_clear_does_not_report_nothing_to_clear(backend, monkeypatch):
await backend.entity_create_line(0, 0, 10, 0, layer="CONSTRUCTION")
release = await _quarantine(backend, monkeypatch)
try:
with pytest.raises(DocumentQuarantineError):
await backend.construction_clear()
finally:
release.set()
async def test_an_ordinary_failure_is_still_degraded_not_raised(backend):
"""The bare excepts exist so one broken check cannot fail the whole pass.
Only the quarantine escapes them; everything else keeps degrading.
"""
from engineering.critique import run_critique
async def _boom(*_a, **_k):
raise RuntimeError("layer_list is having a bad day")
backend.layer_list = _boom
issues = await run_critique(backend, focus=["layer_color"])
assert issues == []