Files
autocad-mcp-pro/engineering/validator.py
T
Umut CANandClaude Opus 5.5 898e775f93 bench: matrix v5 adds mech_assembly; the correctness suite grows with the mechanical and sheet checks
The evidence for tracks B and G, in the shape tracks A and E used.

- benchmarks/tasks_v5.py: 18 tasks, DEFAULT_MATRIX = "v5"; v4, v3 and v2 stay
  addressable so an earlier report reproduces exactly rather than being only
  described. mech_assembly is roadmap criterion 3 - a flange-coupling sheet
  built from the part model and the sheet standard, through the code the
  tools call (apply_titleblock, draw_part/add_view/dimension_part,
  insert_std_part, extract_records -> draw_bom_table, add_balloon), gated on
  two numbers that cannot be negotiated: drawing_critique(focus=None) returns
  zero issues and the finalize score is at least 90 (it scores 100, 18/18 on
  the v5 matrix). The adapter returns the focuses that fired, so a
  regression names itself, and a test drops one balloon to prove the gate
  fails and names mech_bom_balloon_mismatch.
- Seven correctness checks (32 -> 39). mech_section_hatch_area is the one
  that cannot be faked: a sleeve of length 60, outside diameter 40 and bore
  20 cuts two faces totalling 1200 mm^2 exactly, read back out of the drawing
  through entity_measure. sheet_frame_iso5457_a3 pins the A3 frame (20,10 to
  410,287 inside the 420 x 297 trimmed sheet); mech_iso286_on_dimension pins
  40 H7 at +0.025/0 through the layout; std_part_iso4014_m12 pins s = 18 and
  k = 7.5 by value; mech_thread_unrepresented_is_caught is the seventh: a
  quality gate that never fires is not a gate.
- scripts/smoke_mech_com.py executed every new COM path once on AutoCAD 2026
  (25.1s): frame, title block, a sectioned shaft with 12 dimensions, an ISO
  4014 block with 1 ACADMCP_MECH chunk, a weld symbol, the parts list read off
  the drawing, a linked balloon, an xref attached / listed / detached, a
  31687-byte .dwg (AC1032) through ActiveX SaveAs, critique 0 issues, exit 0,
  every scratch document closed. It retries RPC_E_CALL_REJECTED (measured:
  the first call after Documents.Add is refused) and tracks its documents by
  the document list, since SaveAs renames them.
- Defects the evidence found, each fixed with a test that fails on the old
  code:
  * ezdxf dimension_diameter put the text beyond the SECOND chord point;
    ActiveX AddDimDiametric puts it beyond the first (measured live: a d50
    from (x,170) to (x,220), leader 10, text at (x,160)). Headlessly every
    front-view diameter stood on its own chain dimension. ezdxf now follows
    ActiveX.
  * draw_part's anchor goes on MECH; ActiveX refuses entity.Layer on a
    missing layer (measured -0x7ffdfff7 'Key not found'). draw.py ensures
    every target layer before the first entity.
  * bom.extract_records filtered on INSERT; ActiveX names it BLOCKREFERENCE
    (ObjectName AcDbBlockReference) - the live parts list was empty.
  * COM _entity_info reported no dimension text_position: ModelSpace.Item
    narrows to IAcadEntity, which has no TextPosition (measured
    AttributeError); read through _com_unnarrow (TextPosition measured on
    AcDbDiametricDimension, WCS). dim_overlap compared bbox centres and a
    diameter collided with its own bore.
  * pid graph raised KeyError 'start' mid-critique on a LINE whose point read
    AutoCAD refused while busy; such a line is left out.
  * the validator read "ISO 4014 - M12x60" as fake dimension text.
- The published A/B report is re-run over the grown suite rather than
  re-typed (26/39 -> 39/39, 13 fixed, 0 regressed), check_doc_numbers derives
  the task count from the v5 matrix, the token table is remeasured with a
  TOOL_PACKS=core,mech row (193 tools / 61,673 tokens), and the README's
  collected-tests figure is retaken (3990).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd
2026-09-23 13:30:02 +03:00

321 lines
11 KiB
Python
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
"""8-step pre-completion validator for engineering drawings."""
from __future__ import annotations
import re
from dataclasses import dataclass, field
from pathlib import Path
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from backends.base import AutoCADBackend
REAL_DIMENSION_TYPES = {
"DIMENSION",
"DIMLINEAR",
"DIMALIGNED",
"DIMANGULAR",
"DIMRADIUS",
"DIMDIAMETER",
"DIMORDINATE",
}
# Patterns that suggest a TEXT/MTEXT entity is masquerading as a dimension.
DIMENSION_LIKE_TEXT = re.compile(
r"(Ø|⌀|∅|%%c|%%C|R\d|±|%%p|\d+\s*[×x]\s*\d+|\d+\.\d+\s*mm)",
re.IGNORECASE,
)
# A standard-part designation (ISO 4014 - M12x60, DIN 912 - M8x20) names a
# part, it does not measure one: the "12x60" in it is the thread and the
# nominal length the standard's designation grammar spells that way. A parts
# list drawn as text carries one per row, so the designation is taken out
# before DIMENSION_LIKE_TEXT looks - a hand-typed "Ø78" beside it still flags.
STANDARD_DESIGNATION = re.compile(
r"\b(?:ISO|DIN|EN)\s*\d+(?:-\d+)?\s*-\s*M\d+(?:\.\d+)?(?:\s*[x×]\s*\d+(?:\.\d+)?)?",
re.IGNORECASE,
)
def looks_like_dimension(text: str) -> bool:
"""Is this TEXT/MTEXT content masquerading as a dimension?"""
return bool(DIMENSION_LIKE_TEXT.search(STANDARD_DESIGNATION.sub(" ", text or "")))
# R27 — match SPUR / HELICAL as whole words (not substrings of e.g.
# "SPURIOUS") when checking title/helix consistency.
_SPUR_RE = re.compile(r"\bSPUR\b", re.IGNORECASE)
_HELICAL_RE = re.compile(r"\bHELICAL\b", re.IGNORECASE)
# Layers that hold title-block text (scoped scan target for step 5).
_TITLE_LAYERS = {"TITLE", "TITLEBLOCK"}
@dataclass
class ValidationFinding:
"""One issue surfaced by the validator."""
severity: str # "error" | "warning" | "info"
code: str
message: str
detail: dict = field(default_factory=dict)
@dataclass
class ValidationResult:
"""Aggregate of validator findings."""
ok: bool
findings: list[ValidationFinding]
summary: dict
def to_dict(self) -> dict:
return {
"ok": self.ok,
"summary": self.summary,
"findings": [
{
"severity": f.severity,
"code": f.code,
"message": f.message,
"detail": f.detail,
}
for f in self.findings
],
}
class DrawingValidator:
"""8-step pre-completion validator."""
async def run(
self,
backend: AutoCADBackend,
*,
expected: dict | None = None,
) -> ValidationResult:
"""Run all 8 checks; return aggregate ValidationResult."""
findings: list[ValidationFinding] = []
expected = expected or {}
ents: list = []
# 1 — fake dimensions
try:
ents = await backend.entity_list(limit=10000)
text_likes = [
e
for e in ents
if e.type in ("TEXT", "MTEXT")
and looks_like_dimension(getattr(e, "properties", {}).get("text", "") or "")
]
real_dims = [e for e in ents if e.type in REAL_DIMENSION_TYPES]
if text_likes:
findings.append(
ValidationFinding(
"warning",
"fake_dimension_text",
f"{len(text_likes)} TEXT/MTEXT entities look like dimensions "
"(Ø, ±, R, etc.) — use dimension_* tools instead",
{
"sample_handles": [t.handle for t in text_likes[:5]],
"real_dim_count": len(real_dims),
},
)
)
except Exception as exc:
findings.append(
ValidationFinding(
"warning",
"dim_check_failed",
str(exc),
)
)
# 2 — hidden orphans
try:
hidden = [e for e in ents if not e.visible]
if hidden:
findings.append(
ValidationFinding(
"error",
"hidden_orphans",
f"{len(hidden)} entities have visible=False — hide-fallback "
"is forbidden, use entity_delete",
{"sample_handles": [h.handle for h in hidden[:5]]},
)
)
except Exception:
pass
# 3 — linetypes
try:
lts = {x.lower() for x in await backend.linetype_list()}
missing = [lt for lt in ("center", "hidden", "phantom") if lt not in lts]
if missing:
findings.append(
ValidationFinding(
"error",
"linetypes_missing",
f"Standard linetypes not loaded: {', '.join(missing).upper()}",
{"missing": missing},
)
)
except Exception as exc:
findings.append(
ValidationFinding(
"warning",
"linetype_check_failed",
str(exc),
)
)
# 4 — gear views (heuristic, only if expected demands)
if expected.get("must_have_bore") or expected.get("must_have_keyway"):
bore_circles = [e for e in ents if e.type == "CIRCLE" and e.layer == "GEOMETRY"]
if expected.get("must_have_bore") and len(bore_circles) < 1:
findings.append(
ValidationFinding(
"error",
"bore_missing",
"Expected bore but no circle on GEOMETRY layer found",
)
)
# draw_keyed_bore emits the keyway notch as an OPEN 4-point polyline
# on GEOMETRY — requiring `closed` here made this a permanent
# false-negative. Match a small (3-6 point) polyline, open or closed.
keyway_polys = [
e
for e in ents
if e.type in ("LWPOLYLINE", "POLYLINE")
and e.layer == "GEOMETRY"
and 3 <= len(e.properties.get("points", [])) <= 6
]
if expected.get("must_have_keyway") and len(keyway_polys) < 1:
findings.append(
ValidationFinding(
"warning",
"keyway_section_unverified",
"Could not confirm keyway via heuristic "
"(small 3-6 point polyline on GEOMETRY)",
)
)
# 5 — title consistency
helix = expected.get("helix_angle")
if helix:
# R27 — scope the scan to title-block text where possible (TITLE /
# TITLEBLOCK layer); fall back to all TEXT-layer text if no title
# layer carries any text.
title_block_texts = [
e.properties.get("text", "") or ""
for e in ents
if e.type in ("TEXT", "MTEXT") and e.layer in _TITLE_LAYERS
]
if title_block_texts:
title_texts = title_block_texts
else:
title_texts = [
e.properties.get("text", "") or ""
for e in ents
if e.type in ("TEXT", "MTEXT") and e.layer == "TEXT"
]
joined = " | ".join(title_texts)
# R27 — whole-word matching so "SPURIOUS" no longer trips SPUR.
has_spur = bool(_SPUR_RE.search(joined))
has_helical = bool(_HELICAL_RE.search(joined))
if has_spur and has_helical:
findings.append(
ValidationFinding(
"error",
"title_helical_spur",
"Title contains 'HELICAL SPUR GEAR' — invalid; use "
"'HELICAL GEAR' or 'SPUR GEAR'",
)
)
elif has_spur:
findings.append(
ValidationFinding(
"error",
"title_says_spur_for_helical",
f"Title contains SPUR but part is helical (helix_angle={helix})",
{"sample": joined[:200]},
)
)
# 6 — saved to disk
try:
info = await backend.drawing_info()
saved_attr = getattr(info, "saved", None)
full_path = getattr(info, "full_path", "") or ""
if not saved_attr:
findings.append(
ValidationFinding(
"error",
"not_saved",
"Drawing has unsaved changes — call drawing_save before finalize",
{"full_path": full_path},
)
)
elif full_path and not Path(full_path).exists():
findings.append(
ValidationFinding(
"error",
"file_missing",
f"Drawing reports saved=True but file not on disk: {full_path}",
)
)
except Exception as exc:
findings.append(
ValidationFinding(
"warning",
"save_check_failed",
str(exc),
)
)
# 7 — screenshot
try:
shot = await backend.view_screenshot()
if not shot:
findings.append(
ValidationFinding(
"info",
"screenshot_unavailable",
"Screenshot returned no bytes (ezdxf needs matplotlib install)",
)
)
except Exception as exc:
findings.append(
ValidationFinding(
"info",
"screenshot_failed",
str(exc),
)
)
# 8 — dwg path returned
try:
info = await backend.drawing_info()
if not getattr(info, "full_path", ""):
findings.append(
ValidationFinding(
"warning",
"no_path",
"drawing_info.full_path is empty",
)
)
except Exception:
pass
summary = {
sev: sum(1 for f in findings if f.severity == sev)
for sev in ("error", "warning", "info")
}
return ValidationResult(
ok=summary["error"] == 0,
findings=findings,
summary=summary,
)