Files
Umutcan Edizsalan 0c7f64ceb6 fix(correctness-bundle): I16 correctness fixes + dead code removal + ruff clean
N1: COM hatch non-associative (AddHatch associativity=False)
N2: ezdxf refuses DWG saves with clear error; server.py derives fmt from extension
R19: dimension_aligned dist→distance; dimension_angular correct add_angular_dim_2l args (base/line1/line2/location)
R9: polar array 360° no-duplicate (divisor=n when fill_angle%360<ε)
R13: entity_offset side_x/side_y honored in ezdxf via cross-product (LINE) and distance-to-center (CIRCLE)
R18: drawing_new bootstrap failure surfaces status="degraded" instead of swallowing silently
Dead code: engineering/section.py, gear.generate_tooth_profile, CommandResult, _ACI_NAMES, set_layer_active, ensure_linetypes, ComBackend._batch_mode instance attr, engineering/__init__ build-seam comment
Ruff: 38 auto-fixed + 5 manual → codebase is now ruff-clean; CI lint is now blocking

184 tests pass.
2026-06-18 11:34:30 +03:00

20 KiB
Raw Permalink Blame History

AutoCAD MCP Pro — Evidence-Based Roadmap

Mission: become the best AutoCAD MCP for LLM-driven drawing. The durable moat is the standards-validation gate (no competing CAD MCP — build123d / FreeCAD / Blender / Rhino / Onshape — ships any design-rule validation). Everything below is sequenced to widen that moat or to remove the silent no-ops that make it lie.

Priority signal = (impact + differentiation + evidence_strength) / effort, computed on post-challenge adjusted scores, then adjusted by judgment (quick wins and dependency order). DEFER → Backlog; RESCOPE → adjusted scope used; KEEP → as scored.


Scoring table (all ideas, adjusted, sorted by priority signal)

ID Title Imp Eff Diff Ev Signal Verdict Area
I9 Expand deterministic geometry surface 3 2 3 5 5.5 KEEP→rescoped* determinism
I3 ProCAD pre-plan clarification pass 3 2 4 4 5.5 RESCOPE closed-loop
I16 Correctness + hygiene bundle (DIAGNOSTICS) 3 2 2 5 5.0 KEEP hygiene
I6 GD&T primitive (ISO 1101 / Y14.5) 5 3 5 3 4.3 KEEP (impact↑) iso-production
I14 NL→primitive grounding (trim/fillet/chamfer) 3 3 3 4 3.3 RESCOPE determinism
I15 Staged validator (code→geometric→rubric) 5 3 5 5 5.0 RESCOPE metrics
I12 Authenticated remote HTTP 2 1 2 2 6.0 KEEP (quick win) infra
I4 Scalar drawing-score / Invalidity-Ratio 3 2 3 4 5.0 KEEP metrics
I2 Visual critique / render as ImageContent 3 2 4 5 6.0 RESCOPE (cheap half) closed-loop
I18 Real drawing repair (recover + Auditor.fix) 2 2 1 3 3.0 KEEP robustness
I1 Routed iterative Refiner loop 4 4 5 5 3.5 RESCOPE closed-loop
I5 ISO 129 tolerances on dimensions 4 4 3 5 3.0 RESCOPE iso-production
I8 ISO-129 DimStyle + MLEADER 3 3 2 3 2.7 as scored iso-production
I19 COM test harness (mocked) + cov gate 2 2 1 3 3.0 KEEP hygiene
I7 Real plotted sheets + real DWG 3 3 3 3 3.0 RESCOPE (cheap half) iso-production
I17 Fast selection via native query 2 3 1 2 1.7 ezdxf half only perf
I13 Standard-parts / symbol library 3 4 3 2 2.0 RESCOPE breadth
I11 Parametric editability / conv. edit 3 4 3 2 2.0 as scored breadth
I10 3D solids tier (or fix false claim) 2 4 2 2 1.5 split breadth

* I9/I3 share the top signal; I9 edges first because it is zero-risk and directly converts the GeoGramBench failure mode (<50% program-to-geometry) into deterministic tool calls.


Sprint 1 — Quick wins & correctness

Foundation: the closed-loop moat only pays off if the validator/critique signals are not silent no-ops. Land the cheap, CI-verifiable correctness fixes and the highest-signal determinism wins first.

I16 — Correctness + hygiene bundle (DIAGNOSTICS fixes) · imp 3 / eff 2 / diff 2

Why now: Every premium-workflow signal that the moat depends on is currently degraded by confirmed defects (fake DWG, broken aligned/angular dims, polar-array dup, silent bootstrap). Iteration evidence (Self-Improving CAD +13.4pp/round) only pays off if signals aren't dead. Depends on: land alongside I19 (COM-only N1/R5 need a harness). First step: One PR for the CI-verifiable wins: fix R19 (ezdxf_backend.py:696 dist→positional distance; :711 add_angular_dim_2l(base=…, line1=…, line2=…, location=(tx,ty))), flip tests/test_dimensions.py:16,23 to xfail(strict=True), fix R9 divisor (:947), R12 location=(tx,ty) (:715), R13 offset side via cross-product sign; then ruff check --fix . (41 auto) + 6 manual. Verified: ezdxf drawing_save_as ignores fmt and returns {"format": fmt} while writing DXF bytes (ezdxf_backend.py:346-353) — make it derive fmt from extension and refuse to mislabel.

I9 — Expand deterministic geometry surface · imp 3 / eff 2 / diff 3

Why now: Highest priority signal. point_from_snap is a concrete base method sourcing from the shared EntityInfo.properties contract (verified at base.py:585), so new snaps are pure math in one place, zero per-backend code, free on COM+ezdxf. Finishes the pre-committed int stub (plan_spec.py:24). Depends on: none. First step: Implement the deferred point_intersection (line-line / line-circle / circle-circle, ref-point disambiguation) and point_tangent (external-point-to-circle — highest leverage). Demote/drop point_offset (trivial vector add). Either cover LWPOLYLINE or document the gap explicitly. Dual-backend tests in tests/test_meta_tools.py.

I12 — Authenticated remote HTTP · imp 2 / eff 1 / diff 2 — QUICK WIN / latent security bug

Why now: Confirmed: no auth= arg and no TokenVerifier anywhere (grep returned nothing). _validate_http_bind promises bearer-token auth but it is never enforced — a remote operator who sets ALLOW_REMOTE_HTTP+MCP_AUTH_TOKEN gets an unauthenticated server that believes it is authenticated. FastMCP 3.0.1 ships StaticTokenVerifier. ~10 lines. First step: Build _auth = StaticTokenVerifier(tokens={settings.mcp_auth_token: {...}}) if settings.mcp_auth_token else None, pass auth=_auth into FastMCP(...) at server.py:153; add tests/test_http_auth.py (401 on bad/missing token, pass on correct).

I12-adjacent quick win — I2 cheap half (render as ImageContent) · imp 3 / eff ~1 / diff 4

Why now: fastmcp Image is already imported/returned by view_screenshot (server.py:1846+); drawing_finalize writes screenshot bytes to disk and discards them. Returning the existing PNG bytes as ImageContent lets the already-multimodal driving agent (Claude) see the render — captures the CADSmith "render is a first-class signal" win with no model bundling, no torch, no [vision] extra. First step: Add return_image/include_render to drawing_finalize (or a thin visual_inspect tool) that wraps existing bytes in Image. Keep the deterministic critique gate pure — the visual signal is advisory, not a CritiqueFocus member. Defer the local-VLM judge and OCR dim round-trip.


Phase A — Own the moat (closed-loop validation)

This is the differentiation core. Sequence: scalar metric → staged unified validator → clarification gate → refiner loop. The metric makes the loop measurable; the unified validator fixes a real disconnect; the refiner acts on it.

I4 — Scalar drawing-score / Invalidity-Ratio · imp 3 / eff 2 / diff 3

Why now: No quantitative metric exists today; drawing_critique returns a list, validator.run returns severity counts only. MUSE/CadBench grade Invalidity-Ratio, not shape. A scalar is the objective function that makes the repair loop measurable and the smoke test regression-trackable. Depends on: sequence after I16 (so the score isn't averaged over dead checks like R4 dim_overlap). First step: Add engineering/scoring.py::score_findings(severity_counts) → score = 100 − (errors·Wₑ + warnings·Wᵥᵥ + info·Wᵢ) clamped, plus invalidity_ratio = errors/max(1, findings). Consume from both validator.py and critique.py; surface in drawing_finalize payload. Document weights as a convention, not ground truth.

I15 — Staged validator (code→geometric→rubric) · imp 5 / eff 3 / diff 5 — RESCOPE

Why now: Strongest hidden find: drawing_finalize (server.py:2311) calls only DrawingValidator.run() and never run_critique — the 6 premium critique focuses are not part of the finalize gate. Unifying validator's 8 steps + critique's 6 focuses into one staged protocol fixes a real architectural disconnect, not just cosmetics. Depends on: the no-op critique fixes (R4/R12/R19 in I16) must land first or the staged scalar lies. Drop R27 (separate SMELL); defer the VLM rubric stage to I2's deferred half. First step: Define StagedReport (stages: code/geometric) with {rule_id, stage, severity, passed, handles, detail} + the I4 scalar; refactor DrawingValidator.run to tag findings with stage and reuse _FOCUS_DISPATCH for the geometric stage; wire both into drawing_finalize. Assert the scalar in scripts/smoke_premium.py.

I3 — ProCAD pre-plan clarification pass · imp 3 / eff 2 / diff 4 — RESCOPE

Why now: ProCAD's one clarification round cut invalidity 14.6%→0.9% — attacks "garbage in" before any geometry, which no downstream critique recovers. drawing_plan is a thin concrete method building a PlanSpec (base.py:553). Depends on: none, but deep material/tolerance NL-extraction should wait for the I5 tolerance model. Rescope: ship cheap robust checks only — PlanSpec has no structured tolerance/material fields, so deep NL-parsing balloons effort. First step: engineering/plan_spec.py::detect_ambiguities(plan) mirroring _FOCUS_DISPATCH: rule 1 = view_count vs intent keywords (section/orthographic/3-view); rule 2 = scale-vs-sheet sanity; rule 3 = ordinate dim_style needs a datum note. Surface advisory clarifications in the server wrapper return.

I1 — Routed iterative Refiner loop · imp 4 / eff 4 / diff 5 — RESCOPE

Why now: The validation-gate-as-repair-loop is the moat no competitor has, directly operationalizing CADSmith Refiner (38x) and FEA-agent (+13.4pp/round). All repair primitives exist on both backends with transaction rollback. Depends on: I15 (unified staged report) and I16 (R4/R19). Rescope to honest v1: auto-repair only the 3 clean deterministic focuses + iso128-snap; route untrimmed_corner and dim_overlap to the LLM as hints, not auto-fixes. First step: engineering/refine.py with a REPAIR_HINTS registry mapping each CritiqueFocus to (a) a deterministic repair coroutine and (b) a fallback hint. Implement the 4 trivial focuses (construction_left→construction_clear, duplicate_entities→entity_delete(handles[1]), layer_color→layer_modify(color=detail.expected), iso128→snap-to-nearest-ISO). Then drawing_refine(max_rounds=4) looping critique→repair→re-critique. Critical blocker (R22): the ezdxf _undo_stack is shared between user undo and transactions with no txn-id isolation (system_status even reports len(_undo_stack) as transaction_depth) — add a dedicated txn list/guard before the rollback cycle, or it corrupts undo state.


Phase B — Production ISO parity

Make output actually manufacturable. ISO 129 tolerances are the single biggest functional gap; GD&T is the highest-differentiation breadth item.

I6 — GD&T primitive (ISO 1101 / Y14.5) · imp 5 / eff 3 / diff 5 — KEEP (impact↑)

Why now: Maximal differentiation — 2D GD&T is absent from every surveyed text-to-CAD product and every competing MCP. Pure composition over existing primitives (the titleblock_apply_iso_a3 pattern, server.py:2248) — no ABC/dual-backend additions. Impact raised to 5: bundle a thin tolerance-text option and one GD&T critique check (datum letter in an FCF must have a matching datum feature) to convert it from "draws a symbol" to "validates a standard." Depends on: none (composes cleanly). First step: engineering/gdt.py with the 14 ISO 1101 characteristic glyphs + Ⓜ/Ⓛ/Ⓢ modifiers (unicode/MTEXT codes), then draw_feature_control_frame(...) composing the frame from entity_create_line + entity_create_text on DIM layer. Register one gd_frame @mcp.tool. Design constraint: avoid ezdxf's native Tolerance/FCF entity (renders blank through matplotlib) — compose from lines+text. Smoke-test glyph rendering via screenshot.

I5 — ISO 129 tolerances on dimensions · imp 4 / eff 4 / diff 3 — RESCOPE

Why now: A drawing without ±/limit/fit callouts is not manufacturable — the dossier's "biggest functional gap." Empirically verified ezdxf 1.4.3 renders sym/limit/text overrides via override dict. Depends on: R19 fix (I16) is a soft prerequisite — aligned/angular dims are currently broken, so don't bolt tolerances onto them yet. Rescope: ezdxf-first, CI-testable subset; defer aligned/angular tolerances behind R19; COM tolerance rendering best-effort/untested until I19. First step: Add tol_upper/tol_lower/tol_mode/fit/text_override to dimension_linear/radius/diameter in base.py; implement in ezdxf via override dict (sym/limit → DIMTOL/DIMLIM + DIMTP/DIMTM; fit → engineering/iso286_fits.py nominal-band×grade table — the main authored-data cost). Test asserts the DIMENSION carries DIMTP/DIMTM and reads e.g. 20 +0.021/0.

I8 — ISO-129 DimStyle + MLEADER · imp 3 / eff 3 / diff 2 — as scored

Why now: Every dim today inherits the default "Standard" style (no DIMTXT/DIMASZ/precision). Natural sibling to drawing_apply_iso_layers. Low differentiation (table-stakes) — sequence after I5/I6. Depends on: I5 (share the dimstyle plumbing). First step: engineering/layers.py::apply_iso_dimstyle(backend) → doc.dimstyles.new("ISO-25", …); thread dimstyle="ISO-25" through ezdxf dim methods; wire into drawing_apply_iso_layers. Defer COM DimStyles.Add and the MLEADER tool to follow-ups.

I7 — Cheap half: fmt-from-extension + true ISO-128 lineweights · imp 3 / eff 3 / diff 3 — RESCOPE

Why now: Two isolated wins. (1) Fake-DWG is ezdxf-only (COM already writes real DWG via SaveAs(path,12)); (2) ISO-128 true lineweights via ezdxf Configuration (lineweight_policy) on the render path is the genuinely differentiating piece and feeds the render/critique signal. Defer the from-scratch paperspace/viewport/Plot pipeline (multi-day, CI-unverifiable, odafc needs the ODA binary — confirmed not on PATH) to the backlog. Depends on: none. First step: Wire Configuration(lineweight_policy=…) into the PDF/screenshot Frontend; unify the save_as fmt default and fail loud when ezdxf can't write real DWG (use odafc.export_dwg if present, else raise — never mislabel).


Phase C — Breadth & determinism extras

Catch-up capability and convenience. Lower priority signals; do not displace the moat work.

I14 — NL→primitive grounding for trim/fillet/chamfer · imp 3 / eff 3 / diff 3 — RESCOPE

Why now: Repair-side companion to the untrimmed_corner detector; attacks FutureCAD primitive-grounding + GeoGramBench. Rescope: "which corner" is net-new work — analysis_bounding_box returns only a global bbox, so corner→handle mapping re-runs O(n²) endpoint clustering. Ship ezdxf path with bbox-quadrant corner naming that errors with candidates when ambiguous instead of guessing; defer COM until I19 lands COM tests. Depends on: I9 (intersection math), I19 (COM verification).

I18 — Real drawing repair (recover + Auditor.fix) · imp 2 / eff 2 / diff 1 — KEEP

Why now: Confirmed honesty bug: drawing_audit returns only auditor.errors, ignores auditor.fixes, while the docstring claims it "fixes errors"; drawing_open uses bare readfile with no recover.readfile fallback. Near-free robustness win. First step: Return errors+fixes+fixed_count and _mark_dirty() when fixes applied; wrap readfile with a recover.readfile fallback.

I19 — COM test harness (mocked) + scoped coverage gate · imp 2 / eff 2 / diff 1 — KEEP

Why now: R6 confirmed — zero tests touch ComBackend; CI is ubuntu-only so 1651 lines never run. normalize_lineweight (the P0 anti-truncation fix, pure/OS-agnostic) has no regression guard. Unblocks COM verification for I14/I5/I16. Depends on: none; should land before/with I16's COM-only items (N1/R5). First step: tests/test_com_backend.py with a sys.modules fake-COM shim; parametrized normalize_lineweight table tests + _safe_send_command CMDACTIVE/timeout/var-restore tests. Scope --cov-fail-under to backends.base + tested modules, not repo-wide.

Backlog / Later (!)

  • I17 — Fast selection: do the ezdxf-only half (query()/groupby() one-liners, zero-risk, headless-testable); defer the COM SelectAll rewrite (CI-unverifiable, finicky compound filters). Not a moat-widener.
  • I13 — Standard-parts library: ship the cheap library_insert wrapper + a few parametric ISO bolt block defs; the real cost (ISO-correct fastener + P&ID symbol authoring) is large ongoing work. Defer dynamic-block driving (COM-only, breaks dual-backend parity).
  • I11 — Parametric editability: ship single-entity entity_edit_geometry (set radius/endpoints) first; true re-drive needs a constraint graph the architecture has no scaffold for. Catch-up, not moat.
  • I10 — 3D solids: do the near-zero-effort honesty fix now (correct the false "all_entity_types" capability at com_backend.py:1444). Defer the COM-only solids tier; ezdxf has no ACIS kernel so it breaks dual-backend parity, and all cited research targets 2D fidelity.
  • I7 expensive half — paperspace/viewport/Plot pipeline + bundled-VLM judge.

  1. I16 (Correctness bundle) + I19 (COM harness) together. Highest evidence strength; everything in Phase A is built on signals that are currently silent no-ops (R4 dim_overlap, R19 broken dims, fake DWG). The COM harness lands with it so the COM-only fixes (N1, R5) and normalize_lineweight are actually guarded. Foundation for the entire moat.
  2. I9 (Deterministic geometry surface). Top priority signal, zero risk, one-place pure math. Directly converts the GeoGramBench <50% program-to-geometry failure into deterministic tool calls — the cheapest moat-adjacent win available.
  3. I12 (HTTP auth) + I2-cheap (render as ImageContent). Two quick wins. I12 closes a real false-security gap (~10 lines, verified no auth wiring exists). I2-cheap captures the CADSmith "render is a first-class signal" win for ~1 day with no model bundling, by returning the already-produced PNG to the multimodal driver.
  4. I4 → I15 (Scalar metric → unified staged validator). I4 gives the objective function; I15 fixes the confirmed disconnect where drawing_finalize never runs the premium critique focuses. Together they make the moat measurable, regression-testable, and actually enforced at finalize — the prerequisite for the refiner.
  5. I6 (GD&T primitive). Highest-differentiation feature in the dossier, KEEP at effort 3, impact raised to 5 by bundling a tolerance-text option and one datum-consistency critique. Pure composition, no backend surface change — the single biggest "no competitor has this" capability.

Sequencing rationale: items 1–4 are explicitly ordered by depends_on (signals must be honest before the scalar averages them; the scalar must exist before the refiner can optimize it; R22 txn-isolation gates the refiner's rollback cycle). Item 5 is parallelizable — it composes cleanly and shares no dependency with the closed-loop track, so it can run alongside Phase A.


Security note — unverified DIAGNOSTICS items (R7/R8)

The diagnostics flagged two security items that the security-review subagents could not verify: R7 (input-sanitizer wiring — whether system_run_command / system_run_lisp user strings are actually routed through a sanitizer before reaching SendCommand) and R8 (absence of adversarial sanitizer tests). These sit on the live-COM command path, which is the highest-blast-radius surface in the repo (arbitrary AutoLISP/command execution) and is exactly the path with zero COM test coverage (R6).

Recommendation — add a defensive hardening work item (slot in Sprint 1, alongside I19): (a) trace system_run_command/system_run_lisp end-to-end and confirm every user-supplied string passes through an explicit allow/deny sanitizer before _safe_send_command; if no sanitizer is wired, add one (reject embedded newlines, \x03/ESC injection, nested (command …) escapes, and SCRIPT/SHELL/START command words). (b) Add adversarial unit tests on the mocked-COM harness from I19 asserting that known injection payloads are rejected before reaching SendCommand. This is cheap insurance on the one surface where a prompt-injected agent could execute arbitrary OS-level actions via SHELL/START, and it composes directly with the I19 fake-COM fixture.