Umutcan Edizsalan
0c7f64ceb6
N1: COM hatch non-associative (AddHatch associativity=False) N2: ezdxf refuses DWG saves with clear error; server.py derives fmt from extension R19: dimension_aligned dist→distance; dimension_angular correct add_angular_dim_2l args (base/line1/line2/location) R9: polar array 360° no-duplicate (divisor=n when fill_angle%360<ε) R13: entity_offset side_x/side_y honored in ezdxf via cross-product (LINE) and distance-to-center (CIRCLE) R18: drawing_new bootstrap failure surfaces status="degraded" instead of swallowing silently Dead code: engineering/section.py, gear.generate_tooth_profile, CommandResult, _ACI_NAMES, set_layer_active, ensure_linetypes, ComBackend._batch_mode instance attr, engineering/__init__ build-seam comment Ruff: 38 auto-fixed + 5 manual → codebase is now ruff-clean; CI lint is now blocking 184 tests pass.
147 lines
20 KiB
Markdown
147 lines
20 KiB
Markdown
<!-- Generated by the IDEAS/roadmap workflow (wf_02a3262e-ddd), 2026-06-18. 19 ideas scored, 9 stress-tested. Pairs with docs/analysis/01-DIAGNOSTICS.md. -->
|
||
|
||
# AutoCAD MCP Pro — Evidence-Based Roadmap
|
||
|
||
> Mission: become the best AutoCAD MCP for LLM-driven drawing. The durable moat is the **standards-validation gate** (no competing CAD MCP — build123d / FreeCAD / Blender / Rhino / Onshape — ships any design-rule validation). Everything below is sequenced to widen that moat or to remove the silent no-ops that make it lie.
|
||
|
||
Priority signal = `(impact + differentiation + evidence_strength) / effort`, computed on **post-challenge adjusted** scores, then adjusted by judgment (quick wins and dependency order). DEFER → Backlog; RESCOPE → adjusted scope used; KEEP → as scored.
|
||
|
||
---
|
||
|
||
## Scoring table (all ideas, adjusted, sorted by priority signal)
|
||
|
||
| ID | Title | Imp | Eff | Diff | Ev | Signal | Verdict | Area |
|
||
|----|-------|-----|-----|------|----|--------|---------|------|
|
||
| **I9** | Expand deterministic geometry surface | 3 | 2 | 3 | 5 | **5.5** | KEEP→rescoped* | determinism |
|
||
| **I3** | ProCAD pre-plan clarification pass | 3 | 2 | 4 | 4 | **5.5** | RESCOPE | closed-loop |
|
||
| **I16** | Correctness + hygiene bundle (DIAGNOSTICS) | 3 | 2 | 2 | 5 | **5.0** | KEEP | hygiene |
|
||
| **I6** | GD&T primitive (ISO 1101 / Y14.5) | 5 | 3 | 5 | 3 | **4.3** | KEEP (impact↑) | iso-production |
|
||
| **I14** | NL→primitive grounding (trim/fillet/chamfer) | 3 | 3 | 3 | 4 | **3.3** | RESCOPE | determinism |
|
||
| **I15** | Staged validator (code→geometric→rubric) | 5 | 3 | 5 | 5 | **5.0** | RESCOPE | metrics |
|
||
| **I12** | Authenticated remote HTTP | 2 | 1 | 2 | 2 | **6.0** | KEEP (quick win) | infra |
|
||
| **I4** | Scalar drawing-score / Invalidity-Ratio | 3 | 2 | 3 | 4 | **5.0** | KEEP | metrics |
|
||
| **I2** | Visual critique / render as ImageContent | 3 | 2 | 4 | 5 | **6.0** | RESCOPE (cheap half) | closed-loop |
|
||
| **I18** | Real drawing repair (recover + Auditor.fix) | 2 | 2 | 1 | 3 | **3.0** | KEEP | robustness |
|
||
| **I1** | Routed iterative Refiner loop | 4 | 4 | 5 | 5 | **3.5** | RESCOPE | closed-loop |
|
||
| **I5** | ISO 129 tolerances on dimensions | 4 | 4 | 3 | 5 | **3.0** | RESCOPE | iso-production |
|
||
| **I8** | ISO-129 DimStyle + MLEADER | 3 | 3 | 2 | 3 | **2.7** | as scored | iso-production |
|
||
| **I19** | COM test harness (mocked) + cov gate | 2 | 2 | 1 | 3 | **3.0** | KEEP | hygiene |
|
||
| **I7** | Real plotted sheets + real DWG | 3 | 3 | 3 | 3 | **3.0** | RESCOPE (cheap half) | iso-production |
|
||
| **I17** | Fast selection via native query | 2 | 3 | 1 | 2 | **1.7** | ezdxf half only | perf |
|
||
| **I13** | Standard-parts / symbol library | 3 | 4 | 3 | 2 | **2.0** | RESCOPE | breadth |
|
||
| **I11** | Parametric editability / conv. edit | 3 | 4 | 3 | 2 | **2.0** | as scored | breadth |
|
||
| **I10** | 3D solids tier (or fix false claim) | 2 | 4 | 2 | 2 | **1.5** | split | breadth |
|
||
|
||
\* I9/I3 share the top signal; I9 edges first because it is zero-risk and directly converts the GeoGramBench failure mode (<50% program-to-geometry) into deterministic tool calls.
|
||
|
||
---
|
||
|
||
## Sprint 1 — Quick wins & correctness
|
||
|
||
Foundation: the closed-loop moat only pays off if the validator/critique signals are not silent no-ops. Land the cheap, CI-verifiable correctness fixes and the highest-signal determinism wins first.
|
||
|
||
### I16 — Correctness + hygiene bundle (DIAGNOSTICS fixes) · imp 3 / eff 2 / diff 2
|
||
**Why now:** Every premium-workflow signal that the moat depends on is currently degraded by confirmed defects (fake DWG, broken aligned/angular dims, polar-array dup, silent bootstrap). Iteration evidence (Self-Improving CAD +13.4pp/round) only pays off if signals aren't dead. **Depends on:** land alongside I19 (COM-only N1/R5 need a harness).
|
||
**First step:** One PR for the CI-verifiable wins: fix R19 (`ezdxf_backend.py:696` `dist`→positional `distance`; `:711` `add_angular_dim_2l(base=…, line1=…, line2=…, location=(tx,ty))`), flip `tests/test_dimensions.py:16,23` to `xfail(strict=True)`, fix R9 divisor (`:947`), R12 `location=(tx,ty)` (`:715`), R13 offset side via cross-product sign; then `ruff check --fix .` (41 auto) + 6 manual. Verified: ezdxf `drawing_save_as` ignores `fmt` and returns `{"format": fmt}` while writing DXF bytes (`ezdxf_backend.py:346-353`) — make it derive fmt from extension and refuse to mislabel.
|
||
|
||
### I9 — Expand deterministic geometry surface · imp 3 / eff 2 / diff 3
|
||
**Why now:** Highest priority signal. `point_from_snap` is a **concrete** base method sourcing from the shared `EntityInfo.properties` contract (verified at `base.py:585`), so new snaps are pure math in one place, zero per-backend code, free on COM+ezdxf. Finishes the pre-committed `int` stub (`plan_spec.py:24`). **Depends on:** none.
|
||
**First step:** Implement the deferred `point_intersection` (line-line / line-circle / circle-circle, ref-point disambiguation) and `point_tangent` (external-point-to-circle — highest leverage). Demote/drop `point_offset` (trivial vector add). Either cover LWPOLYLINE or document the gap explicitly. Dual-backend tests in `tests/test_meta_tools.py`.
|
||
|
||
### I12 — Authenticated remote HTTP · imp 2 / eff 1 / diff 2 — **QUICK WIN / latent security bug**
|
||
**Why now:** Confirmed: no `auth=` arg and no `TokenVerifier` anywhere (grep returned nothing). `_validate_http_bind` promises bearer-token auth but it is **never enforced** — a remote operator who sets `ALLOW_REMOTE_HTTP`+`MCP_AUTH_TOKEN` gets an unauthenticated server that believes it is authenticated. FastMCP 3.0.1 ships `StaticTokenVerifier`. ~10 lines.
|
||
**First step:** Build `_auth = StaticTokenVerifier(tokens={settings.mcp_auth_token: {...}}) if settings.mcp_auth_token else None`, pass `auth=_auth` into `FastMCP(...)` at `server.py:153`; add `tests/test_http_auth.py` (401 on bad/missing token, pass on correct).
|
||
|
||
### I12-adjacent quick win — I2 cheap half (render as ImageContent) · imp 3 / eff ~1 / diff 4
|
||
**Why now:** `fastmcp Image` is already imported/returned by `view_screenshot` (`server.py:1846+`); `drawing_finalize` writes screenshot bytes to disk and discards them. Returning the existing PNG bytes as `ImageContent` lets the already-multimodal driving agent (Claude) see the render — captures the CADSmith "render is a first-class signal" win with **no model bundling, no torch, no [vision] extra**.
|
||
**First step:** Add `return_image`/`include_render` to `drawing_finalize` (or a thin `visual_inspect` tool) that wraps existing bytes in `Image`. Keep the deterministic critique gate pure — the visual signal is advisory, **not** a `CritiqueFocus` member. Defer the local-VLM judge and OCR dim round-trip.
|
||
|
||
---
|
||
|
||
## Phase A — Own the moat (closed-loop validation)
|
||
|
||
This is the differentiation core. Sequence: scalar metric → staged unified validator → clarification gate → refiner loop. The metric makes the loop measurable; the unified validator fixes a real disconnect; the refiner acts on it.
|
||
|
||
### I4 — Scalar drawing-score / Invalidity-Ratio · imp 3 / eff 2 / diff 3
|
||
**Why now:** No quantitative metric exists today; `drawing_critique` returns a list, `validator.run` returns severity counts only. MUSE/CadBench grade Invalidity-Ratio, not shape. A scalar is the objective function that makes the repair loop measurable and the smoke test regression-trackable. **Depends on:** sequence after I16 (so the score isn't averaged over dead checks like R4 dim_overlap).
|
||
**First step:** Add `engineering/scoring.py::score_findings(severity_counts)` → `score = 100 − (errors·Wₑ + warnings·Wᵥᵥ + info·Wᵢ)` clamped, plus `invalidity_ratio = errors/max(1, findings)`. Consume from both `validator.py` and `critique.py`; surface in `drawing_finalize` payload. Document weights as a convention, not ground truth.
|
||
|
||
### I15 — Staged validator (code→geometric→rubric) · imp 5 / eff 3 / diff 5 — RESCOPE
|
||
**Why now:** Strongest hidden find: `drawing_finalize` (`server.py:2311`) calls **only** `DrawingValidator.run()` and **never** `run_critique` — the 6 premium critique focuses are not part of the finalize gate. Unifying validator's 8 steps + critique's 6 focuses into one staged protocol fixes a real architectural disconnect, not just cosmetics. **Depends on:** the no-op critique fixes (R4/R12/R19 in I16) must land first or the staged scalar lies. Drop R27 (separate SMELL); defer the VLM rubric stage to I2's deferred half.
|
||
**First step:** Define `StagedReport` (stages: code/geometric) with `{rule_id, stage, severity, passed, handles, detail}` + the I4 scalar; refactor `DrawingValidator.run` to tag findings with stage and reuse `_FOCUS_DISPATCH` for the geometric stage; wire **both** into `drawing_finalize`. Assert the scalar in `scripts/smoke_premium.py`.
|
||
|
||
### I3 — ProCAD pre-plan clarification pass · imp 3 / eff 2 / diff 4 — RESCOPE
|
||
**Why now:** ProCAD's one clarification round cut invalidity 14.6%→0.9% — attacks "garbage in" before any geometry, which no downstream critique recovers. `drawing_plan` is a thin concrete method building a `PlanSpec` (`base.py:553`). **Depends on:** none, but deep material/tolerance NL-extraction should wait for the I5 tolerance model. **Rescope:** ship cheap robust checks only — PlanSpec has no structured tolerance/material fields, so deep NL-parsing balloons effort.
|
||
**First step:** `engineering/plan_spec.py::detect_ambiguities(plan)` mirroring `_FOCUS_DISPATCH`: rule 1 = `view_count` vs intent keywords (section/orthographic/3-view); rule 2 = scale-vs-sheet sanity; rule 3 = ordinate dim_style needs a datum note. Surface advisory `clarifications` in the server wrapper return.
|
||
|
||
### I1 — Routed iterative Refiner loop · imp 4 / eff 4 / diff 5 — RESCOPE
|
||
**Why now:** The validation-gate-as-repair-loop is the moat no competitor has, directly operationalizing CADSmith Refiner (38x) and FEA-agent (+13.4pp/round). All repair primitives exist on both backends with transaction rollback. **Depends on:** I15 (unified staged report) and I16 (R4/R19). **Rescope to honest v1:** auto-repair only the 3 clean deterministic focuses + iso128-snap; route `untrimmed_corner` and `dim_overlap` to the LLM as **hints**, not auto-fixes.
|
||
**First step:** `engineering/refine.py` with a `REPAIR_HINTS` registry mapping each `CritiqueFocus` to (a) a deterministic repair coroutine and (b) a fallback hint. Implement the 4 trivial focuses (`construction_left`→`construction_clear`, `duplicate_entities`→`entity_delete(handles[1])`, `layer_color`→`layer_modify(color=detail.expected)`, `iso128`→snap-to-nearest-ISO). Then `drawing_refine(max_rounds=4)` looping critique→repair→re-critique. **Critical blocker (R22):** the ezdxf `_undo_stack` is shared between user undo and transactions with no txn-id isolation (`system_status` even reports `len(_undo_stack)` as transaction_depth) — add a dedicated txn list/guard before the rollback cycle, or it corrupts undo state.
|
||
|
||
---
|
||
|
||
## Phase B — Production ISO parity
|
||
|
||
Make output actually manufacturable. ISO 129 tolerances are the single biggest functional gap; GD&T is the highest-differentiation breadth item.
|
||
|
||
### I6 — GD&T primitive (ISO 1101 / Y14.5) · imp 5 / eff 3 / diff 5 — KEEP (impact↑)
|
||
**Why now:** Maximal differentiation — 2D GD&T is absent from every surveyed text-to-CAD product and every competing MCP. Pure composition over existing primitives (the `titleblock_apply_iso_a3` pattern, `server.py:2248`) — no ABC/dual-backend additions. Impact raised to 5: bundle a thin tolerance-text option and one GD&T critique check (datum letter in an FCF must have a matching datum feature) to convert it from "draws a symbol" to "validates a standard." **Depends on:** none (composes cleanly).
|
||
**First step:** `engineering/gdt.py` with the 14 ISO 1101 characteristic glyphs + Ⓜ/Ⓛ/Ⓢ modifiers (unicode/MTEXT codes), then `draw_feature_control_frame(...)` composing the frame from `entity_create_line` + `entity_create_text` on DIM layer. Register one `gd_frame` `@mcp.tool`. **Design constraint:** avoid ezdxf's native Tolerance/FCF entity (renders blank through matplotlib) — compose from lines+text. Smoke-test glyph rendering via screenshot.
|
||
|
||
### I5 — ISO 129 tolerances on dimensions · imp 4 / eff 4 / diff 3 — RESCOPE
|
||
**Why now:** A drawing without ±/limit/fit callouts is not manufacturable — the dossier's "biggest functional gap." Empirically verified ezdxf 1.4.3 renders sym/limit/text overrides via override dict. **Depends on:** R19 fix (I16) is a soft prerequisite — aligned/angular dims are currently broken, so don't bolt tolerances onto them yet. **Rescope:** ezdxf-first, CI-testable subset; defer aligned/angular tolerances behind R19; COM tolerance rendering best-effort/untested until I19.
|
||
**First step:** Add `tol_upper/tol_lower/tol_mode/fit/text_override` to `dimension_linear/radius/diameter` in `base.py`; implement in ezdxf via override dict (sym/limit → DIMTOL/DIMLIM + DIMTP/DIMTM; fit → `engineering/iso286_fits.py` nominal-band×grade table — the main authored-data cost). Test asserts the DIMENSION carries DIMTP/DIMTM and reads e.g. `20 +0.021/0`.
|
||
|
||
### I8 — ISO-129 DimStyle + MLEADER · imp 3 / eff 3 / diff 2 — as scored
|
||
**Why now:** Every dim today inherits the default "Standard" style (no DIMTXT/DIMASZ/precision). Natural sibling to `drawing_apply_iso_layers`. Low differentiation (table-stakes) — sequence after I5/I6. **Depends on:** I5 (share the dimstyle plumbing).
|
||
**First step:** `engineering/layers.py::apply_iso_dimstyle(backend)` → `doc.dimstyles.new("ISO-25", …)`; thread `dimstyle="ISO-25"` through ezdxf dim methods; wire into `drawing_apply_iso_layers`. Defer COM `DimStyles.Add` and the MLEADER tool to follow-ups.
|
||
|
||
### I7 — Cheap half: fmt-from-extension + true ISO-128 lineweights · imp 3 / eff 3 / diff 3 — RESCOPE
|
||
**Why now:** Two isolated wins. (1) Fake-DWG is ezdxf-only (COM already writes real DWG via `SaveAs(path,12)`); (2) ISO-128 true lineweights via ezdxf `Configuration` (`lineweight_policy`) on the render path is the genuinely differentiating piece and feeds the render/critique signal. **Defer** the from-scratch paperspace/viewport/Plot pipeline (multi-day, CI-unverifiable, `odafc` needs the ODA binary — confirmed not on PATH) to the backlog. **Depends on:** none.
|
||
**First step:** Wire `Configuration(lineweight_policy=…)` into the PDF/screenshot Frontend; unify the save_as fmt default and fail loud when ezdxf can't write real DWG (use `odafc.export_dwg` if present, else raise — never mislabel).
|
||
|
||
---
|
||
|
||
## Phase C — Breadth & determinism extras
|
||
|
||
Catch-up capability and convenience. Lower priority signals; do not displace the moat work.
|
||
|
||
### I14 — NL→primitive grounding for trim/fillet/chamfer · imp 3 / eff 3 / diff 3 — RESCOPE
|
||
**Why now:** Repair-side companion to the `untrimmed_corner` detector; attacks FutureCAD primitive-grounding + GeoGramBench. **Rescope:** "which corner" is net-new work — `analysis_bounding_box` returns only a global bbox, so corner→handle mapping re-runs O(n²) endpoint clustering. Ship ezdxf path with bbox-quadrant corner naming that **errors with candidates when ambiguous** instead of guessing; defer COM until I19 lands COM tests. **Depends on:** I9 (intersection math), I19 (COM verification).
|
||
|
||
### I18 — Real drawing repair (recover + Auditor.fix) · imp 2 / eff 2 / diff 1 — KEEP
|
||
**Why now:** Confirmed honesty bug: `drawing_audit` returns only `auditor.errors`, ignores `auditor.fixes`, while the docstring claims it "fixes errors"; `drawing_open` uses bare `readfile` with no `recover.readfile` fallback. Near-free robustness win.
|
||
**First step:** Return `errors`+`fixes`+`fixed_count` and `_mark_dirty()` when fixes applied; wrap `readfile` with a `recover.readfile` fallback.
|
||
|
||
### I19 — COM test harness (mocked) + scoped coverage gate · imp 2 / eff 2 / diff 1 — KEEP
|
||
**Why now:** R6 confirmed — zero tests touch `ComBackend`; CI is ubuntu-only so 1651 lines never run. `normalize_lineweight` (the P0 anti-truncation fix, pure/OS-agnostic) has no regression guard. Unblocks COM verification for I14/I5/I16. **Depends on:** none; should land **before/with** I16's COM-only items (N1/R5).
|
||
**First step:** `tests/test_com_backend.py` with a `sys.modules` fake-COM shim; parametrized `normalize_lineweight` table tests + `_safe_send_command` CMDACTIVE/timeout/var-restore tests. Scope `--cov-fail-under` to `backends.base` + tested modules, not repo-wide.
|
||
|
||
### Backlog / Later (!)
|
||
|
||
- **I17** — Fast selection: do the **ezdxf-only** half (`query()`/`groupby()` one-liners, zero-risk, headless-testable); defer the COM `SelectAll` rewrite (CI-unverifiable, finicky compound filters). Not a moat-widener.
|
||
- **I13** — Standard-parts library: ship the cheap `library_insert` wrapper + a few parametric ISO bolt block defs; the real cost (ISO-correct fastener + P&ID symbol authoring) is large ongoing work. Defer dynamic-block driving (COM-only, breaks dual-backend parity).
|
||
- **I11** — Parametric editability: ship single-entity `entity_edit_geometry` (set radius/endpoints) first; true re-drive needs a constraint graph the architecture has no scaffold for. Catch-up, not moat.
|
||
- **I10** — 3D solids: **do the near-zero-effort honesty fix now** (correct the false `"all_entity_types"` capability at `com_backend.py:1444`). Defer the COM-only solids tier; ezdxf has no ACIS kernel so it breaks dual-backend parity, and all cited research targets 2D fidelity.
|
||
- **I7 expensive half** — paperspace/viewport/Plot pipeline + bundled-VLM judge.
|
||
|
||
---
|
||
|
||
## Recommended next 5 work items
|
||
|
||
1. **I16 (Correctness bundle) + I19 (COM harness) together.** Highest evidence strength; everything in Phase A is built on signals that are currently silent no-ops (R4 dim_overlap, R19 broken dims, fake DWG). The COM harness lands with it so the COM-only fixes (N1, R5) and `normalize_lineweight` are actually guarded. Foundation for the entire moat.
|
||
2. **I9 (Deterministic geometry surface).** Top priority signal, zero risk, one-place pure math. Directly converts the GeoGramBench <50% program-to-geometry failure into deterministic tool calls — the cheapest moat-adjacent win available.
|
||
3. **I12 (HTTP auth) + I2-cheap (render as ImageContent).** Two quick wins. I12 closes a real false-security gap (~10 lines, verified no auth wiring exists). I2-cheap captures the CADSmith "render is a first-class signal" win for ~1 day with no model bundling, by returning the already-produced PNG to the multimodal driver.
|
||
4. **I4 → I15 (Scalar metric → unified staged validator).** I4 gives the objective function; I15 fixes the confirmed disconnect where `drawing_finalize` never runs the premium critique focuses. Together they make the moat measurable, regression-testable, and actually enforced at finalize — the prerequisite for the refiner.
|
||
5. **I6 (GD&T primitive).** Highest-differentiation feature in the dossier, KEEP at effort 3, impact raised to 5 by bundling a tolerance-text option and one datum-consistency critique. Pure composition, no backend surface change — the single biggest "no competitor has this" capability.
|
||
|
||
Sequencing rationale: items 1–4 are explicitly ordered by `depends_on` (signals must be honest before the scalar averages them; the scalar must exist before the refiner can optimize it; R22 txn-isolation gates the refiner's rollback cycle). Item 5 is parallelizable — it composes cleanly and shares no dependency with the closed-loop track, so it can run alongside Phase A.
|
||
|
||
---
|
||
|
||
## Security note — unverified DIAGNOSTICS items (R7/R8)
|
||
|
||
The diagnostics flagged two security items that the security-review subagents **could not verify**: **R7** (input-sanitizer wiring — whether `system_run_command` / `system_run_lisp` user strings are actually routed through a sanitizer before reaching `SendCommand`) and **R8** (absence of adversarial sanitizer tests). These sit on the live-COM command path, which is the highest-blast-radius surface in the repo (arbitrary AutoLISP/command execution) and is exactly the path with **zero COM test coverage** (R6).
|
||
|
||
**Recommendation — add a defensive hardening work item (slot in Sprint 1, alongside I19):** (a) trace `system_run_command`/`system_run_lisp` end-to-end and confirm every user-supplied string passes through an explicit allow/deny sanitizer before `_safe_send_command`; if no sanitizer is wired, add one (reject embedded newlines, `\x03`/ESC injection, nested `(command …)` escapes, and `SCRIPT`/`SHELL`/`START` command words). (b) Add adversarial unit tests on the mocked-COM harness from I19 asserting that known injection payloads are rejected before reaching `SendCommand`. This is cheap insurance on the one surface where a prompt-injected agent could execute arbitrary OS-level actions via `SHELL`/`START`, and it composes directly with the I19 fake-COM fixture.
|