diff --git a/CHANGELOG.md b/CHANGELOG.md index a37783b..21f46a6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -39,6 +39,16 @@ new `arch` pack, three `arch_*` critique focuses, and English or Turkish labels. No new contract member. Spec: `docs/superpowers/specs/2026-09-23-v1.6-arch-design.md`. +Track H of 1.6: understanding drawings somebody else made. Six tools in two +new sections — understanding and QA (26: a one-call report, a scale check +between a P&ID and its layout, a revision diff, topology checks) and plant +takeoffs (27: pipe and cable quantities whose topology comes from the P&ID +and whose lengths come from the layout) — one DXF snapshot per drawing (a +live document through `Document.Export`, never renamed), a vocabulary in five +languages, three `topo_*` critique focuses, a new `plant` pack and the +`office` extra. Read-only by default; nothing invented. No new contract +member. Spec: `docs/superpowers/specs/2026-09-24-v1.6-understand-design.md`. + ### Added - **Styles — 10 tools, pack `core`.** `dimstyle_list/create/modify/set_current`, @@ -139,6 +149,33 @@ labels. No new contract member. Spec: 2026 (`25.1s (LMS Tech)`): the example plan drawn in one transaction, both rooms read back within 0.1 % of their labels and of the hand-computed net floor, and `drawing_critique(focus=None)` returned 0 issues (record below). +- **Understanding foreign drawings — 4 tools, pack `core`; plant takeoffs — 2 + tools, pack `plant`.** `drawing_understand`, `drawing_scale_check`, + `drawing_diff`, `drawing_topology_check`, `pipe_takeoff`, `cable_takeoff`. + Every one reads a snapshot of frozen WCS records + (`engineering/understand/snapshot.py`); labels are read by one parser + (`%%c`, `\U+2205`, the U+2300 sign, `ø` and MTEXT format runs decoded, + Cyrillic look-alike tags folded to Latin) and layer names and texts by one + EN / TR / RU / NL / DE vocabulary. The takeoff workbook is written as CSV + always and as XLSX with the new `office` extra (`openpyxl`, also in `full`). + Lean gains `drawing_understand` and reaches 65. +- **Benchmark evidence for track H.** `benchmarks/tasks_v7.py` (21 tasks, the + default matrix; v6 to v2 stay addressable) adds `takeoff_roundtrip` and + `understand_foreign`, both on the synthetic plant pair of + `tests/fixtures/plant_pair.py`, whose truth is computed from the positions + it placed; each gate is shown to fail on a broken input. The correctness + suite grows to 48 with `scale_check_detects_schematic`, + `pipe_takeoff_rmst_exact`, `cable_takeoff_roundup`, + `diff_detects_known_edits` and `topology_known_defects`. A/B against + `v1.5.1`: **26 / 48 → 48 / 48, twenty-two `miss → pass`, zero regressed**. +- **`scripts/field_test_takeoff.py`** runs both takeoffs on a user's own P&ID + and layout and writes the report next to them; it refuses an output folder + inside the repository and prints no quantity unless `--show` is given. +- **Live COM smoke.** `scripts/smoke_understand_com.py` executed once on + AutoCAD 2026 (`25.1s (LMS Tech)`): the live snapshots matched + their files type for type in model space, the DWG route read the same + P&ID, both takeoffs matched the truth and the diff reported exactly the + one entity moved (record below). - **P&ID — 9 tools, pack `pid`.** - `pid_symbol_list` / `pid_symbol_insert`: a 43-symbol ISO 10628-2 / @@ -318,8 +355,40 @@ labels. No new contract member. Spec: `KeyError: 'start'` recorded above as a busy-refused point read (not re-measured). +- **Track H, found after the groups landed.** Six defects in the readers, + each with a test that failed before its fix: + - A truncated DXF (an interrupted save, a partial copy) made ezdxf raise + `StopIteration`, which `asyncio.to_thread` cannot hand back to an + awaiting caller, so every track H tool reading one would have hung. The + snapshot now refuses it as "not a readable DXF file". + - `take_snapshot` of a DWG the operator already had open opened it again + read-only and closed "it" unsaved - possibly their copy, with their + unsaved work. An open document is now looked up first, exported as it + stands and left open; only a document the snapshot opened is closed. + - `drawing_diff`: a LINE drawn back to front and moved was reported as + reshaped; an INSERT carrying one attribute tag twice could lose an edit + of the first value (a repeated tag is now compared as the list of its + values); the cluster list is capped like the other lists + (`truncated["clusters"]`, the markup still clouds every cluster); and a + whole-drawing move keeps only the 32 nearest candidates per entity. + - A line number was read as an equipment tag: `100-P-001` yielded the pump + `P-001`. A token that is the tail of a word beginning with a digit is no + longer a tag. + - The pipe network merged ends and dropped stubs within `tol = 1.0` + drawing units - a metre on a metre P&ID. The tolerance now defaults to + 1 mm in the P&ID's inferred unit, and a dropped stub is counted and said. + - A wiring callout was counted as a place its panel is written: + `Wiring to CP-2` made `CP-2` ambiguous in `drawing_scale_check` and + `drawing_understand`, and `Wiring to CP1` a phantom tag `CP1`. On the + synthetic pair the check matched 10 tags and 45 pairs (13.3 % within + ±10 %) where the generator placed 11 and 55 (21.8 %). A callout is no + longer a tag occurrence - the rule the line-network reader already kept. + ### Changed +- **`MAX_DXF_BYTES` defaults to 512 MB** (was 50 MB): a real plant layout read + for track H was a 188 MB DXF. The snapshot's refusal names the file's size, + the limit and the variable; `0` still disables the check. - Three tool groups appear in `system_about` and `docs/tool-inventory.json`: `styles` (10), `page_setup` (6), `environment` (22) — 204 tools in 23 groups; `LEAN_TOOL_NAMES` 55; capability keys `dwgprops`, `dwt_write`, @@ -949,6 +1018,86 @@ The first run exited 1: the first room label was refused because the room reader found 0 wall segments on `A-WALL-E-N` — the `entity_list` narrowing listed under **Fixed**. The run above is the one after the fix. +### Live COM smoke — understanding (track H) + +`scripts/smoke_understand_com.py`, AutoCAD 2026, Windows 11, on the synthetic +plant pair only — `take_snapshot` of `pid.dxf` opened live, the same P&ID +saved as DWG and read back through the read-only DWG route, `take_snapshot` +of `layout.dxf`, `pipe_rows` / `cable_rows` on the live snapshots against the +pair's truth, then one tag moved 10 mm and `diff_snapshots` of the layout +before and after, all through the COM engine, exit 0, every document closed: + +```json +{ + "autocad": "25.1s (LMS Tech)", + "export_s": 0.104, + "live_pid_source": "live:pid.dxf", + "dwg_pid_source": "live:pid.dwg", + "model_space": { + "pid_file": { + "ARC": 1, + "INSERT": 15, + "LINE": 8, + "LWPOLYLINE": 8, + "MTEXT": 8, + "TEXT": 31 + }, + "pid_live": { + "ARC": 1, + "INSERT": 15, + "LINE": 8, + "LWPOLYLINE": 8, + "MTEXT": 8, + "TEXT": 31 + }, + "pid_dwg": { + "ARC": 1, + "INSERT": 15, + "LINE": 8, + "LWPOLYLINE": 8, + "MTEXT": 8, + "TEXT": 31 + }, + "layout_file": { + "CIRCLE": 18, + "LINE": 25, + "LWPOLYLINE": 4, + "MULTILEADER": 8, + "TEXT": 26 + }, + "layout_live": { + "CIRCLE": 18, + "LINE": 25, + "LWPOLYLINE": 4, + "MULTILEADER": 8, + "TEXT": 26 + } + }, + "paper_space_layouts": [], + "takeoffs": { + "scale_verdict": "schematic", + "within_10pct": 0.21818181818181817, + "runs_checked": 9, + "pipe_mismatch": [], + "loads_checked": 4, + "cable_mismatch": [] + }, + "diff": { + "moved": "44", + "added": 0, + "removed": 0, + "changed": 1, + "moved_is_the_change": true + } +} +``` + +The smoke works only in documents it brought in from its own temporary +folder, known by full path; each live snapshot, the DWG save and the move +first check, in the same COM-thread call, that the active document is the +smoke's own, and on the way out it re-activates the document that was active +when it started. + ## [1.5.1] — 2026-08-06 A patch for three defects found *after* 1.5.0 went to PyPI, two of them by the diff --git a/CLAUDE.md b/CLAUDE.md index 7946502..a7c0d17 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -33,6 +33,9 @@ pip install -e ".[com]" # With PDF/screenshot support via matplotlib pip install -e ".[pdf]" +# XLSX workbooks (takeoffs, data_extract) via openpyxl +pip install -e ".[office]" + # Everything pip install -e ".[full]" ``` @@ -97,10 +100,12 @@ Tools are organized into sections (counts are indicative — `system_about` is a 23. Mechanical annotation (5 tools, pack `mech`): `surface_texture` (ISO 21920-1, ISO 1302's older Ra form accepted), `weld_symbol` (ISO 2553), `centre_marks` (ISO 128-23), `section_line` (ISO 128-40 — it returns the plane `mech_view_add(kind="section")` consumes, so the label and the view cannot disagree), `hatch_material` (the ISO 128-50 map; an unknown material is refused with the list, never hatched as steel). 24. Sheet and delivery (11 tools, pack `core`): `sheet_frame` (ISO 5457 A4-A0 — 20 mm filing margin, 10 mm elsewhere on A3, zone grid, centring and trimming marks), `titleblock_apply` (ISO 7200 for every size; `titleblock_apply_iso_a3` is now a thin alias), `revision_add`, `bom_extract` (never modifies the drawing), `bom_table` (ISO 7573), `balloon_add` (ISO 6433, linked by XDATA so a rerun renumbers instead of duplicating), `data_extract` (CSV always; XLSX refused with `xlsx_write` when `openpyxl` is absent), `xref_attach`, `xref_manage`, `image_attach`, `drawing_export_dwg` (COM `SaveAs` natively; headless only with the ODA File Converter, otherwise `dwg_write` refuses and names the install route). The COM paths of SECTIONS 21-24 were executed live once by `scripts/smoke_mech_com.py` (AutoCAD 2026): part, section, dimensions, an ISO 4014 block with its XDATA, the parts list read off the drawing, a linked balloon, an xref attached and detached, a real `.dwg`, and `drawing_critique(focus=None)` clean. 25. Architecture (12 tools, pack `arch`): `arch_wall`, `arch_opening`, `arch_stair`, `arch_room`, `arch_rooms_detect`, `arch_schedule`, `arch_grid`, `arch_symbol`, `arch_catalogue_list`, `arch_catalogue_insert`, `arch_dimension_chains`, `arch_plan_from_spec`. One plan model — a wall network (axis polyline, thickness, justification, material) with hosted doors and windows — and one engine: L / T / X junctions resolved, openings cut, cut faces hatched by material (`engineering/arch/materials.py`: ISO 128-50 general hatching for brick, the JIS A 0150 RC / stone / wood symbols, AutoCAD's AR-CONC / AR-SAND, AAC as a declared convention), and anything it cannot place listed in `omitted: [{element, reason}]`, never dropped. Rooms are *computed*: a label's area is the planar face its point lies in, and `arch_rooms_detect` reads rooms from any plan (confidence 1.0 for our walls, 0.6 for plain lines) without modifying it. Schedules are a read of the `ACADMCP_ARCH` records, drawn as real TABLEs. Labels and schedules speak English, or Turkish with `lang="tr"` (decimal comma). The COM paths were executed live once by `scripts/smoke_arch_com.py` (AutoCAD 2026). +26. Understanding & QA (4 tools, pack `core`): `drawing_understand` (one read-only report of a drawing somebody else made — declared vs inferred units with the evidence, declared vs robust extents and the outlier handles behind them, clusters (sheets, plan copies), every layer's likely discipline and service from an EN / TR / RU / NL / DE vocabulary with a confidence and the deciding keyword, equipment tags, rooms, text languages, blocks, layouts, title-block attributes), `drawing_scale_check` (a drawing against a reference: tags unique within a cluster are matched — a tag seen twice is `ambiguous`, never picked — pairs at least 2 m apart, the median ratio, IQR and the share within ±10 %: `to_scale` at 80 % or more, with its factor; `schematic` at 50 % or less; `partly` between), `drawing_diff` (two revisions matched by signature, then handle, then type + layer + nearest position; `markup=True` draws revision clouds on the newer one — its only write) and `drawing_topology_check` (dangling ends, near misses, interior crossings; the three `topo_*` critique focuses run the same engine under `focus=None` on layers no semantic focus covers). Every one reads one snapshot per drawing (`engineering/understand/snapshot.py`: `ezdxf` directly, or on a live seat `Document.Export(/snap, "DXF", ...)`, which never renames the document and carries model space, paper space and block definitions — measured). +27. Plant takeoffs (2 tools, pack `plant`): `pipe_takeoff` (topology from the P&ID — segments merged at their ends, split at T-junctions, continued through inline fittings, a double-drawn segment counted once; service from the layer overridden by supply / return words; sizes kept as drawn — `Ø51`, `SMS51`, `DN20` are never equated — direct, by continuity or `unassigned`; lengths from the layout: the rectilinear MST of the tags a run connects, labelled an estimate, the P&ID length kept beside it as a comparison; statuses A / B / C) and `cable_takeoff` (one row per load: power from the P&ID's electrical texts, never invented; the panel from the layout's `Wiring to CPn` callouts; the Manhattan length plus the allowance rounded up to the metre; a section only from a caller's `section_rules`). Both write a workbook (Metraj / Özet / Kontrol / Metodoloji, `lang` tr / en / ru, XLSX with the `office` extra, CSV always) and never the drawing. `scripts/field_test_takeoff.py` runs both on a user's own pair and writes next to it, never into this repository. The COM paths were executed live once by `scripts/smoke_understand_com.py` (AutoCAD 2026). -**Tool profiles:** `TOOL_PROFILE=lean|full` (default `full`) controls the advertised surface — `lean` = 64 curated drafting tools (54 with `TOOL_PACKS=core`, which drops the three `pid_*`, the three `mech_*` and the four `arch_*` lean members; the five settings essentials — `drawing_apply_standard`, `dimstyle_set_current`, `textstyle_set_current`, `page_setup_apply`, `batch_plot` — are core-pack tools and stay; including `cad_batch`, since a client with a tight tool cap is exactly the client paying most per turn). Applied in the lifespan; reported by `system_about`. The `core` profile was removed in v1.5.0 (the discovery layer replaced it); `TOOL_PROFILE=core` falls back to `full` with a warning. +**Tool profiles:** `TOOL_PROFILE=lean|full` (default `full`) controls the advertised surface — `lean` = 65 curated drafting tools (55 with `TOOL_PACKS=core`, which drops the three `pid_*`, the three `mech_*` and the four `arch_*` lean members, and keeps `drawing_understand`; the five settings essentials — `drawing_apply_standard`, `dimstyle_set_current`, `textstyle_set_current`, `page_setup_apply`, `batch_plot` — are core-pack tools and stay; including `cad_batch`, since a client with a tight tool cap is exactly the client paying most per turn). Applied in the lifespan; reported by `system_about`. The `core` profile was removed in v1.5.0 (the discovery layer replaced it); `TOOL_PROFILE=core` falls back to `full` with a warning. -**Tool packs:** `TOOL_PACKS=all|core,pid,settings,mech,arch` (default `all`) advertises only the vertical packs a client uses; `core` is always on and is everything no other pack claims; `settings` is the 22 SECTION 20 environment tools, `pid` the nine P&ID tools, `mech` the 14 SECTION 21-23 mechanical tools, `arch` the 12 SECTION 25 architectural tools; SECTION 18/19/24 are core; unknown names are ignored with a warning; a `lean` profile intersects with the enabled packs. `system_about` reports `tool_packs`. +**Tool packs:** `TOOL_PACKS=all|core,pid,settings,mech,arch,plant` (default `all`) advertises only the vertical packs a client uses; `core` is always on and is everything no other pack claims; `settings` is the 22 SECTION 20 environment tools, `pid` the nine P&ID tools, `mech` the 14 SECTION 21-23 mechanical tools, `arch` the 12 SECTION 25 architectural tools, `plant` the two SECTION 27 takeoffs (track D's Plant 3D readers join it); SECTION 18/19/24/26 are core; unknown names are ignored with a warning; a `lean` profile intersects with the enabled packs. `system_about` reports `tool_packs`. **Tool discovery:** `DISCOVERY_MODE=off|search` (default `off`). `search` replaces the advertised catalog with `search_tools` + `call_tool`, ranking hits over an AutoCAD command / synonym corpus (`discovery/aliases.py`) layered on fastmcp's BM25 index. @@ -112,7 +117,7 @@ Tools are organized into sections (counts are indicative — `system_about` is a **Measuring:** never read vertices back and shoelace them — that loses 28.2% of the area on a semicircular edge, silently. `analysis_measure_entity(handle)` reads the real geometry, and its payload states its own accuracy: `exact`, `flatten_tolerance`, `assumed_closed`, `self_intersecting`. A HATCH reports the area it *fills* (outer loops minus islands) plus `hatch_style`; on a curved hatch edge `flatten_tolerance` is not the accuracy knob, because ezdxf hands boundaries over as cubic Beziers whose ~0.028% circle error is already baked in. `boundary_trace` / `boundary_from_entities` report the same number `analysis_measure_entity` gives for the polyline they just drew — if those ever diverge, one of them is approximating and neither says so. -**Benchmark matrix:** `benchmarks/tasks_v6.py` is the current set (19 tasks; `--matrix v5` / `v4` / `v3` / `v2` reproduce the earlier sets). The published competitor reports are pinned v1.4 runs against v2 and are **never** back-filled for the five v3 tasks, the v4 `pid_roundtrip` / `page_setup_truth`, the v5 `mech_assembly` (a flange-coupling sheet gated on a zero-issue `drawing_critique(focus=None)` and a finalize score of at least 90) or the v6 `arch_roundtrip` (a two-room plan drawn by `arch_plan_from_spec`, its rooms read back by `arch_rooms_detect` within 0.1 % of the label and of the hand-computed net floor, the schedules listing the drawn tags, gated like `mech_assembly`) — the chart shows them `not run`, not zero. `benchmarks/correctness_suite.py` (43 checks) is the release A/B gate: `python benchmarks/compare_versions.py v1.5.1` (the published report is `benchmarks/results/published/ab-v1.5.1-vs-v1.6.0-dev.json`; `scripts/check_doc_numbers.py` names it in `AB_REPORT`). +**Benchmark matrix:** `benchmarks/tasks_v7.py` is the current set (21 tasks; `--matrix v6` / `v5` / `v4` / `v3` / `v2` reproduce the earlier sets). The published competitor reports are pinned v1.4 runs against v2 and are **never** back-filled for the five v3 tasks, the v4 `pid_roundtrip` / `page_setup_truth`, the v5 `mech_assembly` (a flange-coupling sheet gated on a zero-issue `drawing_critique(focus=None)` and a finalize score of at least 90) or the v6 `arch_roundtrip` (a two-room plan drawn by `arch_plan_from_spec`, its rooms read back by `arch_rooms_detect` within 0.1 % of the label and of the hand-computed net floor, the schedules listing the drawn tags, gated like `mech_assembly`) or the v7 `takeoff_roundtrip` / `understand_foreign` (the synthetic plant pair of `tests/fixtures/plant_pair.py` read without its truth: the P&ID called schematic, every routable run's layout length equal to the rectilinear MST of its tags, the cable metres rounded up after the allowance; units, the outlier, both plan copies and the service layers found by one `describe`) — the chart shows them `not run`, not zero. `benchmarks/correctness_suite.py` (48 checks) is the release A/B gate: `python benchmarks/compare_versions.py v1.5.1` (the published report is `benchmarks/results/published/ab-v1.5.1-vs-v1.6.0-dev.json`; `scripts/check_doc_numbers.py` names it in `AB_REPORT`). ### Adding a New Tool @@ -139,7 +144,7 @@ Tools are organized into sections (counts are indicative — `system_about` is a - **Engineering layer scaffold**: `drawing_new` auto-bootstraps standard linetypes (CENTER, HIDDEN, PHANTOM) and engineering layers (GEOMETRY, DIM, CENTER, HIDDEN, PHANTOM, HATCH, TEXT, TITLEBLOCK). Pass `bootstrap=False` to opt out. - **Generic contracts added in 1.6:** `block_define(name, entities, attdefs)` (typed primitives + ATTDEFs, both engines; `overwrite` replaces contents so INSERTs keep the name) and `entity_get/set_xdata` (typed by group code; 255-char strings, 16 KB per entity, enforced before writing). - **P&ID reader rules:** `pid_graph` never modifies the drawing, reads geometry from the drawing (an INSERT's real rotation/scale) rather than the catalogue, and reports `source`/`confidence` per node (catalog 1.0, xdata 0.95, heuristic 0.6, inferred 0.3) — never raised by agreement between heuristics. Read `stats.confidence_min` before trusting a foreign drawing. -- **`TOOL_PACKS=all|core,pid,settings,mech,arch`** advertises only the packs a client uses; `core` is always on; `system_about` reports `tool_packs`. +- **`TOOL_PACKS=all|core,pid,settings,mech,arch,plant`** advertises only the packs a client uses; `core` is always on; `system_about` reports `tool_packs`. - **Layer states are ours, not AutoCAD's.** `layer_state_*` stores a portable snapshot in an `ACADMCP_LAYERSTATES` XRECORD; it survives save/reopen on both engines and travels with the file, and it is not listed in AutoCAD's Layer States Manager. Said in the tool docstring, in `system_capabilities.layer_states` (`mode: "xrecord"`, reason `portable_acadmcp_xrecord;not_listed_in_autocad_layer_states_manager`) and in the README. - **Page-setup evidence is the PDF.** `batch_plot` reports `mediabox_mm` parsed from the file's `/MediaBox` (`engineering/standards/pdfinfo.py`); a page setup is verified by what a printer would get, not by the setter's return value. A style or setting write reports `changed` — re-setting a value to itself is not a change. - **Nothing interprets coordinates in UCS.** `ucs_set` / `ucs_restore` store and activate a UCS for the operator's benefit; every tool input and output stays WCS (the repository rule above). @@ -153,6 +158,11 @@ Tools are organized into sections (counts are indicative — `system_about` is a - **Plan text follows the plot scale**: `scale` is a denominator (50 = 1:50) and a paper height `h` is drawn at `h × scale` drawing units — no annotative objects. `lang="tr"` switches words and the decimal separator; any other language is refused with the list. - **Three `arch_*` critique focuses** (`arch_room_unlabelled`, `arch_wall_gap` — two wall ends within 50 mm that do not join, `arch_opening_clash`) run under `focus=None` and inside `drawing_finalize`; they return nothing on a drawing with no `ACADMCP_ARCH` record. - **Nominal, not standard**: the furniture and sanitary blocks carry nominal catalogue dimensions and say so; the stair tool reports the Blondel relation (2R + G within 600–650 mm) as a design rule, never as a building code; the `arch` layer colours are a repository convention (ISO 13567 fixes names, not colours). +- **Read-only by default (track H)**: `drawing_understand`, `drawing_scale_check`, `drawing_topology_check` and both takeoffs never modify the drawing; `drawing_diff` writes only with `markup=True`, and a takeoff writes a workbook file, never the drawing. +- **Lengths come from the inferred unit**: a reader measures in the unit the geometry implies (text heights, wall and door sizes, overall size) and reports metres; a disagreement with `INSUNITS` is a warning with the numbers — a layout declaring inches over millimetre geometry is measured in millimetres, never ×25.4. +- **Scale check before a P&ID length**: a P&ID is a length source only when `drawing_scale_check` calls it `to_scale`; otherwise a run's length is the orthogonal estimate on the layout and the drawn length stays a comparison column (`length_source="pid"` on a schematic P&ID is refused unless `force=True`, quoting the statistics). +- **Nothing is invented (track H)**: a power the P&ID does not state stays empty and becomes an open item, an unreadable size stays `unassigned`, and no cable-section table is authored — a section comes only from the caller's `section_rules`. +- **Client drawings never enter the repository**: every track-H fixture is the synthetic plant pair (`tests/fixtures/plant_pair.py`), whose truth is computed from the positions it placed; `scripts/field_test_takeoff.py` refuses an output folder inside the repository and prints no quantity unless `--show` is given. - **Production drawings**: For real engineering output, use the `engineering/` package primitives via the `gear_*` / `keyway_*` / `titleblock_*` MCP tools — do NOT hand-draw teeth/keyways/sections with raw `entity_create_*` calls. Always end with `drawing_finalize` for the 8-step validator. ### Premium Drawing Rules diff --git a/README.md b/README.md index 2da4c78..73c44ca 100644 --- a/README.md +++ b/README.md @@ -27,24 +27,24 @@ Live through COM on Windows, or headless through ezdxf anywhere — one typed co -> **v1.5 release snapshot:** 247 tools · 8 resources · 5 prompt templates · 4942 collected tests. +> **v1.5 release snapshot:** 247 tools · 8 resources · 5 prompt templates · 4960 collected tests. > 247 is the **registered** count; a default install advertises 242 over `tools/list`, > because `ENABLE_3D` is unset. `system_about` is the runtime authority. ## Why this exists -**A big MCP server is expensive to be connected to.** The full catalog costs a client **78,772 tokens** before it has asked for anything. Discovery mode replaces it with two tools and costs **356**. +**A big MCP server is expensive to be connected to.** The full catalog costs a client **83,758 tokens** before it has asked for anything. Discovery mode replaces it with two tools and costs **356**. **A drafter searches for `FILLET`, not `entity_fillet`.** Those command names appeared in no tool name or description — `df = 0` against a stock index, not badly ranked but *absent*. The fix was data: an authored corpus of **206 AutoCAD command names and 1439 synonym phrases** covering all 247 tools. A test refuses to let a tool exist without one. | Advertised surface | Tools seen | Idle cost | |---|---:|---:| -| `TOOL_PROFILE=full` (default) | 236 | 78,772 tokens | -| `TOOL_PACKS=core,settings` (full profile) | 201 | 60,926 tokens | -| `TOOL_PACKS=core,mech` (full profile) | 193 | 61,872 tokens | -| `TOOL_PACKS=core,arch` (full profile) | 191 | 60,568 tokens | -| `TOOL_PACKS=core` (full profile) | 179 | 54,158 tokens | -| `TOOL_PROFILE=lean` | 64 | 21,939 tokens | +| `TOOL_PROFILE=full` (default) | 242 | 83,758 tokens | +| `TOOL_PACKS=core,settings` (full profile) | 205 | 63,890 tokens | +| `TOOL_PACKS=core,mech` (full profile) | 197 | 64,836 tokens | +| `TOOL_PACKS=core,arch` (full profile) | 195 | 63,532 tokens | +| `TOOL_PACKS=core` (full profile) | 183 | 57,122 tokens | +| `TOOL_PROFILE=lean` | 65 | 22,661 tokens | | `DISCOVERY_MODE=search` | 2 | **356 tokens** | > [!NOTE] @@ -106,6 +106,7 @@ Claude Desktop, Cursor, or any stdio MCP host. For HTTP: `autocad-mcp --transpor | Mechanical parts | one part model (segments or outline + typed features) and one view engine: front / side / top / section / detail with hidden lines, ISO 128-50 cut faces and ISO 129 dimensions; ISO 4014/4017/4032/7089/4762 fasteners, ISO 15 bearings, DIN 471/472, DIN 509, DIN 332 and ISO 3601-2 as real blocks with attributes; ISO 21920-1 surface texture, ISO 2553 welds, ISO 128-40 section lines | | Sheet & delivery | ISO 5457 frames A4-A0 with zones and trim marks, ISO 7200 title blocks for every size, revision blocks with clouds, ISO 7573 parts lists with ISO 6433 balloons linked by XDATA, CSV/XLSX extraction, xrefs and images on both engines, real `.dwg` on a live seat | | Architecture | one plan model — walls with hosted doors and windows — and one engine: L / T / X junctions resolved, openings cut, poché by material; rooms measured from the faces the walls enclose (never typed) and read back from any plan with a confidence; door / window / room schedules as real TABLEs; a furniture and sanitary catalogue at nominal sizes; structural grid, north arrow, section / level / elevation marks; exterior dimension chains; English or Turkish labels (`lang="tr"`, decimal comma) | +| Understanding & takeoffs | `drawing_understand` reads a drawing somebody else made in one call — declared vs inferred units, robust extents and the outlier behind them, plan copies, each layer's discipline and service in five languages, tags, rooms; `drawing_scale_check` says whether a P&ID is to scale against its layout; `pipe_takeoff` / `cable_takeoff` take the topology from the P&ID and the lengths from the layout (rectilinear MST, statuses A/B/C, power never invented) into a Turkish / English / Russian workbook; `drawing_diff` between revisions; `drawing_topology_check` for dangling ends, near misses and crossings | | P&ID | catalogue blocks with ports and tags (ISO 10628-2 / ISA-5.1), port-to-port lines with ISA-5.1 classes and line numbers, `pid_graph` reads any P&ID back with confidence, instrument index / line list / equipment list, `pid_from_spec` one-call sheets | | Styles & standards | ISO-25 / ANSI dimension styles, ISOCP / ROMANS text styles, ISO / ANSI leader styles from authored presets; `drawing_apply_standard("iso")` sets all of it plus units and the `mech` layers in one call; `changed` reports only what moved | | Discovery | `search_tools` ranked over an AutoCAD command and synonym corpus — `FILLET`, `BPOLY`, `QSELECT`, `WBLOCK`, `OVERKILL`, `CHSPACE` each rank **#1** of the 242-tool advertised catalog | @@ -162,23 +163,23 @@ Self-measurement, produced by scripts in [`benchmarks/`](https://github.com/U-C4 ### Correctness — every release re-proves itself -43 deterministic headless checks against the previous tag and the current tree, each in its own subprocess so a hard crash counts as a miss rather than killing the run. +48 deterministic headless checks against the previous tag and the current tree, each in its own subprocess so a hard crash counts as a miss rather than killing the run. | Version | Checks passing | Pass rate | Fixed | Regressed | |---|---:|---:|---:|---:| -| v1.5.1 *(baseline)* | 26 / 43 | 60.5 % | — | — | -| **v1.6.0-dev** *(this branch)* | **43 / 43** | **100 %** | 17 | **0** | +| v1.5.1 *(baseline)* | 26 / 48 | 54.2 % | — | — | +| **v1.6.0-dev** *(this branch)* | **48 / 48** | **100 %** | 22 | **0** | -The seventeen fixed rows are new capability (`miss → pass`). Track F adds four: `arch_junction_l_t_x` (an L corner mitres at (4100, −100) / (3900, 100), a T stem stops on the near face at (2950, 100) / (3050, 100), an X cuts all four faces, and the wall areas are 1 400 000, 1 490 000 and 2 360 000 mm² — computed by hand), `arch_room_area_net` (a 4 × 5 m room between 200 mm walls measures 20.00 m² in its label and its record, not the 21.84 m² its axes enclose), `arch_opening_cuts_wall` (both faces are interrupted across a 900 door and two jambs close it) and `arch_rooms_detect_foreign` (a plan of plain lines on a `WALLS` layer yields 20 and 15 m² at confidence 0.6, and nothing is written). Tracks B and G add seven: `mech_part_roundtrip` (a part read back out of its own `ACADMCP_MECH` XDATA equals the part drawn), `mech_section_hatch_area` (a 60 x ⌀40 sleeve with a ⌀20 bore cuts 1200 mm² exactly, measured out of the drawing by `analysis_measure_entity`), `mech_iso286_on_dimension` (40 H7 is +0.025 / 0 and survives the layout), `mech_thread_unrepresented_is_caught` (the ISO 6410 focus fires on a thread drawn as a plain circle — a gate that never fires is not a gate), `std_part_iso4014_m12` (s = 18, k = 7.5, read by value), `sheet_frame_iso5457_a3` (the frame runs 20,10 to 410,287 inside the 420 × 297 sheet) and `bom_balloon_link` (two identical bolts are one row of quantity 2, and its balloon carries the row's item number). The other six are the three P&ID checks of track A and, from track E, `settings_dimstyle_iso25_values` (the ISO-25 preset lands in the DIMSTYLE table with ISO 129-1's numbers), `settings_layer_state_roundtrip` (save → change → restore puts the layer table back, and the state survives save/reopen because it is an XRECORD in the file) and `settings_pdf_mediabox_a3` (the plotted PDF's own `/MediaBox` reads 420 × 297). The 26 checks v1.5.1 was gated on all still pass. Against the older `v1.4.0` baseline the same 26 reported **21 / 26 → 26 / 26, five fixed, zero regressed** — two of those were repaired defects (`fail → pass`), the diameter and radius callouts, which measured the leader as geometry and dimensioned a 40 mm bore as 60 at default settings. +The twenty-two fixed rows are new capability (`miss → pass`). Track H adds five: `scale_check_detects_schematic` (the synthetic P&ID reads **schematic** against its layout, and a uniform 2× copy of six tags reads **to scale** with factor 2), `pipe_takeoff_rmst_exact` (all eight routable runs of the pair measure exactly the rectilinear MST of their tags), `cable_takeoff_roundup` (9, 17, 19 and 10 m after the 20 % allowance, and the heater's unstated power stays empty), `diff_detects_known_edits` (a move, a text change, an attribute change, an add and a delete — and nothing else) and `topology_known_defects` (a 5 mm near miss, one interior crossing, and a clean T that is neither). Track F adds four: `arch_junction_l_t_x` (an L corner mitres at (4100, −100) / (3900, 100), a T stem stops on the near face at (2950, 100) / (3050, 100), an X cuts all four faces, and the wall areas are 1 400 000, 1 490 000 and 2 360 000 mm² — computed by hand), `arch_room_area_net` (a 4 × 5 m room between 200 mm walls measures 20.00 m² in its label and its record, not the 21.84 m² its axes enclose), `arch_opening_cuts_wall` (both faces are interrupted across a 900 door and two jambs close it) and `arch_rooms_detect_foreign` (a plan of plain lines on a `WALLS` layer yields 20 and 15 m² at confidence 0.6, and nothing is written). Tracks B and G add seven: `mech_part_roundtrip` (a part read back out of its own `ACADMCP_MECH` XDATA equals the part drawn), `mech_section_hatch_area` (a 60 x ⌀40 sleeve with a ⌀20 bore cuts 1200 mm² exactly, measured out of the drawing by `analysis_measure_entity`), `mech_iso286_on_dimension` (40 H7 is +0.025 / 0 and survives the layout), `mech_thread_unrepresented_is_caught` (the ISO 6410 focus fires on a thread drawn as a plain circle — a gate that never fires is not a gate), `std_part_iso4014_m12` (s = 18, k = 7.5, read by value), `sheet_frame_iso5457_a3` (the frame runs 20,10 to 410,287 inside the 420 × 297 sheet) and `bom_balloon_link` (two identical bolts are one row of quantity 2, and its balloon carries the row's item number). The other six are the three P&ID checks of track A and, from track E, `settings_dimstyle_iso25_values` (the ISO-25 preset lands in the DIMSTYLE table with ISO 129-1's numbers), `settings_layer_state_roundtrip` (save → change → restore puts the layer table back, and the state survives save/reopen because it is an XRECORD in the file) and `settings_pdf_mediabox_a3` (the plotted PDF's own `/MediaBox` reads 420 × 297). The 26 checks v1.5.1 was gated on all still pass. Against the older `v1.4.0` baseline the same 26 reported **21 / 26 → 26 / 26, five fixed, zero regressed** — two of those were repaired defects (`fail → pass`), the diameter and radius callouts, which measured the leader as geometry and dimensioned a 40 mm bore as 60 at default settings. ### The task matrix — tasks that can fail -An earlier matrix scored this server 10/10, which carried no information: every task in it exercised something the server was built around. Five were added in 1.5 because they *can* fail, and three did while being written; 1.6 adds four more — one the P&ID reader can fail on its own, one where the PDF file, not the setter, is the witness, one where a whole mechanical sheet has to come out clean, and one where a floor plan's rooms are read back off the drawing. +An earlier matrix scored this server 10/10, which carried no information: every task in it exercised something the server was built around. Five were added in 1.5 because they *can* fail, and three did while being written; 1.6 adds six more — one the P&ID reader can fail on its own, one where the PDF file, not the setter, is the witness, one where a whole mechanical sheet has to come out clean, one where a floor plan's rooms are read back off the drawing, and two where a foreign plant's P&ID and layout are read: the takeoffs against lengths computed in advance, and the one-call report against the defects planted in it. | Task | Verified against | |---|---| | `tool_discovery` | six AutoCAD command names, each ranking #1 | -| `token_budget` | 78,772 → 356 tokens, against a ceiling fixed in advance | +| `token_budget` | 83,758 → 356 tokens, against a ceiling fixed in advance | | `hatch_islands` | 300 filled with the island, 400 ignoring it | | `selection_filter` | window 1, crossing 2, bounding box 3, polygon 1 | | `measure_from_handle` | 139.2699 against the 100.0 a vertex shoelace gives | @@ -186,6 +187,8 @@ An earlier matrix scored this server 10/10, which carried no information: every | `page_setup_truth` | ANSI B then ISO A3 landscape applied to Layout1, each plotted through `batch_plot` and read back from its PDF's `/MediaBox`: 432 × 279 then 420 × 297 mm, not the setter's return value (a fresh layout is already A3, so the B sheet is what a no-op setter cannot fake) | | `mech_assembly` | a flange-coupling A3 sheet — hub with a full section, flange with its bolt-circle end view, two ISO 4014 bolts with ISO 4032 nuts, an ISO 7573 parts list read off the drawing, linked balloons, ISO 5457 frame, ISO 7200 title block — gated on `drawing_critique(focus=None)` returning **zero** issues and a finalize score of at least **90**; a test drops one balloon and watches the gate fail | | `arch_roundtrip` | a two-room plan — an entrance door, an interior door, two windows, a stair — drawn by `arch_plan_from_spec` and read back by `arch_rooms_detect`, which never sees the spec: each room within 0.1 % of its label and of the net floor computed by hand (4825 × 5750 and 3825 × 5750 mm), the door and window schedules listing D1, D2, W1, W2, `drawing_critique(focus=None)` at **zero** and a finalize score of at least **90**; a test removes one room label and watches the gate fail | +| `takeoff_roundtrip` | the synthetic plant pair — a P&ID stretched 1.3× in one room and rearranged in the other, Russian supply / return words, a return drawn on the supply layer, a reducer, a segment drawn twice — read by the scale check (**schematic**: 21.8 % of 55 pairs within ±10 %), `pipe_takeoff` (every routable run's layout length equal to the rectilinear MST of its tags, sizes and services as drawn, the run to a tag the layout lacks status C) and `cable_takeoff` (9 / 17 / 19 / 10 m after the 20 % allowance, the heater's power left empty); a test removes the wiring callouts and watches the cable half fail by name | +| `understand_foreign` | the pair's layout — inches declared over millimetres, a stray line 5,000 km out, two plan copies — comes back as millimetres with an `INSUNITS` warning, the stray line's handle and two clusters; the P&ID's `P_product piping` / `P_cipsupplyline` / `P_cipreturnline` / `P_ijswater` / `E_power` classify as product / CIP supply / CIP return / ice water / electrical; a test makes the layout declare millimetres and watches the units gate fail | ### Headless performance diff --git a/benchmarks/README.md b/benchmarks/README.md index d6b2a40..62a3fe7 100644 --- a/benchmarks/README.md +++ b/benchmarks/README.md @@ -127,6 +127,17 @@ result for it. The competitor reports carry no result for it. +### Matrix v7 (v1.6) — understanding foreign drawings + +`tasks_v7.py` keeps the v6 nineteen unchanged and adds track H's evidence, both on the synthetic plant pair of `tests/fixtures/plant_pair.py` (invented names; its truth is computed from the positions it placed, never typed): + +| Task | Category | Verified against | +|---|---|---| +| `takeoff_roundtrip` | plant | `scale_check(pid, layout)` must say **schematic** (room 1 is a 1.3× stretch, room 2 is rearranged: 21.8 % of the 55 tag pairs within ±10 % of the median ratio); `pipe_rows(..., length_source="auto")` must report all nine runs with the truth's status, each routable run's layout length equal to the rectilinear MST of its tags to a micron (PR1 5500, PR2 13200, CS1 20700 mm, …), its size and service as drawn — the CIP return drawn on the supply layer included — and both sizes across the reducer; `cable_rows` must give M11 / T102 / M12 / T202 their panels, 6900 / 14100 / 15800 / 8000 mm and 9 / 17 / 19 / 10 m after the 20 % allowance, the heater's power empty. `tests/test_benchmark_v7.py` deletes the wiring callouts and the gate names all four loads | +| `understand_foreign` | understand | `describe(layout)` must infer millimetres and warn about `INSUNITS` (the file says inches), name the stray line's handle in its extents and report two clusters (the plan is drawn twice, 100 m apart); `describe(pid)` must classify `P_product piping`, `P_cipsupplyline`, `P_cipreturnline`, `P_ijswater` and `E_power` as product, CIP supply, CIP return, ice water and electrical. `tests/test_benchmark_v7.py` makes the layout declare millimetres and the units gate fails alone | + +The competitor reports carry no result for them. + ## Live competitor lane (v1.4) Two competitor adapters now execute the exact same task matrix, black-box over @@ -244,15 +255,15 @@ python benchmarks/compare_versions.py --json results.json ### Result — this branch (1.6.0-dev) vs v1.5.1 (release gate) -43 checks, ezdxf backend, one subprocess per check. Machine-readable report: +48 checks, ezdxf backend, one subprocess per check. Machine-readable report: [`results/published/ab-v1.5.1-vs-v1.6.0-dev.json`](results/published/ab-v1.5.1-vs-v1.6.0-dev.json). | Version | Checks passing | Pass rate | Fixed | Regressed | |---------|----------------|-----------|-------|-----------| -| **v1.5.1** (baseline) | 26 / 43 | 60.5 % | — | — | -| **v1.6.0-dev** (this branch) | 43 / 43 | 100 % | 17 | **0** | +| **v1.5.1** (baseline) | 26 / 48 | 54.2 % | — | — | +| **v1.6.0-dev** (this branch) | 48 / 48 | 100 % | 22 | **0** | -The seventeen are all `miss → pass` — v1.5.1 has none of the methods. Four are track F: `arch_junction_l_t_x` (L, T and X junction outlines against hand-computed corners and wall areas), `arch_room_area_net` (a 4 × 5 m room between 200 mm walls is labelled 20.00 m², not its 21.84 m² axis area), `arch_opening_cuts_wall` (both faces interrupted across a door, two jambs) and `arch_rooms_detect_foreign` (plain lines on a WALLS layer read as 20 and 15 m² at confidence 0.6, nothing written). Seven +The twenty-two are all `miss → pass` — v1.5.1 has none of the methods. Five are track H: `scale_check_detects_schematic` (the synthetic P&ID is schematic against its layout; a uniform 2× copy is to scale with factor 2), `pipe_takeoff_rmst_exact` (eight routable runs, each exactly the rectilinear MST of its tags), `cable_takeoff_roundup` (9 / 17 / 19 / 10 m, the unstated power left empty), `diff_detects_known_edits` (five known edits and nothing else) and `topology_known_defects` (one near miss, one crossing, a clean T). Four are track F: `arch_junction_l_t_x` (L, T and X junction outlines against hand-computed corners and wall areas), `arch_room_area_net` (a 4 × 5 m room between 200 mm walls is labelled 20.00 m², not its 21.84 m² axis area), `arch_opening_cuts_wall` (both faces interrupted across a door, two jambs) and `arch_rooms_detect_foreign` (plain lines on a WALLS layer read as 20 and 15 m² at confidence 0.6, nothing written). Seven are tracks B and G: `mech_part_roundtrip` (a part read back from its own `ACADMCP_MECH` XDATA equals the part drawn), `mech_section_hatch_area` (a sleeve 60 long, ⌀40 outside, ⌀20 bore: a full section cuts two 60 × 10 faces, diff --git a/benchmarks/adapters/autocad_mcp_pro.py b/benchmarks/adapters/autocad_mcp_pro.py index 19ca6d9..0f39e6f 100644 --- a/benchmarks/adapters/autocad_mcp_pro.py +++ b/benchmarks/adapters/autocad_mcp_pro.py @@ -1,4 +1,4 @@ -"""Reference adapter for this repository, covering the v6 task matrix. +"""Reference adapter for this repository, covering the v7 task matrix. Every task verifies against a value worked out independently of the code under test — a closed-form area, a count of entities placed on purpose, a token @@ -12,7 +12,7 @@ import math from pathlib import Path from benchmarks.adapters.base import BenchmarkAdapter, TaskResult -from benchmarks.tasks_v6 import TaskSpec +from benchmarks.tasks_v7 import TaskSpec from engineering.delivery import deliver_drawing from engineering.layers import ensure_engineering_layers, ensure_standard_linetypes from engineering.refiner import refine_drawing @@ -779,6 +779,147 @@ class AutoCADMCPProAdapter(BenchmarkAdapter): [str(sheet)], ) + async def _task_takeoff_roundtrip(self): + """Pipe and cable quantities off the synthetic plant pair, against its truth. + + Track H's evidence (spec §15). The pair is written fresh into the + artifact folder by ``tests/fixtures/plant_pair.py``, whose truth is + computed from the positions it placed. Then only the two DXFs are read: + + * ``scale_check(pid, layout)`` must say ``schematic`` - room 1 is the + layout stretched 1.3x, room 2 is rearranged, and 21.8 % of the 55 tag + pairs lie within ±10 % of the median ratio; + * ``pipe_rows(..., length_source="auto")`` must report every run of the + P&ID (the ice-water utility included, because ``services`` names it) + with the truth's status; a routable run's layout length must equal the + rectilinear MST of its tags exactly, its sizes (both across the + reducer) and its service must be the drawn ones (the CIP return drawn + on the supply layer included); the run ending on T105, which the + layout lacks, is status C with no layout length; + * ``cable_rows(pid, layout)`` must give every load its panel from the + layout's wiring callouts, its stated power (the heater's stays empty), + the Manhattan length to the panel and the metres rounded up after the + 20 % allowance. + + Any one slipping fails the task, and the metrics name what slipped. + """ + from engineering.understand.network import build_network + from engineering.understand.scale import scale_check + from engineering.understand.snapshot import read_snapshot + from engineering.understand.takeoff import cable_rows, pipe_rows + from tests.fixtures.plant_pair import build_plant_pair + + truth = build_plant_pair(self.artifact_dir / "plant_pair") + pid = read_snapshot(truth["pid"]) + layout = read_snapshot(truth["layout"]) + scale = scale_check(pid, layout) + services = tuple(sorted({run["service"] for run in truth["pipe_runs"]})) + pipe = pipe_rows( + build_network(pid), pid, layout, scale=scale, services=services, length_source="auto" + ) + found = {(run["service"], tuple(sorted(run["tags"]))): run for run in pipe["runs"]} + pipe_mismatch = [] + for run in truth["pipe_runs"]: + got = found.get((run["service"], tuple(run["tags"]))) + if got is None: + pipe_mismatch.append({"run": run["id"], "reason": "no run with these tags"}) + continue + reasons = [] + if got["status"] != run["status"]: + reasons.append(f"status {got['status']} != {run['status']}") + want, have = run["layout_length_mm"], got.get("layout_m") + if want is None and have is not None: + reasons.append(f"a layout length ({have} m) for an unroutable run") + elif want is not None and (have is None or abs(float(have) * 1000.0 - want) > 1e-6): + reasons.append(f"layout length {have} m != {want} mm") + if set(got["diameters"]) != set(run["diameters"]): + reasons.append(f"sizes {sorted(got['diameters'])} != {sorted(run['diameters'])}") + if reasons: + pipe_mismatch.append({"run": run["id"], "reason": "; ".join(reasons)}) + + cable = {row["tag"]: row for row in cable_rows(pid, layout)["rows"]} + cable_mismatch = [] + for want in truth["cable_rows"]: + got = cable.get(want["tag"]) + if got is None: + cable_mismatch.append({"tag": want["tag"], "wrong": ["missing"]}) + continue + wrong = [key for key in ("panel", "kw") if got.get(key) != want[key]] + if got.get("cable_m") != want["roundup_m"]: + wrong.append("cable_m") + length = got.get("length_m") + if length is None or abs(float(length) * 1000.0 - want["length_mm"]) > 1e-6: + wrong.append("length_m") + if wrong: + cable_mismatch.append({"tag": want["tag"], "wrong": wrong}) + + passed = scale["verdict"] == truth["scale_verdict"] and not pipe_mismatch + passed = passed and not cable_mismatch + return ( + passed, + { + "scale_verdict": scale["verdict"], + "within_10pct": scale.get("within_10pct"), + "length_source": pipe.get("length_source"), + "runs_checked": len(truth["pipe_runs"]), + "pipe_mismatch": pipe_mismatch, + "loads_checked": len(truth["cable_rows"]), + "cable_mismatch": cable_mismatch, + }, + [truth["pid"], truth["layout"]], + ) + + async def _task_understand_foreign(self): + """One call on a foreign layout reports what is wrong with it, by evidence. + + Track H's evidence (spec §5, §15), on the synthetic plant pair: the + layout declares inches (``$INSUNITS = 1``) over millimetre geometry, a + stray LINE 5,000 km away stretches its declared extents, and the plan is + drawn twice, 100 m apart. ``describe`` of the layout must infer + millimetres, report that they disagree with the declared unit and warn + about ``INSUNITS``, name the stray entity's handle in its extents, and + report exactly two clusters; ``describe`` of the P&ID must classify its + four service layers and the electrical layer as the generator names them + (English and Dutch names). A warning or an extents report is searched as + JSON, so the gate holds whatever shape the evidence takes. + """ + import json + + from engineering.understand.describe import describe + from engineering.understand.snapshot import read_snapshot + from tests.fixtures.plant_pair import build_plant_pair + + truth = build_plant_pair(self.artifact_dir / "plant_pair") + layout = describe(read_snapshot(truth["layout"])) + pid = describe(read_snapshot(truth["pid"])) + units = layout["units"] + units_flagged = ( + str(units.get("inferred")) == truth["layout_true_unit"] + and units.get("agree") is False + and "INSUNITS" in json.dumps(layout["warnings"], ensure_ascii=False) + ) + outlier_found = f'"{truth["outlier_handle"]}"' in json.dumps(layout["extents"]) + copies_separated = len(layout["clusters"]) == truth["cluster_count"] + read = {row["name"]: row["service"] for row in pid["layers"]} + misread = { + name: read.get(name) + for name, service in truth["layer_services"].items() + if read.get(name) != service + } + passed = units_flagged and outlier_found and copies_separated and not misread + return ( + passed, + { + "units": units, + "units_flagged": units_flagged, + "outlier_found": outlier_found, + "clusters": len(layout["clusters"]), + "copies_separated": copies_separated, + "misread_layers": misread, + }, + [truth["layout"], truth["pid"]], + ) + async def _task_auditable_delivery(self): await ensure_engineering_layers(self.backend) await self.backend.entity_create_line(0, 0, 10, 0, layer="GEOMETRY") diff --git a/benchmarks/correctness_suite.py b/benchmarks/correctness_suite.py index b95c7ec..0038ac8 100644 --- a/benchmarks/correctness_suite.py +++ b/benchmarks/correctness_suite.py @@ -818,6 +818,214 @@ async def arch_rooms_detect_foreign(): ) +# ── Understanding foreign drawings (track H) ─────────────────────────────── + + +def _plant_pair() -> dict: + """The synthetic plant pair, written to a fresh temporary folder.""" + from tests.fixtures.plant_pair import build_plant_pair + + return build_plant_pair(tempfile.mkdtemp(prefix="acadmcp_plant_")) + + +def _quoted(handle: str) -> str: + """A handle as it appears in a JSON dump - quoted, so '2A' never matches '12A3'.""" + return f'"{handle}"' + + +async def scale_check_detects_schematic(): + """The synthetic P&ID is schematic against its layout; a uniform 2x copy is not. + + The pair: room 1 of the P&ID is the layout stretched 1.3x, room 2 is + rearranged, and of the 55 pairs of the 11 shared tags 21.8 % lie within + ±10 % of the median ratio - at most 50 % is ``schematic`` (spec §6). + The control: six tags at (0, 0), (4000, 0), (8000, 0), (0, 3000), + (4000, 3000), (8000, 6000) and the same tags at twice those coordinates - + the nearest pair is 3000 mm apart, every ratio is exactly 2.0, so the + doubled drawing is ``to_scale`` against the first with factor 2. + """ + import ezdxf + + from engineering.understand.scale import scale_check + from engineering.understand.snapshot import read_snapshot + + truth = _plant_pair() + pair = scale_check(read_snapshot(truth["pid"]), read_snapshot(truth["layout"])) + folder = tempfile.mkdtemp(prefix="acadmcp_scale_") + positions = [(0, 0), (4000, 0), (8000, 0), (0, 3000), (4000, 3000), (8000, 6000)] + paths = [] + for factor in (1.0, 2.0): + doc = ezdxf.new("R2018") + msp = doc.modelspace() + for number, (x, y) in enumerate(positions, start=1): + msp.add_circle((x * factor, y * factor), 500.0) + msp.add_text(f"T{number}", dxfattribs={"insert": (x * factor, y * factor)}) + path = os.path.join(folder, f"x{factor:g}.dxf") + doc.saveas(path) + paths.append(path) + control = scale_check(read_snapshot(paths[1]), read_snapshot(paths[0])) + return ( + pair["verdict"] == "schematic" + and control["verdict"] == "to_scale" + and abs(float(control["factor"]) - 2.0) < 1e-9 + ) + + +async def pipe_takeoff_rmst_exact(): + """Every routable pipe run's layout length is the rectilinear MST of its tags. + + Measured on the synthetic pair with ``length_source="layout"``. The truth + is the generator's own Manhattan / Prim arithmetic over the tag positions it + placed, hand-checked in tests/test_plant_pair.py - PR1 5500 (5000 + 500), + PR2 13200 (the three distances 6000 / 7200 / 13200, spanning tree 6000 + + 7200), CS1 20700 (20500 + 200). All eight routable runs must agree to a + micron; the run ending on T105, which the layout lacks, is not routable. + """ + from engineering.understand.network import build_network + from engineering.understand.snapshot import read_snapshot + from engineering.understand.takeoff import pipe_rows + + truth = _plant_pair() + pid, layout = read_snapshot(truth["pid"]), read_snapshot(truth["layout"]) + services = tuple(sorted({run["service"] for run in truth["pipe_runs"]})) + result = pipe_rows(build_network(pid), pid, layout, services=services, length_source="layout") + found = {(r["service"], tuple(sorted(r["tags"]))): r for r in result["runs"]} + routable = [run for run in truth["pipe_runs"] if run["layout_length_mm"] is not None] + for run in routable: + got = found.get((run["service"], tuple(run["tags"]))) + if got is None or got.get("layout_m") is None: + return False + if abs(float(got["layout_m"]) * 1000.0 - run["layout_length_mm"]) > 1e-6: + return False + return len(routable) == 8 + + +async def cable_takeoff_roundup(): + """Cable rows: the panel from the callout, Manhattan length, metres rounded up. + + The truth rows of the synthetic pair, by hand: M11 (5500, 8200) to CP-1 + (1200, 10800) is 6900 mm, 6.9 m x 1.2 = 8.28 -> 9; T102 (8000, 3500) to CP-1 + is 14100 mm -> 16.92 -> 17; M12 (26300, 8400) to CP-2 (35000, 1300) is + 15800 mm -> 18.96 -> 19; T202 (28700, 3000) to CP-2 is 8000 mm -> 9.6 -> 10. + The heater on T202 states no power, and its row must keep an empty one. + """ + from engineering.understand.snapshot import read_snapshot + from engineering.understand.takeoff import cable_rows + + truth = _plant_pair() + rows = { + row["tag"]: row + for row in cable_rows(read_snapshot(truth["pid"]), read_snapshot(truth["layout"]))["rows"] + } + expected = {"M11": ("CP-1", 9), "T102": ("CP-1", 17), "M12": ("CP-2", 19), "T202": ("CP-2", 10)} + for tag, (panel, metres) in expected.items(): + row = rows.get(tag) + if row is None or row.get("panel") != panel or row.get("cable_m") != metres: + return False + return rows["T202"].get("kw") is None and rows["M11"].get("kw") == 5.5 + + +async def diff_detects_known_edits(): + """Two revisions differing by exactly five edits - and the diff says so. + + A move (a LINE shifted 10 mm), a text change (PUMP A -> PUMP B), an + attribute change (TAG T101 -> T102), an add (a new LINE) and a delete (a + LINE removed); a CIRCLE stays untouched. The edits are made on a copy read + back from the first file, so every surviving entity keeps its handle. The + diff must list the added line as added, the removed line as removed, the + three edited entities as changed - and nothing else, the circle included. + """ + import json + + import ezdxf + + from engineering.understand.diff import diff_snapshots + from engineering.understand.snapshot import read_snapshot + + folder = tempfile.mkdtemp(prefix="acadmcp_diff_") + old_path, new_path = os.path.join(folder, "rev_a.dxf"), os.path.join(folder, "rev_b.dxf") + doc = ezdxf.new("R2018") + msp = doc.modelspace() + moved = msp.add_line((0, 0), (100, 0), dxfattribs={"layer": "PIPE"}) + text = msp.add_text("PUMP A", dxfattribs={"insert": (0, 50), "height": 2.5}) + block = doc.blocks.new("TAGGED") + block.add_circle((0, 0), 10) + block.add_attdef("TAG", (0, 0), dxfattribs={"height": 2.5}) + insert = msp.add_blockref("TAGGED", (200, 0)) + insert.add_auto_attribs({"TAG": "T101"}) + gone = msp.add_line((0, 100), (100, 100), dxfattribs={"layer": "PIPE"}) + kept = msp.add_circle((300, 300), 20) + doc.saveas(old_path) + + revision = ezdxf.readfile(old_path) + revision.entitydb[moved.dxf.handle].translate(10, 0, 0) + revision.entitydb[text.dxf.handle].dxf.text = "PUMP B" + revision.entitydb[insert.dxf.handle].get_attrib("TAG").dxf.text = "T102" + revision.modelspace().delete_entity(revision.entitydb[gone.dxf.handle]) + added = revision.modelspace().add_line((500, 500), (600, 500), dxfattribs={"layer": "PIPE"}) + revision.saveas(new_path) + + diff = diff_snapshots(read_snapshot(old_path), read_snapshot(new_path)) + changed = json.dumps(diff["changed"]) + dumped = changed + json.dumps(diff["added"]) + json.dumps(diff["removed"]) + return ( + len(diff["added"]) == 1 + and len(diff["removed"]) == 1 + and len(diff["changed"]) == 3 + and _quoted(added.dxf.handle) in json.dumps(diff["added"]) + and _quoted(gone.dxf.handle) in json.dumps(diff["removed"]) + and all(_quoted(e.dxf.handle) in changed for e in (moved, text, insert)) + and _quoted(kept.dxf.handle) not in dumped + ) + + +async def topology_known_defects(): + """Dangling ends, a near miss and a crossing, placed on purpose - and nothing else. + + On layer PIPE: a closed 1000 x 1000 square (no free end, no defect); a LINE + from (500, -500) that stops at (500, -5), 5 mm short of the square's bottom + edge (a near miss at gap 10, and a dangling end); two LINEs crossing at + (2500, 500), away from all four of their ends (one interior crossing); and a + clean T - a LINE ending on the middle of another at (4500, 0), which is a + junction, not a crossing and not a near miss. + """ + import json + + import ezdxf + + from engineering.understand.snapshot import read_snapshot + from engineering.understand.topology import topology_findings + + doc = ezdxf.new("R2018") + msp = doc.modelspace() + pipe = {"layer": "PIPE"} + ring = msp.add_lwpolyline( + [(0, 0), (1000, 0), (1000, 1000), (0, 1000)], close=True, dxfattribs=pipe + ) + short = msp.add_line((500, -500), (500, -5), dxfattribs=pipe) + cross_a = msp.add_line((2000, 0), (3000, 1000), dxfattribs=pipe) + cross_b = msp.add_line((2000, 1000), (3000, 0), dxfattribs=pipe) + bar = msp.add_line((4000, 0), (5000, 0), dxfattribs=pipe) + stem = msp.add_line((4500, 0), (4500, 500), dxfattribs=pipe) + path = os.path.join(tempfile.mkdtemp(prefix="acadmcp_topo_"), "topology.dxf") + doc.saveas(path) + + found = topology_findings(read_snapshot(path), layers=["PIPE"], gap=10.0) + near = json.dumps(found["near_miss"]) + crossing = json.dumps(found["crossing"]) + dangling = json.dumps(found["dangling"]) + return ( + _quoted(short.dxf.handle) in near + and _quoted(short.dxf.handle) in dangling + and _quoted(ring.dxf.handle) not in dangling + and _quoted(ring.dxf.handle) not in crossing + and len(found["crossing"]) == 1 + and _quoted(cross_a.dxf.handle) in crossing + and _quoted(cross_b.dxf.handle) in crossing + and all(_quoted(e.dxf.handle) not in crossing + near for e in (bar, stem)) + ) + + CHECKS = { "core_line_length": (core_line_length, "Core"), "core_circle_radius": (core_circle_radius, "Core"), @@ -862,6 +1070,11 @@ CHECKS = { "arch_room_area_net": (arch_room_area_net, "Architecture"), "arch_opening_cuts_wall": (arch_opening_cuts_wall, "Architecture"), "arch_rooms_detect_foreign": (arch_rooms_detect_foreign, "Architecture"), + "scale_check_detects_schematic": (scale_check_detects_schematic, "Understanding"), + "pipe_takeoff_rmst_exact": (pipe_takeoff_rmst_exact, "Understanding"), + "cable_takeoff_roundup": (cable_takeoff_roundup, "Understanding"), + "diff_detects_known_edits": (diff_detects_known_edits, "Understanding"), + "topology_known_defects": (topology_known_defects, "Understanding"), } diff --git a/benchmarks/render_matrix_chart.py b/benchmarks/render_matrix_chart.py index 7ff2ba8..226befb 100644 --- a/benchmarks/render_matrix_chart.py +++ b/benchmarks/render_matrix_chart.py @@ -13,7 +13,7 @@ import json from pathlib import Path from typing import Any -from benchmarks.tasks_v6 import TASKS_V6 +from benchmarks.tasks_v7 import TASKS_V7 ROOT = Path(__file__).resolve().parents[1] DEFAULT_INPUT_DIR = ROOT / "benchmarks" / "results" / "published" @@ -66,8 +66,8 @@ def render_chart(reports: list[dict[str, Any]], output: Path) -> None: matplotlib.rcParams["svg.hashsalt"] = "autocad-mcp-taskmatrix-v1" - task_ids = [task.task_id for task in TASKS_V6] - task_labels = [f"{task.task_id} ({task.category})" for task in TASKS_V6] + task_ids = [task.task_id for task in TASKS_V7] + task_labels = [f"{task.task_id} ({task.category})" for task in TASKS_V7] servers = [report["adapter"] for report in reports] status_by_server = { report["adapter"]: {item["task_id"]: item["status"] for item in report["results"]} diff --git a/benchmarks/results/published/ab-v1.5.1-vs-v1.6.0-dev.json b/benchmarks/results/published/ab-v1.5.1-vs-v1.6.0-dev.json index ecb1407..a22e996 100644 --- a/benchmarks/results/published/ab-v1.5.1-vs-v1.6.0-dev.json +++ b/benchmarks/results/published/ab-v1.5.1-vs-v1.6.0-dev.json @@ -1,12 +1,12 @@ { "summary": { "baseline_ref": "v1.5.1", - "total": 43, + "total": 48, "old_pass": 26, - "new_pass": 43, - "old_pct": 60.5, + "new_pass": 48, + "old_pct": 54.2, "new_pct": 100.0, - "fixed": 17, + "fixed": 22, "regressed": 0 }, "rows": [ @@ -267,6 +267,36 @@ "category": "Architecture", "old": "miss", "new": "pass" + }, + { + "check": "scale_check_detects_schematic", + "category": "Understanding", + "old": "miss", + "new": "pass" + }, + { + "check": "pipe_takeoff_rmst_exact", + "category": "Understanding", + "old": "miss", + "new": "pass" + }, + { + "check": "cable_takeoff_roundup", + "category": "Understanding", + "old": "miss", + "new": "pass" + }, + { + "check": "diff_detects_known_edits", + "category": "Understanding", + "old": "miss", + "new": "pass" + }, + { + "check": "topology_known_defects", + "category": "Understanding", + "old": "miss", + "new": "pass" } ] } \ No newline at end of file diff --git a/benchmarks/run_competitors.py b/benchmarks/run_competitors.py index 7e5106b..d0cb401 100644 --- a/benchmarks/run_competitors.py +++ b/benchmarks/run_competitors.py @@ -16,7 +16,7 @@ from pathlib import Path from backends.capability import UnsupportedCapabilityError from benchmarks.adapters.base import BenchmarkAdapter, TaskResult -from benchmarks.tasks_v6 import DEFAULT_MATRIX, MATRICES, TaskSpec, task_by_id +from benchmarks.tasks_v7 import DEFAULT_MATRIX, MATRICES, TaskSpec, task_by_id def _load_registry() -> dict[str, dict]: diff --git a/benchmarks/tasks_v7.py b/benchmarks/tasks_v7.py new file mode 100644 index 0000000..3605c2e --- /dev/null +++ b/benchmarks/tasks_v7.py @@ -0,0 +1,71 @@ +"""The v7 task matrix: v6 plus what track H added. + +Track H teaches the server to read drawings somebody else made, so its two +tasks are gated on a drawing pair whose answers are known in advance - the +synthetic plant pair of ``tests/fixtures/plant_pair.py``, which reproduces +every field finding of the design spec with invented names: + +* ``takeoff_roundtrip`` - the scale check calls the P&ID ``schematic``; every + routable pipe run's layout length equals the rectilinear minimum spanning + tree of its tags exactly; its size, service and status match the generator's + truth; the cable rows equal the Manhattan lengths from each load to its panel + with the allowance rounded up to the metre. +* ``understand_foreign`` - one ``describe`` of the layout flags inches declared + over millimetre geometry, finds the stray entity by handle and separates the + two plan copies; one ``describe`` of the P&ID classifies its English and + Dutch service layers. + +The published competitor reports were never asked these questions; the chart +shows ``not_run`` for them, not zero. +""" + +from __future__ import annotations + +from benchmarks.tasks_v2 import TASKS_V2 +from benchmarks.tasks_v3 import TASKS_V3 +from benchmarks.tasks_v4 import TASKS_V4 +from benchmarks.tasks_v5 import TASKS_V5 +from benchmarks.tasks_v6 import TASKS_V6, TaskSpec + +#: Added in v1.6.0 by track H. +NEW_TASKS_V7: tuple[TaskSpec, ...] = ( + TaskSpec( + "takeoff_roundtrip", + "plant", + "Take pipe and cable quantities off a schematic P&ID and its layout: the P&ID " + "called schematic, every run's layout length equal to the rectilinear MST of " + "its tags, sizes and services as drawn, cable metres rounded up after the " + "allowance", + 1.0, + ), + TaskSpec( + "understand_foreign", + "understand", + "Read a foreign layout in one call: inches declared over millimetres flagged, " + "the stray entity found, both plan copies separated, the service layers of its " + "P&ID classified", + 1.0, + ), +) + +TASKS_V7: tuple[TaskSpec, ...] = TASKS_V6 + NEW_TASKS_V7 + +#: Runner-selectable task sets. v2 to v6 stay addressable so an earlier report +#: can be reproduced exactly rather than only described. +MATRICES: dict[str, tuple[TaskSpec, ...]] = { + "v2": TASKS_V2, + "v3": TASKS_V3, + "v4": TASKS_V4, + "v5": TASKS_V5, + "v6": TASKS_V6, + "v7": TASKS_V7, +} + +DEFAULT_MATRIX = "v7" + + +def task_by_id(task_id: str, matrix: str = DEFAULT_MATRIX) -> TaskSpec: + for task in MATRICES[matrix]: + if task.task_id == task_id: + return task + raise KeyError(f"{task_id} is not in matrix {matrix}") diff --git a/docs/superpowers/specs/2026-09-24-v1.6-understand-design.md b/docs/superpowers/specs/2026-09-24-v1.6-understand-design.md index 07622f5..befa2ce 100644 --- a/docs/superpowers/specs/2026-09-24-v1.6-understand-design.md +++ b/docs/superpowers/specs/2026-09-24-v1.6-understand-design.md @@ -1,6 +1,6 @@ # v1.6 Track H — understanding foreign drawings, takeoffs, diff and topology -**Date:** 2026-09-24 · **Status:** design approved in conversation · +**Date:** 2026-09-24 · **Status:** shipped (v1.6.0-dev, track H) · **Parent:** `2026-09-15-v1.6-roadmap-design.md` (track H; track D stays separate and follows) · **Baseline:** main after track F (`0abbe55`, 241 tools) diff --git a/engineering/understand/describe.py b/engineering/understand/describe.py index d9329e6..f4d359b 100644 --- a/engineering/understand/describe.py +++ b/engineering/understand/describe.py @@ -32,7 +32,7 @@ from collections.abc import Sequence from engineering.arch.faces import Face, face_containing, planar_faces from engineering.pid.graph import ATTRIBUTE_TAGS, KEYWORDS from engineering.understand.clusters import find_clusters -from engineering.understand.labels import parse_tag +from engineering.understand.labels import parse_tag, wiring_target from engineering.understand.snapshot import EntityRecord, Snapshot, length_of from engineering.understand.units import infer_units, robust_extents, wall_segments from engineering.understand.vocab import classify_layer, room_label @@ -113,6 +113,9 @@ def tag_occurrences(snap: Snapshot, *, prep: dict | None = None) -> list[dict]: A tag is read with `labels.parse_tag` from a TEXT / MTEXT / MULTILEADER, or from an INSERT attribute named like a tag (`pid.graph.ATTRIBUTE_TAGS`). + A wiring callout (`labels.wiring_target`: ``Wiring to CP-2``) is not an + occurrence of the panel it names - it stands at the load, not the panel - + the rule `network.tag_occurrences` keeps too. Two readings of one tag in one space closer than :data:`TAG_MERGE_FRACTION` of the diagonal are one occurrence (the attribute's position and confidence win). Rows carry ``tag``, ``handle``, @@ -128,6 +131,8 @@ def tag_occurrences(snap: Snapshot, *, prep: dict | None = None) -> list[dict]: continue readings = [] if rec.type in _TEXT_TYPES and rec.text: + if wiring_target(rec.text): + continue # names the panel it points at; it is not written there readings.append((parse_tag(rec.text), "text", TEXT_TAG_CONFIDENCE)) elif rec.type == "INSERT": for name, value in rec.attribs: diff --git a/scripts/check_doc_numbers.py b/scripts/check_doc_numbers.py index 1fc6837..ab8a881 100644 --- a/scripts/check_doc_numbers.py +++ b/scripts/check_doc_numbers.py @@ -79,7 +79,7 @@ def derived() -> dict[str, object]: } from benchmarks import correctness_suite - from benchmarks.tasks_v6 import TASKS_V6 + from benchmarks.tasks_v7 import TASKS_V7 return { "collected_tests": _collected_tests(), @@ -94,7 +94,7 @@ def derived() -> dict[str, object]: "ab_fixed": ab["fixed"], "ab_regressed": ab["regressed"], "suite_checks": len(correctness_suite.CHECKS), - "matrix_tasks": len(TASKS_V6), + "matrix_tasks": len(TASKS_V7), "perf_ms": perf, } diff --git a/scripts/field_test_takeoff.py b/scripts/field_test_takeoff.py new file mode 100644 index 0000000..92e1e9e --- /dev/null +++ b/scripts/field_test_takeoff.py @@ -0,0 +1,128 @@ +"""Run both takeoffs on a real P&ID and layout, and write the report next to them. + +LOCAL ONLY. This is the field test of track H (spec §15): the user points it at +a client's P&ID and layout, compares what comes out with the hand-made +workbooks, and keeps the result on their own disk. Nothing it reads or writes +may enter this repository (spec §14), so: + +* the report is written next to the P&ID (or into ``--out``) and **the script + refuses any output folder inside this repository**, before reading a byte; +* the console gets file names, row counts and the scale verdict - **never a + quantity**, unless ``--show`` is given on the user's own terminal; +* it is never run in CI and never by a test on a real drawing; the test next + to it (tests/test_field_test_takeoff.py) runs it on the synthetic plant pair. + +What it writes: ``pipe_takeoff.xlsx`` / ``cable_takeoff.xlsx`` (CSV beside +them always, XLSX when the ``office`` extra is installed) through the same +``write_workbook`` the tools use, and ``field_test_report.json`` with the scale +check and both results in full. + + uv run --frozen python scripts/field_test_takeoff.py --pid D:/job/pid.dxf --layout D:/job/layout.dxf + uv run --frozen python scripts/field_test_takeoff.py --pid ... --layout ... --lang en --show +""" + +from __future__ import annotations + +import argparse +import json +import sys +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(ROOT)) + +#: Exit code for a refused run (output inside the repository, a missing file). +REFUSED = 2 + + +def _inside_repository(folder: Path) -> bool: + resolved = folder.resolve() + return resolved == ROOT or ROOT in resolved.parents + + +def _parse(argv) -> argparse.Namespace: + parser = argparse.ArgumentParser( + description="Pipe and cable takeoffs on a real P&ID + layout (local only)." + ) + parser.add_argument("--pid", required=True, help="the P&ID, a .dxf") + parser.add_argument("--layout", required=True, help="the layout, a .dxf") + parser.add_argument("--out", default=None, help="report folder (default: the P&ID's folder)") + parser.add_argument("--lang", default="tr", choices=("tr", "en", "ru")) + parser.add_argument("--allowance", type=float, default=0.20) + parser.add_argument( + "--show", action="store_true", help="also print the totals (quantities) to the console" + ) + return parser.parse_args(argv) + + +def main(argv=None) -> int: + args = _parse(argv) + pid_path, layout_path = Path(args.pid), Path(args.layout) + out = Path(args.out) if args.out else pid_path.resolve().parent + if _inside_repository(out): + print( + f"refused: {out} is inside the repository ({ROOT}); client drawings and " + "anything derived from them never enter it. Pass --out outside it." + ) + return REFUSED + for path in (pid_path, layout_path): + if not path.is_file(): + print(f"refused: {path} does not exist") + return REFUSED + + from engineering.understand.network import build_network + from engineering.understand.report import write_workbook + from engineering.understand.scale import scale_check + from engineering.understand.snapshot import read_snapshot + from engineering.understand.takeoff import cable_rows, pipe_rows + + out.mkdir(parents=True, exist_ok=True) + pid = read_snapshot(str(pid_path)) + layout = read_snapshot(str(layout_path)) + scale = scale_check(pid, layout) + pipe = pipe_rows(build_network(pid), pid, layout, scale=scale, allowance=args.allowance) + cable = cable_rows(pid, layout, allowance=args.allowance) + written = [ + write_workbook(pipe, kind="pipe", lang=args.lang, path=str(out / "pipe_takeoff.xlsx")), + write_workbook(cable, kind="cable", lang=args.lang, path=str(out / "cable_takeoff.xlsx")), + ] + report = out / "field_test_report.json" + report.write_text( + json.dumps( + {"scale": scale, "pipe": pipe, "cable": cable, "written": written}, + indent=2, + ensure_ascii=False, + default=str, + ), + encoding="utf-8", + ) + print( + json.dumps( + { + "scale_verdict": scale["verdict"], + "pipe_rows": len(pipe["rows"]), + "cable_rows": len(cable["rows"]), + "report": str(report), + "workbooks": [ + path + for item in written + for path in ([item["xlsx"]] if item["xlsx"] else []) + list(item["csv"]) + ], + }, + indent=2, + ) + ) + if args.show: + print( + json.dumps( + {"pipe": pipe["rows"], "cable": cable["rows"]}, + indent=2, + ensure_ascii=False, + default=str, + ) + ) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/smoke_understand_com.py b/scripts/smoke_understand_com.py new file mode 100644 index 0000000..d7a5695 --- /dev/null +++ b/scripts/smoke_understand_com.py @@ -0,0 +1,386 @@ +"""Exercise track H once against the live AutoCAD on this machine. + +Builds the synthetic plant pair (tests/fixtures/plant_pair.py) in a temporary +folder and reads it through the live engine - never an operator's drawing: + + open pid.dxf -> take_snapshot(backend) (Document.Export, "DXF") + -> drawing_save_as(pid.dwg), close it + -> take_snapshot(backend, pid.dwg) (opened read-only, exported, closed) + open layout.dxf -> take_snapshot(backend) + pipe_rows / cable_rows on the two live snapshots, against the pair's truth + entity_move one tag text by 10 mm -> take_snapshot -> diff_snapshots + against the snapshot taken before the move + +and prints what came back as JSON. Exit codes: + + 0 every live snapshot holds the same model-space entities (by type) as the + file it came from, the takeoffs match the truth, and the diff reports + exactly the one moved entity + 1 it ran, but something disagreed (the JSON says what) + 2 no live CAD application reachable (pywin32 missing, or no running + instance to attach to) -- the COM paths then remain fake-tested only + +**The operator's drawings are never touched.** AutoCAD may have other +documents open, with unsaved work in them, while this runs. So: + +* the smoke only ever attaches to a running instance (``GetActiveObject``); + it never creates one, because ``Dispatch`` would launch AutoCAD; +* every document it works in is one it brought in itself from the files it + just wrote into its own temporary folder, and it knows them by full path - + never by a name another document could share; +* every call that reads or writes *the current document* (a live snapshot, the + DWG save, the move) first checks, inside the same COM-thread call, that the + active document is the smoke's own - and refuses otherwise; +* on the way out (also when a step fails) it closes its own documents with + ``Close(False)`` and re-activates the document that was active when it + started; unless ``--keep`` is given, then its documents stay open; +* nothing it prints names a document other than its own. + + AUTOCAD_MCP_BACKEND=com uv run --frozen python scripts/smoke_understand_com.py + uv run --frozen python scripts/smoke_understand_com.py --keep # leave them open +""" + +from __future__ import annotations + +import asyncio +import dataclasses +import json +import sys +import tempfile +import time +from collections import Counter +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(ROOT)) + +#: ``RPC_E_CALL_REJECTED``: AutoCAD's message filter refused the call because +#: it was busy (a document still initialising after ``Documents.Open``). A +#: refused call was not executed. +RPC_E_CALL_REJECTED = -2147418111 + +#: How long a refused call is retried before the smoke gives up on it. +REJECTED_PATIENCE_S = 90.0 + +#: Full paths (normalised) of the documents this run opened and has not closed. +_OPENED: list[str] = [] + +#: The run's state shared with the COM thread: the folder its files live in, +#: the document that must be active for the next current-document call (None: +#: the call does not touch the current document), and the document that was +#: active when the run started (a COM object, used on the COM thread only). +_STATE: dict = {"folder": None, "expect": None, "start": None} + + +class NotOurDocument(RuntimeError): + """The active document is not the one the smoke is about to read or write.""" + + +def _key(path) -> str: + return str(Path(str(path)).resolve()).lower() + + +def _ours(path) -> bool: + folder = _STATE["folder"] + return bool(path) and folder is not None and Path(_key(path)).parent == Path(_key(folder)) + + +def _check_active() -> None: + """COM thread: refuse unless the active document is the expected own one.""" + expected = _STATE["expect"] + if expected is None: + return + from backends import com_backend as com + + app = com._acad_app() + active = str(app.ActiveDocument.FullName) if int(app.Documents.Count) else "" + if not active or _key(active) != expected: + # Never name the other document: it may be an operator's drawing. + raise NotOurDocument( + f"refused: the active document is not the smoke's own {Path(expected).name}" + ) + + +def _patient(backend) -> None: + """Retry any call AutoCAD refused with ``RPC_E_CALL_REJECTED``, and run the + active-document check in the same COM-thread call as the call it guards.""" + run = backend._run + + async def patient(func, *args, **kwargs): + def guarded(*a, **kw): + _check_active() + return func(*a, **kw) + + deadline = time.monotonic() + REJECTED_PATIENCE_S + pause = 0.5 + while True: + try: + return await run(guarded, *args, **kwargs) + except RuntimeError as exc: + cause = exc.__cause__ + hr = cause.args[0] if cause is not None and cause.args else None + if hr != RPC_E_CALL_REJECTED or time.monotonic() + pause > deadline: + raise + await asyncio.sleep(pause) + pause = min(pause * 2.0, 5.0) + + backend._run = patient + + +class _Current: + """``async with _Current(path):`` - calls inside it must find ``path`` active.""" + + def __init__(self, path) -> None: + self.path = _key(path) + + async def __aenter__(self): + _STATE["expect"] = self.path + return self + + async def __aexit__(self, *exc): + _STATE["expect"] = None + return False + + +async def _attach(backend) -> str | None: + """Attach to the running instance only, and remember the active document. + + Returns the application's version, or None when nothing is running (the + backend's own ``_acad_app`` would fall back to ``Dispatch`` and launch + AutoCAD - a smoke must never do that).""" + import win32com.client + + import config + from backends import com_backend as com + + def _sync(): + try: + app = win32com.client.GetActiveObject(config.settings.cad_progid) + except Exception: + return None + com._COM_STATE["app"] = app + _STATE["start"] = app.ActiveDocument if int(app.Documents.Count) else None + return str(app.Version) + + return await backend._run(_sync) + + +async def _open_paths(backend) -> set[str]: + """Full paths of the open documents that are the smoke's own (COM thread).""" + + def _sync(): + from backends import com_backend as com + + app = com._acad_app() + paths = (str(app.Documents.Item(i).FullName) for i in range(int(app.Documents.Count))) + return {_key(p) for p in paths if _ours(p)} + + return await backend._run(_sync) + + +async def _open(backend, path) -> str: + """``drawing_open`` of one of the smoke's own files; returns its key.""" + if not _ours(path): + raise NotOurDocument(f"refused: {Path(path).name} is not in the smoke's own folder") + result = await backend.drawing_open(str(path)) + key = _key(result.get("path") or path) + if key not in _OPENED: + _OPENED.append(key) + return key + + +async def _close(backend, key: str) -> None: + """Close one of the smoke's own documents, unsaved (``Close(False)``).""" + if not _ours(key): + raise NotOurDocument("refused: not one of the smoke's own documents") + if key in await _open_paths(backend): + await backend.document_close(key, discard=True) + if key in _OPENED: + _OPENED.remove(key) + + +async def _close_leftovers(backend) -> None: + for key in list(_OPENED): + try: + await _close(backend, key) + except Exception as exc: # best effort on the way out + print(f"could not close {Path(key).name}: {type(exc).__name__}", file=sys.stderr) + if key in _OPENED: + _OPENED.remove(key) + + +async def _reactivate_start(backend) -> None: + """Hand the seat back as it was: the starting document active again.""" + + def _sync(): + start = _STATE["start"] + if start is None: + return + try: + start.Activate() + except Exception: # it may have been closed by its owner meanwhile + pass + + try: + await backend._run(_sync) + except Exception as exc: + print(f"could not re-activate the starting document: {type(exc).__name__}", file=sys.stderr) + + +def _model(snapshot): + """The snapshot with its model-space records only (AutoCAD adds a paper-space + viewport when it initialises a layout; the takeoffs read model space).""" + return dataclasses.replace( + snapshot, records=tuple(r for r in snapshot.records if r.space == "Model") + ) + + +def _types(snapshot) -> dict[str, int]: + return dict(sorted(Counter(r.type for r in _model(snapshot).records).items())) + + +def _check_takeoffs(truth, pid, layout) -> dict: + from engineering.understand.network import build_network + from engineering.understand.scale import scale_check + from engineering.understand.takeoff import cable_rows, pipe_rows + + scale = scale_check(pid, layout) + services = tuple(sorted({run["service"] for run in truth["pipe_runs"]})) + pipe = pipe_rows(build_network(pid), pid, layout, scale=scale, services=services) + found = {(r["service"], tuple(sorted(r["tags"]))): r for r in pipe["runs"]} + pipe_mismatch = [] + for run in truth["pipe_runs"]: + got = found.get((run["service"], tuple(run["tags"]))) + want = run["layout_length_mm"] + have = None if got is None else got.get("layout_m") + agrees = got is not None and got["status"] == run["status"] + if want is None: + agrees = agrees and have is None + else: + agrees = agrees and have is not None and abs(float(have) * 1000.0 - want) <= 1e-6 + if not agrees: + pipe_mismatch.append({"run": run["id"], "want": want, "have": have}) + rows = {row["tag"]: row for row in cable_rows(pid, layout)["rows"]} + cable_mismatch = [ + want["tag"] + for want in truth["cable_rows"] + if rows.get(want["tag"], {}).get("cable_m") != want["roundup_m"] + or rows.get(want["tag"], {}).get("panel") != want["panel"] + ] + return { + "scale_verdict": scale["verdict"], + "within_10pct": scale.get("within_10pct"), + "runs_checked": len(truth["pipe_runs"]), + "pipe_mismatch": pipe_mismatch, + "loads_checked": len(truth["cable_rows"]), + "cable_mismatch": cable_mismatch, + } + + +async def _smoke(backend, folder: Path) -> dict: + from engineering.understand.diff import diff_snapshots + from engineering.understand.snapshot import read_snapshot, take_snapshot + from tests.fixtures.plant_pair import build_plant_pair + + truth = build_plant_pair(folder) + + pid_key = await _open(backend, truth["pid"]) + async with _Current(pid_key): + started = time.perf_counter() + live_pid = await take_snapshot(backend) + export_s = time.perf_counter() - started + dwg = folder / "pid.dwg" + await backend.drawing_save_as(str(dwg), "dwg") + # SaveAs renamed the document: it is now pid.dwg, and still the smoke's own. + dwg_key = _key(dwg) + _OPENED.append(dwg_key) + for key in (pid_key, dwg_key): + await _close(backend, key) + dwg_pid = await take_snapshot(backend, str(dwg)) + + layout_key = await _open(backend, truth["layout"]) + async with _Current(layout_key): + live_layout = await take_snapshot(backend) + + takeoffs = _check_takeoffs(truth, _model(live_pid), _model(live_layout)) + + tag = next( + r + for r in live_layout.records + if r.type == "TEXT" and r.layer == "TAGS" and r.space == "Model" + ) + async with _Current(layout_key): + await backend.entity_move(tag.handle, 10.0, 0.0) + after = await take_snapshot(backend) + diff = diff_snapshots(_model(live_layout), _model(after)) + await _close(backend, layout_key) + + file_pid, file_layout = read_snapshot(truth["pid"]), read_snapshot(truth["layout"]) + return { + "export_s": round(export_s, 3), + "live_pid_source": live_pid.source, + "dwg_pid_source": dwg_pid.source, + "model_space": { + "pid_file": _types(file_pid), + "pid_live": _types(live_pid), + "pid_dwg": _types(dwg_pid), + "layout_file": _types(file_layout), + "layout_live": _types(live_layout), + }, + "paper_space_layouts": sorted({r.space for r in live_layout.records} - {"Model"}), + "takeoffs": takeoffs, + "diff": { + "moved": tag.handle, + "added": len(diff["added"]), + "removed": len(diff["removed"]), + "changed": len(diff["changed"]), + "moved_is_the_change": f'"{tag.handle}"' in json.dumps(diff["changed"]), + }, + } + + +async def main(keep_document: bool = False) -> int: + try: + from backends.com_backend import ComBackend + except ImportError as exc: + print(f"pywin32 not available: {exc}") + return 2 + backend = ComBackend() + try: + await backend.connect() + except Exception as exc: + print(f"no live AutoCAD reachable: {exc}") + return 2 + _patient(backend) + folder = Path(tempfile.mkdtemp(prefix="acadmcp_understand_smoke_")) + _STATE["folder"] = str(folder) + try: + version = await _attach(backend) + if version is None: + print("no live AutoCAD reachable: no running instance to attach to") + return 2 + report = {"autocad": version, **await _smoke(backend, folder)} + print(json.dumps(report, indent=2, ensure_ascii=False)) + spaces = report["model_space"] + ok = ( + spaces["pid_live"] == spaces["pid_file"] == spaces["pid_dwg"] + and spaces["layout_live"] == spaces["layout_file"] + and report["takeoffs"]["scale_verdict"] == "schematic" + and not report["takeoffs"]["pipe_mismatch"] + and not report["takeoffs"]["cable_mismatch"] + and report["diff"]["changed"] == 1 + and report["diff"]["added"] == report["diff"]["removed"] == 0 + and report["diff"]["moved_is_the_change"] + ) + return 0 if ok else 1 + finally: + _STATE["expect"] = None + if not keep_document: + await _close_leftovers(backend) + await _reactivate_start(backend) + _STATE["start"] = None + await backend.disconnect() + + +if __name__ == "__main__": + raise SystemExit(asyncio.run(main(keep_document="--keep" in sys.argv[1:]))) diff --git a/tests/test_benchmark_v4.py b/tests/test_benchmark_v4.py index 8b9d48e..f9c7575 100644 --- a/tests/test_benchmark_v4.py +++ b/tests/test_benchmark_v4.py @@ -49,7 +49,7 @@ def test_correctness_suite_has_the_settings_checks(): ): assert CHECKS[name][1] == "Settings" # v4 left the suite at 32; tracks B + G grew it to 39 (tests/test_benchmark_v5.py). - assert len(CHECKS) == 43 + assert len(CHECKS) == 48 @pytest.mark.asyncio # asyncio_mode is strict; the four tests above are sync diff --git a/tests/test_benchmark_v5.py b/tests/test_benchmark_v5.py index dd2ad28..b127eca 100644 --- a/tests/test_benchmark_v5.py +++ b/tests/test_benchmark_v5.py @@ -48,7 +48,7 @@ def test_the_suite_carries_the_mechanical_and_sheet_checks(): assert CHECKS[name][1] == "Mechanical" assert CHECKS["sheet_frame_iso5457_a3"][1] == "Sheet" assert CHECKS["bom_balloon_link"][1] == "Sheet" - assert len(CHECKS) == 43 + assert len(CHECKS) == 48 @pytest.mark.asyncio diff --git a/tests/test_benchmark_v6.py b/tests/test_benchmark_v6.py index 1ef60f1..36af66b 100644 --- a/tests/test_benchmark_v6.py +++ b/tests/test_benchmark_v6.py @@ -52,7 +52,7 @@ def test_the_suite_carries_the_four_architectural_checks(): "arch_rooms_detect_foreign", ): assert CHECKS[name][1] == "Architecture" - assert len(CHECKS) == 43 + assert len(CHECKS) == 48 @pytest.mark.asyncio diff --git a/tests/test_benchmark_v7.py b/tests/test_benchmark_v7.py new file mode 100644 index 0000000..3d1832d --- /dev/null +++ b/tests/test_benchmark_v7.py @@ -0,0 +1,136 @@ +"""The v7 task matrix: v6 plus what track H added. + +``takeoff_roundtrip`` and ``understand_foreign`` are track H's evidence, both +gated on the synthetic plant pair (tests/fixtures/plant_pair.py) whose truth +is computed from the positions it placed. Each gate is shown to fail on a +broken input - a gate that cannot fail is not evidence. The five correctness +checks run here once too, so a regression fails the suite and not only the +A/B report. +""" + +from __future__ import annotations + +import ezdxf +import pytest + +from benchmarks.tasks_v6 import TASKS_V6 +from benchmarks.tasks_v7 import DEFAULT_MATRIX, MATRICES, NEW_TASKS_V7, TASKS_V7, task_by_id +from tests.fixtures import plant_pair + +UNDERSTANDING_CHECKS = ( + "scale_check_detects_schematic", + "pipe_takeoff_rmst_exact", + "cable_takeoff_roundup", + "diff_detects_known_edits", + "topology_known_defects", +) + + +def test_v7_extends_v6_without_touching_it(): + assert TASKS_V7[: len(TASKS_V6)] == TASKS_V6 + assert [t.task_id for t in NEW_TASKS_V7] == ["takeoff_roundtrip", "understand_foreign"] + assert len(TASKS_V7) == 21 + assert MATRICES["v6"] == TASKS_V6 and MATRICES["v7"] == TASKS_V7 + assert DEFAULT_MATRIX == "v7" + assert task_by_id("takeoff_roundtrip").category == "plant" + assert task_by_id("understand_foreign").category == "understand" + + +def test_the_adapter_implements_every_v7_task(): + from benchmarks.adapters.autocad_mcp_pro import AutoCADMCPProAdapter + + for task in TASKS_V7: + assert hasattr(AutoCADMCPProAdapter, f"_task_{task.task_id}"), task.task_id + + +def test_the_suite_carries_the_five_understanding_checks(): + from benchmarks.correctness_suite import CHECKS + + for name in UNDERSTANDING_CHECKS: + assert CHECKS[name][1] == "Understanding" + assert len(CHECKS) == 48 + + +async def _run(task_id, tmp_path): + from benchmarks.adapters.autocad_mcp_pro import AutoCADMCPProAdapter + + adapter = AutoCADMCPProAdapter(backend="ezdxf") + await adapter.setup(tmp_path) + try: + await adapter._reset() + return await getattr(adapter, f"_task_{task_id}")() + finally: + await adapter.cleanup() + + +def _broken(edit): + """build_plant_pair, then ``edit(truth)`` on the files it wrote.""" + real = plant_pair.build_plant_pair + + def build(directory): + truth = real(directory) + edit(truth) + return truth + + return build + + +@pytest.mark.asyncio +async def test_takeoff_roundtrip_passes_headlessly(tmp_path): + passed, metrics, _artifacts = await _run("takeoff_roundtrip", tmp_path) + assert passed, metrics + assert metrics["scale_verdict"] == "schematic" + assert metrics["runs_checked"] == 9 and metrics["loads_checked"] == 4 + + +@pytest.mark.asyncio +async def test_takeoff_roundtrip_fails_loudly_without_the_wiring_callouts(tmp_path, monkeypatch): + """Every callout gone from the layout: no load has a panel any more, so no + cable length can be measured - and the gate names all four loads, while the + pipe half, untouched, still agrees.""" + + def drop_callouts(truth): + doc = ezdxf.readfile(truth["layout"]) + msp = doc.modelspace() + for entity in list(msp.query("MULTILEADER")): + msp.delete_entity(entity) + doc.saveas(truth["layout"]) + + monkeypatch.setattr(plant_pair, "build_plant_pair", _broken(drop_callouts)) + passed, metrics, _artifacts = await _run("takeoff_roundtrip", tmp_path) + assert passed is False + assert sorted(row["tag"] for row in metrics["cable_mismatch"]) == ["M11", "M12", "T102", "T202"] + assert metrics["pipe_mismatch"] == [] + + +@pytest.mark.asyncio +async def test_understand_foreign_passes_headlessly(tmp_path): + passed, metrics, _artifacts = await _run("understand_foreign", tmp_path) + assert passed, metrics + assert metrics["clusters"] == 2 and metrics["misread_layers"] == {} + + +@pytest.mark.asyncio +async def test_understand_foreign_fails_loudly_when_the_units_tell_the_truth(tmp_path, monkeypatch): + """The same layout declaring millimetres: there is no disagreement left to + flag, so the units gate fails - and only that gate.""" + + def honest_units(truth): + doc = ezdxf.readfile(truth["layout"]) + doc.header["$INSUNITS"] = 4 + doc.saveas(truth["layout"]) + + monkeypatch.setattr(plant_pair, "build_plant_pair", _broken(honest_units)) + passed, metrics, _artifacts = await _run("understand_foreign", tmp_path) + assert passed is False + assert metrics["units_flagged"] is False + assert metrics["outlier_found"] and metrics["copies_separated"] + + +@pytest.mark.asyncio +@pytest.mark.parametrize("name", UNDERSTANDING_CHECKS) +async def test_each_understanding_check_passes(name): + from benchmarks.correctness_suite import CHECKS + + check, _category = CHECKS[name] + assert await check() is True diff --git a/tests/test_field_test_takeoff.py b/tests/test_field_test_takeoff.py new file mode 100644 index 0000000..2e3562e --- /dev/null +++ b/tests/test_field_test_takeoff.py @@ -0,0 +1,69 @@ +"""scripts/field_test_takeoff.py never writes into the repository and never prints a quantity. + +The script exists for client drawings, which never enter this repository +(spec §14). It is exercised here on the synthetic plant pair only - the field +test itself is never run in CI. +""" + +from __future__ import annotations + +import importlib.util +import json +from pathlib import Path + +from tests.fixtures.plant_pair import build_plant_pair + +ROOT = Path(__file__).resolve().parents[1] + + +def _script(): + spec = importlib.util.spec_from_file_location( + "field_test_takeoff", ROOT / "scripts" / "field_test_takeoff.py" + ) + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + +def test_an_output_folder_inside_the_repository_is_refused_before_anything_runs(tmp_path, capsys): + truth = build_plant_pair(tmp_path) + target = ROOT / "field_test_output_must_not_exist" + code = _script().main( + ["--pid", truth["pid"], "--layout", truth["layout"], "--out", str(target)] + ) + assert code == 2 + assert not target.exists() + assert "refused" in capsys.readouterr().out + + +def test_a_missing_drawing_is_refused(tmp_path, capsys): + code = _script().main( + ["--pid", str(tmp_path / "none.dxf"), "--layout", str(tmp_path / "none.dxf")] + ) + assert code == 2 + assert "does not exist" in capsys.readouterr().out + + +def test_the_report_lands_next_to_the_pid_and_the_console_carries_no_quantity(tmp_path, capsys): + truth = build_plant_pair(tmp_path) + code = _script().main(["--pid", truth["pid"], "--layout", truth["layout"]]) + assert code == 0 + summary = json.loads(capsys.readouterr().out) + assert summary["scale_verdict"] == "schematic" + assert Path(summary["report"]).parent == tmp_path + assert all(Path(path).parent == tmp_path for path in summary["workbooks"]) + printed = json.dumps({k: v for k, v in summary.items() if k not in ("report", "workbooks")}) + for run in truth["pipe_runs"]: + if run["layout_length_mm"] is not None: + assert f"{run['layout_length_mm']:g}" not in printed, run["id"] + for row in truth["cable_rows"]: + assert f"{row['length_mm']:g}" not in printed, row["tag"] + report = json.loads(Path(summary["report"]).read_text(encoding="utf-8")) + assert report["scale"]["verdict"] == "schematic" + + +def test_show_prints_the_rows_on_request(tmp_path, capsys): + truth = build_plant_pair(tmp_path) + code = _script().main(["--pid", truth["pid"], "--layout", truth["layout"], "--show"]) + assert code == 0 + assert '"cable"' in capsys.readouterr().out diff --git a/tests/test_understand_scale.py b/tests/test_understand_scale.py index 52ca2f6..1b1c1b6 100644 --- a/tests/test_understand_scale.py +++ b/tests/test_understand_scale.py @@ -190,3 +190,30 @@ def test_the_synthetic_pid_is_schematic_against_its_layout(tmp_path): assert len(result["matched"]) >= 3 # The layout's two plan copies: the check must pick one of them. assert result["scope"]["cluster_b"] is not None + + +def test_a_wiring_callout_is_not_a_second_place_for_its_panel(): + """``Wiring to CP-2`` names the panel it points at; it is not written at the + panel. Read as a tag it would make every panel with a callout ambiguous + (and ``Wiring to CP1`` a phantom tag ``CP1``), so the callout takes no part + - the same rule the line-network reader (`network.tag_occurrences`) keeps.""" + plan = {**PLAN, "CP-2": (3000.0, 15000.0)} + callouts = [ + text("W1", "Wiring to CP-2", 12000.0, 0.0), + text("W2", "Wiring to CP1", 0.0, 9000.0), + ] + result = scale_check(snapshot(tags(plan, "A")), snapshot(tags(plan, "B") + callouts)) + assert result["ambiguous"] == [] + assert "CP-2" in {row["tag"] for row in result["matched"]} + assert result["only_b"] == [] + assert result["pair_count"] == math.comb(len(plan), 2) + + +def test_the_synthetic_pair_reproduces_the_generators_own_statistics(tmp_path): + """The generator computes its truth from the positions it placed: 11 shared + tags, 55 pairs, 21.8 % within +/-10 % of the median. The check must see the + same tags - a wiring callout counted as a panel would drop to 45.""" + truth = build_plant_pair(tmp_path) + result = scale_check(read_snapshot(truth["pid"]), read_snapshot(truth["layout"])) + assert result["pair_count"] == truth["scale"]["overall"]["pairs"] == 55 + assert result["within_10pct"] == pytest.approx(truth["scale"]["overall"]["within_10pct"])