From 023eac7a51ece5a836724e3906981dec51c2c870 Mon Sep 17 00:00:00 2001 From: Umut CAN Date: Thu, 24 Sep 2026 10:43:08 +0300 Subject: [PATCH] =?UTF-8?q?docs(plan):=20track=20H=20=E2=80=94=2012=20TDD?= =?UTF-8?q?=20tasks=20in=20three=20groups?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The skeleton's header, constraints, file map and pinned interfaces now carry their twelve tasks, in order: - main (Tasks 1-2): the snapshot, vocabulary and label parsers, MAX_DXF_BYTES 512 MB and the office extra; the synthetic plant pair with its truth. - U (Tasks 3-4): drawing_understand and drawing_scale_check; SECTION 26 opens. - N (Tasks 5-8): the line network, orthogonal lengths and pipe rows, the takeoff workbook with pipe_takeoff, then cable_takeoff; SECTION 27, pack plant. - D (Tasks 9-10): drawing_diff and drawing_topology_check with the three topo_* focuses, on D's own copy of SECTION 26. - final (Tasks 11-12): packs, lean 65, aliases, golden queries and counts; benchmark v7, five correctness checks, the live smoke, the field-test script and the docs. Interface pass: the U and N texts now say what Task 6 really does with the scale check (it takes one through pipe_rows(scale=...) and otherwise keeps its own stand-ins), and Task 11 moves D's block above SECTION 27 after the merge. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd --- ...26-09-24-v1.6-understand-implementation.md | 17563 ++++++++++++++++ 1 file changed, 17563 insertions(+) diff --git a/docs/superpowers/plans/2026-09-24-v1.6-understand-implementation.md b/docs/superpowers/plans/2026-09-24-v1.6-understand-implementation.md index 05fece5..b00c378 100644 --- a/docs/superpowers/plans/2026-09-24-v1.6-understand-implementation.md +++ b/docs/superpowers/plans/2026-09-24-v1.6-understand-implementation.md @@ -227,3 +227,17566 @@ group sizes (from the live registry); `engineering/critique.py` focus registration (D only). Anything else: `git merge --abort`, record, continue. --- + +### Task 1: Foundation — records, the snapshot, vocabulary, label parsers, the size limit, the office extra + +**Files:** +- Create: `engineering/understand/__init__.py` +- Create: `engineering/understand/labels.py` +- Create: `engineering/understand/vocab.py` +- Create: `engineering/understand/snapshot.py` +- Create: `tests/test_understand_labels.py` +- Create: `tests/test_understand_vocab.py` +- Create: `tests/test_understand_snapshot.py` +- Create: `tests/test_office_extra.py` +- Modify: `config.py` (`DEFAULT_MAX_DXF_BYTES` = 512 MB; `max_dxf_bytes` defaults to it) +- Modify: `pyproject.toml` (new `office = ["openpyxl>=3.1"]` extra; `openpyxl>=3.1` also in `full`) +- Modify: `uv.lock` (`uv lock`: adds `openpyxl` 3.1.5 and `et-xmlfile` 2.0.0) +- Modify: `README.md` (the `MAX_DXF_BYTES` row of the configuration table), `.env.example` (the `MAX_DXF_BYTES` line) +- Test: `tests/test_understand_labels.py`, `tests/test_understand_vocab.py`, `tests/test_understand_snapshot.py`, `tests/test_office_extra.py` + +**Interfaces:** +- Consumes: `backends/ocs.py::to_wcs_2d(extrusion, x, y, z=0.0) -> list[float]`, `wcs_arc_angles(extrusion, center_ocs, radius, start_deg, end_deg) -> tuple[float, float]`, `wcs_bulge(extrusion, bulge) -> float`; `engineering/measure.py::polygon_area_perimeter(vertices, closed) -> (area, perimeter)`; `backends/capability.py::UnsupportedCapabilityError(capability, message)`; `backends/com_backend.py::_acad_app()`, `_acad_doc()`, `_AC_SELECTION_SET_ALL` (= 5), `ComBackend._run(func, *args)`, `ComBackend._wait_out_rejected_call(read)`, `ComBackend._ensure_document_state()`; `EzdxfBackend._async(func)`, `EzdxfBackend._doc`, `EzdxfBackend._doc_path`; `backend.name` (`"com"` | `"ezdxf"`); `config.settings.max_dxf_bytes`. +- Produces — exactly the skeleton's Interfaces block, plus the additive names listed after it: + - `engineering/understand/snapshot.py::Pt = tuple[float, float]`; `EntityRecord` (frozen: `handle, type, layer, space, points=(), bulges=(), closed=False, radius=None, angles=None, text=None, height=None, block=None, attribs=(), bbox=None, rotation=0.0, scale=(1.0, 1.0)`); `Snapshot` (frozen: `source, insunits, extmin, extmax, layers, layouts, records`); `records_from_doc(doc, *, source: str) -> Snapshot`; `read_snapshot(path: str) -> Snapshot`; `async take_snapshot(backend, path: str | None = None) -> Snapshot`; `length_of(rec: EntityRecord) -> float`. + - `engineering/understand/vocab.py::SERVICES`, `DISCIPLINES`, `classify_layer(name) -> {"discipline", "service", "confidence", "keyword", "language"}`, `supply_return(text) -> "supply" | "return" | None`, `room_label(text) -> {"number", "name", "language"} | None`. + - `engineering/understand/labels.py::plain(text) -> str`, `parse_diameters(text) -> tuple[str, ...]`, `parse_tag(text) -> str | None`, `normalize_panel(text) -> str | None`, `parse_electrical(text) -> {"kw", "phases", "voltage", "neutral", "vfd"}`, `wiring_target(text) -> str | None`. + - Additive (not in the skeleton; nothing above changes shape): `vocab.LANGUAGES = ("en", "tr", "ru", "nl", "de")`, `vocab.EQUIPMENT_KINDS`, `vocab.equipment_kind(block_name) -> {"kind", "keyword", "language"} | None` (the spec's "equipment kinds", §3; group N's Task 5 keeps its own `is_reducer`, so no later task depends on it), `vocab.fold(text) -> str`; `labels.CYRILLIC_LOOKALIKES`, `labels.fold_lookalikes(text) -> str`; `config.DEFAULT_MAX_DXF_BYTES = 536_870_912`. + - Record conventions every later task relies on: every point is WCS (OCS translated here); `points` of a MULTILEADER are each leader line's vertices (its arrow tip first), each leader's landing point, then the text insertion **last**; an INSERT's `bbox` is its block geometry **without** its ATTRIB texts; `text` is always `plain()`-ed; `space` is `"Model"` or the paper layout's name; `layouts` is every layout in tab order, `"Model"` first. + +- [ ] **Step 1: Write the failing test for the label parsers (Review Focus 1 and 2)** + +Create `tests/test_understand_labels.py`: + +```python +"""Label parsers: every spelling a foreign drawing uses, read one way. + +Review Focus 1 (AutoCAD text codes) and 2 (Cyrillic look-alike tags) of the +track H plan live here: one test per spelling, because a parser that handles +four of five codes fails silently on the fifth drawing. +""" + +from __future__ import annotations + +import pytest + +from engineering.understand.labels import ( + CYRILLIC_LOOKALIKES, + normalize_panel, + parse_diameters, + parse_electrical, + parse_tag, + plain, + wiring_target, +) + +# -- plain(): AutoCAD codes and MTEXT format runs ------------------------------- + + +def test_plain_decodes_the_percent_codes(): + assert plain("%%c51") == "\u00d851" + assert plain("90%%d") == "90\u00b0" + assert plain("%%p0.1") == "\u00b10.1" + assert plain("50%%%") == "50%" + assert plain("%%uUNDER%%u") == "UNDER" + assert plain("%%065") == "A" + + +def test_plain_decodes_a_unicode_escape(): + assert plain("\\U+2205 51") == "\u2205 51" + assert plain("\\u+00D8") == "\u00d8" + + +def test_plain_drops_mtext_format_runs_and_keeps_the_content(): + assert plain("{\\fArial|b0|i0|c0|p34;\u00d851}") == "\u00d851" + assert plain("\\H2.5;\\C1;SMS51") == "SMS51" + assert plain("\\A1;\\W0.8;\\Q15;\\T1.1;DN20") == "DN20" + assert plain("\\LUNDERLINED\\l") == "UNDERLINED" + + +def test_plain_turns_paragraphs_spaces_and_stacks_into_text(): + assert plain("PUMP\\PM11") == "PUMP\nM11" + assert plain("a\\~b") == "a b" + assert plain("\\S1/2;") == "1/2" + assert plain("\\S+0.1^-0.2;") == "+0.1/-0.2" + + +def test_plain_keeps_escaped_backslashes_and_braces(): + assert plain("\\\\P") == "\\P" + assert plain("\\{x\\}") == "{x}" + + +def test_plain_of_nothing_is_empty(): + assert plain(None) == "" + assert plain("") == "" + + +# -- parse_diameters(): Review Focus 1, one spelling per test -------------------- + + +def test_the_autocad_percent_code_is_a_diameter(): + assert parse_diameters("%%c51") == ("\u00d851",) + + +def test_a_unicode_escaped_diameter_sign_is_a_diameter(): + assert parse_diameters("\\U+2205 51") == ("\u00d851",) + assert parse_diameters("\\U+220551") == ("\u00d851",) + + +def test_the_diameter_sign_u2300_is_a_diameter(): + assert parse_diameters("\u230051") == ("\u00d851",) + + +def test_a_lowercase_o_stroke_is_a_diameter(): + assert parse_diameters("\u00f851") == ("\u00d851",) + + +def test_a_diameter_inside_an_mtext_format_run_is_a_diameter(): + assert parse_diameters("{\\fArial|b0|i0|c0|p34;\u00d851}") == ("\u00d851",) + assert parse_diameters("{\\fArial|b0;\\U+2205}51") == ("\u00d851",) + + +def test_a_diameter_after_a_paragraph_break_is_found(): + assert parse_diameters("PRODUCT\\P%%c51") == ("\u00d851",) + + +def test_labels_are_kept_as_drawn_and_never_equated(): + assert parse_diameters("\u00d825,4") == ("\u00d825,4",) + assert parse_diameters("\u00d825") == ("\u00d825",) + assert parse_diameters("SMS 51") == ("SMS51",) + assert parse_diameters("dn 20") == ("DN20",) + assert parse_diameters("DN-25") == ("DN25",) + assert parse_diameters("20 x 27") == ("20x27",) + assert parse_diameters("20\u044527") == ("20x27",) # Cyrillic х + assert parse_diameters('1,5"') == ('1,5"',) + assert parse_diameters("1/2\u2033") == ('1/2"',) + + +def test_several_labels_in_one_text_come_in_order_and_once(): + assert parse_diameters("\u00d851 / SMS51 / \u00d851") == ("\u00d851", "SMS51") + + +def test_a_cable_cross_section_is_not_a_pipe_size(): + assert parse_diameters("5x4 mm\u00b2") == () + assert parse_diameters("3x2,5 mm2") == () + + +def test_text_without_a_size_yields_nothing(): + assert parse_diameters("CIP SUPPLY") == () + assert parse_diameters("T101") == () + + +# -- parse_tag(): Review Focus 2, one look-alike per test ------------------------- + + +@pytest.mark.parametrize(("cyrillic", "latin"), sorted(CYRILLIC_LOOKALIKES.items())) +def test_every_cyrillic_lookalike_folds_to_its_latin_twin(cyrillic, latin): + assert parse_tag(f"{cyrillic}101") == f"{latin}101" + + +def test_a_cyrillic_t_tag_matches_the_latin_one(): + assert parse_tag("\u04224100") == "T4100" + + +def test_a_cyrillic_m_tag_matches_the_latin_one(): + assert parse_tag("\u041c82") == "M82" + + +def test_a_tag_with_a_cyrillic_prefix_and_a_hyphen(): + assert parse_tag("\u0421\u0420-1") == "CP-1" # С Р typed in Cyrillic + + +def test_a_russian_word_is_never_folded_into_a_tag(): + assert parse_tag("\u041f\u041e\u0414\u0410\u0427\u0410") is None # ПОДАЧА + assert parse_tag("\u041f\u041e\u04171") is None # ПОЗ1: П has no Latin twin + + +def test_tag_shapes(): + assert parse_tag("T4100") == "T4100" + assert parse_tag("pump m41a") == "M41A" + assert parse_tag("M47.1") == "M47.1" + assert parse_tag("CP-1") == "CP-1" + assert parse_tag("CP-M82") == "CP-M82" + assert parse_tag("JB-2.") == "JB-2" + assert parse_tag("CIP TANK T101") == "T101" + + +def test_sizes_ratings_and_ip_codes_are_not_tags(): + assert parse_tag("DN20") is None + assert parse_tag("SMS51") is None + assert parse_tag("PN16") is None + assert parse_tag("IP65") is None + assert parse_tag("DN20 T101") == "T101" + + +def test_text_without_a_tag(): + assert parse_tag("ROOM 1 PROCESS HALL") is None + assert parse_tag(None) is None + + +# -- panels and wiring callouts ---------------------------------------------------- + + +def test_panel_spellings_become_one(): + assert normalize_panel("CP 1") == "CP-1" + assert normalize_panel("CP1") == "CP-1" + assert normalize_panel("cp-1") == "CP-1" + assert normalize_panel("CP M82") == "CP-M82" + assert normalize_panel("\u0421\u0420 2") == "CP-2" + assert normalize_panel("PANEL") is None + assert normalize_panel("") is None + + +def test_wiring_callouts_in_four_languages(): + assert wiring_target("Wiring to CP1") == "CP-1" + assert wiring_target("WIRING TO CP 2") == "CP-2" + assert wiring_target("wiring to the CP-M82") == "CP-M82" + # подключение к шкафу CP-1 + assert ( + wiring_target( + "\u043f\u043e\u0434\u043a\u043b\u044e\u0447\u0435\u043d\u0438\u0435 " + "\u043a \u0448\u043a\u0430\u0444\u0443 CP-1" + ) + == "CP-1" + ) + assert wiring_target("bedrading naar CP3") == "CP-3" + assert wiring_target("CP 2 panosuna") == "CP-2" + + +def test_a_callout_without_a_wiring_phrase_names_no_panel(): + assert wiring_target("CP1") is None + assert wiring_target("Wiring to") is None + + +# -- electrical data --------------------------------------------------------------- + + +def test_russian_electrical_text(): + # P = 18,5 кВт, 3 ф. 400 В + N + PE, ЧРП + text = "P = 18,5 \u043a\u0412\u0442, 3 \u0444. 400 \u0412 + N + PE, \u0427\u0420\u041f" + assert parse_electrical(text) == { + "kw": 18.5, + "phases": 3, + "voltage": 400, + "neutral": True, + "vfd": True, + } + + +def test_english_and_turkish_electrical_text(): + assert parse_electrical("5.5 kW 3ph 400V VFD") == { + "kw": 5.5, + "phases": 3, + "voltage": 400, + "neutral": False, + "vfd": True, + } + assert parse_electrical("2,2 kW 3 faz 400 V frekans konvert\u00f6r\u00fc")["vfd"] is True + assert parse_electrical("3/N/PE ~ 400 V")["neutral"] is True + assert parse_electrical("24 VDC")["voltage"] == 24 + + +def test_a_power_that_is_not_written_is_never_invented(): + # Нагреватель, 3 ф. 400 В + heater = parse_electrical( + "\u041d\u0430\u0433\u0440\u0435\u0432\u0430\u0442\u0435\u043b\u044c, 3 \u0444. 400 \u0412" + ) + assert heater["kw"] is None + assert heater["phases"] == 3 and heater["voltage"] == 400 + + +def test_energy_is_not_power(): + assert parse_electrical("1200 kWh")["kw"] is None +``` + +- [ ] **Step 2: Run it to verify it fails** + +Run: `uv run --frozen pytest tests/test_understand_labels.py -q` +Expected: FAIL at collection — `ModuleNotFoundError: No module named 'engineering.understand'`. + +- [ ] **Step 3: Create the package and `engineering/understand/labels.py`** + +`engineering/understand/__init__.py` (the docstring only for now; Step 17 adds the re-exports once all three modules exist): + +```python +"""Understanding drawings somebody else made (v1.6 track H). + +One bulk read per drawing (:mod:`.snapshot`), multilingual vocabulary +(:mod:`.vocab`) and label parsers (:mod:`.labels`) are the foundation; the +readers built on them - the one-call report, the scale check, the line +networks and takeoffs, the diff and the topology checks - add their modules +here. Everything except the snapshot's live export is pure: records in, +results out, the drawing never modified. +""" +``` + +`engineering/understand/labels.py`: + +```python +"""Label parsers for foreign drawings: diameters, equipment tags, panels, electrical data. + +Pure functions over one string each - no ezdxf, no I/O - so every rule here is +tested on the exact spellings the field drawings of 2026-09-24 used, and the +readers above it (the network, the takeoffs, the scale check) share one +reading of a label instead of three. + +What a label *says* is kept exactly as drawn. ``Ø51``, ``SMS51``, ``DN20``, +``Ø25,4`` and ``Ø25`` are five different sizes: the parser normalises only the +spelling of the diameter sign and the whitespace, never the number, and never +equates two sizing systems (an SMS size and a DN size are different series; the +drawing has to say they are the same pipe, not this module). + +Two kinds of foreign-drawing noise are decoded before anything is parsed: + +* **AutoCAD text codes.** ``%%c`` is AutoCAD's diameter sign, ``%%d`` the degree + sign, ``%%p`` plus-minus; ``\\U+2205`` is a Unicode escape (AutoCAD's help + lists U+2205 as its diameter symbol). MTEXT carries format runs + (``{\\fArial|b0;Ø51}``, ``\\H2.5;``, ``\\P`` for a new paragraph) that are + formatting, not content. +* **Cyrillic look-alikes in tags.** A Russian-labelled P&ID typed on a Russian + keyboard writes ``Т4100`` with a Cyrillic Т (U+0422); it must match the + layout's Latin ``T4100``. Eleven capitals have a Latin twin of identical + shape (Т М С Р А В Е К Н О Х) and are folded; a Cyrillic letter without one + (П, Щ, Ш, ...) is left alone, so a Russian word never turns into a tag. +""" + +from __future__ import annotations + +import re + +__all__ = [ + "CYRILLIC_LOOKALIKES", + "fold_lookalikes", + "normalize_panel", + "parse_diameters", + "parse_electrical", + "parse_tag", + "plain", + "wiring_target", +] + +#: Cyrillic capitals drawn identically to a Latin capital, and that capital. +CYRILLIC_LOOKALIKES: dict[str, str] = { + "\u0422": "T", # Т + "\u041c": "M", # М + "\u0421": "C", # С + "\u0420": "P", # Р + "\u0410": "A", # А + "\u0412": "B", # В + "\u0415": "E", # Е + "\u041a": "K", # К + "\u041d": "H", # Н + "\u041e": "O", # О + "\u0425": "X", # Х +} +_LOOKALIKE_TABLE = str.maketrans(CYRILLIC_LOOKALIKES) + +#: ``%%`` codes that stand for a character (AutoCAD's control codes). +_PERCENT_CHARS = {"c": "\u00d8", "d": "\u00b0", "p": "\u00b1"} +#: MTEXT codes that carry an argument up to the next ``;`` and draw nothing. +_ARG_CODES = frozenset("fFHWQTACcp") +#: MTEXT on/off toggles (underline, overline, strike-through, column break). +_TOGGLE_CODES = frozenset("LlOoKkN") + +_UNICODE_ESCAPE = re.compile(r"\\[Uu]\+([0-9A-Fa-f]{4})") +_PERCENT_CODE = re.compile(r"%%(\d{3}|[cdpCDP]|[uoUOkK])") + +#: Every spelling of the diameter sign met in drawings: Ø (U+00D8), ø (U+00F8), +#: ∅ (U+2205, AutoCAD's listed diameter symbol), ⌀ (U+2300, +#: the Unicode DIAMETER SIGN). +_DIAMETER_SIGNS = "\u00d8\u00f8\u2205\u2300" +_NUMBER = r"\d+(?:[.,]\d+)?" +_DIAMETER_RE = re.compile( + rf"(?P[{_DIAMETER_SIGNS}])\s*(?P{_NUMBER})" + rf"|(?SMS)\s*-?\s*(?P{_NUMBER})" + rf"|(?DN)\s*-?\s*(?P\d+)" + rf"|(?{_NUMBER})\s*[xX\u00d7\u0445\u0425]\s*(?P{_NUMBER})(?![.,]?\d)" + r"(?!\s*(?:mm2|mm\u00b2|\u043c\u043c2|\u043c\u043c\u00b2))" + rf"|(?\d+(?:[.,]\d+)?(?:\s*/\s*\d+)?)\s*(?:\"|''|\u201d|\u2033)", + re.IGNORECASE, +) + +#: A token that may be a tag: starts with a letter, keeps letters, digits, dots +#: and hyphens (``CP-M82``, ``M47.1``). +_TAG_TOKEN = re.compile(r"[A-Za-z\u0400-\u04ff][A-Za-z0-9\u0400-\u04ff.\-]*") +#: An equipment tag: 1-4 letters, an optional second letter group after a +#: hyphen (``CP-M82``), an optional hyphen, 1-5 digits, an optional letter +#: suffix (``M41A``) and an optional ``.n`` (``M47.1``). +_TAG_RE = re.compile(r"[A-Z]{1,4}(?:-[A-Z]{1,4})?-?\d{1,5}[A-Z]?(?:\.\d{1,3})?") +#: Prefixes that look like a tag and are not one: nominal sizes (DN20, SMS51), +#: a pressure rating (PN16) and an ingress-protection code (IP65). +_NOT_TAG_PREFIXES = ("DN", "SMS", "PN", "IP") + +_PANEL_RE = re.compile( + r"(? str: + """The text a reader sees: codes decoded, MTEXT formatting removed. + + ``\\U+XXXX`` becomes its character; ``%%c`` / ``%%d`` / ``%%p`` become Ø / ° / + ±, ``%%nnn`` the character with that code, ``%%%`` a percent sign, and the + ``%%u`` / ``%%o`` / ``%%k`` toggles disappear. In MTEXT, ``\\P`` is a line + break, ``\\~`` a space, ``\\S a^b;`` / ``\\S a/b;`` the stacked ``a/b``, + ``\\\\`` ``\\{`` ``\\}`` the literal characters; every argument code + (``\\f...;``, ``\\H...;``, ``\\C...;`` ...) and every toggle is dropped, and so + are the grouping braces. + """ + if not text: + return "" + source = _UNICODE_ESCAPE.sub(lambda m: chr(int(m.group(1), 16)), str(text)) + source = source.replace("%%%", "\x00") + + def percent(match: re.Match) -> str: + code = match.group(1) + if code.isdigit(): + return chr(int(code)) + return _PERCENT_CHARS.get(code.lower(), "") + + source = _PERCENT_CODE.sub(percent, source).replace("\x00", "%") + out: list[str] = [] + i, n = 0, len(source) + while i < n: + ch = source[i] + if ch == "\\" and i + 1 < n: + code = source[i + 1] + if code in "\\{}": + out.append(code) + i += 2 + elif code == "P": + out.append("\n") + i += 2 + elif code == "~": + out.append(" ") + i += 2 + elif code == "S": + end = source.find(";", i + 2) + end = n if end < 0 else end + out.append(re.sub(r"[\^#]", "/", source[i + 2 : end])) + i = end + 1 + elif code in _ARG_CODES: + end = source.find(";", i + 2) + i = n if end < 0 else end + 1 + elif code in _TOGGLE_CODES: + i += 2 + else: + out.append(code) + i += 2 + elif ch in "{}": + i += 1 + else: + out.append(ch) + i += 1 + return "".join(out).strip() + + +def parse_diameters(text: str | None) -> tuple[str, ...]: + """Every size label in ``text``, in order, each once, spelled one way. + + ``Ø51`` / ``%%c51`` / ``\\U+2205 51`` / ``⌀51`` / ``ø51`` -> ``"Ø51"``; + ``Ø25,4`` keeps its comma; ``SMS 51`` -> ``"SMS51"``; ``DN 20`` -> + ``"DN20"``; ``20 x 27`` / ``20х27`` (Cyrillic х) -> ``"20x27"``; ``1,5"`` -> + ``'1,5"'``. An ``a x b`` followed by mm² is a cable cross-section, not a + pipe, and is skipped. + """ + found: list[str] = [] + for match in _DIAMETER_RE.finditer(plain(text)): + if match.group("sign"): + token = "\u00d8" + match.group("sign_n") + elif match.group("sms"): + token = "SMS" + match.group("sms_n") + elif match.group("dn"): + token = "DN" + match.group("dn_n") + elif match.group("a"): + token = f"{match.group('a')}x{match.group('b')}" + else: + token = re.sub(r"\s+", "", match.group("inch")) + '"' + if token not in found: + found.append(token) + return tuple(found) + + +def fold_lookalikes(text: str) -> str: + """Uppercase, with the eleven Cyrillic look-alike capitals made Latin.""" + return str(text).upper().translate(_LOOKALIKE_TABLE) + + +def parse_tag(text: str | None) -> str | None: + """The first equipment tag in ``text``, uppercase, look-alikes folded. + + ``"Т4100"`` (Cyrillic Т) -> ``"T4100"``; ``"pump M41A"`` -> ``"M41A"``; + ``"CP-M82"`` stays ``"CP-M82"``. A size label (``DN20``, ``SMS51``), a + pressure rating (``PN16``) or an IP code (``IP65``) is not a tag; a token + that keeps a Cyrillic letter without a Latin twin is not one either. + """ + for match in _TAG_TOKEN.finditer(plain(text)): + token = fold_lookalikes(match.group(0)).rstrip(".-") + if not _TAG_RE.fullmatch(token): + continue + if re.match(r"[A-Z]+", token).group(0) in _NOT_TAG_PREFIXES: + continue + return token + return None + + +def normalize_panel(text: str | None) -> str | None: + """A panel name spelled one way: ``CP 1`` / ``CP1`` / ``cp-1`` -> ``CP-1``. + + The letters, one hyphen, then the rest: ``CP M82`` -> ``CP-M82``. Cyrillic + look-alikes are folded first (``СР-1`` typed in Cyrillic is ``CP-1``). + Returns None when the text holds no panel-shaped token. + """ + if not text: + return None + match = _PANEL_RE.search(fold_lookalikes(plain(text))) + if match is None: + return None + return f"{match.group(1)}-{match.group(2)}" + + +def parse_electrical(text: str | None) -> dict: + """Power, phases, voltage, neutral and drive type stated in ``text``. + + ``"P = 18,5 кВт, 3 ф. 400 В + N + PE, ЧРП"`` -> + ``{"kw": 18.5, "phases": 3, "voltage": 400, "neutral": True, "vfd": True}``. + What the text does not state stays ``None`` / ``False``: a power that is not + written is never estimated. + """ + body = plain(text) + lowered = body.lower() + kw = _KW_RE.search(body) + phases = _PHASE_RE.search(body) + volts = _VOLT_RE.search(body) + return { + "kw": float(kw.group(1).replace(",", ".")) if kw else None, + "phases": int(phases.group(1)) if phases else None, + "voltage": int(volts.group(1)) if volts else None, + "neutral": bool(_NEUTRAL_RE.search(body)), + "vfd": any(word in lowered for word in _VFD_WORDS), + } + + +def wiring_target(text: str | None) -> str | None: + """The panel a wiring callout points to, normalised. + + ``"Wiring to CP1"`` -> ``"CP-1"``; ``"подключение к шкафу CP-1"`` -> + ``"CP-1"``; ``"CP 2 panosuna"`` -> ``"CP-2"`` (Turkish names the panel + first). None when the text has no wiring phrase or no panel beside it. + """ + body = plain(text) + lowered = body.lower() + for phrase in _WIRING_PHRASES: + at = lowered.find(phrase) + if at >= 0: + panel = normalize_panel(body[at + len(phrase) :]) + if panel is not None: + return panel + for phrase in _WIRING_PHRASES_AFTER: + at = lowered.find(phrase) + if at > 0: + matches = list(_PANEL_RE.finditer(fold_lookalikes(body[:at]))) + if matches: + return f"{matches[-1].group(1)}-{matches[-1].group(2)}" + return None +``` + +- [ ] **Step 4: Run it to verify it passes** + +Run: `uv run --frozen pytest tests/test_understand_labels.py -q` +Expected: `41 passed` (the look-alike test is parametrized over the eleven letters). A failing spelling is fixed in `labels.py`, never by deleting its test: each one is a spelling a field drawing used. + +- [ ] **Step 5: Write the failing test for the vocabulary** + +Create `tests/test_understand_vocab.py`: + +```python +"""The multilingual vocabulary: layer names, direction words, room labels, block kinds. + +Every layer name below is one a drafter would write; the ones in the synthetic +plant pair (tests/fixtures/plant_pair.py) are here too, because that is what the +readers above this module are measured on. +""" + +from __future__ import annotations + +import pytest + +from engineering.understand.vocab import ( + DISCIPLINES, + EQUIPMENT_KINDS, + LANGUAGES, + SERVICES, + classify_layer, + equipment_kind, + fold, + room_label, + supply_return, +) + + +def test_the_closed_lists(): + assert LANGUAGES == ("en", "tr", "ru", "nl", "de") + assert SERVICES == ( + "product", + "cip_supply", + "cip_return", + "glycol_supply", + "glycol_return", + "steam", + "condensate", + "cold_water", + "hot_water", + "ice_water", + "compressed_air", + "drain", + "electrical", + "unknown", + ) + assert DISCIPLINES == ( + "piping", + "electrical", + "architecture", + "equipment", + "annotation", + "structure", + "unknown", + ) + assert EQUIPMENT_KINDS == ("tank", "pump", "valve", "reducer", "motor", "agitator", "panel") + + +def test_fold_strips_diacritics_and_the_turkish_dotless_i(): + assert fold("DÖNÜŞ") == "donus" + assert fold("GİDİŞ") == "gidis" + assert fold("çıkış") == "cikis" + assert fold("RückLauf") == "rucklauf" + assert fold("ПОДАЧА") == "подача" + assert fold(None) == "" + + +@pytest.mark.parametrize( + ("name", "service", "keyword", "language"), + [ + ("P_product piping", "product", "product", "en"), + ("P_cipsupplyline", "cip_supply", "cip", "en"), + ("P_cipreturnline", "cip_return", "cip", "en"), + ("P_ijswater", "ice_water", "ijswater", "nl"), + ("CIP ВОЗВРАТ", "cip_return", "cip", "en"), + ("glikol dönüş", "glycol_return", "glikol", "tr"), + ("GLYKOL-VORLAUF", "glycol_supply", "glykol", "de"), + ("buhar hattı", "steam", "buhar", "tr"), + ("КОНДЕНСАТ", "condensate", "конденсат", "ru"), + ("P_ICE WATER", "ice_water", "ice water", "en"), + ("sıcak su", "hot_water", "sicak su", "tr"), + ("DRUCKLUFT", "compressed_air", "druckluft", "de"), + ("air_line", "compressed_air", "air", "en"), + ("RIOOL", "drain", "riool", "nl"), + ], +) +def test_service_layers(name, service, keyword, language): + row = classify_layer(name) + assert row["service"] == service + assert row["discipline"] == "piping" + assert row["confidence"] == 0.9 + assert row["keyword"] == keyword + assert row["language"] == language + + +def test_electrical_layers_are_their_own_service(): + for name in ("E_power", "ELEKTRIK", "ЭЛЕКТРИКА", "kabel"): + row = classify_layer(name) + assert (row["discipline"], row["service"], row["confidence"]) == ( + "electrical", + "electrical", + 0.9, + ), name + + +def test_a_cip_layer_without_a_direction_is_not_guessed(): + row = classify_layer("P_CIP") + assert row == { + "discipline": "piping", + "service": "unknown", + "confidence": 0.6, + "keyword": "cip", + "language": "en", + } + + +def test_a_cip_layer_naming_both_directions_is_not_guessed_either(): + assert classify_layer("CIP supply return")["service"] == "unknown" + + +def test_a_water_layer_with_no_temperature_is_piping_of_unknown_service(): + row = classify_layer("P_WATER") + assert (row["discipline"], row["service"], row["confidence"]) == ("piping", "unknown", 0.6) + + +@pytest.mark.parametrize( + ("name", "discipline", "language"), + [ + ("KLEPPEN", "piping", "nl"), + ("VANALAR", "piping", "tr"), + ("WALLS", "architecture", "en"), + ("MUREN", "architecture", "nl"), + ("СТЕНЫ", "architecture", "ru"), + ("EQUIPMENT", "equipment", "en"), + ("POMPALAR", "equipment", "tr"), + ("DIM", "annotation", "en"), + ("ÖLÇÜ", "annotation", "tr"), + ("KOLONLAR", "structure", "tr"), + ], +) +def test_discipline_layers(name, discipline, language): + row = classify_layer(name) + assert (row["discipline"], row["service"], row["confidence"], row["language"]) == ( + discipline, + "unknown", + 0.9, + language, + ) + + +def test_a_short_keyword_must_start_a_word(): + assert classify_layer("STAIRS")["service"] != "compressed_air" + assert classify_layer("CHAIRS")["discipline"] == "unknown" + + +def test_a_layer_with_no_keyword(): + assert classify_layer("0") == { + "discipline": "unknown", + "service": "unknown", + "confidence": 0.0, + "keyword": None, + "language": None, + } + assert classify_layer("Defpoints")["discipline"] == "unknown" + + +@pytest.mark.parametrize( + ("text", "direction"), + [ + ("ПОДАЧА", "supply"), + ("ВОЗВРАТ", "return"), + ("обратка CIP", "return"), # обратка + ("BESLEME", "supply"), + ("DÖNÜŞ", "return"), + ("donus hatti", "return"), + ("CIP SUPPLY", "supply"), + ("return line", "return"), + ("AANVOER", "supply"), + ("RETOUR", "return"), + ("Vorlauf", "supply"), + ("RÜCKLAUF", "return"), + ], +) +def test_supply_and_return_words_in_five_languages(text, direction): + assert supply_return(text) == direction + + +def test_a_text_naming_both_or_neither_direction_is_none(): + assert supply_return("CIP SUPPLY / RETURN") is None + assert supply_return("Ø51") is None + assert supply_return(None) is None + + +@pytest.mark.parametrize( + ("text", "expected"), + [ + ("ROOM 1 PROCESS HALL", {"number": "1", "name": "PROCESS HALL", "language": "en"}), + ("ODA 2", {"number": "2", "name": None, "language": "tr"}), + ("MAHAL 1.05 SOĞUK ODA", {"number": "1.05", "name": "SOĞUK ODA", "language": "tr"}), + ( + "ПОМЕЩЕНИЕ 2 МОЙКА", + {"number": "2", "name": "МОЙКА", "language": "ru"}, + ), + ("RUIMTE 3 opslag", {"number": "3", "name": "opslag", "language": "nl"}), + ("Raum 4", {"number": "4", "name": None, "language": "de"}), + ], +) +def test_room_labels_in_five_languages(text, expected): + assert room_label(text) == expected + + +def test_a_text_without_a_room_word_is_not_a_room_label(): + assert room_label("T101") is None + assert room_label("PROCESS HALL") is None + assert room_label("") is None + + +@pytest.mark.parametrize( + ("block", "kind", "language"), + [ + ("KLEP_VLINDER", "valve", "nl"), + ("VERLOOPSTUK", "reducer", "nl"), + ("REDUCER_CONC", "reducer", "en"), + ("TANK_V", "tank", "en"), + ("POMP", "pump", "nl"), + ("SCHAKELKAST", "panel", "nl"), + ("НАСОС", "pump", "ru"), + ("VANA_KELEBEK", "valve", "tr"), + ], +) +def test_block_names_say_their_kind(block, kind, language): + row = equipment_kind(block) + assert row["kind"] == kind and row["language"] == language + + +def test_an_anonymous_block_has_no_kind(): + assert equipment_kind("*U12") is None + assert equipment_kind(None) is None +``` + +- [ ] **Step 6: Run it to verify it fails** + +Run: `uv run --frozen pytest tests/test_understand_vocab.py -q` +Expected: FAIL at collection — `ModuleNotFoundError: No module named 'engineering.understand.vocab'`. + +- [ ] **Step 7: Write `engineering/understand/vocab.py`** + +The tables are vocabulary a drafter writes, each row with its language — not a standard, so no row needs a source beyond the language it is in; the Russian stems carry an English gloss. + +```python +"""Multilingual keyword tables for reading foreign drawings: EN / TR / RU / NL / DE. + +A foreign drawing says what a layer carries in its name and in the texts on it, +in whatever language the office that drew it speaks: the field P&ID of +2026-09-24 named its piping layers in English and Dutch (``P_cipsupplyline``, +``KLEPPEN``) and labelled its lines in Russian. These tables are what lets the +reader tell a CIP supply line from a product line without a human. + +They are **vocabulary, not a standard**: words a drafter writes, each with the +language it belongs to. A classification made from them carries a confidence +(0.9 for a direct keyword, 0.6 for weaker evidence) and the keyword that +decided it, so a reader can always see why. + +Matching is done on a *folded* spelling - casefolded, diacritics stripped, +Turkish dotless ı made i - so ``DÖNÜŞ``, ``dönüş`` and ``donus`` are one word. +Cyrillic keeps its letters (``подача`` stays ``подача``). A keyword of five +letters or more matches anywhere in a layer name written without separators +(``cipsupplyline`` holds ``supply``); a shorter one must start a word, so +``air`` finds ``air_line`` and not ``stairs``. +""" + +from __future__ import annotations + +import re +import unicodedata + +__all__ = [ + "DISCIPLINES", + "EQUIPMENT_KINDS", + "LANGUAGES", + "SERVICES", + "classify_layer", + "equipment_kind", + "fold", + "room_label", + "supply_return", +] + +LANGUAGES = ("en", "tr", "ru", "nl", "de") + +SERVICES: tuple[str, ...] = ( + "product", + "cip_supply", + "cip_return", + "glycol_supply", + "glycol_return", + "steam", + "condensate", + "cold_water", + "hot_water", + "ice_water", + "compressed_air", + "drain", + "electrical", + "unknown", +) + +DISCIPLINES: tuple[str, ...] = ( + "piping", + "electrical", + "architecture", + "equipment", + "annotation", + "structure", + "unknown", +) + +EQUIPMENT_KINDS: tuple[str, ...] = ( + "tank", + "pump", + "valve", + "reducer", + "motor", + "agitator", + "panel", +) + +#: Confidence of a classification decided by one direct keyword. +DIRECT = 0.9 +#: Confidence of a classification from weaker evidence: a CIP or glycol layer +#: whose name says no direction, a generic water layer. +WEAK = 0.6 +#: A keyword this long matches anywhere in a separator-free name. +_SUBSTRING_MIN = 5 + +# (keyword, language) rows. Order matters inside a table: the first row that +# matches decides, so a longer or more specific word comes before a word it +# contains. + +_SUPPLY_WORDS = ( + ("supply", "en"), + ("besleme", "tr"), + ("gidis", "tr"), + ("подач", "ru"), # подача: supply + ("подающ", "ru"), # подающий: supplying + ("aanvoer", "nl"), + ("vorlauf", "de"), +) +_RETURN_WORDS = ( + ("return", "en"), + ("donus", "tr"), + ("возврат", "ru"), # return + ("обрат", "ru"), # обратка / обратный: return + ("retour", "nl"), + ("rucklauf", "de"), +) + +_CIP_WORDS = ( + ("cip", "en"), + ("мойк", "ru"), # мойка: washing, CIP +) +_GLYCOL_WORDS = ( + ("glycol", "en"), + ("glikol", "tr"), + ("гликол", "ru"), # гликоль: glycol + ("glykol", "de"), +) + +#: Services a layer name can state directly (CIP and glycol are decided with +#: their direction words above). +_SERVICE_WORDS = ( + ("product", "product", "en"), + ("product", "urun", "tr"), + ("product", "продукт", "ru"), # product + ("product", "produkt", "de"), + ("steam", "steam", "en"), + ("steam", "buhar", "tr"), + ("steam", "пар", "ru"), # steam + ("steam", "stoom", "nl"), + ("steam", "dampf", "de"), + ("condensate", "condensate", "en"), + ("condensate", "kondensat", "de"), + ("condensate", "kondens", "tr"), + ("condensate", "конденсат", "ru"), # condensate + ("condensate", "condensaat", "nl"), + ("ice_water", "ice water", "en"), + ("ice_water", "buzlu su", "tr"), + ("ice_water", "ледяная вода", "ru"), + ("ice_water", "ijswater", "nl"), + ("ice_water", "eiswasser", "de"), + ("hot_water", "hot water", "en"), + ("hot_water", "sicak su", "tr"), + ("hot_water", "горячая вода", "ru"), + ("hot_water", "warm water", "nl"), + ("hot_water", "warmwasser", "de"), + ("cold_water", "cold water", "en"), + ("cold_water", "soguk su", "tr"), + ("cold_water", "холодная вода", "ru"), + ("cold_water", "koud water", "nl"), + ("cold_water", "kaltwasser", "de"), + ("compressed_air", "compressed air", "en"), + ("compressed_air", "basincli hava", "tr"), + ("compressed_air", "сжатый воздух", "ru"), + ("compressed_air", "perslucht", "nl"), + ("compressed_air", "druckluft", "de"), + ("compressed_air", "air", "en"), + ("compressed_air", "hava", "tr"), + ("compressed_air", "воздух", "ru"), # air + ("drain", "drain", "en"), + ("drain", "sewer", "en"), + ("drain", "drenaj", "tr"), + ("drain", "atik su", "tr"), + ("drain", "дренаж", "ru"), # drainage + ("drain", "канализац", "ru"), # канализация: sewer + ("drain", "riool", "nl"), + ("drain", "abwasser", "de"), +) + +_ELECTRICAL_WORDS = ( + ("electrical", "en"), + ("electric", "en"), + ("power", "en"), + ("cable", "en"), + ("elektrik", "tr"), + ("kablo", "tr"), + ("электр", "ru"), # электрика: electrical + ("кабел", "ru"), # кабель: cable + ("elektra", "nl"), + ("kabel", "nl"), + ("elektro", "de"), +) + +#: Water words that name no temperature: a piping layer of unknown service. +_WATER_WORDS = ( + ("water", "en"), + ("вода", "ru"), # water + ("wasser", "de"), +) + +_DISCIPLINE_WORDS = ( + ("piping", "piping", "en"), + ("piping", "pipe", "en"), + ("piping", "valve", "en"), + ("piping", "boru", "tr"), + ("piping", "tesisat", "tr"), + ("piping", "vana", "tr"), + ("piping", "труб", "ru"), # трубопровод: pipeline + ("piping", "клапан", "ru"), # valve + ("piping", "арматур", "ru"), # арматура: valves and fittings + ("piping", "leiding", "nl"), + ("piping", "klep", "nl"), + ("piping", "afsluiter", "nl"), + ("piping", "rohr", "de"), + ("piping", "ventil", "de"), + ("piping", "armatur", "de"), + ("architecture", "wall", "en"), + ("architecture", "door", "en"), + ("architecture", "window", "en"), + ("architecture", "duvar", "tr"), + ("architecture", "kapi", "tr"), + ("architecture", "pencere", "tr"), + ("architecture", "стен", "ru"), # стена: wall + ("architecture", "двер", "ru"), # дверь: door + ("architecture", "окн", "ru"), # окно: window + ("architecture", "muur", "nl"), + ("architecture", "muren", "nl"), + ("architecture", "deur", "nl"), + ("architecture", "raam", "nl"), + ("architecture", "wand", "de"), + ("architecture", "fenster", "de"), + ("equipment", "equipment", "en"), + ("equipment", "tank", "en"), + ("equipment", "pump", "en"), + ("equipment", "ekipman", "tr"), + ("equipment", "pompa", "tr"), + ("equipment", "оборудован", "ru"), # оборудование: equipment + ("equipment", "насос", "ru"), # pump + ("equipment", "apparatuur", "nl"), + ("equipment", "pomp", "nl"), + ("equipment", "ausrustung", "de"), + ("equipment", "pumpe", "de"), + ("annotation", "text", "en"), + ("annotation", "dim", "en"), + ("annotation", "anno", "en"), + ("annotation", "note", "en"), + ("annotation", "title", "en"), + ("annotation", "yazi", "tr"), + ("annotation", "olcu", "tr"), + ("annotation", "antet", "tr"), + ("annotation", "текст", "ru"), # text + ("annotation", "размер", "ru"), # dimension + ("annotation", "штамп", "ru"), # title block + ("annotation", "tekst", "nl"), + ("annotation", "maat", "nl"), + ("annotation", "bemassung", "de"), + ("structure", "column", "en"), + ("structure", "beam", "en"), + ("structure", "steel", "en"), + ("structure", "grid", "en"), + ("structure", "kolon", "tr"), + ("structure", "kiris", "tr"), + ("structure", "celik", "tr"), + ("structure", "aks", "tr"), + ("structure", "колонн", "ru"), # колонна: column + ("structure", "балк", "ru"), # балка: beam + ("structure", "kolom", "nl"), + ("structure", "balk", "nl"), + ("structure", "staal", "nl"), + ("structure", "stutze", "de"), + ("structure", "trager", "de"), + ("structure", "stahl", "de"), +) + +_ROOM_WORDS = ( + ("room", "en"), + ("oda", "tr"), + ("mahal", "tr"), + ("помещ", "ru"), # помещение: room + ("комнат", "ru"), # комната: room + ("ruimte", "nl"), + ("kamer", "nl"), + ("raum", "de"), + ("zimmer", "de"), +) + +_EQUIPMENT_KIND_WORDS = ( + ("reducer", "reducer", "en"), + ("reducer", "reduksiyon", "tr"), + ("reducer", "переход", "ru"), # reducer + ("reducer", "verloop", "nl"), + ("reducer", "reduzier", "de"), + ("valve", "valve", "en"), + ("valve", "vlv", "en"), + ("valve", "vana", "tr"), + ("valve", "клапан", "ru"), # valve + ("valve", "кран", "ru"), # valve (cock) + ("valve", "klep", "nl"), + ("valve", "afsluiter", "nl"), + ("valve", "ventil", "de"), + ("valve", "armatur", "de"), + ("agitator", "agitator", "en"), + ("agitator", "mixer", "en"), + ("agitator", "karistirici", "tr"), + ("agitator", "мешалк", "ru"), # мешалка: agitator + ("agitator", "roerwerk", "nl"), + ("agitator", "ruhrwerk", "de"), + ("pump", "pump", "en"), + ("pump", "pompa", "tr"), + ("pump", "насос", "ru"), # pump + ("pump", "pomp", "nl"), + ("pump", "pumpe", "de"), + ("tank", "tank", "en"), + ("tank", "vessel", "en"), + ("tank", "емкост", "ru"), # емкость: vessel + ("tank", "бак", "ru"), # tank + ("tank", "vat", "nl"), + ("tank", "behalter", "de"), + ("motor", "motor", "en"), + ("motor", "мотор", "ru"), # motor + ("motor", "электродвиг", "ru"), # электродвигатель: electric motor + ("panel", "panel", "en"), + ("panel", "cabinet", "en"), + ("panel", "pano", "tr"), + ("panel", "шкаф", "ru"), # cabinet + ("panel", "щит", "ru"), # switchboard + ("panel", "schakelkast", "nl"), + ("panel", "kast", "nl"), + ("panel", "schrank", "de"), +) + + +def fold(text: str | None) -> str: + """Casefolded, diacritics stripped, Turkish ı made i; Cyrillic letters kept.""" + if not text: + return "" + decomposed = unicodedata.normalize("NFKD", str(text).casefold().replace("ı", "i")) + return "".join(ch for ch in decomposed if not unicodedata.combining(ch)) + + +def _words(folded: str) -> list[str]: + return [w for w in re.split(r"[^\w]+|_|\d+", folded) if w] + + +def _compact(folded: str) -> str: + return "".join(w for w in _words(folded)) + + +def _matches(keyword: str, folded: str) -> bool: + key = fold(keyword) + if " " in key or len(key) >= _SUBSTRING_MIN: + return key.replace(" ", "") in _compact(folded) + return any(word.startswith(key) for word in _words(folded)) + + +def _first(rows, folded: str): + for row in rows: + if _matches(row[-2], folded): + return row + return None + + +def _direction(folded: str) -> tuple[str, str, str] | None: + """("supply" | "return", keyword, language), or None when neither or both.""" + supply = _first(_SUPPLY_WORDS, folded) + ret = _first(_RETURN_WORDS, folded) + if (supply is None) == (ret is None): + return None + if supply is not None: + return ("supply", *supply) + return ("return", *ret) + + +def _result(discipline, service, confidence, keyword, language) -> dict: + return { + "discipline": discipline, + "service": service, + "confidence": confidence, + "keyword": keyword, + "language": language, + } + + +def classify_layer(name: str) -> dict: + """What a layer most likely carries, decided by the words in its name. + + ``{"discipline", "service", "confidence", "keyword", "language"}``. Order: + electrical words; CIP and glycol with a direction word (``P_cipsupplyline`` + -> ``cip_supply``; without one, the service stays ``unknown`` at 0.6 - the + layer alone is not trusted to say which way a CIP line runs); the stated + services; a water word with no temperature (piping, ``unknown``, 0.6); the + discipline words. Nothing matched: ``unknown`` / ``unknown`` at 0.0. + """ + folded = fold(name) + row = _first(_ELECTRICAL_WORDS, folded) + if row is not None: + return _result("electrical", "electrical", DIRECT, *row) + for rows, stem in ((_CIP_WORDS, "cip"), (_GLYCOL_WORDS, "glycol")): + row = _first(rows, folded) + if row is None: + continue + direction = _direction(folded) + if direction is None: + return _result("piping", "unknown", WEAK, *row) + return _result("piping", f"{stem}_{direction[0]}", DIRECT, row[0], row[1]) + row = _first(_SERVICE_WORDS, folded) + if row is not None: + return _result("piping", row[0], DIRECT, row[1], row[2]) + row = _first(_WATER_WORDS, folded) + if row is not None: + return _result("piping", "unknown", WEAK, *row) + row = _first(_DISCIPLINE_WORDS, folded) + if row is not None: + return _result(row[0], "unknown", DIRECT, row[1], row[2]) + return _result("unknown", "unknown", 0.0, None, None) + + +def supply_return(text: str | None) -> str | None: + """``"supply"`` or ``"return"`` when the text says one of them, else None. + + ``ПОДАЧА`` / ``BESLEME`` / ``SUPPLY`` / ``AANVOER`` / ``VORLAUF`` are supply; + ``ВОЗВРАТ`` / ``DÖNÜŞ`` / ``RETURN`` / ``RETOUR`` / ``RÜCKLAUF`` return. A + text naming both (``CIP SUPPLY / RETURN``) says neither and is None. + """ + direction = _direction(fold(text)) + return None if direction is None else direction[0] + + +def room_label(text: str | None) -> dict | None: + """A room label read: ``{"number", "name", "language"}``, or None. + + A text is a room label when one of its words is a room word (*room*, + *oda*, *mahal*, *помещение*, *комната*, *ruimte*, *kamer*, *Raum*, + *Zimmer*). The number is the first word holding a digit; the name is what + is left, in the text's own spelling, or None. ``"ROOM 1 PROCESS HALL"`` -> + ``{"number": "1", "name": "PROCESS HALL", "language": "en"}``. + """ + if not text: + return None + words = str(text).split() + for index, word in enumerate(words): + folded = fold(word) + for keyword, language in _ROOM_WORDS: + if folded.startswith(fold(keyword)): + rest = words[:index] + words[index + 1 :] + number = next((w for w in rest if any(ch.isdigit() for ch in w)), None) + name = " ".join(w for w in rest if w != number).strip() or None + return {"number": number, "name": name, "language": language} + return None + + +def equipment_kind(block_name: str | None) -> dict | None: + """The kind of equipment a block name says it is: ``{"kind", "keyword", "language"}``. + + ``KLEP_VLINDER`` -> valve (nl); ``VERLOOPSTUK`` -> reducer (nl); an + anonymous ``*U12`` or a name with no keyword -> None. + """ + folded = fold(block_name) + row = _first(_EQUIPMENT_KIND_WORDS, folded) + if row is None: + return None + return {"kind": row[0], "keyword": row[1], "language": row[2]} +``` + +- [ ] **Step 8: Run it to verify it passes** + +Run: `uv run --frozen pytest tests/test_understand_vocab.py -q` +Expected: `61 passed`. + +- [ ] **Step 9: Write the failing test for the snapshot and the size limit** + +Create `tests/test_understand_snapshot.py`: + +```python +"""The snapshot: one bulk read of a drawing into frozen WCS records. + +Every expected coordinate and length is worked out by hand in the comment +beside it. The live engine's export is exercised against a fake document that +behaves the way AutoCAD 2026 was measured to behave (the extension must be +"DXF", the file lands as snap.DXF, the document is not renamed). +""" + +from __future__ import annotations + +import math +from pathlib import Path + +import ezdxf +import pytest +from ezdxf.math import Vec2 + +import backends.com_backend as cb +import config +from backends.base import UnsupportedCapabilityError +from backends.com_backend import ComBackend +from backends.ezdxf_backend import EzdxfBackend +from engineering.understand.snapshot import ( + EntityRecord, + Snapshot, + length_of, + read_snapshot, + records_from_doc, + take_snapshot, +) + +EPS = 1e-9 + + +def _doc(): + """A drawing with one entity of each kind the readers use.""" + doc = ezdxf.new("R2018") + doc.header["$INSUNITS"] = 4 + doc.layers.add("PIPE", color=3, linetype="Continuous") + msp = doc.modelspace() + msp.add_line((0, 0), (3, 4), dxfattribs={"layer": "PIPE"}) + # a closed square whose first edge bulges into a semicircle (bulge 1) + msp.add_lwpolyline( + [(0, 0, 1.0), (10, 0, 0.0), (10, 10, 0.0), (0, 10, 0.0)], + format="xyb", + close=True, + dxfattribs={"layer": "PIPE"}, + ) + msp.add_polyline2d([(0, 0), (0, 5), (5, 5)], dxfattribs={"layer": "PIPE"}) + msp.add_arc((100, 0), 10, 0, 90, dxfattribs={"layer": "PIPE"}) + # the same quarter arc mirrored: extrusion (0, 0, -1), OCS centre (-200, 0) + # is WCS (200, 0), and the arc runs from 90 to 180 degrees in WCS + msp.add_arc((-200, 0), 10, 0, 90, dxfattribs={"layer": "PIPE", "extrusion": (0, 0, -1)}) + msp.add_circle((50, 50), 5) + msp.add_text("%%c51", dxfattribs={"insert": (20, 20), "height": 2.5, "layer": "TXT"}) + msp.add_mtext("{\\fArial|b0;SMS51}\\PDN20", dxfattribs={"insert": (30, 30), "char_height": 3.5}) + block = doc.blocks.new("TANK_V") + block.add_circle((0, 0), 600) + block.add_attdef("TAG", (0, 0), dxfattribs={"height": 250}) + insert = msp.add_blockref("TANK_V", (1000, 2000), dxfattribs={"rotation": 90}) + insert.add_auto_attribs({"TAG": "Т4100"}) # Cyrillic Т + leader = msp.add_multileader_mtext("Standard") + leader.set_content("Wiring to CP1", char_height=2.5) + leader.add_leader_line(ezdxf.render.mleader.ConnectionSide.left, [Vec2(0, 0)]) + leader.build(insert=Vec2(50, 20)) + doc.layouts.get("Layout1").add_line((1, 1), (2, 1)) + return doc + + +def _by_type(snap: Snapshot) -> dict[str, list[EntityRecord]]: + out: dict[str, list[EntityRecord]] = {} + for rec in snap.records: + out.setdefault(rec.type, []).append(rec) + return out + + +def test_records_carry_wcs_geometry_plain_text_and_the_space(): + snap = records_from_doc(_doc(), source="memory") + kinds = _by_type(snap) + assert snap.source == "memory" + assert snap.insunits == 4 + assert snap.layouts == ("Model", "Layout1") + assert snap.layers["PIPE"] == {"color": 3, "linetype": "Continuous"} + (line_model,) = [r for r in kinds["LINE"] if r.space == "Model"] + (line_paper,) = [r for r in kinds["LINE"] if r.space == "Layout1"] + assert line_model.points == ((0.0, 0.0), (3.0, 4.0)) + assert line_model.layer == "PIPE" + assert line_paper.points == ((1.0, 1.0), (2.0, 1.0)) + (square,) = kinds["LWPOLYLINE"] + assert square.closed is True + assert square.bulges == (1.0, 0.0, 0.0, 0.0) + (poly,) = kinds["POLYLINE"] + assert poly.points == ((0.0, 0.0), (0.0, 5.0), (5.0, 5.0)) and poly.closed is False + arcs = sorted(kinds["ARC"], key=lambda r: r.points[0][0]) + assert arcs[0].points == ((100.0, 0.0),) and arcs[0].angles == pytest.approx((0.0, 90.0)) + assert arcs[1].points[0] == pytest.approx((200.0, 0.0), abs=EPS) + assert arcs[1].angles == pytest.approx((90.0, 180.0), abs=EPS) + (text,) = kinds["TEXT"] + assert text.text == "Ø51" and text.height == 2.5 and text.points == ((20.0, 20.0),) + (mtext,) = kinds["MTEXT"] + assert mtext.text == "SMS51\nDN20" and mtext.height == 3.5 + (insert,) = kinds["INSERT"] + assert insert.block == "TANK_V" and insert.points == ((1000.0, 2000.0),) + assert insert.attribs == (("TAG", "Т4100"),) + assert insert.rotation == 90.0 + assert insert.bbox == pytest.approx((400.0, 1400.0, 1600.0, 2600.0), abs=1e-6) + (leader,) = kinds["MULTILEADER"] + assert leader.text == "Wiring to CP1" + assert leader.points[0] == (0.0, 0.0) # the arrow tip comes first + assert leader.points[-1] == (50.0, 20.0) # the text insertion comes last + assert snap.extmin is None and snap.extmax is None # never computed + + +def test_every_record_is_frozen(): + rec = records_from_doc(_doc(), source="memory").records[0] + with pytest.raises(AttributeError): + rec.layer = "OTHER" + + +def test_lengths(): + kinds = _by_type(records_from_doc(_doc(), source="memory")) + (line_model,) = [r for r in kinds["LINE"] if r.space == "Model"] + assert length_of(line_model) == pytest.approx(5.0, abs=EPS) # 3-4-5 + # semicircle of diameter 10 (5 pi) + three straight sides of 10 + (square,) = kinds["LWPOLYLINE"] + assert length_of(square) == pytest.approx(5.0 * math.pi + 30.0, abs=EPS) + (poly,) = kinds["POLYLINE"] + assert length_of(poly) == pytest.approx(10.0, abs=EPS) + for arc in kinds["ARC"]: # a quarter of a radius-10 circle + assert length_of(arc) == pytest.approx(5.0 * math.pi, abs=EPS) + (circle,) = kinds["CIRCLE"] + assert length_of(circle) == pytest.approx(10.0 * math.pi, abs=EPS) + assert length_of(kinds["TEXT"][0]) == 0.0 + + +def test_the_size_limit_defaults_to_512_mb(monkeypatch): + monkeypatch.delenv("MAX_DXF_BYTES", raising=False) + assert config.Settings().max_dxf_bytes == 512 * 1024 * 1024 + assert config.DEFAULT_MAX_DXF_BYTES == 536_870_912 + + +def test_read_snapshot_reads_a_file(tmp_path): + path = tmp_path / "plant.dxf" + _doc().saveas(path) + snap = read_snapshot(str(path)) + assert snap.source == str(path) + assert len(snap.records) == len(records_from_doc(_doc(), source="x").records) + + +def test_read_snapshot_refuses_a_file_over_the_limit_naming_size_and_variable( + tmp_path, monkeypatch +): + path = tmp_path / "big.dxf" + _doc().saveas(path) + size = path.stat().st_size + monkeypatch.setattr(config.settings, "max_dxf_bytes", size - 1) + with pytest.raises(ValueError) as excinfo: + read_snapshot(str(path)) + message = str(excinfo.value) + assert "MAX_DXF_BYTES" in message + assert f"{size:,} bytes" in message + monkeypatch.setattr(config.settings, "max_dxf_bytes", 0) # 0 disables the check + assert read_snapshot(str(path)).records + + +def test_read_snapshot_refuses_missing_and_unreadable_files(tmp_path): + with pytest.raises(FileNotFoundError): + read_snapshot(str(tmp_path / "nope.dxf")) + junk = tmp_path / "junk.dxf" + junk.write_text("not a dxf", encoding="utf-8") + with pytest.raises(ValueError) as excinfo: + read_snapshot(str(junk)) + assert "not a readable DXF" in str(excinfo.value) + + +# -- take_snapshot on the headless engine --------------------------------------- + + +@pytest.mark.asyncio +async def test_the_headless_engine_snapshots_its_current_document_without_touching_it(): + backend = EzdxfBackend() + await backend.connect() + await backend.drawing_new() + line = await backend.entity_create_line(0, 0, 10, 0, layer="PIPE") + before = len(await backend.entity_list(limit=1000)) + snap = await take_snapshot(backend) + assert snap.source == "memory:untitled" + assert [r.handle for r in snap.records if r.type == "LINE"] == [line.handle] + assert len(await backend.entity_list(limit=1000)) == before + + +@pytest.mark.asyncio +async def test_a_dxf_path_is_read_directly(tmp_path): + path = tmp_path / "plant.dxf" + _doc().saveas(path) + backend = EzdxfBackend() + await backend.connect() + snap = await take_snapshot(backend, str(path)) + assert snap.source == str(path) and snap.records + + +@pytest.mark.asyncio +async def test_a_dwg_is_refused_headlessly_with_the_dwg_capability(tmp_path): + path = tmp_path / "plant.dwg" + path.write_bytes(b"AC1032") + backend = EzdxfBackend() + await backend.connect() + with pytest.raises(UnsupportedCapabilityError) as excinfo: + await take_snapshot(backend, str(path)) + assert excinfo.value.capability == "dwg" + assert "DXF" in str(excinfo.value) + with pytest.raises(FileNotFoundError): + await take_snapshot(backend, str(tmp_path / "missing.dwg")) + with pytest.raises(ValueError): + await take_snapshot(backend, str(tmp_path / "plant.txt")) + + +# -- take_snapshot on the live engine (a fake that behaves as measured) ---------- + + +class _Selection: + def __init__(self, log): + self.log = log + + def Select(self, mode, *args): # noqa: N802 - ActiveX member name + self.log.append(("select", mode)) + + def Delete(self): # noqa: N802 + self.log.append(("delete_selection",)) + + +class _Selections: + def __init__(self, log): + self.log = log + + def Add(self, name): # noqa: N802 + self.log.append(("add_selection", name)) + return _Selection(self.log) + + +class _Doc: + def __init__(self, name, log): + self.Name = name + self.log = log + self.SelectionSets = _Selections(log) + + def Export(self, base, extension, selection): # noqa: N802 + if extension != "DXF": + raise ValueError("Invalid argument") # what AutoCAD 2026 answers to ".dxf" + self.log.append(("export", Path(base).name, extension)) + _doc().saveas(base + ".DXF") # AutoCAD writes the upper-case extension + + def SaveAs(self, *args): # noqa: N802 + raise AssertionError("a snapshot must never rename the document") + + def Close(self, save): # noqa: N802 + self.log.append(("close", save)) + + +class _Documents: + def __init__(self, log): + self.log = log + + def Open(self, path, read_only): # noqa: N802 + self.log.append(("open", Path(path).name, read_only)) + return _Doc(Path(path).name, self.log) + + +class _App: + def __init__(self, log): + self.Documents = _Documents(log) + + +def _live(monkeypatch, log): + backend = ComBackend() + + async def run_inline(func, *args, **kwargs): # bypass the STA executor + return func(*args, **kwargs) + + async def ensure_state(): + log.append(("ensure_document_state",)) + + backend._run = run_inline + backend._ensure_document_state = ensure_state + active = _Doc("Drawing1.dwg", log) + monkeypatch.setattr(cb, "_acad_app", lambda: _App(log)) + monkeypatch.setattr(cb, "_acad_doc", lambda: active) + return backend, active + + +@pytest.mark.asyncio +async def test_the_live_engine_exports_a_dxf_snapshot_and_deletes_it(monkeypatch, tmp_path): + log: list[tuple] = [] + made: list[str] = [] + real_mkdtemp = __import__("tempfile").mkdtemp + + def mkdtemp(prefix=""): + folder = real_mkdtemp(prefix=prefix, dir=tmp_path) + made.append(folder) + return folder + + monkeypatch.setattr("engineering.understand.snapshot.tempfile.mkdtemp", mkdtemp) + backend, active = _live(monkeypatch, log) + snap = await take_snapshot(backend) + assert snap.source == "live:Drawing1.dwg" + assert active.Name == "Drawing1.dwg" + assert ("select", cb._AC_SELECTION_SET_ALL) in log + assert ("export", "snap", "DXF") in log + assert ("delete_selection",) in log + assert {r.type for r in snap.records} >= {"LINE", "INSERT", "MULTILEADER"} + assert made and not Path(made[0]).exists(), "the temporary snapshot must be deleted" + + +@pytest.mark.asyncio +async def test_a_dwg_on_the_live_engine_is_opened_read_only_and_closed_unsaved( + monkeypatch, tmp_path +): + path = tmp_path / "plant.dwg" + path.write_bytes(b"AC1032") + log: list[tuple] = [] + backend, _active = _live(monkeypatch, log) + snap = await take_snapshot(backend, str(path)) + assert snap.source == "live:plant.dwg" + assert ("open", "plant.dwg", True) in log + assert ("close", False) in log + assert log[-1] == ("ensure_document_state",) +``` + +- [ ] **Step 10: Run it to verify it fails** + +Run: `uv run --frozen pytest tests/test_understand_snapshot.py -q` +Expected: FAIL at collection — `ModuleNotFoundError: No module named 'engineering.understand.snapshot'`. + +- [ ] **Step 11: Raise the size limit in `config.py`** + +In `config.py`, replace + +```python +DEFAULT_CAD_PROGID = "AutoCAD.Application" +``` + +with + +```python +DEFAULT_CAD_PROGID = "AutoCAD.Application" + +#: Default ``MAX_DXF_BYTES``: 512 MB. +DEFAULT_MAX_DXF_BYTES = 512 * 1024 * 1024 +``` + +and replace + +```python + # Reject opening DXF files larger than this many bytes (default 50 MB). + # Set to 0 to disable the check. + self.max_dxf_bytes: int = int(os.environ.get("MAX_DXF_BYTES", str(50 * 1024 * 1024))) +``` + +with + +```python + # Reject opening DXF files larger than this many bytes (default 512 MB). + # Set to 0 to disable the check. It was 50 MB until v1.6's track H: a + # real plant layout read on 2026-09-24 was a 188 MB DXF, and the + # understanding tools exist for exactly such drawings. The refusal + # names the file's size and this variable. + self.max_dxf_bytes: int = int(os.environ.get("MAX_DXF_BYTES", str(DEFAULT_MAX_DXF_BYTES))) +``` + +`drawing_open` (`backends/ezdxf_backend.py`) reads the same setting and already names the size and the variable in its refusal; it is not touched. + +- [ ] **Step 12: Write `engineering/understand/snapshot.py`** + +What the module docstring states about the live export was measured on this machine's AutoCAD 2026 (`25.1s (LMS Tech)`) on 2026-09-24, in a scratch document created and closed by the measuring script (no operator document was open); Step 19 re-measures it. + +```python +"""One bulk read of a drawing into frozen records - the input of every track H reader. + +Reading a 35,000-entity P&ID one COM call at a time takes minutes; reading its +DXF takes about a second. So every analysis in ``engineering/understand/`` runs +on a :class:`Snapshot`: the drawing read once, into small frozen +:class:`EntityRecord` s, and never touched again. + +Where the snapshot comes from: + +* **A DXF file** - :func:`read_snapshot`, ``ezdxf.readfile`` after the size + check (``MAX_DXF_BYTES``, 512 MB by default; the refusal names the size and + the variable). +* **The headless engine's current document** - :func:`records_from_doc` on the + in-memory ``ezdxf`` document, inside the backend's own lock. +* **The live engine's current document** - ``Document.Export(/snap, "DXF", + )``, then :func:`read_snapshot` on the file, which is + deleted afterwards. Measured on AutoCAD 2026 (``25.1s (LMS Tech)``) on + 2026-09-24, in a new scratch document closed afterwards: + + - the extension argument must be ``"DXF"``; ``".dxf"`` fails with *Invalid + argument*; the file is written as ``snap.DXF`` (upper-case extension); + - the document keeps its name (``Export`` is not ``SaveAs``), and an unsaved + document exports; + - **model space, every paper-space layout and every block definition reach + the snapshot** - a LINE in Layout1 was read back in Layout1, and an unused + block definition was in the BLOCKS section; + - **the selection set does not narrow a DXF export**: an empty selection set + wrote the same model space, paper space and blocks as the set of all. The + set of all is passed anyway, because the ActiveX signature requires one. + - xrefs were not measured. + + A DWG path on the live engine is opened read-only, exported the same way and + closed without saving. + +Records hold what the readers need and nothing else: WCS points (OCS is +translated here with ``backends/ocs.py``, so a mirrored ARC or LWPOLYLINE is +where it looks), plain text (``labels.plain``: AutoCAD codes and MTEXT format +runs decoded), block names and attributes, a WCS box. Block definitions are not +exploded: an INSERT is one record with its block name, attributes and box. +""" + +from __future__ import annotations + +import asyncio +import math +import os +import shutil +import tempfile +import uuid +from dataclasses import dataclass +from pathlib import Path + +import ezdxf +from ezdxf import bbox as ezbbox + +import config +from backends.capability import UnsupportedCapabilityError +from backends.ocs import to_wcs_2d, wcs_arc_angles, wcs_bulge +from engineering.measure import polygon_area_perimeter +from engineering.understand.labels import plain + +__all__ = [ + "EntityRecord", + "Pt", + "Snapshot", + "length_of", + "read_snapshot", + "records_from_doc", + "take_snapshot", +] + +Pt = tuple[float, float] + +#: ezdxf's placeholder for an extents header nobody computed. +_UNSET_EXTENT = 1e19 +#: The live export's base name inside its temporary folder. +_SNAP_BASE = "snap" + + +@dataclass(frozen=True) +class EntityRecord: + """One entity of a layout, in WCS, as the readers need it.""" + + handle: str + type: str + layer: str + space: str + points: tuple[Pt, ...] = () + bulges: tuple[float, ...] = () + closed: bool = False + radius: float | None = None + angles: tuple[float, float] | None = None + text: str | None = None + height: float | None = None + block: str | None = None + attribs: tuple[tuple[str, str], ...] = () + bbox: tuple[float, float, float, float] | None = None + rotation: float = 0.0 + scale: tuple[float, float] = (1.0, 1.0) + + +@dataclass(frozen=True) +class Snapshot: + """A whole drawing, read once.""" + + source: str + insunits: int + extmin: Pt | None + extmax: Pt | None + layers: dict[str, dict] + layouts: tuple[str, ...] + records: tuple[EntityRecord, ...] + + +def _xy(point) -> Pt: + return (float(point[0]), float(point[1])) + + +def _box(entity) -> tuple[float, float, float, float] | None: + """The entity's WCS box. An INSERT's box is its block geometry only: the + ATTRIB texts are left out, because a TAG lettered beside a tank would push + the tank's box past its body (the rule `engineering/pid/graph.py` keeps).""" + try: + if entity.dxftype() == "INSERT": + box = ezbbox.extents(entity.virtual_entities(), fast=True) + else: + box = ezbbox.extents([entity], fast=True) + except Exception: # a proxy or a malformed entity has no box; the record stays + return None + if not box.has_data: + return None + return ( + float(box.extmin.x), + float(box.extmin.y), + float(box.extmax.x), + float(box.extmax.y), + ) + + +def _header_point(doc, name: str) -> Pt | None: + try: + value = doc.header.get(name) + except Exception: + return None + if value is None or abs(float(value[0])) >= _UNSET_EXTENT: + return None + return _xy(value) + + +def _extrusion(entity): + """The entity's extrusion, or +Z for a type that has none (MULTILEADER, ...).""" + if entity.dxf.is_supported("extrusion"): + return entity.dxf.get("extrusion", (0.0, 0.0, 1.0)) + return (0.0, 0.0, 1.0) + + +def _polyline(entity) -> tuple[tuple[Pt, ...], tuple[float, ...], bool]: + extrusion = _extrusion(entity) + if entity.dxftype() == "LWPOLYLINE": + elevation = float(entity.dxf.get("elevation", 0.0)) + rows = [(float(x), float(y), float(b)) for x, y, b in entity.get_points("xyb")] + points = tuple(tuple(to_wcs_2d(extrusion, x, y, elevation)) for x, y, _ in rows) + bulges = tuple(wcs_bulge(extrusion, b) for _, _, b in rows) + return points, bulges, bool(entity.closed) + vertices = list(entity.vertices) + if entity.is_2d_polyline: + points = tuple( + tuple(to_wcs_2d(extrusion, v.dxf.location.x, v.dxf.location.y, v.dxf.location.z)) + for v in vertices + ) + bulges = tuple(wcs_bulge(extrusion, v.dxf.get("bulge", 0.0)) for v in vertices) + else: # a 3D polyline is WCS already; its z is dropped + points = tuple(_xy(v.dxf.location) for v in vertices) + bulges = tuple(0.0 for _ in vertices) + return points, bulges, bool(entity.is_closed) + + +def _record(entity, space: str) -> EntityRecord | None: + kind = entity.dxftype() + dxf = entity.dxf + base = { + "handle": str(dxf.handle), + "type": kind, + "layer": str(dxf.get("layer", "0")), + "space": space, + "bbox": _box(entity), + } + extrusion = _extrusion(entity) + if kind == "LINE": + return EntityRecord(**base, points=(_xy(dxf.start), _xy(dxf.end))) + if kind in ("LWPOLYLINE", "POLYLINE"): + if kind == "POLYLINE" and (entity.is_poly_face_mesh or entity.is_polygon_mesh): + return EntityRecord(**base) + points, bulges, closed = _polyline(entity) + return EntityRecord(**base, points=points, bulges=bulges, closed=closed) + if kind in ("ARC", "CIRCLE"): + centre = tuple(to_wcs_2d(extrusion, dxf.center.x, dxf.center.y, dxf.center.z)) + angles = None + if kind == "ARC": + angles = wcs_arc_angles( + extrusion, dxf.center, dxf.radius, dxf.start_angle, dxf.end_angle + ) + return EntityRecord(**base, points=(centre,), radius=float(dxf.radius), angles=angles) + if kind == "TEXT": + insert = dxf.insert + return EntityRecord( + **base, + points=(tuple(to_wcs_2d(extrusion, insert.x, insert.y, insert.z)),), + text=plain(dxf.get("text", "")), + height=float(dxf.get("height", 0.0)), + rotation=float(dxf.get("rotation", 0.0)), + ) + if kind == "MTEXT": + return EntityRecord( + **base, + points=(_xy(dxf.insert),), + text=plain(entity.text), + height=float(dxf.get("char_height", 0.0)), + rotation=float(entity.get_rotation()), + ) + if kind == "INSERT": + insert = dxf.insert + attribs = tuple((str(a.dxf.tag), plain(a.dxf.get("text", ""))) for a in entity.attribs) + return EntityRecord( + **base, + points=(tuple(to_wcs_2d(extrusion, insert.x, insert.y, insert.z)),), + block=str(dxf.name), + attribs=attribs, + rotation=float(dxf.get("rotation", 0.0)), + scale=(float(dxf.get("xscale", 1.0)), float(dxf.get("yscale", 1.0))), + ) + if kind == "MULTILEADER": + context = entity.context + points: list[Pt] = [] + for leader in context.leaders: + for line in leader.lines: + points.extend(_xy(v) for v in line.vertices) + points.append(_xy(leader.last_leader_point)) + text = None + height = None + if context.mtext is not None: + text = plain(context.mtext.default_content) + points.append(_xy(context.mtext.insert)) + height = float(context.char_height) + return EntityRecord(**base, points=tuple(points), text=text, height=height) + if kind == "DIMENSION": + override = str(dxf.get("text", "") or "") + text = plain(override) if override not in ("", "<>") else None + middle = dxf.get("text_midpoint", dxf.defpoint) + points = (_xy(dxf.defpoint), tuple(to_wcs_2d(extrusion, middle.x, middle.y, middle.z))) + return EntityRecord(**base, points=points, text=text) + if kind == "LEADER": + return EntityRecord(**base, points=tuple(_xy(v) for v in entity.vertices)) + if kind == "POINT": + return EntityRecord(**base, points=(_xy(dxf.location),)) + if kind == "SPLINE": + return EntityRecord(**base, points=tuple(_xy(p) for p in entity.control_points)) + if kind == "ELLIPSE": + return EntityRecord(**base, points=(_xy(dxf.center),)) + return EntityRecord(**base) + + +def records_from_doc(doc, *, source: str) -> Snapshot: + """Every entity of every layout of an ``ezdxf`` document, as records. + + Model space first, then each paper layout in tab order. ``insunits`` is + ``$INSUNITS`` (0 when absent); ``extmin`` / ``extmax`` are the declared + ``$EXTMIN`` / ``$EXTMAX``, None when the header holds ezdxf's "never + computed" placeholder. Nothing in ``doc`` is modified. + """ + records: list[EntityRecord] = [] + layouts = tuple(doc.layouts.names_in_taborder()) + for name in layouts: + for entity in doc.layouts.get(name): + record = _record(entity, name) + if record is not None: + records.append(record) + layers = { + str(layer.dxf.name): { + "color": int(layer.dxf.get("color", 7)), + "linetype": str(layer.dxf.get("linetype", "Continuous")), + } + for layer in doc.layers + } + return Snapshot( + source=source, + insunits=int(doc.header.get("$INSUNITS", 0) or 0), + extmin=_header_point(doc, "$EXTMIN"), + extmax=_header_point(doc, "$EXTMAX"), + layers=layers, + layouts=layouts, + records=tuple(records), + ) + + +def _read(path: str, *, source: str) -> Snapshot: + try: + doc = ezdxf.readfile(path) + except (OSError, ezdxf.DXFError) as exc: + raise ValueError(f"{path} is not a readable DXF file: {exc}") from exc + return records_from_doc(doc, source=source) + + +def read_snapshot(path: str) -> Snapshot: + """The snapshot of a DXF file on disk, after the size check. + + Refuses a missing file (``FileNotFoundError``) and a file larger than + ``MAX_DXF_BYTES`` (``ValueError`` naming the size, the limit and the + variable; 0 disables the check) before reading a byte of it. + """ + target = Path(path) + if not target.is_file(): + raise FileNotFoundError(f"{path}: no such file") + limit = int(config.settings.max_dxf_bytes) + size = target.stat().st_size + if limit > 0 and size > limit: + raise ValueError( + f"{path} is {size:,} bytes ({size / 2**20:.1f} MB), over the MAX_DXF_BYTES " + f"limit of {limit:,} bytes ({limit / 2**20:.0f} MB). Raise MAX_DXF_BYTES " + "(0 disables the check) to read it." + ) + return _read(str(target), source=str(target)) + + +def _com_export(backend, path: str | None) -> tuple[str, str, str]: + """(snapshot file, its temporary folder, source) - runs on the COM thread.""" + from backends import com_backend as com + + app = com._acad_app() + opened = None + if path is None: + doc = com._acad_doc() + else: + opened = app.Documents.Open(str(path), True) # read-only + doc = opened + folder = tempfile.mkdtemp(prefix="acadmcp_snap_") + try: + name = str(backend._wait_out_rejected_call(lambda: doc.Name)) + selection = doc.SelectionSets.Add(f"_SNAP_{uuid.uuid4().hex[:8]}") + try: + selection.Select(com._AC_SELECTION_SET_ALL) + # "DXF", never ".dxf": the dotted spelling is refused with + # "Invalid argument" (measured, AutoCAD 2026). + doc.Export(os.path.join(folder, _SNAP_BASE), "DXF", selection) + finally: + try: + selection.Delete() + except Exception: # a selection set left behind is harmless; the export is not + pass + except Exception: + shutil.rmtree(folder, ignore_errors=True) + raise + finally: + if opened is not None: + opened.Close(False) + written = sorted(Path(folder).glob(f"{_SNAP_BASE}.*")) + if not written: + shutil.rmtree(folder, ignore_errors=True) + raise RuntimeError(f"AutoCAD reported the export of {name} done but wrote no file") + return str(written[0]), folder, f"live:{name}" + + +async def take_snapshot(backend, path: str | None = None) -> Snapshot: + """The snapshot of ``path``, or of the backend's current document. + + ``path`` given: a ``.dxf`` is read directly (on either engine); a ``.dwg`` + needs the live engine, which opens it read-only, exports it and closes it - + headlessly it is refused with the ``dwg`` capability. ``path`` None: the + headless engine reads its in-memory document under its own lock; the live + engine exports the active document to a temporary DXF (see the module + docstring for what was measured) and deletes it afterwards. The snapshot of + a live document is not size-checked: it is already in AutoCAD's memory. + """ + if path is not None: + suffix = Path(path).suffix.lower() + if suffix == ".dxf": + return await asyncio.to_thread(read_snapshot, str(path)) + if suffix != ".dwg": + raise ValueError(f"{path}: a snapshot reads a .dxf or a .dwg, not {suffix or 'this'}") + if not Path(path).is_file(): + raise FileNotFoundError(f"{path}: no such file") + if backend.name != "com": + raise UnsupportedCapabilityError( + "dwg", + f"{path} is a DWG, which the headless engine cannot read. Export it to " + "DXF (DXFOUT or SAVEAS in AutoCAD) and pass the .dxf, or run the server " + "on the live engine (AUTOCAD_MCP_BACKEND=com).", + ) + if backend.name == "com": + written, folder, source = await backend._run(_com_export, backend, path) + if path is not None: + await backend._ensure_document_state() + try: + return await asyncio.to_thread(_read, written, source=source) + finally: + shutil.rmtree(folder, ignore_errors=True) + source = backend._doc_path or "memory:untitled" + return await backend._async(lambda: records_from_doc(backend._doc, source=source)) + + +def length_of(rec: EntityRecord) -> float: + """Drawn length in drawing units: LINE, LWPOLYLINE / POLYLINE with their + bulges (``engineering.measure``, the same arithmetic ``analysis_measure_entity`` + reports), ARC, CIRCLE. Every other type is 0.0. + """ + if rec.type == "LINE" and len(rec.points) == 2: + return math.dist(rec.points[0], rec.points[1]) + if rec.type in ("LWPOLYLINE", "POLYLINE") and len(rec.points) >= 2: + bulges = rec.bulges or tuple(0.0 for _ in rec.points) + vertices = [(x, y, b) for (x, y), b in zip(rec.points, bulges, strict=False)] + return polygon_area_perimeter(vertices, rec.closed)[1] + if rec.type == "ARC" and rec.radius is not None and rec.angles is not None: + sweep = (rec.angles[1] - rec.angles[0]) % 360.0 + return rec.radius * math.radians(sweep) + if rec.type == "CIRCLE" and rec.radius is not None: + return 2.0 * math.pi * rec.radius + return 0.0 +``` + +- [ ] **Step 13: Run it to verify it passes** + +Run: `uv run --frozen pytest tests/test_understand_snapshot.py -q` +Expected: `12 passed`. If the MULTILEADER assertions fail on the ezdxf of the lock, print `[type(e).__name__ for e in leader.context.leaders]` and the leader's `lines[0].vertices` / `last_leader_point` (the API checked for this plan: `context.leaders[i].lines[j].vertices`, `context.leaders[i].last_leader_point`, `context.mtext.default_content`, `context.mtext.insert`, `context.char_height`) and fix `_record`, not the test. + +- [ ] **Step 14: Write the failing test for the `office` extra** + +Create `tests/test_office_extra.py`: + +```python +"""The `office` extra: openpyxl for the XLSX workbooks, and in `full` too. + +Declared and locked is what is tested - not installed: the CI lane that +installs `[pdf]` has no openpyxl and must stay green, because every XLSX +writer refuses with the `xlsx_write` capability and writes CSV instead. +""" + +from __future__ import annotations + +import tomllib +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[1] + + +def _extras() -> dict: + with open(ROOT / "pyproject.toml", "rb") as handle: + return tomllib.load(handle)["project"]["optional-dependencies"] + + +def test_the_office_extra_carries_openpyxl_and_full_carries_it_too(): + extras = _extras() + assert extras["office"] == ["openpyxl>=3.1"] + assert "openpyxl>=3.1" in extras["full"] + + +def test_the_lock_records_openpyxl_for_the_office_extra(): + lock = (ROOT / "uv.lock").read_text(encoding="utf-8") + assert '\nname = "openpyxl"\n' in lock + assert '{ name = "openpyxl" }' in lock +``` + +Run: `uv run --frozen pytest tests/test_office_extra.py -q` +Expected: FAIL — `KeyError: 'office'` in `test_the_office_extra_carries_openpyxl_and_full_carries_it_too`, and `AssertionError` in the lock test. + +- [ ] **Step 15: Declare the extra, lock it, install it — the one `uv sync` of this track** + +In `pyproject.toml`, replace + +```toml +full = [ + # Same marker as `com` above, and for the same reason: `[full]` is the + # extra this project recommends to contributors, and it was + # uninstallable on every non-Windows machine. + 'pywin32>=306; sys_platform == "win32"', + "Pillow>=10.0", + "matplotlib>=3.8", +] +``` + +with + +```toml +office = [ + # XLSX output: the takeoff workbooks (pipe_takeoff / cable_takeoff) and + # data_extract(format="xlsx"). Without it every one of them still writes + # CSV and refuses XLSX with the `xlsx_write` capability, naming this extra. + "openpyxl>=3.1", +] +full = [ + # Same marker as `com` above, and for the same reason: `[full]` is the + # extra this project recommends to contributors, and it was + # uninstallable on every non-Windows machine. + 'pywin32>=306; sys_platform == "win32"', + "Pillow>=10.0", + "matplotlib>=3.8", + "openpyxl>=3.1", +] +``` + +Then, exactly once in this track: + +``` +uv lock +uv sync --frozen --all-extras +uv run --frozen python -c "import openpyxl; print(openpyxl.__version__)" +``` + +Expected: `uv lock` prints `Added et-xmlfile v2.0.0` and `Added openpyxl v3.1.5` (measured against PyPI for this plan on 2026-09-24; a newer 3.1.x is fine — the lock is the record), and the last line prints the version. If `uv sync` fails with `os error 32` on `.venv/Scripts/autocad-mcp.exe` (a running MCP server holds it), run `uv sync --frozen --all-extras --no-install-project` instead, use `uv run --frozen --no-sync …` for the rest of this task, and say so in the report. + +Run: `uv run --frozen pytest tests/test_office_extra.py tests/test_sheet_extract.py tests/test_refs_contract.py tests/test_release_consistency.py::test_uv_lock_records_the_pyproject_version tests/test_release_consistency.py::test_windows_only_dependencies_carry_a_platform_marker -q` +Expected: all pass. With openpyxl now installed, `test_xlsx_refuses_with_the_capability_key_and_names_the_extra` takes its positive branch (it writes the XLSX) — that is the test working, not a change of it. + +- [ ] **Step 16: Say the new default where a user reads it** + +In `README.md`, replace the configuration-table row + +``` +| `MAX_DXF_BYTES` | `52428800` | Reject larger DXF input; `0` disables | +``` + +with + +``` +| `MAX_DXF_BYTES` | `536870912` | Reject larger DXF input (512 MB); `0` disables | +``` + +In `.env.example`, replace + +``` +# Reject DXF input larger than this many bytes. 0 disables the check. +MAX_DXF_BYTES=52428800 +``` + +with + +``` +# Reject DXF input larger than this many bytes (512 MB). 0 disables the check. +MAX_DXF_BYTES=536870912 +``` + +- [ ] **Step 17: Re-export the foundation from the package** + +Replace `engineering/understand/__init__.py` with (later tasks append their own names to the imports and to `__all__`; the merge of those appends is additive): + +```python +"""Understanding drawings somebody else made (v1.6 track H). + +One bulk read per drawing (:mod:`.snapshot`), multilingual vocabulary +(:mod:`.vocab`) and label parsers (:mod:`.labels`) are the foundation; the +readers built on them - the one-call report, the scale check, the line +networks and takeoffs, the diff and the topology checks - add their modules +here. Everything except the snapshot's live export is pure: records in, +results out, the drawing never modified. +""" + +from engineering.understand.labels import ( + normalize_panel, + parse_diameters, + parse_electrical, + parse_tag, + plain, + wiring_target, +) +from engineering.understand.snapshot import ( + EntityRecord, + Pt, + Snapshot, + length_of, + read_snapshot, + records_from_doc, + take_snapshot, +) +from engineering.understand.vocab import ( + DISCIPLINES, + SERVICES, + classify_layer, + equipment_kind, + room_label, + supply_return, +) + +__all__ = [ + "DISCIPLINES", + "SERVICES", + "EntityRecord", + "Pt", + "Snapshot", + "classify_layer", + "equipment_kind", + "length_of", + "normalize_panel", + "parse_diameters", + "parse_electrical", + "parse_tag", + "plain", + "read_snapshot", + "records_from_doc", + "room_label", + "supply_return", + "take_snapshot", + "wiring_target", +] +``` + +Run: `uv run --frozen python -c "import engineering.understand as u; print(len(u.__all__), u.parse_tag('\u04224100'))"` +Expected: `19 T4100`. + +- [ ] **Step 18: Run the four new files together** + +Run: `uv run --frozen pytest tests/test_understand_labels.py tests/test_understand_vocab.py tests/test_understand_snapshot.py tests/test_office_extra.py -q` +Expected: `116 passed`. + +- [ ] **Step 19: Re-measure the live export on the running AutoCAD (skip, and say so, if none is running)** + +The snapshot docstring states what `Document.Export` puts into the file. Measure it again here rather than trusting this plan — the script adds a scratch document, never touches an operator document, and closes what it added: + +``` +uv run --frozen python - <<'PY' +import json, os, tempfile, time +import ezdxf, pythoncom, win32com.client +from win32com.client import VARIANT + +def retry(read, patience=60.0): + t0, pause = time.monotonic(), 0.3 + while True: + try: + return read() + except pythoncom.com_error as exc: + if exc.args[0] != -2147418111 or time.monotonic() - t0 > patience: + raise + time.sleep(pause) + pause = min(pause * 2, 3.0) + +def pt(x, y): + return VARIANT(pythoncom.VT_ARRAY | pythoncom.VT_R8, (float(x), float(y), 0.0)) + +pythoncom.CoInitialize() +app = win32com.client.GetActiveObject("AutoCAD.Application") +doc = retry(lambda: app.Documents.Add()) +name = retry(lambda: str(doc.Name)) +out = {"version": str(app.Version)} +try: + retry(lambda: doc.ModelSpace.AddLine(pt(0, 0), pt(100, 0))) + unused = retry(lambda: doc.Blocks.Add(pt(0, 0), "MEAS_UNUSED")) + retry(lambda: unused.AddLine(pt(0, 0), pt(1, 1))) + retry(lambda: doc.Layouts.Item("Layout1").Block.AddLine(pt(10, 10), pt(20, 10))) + folder = tempfile.mkdtemp(prefix="acadmcp_measure_") + base = os.path.join(folder, "snap") + everything = retry(lambda: doc.SelectionSets.Add("ACADMCP_MEASURE_ALL")) + empty = retry(lambda: doc.SelectionSets.Add("ACADMCP_MEASURE_EMPTY")) + try: + retry(lambda: everything.Select(5)) + retry(lambda: doc.Export(base, "DXF", everything)) + retry(lambda: doc.Export(base + "_empty", "DXF", empty)) + try: + doc.Export(base + "_dot", ".dxf", everything) + out["dot_dxf"] = "accepted" + except pythoncom.com_error as exc: + out["dot_dxf"] = exc.args[2][2] if exc.args[2] else str(exc) + finally: + everything.Delete() + empty.Delete() + out["files"] = sorted(os.listdir(folder)) + out["name_kept"] = retry(lambda: str(doc.Name)) == name + for label, path in (("all", base + ".dxf"), ("empty", base + "_empty.dxf")): + read = ezdxf.readfile(path) + out[label] = { + "model": sorted(e.dxftype() for e in read.modelspace()), + "Layout1": sorted(e.dxftype() for e in read.layouts.get("Layout1")), + "has_unused_block": "MEAS_UNUSED" in read.blocks, + } +finally: + retry(lambda: doc.Close(False)) +print(json.dumps(out, indent=1)) +PY +``` + +Expected (what this plan measured on 2026-09-24): `"dot_dxf": "Invalid argument"`, `"files": ["snap.DXF", "snap_empty.DXF"]`, `"name_kept": true`, and for both `all` and `empty`: `"model": ["LINE"]`, `"Layout1": ["LINE"]`, `"has_unused_block": true`. If the live seat answers differently, the docstring of `snapshot.py` is corrected to what it printed (and the fake in `tests/test_understand_snapshot.py` with it) — the measurement is the authority, not this plan. If AutoCAD is not running, the COM path stays fake-tested: say so in the task report; Task 12's smoke runs it live. + +- [ ] **Step 20: Gates** + +``` +uv run --frozen ruff format engineering/understand config.py tests/test_understand_labels.py tests/test_understand_vocab.py tests/test_understand_snapshot.py tests/test_office_extra.py +uv run --frozen ruff check engineering/understand config.py tests +uv run --frozen pytest -q --deselect tests/test_release_consistency.py::test_document_numbers_match_the_tree +``` + +Expected: ruff clean; the whole suite green. `test_document_numbers_match_the_tree` is deselected, and the report says so: this task adds 116 tests, so the README collected-tests figure drifts — Task 11 retakes it on main (Global Constraints, known red). + +- [ ] **Step 21: Commit** + +``` +git add engineering/understand config.py pyproject.toml uv.lock README.md .env.example tests/test_understand_labels.py tests/test_understand_vocab.py tests/test_understand_snapshot.py tests/test_office_extra.py +git -c user.name="Umut CAN" -c user.email="umutcan.edizaslan@gmail.com" commit -m "$(cat <<'MSG' +feat(understand): the snapshot, the vocabulary and the label parsers; MAX_DXF_BYTES 512 MB; the office extra + +Track H's foundation, landed on main before its three groups branch. + +- engineering/understand/snapshot.py: one bulk read of a drawing into frozen + WCS records (OCS translated through backends/ocs.py, text decoded, an + INSERT boxed by its block geometry without its attribute texts). A DXF is + read with ezdxf after the size check; the headless engine's document is + read under its own lock; the live engine exports the active document with + Document.Export(/snap, "DXF", ) - never renamed, deleted + afterwards - and a DWG is opened read-only, exported and closed. Measured on + AutoCAD 2026: ".dxf" is refused with Invalid argument, the file lands as + snap.DXF, model space, paper space and unused block definitions all reach + it, and the selection set does not narrow a DXF export. +- engineering/understand/labels.py: %%c, \U+2205, the U+2300 sign, o-stroke + and MTEXT format runs all read as one diameter spelling; sizes are kept as + drawn and never equated; Cyrillic look-alike capitals fold into Latin tags; + panels, wiring callouts and electrical data (kW, phases, volts, N, VFD) in + EN / TR / RU / NL / DE. A power that is not written stays None. +- engineering/understand/vocab.py: layer classification with a confidence and + the deciding keyword, supply / return words, room labels and block kinds in + five languages. A CIP layer without a direction word is not guessed. +- MAX_DXF_BYTES defaults to 512 MB (a real layout was 188 MB); the refusal + names the size and the variable. New `office` extra (openpyxl), also in + `full`; uv.lock regenerated. + +Co-Authored-By: Claude Opus 5.5 (1M context) +Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd +MSG +)" +``` + +--- + +### Task 2: The synthetic plant pair + +**Files:** +- Create: `tests/fixtures/__init__.py` +- Create: `tests/fixtures/plant_pair.py` +- Create: `tests/test_plant_pair.py` +- Test: `tests/test_plant_pair.py` + +**Interfaces:** +- Consumes: `ezdxf` 1.4 (`ezdxf.new`, `add_blockref`, `add_lwpolyline`, `add_multileader_mtext(...).set_content / add_leader_line(ConnectionSide.left, [Vec2]) / build(insert=Vec2)`, `ezdxf.bbox.extents`, the model-space layout's `dxf.extmin` / `dxf.extmax`, which ezdxf copies into `$EXTMIN` / `$EXTMAX` on save — a direct header write is overwritten, verified for this plan); from Task 1 — `engineering/understand/labels.py::parse_diameters, parse_tag, parse_electrical, wiring_target`, `engineering/understand/vocab.py::classify_layer, supply_return, room_label`, `engineering/understand/snapshot.py::read_snapshot` (used by one cross-check test only; the generator itself never calls the code it measures). +- Produces: `tests/fixtures/plant_pair.py::build_plant_pair(directory: str | Path) -> dict` — writes `pid.dxf` and `layout.dxf`, returns the skeleton's truth dict exactly: + `{"pid": str, "layout": str, "scale_verdict": "schematic", "layout_insunits": 1, "layout_true_unit": "mm", "outlier_handle": str, "cluster_count": 2, "layer_services": {layer: service}, "pipe_runs": [{"service", "diameter", "tags": [..], "layout_length_mm": float, "status": "A"|"B"|"C"}], "cable_rows": [{"tag", "panel", "kw": float | None, "length_mm": float, "roundup_m": int}], "rooms": [{"number", "name", "polygon": [[x, y], ...]}]}` + plus additive keys that no pinned key changes: per run `"id"`, `"layer"`, `"diameters": {size: schematic length on the P&ID}`, `"pid_length_mm"`; per cable row `"phases"`, `"voltage"`, `"vfd"`; top level `"pid_insunits": 4`, `"copy_offset": [100000.0, 0.0]`, `"allowance": 0.20`, `"label_offset_mm": 300.0`, `"tag_positions": {"layout": {tag: (x, y)}, "pid": {tag: (x, y)}}`, `"scale": {"overall", "room_1", "room_2": {"pairs", "median", "within_10pct"}, "room_1_factor": 1.3}`. Also module constants `RUNS`, `LAYOUT_TAGS`, `ROOM1_TAGS`, `PID_ROOM2`, `LAYER_SERVICES`, `ALLOWANCE`, `LABEL_OFFSET`, and the helpers `manhattan(a, b)`, `rmst_length(points)`, `scale_statistics(pid, layout, tags)`, `pid_tags()`. + Conventions later tasks rely on: `tags` is **sorted**; a layout tag's position is its TEXT insertion, which is also the centre of the equipment circle it labels (so "tag position" and "equipment position" are one point by construction); `layout_length_mm` is `None` for the status-C run (`PR4`, ending on `T105`, which only the P&ID has); `PR3` crosses a reducer, so it has two sizes — `"diameter"` is the one at its first tag (`Ø51`) and `"diameters"` gives both; the layout draws the plan twice, so each shared tag appears twice on it and is unique only within a copy (Review Focus 3). + +The truth, computed by the generator and worked by hand in the test comments: nine runs — `PR1` product Ø51 A 5500 mm; `PR2` product SMS51 B (a T-junction, an inline valve, an arc bend, one segment drawn twice) 13200; `PR3` product Ø51→Ø38 A (a reducer) 9000; `PR4` product DN20 C (to T105) —; `CS1` CIP supply Ø38 A 20700; `CS2` CIP supply 20x27 B (an inline valve) 8000; `CR1` CIP return Ø38 A 25700; `CR2` CIP return Ø38 A **drawn on the supply layer** 10100; `UT1` ice water DN25 A 7100 (a utility, outside the default takeoff scope). Cables: M11 → CP-1 6900 mm, 9 m; T102 → CP-1 14100, 17; M12 → CP-2 15800, 19; T202 → CP-2 8000, 10 (no power stated). Every name is invented; no value comes from a client drawing. + +- [ ] **Step 1: Write the failing test** + +Create `tests/test_plant_pair.py`: + +```python +"""The synthetic plant pair opens, holds what it claims, and its truth is right. + +Every expected number is worked out by hand in the comment beside it, from the +positions listed in tests/fixtures/plant_pair.py - never read back from the +generator's own arithmetic. +""" + +from __future__ import annotations + +import math + +import ezdxf +import pytest + +from tests.fixtures import plant_pair +from tests.fixtures.plant_pair import build_plant_pair + + +@pytest.fixture(scope="module") +def pair(tmp_path_factory): + return build_plant_pair(tmp_path_factory.mktemp("plant")) + + +def _count(msp, kind, layer=None): + return sum(1 for e in msp if e.dxftype() == kind and (layer is None or e.dxf.layer == layer)) + + +def _texts(msp): + out = [] + for e in msp: + if e.dxftype() == "TEXT": + out.append(e.dxf.text) + elif e.dxftype() == "MTEXT": + out.append(e.text) + return out + + +def test_both_files_open_with_their_declared_units(pair): + pid = ezdxf.readfile(pair["pid"]) + layout = ezdxf.readfile(pair["layout"]) + assert pid.header["$INSUNITS"] == 4 + assert layout.header["$INSUNITS"] == pair["layout_insunits"] == 1 + assert pair["layout_true_unit"] == "mm" + + +def test_the_pid_holds_the_symbols_the_runs_and_the_labels(pair): + msp = ezdxf.readfile(pair["pid"]).modelspace() + # 8 tanks (T101-T105, T201-T203), 2 pumps, 2 panels, 2 valves, 1 reducer + assert _count(msp, "INSERT") == 15 + assert _count(msp, "INSERT", "KLEPPEN") == 3 + assert _count(msp, "TEXT", "TAGS") == 12 # the 11 shared tags and T105 + assert _count(msp, "ARC", "P_product piping") == 1 + texts = _texts(msp) + for spelling in ( + "{\\fArial|b0;Ø51}", + "%%c38", + "\\U+2205 38", + "⌀38", + "ø38", + "SMS51", + "DN20", + "DN25", + "20x27", + ): + assert spelling in texts, spelling + assert texts.count("ПОДАЧА") == 2 # CS1, CS2 + assert texts.count("ВОЗВРАТ") == 2 # CR1, CR2 + assert _count(msp, "MTEXT", "E_power") == 4 + + +def test_one_segment_is_drawn_twice_a_line_over_a_polyline(pair): + msp = ezdxf.readfile(pair["pid"]).modelspace() + start, end = (14150.0, 7000.0), (17250.0, 7000.0) + lines = [ + e + for e in msp.query("LINE") + if (e.dxf.start.x, e.dxf.start.y) == start and (e.dxf.end.x, e.dxf.end.y) == end + ] + polylines = [ + e for e in msp.query("LWPOLYLINE") if [tuple(p) for p in e.get_points("xy")] == [start, end] + ] + assert len(lines) == 1 and len(polylines) == 1 + + +def test_a_cip_return_is_drawn_on_the_supply_layer(pair): + (cr2,) = [run for run in pair["pipe_runs"] if run["id"] == "CR2"] + assert cr2["layer"] == "P_cipsupplyline" and cr2["service"] == "cip_return" + + +def test_the_layout_carries_two_copies_one_outlier_and_the_rooms(pair): + doc = ezdxf.readfile(pair["layout"]) + msp = doc.modelspace() + # per copy: 7 tank and 2 pump circles, 2 panel boxes, 11 tag texts, + # 2 room labels, 4 callouts, 12 wall lines; plus one stray line + assert _count(msp, "CIRCLE") == 18 + assert _count(msp, "LWPOLYLINE", "E-PANEL") == 4 + assert _count(msp, "TEXT", "TAGS") == 22 + assert _count(msp, "TEXT", "ROOMS") == 4 + assert _count(msp, "MULTILEADER") == 8 + assert _count(msp, "LINE", "WALLS") == 24 + tags = [e.dxf.text for e in msp.query("TEXT") if e.dxf.layer == "TAGS"] + assert all(tags.count(tag) == 2 for tag in set(tags)) + assert "T105" not in tags + outlier = doc.entitydb[pair["outlier_handle"]] + assert outlier.dxftype() == "LINE" and outlier.dxf.start.x == -5.0e9 + assert doc.header["$EXTMIN"][0] == -5.0e9 # the declared extents include it + assert pair["cluster_count"] == 2 and pair["copy_offset"] == [100000.0, 0.0] + + +def test_the_wiring_callouts_point_at_their_loads(pair): + msp = ezdxf.readfile(pair["layout"]).modelspace() + first_copy = {} + for e in msp.query("MULTILEADER"): + tip = e.context.leaders[0].lines[0].vertices[0] + if tip.x < 50000.0: + first_copy[e.context.mtext.default_content] = (tip.x, tip.y) + assert first_copy == { + "Wiring to CP1": (5500.0, 8200.0), # M11 + "Wiring to CP 1": (8000.0, 3500.0), # T102 + "Wiring to CP-2": (26300.0, 8400.0), # M12 + "WIRING TO CP2": (28700.0, 3000.0), # T202 + } + + +def test_the_layer_services_truth(pair): + assert pair["layer_services"] == { + "P_product piping": "product", + "P_cipsupplyline": "cip_supply", + "P_cipreturnline": "cip_return", + "P_ijswater": "ice_water", + "E_power": "electrical", + } + + +def test_the_pid_is_schematic_by_its_own_placement(pair): + scale = pair["scale"] + assert pair["scale_verdict"] == "schematic" + assert scale["overall"]["pairs"] == 55 # 11 shared tags, every pair over 2 m + assert scale["overall"]["within_10pct"] <= 0.5 + assert scale["room_1"]["within_10pct"] == 1.0 + assert scale["room_1"]["median"] == pytest.approx(1.3, abs=1e-12) + # T101 (3000, 3000) and T102 (8000, 3500) on the layout; x1.3 on the P&ID + pid = pair["tag_positions"]["pid"] + assert pid["T101"] == pytest.approx((3900.0, 3900.0)) + assert pid["T102"] == pytest.approx((10400.0, 4550.0)) + + +def test_layout_lengths_are_manhattan_and_the_rectilinear_mst(pair): + runs = {run["id"]: run for run in pair["pipe_runs"]} + # PR1: T101 (3000, 3000) - T102 (8000, 3500): 5000 + 500 + assert runs["PR1"]["layout_length_mm"] == 5500.0 + # PR2: T102 (8000, 3500), T103 (13500, 3000), M11 (5500, 8200): + # T102-T103 6000, T102-M11 7200, T103-M11 13200 -> MST 6000 + 7200 + assert runs["PR2"]["layout_length_mm"] == 13200.0 + # CS1: T201 (23500, 3200) - T101 (3000, 3000): 20500 + 200 + assert runs["CS1"]["layout_length_mm"] == 20700.0 + # PR4 ends on T105, which the layout does not have + assert runs["PR4"]["layout_length_mm"] is None and runs["PR4"]["status"] == "C" + assert [runs[i]["status"] for i in ("PR1", "PR2", "PR3", "CS2")] == ["A", "B", "A", "B"] + + +def test_schematic_lengths_count_the_double_drawn_segment_once(pair): + runs = {run["id"]: run for run in pair["pipe_runs"]} + # PR2: 1850 + 6700 + 3100 + 2200 + 3360 straight, and a quarter arc of r 300 + assert runs["PR2"]["pid_length_mm"] == pytest.approx(17210.0 + 150.0 * math.pi, abs=1e-9) + # PR3 across the reducer: 1850 + 3100 at Ø51, 3750 + 1500 at Ø38 + assert runs["PR3"]["diameters"] == {"Ø51": 4950.0, "Ø38": 5250.0} + + +def test_cable_truth_is_manhattan_then_the_allowance_rounded_up(pair): + rows = {row["tag"]: row for row in pair["cable_rows"]} + # M11 (5500, 8200) to CP-1 (1200, 10800): 4300 + 2600 = 6900 mm; + # 6.9 m x 1.2 = 8.28 -> 9 + assert (rows["M11"]["panel"], rows["M11"]["length_mm"], rows["M11"]["roundup_m"]) == ( + "CP-1", + 6900.0, + 9, + ) + # M12 (26300, 8400) to CP-2 (35000, 1300): 8700 + 7100 = 15800; 18.96 -> 19 + assert (rows["M12"]["length_mm"], rows["M12"]["roundup_m"], rows["M12"]["vfd"]) == ( + 15800.0, + 19, + True, + ) + assert rows["T202"]["kw"] is None, "the heater states no power; none is invented" + assert [rows[t]["kw"] for t in ("M11", "T102", "M12")] == [5.5, 2.2, 7.5] + + +def test_the_rooms_truth(pair): + assert [(r["number"], r["name"]) for r in pair["rooms"]] == [ + ("1", "PROCESS HALL"), + ("2", "МОЙКА"), + ] + assert pair["rooms"][1]["polygon"] == [ + [20200.0, 0.0], + [36000.0, 0.0], + [36000.0, 12000.0], + [20200.0, 12000.0], + ] + + +def test_the_generator_refuses_a_pair_that_would_read_to_scale(tmp_path, monkeypatch): + stretched = { + tag: (1.3 * x, 1.3 * y) + for tag, (x, y) in plant_pair.LAYOUT_TAGS.items() + if tag not in plant_pair.ROOM1_TAGS + } + monkeypatch.setattr(plant_pair, "PID_ROOM2", stretched) + with pytest.raises(AssertionError, match="schematic"): + build_plant_pair(tmp_path) + assert not (tmp_path / "pid.dxf").exists() + + +def test_every_label_reads_as_the_truth_says_through_the_task_1_parsers(pair): + """The fixture's declared readings agree with engineering/understand's parsers.""" + from engineering.understand.labels import ( + parse_diameters, + parse_electrical, + parse_tag, + wiring_target, + ) + from engineering.understand.snapshot import read_snapshot + from engineering.understand.vocab import classify_layer, room_label, supply_return + + for run in plant_pair.RUNS: + for _kind, raw, _at, read in run.labels: + assert parse_diameters(raw) == (read,), (run.id, raw) + for word, _at in run.words: + expected = "supply" if run.service.endswith("supply") else "return" + assert supply_return(word) == expected, run.id + for layer, service in pair["layer_services"].items(): + assert classify_layer(layer)["service"] == service, layer + pid = read_snapshot(pair["pid"]) + tags = [parse_tag(r.text) for r in pid.records if r.type == "TEXT" and r.layer == "TAGS"] + assert sorted(tags) == sorted([*pair["tag_positions"]["layout"], "T105"]) + electrical = [r.text for r in pid.records if r.layer == "E_power"] + assert sorted(parse_electrical(t)["kw"] or 0.0 for t in electrical) == [0.0, 2.2, 5.5, 7.5] + layout = read_snapshot(pair["layout"]) + rooms = [room_label(r.text) for r in layout.records if r.layer == "ROOMS"] + assert {(r["number"], r["name"]) for r in rooms} == { + (room["number"], room["name"]) for room in pair["rooms"] + } + panels = {wiring_target(r.text) for r in layout.records if r.type == "MULTILEADER"} + assert panels == {"CP-1", "CP-2"} +``` + +- [ ] **Step 2: Run it to verify it fails** + +Run: `uv run --frozen pytest tests/test_plant_pair.py -q` +Expected: FAIL at collection — `ModuleNotFoundError: No module named 'tests.fixtures'`. + +- [ ] **Step 3: Write the fixture package and the generator** + +`tests/fixtures/__init__.py`: + +```python +"""Synthetic drawings the readers are measured on (never a client's drawing).""" +``` + +`tests/fixtures/plant_pair.py`: + +```python +"""The synthetic plant pair: a schematic P&ID and its layout, with known answers. + +Every track H reader is measured on this pair and never on a client drawing +(spec §14). It reproduces each field finding of the design spec (§1, §15) with +invented names and invented numbers: + +* **Tags in both drawings** - T101..T104, M11, CP-1 in room 1; T201..T203, M12, + CP-2 in room 2. T105 is on the P&ID only, so the run to it is not routable. +* **The P&ID is not to scale** - room 1 is the layout stretched 1.3x about the + origin, room 2 is rearranged outright. The generator computes the scale + statistics of its own placement (pairs at least 2 m apart, the ratio of their + distances, the share within ±10 % of the median) and refuses to write a pair + that would not read as ``schematic``. +* **Service layers in English and Dutch** - ``P_product piping``, + ``P_cipsupplyline``, ``P_cipreturnline``, ``P_ijswater`` (ice water, a utility + outside the default takeoff scope), valves and the reducer on ``KLEPPEN``. +* **Russian supply / return words** - ПОДАЧА beside the CIP supply runs, + ВОЗВРАТ beside the returns, and one return (``CR2``) drawn on the *supply* + layer: only the word says which way it runs. +* **Size labels as drafters write them** - ``{\\fArial|b0;Ø51}`` (an MTEXT format + run), ``Ø51``, ``%%c38``, ``\\U+2205 38``, ``⌀38``, ``ø38``, ``SMS51``, ``DN20``, + ``DN25``, ``20x27``; each 300 mm from the segment it names and at least 600 mm + from every other run (the generator checks it). +* **Fittings on runs** - a reducer (``VERLOOPSTUK``) between Ø51 and Ø38 on + ``PR3``, inline valves (``KLEP_VLINDER``) on ``PR2`` and ``CS2``; the pipe stops + at each fitting's box and continues on its far side. +* **A double-drawn segment** - on ``PR2`` a LINE lies exactly over an LWPOLYLINE + segment; it must count once. +* **Electrical texts in Russian** - power, phases, voltage, N and PE, ЧРП (VFD) + 1500 mm right of each load's tag; the heater on T202 states no power. +* **The layout lies about its units** - ``$INSUNITS = 1`` (inches) over + millimetre geometry; one stray LINE 5,000 km away; two copies of the plan + 100 m apart, so every tag is on the layout twice and is unique only within a + copy. +* **Rooms** - wall outlines (200 mm walls) and labels ``ROOM 1 PROCESS HALL`` / + ``ПОМЕЩЕНИЕ 2 МОЙКА``; ``Wiring to CP1`` style MULTILEADER callouts from each + load to its panel, in four spellings. + +Truth values are **computed here from the positions placed here** - Manhattan +distances, the rectilinear minimum spanning tree of each run's tags, the cable +lengths rounded up after the allowance - never typed. A position on the layout +is the insertion point of the tag's TEXT, which is also the centre of the +equipment circle it labels, so "the tag's position" and "the equipment's +position" are one point by construction. +""" + +from __future__ import annotations + +import itertools +import math +import statistics +from dataclasses import dataclass, field +from pathlib import Path + +import ezdxf +from ezdxf import bbox as ezbbox +from ezdxf.math import Vec2 +from ezdxf.render.mleader import ConnectionSide + +Pt = tuple[float, float] + +__all__ = ["ALLOWANCE", "LABEL_OFFSET", "build_plant_pair"] + +#: The takeoffs' default allowance; the truth's ROUNDUP metres use it. +ALLOWANCE = 0.20 +#: How far a size label or a service word sits from the segment it names. +LABEL_OFFSET = 300.0 +#: The stretch of room 1 on the P&ID. +ROOM1_FACTOR = 1.3 +#: The second plan copy on the layout is this far to the right of the first. +COPY_OFFSET: Pt = (100_000.0, 0.0) +#: The stray entity on the layout. +OUTLIER_START: Pt = (-5.0e9, 3.0e9) +#: Tank, pump and panel sizes (P&ID radii / layout radii / panel box). +PID_TANK_R, PID_PUMP_R = 600.0, 300.0 +LAYOUT_TANK_R, LAYOUT_PUMP_R = 1000.0, 400.0 +PANEL_W, PANEL_H = 1000.0, 400.0 +TEXT_H = 250.0 + +#: Tag positions on the layout (first copy), mm. The P&ID positions follow. +LAYOUT_TAGS: dict[str, Pt] = { + "T101": (3000.0, 3000.0), + "T102": (8000.0, 3500.0), + "T103": (13500.0, 3000.0), + "T104": (17000.0, 8500.0), + "M11": (5500.0, 8200.0), + "CP-1": (1200.0, 10800.0), + "T201": (23500.0, 3200.0), + "T202": (28700.0, 3000.0), + "T203": (33000.0, 8800.0), + "M12": (26300.0, 8400.0), + "CP-2": (35000.0, 1300.0), +} +ROOM1_TAGS = ("T101", "T102", "T103", "T104", "M11", "CP-1") +#: Room 2 on the P&ID: rearranged, not scaled. +PID_ROOM2: dict[str, Pt] = { + "T201": (25000.0, 3000.0), + "T202": (29000.0, 14000.0), + "T203": (25500.0, 11000.0), + "M12": (32000.0, 4000.0), + "CP-2": (28000.0, 7500.0), +} +#: On the P&ID only: the run to it cannot be routed on the layout. +PID_ONLY = {"T105": (22100.0, 16500.0)} + +ROOMS = ( + {"number": "1", "name": "PROCESS HALL", "box": (0.0, 0.0, 20000.0, 12000.0)}, + { + "number": "2", + "name": "МОЙКА", + "box": (20200.0, 0.0, 36000.0, 12000.0), + }, +) +ROOM_LABELS = ( + ("ROOM 1 PROCESS HALL", (10000.0, 11000.0)), + ( + "ПОМЕЩЕНИЕ 2 МОЙКА", + (28000.0, 11000.0), + ), +) +OUTER_WALL = (-200.0, -200.0, 36200.0, 12200.0) + +SUPPLY_WORD = "ПОДАЧА" +RETURN_WORD = "ВОЗВРАТ" + +#: (tag, the Russian electrical text beside it, stated kW or None, VFD). +LOADS = ( + ("M11", "Насос P = 5,5 кВт\\P3 ф. 400 В + N + PE", 5.5, False), + ("T102", "Мешалка P = 2,2 кВт\\P3 ф. 400 В + N + PE", 2.2, False), + ("M12", "Насос CIP P = 7,5 кВт\\P3 ф. 400 В + N + PE\\PЧРП", 7.5, True), + ("T202", "Нагреватель\\P3 ф. 400 В + N + PE", None, False), +) +#: (load tag, callout text on the layout, the panel it names). +CALLOUTS = ( + ("M11", "Wiring to CP1", "CP-1"), + ("T102", "Wiring to CP 1", "CP-1"), + ("M12", "Wiring to CP-2", "CP-2"), + ("T202", "WIRING TO CP2", "CP-2"), +) +ELECTRICAL_DX = 1500.0 +CALLOUT_DX, CALLOUT_DY = 1500.0, 1500.0 + + +@dataclass(frozen=True) +class _Run: + """One pipe run of the P&ID and what the takeoffs must find for it.""" + + id: str + layer: str + service: str + diameter: str + tags: tuple[str, ...] + status: str + paths: tuple[tuple[Pt, ...], ...] + labels: tuple[tuple[str, str, Pt, str], ...] # (TEXT | MTEXT, as drawn, insert, as read) + arcs: tuple[tuple[Pt, float, float, float], ...] = () # centre, radius, start, end (deg) + fittings: tuple[tuple[str, Pt, float], ...] = () # block, centre, rotation + words: tuple[tuple[str, Pt], ...] = () + duplicate: tuple[Pt, Pt] | None = None + notes: tuple[str, ...] = field(default_factory=tuple) + + +RUNS: tuple[_Run, ...] = ( + _Run( + "PR1", + "P_product piping", + "product", + "Ø51", + ("T101", "T102"), + "A", + paths=(((4500.0, 4200.0), (9800.0, 4200.0)),), + labels=(("MTEXT", "{\\fArial|b0;Ø51}", (7150.0, 4500.0), "Ø51"),), + ), + _Run( + "PR2", + "P_product piping", + "product", + "SMS51", + ("M11", "T102", "T103"), + "B", + paths=( + ((10400.0, 5150.0), (10400.0, 7000.0)), + ((7150.0, 7000.0), (13850.0, 7000.0)), + ((14150.0, 7000.0), (17250.0, 7000.0)), + ((17550.0, 6700.0), (17550.0, 4500.0)), + ((7150.0, 7000.0), (7150.0, 10360.0)), + ), + labels=(("TEXT", "SMS51", (10700.0, 6075.0), "SMS51"),), + arcs=(((17250.0, 6700.0), 300.0, 0.0, 90.0),), + fittings=(("KLEP_VLINDER", (14000.0, 7000.0), 0.0),), + duplicate=((14150.0, 7000.0), (17250.0, 7000.0)), + notes=("T-junction at (10400, 7000)", "valve continued", "one segment drawn twice"), + ), + _Run( + "PR3", + "P_product piping", + "product", + "Ø51", + ("T103", "T104"), + "A", + paths=( + ((18150.0, 3900.0), (20000.0, 3900.0), (20000.0, 7000.0)), + ((20000.0, 7300.0), (20000.0, 11050.0), (21500.0, 11050.0)), + ), + labels=( + ("TEXT", "Ø51", (19075.0, 4200.0), "Ø51"), + ("TEXT", "Ø51", (20300.0, 5450.0), "Ø51"), + ("TEXT", "%%c38", (20300.0, 9175.0), "Ø38"), + ("TEXT", "%%c38", (20750.0, 11350.0), "Ø38"), + ), + fittings=(("VERLOOPSTUK", (20000.0, 7150.0), 90.0),), + notes=("reducer between the two sizes",), + ), + _Run( + "PR4", + "P_product piping", + "product", + "DN20", + ("T104", "T105"), + "C", + paths=(((22100.0, 11650.0), (22100.0, 15900.0)),), + labels=(("TEXT", "DN20", (22400.0, 13775.0), "DN20"),), + notes=("T105 is not on the layout",), + ), + _Run( + "CS1", + "P_cipsupplyline", + "cip_supply", + "Ø38", + ("T101", "T201"), + "A", + paths=(((25000.0, 2400.0), (25000.0, 1500.0), (3900.0, 1500.0), (3900.0, 3300.0)),), + labels=( + ("MTEXT", "\\U+2205 38", (25300.0, 1950.0), "Ø38"), + ("MTEXT", "\\U+2205 38", (14450.0, 1800.0), "Ø38"), + ("MTEXT", "\\U+2205 38", (4200.0, 2400.0), "Ø38"), + ), + words=((SUPPLY_WORD, (14450.0, 1200.0)),), + ), + _Run( + "CS2", + "P_cipsupplyline", + "cip_supply", + "20x27", + ("M12", "T201"), + "B", + paths=( + ((25600.0, 3000.0), (27850.0, 3000.0)), + ((28150.0, 3000.0), (32000.0, 3000.0), (32000.0, 3700.0)), + ), + labels=(("TEXT", "20x27", (26725.0, 3300.0), "20x27"),), + fittings=(("KLEP_VLINDER", (28000.0, 3000.0), 0.0),), + words=((SUPPLY_WORD, (30075.0, 2700.0)),), + ), + _Run( + "CR1", + "P_cipreturnline", + "cip_return", + "Ø38", + ("T101", "T202"), + "A", + paths=( + ( + (3300.0, 3900.0), + (2500.0, 3900.0), + (2500.0, 19000.0), + (29000.0, 19000.0), + (29000.0, 14600.0), + ), + ), + labels=( + ("TEXT", "⌀38", (2900.0, 4200.0), "Ø38"), + ("TEXT", "⌀38", (2200.0, 11450.0), "Ø38"), + ("TEXT", "⌀38", (15750.0, 19300.0), "Ø38"), + ("TEXT", "⌀38", (29300.0, 16800.0), "Ø38"), + ), + words=((RETURN_WORD, (15750.0, 18700.0)),), + ), + _Run( + "CR2", + "P_cipsupplyline", + "cip_return", + "Ø38", + ("T202", "T203"), + "A", + paths=(((25500.0, 11600.0), (25500.0, 14000.0), (28400.0, 14000.0)),), + labels=( + ("TEXT", "ø38", (25200.0, 12800.0), "Ø38"), + ("TEXT", "ø38", (26950.0, 14300.0), "Ø38"), + ), + words=((RETURN_WORD, (26950.0, 13700.0)),), + notes=("a return drawn on the supply layer",), + ), + _Run( + "UT1", + "P_ijswater", + "ice_water", + "DN25", + ("M12", "T203"), + "A", + paths=(((32000.0, 4300.0), (32000.0, 11000.0), (26100.0, 11000.0)),), + labels=( + ("TEXT", "DN25", (32300.0, 7650.0), "DN25"), + ("TEXT", "DN25", (29050.0, 11300.0), "DN25"), + ), + notes=("a utility: outside the default takeoff scope",), + ), +) + +LAYER_SERVICES = { + "P_product piping": "product", + "P_cipsupplyline": "cip_supply", + "P_cipreturnline": "cip_return", + "P_ijswater": "ice_water", + "E_power": "electrical", +} + + +def pid_tags() -> dict[str, Pt]: + """Tag positions on the P&ID: room 1 stretched, room 2 rearranged, T105 alone.""" + out = { + tag: (ROOM1_FACTOR * x, ROOM1_FACTOR * y) + for tag, (x, y) in LAYOUT_TAGS.items() + if tag in ROOM1_TAGS + } + out.update(PID_ROOM2) + out.update(PID_ONLY) + return out + + +def manhattan(a: Pt, b: Pt) -> float: + return abs(a[0] - b[0]) + abs(a[1] - b[1]) + + +def rmst_length(points: list[Pt]) -> float: + """Rectilinear minimum spanning tree length (Prim, Manhattan weights).""" + if len(points) < 2: + return 0.0 + inside = {0} + total = 0.0 + while len(inside) < len(points): + best = min( + (manhattan(points[i], points[j]), j) + for i in inside + for j in range(len(points)) + if j not in inside + ) + total += best[0] + inside.add(best[1]) + return total + + +def scale_statistics(pid: dict[str, Pt], layout: dict[str, Pt], tags) -> dict: + """Spec §6 over ``tags``: pairs at least 2 m apart on both drawings.""" + ratios = [] + for a, b in itertools.combinations(sorted(tags), 2): + on_layout = math.dist(layout[a], layout[b]) + on_pid = math.dist(pid[a], pid[b]) + if on_layout >= 2000.0 and on_pid >= 2000.0: + ratios.append(on_pid / on_layout) + median = statistics.median(ratios) + within = sum(1 for r in ratios if abs(r - median) <= 0.1 * median) / len(ratios) + return {"pairs": len(ratios), "median": median, "within_10pct": within} + + +def _segments(run: _Run) -> list[tuple[Pt, Pt]]: + out = [] + for path in run.paths: + out.extend(zip(path, path[1:], strict=False)) + return out + + +def _segment_distance(p: Pt, a: Pt, b: Pt) -> float: + ax, ay = a + bx, by = b + dx, dy = bx - ax, by - ay + t = max(0.0, min(1.0, ((p[0] - ax) * dx + (p[1] - ay) * dy) / (dx * dx + dy * dy))) + return math.hypot(p[0] - (ax + t * dx), p[1] - (ay + t * dy)) + + +def _arc_length(arc) -> float: + _centre, radius, start, end = arc + return radius * math.radians((end - start) % 360.0) + + +def _check_labels() -> None: + """Every label names its own segment at LABEL_OFFSET and no other run within 2x.""" + for run in RUNS: + own = _segments(run) + others = [s for other in RUNS if other is not run for s in _segments(other)] + for _kind, _raw, at, _read in run.labels: + mine = min(_segment_distance(at, a, b) for a, b in own) + theirs = min(_segment_distance(at, a, b) for a, b in others) + if abs(mine - LABEL_OFFSET) > 1e-9 or theirs <= 2 * LABEL_OFFSET: + raise AssertionError(f"{run.id} label at {at}: own {mine}, other run {theirs}") + if run.status == "A": + for a, b in own: + near = min(_segment_distance(lab[2], a, b) for lab in run.labels) + if abs(near - LABEL_OFFSET) > 1e-9: + raise AssertionError(f"{run.id}: segment {a}-{b} has no direct label") + + +def _segment_diameters(run: _Run) -> dict[str, float]: + """Drawn length per size: a segment takes the label 300 mm from it (direct), + else the run's size (continuity); arcs are continuity.""" + lengths: dict[str, float] = {} + for a, b in _segments(run): + size = run.diameter + for _kind, _raw, at, read in run.labels: + if abs(_segment_distance(at, a, b) - LABEL_OFFSET) <= 1e-9: + size = read + break + lengths[size] = lengths.get(size, 0.0) + math.dist(a, b) + for arc in run.arcs: + lengths[run.diameter] = lengths.get(run.diameter, 0.0) + _arc_length(arc) + return lengths + + +def _blocks(doc) -> None: + tank = doc.blocks.new("TANK_V") + tank.add_circle((0.0, 0.0), PID_TANK_R) + pump = doc.blocks.new("POMP") + pump.add_circle((0.0, 0.0), PID_PUMP_R) + pump.add_lwpolyline([(-150.0, -150.0), (150.0, 0.0), (-150.0, 150.0)], close=True) + panel = doc.blocks.new("SCHAKELKAST") + panel.add_lwpolyline( + [ + (-PANEL_W / 2, -PANEL_H / 2), + (PANEL_W / 2, -PANEL_H / 2), + (PANEL_W / 2, PANEL_H / 2), + (-PANEL_W / 2, PANEL_H / 2), + ], + close=True, + ) + valve = doc.blocks.new("KLEP_VLINDER") + valve.add_lwpolyline( + [(-150.0, -100.0), (150.0, 100.0), (150.0, -100.0), (-150.0, 100.0)], close=True + ) + reducer = doc.blocks.new("VERLOOPSTUK") + reducer.add_lwpolyline( + [(-150.0, -100.0), (150.0, -60.0), (150.0, 60.0), (-150.0, 100.0)], close=True + ) + + +def _write_pid(path: Path) -> None: + doc = ezdxf.new("R2018") + doc.header["$INSUNITS"] = 4 + for name in (*LAYER_SERVICES, "KLEPPEN", "TEKST", "TAGS", "EQUIPMENT"): + if name not in doc.layers: + doc.layers.add(name) + _blocks(doc) + msp = doc.modelspace() + for tag, at in pid_tags().items(): + block = ( + "SCHAKELKAST" if tag.startswith("CP") else "POMP" if tag.startswith("M") else "TANK_V" + ) + msp.add_blockref(block, at, dxfattribs={"layer": "EQUIPMENT"}) + msp.add_text(tag, dxfattribs={"insert": at, "height": TEXT_H, "layer": "TAGS"}) + for run in RUNS: + attribs = {"layer": run.layer} + for pts in run.paths: + # a two-point path is a LINE - except the one drawn twice, which is + # an LWPOLYLINE, so its duplicate is a line lying over a polyline + if len(pts) == 2 and tuple(pts) != run.duplicate: + msp.add_line(pts[0], pts[1], dxfattribs=attribs) + else: + msp.add_lwpolyline(pts, dxfattribs=attribs) + for centre, radius, start, end in run.arcs: + msp.add_arc(centre, radius, start, end, dxfattribs=attribs) + if run.duplicate is not None: + msp.add_line(run.duplicate[0], run.duplicate[1], dxfattribs=attribs) + for block, at, rotation in run.fittings: + msp.add_blockref(block, at, dxfattribs={"layer": "KLEPPEN", "rotation": rotation}) + for kind, raw, at, _read in run.labels: + if kind == "MTEXT": + msp.add_mtext( + raw, dxfattribs={"insert": at, "char_height": TEXT_H, "layer": "TEKST"} + ) + else: + msp.add_text(raw, dxfattribs={"insert": at, "height": TEXT_H, "layer": "TEKST"}) + for word, at in run.words: + msp.add_text(word, dxfattribs={"insert": at, "height": TEXT_H, "layer": "TEKST"}) + tags = pid_tags() + for tag, text, _kw, _vfd in LOADS: + x, y = tags[tag] + msp.add_mtext( + text, + dxfattribs={ + "insert": (x + ELECTRICAL_DX, y), + "char_height": TEXT_H, + "layer": "E_power", + }, + ) + _set_extents(doc) + doc.saveas(path) + + +def _set_extents(doc) -> None: + """Declare the model-space extents the way AutoCAD would: over everything. + + ezdxf copies the model-space layout's ``extmin`` / ``extmax`` into + ``$EXTMIN`` / ``$EXTMAX`` when it saves (``Drawing.update_extents``), so the + layout's attributes are what is set; a header write alone is overwritten. + """ + msp = doc.modelspace() + box = ezbbox.extents(msp, fast=True) + msp.dxf.extmin = (box.extmin.x, box.extmin.y, 0.0) + msp.dxf.extmax = (box.extmax.x, box.extmax.y, 0.0) + + +def _rect(msp, box, layer) -> None: + x0, y0, x1, y1 = box + corners = [(x0, y0), (x1, y0), (x1, y1), (x0, y1)] + for a, b in zip(corners, corners[1:] + corners[:1], strict=True): + msp.add_line(a, b, dxfattribs={"layer": layer}) + + +def _write_layout(path: Path) -> str: + doc = ezdxf.new("R2018") + doc.header["$INSUNITS"] = 1 # inches - the geometry is millimetres + for name in ("WALLS", "ROOMS", "EQUIPMENT", "TAGS", "E-PANEL", "E-CALLOUT"): + doc.layers.add(name) + msp = doc.modelspace() + for copy in range(2): + ox, oy = COPY_OFFSET[0] * copy, COPY_OFFSET[1] * copy + + def at(p: Pt, ox=ox, oy=oy) -> Pt: + return (p[0] + ox, p[1] + oy) + + x0, y0, x1, y1 = OUTER_WALL + _rect(msp, (x0 + ox, y0 + oy, x1 + ox, y1 + oy), "WALLS") + for room in ROOMS: + bx0, by0, bx1, by1 = room["box"] + _rect(msp, (bx0 + ox, by0 + oy, bx1 + ox, by1 + oy), "WALLS") + for text, where in ROOM_LABELS: + msp.add_text(text, dxfattribs={"insert": at(where), "height": TEXT_H, "layer": "ROOMS"}) + for tag, where in LAYOUT_TAGS.items(): + centre = at(where) + if tag.startswith("CP"): + cx, cy = centre + msp.add_lwpolyline( + [ + (cx - PANEL_W / 2, cy - PANEL_H / 2), + (cx + PANEL_W / 2, cy - PANEL_H / 2), + (cx + PANEL_W / 2, cy + PANEL_H / 2), + (cx - PANEL_W / 2, cy + PANEL_H / 2), + ], + close=True, + dxfattribs={"layer": "E-PANEL"}, + ) + else: + radius = LAYOUT_PUMP_R if tag.startswith("M") else LAYOUT_TANK_R + msp.add_circle(centre, radius, dxfattribs={"layer": "EQUIPMENT"}) + msp.add_text(tag, dxfattribs={"insert": centre, "height": TEXT_H, "layer": "TAGS"}) + for tag, text, _panel in CALLOUTS: + tip = at(LAYOUT_TAGS[tag]) + builder = msp.add_multileader_mtext("Standard", dxfattribs={"layer": "E-CALLOUT"}) + builder.set_content(text, char_height=TEXT_H) + builder.add_leader_line(ConnectionSide.left, [Vec2(tip)]) + builder.build(insert=Vec2(tip[0] + CALLOUT_DX, tip[1] + CALLOUT_DY)) + stray = msp.add_line(OUTLIER_START, (OUTLIER_START[0] + 1000.0, OUTLIER_START[1])) + _set_extents(doc) + doc.saveas(path) + return str(stray.dxf.handle) + + +def build_plant_pair(directory: str | Path) -> dict: + """Write ``pid.dxf`` and ``layout.dxf`` into ``directory``; return the truth. + + Refuses (``AssertionError``) to write a pair that would not test what it + claims: a label nearer another run than its own, a status-A segment with no + direct label, a P&ID that is not schematic overall or not 1.3x in room 1, + or a cable length whose rounding sits on an integer. + """ + _check_labels() + layout_tags = dict(LAYOUT_TAGS) + pid = pid_tags() + overall = scale_statistics(pid, layout_tags, LAYOUT_TAGS) + room1 = scale_statistics(pid, layout_tags, ROOM1_TAGS) + room2 = scale_statistics(pid, layout_tags, [t for t in LAYOUT_TAGS if t not in ROOM1_TAGS]) + if overall["within_10pct"] > 0.5 or room1["within_10pct"] != 1.0: + raise AssertionError(f"the pair does not read as schematic: {overall}, {room1}") + + out = Path(directory) + out.mkdir(parents=True, exist_ok=True) + pid_path, layout_path = out / "pid.dxf", out / "layout.dxf" + _write_pid(pid_path) + outlier = _write_layout(layout_path) + + pipe_runs = [] + for run in RUNS: + routable = all(tag in layout_tags for tag in run.tags) + pieces = sum(math.dist(a, b) for a, b in _segments(run)) + sum( + _arc_length(a) for a in run.arcs + ) + pipe_runs.append( + { + "id": run.id, + "layer": run.layer, + "service": run.service, + "diameter": run.diameter, + "diameters": _segment_diameters(run), + "tags": sorted(run.tags), + "status": run.status, + "pid_length_mm": pieces, + "layout_length_mm": rmst_length([layout_tags[t] for t in run.tags]) + if routable + else None, + } + ) + + cable_rows = [] + for (tag, _text, kw, vfd), (callout_tag, _callout, panel) in zip(LOADS, CALLOUTS, strict=True): + assert tag == callout_tag + length = manhattan(layout_tags[tag], layout_tags[panel]) + metres = length / 1000.0 * (1.0 + ALLOWANCE) + if abs(metres - round(metres)) < 1e-6: + raise AssertionError(f"{tag}: {metres} m sits on an integer; move the tag") + cable_rows.append( + { + "tag": tag, + "panel": panel, + "kw": kw, + "phases": 3, + "voltage": 400, + "vfd": vfd, + "length_mm": length, + "roundup_m": math.ceil(metres), + } + ) + + rooms = [] + for room in ROOMS: + x0, y0, x1, y1 = room["box"] + rooms.append( + { + "number": room["number"], + "name": room["name"], + "polygon": [[x0, y0], [x1, y0], [x1, y1], [x0, y1]], + } + ) + + return { + "pid": str(pid_path), + "layout": str(layout_path), + "scale_verdict": "schematic", + "layout_insunits": 1, + "layout_true_unit": "mm", + "pid_insunits": 4, + "outlier_handle": outlier, + "cluster_count": 2, + "copy_offset": list(COPY_OFFSET), + "layer_services": dict(LAYER_SERVICES), + "pipe_runs": pipe_runs, + "cable_rows": cable_rows, + "rooms": rooms, + "allowance": ALLOWANCE, + "label_offset_mm": LABEL_OFFSET, + "tag_positions": {"layout": layout_tags, "pid": pid}, + "scale": { + "overall": overall, + "room_1": room1, + "room_2": room2, + "room_1_factor": ROOM1_FACTOR, + }, + } +``` + +- [ ] **Step 4: Run it to verify it passes** + +Run: `uv run --frozen pytest tests/test_plant_pair.py -q` +Expected: `14 passed`. The generator refuses (with `AssertionError`) to write a pair whose own premises fail — a label nearer another run than its own, a status-A segment with no direct label, a P&ID that is not schematic overall or not exactly 1.3× in room 1, a cable length whose rounding sits on an integer. If one fires, move the placement in the generator's tables until it holds; never loosen the check. + +- [ ] **Step 5: Confirm what the pair is made of, and that nothing confidential is in it** + +``` +uv run --frozen python -c "import tempfile, json; from tests.fixtures.plant_pair import build_plant_pair; t = build_plant_pair(tempfile.mkdtemp()); print(json.dumps({k: t[k] for k in ('scale_verdict', 'outlier_handle', 'cluster_count', 'scale')}, default=str)); print([(r['id'], r['status'], r['layout_length_mm']) for r in t['pipe_runs']]); print([(r['tag'], r['length_mm'], r['roundup_m']) for r in t['cable_rows']])" +git grep -n -i -E "client|customer|site name" -- tests/fixtures tests/test_plant_pair.py +``` + +Expected: `scale.overall.within_10pct` 0.2181…, `room_1.within_10pct` 1.0, `room_2.within_10pct` 0.0; runs `PR1 A 5500.0`, `PR2 B 13200.0`, `PR3 A 9000.0`, `PR4 C None`, `CS1 A 20700.0`, `CS2 B 8000.0`, `CR1 A 25700.0`, `CR2 A 10100.0`, `UT1 A 7100.0`; cables `M11 6900.0 9`, `T102 14100.0 17`, `M12 15800.0 19`, `T202 8000.0 10`. The `git grep` prints exactly two lines — the docstrings of `tests/fixtures/__init__.py` and `tests/fixtures/plant_pair.py` saying the pair is never a client drawing (spec §14: every name here is invented; nothing is copied from the user's archive). Any other hit is removed before committing. + +- [ ] **Step 6: Gates** + +``` +uv run --frozen ruff format tests/fixtures tests/test_plant_pair.py +uv run --frozen ruff check tests/fixtures tests/test_plant_pair.py +uv run --frozen pytest -q --deselect tests/test_release_consistency.py::test_document_numbers_match_the_tree +``` + +Expected: ruff clean; the suite green. `test_document_numbers_match_the_tree` is deselected for the collected-tests figure only, and the report says so (Task 11 retakes it on main). + +- [ ] **Step 7: Commit** + +``` +git add tests/fixtures/__init__.py tests/fixtures/plant_pair.py tests/test_plant_pair.py +git -c user.name="Umut CAN" -c user.email="umutcan.edizaslan@gmail.com" commit -m "$(cat <<'MSG' +test(understand): the synthetic plant pair - a schematic P&ID and its layout with known answers + +Every track H reader is measured on this pair, never on a client drawing +(spec section 14). tests/fixtures/plant_pair.py writes pid.dxf and layout.dxf +with invented names and returns their truth, computed from the positions it +placed: + +- the P&ID is room 1 of the layout stretched 1.3x and room 2 rearranged, so + 21.8 % of the 55 tag pairs fall within 10 % of the median ratio (schematic) + while room 1 alone is 100 % at 1.3; +- service layers in English and Dutch, Russian supply / return words, one CIP + return drawn on the supply layer, sizes spelled as drafters do (an MTEXT + format run, %%c, \U+2205, the U+2300 sign, o-stroke, SMS, DN, 20x27), a + reducer, inline valves, an arc bend, a segment drawn twice; +- electrical texts in Russian with the power stated for all loads but one; +- a layout declaring inches over millimetre geometry, a stray line 5,000 km + out, two plan copies 100 m apart, wall outlines, room labels and wiring + callouts in four spellings. + +Run lengths are the rectilinear MST of each run's tags, cable metres are +the Manhattan length rounded up after the 20 % allowance; the generator +refuses to write a pair whose own premises fail. + +Co-Authored-By: Claude Opus 5.5 (1M context) +Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd +MSG +)" +``` + +--- + +### Task 3: drawing_understand — units, extents, clusters, layer meaning, tags, rooms + +**Files:** +- Create: `engineering/understand/units.py` +- Create: `engineering/understand/clusters.py` +- Create: `engineering/understand/describe.py` +- Modify: `server.py` (open `# SECTION 26: Understanding & QA (1 tool)` with `drawing_understand`, between the end of SECTION 25 and the RESOURCES banner) +- Modify: `discovery/aliases.py` (the `drawing_understand` record, appended at the end of `TOOL_ALIASES`) +- Modify: `docs/tool-inventory.json` (regenerated), `README.md` (release-snapshot tool counts 241 → 242), `tests/test_tool_registry.py` (frozen group sizes: `analysis` 12 → 13, total 242) +- Test: `tests/test_understand_units.py`, `tests/test_understand_clusters.py`, `tests/test_understand_describe.py`, `tests/test_understand_tools.py` + +**Interfaces:** +- Consumes: + - `engineering/understand/snapshot.py` (Task 1): `Pt`, `EntityRecord`, `Snapshot` exactly as the skeleton pins them; `read_snapshot(path: str) -> Snapshot`; `async def take_snapshot(backend, path: str | None = None) -> Snapshot`; `length_of(rec: EntityRecord) -> float`. + - `engineering/understand/vocab.py` (Task 1): `classify_layer(name: str) -> dict` (`discipline`, `service`, `confidence`, `keyword`, `language`); `room_label(text: str) -> dict | None` (`number`, `name`, `language`). + - `engineering/understand/labels.py` (Task 1): `parse_tag(text: str) -> str | None` (Cyrillic look-alikes folded). + - `tests/fixtures/plant_pair.py` (Task 2): `build_plant_pair(directory: str | Path) -> dict` — the truth keys `pid`, `layout`, `layout_insunits`, `layout_true_unit`, `outlier_handle`, `cluster_count`, `layer_services`, `pipe_runs[].tags`, `cable_rows[].tag`, `rooms[].{number, name, polygon}`. + - Existing: `engineering.arch.faces.planar_faces(segments, *, tol) -> tuple[Face, ...]`, `face_containing(faces, point) -> Face | None`, `Face`; `engineering.pid.graph.ATTRIBUTE_TAGS`, `KEYWORDS`; `server.validate_path`, `_backend`, `cad_tool`. +- Produces: + - `engineering/understand/units.py`: `MM_PER_UNIT: dict[str, float]`; `entity_box(rec: EntityRecord) -> tuple[float, float, float, float] | None`; `robust_extents(records: Sequence[EntityRecord]) -> dict` (`computed`, `robust`, `outliers: [{handle, type, layer, at, distance, confidence}]` farthest first, `stretch`, `method`); `wall_segments(records, *, is_wall=None) -> tuple[list[tuple[Pt, Pt]], int]`; `infer_units(snap: Snapshot, *, records=None, room_areas=()) -> dict` (`declared: {code, name, label}`, `inferred: str | None`, `mm_per_unit: float | None`, `agree: bool`, `confidence`, `scores`, `evidence`, `warning: str | None`). + - `engineering/understand/clusters.py`: `find_clusters(records, *, exclude=(), cell=None) -> tuple[list[dict], dict[str, str]]` — rows `{id ("C1", "C2", … largest first), box, size, entities, layers, labels}` and handle → cluster id. + - `engineering/understand/describe.py`: `describe(snap: Snapshot) -> dict` with the skeleton's keys `units`, `extents`, `clusters`, `layers`, `equipment_tags`, `rooms`, `languages`, `blocks`, `layouts`, `title_block`, `warnings` (plus `source`, `entities`, `room_faces`); `prepare(snap: Snapshot) -> dict` (`model`, `extents`, `kept`, `clusters`, `membership`, `diag`); `tag_occurrences(snap: Snapshot, *, prep: dict | None = None) -> list[dict]` (`tag`, `handle`, `at`, `space`, `layer`, `cluster`, `source`, `confidence`); `find_rooms(records, *, tol: float, membership: dict) -> dict` (`rooms`, `labelled: [(row, Face)]`, `faces`, `faces_unlabelled`, `wall_segments`, `arcs_skipped`, `skipped`); `text_language(text: str) -> str`; `FACE_TOL_FRACTION`. Task 4 imports `prepare`, `tag_occurrences`, `find_rooms` and `FACE_TOL_FRACTION`. The takeoffs (Task 6, group N) do not import them: group N is cut from `main` beside this branch, so `takeoff.py` carries its own narrow `unit_of` / `room_regions` (Task 6, "Why this module carries stand-ins"). + - MCP tool `drawing_understand(path: str | None = None) -> dict` (core pack, read-only, tags `analysis` + `query`). + +- [ ] **Step 1: Write the failing units test** + +Create `tests/test_understand_units.py`: + +```python +"""Declared versus inferred units, and robust extents, on hand-built snapshots. + +Every expected number is worked out in the comment beside it from the +coordinates written here; nothing is read back from the implementation. +""" + +from __future__ import annotations + +import math + +import pytest + +from engineering.understand.snapshot import EntityRecord, Snapshot +from engineering.understand.units import entity_box, infer_units, robust_extents + +EPS = 1e-9 + + +def line(handle, a, b, layer="WALL"): + return EntityRecord(handle=handle, type="LINE", layer=layer, space="Model", points=(a, b)) + + +def text(handle, value, x, y, height, layer="TEXT"): + return EntityRecord( + handle=handle, + type="TEXT", + layer=layer, + space="Model", + points=((x, y),), + text=value, + height=height, + ) + + +def rect(prefix, x0, y0, x1, y1, layer="WALL"): + return [ + line(f"{prefix}1", (x0, y0), (x1, y0), layer), + line(f"{prefix}2", (x1, y0), (x1, y1), layer), + line(f"{prefix}3", (x1, y1), (x0, y1), layer), + line(f"{prefix}4", (x0, y1), (x0, y0), layer), + ] + + +def snapshot(records, insunits): + return Snapshot( + source="test", + insunits=insunits, + extmin=None, + extmax=None, + layers={}, + layouts=("Model",), + records=tuple(records), + ) + + +def room_in_mm(): + """One 6000 x 4000 room between 200 mm double-line walls, labelled with + 250-unit text: outer outline (0,0)-(6400,4400), inner (200,200)-(6200,4200).""" + return [ + *rect("O", 0.0, 0.0, 6400.0, 4400.0), + *rect("I", 200.0, 200.0, 6200.0, 4200.0), + text("T1", "ROOM 101 PROCESS", 1000.0, 2000.0, 250.0), + text("T2", "T4100", 3000.0, 1000.0, 250.0), + ] + + +def test_inches_declared_over_millimetre_geometry_is_flagged(): + # Inner room face 6000 x 4000 = 2.4e7 unit^2: 24 m2 as mm (plausible), + # 15 484 m2 as inches (out of the 1-5000 m2 band). Wall pairs 200 apart: + # 0.2 m as mm, 5.08 m as inches. Text 250: 0.25 m as mm, 6.35 m as inches. + result = infer_units(snapshot(room_in_mm(), insunits=1), room_areas=[6000.0 * 4000.0]) + assert result["declared"] == {"code": 1, "name": "in", "label": "Inches"} + assert result["inferred"] == "mm" + assert result["mm_per_unit"] == 1.0 + assert result["agree"] is False + assert result["warning"].startswith("INSUNITS declares Inches (1) but the geometry reads as mm") + kinds = {e["kind"]: e for e in result["evidence"]} + assert kinds["wall_thickness"]["value"] == pytest.approx(200.0, abs=EPS) + assert kinds["wall_thickness"]["plausible"] == ["mm"] + assert kinds["room_area"]["as"]["mm"] == pytest.approx(24.0, abs=EPS) + assert kinds["room_area"]["as"]["in"] == pytest.approx(24_000_000.0 * 0.0254**2, abs=EPS) + assert "in" not in kinds["text_height"]["plausible"] # 6.35 m of lettering + + +def test_millimetres_declared_over_the_same_geometry_agree(): + result = infer_units(snapshot(room_in_mm(), insunits=4), room_areas=[24_000_000.0]) + assert result["inferred"] == "mm" + assert result["agree"] is True + assert result["warning"] is None + assert 0.0 < result["confidence"] <= 0.9 + + +def test_one_weak_item_never_overrules_the_header(): + # Only an overall size: 20 000 units is 20 m as mm, 200 m as cm, 508 m as + # inches (all in the 0.01-2000 m band) and 20 km as metres (out of it). + # mm, cm and in score the size's 0.25 each; "m" scores the header's 0.5 + # alone and still wins - the declared metre stands. + records = [ + line("A", (0.0, 0.0), (20000.0, 0.0), "0"), + line("B", (0.0, 0.0), (0.0, 5000.0), "0"), + ] + result = infer_units(snapshot(records, insunits=6)) + assert result["inferred"] == "m" + assert result["agree"] is True + + +def test_no_evidence_keeps_the_declared_unit_at_low_confidence(): + result = infer_units(snapshot([], insunits=4)) + assert result["inferred"] == "mm" + assert result["confidence"] == 0.4 + assert result["evidence"] == [] + + +def test_unitless_with_no_evidence_settles_nothing(): + result = infer_units(snapshot([], insunits=0)) + assert result["inferred"] is None + assert result["mm_per_unit"] is None + assert result["warning"].endswith("lengths stay in drawing units.") + + +def test_a_far_outlier_is_found_by_handle_and_left_out_of_the_robust_box(): + records = [ + line(f"L{i}", (i * 1000.0, 0.0), (i * 1000.0 + 500.0, 800.0), "0") for i in range(12) + ] + far = line("FAR", (-5_000_000_000.0, 0.0), (-5_000_000_000.0 + 10.0, 0.0), "0") + result = robust_extents([*records, far]) + assert [o["handle"] for o in result["outliers"]] == ["FAR"] + # Robust box: x from 0 to 11 * 1000 + 500 = 11500, y from 0 to 800. + assert result["robust"] == pytest.approx([0.0, 0.0, 11500.0, 800.0], abs=EPS) + assert result["computed"][0] == pytest.approx(-5_000_000_000.0, abs=EPS) + # The outlier's centre (-4 999 999 995, 0) is 4 999 999 995 from the box. + assert result["outliers"][0]["distance"] == pytest.approx(4_999_999_995.0, abs=1e-3) + # Stretch: hypot(5 000 011 500, 800) / hypot(11 500, 800) - the computed + # diagonal over the robust one. + expected = math.hypot(5_000_011_500.0, 800.0) / math.hypot(11500.0, 800.0) + assert result["stretch"] == pytest.approx(expected, rel=1e-12) + + +def test_the_edge_walls_of_a_plan_are_not_outliers(): + # The room's centres crowd its middle (the vertical walls centre on + # y 2200, the label on y 2000), so a per-axis fence on centres would cut + # the top and bottom walls off; grown from the middle, every wall joins. + result = robust_extents(room_in_mm()) + assert result["outliers"] == [] + assert result["robust"] == pytest.approx([0.0, 0.0, 6400.0, 4400.0], abs=EPS) + assert result["stretch"] == pytest.approx(1.0, abs=EPS) + + +def test_fewer_than_four_entities_have_no_outliers(): + records = [line("A", (0.0, 0.0), (1.0, 0.0), "0"), line("B", (1e9, 0.0), (1e9 + 1.0, 0.0), "0")] + assert robust_extents(records)["outliers"] == [] + + +def test_entity_box_grows_a_circle_by_its_radius(): + circle = EntityRecord( + handle="C", type="CIRCLE", layer="0", space="Model", points=((10.0, 20.0),), radius=5.0 + ) + assert entity_box(circle) == (5.0, 15.0, 15.0, 25.0) + boxed = EntityRecord(handle="B", type="INSERT", layer="0", space="Model", bbox=(1, 2, 3, 4)) + assert entity_box(boxed) == (1.0, 2.0, 3.0, 4.0) + assert entity_box(EntityRecord(handle="N", type="HATCH", layer="0", space="Model")) is None +``` + +- [ ] **Step 2: Run it to verify it fails** + +Run: `uv run --frozen pytest tests/test_understand_units.py -q` +Expected: FAIL — collection error `ModuleNotFoundError: No module named 'engineering.understand.units'`. + +- [ ] **Step 3: Write `engineering/understand/units.py`** + +The bands are named in the module docstring with their basis: text height is the only one derived from standards (ISO 3098-0's lettering heights 1.8-20 mm at ISO 5455's scales, with 1:200 as this reader's ceiling); the room-area, wall, door and overall-size bands are this reader's heuristics and say so. The inch and foot factors are exact by definition (1959 international yard and pound agreement); ezdxf's `METER_FACTOR` is rounded (`39.37007874` in/m, measured) and is not used. + +```python +"""Declared versus inferred drawing units, and extents that one stray entity cannot stretch. + +`$INSUNITS` is a claim the drawing makes about itself, and a foreign drawing +can make it wrongly: a layout that declares inches over millimetre geometry +reads 25.4 times too large if the header is believed. This module never corrects +the claim silently. It measures what the geometry implies and reports both, +with the numbers that decided it. + +**Evidence.** Each item is one median measured in drawing units and the band a +real drawing puts it in, in metres. A candidate unit is *plausible* for an item +when the measurement, converted by that unit, lands in the band. An item that +every candidate or no candidate satisfies says nothing and is dropped as +inconclusive. + +======================== ====================== ====== ==================================== +item band weight basis +======================== ====================== ====== ==================================== +``room_area`` 1 m² … 5 000 m² 1.0 a labelled room's face (this reader's + heuristic) +``wall_thickness`` 0.05 m … 1.0 m 1.0 nearest parallel wall line (heuristic) +``door_leaf`` 0.5 m … 1.5 m 1.0 radius of a quarter-circle arc, a door + swing (heuristic) +``text_height`` 1.8 mm … 4.0 m 0.5 ISO 3098-0 lettering heights 1.8-20 mm + at ISO 5455 scales 1:1 to 1:200 (the + 1:200 ceiling is this reader's choice + for plant drawings) +``overall_size`` 0.01 m … 2 000 m 0.25 the larger side of the robust extents + (heuristic: a plant or a building) +======================== ====================== ====== ==================================== + +**Decision.** A candidate scores the weights of the items it is plausible for; +the header is evidence too and adds :data:`HEADER_WEIGHT` to the declared unit. +The highest score wins and a tie goes to the declared unit, then to the order of +:data:`PREFERENCE`. So the header is overruled only when the geometry +outweighs it - one weak item (the overall size) never does. + +**Extents.** The body of the drawing is grown from its middle: the seed is +the quartile box of the entity centres (Q1..Q3 on each axis), and every +entity whose box lies within :data:`OUTLIER_REACH` body diagonals of the body +joins it, pass after pass, until a pass adds nothing. What never joins is an +outlier - an entity farther from everything else than the drawing is large. +The robust extents are the body's box. This is this reader's heuristic, not +a statistical standard: a per-axis fence on centres (Tukey's) flags the edge +walls of a plan whose centres crowd its middle, and a quantile range is +dragged by the outlier itself in a small drawing. Fewer than four entities +have no outliers. + +Pure: records in, dicts out. The drawing is never touched. +""" + +from __future__ import annotations + +import math +import statistics +from collections.abc import Iterable, Sequence + +from ezdxf import units as ezdxf_units + +from engineering.understand.snapshot import EntityRecord, Snapshot +from engineering.understand.vocab import classify_layer + +__all__ = [ + "BANDS", + "CANDIDATES", + "HEADER_WEIGHT", + "MM_PER_UNIT", + "PREFERENCE", + "entity_box", + "infer_units", + "robust_extents", + "wall_segments", +] + +#: Millimetres per drawing unit, exact by definition: the SI prefixes, and the +#: inch (25.4 mm) and foot (304.8 mm) of the 1959 international yard and pound +#: agreement. ezdxf's own ``METER_FACTOR`` is rounded (39.37007874 in/m), so it +#: is not used for conversion. +MM_PER_UNIT: dict[str, float] = {"mm": 1.0, "cm": 10.0, "m": 1000.0, "in": 25.4, "ft": 304.8} +CANDIDATES: tuple[str, ...] = ("mm", "cm", "m", "in", "ft") +#: Tie-break after the declared unit. +PREFERENCE: tuple[str, ...] = ("mm", "m", "cm", "in", "ft") +#: What the header's own claim weighs against the geometry. +HEADER_WEIGHT = 0.5 +#: (low, high, weight, power): the band in metres (m² for power 2). +BANDS: dict[str, tuple[float, float, float, int]] = { + "room_area": (1.0, 5000.0, 1.0, 2), + "wall_thickness": (0.05, 1.0, 1.0, 1), + "door_leaf": (0.5, 1.5, 1.0, 1), + "text_height": (0.0018, 4.0, 0.5, 1), + "overall_size": (0.01, 2000.0, 0.25, 1), +} +#: How far (in body diagonals) an entity may lie from the body and still join it. +OUTLIER_REACH = 1.0 +#: A door swing is a quarter circle; the sweep may be drawn a little off. +DOOR_SWEEP = (85.0, 95.0) +#: Wall-thickness pairing reads at most this many wall segments. +WALL_SEGMENT_CAP = 3000 + +_TEXT_TYPES = frozenset({"TEXT", "MTEXT"}) +_POLY_TYPES = frozenset({"LWPOLYLINE", "POLYLINE"}) + + +def entity_box(rec: EntityRecord) -> tuple[float, float, float, float] | None: + """The record's WCS box: its own ``bbox``, else its points (a circle or arc + grown by its radius about its centre), else None.""" + if rec.bbox is not None: + return tuple(float(v) for v in rec.bbox) + if not rec.points: + return None + xs = [float(p[0]) for p in rec.points] + ys = [float(p[1]) for p in rec.points] + grow = float(rec.radius) if rec.type in ("CIRCLE", "ARC") and rec.radius else 0.0 + return (min(xs) - grow, min(ys) - grow, max(xs) + grow, max(ys) + grow) + + +def _union(boxes: Iterable[tuple[float, float, float, float]]): + boxes = list(boxes) + if not boxes: + return None + return ( + min(b[0] for b in boxes), + min(b[1] for b in boxes), + max(b[2] for b in boxes), + max(b[3] for b in boxes), + ) + + +def _diag(box) -> float: + return math.hypot(box[2] - box[0], box[3] - box[1]) if box else 0.0 + + +def _box_gap(a, b) -> float: + """Distance between two boxes; 0 when they touch or overlap.""" + dx = max(a[0] - b[2], b[0] - a[2], 0.0) + dy = max(a[1] - b[3], b[1] - a[3], 0.0) + return math.hypot(dx, dy) + + +def _grow_body(boxes: Sequence[tuple[float, float, float, float]]) -> set[int]: + """Indices of the boxes that never join the body grown from the centre.""" + if len(boxes) < 4: + return set() + xs = [(b[0] + b[2]) / 2.0 for b in boxes] + ys = [(b[1] + b[3]) / 2.0 for b in boxes] + qx = statistics.quantiles(xs, n=4, method="inclusive") + qy = statistics.quantiles(ys, n=4, method="inclusive") + body = (qx[0], qy[0], qx[2], qy[2]) + floor = statistics.median(_diag(b) for b in boxes) + pending = set(range(len(boxes))) + while pending: + reach = OUTLIER_REACH * max(_diag(body), floor) + joined = [i for i in pending if _box_gap(boxes[i], body) <= reach] + if not joined: + break + for i in joined: + body = _union([body, boxes[i]]) + pending.difference_update(joined) + return pending + + +def _distance_to_box(x: float, y: float, box) -> float: + dx = max(box[0] - x, 0.0, x - box[2]) + dy = max(box[1] - y, 0.0, y - box[3]) + return math.hypot(dx, dy) + + +def robust_extents(records: Sequence[EntityRecord]) -> dict: + """Computed and robust extents of ``records`` and the outliers between them. + + ``outliers`` are sorted farthest first, each with its handle, type, layer, + centre and its distance from the robust box. ``stretch`` is the computed + diagonal over the robust one (None when the robust box is a point). + """ + items = [(rec, box) for rec in records if (box := entity_box(rec)) is not None] + centres = [((b[0] + b[2]) / 2.0, (b[1] + b[3]) / 2.0) for _rec, b in items] + skip = _grow_body([b for _rec, b in items]) + outside = sorted(skip) + computed = _union(b for _rec, b in items) + robust = _union(b for i, (_rec, b) in enumerate(items) if i not in skip) + outliers = [] + for i in outside: + rec, _box = items[i] + cx, cy = centres[i] + outliers.append( + { + "handle": rec.handle, + "type": rec.type, + "layer": rec.layer, + "at": [cx, cy], + "distance": _distance_to_box(cx, cy, robust) if robust else None, + "confidence": 0.9, + } + ) + outliers.sort(key=lambda o: (-(o["distance"] or 0.0), o["handle"])) + robust_diag = _diag(robust) + return { + "computed": list(computed) if computed else None, + "robust": list(robust) if robust else None, + "outliers": outliers, + "stretch": (_diag(computed) / robust_diag) if robust_diag > 0.0 else None, + "method": ( + "body grown from the quartile box of entity centres; an entity farther from it " + f"than {OUTLIER_REACH:g} body diagonal(s) never joins and is an outlier" + ), + } + + +def wall_segments( + records: Sequence[EntityRecord], *, is_wall=None +) -> tuple[list[tuple[tuple[float, float], tuple[float, float]]], int]: + """Straight segments of LINE and polyline records on wall layers, plus the + number of bulged (arc) polyline segments left out. + + ``is_wall(layer)`` defaults to `vocab.classify_layer` naming the + ``architecture`` discipline. + """ + cache: dict[str, bool] = {} + + def wall(layer: str) -> bool: + if layer not in cache: + if is_wall is not None: + cache[layer] = bool(is_wall(layer)) + else: + cache[layer] = classify_layer(layer).get("discipline") == "architecture" + return cache[layer] + + segments: list[tuple[tuple[float, float], tuple[float, float]]] = [] + bulged = 0 + for rec in records: + if not wall(rec.layer): + continue + if rec.type == "LINE" and len(rec.points) >= 2: + segments.append((tuple(rec.points[0]), tuple(rec.points[1]))) + elif rec.type in _POLY_TYPES and len(rec.points) >= 2: + pts = list(rec.points) + count = len(pts) if rec.closed else len(pts) - 1 + for i in range(count): + a, b = pts[i], pts[(i + 1) % len(pts)] + bulge = rec.bulges[i] if i < len(rec.bulges) else 0.0 + if abs(bulge) > 1e-12: + bulged += 1 + continue + if a != b: + segments.append((tuple(a), tuple(b))) + return segments, bulged + + +def _wall_thickness(segments) -> tuple[float | None, int]: + """Median distance from an axis-parallel wall segment to its nearest parallel + neighbour on either side that overlaps it; (None, 0) without a pair. + + Both sides, because the inner face of a double-line wall has the room on + one side and its own outer face on the other: looking one way only reads + half the faces as room widths and the median lands on neither.""" + horizontal: list[tuple[float, float, float]] = [] # (y, x0, x1) + vertical: list[tuple[float, float, float]] = [] # (x, y0, y1) + for (ax, ay), (bx, by) in segments[:WALL_SEGMENT_CAP]: + length = math.hypot(bx - ax, by - ay) + if length <= 0.0: + continue + if abs(by - ay) <= 1e-9 * length: + horizontal.append((ay, min(ax, bx), max(ax, bx))) + elif abs(bx - ax) <= 1e-9 * length: + vertical.append((ax, min(ay, by), max(ay, by))) + gaps: list[float] = [] + for group in (horizontal, vertical): + group.sort() + for i, (c, lo, hi) in enumerate(group): + best = None + for step in (1, -1): + j = i + step + while 0 <= j < len(group): + c2, lo2, hi2 = group[j] + d = abs(c2 - c) + if best is not None and d >= best: + break + if d > 0.0 and lo2 < hi and lo < hi2: + best = d + j += step + if best is not None: + gaps.append(best) + if not gaps: + return None, 0 + return statistics.median(gaps), len(gaps) + + +def _declared(code: int) -> dict: + try: + name = ezdxf_units.decode(int(code)) + label = ezdxf_units.unit_name(int(code)) + except (IndexError, ValueError, TypeError): + name, label = None, None + return {"code": int(code), "name": name, "label": label or "Unitless"} + + +def infer_units( + snap: Snapshot, + *, + records: Sequence[EntityRecord] | None = None, + room_areas: Sequence[float] = (), +) -> dict: + """The unit the geometry implies, against the one ``$INSUNITS`` declares. + + ``records`` defaults to the snapshot's model-space records; pass the ones + left after the outliers are removed. ``room_areas`` are labelled room faces + in square drawing units. Every item of ``evidence`` carries its median, + its sample count, what it is in metres under each candidate, the + candidates it is plausible for and whether it was informative. + """ + recs = [r for r in snap.records if r.space == "Model"] if records is None else list(records) + measured: dict[str, tuple[float, int]] = {} + + heights = [float(r.height) for r in recs if r.type in _TEXT_TYPES and r.height and r.height > 0] + if heights: + measured["text_height"] = (statistics.median(heights), len(heights)) + radii = [] + for r in recs: + if r.type == "ARC" and r.radius and r.angles: + sweep = (float(r.angles[1]) - float(r.angles[0])) % 360.0 + if DOOR_SWEEP[0] <= sweep <= DOOR_SWEEP[1]: + radii.append(float(r.radius)) + if radii: + measured["door_leaf"] = (statistics.median(radii), len(radii)) + thickness, pairs = _wall_thickness(wall_segments(recs)[0]) + if thickness is not None: + measured["wall_thickness"] = (thickness, pairs) + areas = [float(a) for a in room_areas if a and a > 0] + if areas: + measured["room_area"] = (statistics.median(areas), len(areas)) + box = robust_extents(recs)["robust"] + if box: + side = max(box[2] - box[0], box[3] - box[1]) + if side > 0: + measured["overall_size"] = (side, len(recs)) + + evidence = [] + for kind, (value, count) in measured.items(): + low, high, weight, power = BANDS[kind] + physical = {u: value * (MM_PER_UNIT[u] / 1000.0) ** power for u in CANDIDATES} + plausible = [u for u in CANDIDATES if low <= physical[u] <= high] + evidence.append( + { + "kind": kind, + "value": value, + "count": count, + "band": [low, high], + "band_unit": "m2" if power == 2 else "m", + "weight": weight, + "as": physical, + "plausible": plausible, + "informative": 0 < len(plausible) < len(CANDIDATES), + } + ) + + declared = _declared(snap.insunits) + name = declared["name"] + informative = [e for e in evidence if e["informative"]] + scores = {u: sum(e["weight"] for e in informative if u in e["plausible"]) for u in CANDIDATES} + if name in scores: + scores[name] += HEADER_WEIGHT + if informative: + best = max(scores.values()) + order = ([name] if name in scores else []) + [u for u in PREFERENCE if u != name] + inferred = next(u for u in order if scores[u] == best) + total = sum(e["weight"] for e in informative) + (HEADER_WEIGHT if name in scores else 0.0) + confidence = round(0.9 * scores[inferred] / total, 2) + elif name in MM_PER_UNIT: + inferred, confidence = name, 0.4 + else: + inferred, confidence = None, 0.0 + + warning = None + if inferred is not None and inferred != name: + support = [e for e in informative if inferred in e["plausible"]] + detail = "; ".join( + f"{e['kind']} {e['value']:g} = {e['as'][inferred]:.4g} {e['band_unit']} as {inferred}" + + (f", {e['as'][name]:.4g} {e['band_unit']} as {name}" if name in MM_PER_UNIT else "") + for e in support + ) + warning = ( + f"INSUNITS declares {declared['label']} ({declared['code']}) but the geometry reads " + f"as {inferred}: {detail}. Lengths are measured in {inferred}; the header is not " + "changed." + ) + elif inferred is None: + warning = ( + f"INSUNITS declares {declared['label']} ({declared['code']}) and no measurement " + "settles the unit; lengths stay in drawing units." + ) + return { + "declared": declared, + "inferred": inferred, + "mm_per_unit": MM_PER_UNIT.get(inferred) if inferred else None, + "agree": inferred is not None and inferred == name, + "confidence": confidence, + "scores": scores, + "evidence": evidence, + "warning": warning, + } +``` + +- [ ] **Step 4: Run the units test to verify it passes** + +Run: `uv run --frozen pytest tests/test_understand_units.py -q` +Expected: `9 passed`. + +- [ ] **Step 5: Write the failing clusters test** + +Create `tests/test_understand_clusters.py`: + +```python +"""Bodies of model-space content: two plan copies are two clusters.""" + +from __future__ import annotations + +import pytest + +from engineering.understand.clusters import find_clusters +from engineering.understand.snapshot import EntityRecord + +EPS = 1e-9 + + +def line(handle, a, b, layer="WALL"): + return EntityRecord(handle=handle, type="LINE", layer=layer, space="Model", points=(a, b)) + + +def text(handle, value, x, y, height=250.0): + return EntityRecord( + handle=handle, + type="TEXT", + layer="TEXT", + space="Model", + points=((x, y),), + text=value, + height=height, + ) + + +def plan(prefix, dx): + """A 20 000 x 12 000 outline with a partition, a tag in each half and a + title under it, shifted ``dx`` along x. The tags float 4 000 from every + line - far more than one cell - so only the absorb step keeps them in.""" + return [ + line(f"{prefix}1", (dx, 0.0), (dx + 20000.0, 0.0)), + line(f"{prefix}2", (dx + 20000.0, 0.0), (dx + 20000.0, 12000.0)), + line(f"{prefix}3", (dx + 20000.0, 12000.0), (dx, 12000.0)), + line(f"{prefix}4", (dx, 12000.0), (dx, 0.0)), + line(f"{prefix}5", (dx + 10000.0, 0.0), (dx + 10000.0, 12000.0)), + text(f"{prefix}T1", "T4100", dx + 4000.0, 6000.0), + text(f"{prefix}T2", "T4200", dx + 14000.0, 6000.0), + text(f"{prefix}T3", "PLANT LAYOUT", dx + 2000.0, 12500.0, height=500.0), + ] + + +def test_two_plan_copies_side_by_side_are_two_clusters(): + # Copy A spans x 0..20 000, copy B 40 000..60 000: a 20 000 gap. The + # content box is 60 000 x 12 750 (the title's insertion is at y 12 500... + # its point box ends there), so a cell is about 1 % of 61 340 = 613 units + # and the gap is over 30 empty cells. + clusters, membership = find_clusters([*plan("A", 0.0), *plan("B", 40000.0)]) + assert len(clusters) == 2 + left, right = sorted(clusters, key=lambda c: c["box"][0]) + assert left["entities"] == right["entities"] == 8 + assert left["box"] == pytest.approx([0.0, 0.0, 20000.0, 12500.0], abs=EPS) + assert right["box"] == pytest.approx([40000.0, 0.0, 60000.0, 12500.0], abs=EPS) + assert membership["AT1"] == membership["A1"] == left["id"] + assert membership["BT2"] == membership["B5"] == right["id"] + # The tallest text leads the labels. + assert left["labels"][0] == "PLANT LAYOUT" + assert left["layers"][0] == "WALL" + + +def test_ids_run_largest_first_then_left_to_right(): + small = [line("S1", (100000.0, 0.0), (100500.0, 0.0))] + clusters, _ = find_clusters([*small, *plan("A", 40000.0), *plan("B", 0.0)]) + assert [c["id"] for c in clusters] == ["C1", "C2", "C3"] + assert [c["box"][0] for c in clusters] == pytest.approx([0.0, 40000.0, 100000.0], abs=EPS) + assert [c["entities"] for c in clusters] == [8, 8, 1] + + +def test_excluded_handles_take_no_part(): + far = line("FAR", (-5_000_000_000.0, 0.0), (-4_999_999_000.0, 0.0)) + clusters, membership = find_clusters([*plan("A", 0.0), far], exclude={"FAR"}) + assert len(clusters) == 1 + assert "FAR" not in membership + assert clusters[0]["box"] == pytest.approx([0.0, 0.0, 20000.0, 12500.0], abs=EPS) + + +def test_nothing_to_cluster_is_an_empty_answer(): + assert find_clusters([]) == ([], {}) + + +def test_a_non_positive_cell_is_refused(): + with pytest.raises(ValueError, match="cell: must be greater than zero"): + find_clusters(plan("A", 0.0), cell=-1.0) +``` + +- [ ] **Step 6: Run it to verify it fails** + +Run: `uv run --frozen pytest tests/test_understand_clusters.py -q` +Expected: FAIL — collection error `ModuleNotFoundError: No module named 'engineering.understand.clusters'`. + +- [ ] **Step 7: Write `engineering/understand/clusters.py`** + +The cell is 2 % of the content's diagonal. Measured while writing this plan: at 1 % a title lettered 500 mm above a 20 m plan fell two cells away and became a cluster of its own; at 2 % it stays with its plan, and two copies of a plan still separate whenever the gap between them exceeds 4 % of the whole diagonal. + +```python +"""Separate bodies of model-space content: sheets, plan copies, details. + +A foreign model space is often several drawings side by side - two copies of +one plan, a detail beside it, a legend. Every reader after this one must know +which body a tag or a room belongs to, or it matches a tag in one copy against +the same tag in the other. + +The algorithm, in the order it runs: + +1. **Raster.** The box of the records is divided into square cells of + ``cell`` drawing units (default :data:`CELL_FRACTION` of its diagonal, so + the grid never exceeds about 50 x 50 cells). Every record marks the cells + its box covers. +2. **Connect.** Occupied cells that touch, corners included, are one + component; a record belongs to the component of its cells. Two bodies stay + apart when an empty cell separates them - a gap of two cells is always + enough. +3. **Absorb.** Components whose boxes intersect are merged, repeatedly: a tag + floating in the middle of a room lies inside the plan's box and belongs to + the plan, not to a cluster of its own. + +Ids are ``C1``, ``C2``, ... by entity count, largest first (ties left to right, +then bottom to top). Outliers are the caller's to exclude: one entity thousands of +kilometres away would otherwise set the cell size. Pure; the drawing is never touched. +""" + +from __future__ import annotations + +import math +from collections.abc import Iterable, Sequence + +from engineering.understand.snapshot import EntityRecord +from engineering.understand.units import entity_box + +__all__ = ["CELL_FRACTION", "find_clusters"] + +#: Cell side as a fraction of the content's diagonal. +CELL_FRACTION = 0.02 +#: Labels listed per cluster. +LABELS_PER_CLUSTER = 5 + +_TEXT_TYPES = frozenset({"TEXT", "MTEXT", "MULTILEADER"}) + + +def _intersect(a, b) -> bool: + return a[0] <= b[2] and b[0] <= a[2] and a[1] <= b[3] and b[1] <= a[3] + + +def _merge(a, b): + return (min(a[0], b[0]), min(a[1], b[1]), max(a[2], b[2]), max(a[3], b[3])) + + +def _root(parent: list[int], i: int) -> int: + """Union-find root with path halving.""" + while parent[i] != i: + parent[i] = parent[parent[i]] + i = parent[i] + return i + + +def _absorb(bodies: list[tuple[tuple, list[int]]]) -> list[tuple[tuple, list[int]]]: + """Merge bodies whose boxes intersect until none do. + + One pass is a sweep over boxes sorted by their left edge (a pair can only + intersect while the next left edge is inside the current box) feeding a + union-find; a merged box can reach a body it did not touch before, so + passes repeat until one merges nothing. + """ + while True: + order = sorted(range(len(bodies)), key=lambda i: bodies[i][0][0]) + parent = list(range(len(bodies))) + joined = False + for n, i in enumerate(order): + box = bodies[i][0] + for j in order[n + 1 :]: + if bodies[j][0][0] > box[2]: + break + ri, rj = _root(parent, i), _root(parent, j) + if ri != rj and _intersect(box, bodies[j][0]): + parent[rj] = ri + joined = True + if not joined: + return bodies + merged: dict[int, tuple[tuple, list[int]]] = {} + for i, (box, members) in enumerate(bodies): + r = _root(parent, i) + if r in merged: + merged[r] = (_merge(merged[r][0], box), merged[r][1] + members) + else: + merged[r] = (box, list(members)) + bodies = list(merged.values()) + + +def find_clusters( + records: Sequence[EntityRecord], + *, + exclude: Iterable[str] = (), + cell: float | None = None, +) -> tuple[list[dict], dict[str, str]]: + """``(clusters, membership)``: one row per body of content, and handle -> id. + + A row is ``{"id", "box": [x0, y0, x1, y1], "size": [w, h], "entities", + "layers": [the three commonest], "labels": [the tallest texts]}``. + ``exclude`` names handles to leave out (the outliers). A record without a + box (no geometry the snapshot could read) belongs to no cluster. + """ + skip = set(exclude) + items = [(rec, box) for rec in records if rec.handle not in skip and (box := entity_box(rec))] + if not items: + return [], {} + x0 = min(b[0] for _r, b in items) + y0 = min(b[1] for _r, b in items) + x1 = max(b[2] for _r, b in items) + y1 = max(b[3] for _r, b in items) + if cell is None: + diag = math.hypot(x1 - x0, y1 - y0) + cell = diag * CELL_FRACTION if diag > 0.0 else 1.0 + if not cell > 0.0: + raise ValueError(f"cell: must be greater than zero, got {cell!r}") + + def span(lo: float, hi: float, origin: float) -> range: + return range( + int(math.floor((lo - origin) / cell)), int(math.floor((hi - origin) / cell)) + 1 + ) + + owner: dict[tuple[int, int], list[int]] = {} + first_cell: list[tuple[int, int]] = [] + for k, (_rec, b) in enumerate(items): + cols, rows = span(b[0], b[2], x0), span(b[1], b[3], y0) + first_cell.append((cols[0], rows[0])) + for i in cols: + for j in rows: + owner.setdefault((i, j), []).append(k) + + component: dict[tuple[int, int], int] = {} + count = 0 + for start in owner: + if start in component: + continue + component[start] = count + stack = [start] + while stack: + ci, cj = stack.pop() + for di in (-1, 0, 1): + for dj in (-1, 0, 1): + nxt = (ci + di, cj + dj) + if nxt in owner and nxt not in component: + component[nxt] = count + stack.append(nxt) + count += 1 + + groups: dict[int, list[int]] = {} + for k in range(len(items)): + groups.setdefault(component[first_cell[k]], []).append(k) + bodies = [] + for members in groups.values(): + box = items[members[0]][1] + for k in members[1:]: + box = _merge(box, items[k][1]) + bodies.append((box, members)) + bodies = _absorb(bodies) + + bodies.sort(key=lambda body: (-len(body[1]), body[0][0], body[0][1])) + clusters: list[dict] = [] + membership: dict[str, str] = {} + for n, (box, members) in enumerate(bodies, start=1): + cid = f"C{n}" + layers: dict[str, int] = {} + texts = [] + for k in members: + rec = items[k][0] + membership[rec.handle] = cid + layers[rec.layer] = layers.get(rec.layer, 0) + 1 + if rec.type in _TEXT_TYPES and rec.text and rec.text.strip(): + texts.append((-(rec.height or 0.0), rec.text.strip())) + labels: list[str] = [] + for _h, text in sorted(texts): + if text not in labels: + labels.append(text) + if len(labels) == LABELS_PER_CLUSTER: + break + clusters.append( + { + "id": cid, + "box": list(box), + "size": [box[2] - box[0], box[3] - box[1]], + "entities": len(members), + "layers": [ + name for name, _c in sorted(layers.items(), key=lambda kv: (-kv[1], kv[0]))[:3] + ], + "labels": labels, + } + ) + return clusters, membership +``` + +- [ ] **Step 8: Run the clusters test to verify it passes** + +Run: `uv run --frozen pytest tests/test_understand_clusters.py -q` +Expected: `5 passed`. + +- [ ] **Step 9: Write the failing describe test** + +Create `tests/test_understand_describe.py`. The first seven tests read the Task 2 plant pair and assert only its truth dict; the rest are hand-built snapshots whose vocabulary expectations are read from `vocab` itself (the test asserts the precondition, e.g. that `classify_layer("WALL")` is `architecture`, rather than restating Task 1's tables). + +```python +"""drawing_understand's report: the synthetic plant pair, then hand-built snapshots. + +The plant-pair facts come from the generator's truth dict (Task 2), never from +a real drawing; the hand-built cases compute their expectations in comments. +""" + +from __future__ import annotations + +import copy + +import pytest + +from engineering.understand.describe import describe, tag_occurrences, text_language +from engineering.understand.labels import parse_tag +from engineering.understand.snapshot import EntityRecord, Snapshot, read_snapshot +from engineering.understand.vocab import classify_layer, room_label +from tests.fixtures.plant_pair import build_plant_pair + +EPS = 1e-9 + + +def shoelace(polygon): + total = 0.0 + for (x1, y1), (x2, y2) in zip(polygon, polygon[1:] + polygon[:1], strict=True): + total += x1 * y2 - x2 * y1 + return abs(total) / 2.0 + + +@pytest.fixture(scope="module") +def plant(tmp_path_factory): + truth = build_plant_pair(tmp_path_factory.mktemp("plant")) + return { + "truth": truth, + "pid": describe(read_snapshot(truth["pid"])), + "layout": describe(read_snapshot(truth["layout"])), + } + + +# ── the synthetic plant pair ──────────────────────────────────────────────── + + +def test_the_layout_declares_inches_over_millimetre_geometry(plant): + units = plant["layout"]["units"] + assert units["declared"]["code"] == plant["truth"]["layout_insunits"] + assert units["inferred"] == plant["truth"]["layout_true_unit"] + assert units["agree"] is False + assert any(w.startswith("INSUNITS declares Inches (1)") for w in plant["layout"]["warnings"]) + + +def test_the_far_outlier_is_named_by_handle(plant): + outliers = plant["layout"]["extents"]["outliers"] + assert outliers[0]["handle"] == plant["truth"]["outlier_handle"] + assert any(plant["truth"]["outlier_handle"] in w for w in plant["layout"]["warnings"]) + + +def test_the_two_plan_copies_are_two_clusters(plant): + assert len(plant["layout"]["clusters"]) == plant["truth"]["cluster_count"] + + +def test_service_layers_are_classified(plant): + rows = {r["name"]: r for r in plant["layout"]["layers"]} + rows.update({r["name"]: r for r in plant["pid"]["layers"]}) + for layer, service in plant["truth"]["layer_services"].items(): + assert rows[layer]["service"] == service, layer + assert rows[layer]["confidence"] > 0.0, layer + + +def test_every_equipment_tag_of_the_truth_is_found_on_the_pid(plant): + found = {t["tag"] for t in plant["pid"]["equipment_tags"]} + expected = {tag for run in plant["truth"]["pipe_runs"] for tag in run["tags"]} + expected |= {row["tag"] for row in plant["truth"]["cable_rows"]} + assert expected <= found + + +def test_layout_rooms_carry_the_truth_numbers_names_and_measured_faces(plant): + for room in plant["truth"]["rooms"]: + rows = [ + r + for r in plant["layout"]["rooms"] + if r["number"] == room["number"] and r["name"] == room["name"] + ] + assert rows, room["number"] + expected = shoelace([tuple(p) for p in room["polygon"]]) + for row in rows: + assert row["face"] is not None, room["number"] + assert abs(row["face"]["area"] - expected) <= 1e-9 * expected, room["number"] + assert row["confidence"] == 0.9 + + +def test_the_pid_texts_are_read_as_cyrillic(plant): + assert plant["pid"]["languages"]["by_class"]["cyrillic"] > 0 + + +# ── hand-built snapshots ──────────────────────────────────────────────────── + + +def line(handle, a, b, layer="WALL", space="Model"): + return EntityRecord(handle=handle, type="LINE", layer=layer, space=space, points=(a, b)) + + +def text(handle, value, x, y, layer="TEXT", height=250.0): + return EntityRecord( + handle=handle, + type="TEXT", + layer=layer, + space="Model", + points=((x, y),), + text=value, + height=height, + ) + + +def insert(handle, block, x, y, attribs=(), space="Model"): + return EntityRecord( + handle=handle, + type="INSERT", + layer="0", + space=space, + points=((x, y),), + block=block, + attribs=tuple(attribs), + ) + + +def rect(prefix, x0, y0, x1, y1, layer="WALL"): + return [ + line(f"{prefix}1", (x0, y0), (x1, y0), layer), + line(f"{prefix}2", (x1, y0), (x1, y1), layer), + line(f"{prefix}3", (x1, y1), (x0, y1), layer), + line(f"{prefix}4", (x0, y1), (x0, y0), layer), + ] + + +def snapshot(records, insunits=4, layers=None, layouts=("Model",)): + return Snapshot( + source="test", + insunits=insunits, + extmin=None, + extmax=None, + layers=layers or {}, + layouts=layouts, + records=tuple(records), + ) + + +def test_a_room_label_inside_the_wall_faces_is_measured(): + assert classify_layer("WALL")["discipline"] == "architecture" + parsed = room_label("ROOM 101 PROCESS") + assert parsed is not None + # Inner face (200,200)-(6200,4200): 6000 x 4000 = 24 000 000 unit^2. + records = [ + *rect("O", 0.0, 0.0, 6400.0, 4400.0), + *rect("I", 200.0, 200.0, 6200.0, 4200.0), + text("R", "ROOM 101 PROCESS", 1000.0, 2000.0), + ] + report = describe(snapshot(records)) + (room,) = report["rooms"] + assert room["number"] == parsed["number"] + assert room["name"] == parsed["name"] + assert room["face"]["area"] == pytest.approx(24_000_000.0, abs=EPS) + assert room["face"]["polygon"] == [ + [200.0, 200.0], + [6200.0, 200.0], + [6200.0, 4200.0], + [200.0, 4200.0], + ] + assert room["source"] == "label+face" + assert room["confidence"] == 0.9 + # The wall ring between the outlines is the one unlabelled face. + assert report["room_faces"]["faces"] == 2 + assert report["room_faces"]["unlabelled"] == 1 + + +def test_an_attribute_and_a_text_of_one_tag_are_one_occurrence_and_a_copy_is_two(): + assert parse_tag("T4100") == "T4100" + records = [ + line("L", (0.0, 0.0), (20000.0, 0.0), "0"), + insert("B1", "TANK", 1000.0, 0.0, attribs=[("TAG", "T4100")]), + # 150 units away: inside 1 % of the 20 000 diagonal (200), so the same one. + text("X1", "T4100", 1150.0, 0.0), + # 18 000 away: a second occurrence. + text("X2", "T4100", 19000.0, 0.0), + ] + snap = snapshot(records) + occurrences = tag_occurrences(snap) + assert [(o["handle"], o["source"]) for o in occurrences] == [ + ("B1", "attribute"), + ("X2", "text"), + ] + report = describe(snap) + (row,) = report["equipment_tags"] + assert row["tag"] == "T4100" + assert row["count"] == 2 + assert row["ambiguous"] is True + assert row["confidence"] == 0.95 + assert any("appear more than once" in w for w in report["warnings"]) + + +def test_text_languages_are_counted_by_character_class(): + assert text_language("ПОДАЧА CIP") == "cyrillic" + assert text_language("DÖNÜŞ") == "turkish" + assert text_language("DÖNÜS") == "latin" # ö and ü alone are German too + assert text_language("SUPPLY") == "latin" + assert text_language("3,5") == "none" + records = [ + text("A", "ПОДАЧА", 0.0, 0.0), + text("B", "BESLEME DÖNÜŞ", 0.0, 500.0), + text("C", "SUPPLY", 0.0, 1000.0), + text("D", "12", 0.0, 1500.0), + ] + languages = describe(snapshot(records))["languages"] + assert languages["by_class"] == {"latin": 1, "turkish": 1, "cyrillic": 1, "none": 1} + assert languages["share"]["cyrillic"] == 0.25 + assert languages["samples"]["cyrillic"] == ["A"] + + +def test_blocks_are_split_named_and_anonymous_with_kinds_by_name(): + records = [ + insert("A1", "*U12", 0.0, 0.0), + insert("A2", "*U12", 100.0, 0.0), + insert("G1", "GATE_VALVE", 200.0, 0.0), + insert("G2", "GATE_VALVE", 300.0, 0.0), + insert("G3", "GATE_VALVE", 400.0, 0.0), + insert("P1", "PUMP_A", 500.0, 0.0), + ] + blocks = describe(snapshot(records))["blocks"] + assert blocks["inserts"] == 6 + assert [(r["name"], r["inserts"], r["kind"]) for r in blocks["named"]] == [ + ("GATE_VALVE", 3, "valve"), + ("PUMP_A", 1, "equipment"), + ] + assert blocks["anonymous"] == {"names": 1, "inserts": 2} + assert blocks["kinds"] == {"equipment": 1, "valve": 1} + + +def test_the_title_block_is_the_named_paper_space_block_with_its_attributes(): + title = insert( + "TB", + "A1_TITLE", + 0.0, + 0.0, + attribs=[("DWG_NO", "001"), ("REV", "B"), ("SCALE", "1:100"), ("DATE", "2026-09-24")], + space="Layout1", + ) + other = insert( + "M", + "NOTE_BOX", + 0.0, + 0.0, + attribs=[("A", "1"), ("B", "2"), ("C", "3"), ("D", "4"), ("E", "5")], + ) + report = describe(snapshot([title, other], layouts=("Model", "Layout1"))) + block = report["title_block"] + assert block["found"] is True + assert block["handle"] == "TB" + assert block["attributes"] == { + "DWG_NO": "001", + "REV": "B", + "SCALE": "1:100", + "DATE": "2026-09-24", + } + assert block["confidence"] == 0.9 + assert report["layouts"] == [ + {"name": "Model", "entities": 1}, + {"name": "Layout1", "entities": 1}, + ] + + +def test_no_title_block_is_said_not_invented(): + block = describe(snapshot([insert("M", "X", 0.0, 0.0, attribs=[("A", "1")])]))["title_block"] + assert block == {"found": False, "reason": "no block reference with three or more attributes"} + + +def test_a_layer_without_a_keyword_is_classified_by_its_texts(): + expected = classify_layer("CIP SUPPLY") + assert expected["service"] != "unknown" + assert classify_layer("L-17")["service"] == "unknown" + assert classify_layer("L-17")["discipline"] == "unknown" + records = [ + line("P1", (0.0, 0.0), (3000.0, 0.0), "L-17"), + line("P2", (3000.0, 0.0), (3000.0, 4000.0), "L-17"), + text("T1", "CIP SUPPLY", 1000.0, 100.0, layer="L-17"), + text("T2", "CIP SUPPLY", 3100.0, 2000.0, layer="L-17"), + ] + layers = {r["name"]: r for r in describe(snapshot(records))["layers"]} + row = layers["L-17"] + assert row["service"] == expected["service"] + assert row["discipline"] == expected["discipline"] + assert row["confidence"] == 0.6 + assert row["entities"] == 4 + assert row["length"] == pytest.approx(7000.0, abs=EPS) # 3000 + 4000 + assert row["length_m"] == pytest.approx(7.0, abs=EPS) + assert row["evidence"].startswith("2 of 2 texts on the layer read as") + + +def test_describe_leaves_the_snapshot_as_it_was(): + records = [*rect("O", 0.0, 0.0, 6400.0, 4400.0), text("R", "T4100", 100.0, 100.0)] + snap = snapshot(records) + before = copy.deepcopy(snap) + describe(snap) + assert snap == before +``` + +- [ ] **Step 10: Run it to verify it fails** + +Run: `uv run --frozen pytest tests/test_understand_describe.py -q` +Expected: FAIL — collection error `ModuleNotFoundError: No module named 'engineering.understand.describe'`. + +- [ ] **Step 11: Write `engineering/understand/describe.py`** + +```python +"""drawing_understand: one read of a foreign drawing, every claim with its evidence. + +`describe` assembles the report from the pure readers - units and extents +(`units`), bodies of content (`clusters`), and here: layers and what they +carry, equipment tags, rooms, text languages, blocks, layouts and the title +block. Each claim carries its evidence (what was measured or matched) and a +confidence; nothing is corrected silently, and the drawing is never modified +(the snapshot is a frozen copy). + +Confidences, in one place: + +* a layer name matching the vocabulary: `vocab.classify_layer`'s own (0.9 for + a direct keyword); the texts on an unnamed layer voting instead: + :data:`TEXT_VOTE_CONFIDENCE` when most of them agree, :data:`WEAK_VOTE_CONFIDENCE` + otherwise; +* a tag read from a block attribute :data:`ATTRIBUTE_TAG_CONFIDENCE`, from a + text :data:`TEXT_TAG_CONFIDENCE`; +* a room label that sits in a face of the wall layers + :data:`ROOM_FACE_CONFIDENCE`, a label alone :data:`ROOM_LABEL_CONFIDENCE`; +* a title block whose name says so :data:`TITLE_NAME_CONFIDENCE`, one chosen by + its attribute count alone :data:`TITLE_ATTRIBUTE_CONFIDENCE`; +* character classes are exact: a text either holds a Cyrillic letter or not. +""" + +from __future__ import annotations + +import math +import re +from collections import Counter +from collections.abc import Sequence + +from engineering.arch.faces import Face, face_containing, planar_faces +from engineering.pid.graph import ATTRIBUTE_TAGS, KEYWORDS +from engineering.understand.clusters import find_clusters +from engineering.understand.labels import parse_tag +from engineering.understand.snapshot import EntityRecord, Snapshot, length_of +from engineering.understand.units import infer_units, robust_extents, wall_segments +from engineering.understand.vocab import classify_layer, room_label + +__all__ = [ + "describe", + "find_rooms", + "prepare", + "tag_occurrences", + "text_language", +] + +TEXT_VOTE_CONFIDENCE = 0.6 +WEAK_VOTE_CONFIDENCE = 0.4 +ATTRIBUTE_TAG_CONFIDENCE = 0.95 +TEXT_TAG_CONFIDENCE = 0.8 +ROOM_FACE_CONFIDENCE = 0.9 +ROOM_LABEL_CONFIDENCE = 0.6 +TITLE_NAME_CONFIDENCE = 0.9 +TITLE_ATTRIBUTE_CONFIDENCE = 0.6 +#: The face finder pairs every wall segment with every other; above this many +#: the rooms are listed without faces and a warning says so. +FACE_SEGMENT_CAP = 3000 +#: Endpoints closer than this fraction of the robust diagonal are one corner +#: (1 mm on a 50 m plan drawn in millimetres). +FACE_TOL_FRACTION = 2e-5 +#: The same tag lettered twice closer than this fraction of the robust diagonal +#: (an attribute and a text beside one symbol) is one occurrence. +TAG_MERGE_FRACTION = 0.01 +#: Rows listed for named blocks, most inserted first. +BLOCK_ROW_CAP = 100 +#: Block-name words that mark a title block (EN / NL / DE / TR / RU). +TITLE_WORDS = ("TITLE", "TITEL", "STEMPEL", "SCHRIFTFELD", "ANTET", "ШТАМП") + +_TEXT_TYPES = frozenset({"TEXT", "MTEXT", "MULTILEADER"}) +_TOKEN_RE = re.compile(r"[A-Z]+") +_CYRILLIC = re.compile(r"[\u0400-\u04FF]") # the Cyrillic block +#: Letters only Turkish (of the five languages) writes; ç, ö and ü are shared +#: with German and French and are not evidence on their own. +_TURKISH = re.compile(r"[ĞğİıŞş]") +_LATIN = re.compile(r"[A-Za-z\u00C0-\u024F]") # ASCII and Latin-1 to Latin Extended-B + + +def _anchor(rec: EntityRecord) -> tuple[float, float] | None: + if rec.points: + return (float(rec.points[0][0]), float(rec.points[0][1])) + if rec.bbox is not None: + b = rec.bbox + return ((b[0] + b[2]) / 2.0, (b[1] + b[3]) / 2.0) + return None + + +def prepare(snap: Snapshot) -> dict: + """The shared first pass: model-space records, extents and outliers, the + records kept, clusters and membership. `describe` and `scale.scale_check` + both start here, so they agree on which entity is an outlier and which + cluster a tag sits in.""" + model = [r for r in snap.records if r.space == "Model"] + extents = robust_extents(model) + outliers = {o["handle"] for o in extents["outliers"]} + kept = [r for r in model if r.handle not in outliers] + clusters, membership = find_clusters(kept) + robust = extents["robust"] + diag = math.hypot(robust[2] - robust[0], robust[3] - robust[1]) if robust else 0.0 + return { + "model": model, + "extents": extents, + "kept": kept, + "clusters": clusters, + "membership": membership, + "diag": diag, + } + + +def tag_occurrences(snap: Snapshot, *, prep: dict | None = None) -> list[dict]: + """Every equipment tag on the drawing, one row per occurrence. + + A tag is read with `labels.parse_tag` from a TEXT / MTEXT / MULTILEADER, + or from an INSERT attribute named like a tag (`pid.graph.ATTRIBUTE_TAGS`). + Two readings of one tag in one space closer than + :data:`TAG_MERGE_FRACTION` of the diagonal are one occurrence (the + attribute's position and confidence win). Rows carry ``tag``, ``handle``, + ``at``, ``space``, ``layer``, ``cluster`` (model space only), ``source`` + and ``confidence``. + """ + prep = prep or prepare(snap) + radius = prep["diag"] * TAG_MERGE_FRACTION + found: list[dict] = [] + for rec in snap.records: + at = _anchor(rec) + if at is None: + continue + readings = [] + if rec.type in _TEXT_TYPES and rec.text: + readings.append((parse_tag(rec.text), "text", TEXT_TAG_CONFIDENCE)) + elif rec.type == "INSERT": + for name, value in rec.attribs: + if name.upper() in ATTRIBUTE_TAGS and value: + readings.append((parse_tag(value), "attribute", ATTRIBUTE_TAG_CONFIDENCE)) + for tag, source, confidence in readings: + if not tag: + continue + found.append( + { + "tag": tag, + "handle": rec.handle, + "at": [at[0], at[1]], + "space": rec.space, + "layer": rec.layer, + "cluster": prep["membership"].get(rec.handle) if rec.space == "Model" else None, + "source": source, + "confidence": confidence, + } + ) + found.sort(key=lambda o: (-o["confidence"], o["tag"], o["handle"])) + kept: list[dict] = [] + for occ in found: + twin = next( + ( + k + for k in kept + if k["tag"] == occ["tag"] + and k["space"] == occ["space"] + and math.dist(k["at"], occ["at"]) <= radius + ), + None, + ) + if twin is None: + kept.append(occ) + kept.sort(key=lambda o: (o["tag"], o["space"], o["at"][0], o["at"][1])) + return kept + + +def find_rooms(records: Sequence[EntityRecord], *, tol: float, membership: dict) -> dict: + """Room labels and the faces of the wall layers they sit in. + + A label is any text `vocab.room_label` recognises. Wall layers are the + ones `vocab.classify_layer` files under ``architecture``; their straight + segments go through `engineering.arch.faces.planar_faces` (arcs are left + out and counted). ``labelled`` pairs each room row with its `Face` for a + caller that needs the geometry; ``rooms`` is JSON-ready. + """ + labels = [] + for rec in records: + if rec.type in _TEXT_TYPES and rec.text: + parsed = room_label(rec.text) + at = _anchor(rec) + if parsed and at is not None: + labels.append((rec, parsed, at)) + segments, bulged = wall_segments(records) + faces: tuple[Face, ...] = () + skipped = None + if len(segments) > FACE_SEGMENT_CAP: + skipped = ( + f"{len(segments)} wall segments exceed the {FACE_SEGMENT_CAP}-segment cap of the " + "face finder; rooms are listed by label only" + ) + elif segments: + faces = planar_faces(segments, tol=tol) + rooms: list[dict] = [] + labelled: list[tuple[dict, Face]] = [] + used: set[int] = set() + for rec, parsed, at in labels: + face = face_containing(faces, at) if faces else None + row = { + "number": parsed.get("number"), + "name": parsed.get("name"), + "language": parsed.get("language"), + "text": rec.text, + "handle": rec.handle, + "at": [at[0], at[1]], + "cluster": membership.get(rec.handle), + "face": None, + "source": "label", + "confidence": ROOM_LABEL_CONFIDENCE, + } + if face is not None: + used.add(id(face)) + row["face"] = { + "area": face.area, + "polygon": [[p[0], p[1]] for p in face.loop], + "centroid": [face.centroid[0], face.centroid[1]], + } + row["source"] = "label+face" + row["confidence"] = ROOM_FACE_CONFIDENCE + labelled.append((row, face)) + rooms.append(row) + rooms.sort(key=lambda r: (r["cluster"] or "", r["number"] or "", r["name"] or "", r["at"])) + return { + "rooms": rooms, + "labelled": labelled, + "faces": len(faces), + "faces_unlabelled": sum(1 for f in faces if id(f) not in used), + "wall_segments": len(segments), + "arcs_skipped": bulged, + "skipped": skipped, + } + + +def text_language(text: str) -> str: + """``cyrillic`` | ``turkish`` | ``latin`` | ``none`` by character class, in + that precedence (a Russian label with a Latin tag in it is Cyrillic).""" + if _CYRILLIC.search(text): + return "cyrillic" + if _TURKISH.search(text): + return "turkish" + if _LATIN.search(text): + return "latin" + return "none" + + +def _layers(snap: Snapshot, unit_mm: float | None) -> list[dict]: + counts: Counter = Counter() + lengths: dict[str, float] = {} + texts: dict[str, list[str]] = {} + for rec in snap.records: + counts[rec.layer] += 1 + lengths[rec.layer] = lengths.get(rec.layer, 0.0) + length_of(rec) + if rec.type in _TEXT_TYPES and rec.text: + texts.setdefault(rec.layer, []).append(rec.text) + rows = [] + for name in sorted(set(snap.layers) | set(counts)): + props = snap.layers.get(name, {}) + c = classify_layer(name) + row = { + "name": name, + "entities": counts.get(name, 0), + "length": lengths.get(name, 0.0), + "length_m": lengths.get(name, 0.0) * unit_mm / 1000.0 if unit_mm else None, + "color": props.get("color"), + "linetype": props.get("linetype"), + "discipline": c.get("discipline", "unknown"), + "service": c.get("service", "unknown"), + "keyword": c.get("keyword"), + "language": c.get("language"), + "confidence": c.get("confidence", 0.0), + "evidence": f"layer name keyword {c.get('keyword')!r}" if c.get("keyword") else None, + } + if row["discipline"] == "unknown" and row["service"] == "unknown": + votes: Counter = Counter() + said = texts.get(name, []) + for text in said: + t = classify_layer(text) + if ( + t.get("discipline", "unknown") != "unknown" + or t.get("service", "unknown") != "unknown" + ): + votes[(t.get("discipline", "unknown"), t.get("service", "unknown"))] += 1 + if votes: + (discipline, service), n = sorted(votes.items(), key=lambda kv: (-kv[1], kv[0]))[0] + row.update( + discipline=discipline, + service=service, + confidence=( + TEXT_VOTE_CONFIDENCE if n * 2 > len(said) else WEAK_VOTE_CONFIDENCE + ), + evidence=f"{n} of {len(said)} texts on the layer read as {service} / {discipline}", + ) + rows.append(row) + rows.sort(key=lambda r: (-r["entities"], r["name"])) + return rows + + +def _block_kind(name: str) -> str | None: + for token in _TOKEN_RE.findall(name.upper()): + kind = KEYWORDS.get(token) + if kind: + return kind + return None + + +def _blocks(snap: Snapshot) -> dict: + inserts = [r for r in snap.records if r.type == "INSERT" and r.block] + named = Counter(r.block for r in inserts if not r.block.startswith("*")) + anonymous = Counter(r.block for r in inserts if r.block.startswith("*")) + rows = [ + { + "name": name, + "inserts": n, + "kind": _block_kind(name), + "discipline": classify_layer(name).get("discipline", "unknown"), + } + for name, n in sorted(named.items(), key=lambda kv: (-kv[1], kv[0])) + ] + kinds = Counter(row["kind"] for row in rows if row["kind"]) + return { + "inserts": len(inserts), + "named": rows[:BLOCK_ROW_CAP], + "named_total": len(rows), + "anonymous": {"names": len(anonymous), "inserts": sum(anonymous.values())}, + "kinds": dict(sorted(kinds.items())), + } + + +def _title_block(snap: Snapshot) -> dict: + best = None + for rec in snap.records: + if rec.type != "INSERT" or len(rec.attribs) < 3: + continue + upper = (rec.block or "").upper() + named = any(word in upper for word in TITLE_WORDS) + key = (named, rec.space != "Model", len(rec.attribs)) + if best is None or key > best[0]: + best = (key, rec, named) + if best is None: + return {"found": False, "reason": "no block reference with three or more attributes"} + _key, rec, named = best + return { + "found": True, + "handle": rec.handle, + "block": rec.block, + "space": rec.space, + "attributes": {tag: value for tag, value in rec.attribs}, + "confidence": TITLE_NAME_CONFIDENCE if named else TITLE_ATTRIBUTE_CONFIDENCE, + "evidence": ( + f"block name {rec.block!r} reads as a title block" + if named + else f"the block reference with the most attributes ({len(rec.attribs)})" + ), + } + + +def _languages(snap: Snapshot, layer_rows: list[dict]) -> dict: + by_class: Counter = Counter() + samples: dict[str, list[str]] = {} + for rec in snap.records: + if rec.type in _TEXT_TYPES and rec.text: + lang = text_language(rec.text) + by_class[lang] += 1 + if len(samples.setdefault(lang, [])) < 3: + samples[lang].append(rec.handle) + total = sum(by_class.values()) + layer_langs = Counter(row["language"] for row in layer_rows if row.get("language")) + return { + "texts": total, + "by_class": {k: by_class.get(k, 0) for k in ("latin", "turkish", "cyrillic", "none")}, + "share": { + k: (by_class.get(k, 0) / total if total else 0.0) + for k in ("latin", "turkish", "cyrillic", "none") + }, + "samples": samples, + "layer_name_languages": dict(sorted(layer_langs.items())), + "confidence": 1.0, + } + + +def describe(snap: Snapshot) -> dict: + """The whole report; see the module docstring for the confidences.""" + prep = prepare(snap) + warnings: list[str] = [] + tol = prep["diag"] * FACE_TOL_FRACTION if prep["diag"] > 0 else 1.0 + rooms = find_rooms(prep["kept"], tol=tol, membership=prep["membership"]) + units = infer_units( + snap, + records=prep["kept"], + room_areas=[row["face"]["area"] for row, _face in rooms["labelled"]], + ) + if units["warning"]: + warnings.append(units["warning"]) + extents = dict(prep["extents"]) + extents["declared"] = ( + {"min": list(snap.extmin), "max": list(snap.extmax)} + if snap.extmin is not None and snap.extmax is not None + else None + ) + if extents["outliers"]: + handles = ", ".join(o["handle"] for o in extents["outliers"][:5]) + stretch = extents["stretch"] + warnings.append( + f"{len(extents['outliers'])} entit{'y' if len(extents['outliers']) == 1 else 'ies'} " + f"far outside the content stretch the extents" + + (f" {stretch:.3g}x" if stretch else "") + + f": {handles}" + ) + if rooms["skipped"]: + warnings.append(rooms["skipped"]) + + occurrences = tag_occurrences(snap, prep=prep) + by_tag: dict[str, list[dict]] = {} + for occ in occurrences: + by_tag.setdefault(occ["tag"], []).append(occ) + tags = [ + { + "tag": tag, + "count": len(rows), + "ambiguous": len(rows) > 1, + "confidence": max(r["confidence"] for r in rows), + "occurrences": [ + {k: r[k] for k in ("handle", "at", "space", "cluster", "layer", "source")} + for r in rows + ], + } + for tag, rows in sorted(by_tag.items()) + ] + repeated = [t["tag"] for t in tags if t["ambiguous"]] + if repeated: + warnings.append( + f"{len(repeated)} tag(s) appear more than once (plan copies, legends, second " + f"sheets): {', '.join(repeated[:5])}" + ) + + layer_rows = _layers(snap, units["mm_per_unit"]) + layout_names = list(snap.layouts) + if "Model" not in layout_names: + layout_names.insert(0, "Model") + space_counts = Counter(r.space for r in snap.records) + return { + "source": snap.source, + "entities": len(snap.records), + "units": units, + "extents": extents, + "clusters": prep["clusters"], + "layers": layer_rows, + "equipment_tags": tags, + "rooms": rooms["rooms"], + "room_faces": { + "faces": rooms["faces"], + "unlabelled": rooms["faces_unlabelled"], + "wall_segments": rooms["wall_segments"], + "arcs_skipped": rooms["arcs_skipped"], + "tolerance": tol, + }, + "languages": _languages(snap, layer_rows), + "blocks": _blocks(snap), + "layouts": [{"name": n, "entities": space_counts.get(n, 0)} for n in layout_names], + "title_block": _title_block(snap), + "warnings": warnings, + } +``` + +- [ ] **Step 12: Run the describe test to verify it passes** + +Run: `uv run --frozen pytest tests/test_understand_describe.py -q` +Expected: `15 passed`. + +- [ ] **Step 13: Write the failing wire test** + +Create `tests/test_understand_tools.py`: + +```python +"""drawing_understand over the MCP wire, on a fresh headless document.""" + +from __future__ import annotations + +import pytest +import pytest_asyncio +from fastmcp import Client + +import server +from tests.fixtures.plant_pair import build_plant_pair + +pytestmark = pytest.mark.asyncio + + +@pytest_asyncio.fixture +async def client(monkeypatch): + """In-memory client on a fresh headless document - never the operator's AutoCAD.""" + monkeypatch.setenv("AUTOCAD_MCP_BACKEND", "ezdxf") + async with Client(server.mcp) as connected: + status = (await connected.call_tool("system_status", {})).structured_content or {} + assert status.get("backend") == "ezdxf", "these tests must never touch live AutoCAD" + await connected.call_tool("drawing_new", {}) + yield connected + + +async def test_drawing_understand_reads_the_synthetic_layout_file(client, tmp_path): + truth = build_plant_pair(tmp_path) + result = await client.call_tool("drawing_understand", {"path": truth["layout"]}) + report = result.structured_content + assert report["units"]["declared"]["code"] == truth["layout_insunits"] + assert report["units"]["inferred"] == truth["layout_true_unit"] + assert report["extents"]["outliers"][0]["handle"] == truth["outlier_handle"] + assert len(report["clusters"]) == truth["cluster_count"] + + +async def test_drawing_understand_reads_the_current_document_and_leaves_it_alone(client): + # A 6400 x 4400 outline on WALL (perimeter 2 x (6400 + 4400) = 21 600) and + # one tag. + corners = [(0.0, 0.0), (6400.0, 0.0), (6400.0, 4400.0), (0.0, 4400.0)] + for (x1, y1), (x2, y2) in zip(corners, corners[1:] + corners[:1], strict=True): + await client.call_tool( + "entity_create_line", {"x1": x1, "y1": y1, "x2": x2, "y2": y2, "layer": "WALL"} + ) + await client.call_tool( + "entity_create_text", + {"text": "T4100", "x": 1000.0, "y": 1000.0, "height": 250.0, "layer": "TAGS"}, + ) + before = (await client.call_tool("analysis_entity_stats", {})).structured_content + report = (await client.call_tool("drawing_understand", {})).structured_content + after = (await client.call_tool("analysis_entity_stats", {})).structured_content + assert after == before + layers = {row["name"]: row for row in report["layers"]} + assert layers["WALL"]["entities"] == 4 + assert layers["WALL"]["length"] == pytest.approx(21600.0, abs=1e-9) + assert [t["tag"] for t in report["equipment_tags"]] == ["T4100"] + + +async def test_a_missing_file_is_refused_by_path(client, tmp_path): + missing = tmp_path / "nowhere.dxf" + result = await client.call_tool( + "drawing_understand", {"path": str(missing)}, raise_on_error=False + ) + assert result.is_error is True + assert "path: no such file" in result.content[0].text + assert "nowhere.dxf" in result.content[0].text + + +async def test_a_file_the_engine_cannot_parse_is_refused(client, tmp_path): + # Not a DXF: the snapshot's parse error comes back as a tool error that + # names the file, never as a traceback. + garbage = tmp_path / "not_a_drawing.dxf" + garbage.write_text("this is not a drawing", encoding="utf-8") + result = await client.call_tool( + "drawing_understand", {"path": str(garbage)}, raise_on_error=False + ) + assert result.is_error is True + assert "not_a_drawing.dxf" in result.content[0].text + + +async def test_drawing_understand_is_advertised_read_only(client): + tools = {tool.name: tool for tool in await client.list_tools()} + assert tools["drawing_understand"].annotations.readOnlyHint is True +``` + +- [ ] **Step 14: Run it to verify it fails** + +Run: `uv run --frozen pytest tests/test_understand_tools.py -q` +Expected: `5 failed` — the two calls raise `fastmcp.exceptions.ToolError: Not found: Unknown tool: 'drawing_understand'`, the two refusal tests fail their message assertions (the text is the unknown-tool error), and the annotation test raises `KeyError: 'drawing_understand'`. + +- [ ] **Step 15: Register `drawing_understand` and open SECTION 26** + +In `server.py`, replace the end of SECTION 25 and the RESOURCES banner + +```python + try: + return await draw_plan_from_spec(_backend(ctx), spec) + except ValueError as exc: + raise ToolError(str(exc)) from exc + + +# --------------------------------------------------------------------------- +# ── RESOURCES ─────────────────────────────────────────────────────────────── +# --------------------------------------------------------------------------- +``` + +with + +```python + try: + return await draw_plan_from_spec(_backend(ctx), spec) + except ValueError as exc: + raise ToolError(str(exc)) from exc + + +# --------------------------------------------------------------------------- +# ── SECTION 26: Understanding & QA (1 tool) ───────────────────────────────── +# --------------------------------------------------------------------------- + + +@cad_tool( + summary="Read a foreign drawing: units, extents, clusters, layers, tags, rooms, languages.", + cost="read", +) +@mcp.tool( + annotations={"title": "Understand: Describe Drawing", "readOnlyHint": True}, + tags={"analysis", "query"}, +) +async def drawing_understand( + path: Annotated[ + str | None, + Field( + default=None, + description="DXF file to read (a DWG only on a live seat); omitted: the current " + "document.", + ), + ] = None, + ctx: Context = None, +) -> dict: + """One report on a drawing somebody else made, every claim with its evidence + and a confidence. Never modifies the drawing: a file is read as it is, and + the current document through a snapshot (headless: the document in memory; + live: a `Document.Export` DXF copy in a temporary folder, deleted after). + + - `units`: `$INSUNITS` as declared against the unit the geometry implies + (text heights, wall thickness, door swings, labelled room areas, overall + size - each item with its median and its value under every candidate + unit). A disagreement is a warning with the numbers, never a silent + correction. + - `extents`: declared, computed and robust boxes and the `outliers` that + stretch them, farthest first, by handle. + - `clusters`: separate bodies of model-space content (plan copies, sheets, + details), each with its box, entity count, commonest layers and tallest + labels. + - `layers`: entity count, drawn length (and metres in the inferred unit) + and the discipline / service each most likely carries - the vocabulary's + confidence (0.9) for a name keyword, 0.6 when most texts on the layer + agree, 0.4 when fewer do. + - `equipment_tags` (a tag found more than once is `ambiguous`), `rooms` + (room labels in five languages, each with the wall-layer face it sits in + and its measured area), `languages` (Latin / Turkish / Cyrillic character + classes), `blocks` (named / anonymous, kinds by name), `layouts`, + `title_block` and `warnings`. + + Refused before anything is read: a `path` that does not exist, a DXF over + `MAX_DXF_BYTES` (the refusal names the size and the variable), and a file + the engine cannot parse (a DWG headlessly: ezdxf reads DXF only). + """ + import asyncio + + from ezdxf.lldxf.const import DXFError + + from engineering.understand.describe import describe + from engineering.understand.snapshot import take_snapshot + + target = None + if path: + target = validate_path(path) + if not target.is_file(): + raise ToolError(f"path: no such file {str(target)!r}") + await ctx.info(f"drawing_understand: {target or 'current document'}") + try: + snap = await take_snapshot(_backend(ctx), str(target) if target else None) + except (ValueError, OSError, RuntimeError, DXFError) as exc: + raise ToolError(str(exc)) from exc + return await asyncio.to_thread(describe, snap) + + +# --------------------------------------------------------------------------- +# ── RESOURCES ─────────────────────────────────────────────────────────────── +# --------------------------------------------------------------------------- +``` + +The banner says `(1 tool)`: `tests/test_release_consistency.py::test_section_header_counts_match_decorators` counts one `@mcp.tool` under it, and Task 4 grows it to `(2 tools)` — the banner protocol of the skeleton (group D opens the same banner on its own branch; the merge keeps one at 4). The tool is tagged `analysis` and `query` and nothing new is added to `_GROUP_TAG_PRIORITY`, `_GROUP_TAG_LABELS`, `TOOL_PACK_NAMES`, `PACK_TOOL_NAMES` or `LEAN_TOOL_NAMES`: `analysis` already ranks it, the tool is core-pack by default, and the lean entry is Task 11's (an edit to those lists on three branches would conflict three ways). `ToolError`, `Annotated`, `Field`, `Context`, `validate_path`, `_backend` and `cad_tool` are already imported or defined in `server.py`; `asyncio`, `DXFError` and the understand modules are imported inside the function, the house pattern for a section's engine. + +- [ ] **Step 16: Add the alias record** + +In `discovery/aliases.py`, replace the end of `TOOL_ALIASES` + +```python + "mimari plan", + ), + ), +} +``` + +with + +```python + "mimari plan", + ), + ), + # ── Understanding & QA (SECTION 26, track H group U) ──────────────────── + # No AutoCAD command reads a foreign drawing back as a report, so `acad` + # is empty. UNITS stays with drawing_settings (it sets units; this reads + # what the geometry implies), and no synonym here says "layers", "blocks" + # or "how big", the words the golden layer_list / block_list / + # analysis_bounding_box queries lean on. + "drawing_understand": ToolAliases( + acad=(), + synonyms=( + "understand this drawing", + "read a foreign drawing", + "summarise a client drawing", + "drawing report", + "which unit is this drawing really in", + "wrong insunits", + "inches or millimetres", + "two copies of the plan", + "stray entity far away", + "what is on this p&id", + "birim", + "birimi ne", + "çizimi anla", + "cizimi anla", + "çizim raporu", + "cizim raporu", + "yabancı çizim", + "yabanci cizim", + ), + ), +} +``` + +Measured before writing this plan (the tool registered in a scratch copy of `main`): `tests/test_discovery_aliases.py` and the golden set in `tests/test_discovery_transform.py` stay green with these synonyms — none of them contains `layers`, `blocks` or `how big`, the words the golden `layer_list` / `block_list` / `analysis_bounding_box` queries lean on. + +- [ ] **Step 17: Run the wire test to verify it passes** + +Run: `uv run --frozen pytest tests/test_understand_tools.py -q` +Expected: `5 passed`. + +- [ ] **Step 18: Regenerate the inventory and bump the documented counts** + +Run: `uv run --frozen python scripts/generate_tool_inventory.py` +Expected output: `wrote docs/tool-inventory.json: 242 tools, 8 resources, 5 prompts, 26 groups` + +In `README.md` change the release-snapshot lines from + +``` +> **v1.5 release snapshot:** 241 tools · 8 resources · 5 prompt templates · 4516 collected tests. +> 241 is the **registered** count; a default install advertises 236 over `tools/list`, +``` + +to + +``` +> **v1.5 release snapshot:** 242 tools · 8 resources · 5 prompt templates · 4516 collected tests. +> 242 is the **registered** count; a default install advertises 237 over `tools/list`, +``` + +and the caption under the "What you get" table from `241 tools in 26 groups;` to `242 tools in 26 groups;`. Leave the collected-tests figure alone (the known red; Task 11 retakes it on main) and leave the "205 AutoCAD command names and 1318 synonym phrases … covering all 241 tools" sentence alone: all three group branches add alias records, so that sentence is recomputed once after the merge. + +- [ ] **Step 19: Bump the frozen group-size snapshot** + +In `tests/test_tool_registry.py::test_tool_group_sizes_are_unchanged`: + +- the docstring's first line `"""Frozen snapshot of the surface (241 tools, 26 groups).` becomes `"""Frozen snapshot of the surface (242 tools, 26 groups).`; +- in the asserted dict, `"analysis": 12,` becomes `"analysis": 13,`; +- `assert sum(sizes.values()) == 241` becomes `assert sum(sizes.values()) == 242`; +- replace the docstring's closing lines + +```python + had grown while the tag was unranked on them returned to its + pre-track-F value. + """ +``` + +with + +```python + had grown while the tag was unranked on them returned to its + pre-track-F value. + + Track H group U opened SECTION 26 (Understanding & QA) with + `drawing_understand`, tagged `analysis` and `query`: `analysis` outranks + `query` in `_GROUP_TAG_PRIORITY`, so the reader files under `analysis` + (12 -> 13) beside the other whole-drawing queries. This snapshot is + branch-local: the track H merge task recomputes it once the understand, + network and diff branches have landed. + """ +``` + +- [ ] **Step 20: Gates** + +Run each and require it clean: + +``` +uv run --frozen ruff check engineering/understand/units.py engineering/understand/clusters.py engineering/understand/describe.py server.py discovery/aliases.py tests/test_understand_units.py tests/test_understand_clusters.py tests/test_understand_describe.py tests/test_understand_tools.py tests/test_tool_registry.py +uv run --frozen ruff format engineering/understand/units.py engineering/understand/clusters.py engineering/understand/describe.py tests/test_understand_units.py tests/test_understand_clusters.py tests/test_understand_describe.py tests/test_understand_tools.py +uv run --frozen ruff format --check server.py discovery/aliases.py tests/test_tool_registry.py +uv run --frozen python scripts/generate_tool_inventory.py --check +uv run --frozen pytest tests/test_understand_units.py tests/test_understand_clusters.py tests/test_understand_describe.py tests/test_understand_tools.py -q +uv run --frozen pytest tests/test_tool_registry.py tests/test_tool_inventory.py tests/test_tool_packs.py tests/test_discovery_aliases.py tests/test_discovery_transform.py tests/test_capability_contract.py -q +uv run --frozen pytest tests/test_release_consistency.py -q --deselect tests/test_release_consistency.py::test_document_numbers_match_the_tree +uv run --frozen pytest -q --deselect tests/test_release_consistency.py::test_document_numbers_match_the_tree +``` + +`generate_tool_inventory.py --check` prints `ok: docs/tool-inventory.json matches the live surface`. The four understand files give `34 passed`. The deselect is the known-red collected-tests drift named in the Global Constraints — record it in the task report. `test_section_header_counts_match_decorators` must pass with the banner at `(1 tool)`. No capability key is added (the tool composes Task 1's snapshot, which both engines serve), so `tests/test_capability_contract.py` passes untouched. `ruff format` must leave `server.py` byte-identical outside the new section. + +- [ ] **Step 21: Commit** + +``` +git add engineering/understand/units.py engineering/understand/clusters.py engineering/understand/describe.py server.py discovery/aliases.py docs/tool-inventory.json README.md tests/test_tool_registry.py tests/test_understand_units.py tests/test_understand_clusters.py tests/test_understand_describe.py tests/test_understand_tools.py +git -c user.name="Umut CAN" -c user.email="umutcan.edizaslan@gmail.com" commit -m "$(cat <<'MSG' +feat(understand): drawing_understand - units, extents, clusters, layers, tags, rooms + +A drawing somebody else made, read back as one report in which every +claim carries its evidence and a confidence; the drawing is never +modified. + +- units.py: $INSUNITS against the unit the geometry implies (text + heights, wall thickness, door swings, labelled room areas, overall + size). The header is overruled only when the evidence outweighs it, + and then with a warning that quotes the numbers. Robust extents grow + from the quartile box of entity centres; what never joins is an + outlier, named by handle. +- clusters.py: separate bodies of model-space content by raster, + 8-connectivity and box absorption - two copies of one plan are two + clusters. +- describe.py: layers with drawn length and the vocabulary's meaning + (name keyword, else the texts on the layer), equipment tags (one tag + lettered twice beside its symbol merged, repeats flagged ambiguous), + room labels in the wall-layer faces of engineering.arch.faces, text + languages by character class, blocks, layouts, the title block. +- server.py opens SECTION 26 (Understanding & QA) with + drawing_understand: core, read-only, tagged analysis/query. Alias + record, inventory, README release snapshot 242, frozen group sizes + (analysis 13). + +Co-Authored-By: Claude Opus 5.5 (1M context) +Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd +MSG +)" +``` + +--- + +### Task 4: drawing_scale_check — is this drawing to scale against that one + +**Files:** +- Create: `engineering/understand/scale.py` +- Modify: `server.py` (SECTION 26 banner `(1 tool)` → `(2 tools)`; `drawing_scale_check` after `drawing_understand`) +- Modify: `discovery/aliases.py` (the `drawing_scale_check` record, appended after `drawing_understand`'s) +- Modify: `docs/tool-inventory.json` (regenerated), `README.md` (release-snapshot tool counts 242 → 243), `tests/test_tool_registry.py` (frozen group sizes: `analysis` 13 → 14, total 243) +- Test: `tests/test_understand_scale.py`, `tests/test_understand_scale_tools.py` + +**Interfaces:** +- Consumes: + - Task 3: `engineering.understand.describe.prepare(snap) -> dict`, `tag_occurrences(snap, *, prep=None) -> list[dict]`, `find_rooms(records, *, tol, membership) -> dict`, `FACE_TOL_FRACTION`; `engineering.understand.units.infer_units(snap, *, records=None, room_areas=()) -> dict`; the MCP tool `drawing_understand` already under the SECTION 26 banner. + - Task 1: `EntityRecord`, `Snapshot`, `read_snapshot(path: str) -> Snapshot`, `async def take_snapshot(backend, path: str | None = None) -> Snapshot`; `labels.parse_tag` (through `tag_occurrences`). + - Task 2: `build_plant_pair(directory) -> dict` (`pid`, `layout`, `scale_verdict`). + - Existing: `engineering.arch.faces.face_containing(faces, point) -> Face | None`. +- Produces: + - `engineering/understand/scale.py`: `match_tags(a: Snapshot, b: Snapshot, *, cluster_a=None, cluster_b=None) -> dict` — `{"matched": {tag: (pa, pb)}, "ambiguous": [tag, …], "occurrences": {tag: {"a": n, "b": m}}, "only_a": [...], "only_b": [...], "cluster_a", "cluster_b"}`; a cluster id is one of `find_clusters`' `"C1"`, `"C2"`, …, None is the whole model space, and an unknown id is refused with the ids the drawing has. + - `scale_check(a: Snapshot, b: Snapshot, *, min_pair_mm=2000.0) -> dict` — `{"verdict": "to_scale"|"schematic"|"partly"|"insufficient", "factor", "median", "q1", "q3", "iqr", "within_10pct", "pair_count", "pairs", "pairs_truncated", "pairs_below_floor", "matched": [{tag, a, b}], "ambiguous", "occurrences", "only_a", "only_b", "groups": [{group, kind, tags, verdict, factor, median, q1, q3, iqr, within_10pct, pair_count}], "scope": {"cluster_a", "cluster_b"}, "units": {"a", "b"}, "min_pair_mm", "thresholds", "notes"}`; `b` is the reference. + - Constants `WITHIN = 0.10`, `TO_SCALE_SHARE = 0.80`, `SCHEMATIC_SHARE = 0.50`, `MIN_TAGS = 3` (spec §6). Task 6 (group N, cut beside this branch) cannot import this module: `pipe_rows(..., scale=...)` accepts a `scale_check(pid, layout)` result in this shape (Task 12's benchmark, smoke and field-test script pass one) and otherwise computes its own `scale_stand_in`; its layout tag positions come from its own `layout_tags`, which applies the same rule — tags unique within the chosen cluster only, the rest `ambiguous` — so Review Focus 3 is reused as a rule, not as an import. + - MCP tool `drawing_scale_check(reference: str, path: str | None = None, min_pair_mm: float = 2000.0) -> dict` (core pack, read-only, tags `analysis` + `query`). + +- [ ] **Step 1: Write the failing scale test** + +Create `tests/test_understand_scale.py`. `test_a_tag_found_twice_is_ambiguous_and_takes_no_part` and `test_two_plan_copies_on_the_reference_resolve_to_one_copy` are Review Focus 3 (the same tag more than once); `test_cyrillic_look_alike_tags_match_their_latin_twins` checks that Review Focus 2's folding (Task 1) reaches the matcher. + +```python +"""drawing_scale_check: tag matching and ratio statistics. + +The hand-built plans put six tags at coordinates written below; every pair +distance and ratio in the comments is worked out from them. +""" + +from __future__ import annotations + +import math +from itertools import combinations + +import pytest + +from engineering.understand.scale import match_tags, scale_check +from engineering.understand.snapshot import EntityRecord, Snapshot, read_snapshot +from tests.fixtures.plant_pair import build_plant_pair + +EPS = 1e-12 + +#: Six tags of one plant, in millimetres. The closest pair is T4100-T4500, +#: hypot(6000, 4000) = 7211.1, so every pair clears the 2 m floor. +PLAN = { + "T4100": (0.0, 0.0), + "T4200": (12000.0, 0.0), + "T4300": (12000.0, 9000.0), + "T4400": (0.0, 9000.0), + "T4500": (6000.0, 4000.0), + "T4600": (20000.0, 15000.0), +} + + +def text(handle, value, x, y): + return EntityRecord( + handle=handle, type="TEXT", layer="TAGS", space="Model", points=((x, y),), text=value + ) + + +def outline(handle, x0, y0, x1, y1): + return EntityRecord( + handle=handle, + type="LWPOLYLINE", + layer="0", + space="Model", + points=((x0, y0), (x1, y0), (x1, y1), (x0, y1)), + bulges=(0.0, 0.0, 0.0, 0.0), + closed=True, + ) + + +def snapshot(records): + return Snapshot( + source="test", + insunits=4, + extmin=None, + extmax=None, + layers={}, + layouts=("Model",), + records=tuple(records), + ) + + +def tags(positions, prefix="H", factor=1.0, dx=0.0): + return [ + text(f"{prefix}{i}", tag, x * factor + dx, y * factor) + for i, (tag, (x, y)) in enumerate(sorted(positions.items())) + ] + + +def test_a_drawing_at_exactly_half_scale_is_to_scale_with_factor_one_half(): + result = scale_check(snapshot(tags(PLAN, "A", 0.5)), snapshot(tags(PLAN, "B"))) + assert result["verdict"] == "to_scale" + assert result["factor"] == pytest.approx(0.5, abs=EPS) + assert result["median"] == pytest.approx(0.5, abs=EPS) + assert result["iqr"] == pytest.approx(0.0, abs=EPS) + assert result["within_10pct"] == 1.0 + assert result["pair_count"] == 15 # C(6, 2) + assert result["pairs_below_floor"] == 0 + assert [m["tag"] for m in result["matched"]] == sorted(PLAN) + assert result["ambiguous"] == [] + assert result["units"]["a"]["unit"] == result["units"]["b"]["unit"] == "mm" + + +def test_a_rearranged_drawing_is_schematic(): + # The same six tags, shuffled onto each other's places and one moved off. + rearranged = { + "T4100": (12000.0, 9000.0), + "T4200": (0.0, 0.0), + "T4300": (20000.0, 15000.0), + "T4400": (6000.0, 4000.0), + "T4500": (12000.0, 0.0), + "T4600": (0.0, 20000.0), + } + result = scale_check(snapshot(tags(rearranged, "A")), snapshot(tags(PLAN, "B"))) + ratios = sorted( + math.dist(rearranged[i], rearranged[j]) / math.dist(PLAN[i], PLAN[j]) + for i, j in combinations(sorted(PLAN), 2) + ) + median = ratios[7] # 15 ratios: the 8th + within = sum(1 for r in ratios if abs(r - median) <= 0.1 * median) / 15 + assert result["median"] == pytest.approx(median, abs=EPS) + assert result["within_10pct"] == pytest.approx(within, abs=EPS) + assert within <= 0.5 + assert result["verdict"] == "schematic" + assert result["factor"] is None + + +def test_a_tag_found_twice_is_ambiguous_and_takes_no_part(): + # T4400 appears twice on the reference: neither copy is matched. + reference = [*tags(PLAN, "B"), text("DUP", "T4400", 3000.0, 12000.0)] + matched = match_tags(snapshot(tags(PLAN, "A", 0.5)), snapshot(reference)) + assert "T4400" not in matched["matched"] + assert matched["ambiguous"] == ["T4400"] + assert matched["occurrences"] == {"T4400": {"a": 1, "b": 2}} + assert matched["matched"]["T4200"] == ((6000.0, 0.0), (12000.0, 0.0)) + + result = scale_check(snapshot(tags(PLAN, "A", 0.5)), snapshot(reference)) + assert result["ambiguous"] == ["T4400"] + assert "T4400" not in [m["tag"] for m in result["matched"]] + assert result["pair_count"] == 10 # C(5, 2): the ambiguous tag is in no pair + assert all("T4400" not in row["tags"] for row in result["pairs"]) + assert result["verdict"] == "to_scale" + assert result["factor"] == pytest.approx(0.5, abs=EPS) + + +def test_two_plan_copies_on_the_reference_resolve_to_one_copy(): + # The reference holds the plan twice, 60 000 apart, each inside its own + # outline: over the whole drawing every tag is ambiguous, inside one copy + # every tag is unique. + reference = [ + outline("O1", -1000.0, -1000.0, 21000.0, 16000.0), + *tags(PLAN, "L"), + outline("O2", 59000.0, -1000.0, 81000.0, 16000.0), + *tags(PLAN, "R", dx=60000.0), + ] + whole = match_tags(snapshot(tags(PLAN, "A", 0.5)), snapshot(reference)) + assert whole["matched"] == {} + assert whole["ambiguous"] == sorted(PLAN) + result = scale_check(snapshot(tags(PLAN, "A", 0.5)), snapshot(reference)) + assert result["scope"] == {"cluster_a": None, "cluster_b": "C1"} + assert len(result["matched"]) == 6 + assert result["ambiguous"] == [] + assert result["verdict"] == "to_scale" + assert result["factor"] == pytest.approx(0.5, abs=EPS) + + +def test_cyrillic_look_alike_tags_match_their_latin_twins(): + # "Т4100" typed with a Cyrillic Т (U+0422) is the layout's Latin T4100. + cyrillic = {("Т" + tag[1:] if tag == "T4100" else tag): xy for tag, xy in PLAN.items()} + matched = match_tags(snapshot(tags(cyrillic, "A")), snapshot(tags(PLAN, "B"))) + assert "T4100" in matched["matched"] + assert len(matched["matched"]) == 6 + + +def test_fewer_than_three_matched_tags_is_insufficient(): + two = {k: PLAN[k] for k in ("T4100", "T4200")} + result = scale_check(snapshot(tags(two, "A")), snapshot(tags(PLAN, "B"))) + assert result["verdict"] == "insufficient" + assert result["factor"] is None + assert result["pair_count"] == 1 + + +def test_pairs_closer_than_the_floor_on_the_reference_are_left_out(): + # With a 13 m floor, the pairs of PLAN at least 13 000 apart on the + # reference are T4100-T4300 (15 000), T4100-T4600 (25 000), T4200-T4600 + # (hypot(8000, 15000) = 17 000), T4400-T4600 (hypot(20000, 6000) = 20 881), + # T4500-T4600 (hypot(14000, 11000) = 17 804), T4200-T4400 (15 000): six. + result = scale_check( + snapshot(tags(PLAN, "A", 0.5)), snapshot(tags(PLAN, "B")), min_pair_mm=13000.0 + ) + assert result["pair_count"] == 6 + assert result["pairs_below_floor"] == 9 + assert result["verdict"] == "to_scale" + + +@pytest.mark.parametrize("floor", [0.0, -5.0, float("nan"), float("inf")]) +def test_a_floor_that_is_not_a_positive_number_is_refused(floor): + with pytest.raises(ValueError, match="min_pair_mm: must be a positive number"): + scale_check(snapshot(tags(PLAN, "A")), snapshot(tags(PLAN, "B")), min_pair_mm=floor) + + +def test_an_unknown_cluster_is_refused_with_the_ids_the_drawing_has(): + with pytest.raises(ValueError, match=r"cluster_b: no cluster 'C99'"): + match_tags(snapshot(tags(PLAN, "A")), snapshot(tags(PLAN, "B")), cluster_b="C99") + + +def test_the_synthetic_pid_is_schematic_against_its_layout(tmp_path): + truth = build_plant_pair(tmp_path) + result = scale_check(read_snapshot(truth["pid"]), read_snapshot(truth["layout"])) + assert result["verdict"] == truth["scale_verdict"] + assert len(result["matched"]) >= 3 + # The layout's two plan copies: the check must pick one of them. + assert result["scope"]["cluster_b"] is not None +``` + +- [ ] **Step 2: Run it to verify it fails** + +Run: `uv run --frozen pytest tests/test_understand_scale.py -q` +Expected: FAIL — collection error `ModuleNotFoundError: No module named 'engineering.understand.scale'`. + +- [ ] **Step 3: Write `engineering/understand/scale.py`** + +The ±10 % band, the 80 % / 50 % shares, the three-tag minimum and the 2 m floor are spec §6's numbers, not standards. The floor is measured on the reference drawing only: a P&ID drawn at sheet scale would otherwise lose every pair, and the floor exists to absorb label offsets on the drawing whose distances are taken as physical. + +```python +"""drawing_scale_check: is this drawing to scale against that one? + +A P&ID and a layout of one plant carry the same equipment tags. If the P&ID +were drawn to scale, the distance between two tanks on it would be the distance +on the layout times one factor, for every pair. On a schematic it is not - and +a pipe length measured on a schematic is a length of drawn line, not of pipe. + +The check, in the order it runs: + +1. **Scope.** Model-space tags are read (`describe.tag_occurrences`) and the + drawings split into clusters (`clusters.find_clusters`, outliers left out). + Every pair of scopes - the whole drawing or one of its eight largest + clusters, on each side - is tried, and the pair that matches the most tags + wins (ties: the whole drawing first, then the larger cluster). Two copies + of one plan side by side therefore resolve to one copy instead of making + every tag ambiguous. +2. **Match.** Only a tag found exactly once in each scope is matched. A tag + found more than once in either (a plan copy, a legend, a second sheet) is + ``ambiguous`` and takes no part: picking one of its positions would be a + guess. +3. **Pairs.** For every two matched tags whose distance on the reference + drawing ``b`` is at least ``min_pair_mm``, the ratio of their distance on + ``a`` to that on ``b`` is taken, both converted to millimetres by each + drawing's inferred unit (`units.infer_units`). +4. **Statistics.** The median ratio, the quartiles and their range, and the + share of pairs within +/-10 % of the median - overall, and per room of the + reference drawing (the labelled faces `describe.find_rooms` finds) or, for + tags in no room, per cluster. +5. **Verdict.** ``to_scale`` when at least 80 % of the pairs are within the + band (``factor`` is then the median: a drawing at a different but uniform + scale is recognised); ``schematic`` at 50 % or less; ``partly`` in between; + ``insufficient`` with fewer than three matched tags or no pair above the + floor. + +A tag label is not an equipment centre: the band and the floor are there to +absorb the offset between the two, and the result says so. Pure; neither +drawing is touched. +""" + +from __future__ import annotations + +import math +import statistics +from itertools import combinations + +from engineering.arch.faces import face_containing +from engineering.understand.describe import FACE_TOL_FRACTION, find_rooms, prepare, tag_occurrences +from engineering.understand.snapshot import Snapshot +from engineering.understand.units import infer_units + +__all__ = ["MIN_TAGS", "SCHEMATIC_SHARE", "TO_SCALE_SHARE", "WITHIN", "match_tags", "scale_check"] + +#: The band around the median a pair must fall in to count as consistent. +WITHIN = 0.10 +TO_SCALE_SHARE = 0.80 +SCHEMATIC_SHARE = 0.50 +MIN_TAGS = 3 +#: Clusters per drawing tried as a scope, largest first. +MAX_SCOPE_CLUSTERS = 8 +#: Pair rows listed in the result, farthest from the median first. +PAIR_ROW_CAP = 200 +NOTE = ( + "Tag labels are not equipment centres; the +/-10 % band and the min_pair_mm floor absorb " + "the offset between a label and what it labels." +) + + +def _side(snap: Snapshot) -> dict: + prep = prepare(snap) + prep["tags"] = [o for o in tag_occurrences(snap, prep=prep) if o["space"] == "Model"] + return prep + + +def _scoped(side: dict, cluster: str | None, name: str) -> list[dict]: + if cluster is None: + return side["tags"] + ids = [c["id"] for c in side["clusters"]] + if cluster not in ids: + raise ValueError(f"{name}: no cluster {cluster!r}; the drawing has {ids}") + return [o for o in side["tags"] if o["cluster"] == cluster] + + +def _match(side_a: dict, side_b: dict, cluster_a: str | None, cluster_b: str | None) -> dict: + occ_a = _scoped(side_a, cluster_a, "cluster_a") + occ_b = _scoped(side_b, cluster_b, "cluster_b") + count_a: dict[str, list[dict]] = {} + count_b: dict[str, list[dict]] = {} + for o in occ_a: + count_a.setdefault(o["tag"], []).append(o) + for o in occ_b: + count_b.setdefault(o["tag"], []).append(o) + matched = { + tag: (tuple(count_a[tag][0]["at"]), tuple(count_b[tag][0]["at"])) + for tag in sorted(set(count_a) & set(count_b)) + if len(count_a[tag]) == 1 and len(count_b[tag]) == 1 + } + ambiguous = sorted( + tag + for tag in set(count_a) | set(count_b) + if len(count_a.get(tag, ())) > 1 or len(count_b.get(tag, ())) > 1 + ) + return { + "matched": matched, + "ambiguous": ambiguous, + "occurrences": { + tag: {"a": len(count_a.get(tag, ())), "b": len(count_b.get(tag, ()))} + for tag in ambiguous + }, + "only_a": sorted(t for t in count_a if t not in count_b and len(count_a[t]) == 1), + "only_b": sorted(t for t in count_b if t not in count_a and len(count_b[t]) == 1), + "cluster_a": cluster_a, + "cluster_b": cluster_b, + } + + +def match_tags(a: Snapshot, b: Snapshot, *, cluster_a=None, cluster_b=None) -> dict: + """``{"matched": {tag: (pa, pb)}, "ambiguous": [tag, ...], ...}`` within the + given scopes (a cluster id such as ``"C1"``, or None for the whole model + space). ``pa`` / ``pb`` are the tag's positions in drawing units. Also + ``occurrences`` (per ambiguous tag, how often each side has it), + ``only_a`` / ``only_b`` and the scopes used. An unknown cluster id is + refused with the ids the drawing has.""" + return _match(_side(a), _side(b), cluster_a, cluster_b) + + +def _stats(ratios: list[float]) -> dict: + if not ratios: + return {"median": None, "q1": None, "q3": None, "iqr": None, "within_10pct": None} + median = statistics.median(ratios) + if len(ratios) >= 2: + q1, _q2, q3 = statistics.quantiles(ratios, n=4, method="inclusive") + else: + q1 = q3 = ratios[0] + within = sum(1 for r in ratios if abs(r - median) <= WITHIN * median) / len(ratios) + return {"median": median, "q1": q1, "q3": q3, "iqr": q3 - q1, "within_10pct": within} + + +def _verdict(tag_count: int, pair_count: int, stats: dict) -> str: + if tag_count < MIN_TAGS or pair_count == 0 or not stats["median"] or stats["median"] <= 0: + return "insufficient" + if stats["within_10pct"] >= TO_SCALE_SHARE: + return "to_scale" + if stats["within_10pct"] <= SCHEMATIC_SHARE: + return "schematic" + return "partly" + + +def _summary(tags: list[str], rows: list[dict]) -> dict: + stats = _stats([r["ratio"] for r in rows]) + verdict = _verdict(len(tags), len(rows), stats) + return { + "verdict": verdict, + "factor": stats["median"] if verdict == "to_scale" else None, + **stats, + "pair_count": len(rows), + } + + +def scale_check(a: Snapshot, b: Snapshot, *, min_pair_mm=2000.0) -> dict: + """Is drawing ``a`` to scale against the reference drawing ``b``? + + Returns ``{"verdict", "factor", "median", "q1", "q3", "iqr", "within_10pct", + "pair_count", "pairs", "pairs_truncated", "pairs_below_floor", "matched", + "ambiguous", "occurrences", "only_a", "only_b", "groups", "scope", "units", + "min_pair_mm", "thresholds", "notes"}``. ``pairs`` lists at most 200 rows, + farthest from the median first; ``matched`` is ``[{"tag", "a": [x, y], + "b": [x, y]}]`` in drawing units. Refused before anything is computed: a + ``min_pair_mm`` that is not a positive finite number. + """ + try: + floor = float(min_pair_mm) + except (TypeError, ValueError): + raise ValueError(f"min_pair_mm: expected a number, got {min_pair_mm!r}") from None + if not math.isfinite(floor) or floor <= 0.0: + raise ValueError(f"min_pair_mm: must be a positive number of millimetres, got {floor:g}") + side_a, side_b = _side(a), _side(b) + + best = None + for ca in [None] + [c["id"] for c in side_a["clusters"][:MAX_SCOPE_CLUSTERS]]: + for cb in [None] + [c["id"] for c in side_b["clusters"][:MAX_SCOPE_CLUSTERS]]: + m = _match(side_a, side_b, ca, cb) + if best is None or len(m["matched"]) > len(best["matched"]): + best = m + + tol_b = side_b["diag"] * FACE_TOL_FRACTION if side_b["diag"] > 0 else 1.0 + rooms_b = find_rooms(side_b["kept"], tol=tol_b, membership=side_b["membership"]) + units = {} + notes = [NOTE] + for key, snap, side, rooms in ( + ("a", a, side_a, None), + ("b", b, side_b, rooms_b), + ): + areas = [row["face"]["area"] for row, _f in rooms["labelled"]] if rooms else [] + u = infer_units(snap, records=side["kept"], room_areas=areas) + units[key] = { + "unit": u["inferred"], + "mm_per_unit": u["mm_per_unit"], + "declared": u["declared"], + } + if u["mm_per_unit"] is None: + notes.append( + f"drawing {key}: no unit could be settled; its drawing units are taken as mm" + ) + fa = units["a"]["mm_per_unit"] or 1.0 + fb = units["b"]["mm_per_unit"] or 1.0 + + matched = best["matched"] + tags = sorted(matched) + rows: list[dict] = [] + below = 0 + for ti, tj in combinations(tags, 2): + d_b = math.dist(matched[ti][1], matched[tj][1]) * fb + if d_b < floor: + below += 1 + continue + d_a = math.dist(matched[ti][0], matched[tj][0]) * fa + rows.append({"tags": [ti, tj], "a_mm": d_a, "b_mm": d_b, "ratio": d_a / d_b}) + overall = _summary(tags, rows) + + faces = [face for _row, face in rooms_b["labelled"]] + room_of = {id(face): row for row, face in rooms_b["labelled"]} + groups: dict[str, dict] = {} + for tag in tags: + pb = matched[tag][1] + face = face_containing(faces, pb) if faces else None + if face is not None: + row = room_of[id(face)] + key = " ".join(p for p in ("room", row["number"], row["name"]) if p) + kind = "room" + else: + cluster = next( + (o["cluster"] for o in side_b["tags"] if o["tag"] == tag and tuple(o["at"]) == pb), + None, + ) + key, kind = (cluster or "unclustered"), "cluster" + groups.setdefault(key, {"group": key, "kind": kind, "tags": []})["tags"].append(tag) + group_rows = [] + for key in sorted(groups): + g = groups[key] + if len(g["tags"]) < 2: + continue + members = set(g["tags"]) + inside = [r for r in rows if r["tags"][0] in members and r["tags"][1] in members] + group_rows.append({**g, **_summary(g["tags"], inside)}) + + median = overall["median"] + listed = sorted( + rows, + key=lambda r: (-(abs(r["ratio"] - median) if median else 0.0), r["tags"]), + )[:PAIR_ROW_CAP] + return { + **overall, + "pairs": listed, + "pairs_truncated": len(rows) > PAIR_ROW_CAP, + "pairs_below_floor": below, + "matched": [{"tag": t, "a": list(matched[t][0]), "b": list(matched[t][1])} for t in tags], + "ambiguous": best["ambiguous"], + "occurrences": best["occurrences"], + "only_a": best["only_a"], + "only_b": best["only_b"], + "groups": group_rows, + "scope": {"cluster_a": best["cluster_a"], "cluster_b": best["cluster_b"]}, + "units": units, + "min_pair_mm": floor, + "thresholds": { + "within": WITHIN, + "to_scale_share": TO_SCALE_SHARE, + "schematic_share": SCHEMATIC_SHARE, + "min_tags": MIN_TAGS, + }, + "notes": notes, + } +``` + +- [ ] **Step 4: Run the scale test to verify it passes** + +Run: `uv run --frozen pytest tests/test_understand_scale.py -q` +Expected: `13 passed`. + +- [ ] **Step 5: Write the failing wire test** + +Create `tests/test_understand_scale_tools.py`: + +```python +"""drawing_scale_check over the MCP wire, on a fresh headless document.""" + +from __future__ import annotations + +import ezdxf +import pytest +import pytest_asyncio +from fastmcp import Client + +import server +from tests.fixtures.plant_pair import build_plant_pair + +pytestmark = pytest.mark.asyncio + +#: Six tags of one plant in millimetres; the closest pair is 7 211 apart. +PLAN = { + "T4100": (0.0, 0.0), + "T4200": (12000.0, 0.0), + "T4300": (12000.0, 9000.0), + "T4400": (0.0, 9000.0), + "T4500": (6000.0, 4000.0), + "T4600": (20000.0, 15000.0), +} + + +@pytest_asyncio.fixture +async def client(monkeypatch): + """In-memory client on a fresh headless document - never the operator's AutoCAD.""" + monkeypatch.setenv("AUTOCAD_MCP_BACKEND", "ezdxf") + async with Client(server.mcp) as connected: + status = (await connected.call_tool("system_status", {})).structured_content or {} + assert status.get("backend") == "ezdxf", "these tests must never touch live AutoCAD" + await connected.call_tool("drawing_new", {}) + yield connected + + +def reference_file(tmp_path): + """The plan's tags at full size in a millimetre DXF (INSUNITS 4).""" + doc = ezdxf.new(dxfversion="R2010", units=4) + msp = doc.modelspace() + for tag, (x, y) in PLAN.items(): + msp.add_text(tag, height=250.0, dxfattribs={"insert": (x, y), "layer": "TAGS"}) + path = tmp_path / "reference.dxf" + doc.saveas(path) + return str(path) + + +async def test_the_synthetic_pid_is_schematic_against_its_layout(client, tmp_path): + truth = build_plant_pair(tmp_path) + result = await client.call_tool( + "drawing_scale_check", {"path": truth["pid"], "reference": truth["layout"]} + ) + assert result.structured_content["verdict"] == truth["scale_verdict"] + + +async def test_the_current_document_at_half_scale_is_to_scale_with_factor_one_half( + client, tmp_path +): + # The same tags at half the coordinates in the current document. Its + # header says metres (drawing_new keeps ezdxf's default INSUNITS 6), but + # 250-unit text and a 10 000-unit spread read as millimetres, and the + # check converts by the inferred unit: the factor stays 0.5. + for tag, (x, y) in PLAN.items(): + await client.call_tool( + "entity_create_text", + {"text": tag, "x": x * 0.5, "y": y * 0.5, "height": 250.0, "layer": "TAGS"}, + ) + before = (await client.call_tool("analysis_entity_stats", {})).structured_content + result = await client.call_tool("drawing_scale_check", {"reference": reference_file(tmp_path)}) + after = (await client.call_tool("analysis_entity_stats", {})).structured_content + assert after == before + report = result.structured_content + assert report["units"]["a"]["unit"] == report["units"]["b"]["unit"] == "mm" + assert report["verdict"] == "to_scale" + assert report["factor"] == pytest.approx(0.5, abs=1e-12) + assert report["pair_count"] == 15 + assert report["ambiguous"] == [] + + +async def test_a_missing_reference_is_refused_by_name(client, tmp_path): + result = await client.call_tool( + "drawing_scale_check", {"reference": str(tmp_path / "none.dxf")}, raise_on_error=False + ) + assert result.is_error is True + assert "reference: no such file" in result.content[0].text + + +async def test_a_non_positive_floor_is_refused(client, tmp_path): + result = await client.call_tool( + "drawing_scale_check", + {"reference": reference_file(tmp_path), "min_pair_mm": 0.0}, + raise_on_error=False, + ) + assert result.is_error is True + assert "min_pair_mm" in result.content[0].text + + +async def test_drawing_scale_check_is_advertised_read_only(client): + tools = {tool.name: tool for tool in await client.list_tools()} + assert tools["drawing_scale_check"].annotations.readOnlyHint is True +``` + +- [ ] **Step 6: Run it to verify it fails** + +Run: `uv run --frozen pytest tests/test_understand_scale_tools.py -q` +Expected: `5 failed` — the calls raise `fastmcp.exceptions.ToolError: Not found: Unknown tool: 'drawing_scale_check'` (the two refusal tests fail their message assertions on that text), and the annotation test raises `KeyError: 'drawing_scale_check'`. + +- [ ] **Step 7: Register `drawing_scale_check` and grow the banner** + +In `server.py`, change the SECTION 26 banner line + +```python +# ── SECTION 26: Understanding & QA (1 tool) ───────────────────────────────── +``` + +to + +```python +# ── SECTION 26: Understanding & QA (2 tools) ──────────────────────────────── +``` + +(both lines are 78 characters, the house width), then replace the end of `drawing_understand` and the RESOURCES banner + +```python + except (ValueError, OSError, RuntimeError, DXFError) as exc: + raise ToolError(str(exc)) from exc + return await asyncio.to_thread(describe, snap) + + +# --------------------------------------------------------------------------- +# ── RESOURCES ─────────────────────────────────────────────────────────────── +# --------------------------------------------------------------------------- +``` + +with + +```python + except (ValueError, OSError, RuntimeError, DXFError) as exc: + raise ToolError(str(exc)) from exc + return await asyncio.to_thread(describe, snap) + + +@cad_tool( + summary="Is this drawing to scale against a reference? Tag distances, ratios, a verdict.", + cost="read", +) +@mcp.tool( + annotations={"title": "Understand: Scale Check", "readOnlyHint": True}, + tags={"analysis", "query"}, +) +async def drawing_scale_check( + reference: Annotated[ + str, + Field( + description="The reference drawing of the same plant (a DXF, usually the layout): " + "distances on it are the ones the other drawing is measured against." + ), + ], + path: Annotated[ + str | None, + Field( + default=None, + description="The drawing to check (usually the P&ID); omitted: the current document.", + ), + ] = None, + min_pair_mm: Annotated[ + float, + Field( + default=2000.0, + gt=0, + description="Tag pairs closer than this on the reference (mm) are left out: a tag " + "label is not an equipment centre.", + ), + ] = 2000.0, + ctx: Context = None, +) -> dict: + """Is the drawing to scale against the reference - or a schematic whose lengths + are lengths of drawn line, not of pipe? Read-only on both drawings. + + Equipment tags are matched between the two (a Cyrillic look-alike such as + `Т4100` is the Latin `T4100`). Only a tag found exactly once in each is used; + a tag found more than once (plan copies, a legend, a second sheet) is listed + in `ambiguous` and takes no part. When a drawing holds several bodies of + content (two copies of one plan), the pair of scopes - whole drawing or one + cluster on each side - that matches the most tags is used and named in + `scope`. For every two matched tags at least `min_pair_mm` apart on the + reference, the ratio of their distances is taken (both in millimetres by + each drawing's inferred unit, see `drawing_understand`). The result gives + the matched tags, the median ratio, the quartiles and their range, the + share of pairs within +/-10 % of the median, the same per room of the + reference (or per cluster for tags in no room) in `groups`, and a verdict: + `to_scale` (at least 80 % within the band; `factor` is the median, so a + uniform other scale is recognised), `schematic` (50 % or less), `partly` + in between, `insufficient` with fewer than three matched tags or no pair + above the floor. Tag labels are not equipment centres; the band and the + floor absorb that offset, and the result's `notes` say so. + + Refused before anything is read: a `reference` or `path` that does not + exist, a DXF over `MAX_DXF_BYTES` (the refusal names the size and the + variable), a file the engine cannot parse (a DWG headlessly), and a + `min_pair_mm` that is not a positive finite number. + """ + import asyncio + + from ezdxf.lldxf.const import DXFError + + from engineering.understand.scale import scale_check + from engineering.understand.snapshot import take_snapshot + + ref = validate_path(reference) + if not ref.is_file(): + raise ToolError(f"reference: no such file {str(ref)!r}") + target = None + if path: + target = validate_path(path) + if not target.is_file(): + raise ToolError(f"path: no such file {str(target)!r}") + await ctx.info(f"drawing_scale_check: {target or 'current document'} against {ref}") + backend = _backend(ctx) + try: + checked = await take_snapshot(backend, str(target) if target else None) + against = await take_snapshot(backend, str(ref)) + return await asyncio.to_thread(scale_check, checked, against, min_pair_mm=min_pair_mm) + except (ValueError, OSError, RuntimeError, DXFError) as exc: + raise ToolError(str(exc)) from exc + + +# --------------------------------------------------------------------------- +# ── RESOURCES ─────────────────────────────────────────────────────────────── +# --------------------------------------------------------------------------- +``` + +`reference` comes first because it is required. Measured while writing this plan: a new headless drawing (`drawing_new`) keeps ezdxf's default `$INSUNITS` 6 (metres), so the wire test's current document declares metres over millimetre text — the check converts by the *inferred* unit, which is why its factor stays 0.5. + +- [ ] **Step 8: Add the alias record** + +In `discovery/aliases.py`, replace the end of `TOOL_ALIASES` (the close of Task 3's record) + +```python + "yabanci cizim", + ), + ), +} +``` + +with + +```python + "yabanci cizim", + ), + ), + "drawing_scale_check": ToolAliases( + acad=(), + synonyms=( + "scale check", + "is the p&id to scale", + "is this drawing to scale", + "schematic or to scale", + "unscaled p&id", + "tag distances against the layout", + "can i measure pipe on the p&id", + "ölçek", + "olcek", + "ölçek kontrolü", + "olcek kontrolu", + "ölçekli mi", + "olcekli mi", + "масштаб", + ), + ), +} +``` + +`compare`, `fark` and `DWGCOMPARE` are left to `drawing_diff` (group D), so no command is shared and `SHARED_ACAD_COMMANDS` is untouched. Measured in the same scratch copy: the discovery golden set stays green. + +- [ ] **Step 9: Run the wire test to verify it passes** + +Run: `uv run --frozen pytest tests/test_understand_scale_tools.py -q` +Expected: `5 passed`. + +- [ ] **Step 10: Regenerate the inventory and bump the documented counts** + +Run: `uv run --frozen python scripts/generate_tool_inventory.py` +Expected output: `wrote docs/tool-inventory.json: 243 tools, 8 resources, 5 prompts, 26 groups` + +In `README.md` change the release-snapshot lines from + +``` +> **v1.5 release snapshot:** 242 tools · 8 resources · 5 prompt templates · 4516 collected tests. +> 242 is the **registered** count; a default install advertises 237 over `tools/list`, +``` + +to + +``` +> **v1.5 release snapshot:** 243 tools · 8 resources · 5 prompt templates · 4516 collected tests. +> 243 is the **registered** count; a default install advertises 238 over `tools/list`, +``` + +and the caption from `242 tools in 26 groups;` to `243 tools in 26 groups;`. The collected-tests figure and the alias-corpus sentence stay as they are (Task 11). + +- [ ] **Step 11: Bump the frozen group-size snapshot** + +In `tests/test_tool_registry.py::test_tool_group_sizes_are_unchanged`: + +- `"""Frozen snapshot of the surface (242 tools, 26 groups).` becomes `"""Frozen snapshot of the surface (243 tools, 26 groups).`; +- `"analysis": 13,` becomes `"analysis": 14,`; +- `assert sum(sizes.values()) == 242` becomes `assert sum(sizes.values()) == 243`; +- replace the paragraph Task 3 added + +```python + Track H group U opened SECTION 26 (Understanding & QA) with + `drawing_understand`, tagged `analysis` and `query`: `analysis` outranks + `query` in `_GROUP_TAG_PRIORITY`, so the reader files under `analysis` + (12 -> 13) beside the other whole-drawing queries. This snapshot is + branch-local: the track H merge task recomputes it once the understand, + network and diff branches have landed. + """ +``` + +with + +```python + Track H group U opened SECTION 26 (Understanding & QA) with + `drawing_understand`, tagged `analysis` and `query`: `analysis` outranks + `query` in `_GROUP_TAG_PRIORITY`, so the reader files under `analysis` + (12 -> 13) beside the other whole-drawing queries, and + `drawing_scale_check`, tagged the same, followed it (13 -> 14). This + snapshot is branch-local: the track H merge task recomputes it once the + understand, network and diff branches have landed. + """ +``` + +- [ ] **Step 12: Gates** + +Run each and require it clean: + +``` +uv run --frozen ruff check engineering/understand/scale.py server.py discovery/aliases.py tests/test_understand_scale.py tests/test_understand_scale_tools.py tests/test_tool_registry.py +uv run --frozen ruff format engineering/understand/scale.py tests/test_understand_scale.py tests/test_understand_scale_tools.py +uv run --frozen ruff format --check server.py discovery/aliases.py tests/test_tool_registry.py +uv run --frozen python scripts/generate_tool_inventory.py --check +uv run --frozen pytest tests/test_understand_units.py tests/test_understand_clusters.py tests/test_understand_describe.py tests/test_understand_tools.py tests/test_understand_scale.py tests/test_understand_scale_tools.py -q +uv run --frozen pytest tests/test_tool_registry.py tests/test_tool_inventory.py tests/test_tool_packs.py tests/test_discovery_aliases.py tests/test_discovery_transform.py tests/test_capability_contract.py -q +uv run --frozen pytest tests/test_release_consistency.py -q --deselect tests/test_release_consistency.py::test_document_numbers_match_the_tree +uv run --frozen pytest -q --deselect tests/test_release_consistency.py::test_document_numbers_match_the_tree +``` + +`generate_tool_inventory.py --check` prints `ok: docs/tool-inventory.json matches the live surface`. The six understand files give `52 passed`. `test_section_header_counts_match_decorators` must pass with the banner at `(2 tools)`. The deselect is the known-red collected-tests drift — record it in the task report. No capability key is added. + +- [ ] **Step 13: Commit** + +``` +git add engineering/understand/scale.py server.py discovery/aliases.py docs/tool-inventory.json README.md tests/test_tool_registry.py tests/test_understand_scale.py tests/test_understand_scale_tools.py +git -c user.name="Umut CAN" -c user.email="umutcan.edizaslan@gmail.com" commit -m "$(cat <<'MSG' +feat(understand): drawing_scale_check - is this drawing to scale against that one + +Equipment tags found exactly once in each drawing are matched (a tag +found twice - a plan copy, a legend - is reported ambiguous and takes no +part; two plan copies resolve to the cluster that matches the most +tags). For every pair of matched tags at least min_pair_mm apart on the +reference, the ratio of their distances is taken in millimetres by each +drawing's inferred unit; the median, the quartiles, the share within ++/-10 % of the median, the same per room of the reference, and the +verdict: to_scale (>= 80 %, with its factor), schematic (<= 50 %), +partly, or insufficient under three matched tags. + +server.py grows SECTION 26 to two tools with drawing_scale_check (core, +read-only, tagged analysis/query); alias record, inventory, README +release snapshot 243, frozen group sizes (analysis 14). + +Co-Authored-By: Claude Opus 5.5 (1M context) +Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd +MSG +)" +``` + +--- + +### Task 5: The line-network engine — runs, services, diameters, equipment attachment + +**Files:** +- Create: `engineering/understand/network.py` +- Test: `tests/test_understand_network.py` + +**Interfaces:** +- Consumes (Task 1, on `main`): `engineering.understand.snapshot` — `Pt`, `EntityRecord` (fields exactly as the skeleton pins them; this module also relies on an `ARC` record carrying its centre as `points[0]` beside `radius` and `angles` — Task 1 fills CIRCLE/ARC that way; if it does not, fix `_arc_segment` to the verified field and record it), `Snapshot`, `read_snapshot(path) -> Snapshot`; `engineering.understand.vocab` — `classify_layer(name) -> dict`, `supply_return(text) -> str | None`; `engineering.understand.labels` — `parse_diameters(text) -> tuple[str, ...]`, `parse_tag(text) -> str | None`, `wiring_target(text) -> str | None`. (Task 2, on `main`): `tests.fixtures.plant_pair.build_plant_pair(directory) -> dict` and its truth keys `pid`, `layer_services`, `pipe_runs[{service, diameter, tags, layout_length_mm, status}]`. +- Produces (Tasks 6-8 rely on these exact names): `Run` (frozen dataclass `id: str; edges: tuple[dict, ...]; service: str; ends: tuple[Pt, ...]; tags: tuple[str, ...]` — the skeleton's shape); `build_network(snap: Snapshot, *, layers=None, tol=1.0, label_search=None) -> dict` returning `{"runs": [Run], "unassigned": [{run, edge, length, reason}], "attachments": {run_id: [{"at": Pt, "tag": str | None}]}, "equipment": {tag: [{"box", "handle", "at"}]}, "stats": {layers, tol, label_search, segments, pieces, overlap_removed, overlap_length, fittings, reducers, fittings_skipped, runs, labels_skipped, diameter_conflicts, service_conflicts}}` (the skeleton's three keys plus `attachments` and `equipment`, additive); each edge dict is `{id, a, b, bulge, length, layer, layers, handles, service, diameter, diameter_source: "direct"|"continuity"|"unassigned", label}`; `tag_occurrences(snap) -> dict[str, list[dict]]` (`{tag: [{tag, at, handle, box, insert}]}`); `equipment_boxes(snap, occurrences, *, exclude_layers=(), search) -> dict`; `default_layers(snap) -> tuple[str, ...]`; `label_search_default(snap, *, tol=1.0) -> float`; `is_reducer(block) -> bool`; constants `MODEL = "Model"`, `TEXT_TYPES`, `LINE_TYPES`, `REDUCER_WORDS`, `LABEL_SEARCH_FACTOR = 5.0`, `EQUIPMENT_SEARCH_SHARE = 2.0`, `MAX_FITTING_ENDS = 4`, `NETWORK_CONFIDENCE = 0.9`. + +Every rule this engine applies is spec §7 or a named heuristic of this module (a fitting holds two to four free ends; a shape holding two different tags is not equipment; a tag is looked for twice the label search away). No standards value is involved. Review Focus item 4 (double-drawn pipe) is owned here: `test_a_line_drawn_twice_is_counted_once` and `test_a_line_drawn_over_a_polyline_is_counted_once` below. + +- [ ] **Step 1: Write the failing test** + +Create `tests/test_understand_network.py`: + +```python +"""The line-network engine: runs, duplicates once, fittings, diameters, services, equipment. + +Every drawing below is built record by record, so each expected length is the +hand sum written beside it; nothing is read back from the implementation. The +last three tests read the synthetic plant pair of Task 2 and compare every run +with the generator's own truth. +""" + +from __future__ import annotations + +import pytest + +from engineering.understand.network import ( + Run, + build_network, + default_layers, + is_reducer, + label_search_default, +) +from engineering.understand.snapshot import EntityRecord, Snapshot, read_snapshot +from engineering.understand.vocab import classify_layer +from tests.fixtures.plant_pair import build_plant_pair + +EPS = 1e-9 +PIPE = "PRODUCT" + + +def rec(handle, kind, layer=PIPE, points=(), **extra): + return EntityRecord( + handle=handle, type=kind, layer=layer, space="Model", points=tuple(points), **extra + ) + + +def line(handle, a, b, layer=PIPE): + return rec(handle, "LINE", layer, (a, b)) + + +def text(handle, value, at, height=5.0, layer="TEXT"): + x, y = at + return rec( + handle, + "TEXT", + layer, + (at,), + text=value, + height=height, + bbox=(x, y, x + height * len(value) * 0.6, y + height), + ) + + +def tank(handle, tag, box, layer="EQUIPMENT"): + """A closed rectangle with its tag written inside it.""" + x0, y0, x1, y1 = box + outline = rec( + handle, "LWPOLYLINE", layer, ((x0, y0), (x1, y0), (x1, y1), (x0, y1)), closed=True, bbox=box + ) + return [outline, text(handle + "T", tag, ((x0 + x1) / 2.0 - 10.0, (y0 + y1) / 2.0))] + + +def insert(handle, block, box, layer=PIPE): + return rec( + handle, + "INSERT", + layer, + (((box[0] + box[2]) / 2.0, (box[1] + box[3]) / 2.0),), + block=block, + bbox=box, + ) + + +def snap(*records, insunits=4): + flat = [] + for item in records: + flat.extend(item if isinstance(item, list) else [item]) + layers = {r.layer: {"color": 7, "linetype": "Continuous"} for r in flat} + return Snapshot( + source="test", + insunits=insunits, + extmin=None, + extmax=None, + layers=layers, + layouts=(), + records=tuple(flat), + ) + + +def total(run: Run) -> float: + return sum(edge["length"] for edge in run.edges) + + +def net(*records, **kw): + return build_network(snap(*records), layers=[PIPE], **kw) + + +# -- topology ------------------------------------------------------------------ + + +def test_ends_within_tol_merge_into_one_run(): + result = net(line("1", (0.0, 0.0), (1000.0, 0.0)), line("2", (1000.4, 0.0), (2000.0, 0.0))) + (run,) = result["runs"] + # the second line starts at the merged vertex (1000, 0): 1000 + 1000 + assert abs(total(run) - 2000.0) < EPS + assert run.ends == ((0.0, 0.0), (2000.0, 0.0)) + + +def test_a_branch_ending_on_a_header_is_a_t_junction(): + result = net(line("1", (0.0, 0.0), (2000.0, 0.0)), line("2", (1000.0, 0.0), (1000.0, 800.0))) + (run,) = result["runs"] + assert abs(total(run) - 2800.0) < EPS # 2000 header + 800 branch + assert len(run.edges) == 3 # header split at the tee + assert run.ends == ((0.0, 0.0), (1000.0, 800.0), (2000.0, 0.0)) + + +def test_two_lines_crossing_without_an_end_do_not_connect(): + result = net(line("1", (0.0, 0.0), (2000.0, 0.0)), line("2", (1000.0, -500.0), (1000.0, 500.0))) + assert len(result["runs"]) == 2 + + +# -- Review Focus 4: double-drawn pipe counts once ---------------------------- + + +def test_a_line_drawn_twice_is_counted_once(): + result = net(line("1", (0.0, 0.0), (2000.0, 0.0)), line("2", (0.0, 0.0), (2000.0, 0.0))) + (run,) = result["runs"] + assert abs(total(run) - 2000.0) < EPS # not 4000 + assert abs(result["stats"]["overlap_length"] - 2000.0) < EPS + assert run.edges[0]["handles"] == ("1", "2") + + +def test_a_line_drawn_over_a_polyline_is_counted_once(): + poly = rec("P", "LWPOLYLINE", PIPE, ((0.0, 0.0), (1000.0, 0.0), (3000.0, 0.0))) + result = net(line("L", (0.0, 0.0), (3000.0, 0.0)), poly) + (run,) = result["runs"] + # the line splits at the polyline's vertex (1000, 0): pieces 1000 + 2000, each once + assert abs(total(run) - 3000.0) < EPS + assert abs(result["stats"]["overlap_length"] - 3000.0) < EPS + assert all(edge["handles"] == ("L", "P") for edge in run.edges) + + +def test_a_partial_collinear_overlap_counts_the_shared_stretch_once(): + result = net(line("1", (0.0, 0.0), (2000.0, 0.0)), line("2", (1500.0, 0.0), (3000.0, 0.0))) + (run,) = result["runs"] + assert abs(total(run) - 3000.0) < EPS # 0..3000 once; 1500..2000 was drawn twice + assert abs(result["stats"]["overlap_length"] - 500.0) < EPS + + +def test_an_arc_keeps_its_true_length(): + # a quarter circle of radius 1000 centred on the origin, from 0 deg to 90 deg + arc = rec("A", "ARC", PIPE, ((0.0, 0.0),), radius=1000.0, angles=(0.0, 90.0)) + (run,) = net(arc)["runs"] + assert abs(total(run) - 500.0 * 3.141592653589793) < 1e-6 # 2*pi*1000/4 + ends = [c for point in run.ends for c in point] + assert ends == pytest.approx([0.0, 1000.0, 1000.0, 0.0], abs=1e-9) + + +# -- fittings ---------------------------------------------------------------- + + +def test_an_inline_valve_joins_the_two_lines_and_adds_no_length(): + result = net( + line("1", (0.0, 0.0), (1000.0, 0.0)), + line("2", (1100.0, 0.0), (2000.0, 0.0)), + insert("V", "GATE_VALVE", (1000.0, -50.0, 1100.0, 50.0)), + ) + (run,) = result["runs"] + assert abs(total(run) - 1900.0) < EPS # 1000 + 900; the valve body is not pipe + assert run.ends == ((0.0, 0.0), (2000.0, 0.0)) + assert result["stats"]["fittings"] == 1 + + +@pytest.mark.parametrize( + ("block", "reducer"), + [ + ("REDUCER_51_38", True), + ("ПЕРЕХОД_51", True), + ("REDÜKSİYON-51", True), + ("verloop 51x38", True), + ("GATE_VALVE", False), + (None, False), + ], +) +def test_reducer_words_in_four_languages(block, reducer): + assert is_reducer(block) is reducer + + +# -- diameters --------------------------------------------------------------- + + +def test_a_label_is_direct_and_carries_by_continuity_across_a_tee(): + result = net( + line("1", (0.0, 0.0), (1000.0, 0.0)), + line("2", (1000.0, 0.0), (2000.0, 0.0)), + line("3", (1000.0, 0.0), (1000.0, 800.0)), + text("D", "Ø51", (400.0, 5.0)), + ) + (run,) = result["runs"] + by_handle = {edge["handles"]: edge for edge in run.edges} + assert by_handle[("1",)]["diameter_source"] == "direct" + assert by_handle[("1",)]["label"] == "D" + for handle in (("2",), ("3",)): + assert by_handle[handle]["diameter"] == "Ø51" + assert by_handle[handle]["diameter_source"] == "continuity" + assert result["unassigned"] == [] + + +def test_a_label_holds_for_every_piece_of_its_drawn_entity(): + poly = rec("P", "LWPOLYLINE", PIPE, ((0.0, 0.0), (1000.0, 0.0), (1000.0, 1000.0))) + (run,) = net(poly, text("D", "DN20", (400.0, 5.0)))["runs"] + assert [edge["diameter_source"] for edge in run.edges] == ["direct", "direct"] + assert {edge["diameter"] for edge in run.edges} == {"DN20"} + + +def test_continuity_stops_at_a_reducer(): + result = net( + line("1", (0.0, 0.0), (1000.0, 0.0)), + line("2", (1060.0, 0.0), (2000.0, 0.0)), + insert("R", "REDUCER_51_38", (1000.0, -30.0, 1060.0, 30.0)), + text("D", "Ø51", (400.0, 5.0)), + ) + (run,) = result["runs"] + by_handle = {edge["handles"]: edge for edge in run.edges} + assert by_handle[("1",)]["diameter"] == "Ø51" + assert by_handle[("2",)]["diameter"] is None + assert by_handle[("2",)]["diameter_source"] == "unassigned" + assert result["unassigned"][0]["reason"].startswith("no diameter label") + assert result["stats"]["reducers"] == 1 + + +def test_continuity_crosses_a_valve(): + result = net( + line("1", (0.0, 0.0), (1000.0, 0.0)), + line("2", (1100.0, 0.0), (2000.0, 0.0)), + insert("V", "GATE_VALVE", (1000.0, -50.0, 1100.0, 50.0)), + text("D", "Ø51", (400.0, 5.0)), + ) + edge = next(e for e in result["runs"][0].edges if e["handles"] == ("2",)) + assert (edge["diameter"], edge["diameter_source"]) == ("Ø51", "continuity") + + +def test_a_conflicting_label_stops_continuity_and_the_nearer_one_wins(): + result = net( + line("1", (0.0, 0.0), (1000.0, 0.0)), + line("2", (1000.0, 0.0), (2000.0, 0.0)), + line("3", (2000.0, 0.0), (3000.0, 0.0)), + text("A", "Ø51", (400.0, 5.0)), + text("B", "DN20", (1400.0, 5.0)), + ) + by_handle = {edge["handles"]: edge for edge in result["runs"][0].edges} + assert by_handle[("1",)]["diameter"] == "Ø51" + assert by_handle[("2",)]["diameter"] == "DN20" + # piece 3 is next to piece 2 (DN20), one piece further from piece 1 (Ø51) + assert (by_handle[("3",)]["diameter"], by_handle[("3",)]["diameter_source"]) == ( + "DN20", + "continuity", + ) + + +def test_two_diameters_equally_near_leave_the_piece_unassigned(): + result = net( + line("1", (0.0, 0.0), (1000.0, 0.0)), + line("2", (1000.0, 0.0), (2000.0, 0.0)), + line("3", (2000.0, 0.0), (3000.0, 0.0)), + text("A", "Ø51", (400.0, 5.0)), + text("B", "DN20", (2400.0, 5.0)), + ) + middle = next(e for e in result["runs"][0].edges if e["handles"] == ("2",)) + assert middle["diameter_source"] == "unassigned" + assert result["unassigned"][0]["reason"] == "two different diameters equally near" + + +def test_labels_are_kept_exactly_as_drawn(): + result = net(line("1", (0.0, 0.0), (1000.0, 0.0)), text("D", "SMS51", (400.0, 5.0))) + assert result["runs"][0].edges[0]["diameter"] == "SMS51" + + +def test_a_label_beyond_the_search_distance_is_not_read(): + # median text height 5 -> label_search 25; the label's centre stands 200 away + result = net(line("1", (0.0, 0.0), (1000.0, 0.0)), text("D", "Ø51", (400.0, 200.0))) + assert result["stats"]["label_search"] == 25.0 + assert result["runs"][0].edges[0]["diameter_source"] == "unassigned" + + +# -- services ---------------------------------------------------------------- + + +def test_a_return_word_overrides_the_layer_for_the_whole_run(): + layer = "CIP_LINES" + base = classify_layer(layer)["service"] + assert base in ("cip_supply", "cip_return"), "the vocabulary must file CIP_LINES as CIP" + records = [ + line("1", (0.0, 0.0), (1000.0, 0.0), layer), + line("2", (1000.0, 0.0), (1000.0, 1000.0), layer), + text("W", "ВОЗВРАТ", (400.0, 5.0)), + ] + result = build_network(snap(*records), layers=[layer]) + (run,) = result["runs"] + assert run.service == "cip_return" + assert {edge["service"] for edge in run.edges} == {"cip_return"} + + +def test_without_a_word_the_run_keeps_its_layer_service(): + (run,) = net(line("1", (0.0, 0.0), (1000.0, 0.0)))["runs"] + assert run.service == classify_layer(PIPE)["service"] + + +# -- equipment --------------------------------------------------------------- + + +def test_run_ends_attach_to_the_tags_whose_geometry_holds_them(): + result = net( + tank("A", "T4100", (-500.0, -500.0, 0.0, 500.0)), + tank("B", "T4101", (2000.0, -500.0, 2500.0, 500.0)), + line("1", (0.0, 0.0), (2000.0, 0.0)), + ) + (run,) = result["runs"] + assert run.tags == ("T4100", "T4101") + assert result["attachments"]["R1"] == [ + {"at": (0.0, 0.0), "tag": "T4100"}, + {"at": (2000.0, 0.0), "tag": "T4101"}, + ] + + +def test_a_tagged_insert_is_equipment_not_a_fitting(): + pump = rec( + "PU", + "INSERT", + "EQUIPMENT", + ((1050.0, 0.0),), + block="*U12", + attribs=(("TAG", "M82"),), + bbox=(1000.0, -50.0, 1100.0, 50.0), + ) + result = net( + line("1", (0.0, 0.0), (1000.0, 0.0)), line("2", (1100.0, 0.0), (2000.0, 0.0)), pump + ) + assert len(result["runs"]) == 2 + assert all("M82" in run.tags for run in result["runs"]) + assert result["stats"]["fittings"] == 0 + + +def test_a_room_outline_around_two_tags_is_not_equipment(): + room = rec( + "ROOM", + "LWPOLYLINE", + "ROOMS", + ((-5000.0, -5000.0), (9000.0, -5000.0), (9000.0, 5000.0), (-5000.0, 5000.0)), + closed=True, + bbox=(-5000.0, -5000.0, 9000.0, 5000.0), + ) + outline = rec( + "A", + "LWPOLYLINE", + "EQUIPMENT", + ((-500.0, -500.0), (0.0, -500.0), (0.0, 500.0), (-500.0, 500.0)), + closed=True, + bbox=(-500.0, -500.0, 0.0, 500.0), + ) + # T4100 is written 30 above its tank, inside the room: the room holds T4100 + # and T4101, so it is not equipment, and the tank is the nearest shape + # within 2 x label_search = 50 + result = net( + room, + outline, + text("AT", "T4100", (-400.0, 530.0)), + tank("B", "T4101", (2000.0, -500.0, 2500.0, 500.0)), + line("1", (0.0, 0.0), (2000.0, 0.0)), + ) + assert result["equipment"]["T4100"][0]["handle"] == "A" + assert result["runs"][0].tags == ("T4100", "T4101") + + +def test_a_free_end_attaches_to_nothing(): + result = net( + tank("A", "T4100", (-500.0, -500.0, 0.0, 500.0)), line("1", (0.0, 0.0), (2000.0, 0.0)) + ) + assert result["attachments"]["R1"][1] == {"at": (2000.0, 0.0), "tag": None} + + +# -- layer choice and refusals ------------------------------------------------- + + +def test_default_layers_are_the_confidently_classified_network_layers(): + drawing = snap(line("1", (0.0, 0.0), (1.0, 0.0)), line("2", (0.0, 0.0), (1.0, 0.0), "MISC")) + expected = tuple( + name + for name in ("MISC", PIPE) + if classify_layer(name)["discipline"] in ("piping", "electrical") + and classify_layer(name)["confidence"] >= 0.9 + ) + assert default_layers(drawing) == expected + + +def test_label_search_defaults_to_five_median_text_heights(): + drawing = snap( + text("a", "x", (0, 0), 2.0), text("b", "y", (0, 0), 4.0), text("c", "z", (0, 0), 9.0) + ) + assert label_search_default(drawing) == 20.0 # 5 x median(2, 4, 9) + + +@pytest.mark.parametrize( + ("kwargs", "message"), + [ + ({"tol": 0.0}, "tol: must be a finite number greater than zero"), + ({"tol": float("nan")}, "tol: must be a finite number greater than zero"), + ({"label_search": -1.0}, "label_search: must be a finite number greater than zero"), + ({"layers": ["NOPE"]}, "layers: NOPE not in the drawing; it has: PRODUCT"), + ({"layers": []}, "layers: an empty list reads nothing"), + ], +) +def test_refusals_name_the_parameter(kwargs, message): + drawing = snap(line("1", (0.0, 0.0), (1.0, 0.0))) + kwargs = {"layers": [PIPE], **kwargs} + with pytest.raises(ValueError, match="^" + message.replace("(", r"\(")): + build_network(drawing, **kwargs) + + +# -- the synthetic plant pair (Task 2) ------------------------------------------- + + +@pytest.fixture(scope="module") +def plant(tmp_path_factory): + truth = build_plant_pair(tmp_path_factory.mktemp("plant")) + return truth, read_snapshot(truth["pid"]) + + +def run_key(tags, service): + return (tuple(sorted(tags)), service) + + +def test_the_generator_names_each_run_by_its_tags_and_service(plant): + truth, _pid = plant + keys = [run_key(run["tags"], run["service"]) for run in truth["pipe_runs"]] + assert len(keys) == len(set(keys)) + + +def test_every_synthetic_run_has_its_service_diameter_and_tags(plant): + truth, pid = plant + found: dict = {} + for run in build_network(pid)["runs"]: + found.setdefault(run_key(run.tags, run.service), []).append(run) + for expected in truth["pipe_runs"]: + key = run_key(expected["tags"], expected["service"]) + runs = found.get(key, []) + assert len(runs) == 1, f"{key}: {len(runs)} runs carry these tags and this service" + if expected["status"] == "C": + continue + edges = runs[0].edges + assert expected["diameter"] in {edge["diameter"] for edge in edges}, key + sources = {edge["diameter_source"] for edge in edges} + if expected["status"] == "A": + assert sources == {"direct"}, key + else: + assert sources & {"continuity", "unassigned"}, key + + +def test_the_synthetic_service_layers_are_read_by_default(plant): + truth, pid = plant + layers = set(build_network(pid)["stats"]["layers"]) + service_layers = { + layer + for layer, service in truth["layer_services"].items() + if service not in ("electrical", "unknown") + } + assert service_layers <= layers +``` + +- [ ] **Step 2: Run it to verify it fails** + +Run: `uv run --frozen pytest tests/test_understand_network.py -q` +Expected: FAIL — collection error `ModuleNotFoundError: No module named 'engineering.understand.network'`. + +- [ ] **Step 3: Write the engine** + +Create `engineering/understand/network.py`: + +```python +"""Line networks read off a foreign P&ID: runs, services, diameters, equipment. + +`engineering.pid.graph` reads a P&ID this server drew; on a foreign one it sees +lines of class `unknown` between blocks of kind `unknown_block` (spec §1). This +module is the reader the takeoffs stand on (spec §7). It works on a +`Snapshot` - one bulk read - and never touches a drawing. + +The algorithm, in the order it runs: + +1. **Segments.** LINE, LWPOLYLINE, POLYLINE and ARC records in model space on + the chosen layers (default: every layer `classify_layer` files under piping + or electrical with confidence >= 0.9). A bulged polyline segment and an ARC + stay one arc piece: never split, never flattened. +2. **Merge.** Endpoints closer than `tol` become one vertex; the first one seen + keeps its coordinates. +3. **Split.** A straight segment is split at every vertex lying on it within + `tol` - the T-junction, where a branch ends on a header. A crossing where + neither line ends is *not* a junction: two P&ID lines that cross without a + dot do not connect. +4. **Duplicates once.** The same split turns a line drawn twice, or a line + drawn over a polyline, into identical pieces; each piece is kept once and + the length the copies would have added is `stats.overlap_length`. +5. **Fittings.** An INSERT whose box (grown by `tol`) holds two to four free + ends is an inline fitting - valve, coupling, reducer - and those ends are + joined through it; the fitting adds no length. An INSERT that a tag labels + is equipment and never a fitting. A block whose name contains reducer / + переход / redüksiyon / verloop is a reducer. +6. **Runs.** A connected component of pieces and fittings is a run. +7. **Diameter.** Each diameter label goes to the piece nearest to it within + `label_search` (default 5 x the median text height) and holds for every + piece of that drawn entity (*direct*; when two labels reach one piece, the + nearer wins and the pair is listed in `stats.diameter_conflicts`). An + unlabelled piece takes the diameter of the nearest direct piece along the + run, measured along the pipe and never across a reducer (*continuity*); two + different diameters equally near, or none, leave it *unassigned*. Labels are + kept as drawn: Ø51, SMS51 and DN20 are different sizes. +8. **Service.** A run's service is the length-weighted service of its layers, + overridden for the whole run by a supply / return word within + `label_search` of one of its pieces - a layer alone never splits CIP supply + from CIP return. +9. **Equipment.** A run end inside the box (grown by `tol`) of the geometry a + tag labels is attached to that tag; the smallest such box wins. A tag labels + the smallest closed shape holding it, else the nearest within twice + `label_search`; a shape holding two different tags (a room, a skid, a sheet + frame) is never equipment. + +Arc pieces are measured to their chord for the label search only; their length +is the true arc length. +""" + +from __future__ import annotations + +import heapq +import math +import statistics +from bisect import bisect_left, bisect_right +from dataclasses import dataclass + +from engineering.understand.labels import parse_diameters, parse_tag, wiring_target +from engineering.understand.snapshot import EntityRecord, Pt, Snapshot +from engineering.understand.vocab import classify_layer, supply_return + +__all__ = [ + "EQUIPMENT_SEARCH_SHARE", + "LABEL_SEARCH_FACTOR", + "LINE_TYPES", + "MAX_FITTING_ENDS", + "NETWORK_CONFIDENCE", + "REDUCER_WORDS", + "Run", + "build_network", + "default_layers", + "equipment_boxes", + "is_reducer", + "label_search_default", + "tag_occurrences", +] + +MODEL = "Model" +LINE_TYPES = frozenset({"LINE", "LWPOLYLINE", "POLYLINE", "ARC"}) +TEXT_TYPES = frozenset({"TEXT", "MTEXT", "MULTILEADER"}) +#: Words that make an INSERT a reducer, matched in its casefolded block name. +REDUCER_WORDS = ("reducer", "переход", "redüksiyon", "verloop") +NETWORK_DISCIPLINES = ("piping", "electrical") +NETWORK_CONFIDENCE = 0.9 +#: A box touched by more free ends than this is not an inline fitting (a +#: valve has two ports, a three- or four-way valve three or four). +MAX_FITTING_ENDS = 4 +LABEL_SEARCH_FACTOR = 5.0 +#: A tag written beside its symbol is looked for twice as far as a label. +EQUIPMENT_SEARCH_SHARE = 2.0 +#: With no text on the drawing at all, labels are searched this many `tol` away. +LABEL_SEARCH_FALLBACK = 50.0 +_PAIRS = {"cip": ("cip_supply", "cip_return"), "glycol": ("glycol_supply", "glycol_return")} +_EPS = 1e-9 + + +@dataclass(frozen=True) +class Run: + """One connected line network: its pieces, its service, its free ends and the tags on them.""" + + id: str + edges: tuple[dict, ...] + service: str + ends: tuple[Pt, ...] + tags: tuple[str, ...] + + +def _positive(value, name: str) -> float: + try: + number = float(value) + except (TypeError, ValueError): + raise ValueError(f"{name}: expected a number, got {value!r}") from None + if not math.isfinite(number) or number <= 0.0: + raise ValueError(f"{name}: must be a finite number greater than zero, got {value!r}") + return number + + +def _in_box(p: Pt, box, grow: float) -> bool: + return box[0] - grow <= p[0] <= box[2] + grow and box[1] - grow <= p[1] <= box[3] + grow + + +def _box_area(box) -> float: + return (box[2] - box[0]) * (box[3] - box[1]) + + +def _box_distance(p: Pt, box) -> float: + dx = max(box[0] - p[0], 0.0, p[0] - box[2]) + dy = max(box[1] - p[1], 0.0, p[1] - box[3]) + return math.hypot(dx, dy) + + +def _centre(rec: EntityRecord) -> Pt | None: + """Where a label stands: its box centre, else its insertion point.""" + if rec.bbox is not None: + return ((rec.bbox[0] + rec.bbox[2]) / 2.0, (rec.bbox[1] + rec.bbox[3]) / 2.0) + return rec.points[0] if rec.points else None + + +def _segment_distance(p: Pt, a: Pt, b: Pt) -> float: + dx, dy = b[0] - a[0], b[1] - a[1] + length2 = dx * dx + dy * dy + if length2 == 0.0: + return math.dist(p, a) + t = max(0.0, min(1.0, ((p[0] - a[0]) * dx + (p[1] - a[1]) * dy) / length2)) + return math.dist(p, (a[0] + t * dx, a[1] + t * dy)) + + +def _along(p: Pt, a: Pt, b: Pt) -> tuple[float, float]: + """(distance along a->b of p's foot, distance of p from the line).""" + dx, dy = b[0] - a[0], b[1] - a[1] + length = math.hypot(dx, dy) + ux, uy = dx / length, dy / length + px, py = p[0] - a[0], p[1] - a[1] + return (px * ux + py * uy, abs(ux * py - uy * px)) + + +def _piece_length(a: Pt, b: Pt, bulge: float) -> float: + chord = math.dist(a, b) + if bulge == 0.0 or chord == 0.0: + return chord + theta = 4.0 * math.atan(abs(bulge)) + return chord / (2.0 * math.sin(theta / 2.0)) * theta + + +def is_reducer(block: str | None) -> bool: + """True when the block name contains a reducer word (Turkish dotted I folded).""" + if not block: + return False + name = block.casefold().replace("̇", "") + return any(word in name for word in REDUCER_WORDS) + + +def _layer_names(snap: Snapshot) -> list[str]: + return sorted(set(snap.layers) | {rec.layer for rec in snap.records}) + + +def default_layers(snap: Snapshot) -> tuple[str, ...]: + """Every layer `classify_layer` files under piping or electrical with confidence >= 0.9.""" + chosen = [] + for name in _layer_names(snap): + found = classify_layer(name) + if found["discipline"] in NETWORK_DISCIPLINES and found["confidence"] >= NETWORK_CONFIDENCE: + chosen.append(name) + return tuple(chosen) + + +def label_search_default(snap: Snapshot, *, tol: float = 1.0) -> float: + """5 x the median model-space text height; 50 x `tol` on a drawing with no text.""" + heights = [ + rec.height + for rec in snap.records + if rec.space == MODEL and rec.type in TEXT_TYPES and rec.height and rec.height > 0.0 + ] + if heights: + return LABEL_SEARCH_FACTOR * statistics.median(heights) + return LABEL_SEARCH_FALLBACK * tol + + +def tag_occurrences(snap: Snapshot) -> dict[str, list[dict]]: + """Every model-space place a tag is written, by tag. + + A TEXT / MTEXT / MULTILEADER whose plain text `parse_tag` reads - unless it + is a diameter label or a wiring callout - and an INSERT attribute whose tag + name contains TAG. ``at`` is the insertion point; ``insert`` is the handle + of the INSERT that carries the tag, or None for a text. + """ + found: dict[str, list[dict]] = {} + for rec in snap.records: + if rec.space != MODEL or not rec.points: + continue + if rec.type in TEXT_TYPES and rec.text: + if parse_diameters(rec.text) or wiring_target(rec.text): + continue + tag = parse_tag(rec.text) + if tag: + found.setdefault(tag, []).append( + { + "tag": tag, + "at": rec.points[0], + "handle": rec.handle, + "box": rec.bbox, + "insert": None, + } + ) + elif rec.type == "INSERT": + for name, value in rec.attribs: + tag = parse_tag(value) if "TAG" in name.upper() else None + if tag: + found.setdefault(tag, []).append( + { + "tag": tag, + "at": rec.points[0], + "handle": rec.handle, + "box": rec.bbox, + "insert": rec.handle, + } + ) + return found + + +def _is_shape(rec: EntityRecord) -> bool: + if rec.type in ("CIRCLE", "ELLIPSE", "INSERT"): + return True + return rec.type in ("LWPOLYLINE", "POLYLINE") and rec.closed + + +def equipment_boxes( + snap: Snapshot, occurrences: dict[str, list[dict]], *, exclude_layers=(), search: float +) -> dict[str, list[dict]]: + """The box of the geometry each tag occurrence labels. + + An INSERT carrying the tag is its own geometry. A text labels the smallest + closed shape (CIRCLE, ELLIPSE, closed polyline, INSERT) off the network + layers whose box holds its insertion point, else the nearest one within + `search`, else only itself (its own box, or its insertion point). A shape + whose box holds the tags of two or more different items is a room, a skid + or a sheet frame - not equipment - and is never chosen. + """ + excluded = set(exclude_layers) + anchors = [(tag, place["at"]) for tag, places in occurrences.items() for place in places] + + def holds_one_item_at_most(rec: EntityRecord) -> bool: + held = {tag for tag, at in anchors if _in_box(at, rec.bbox, 0.0)} + return len(held) <= 1 + + shapes = [ + rec + for rec in snap.records + if rec.space == MODEL + and rec.bbox is not None + and rec.layer not in excluded + and _is_shape(rec) + and holds_one_item_at_most(rec) + ] + out: dict[str, list[dict]] = {} + for tag, places in sorted(occurrences.items()): + for place in places: + x, y = place["at"] + if place["insert"] is not None and place["box"] is not None: + box, handle = place["box"], place["insert"] + else: + inside = [s for s in shapes if _in_box((x, y), s.bbox, 0.0)] + near = [ + (_box_distance((x, y), s.bbox), _box_area(s.bbox), s.handle, s) for s in shapes + ] + near = [n for n in near if n[0] <= search] + if inside: + best = min(inside, key=lambda s: (_box_area(s.bbox), s.handle)) + box, handle = best.bbox, best.handle + elif near: + best = min(near, key=lambda n: n[:3])[3] + box, handle = best.bbox, best.handle + else: + box, handle = place["box"] or (x, y, x, y), place["handle"] + out.setdefault(tag, []).append({"box": box, "handle": handle, "at": (x, y)}) + return out + + +class _Vertices: + """Snap points to vertices: a point within ``tol`` of a vertex *is* that vertex.""" + + def __init__(self, tol: float) -> None: + self.tol = tol + self.points: list[Pt] = [] + self._grid: dict[tuple[int, int], list[int]] = {} + + def index(self, p: Pt) -> int: + cx, cy = math.floor(p[0] / self.tol), math.floor(p[1] / self.tol) + best, best_d = None, self.tol + for dx in (-1, 0, 1): + for dy in (-1, 0, 1): + for i in self._grid.get((cx + dx, cy + dy), ()): + d = math.dist(self.points[i], p) + if d <= best_d: + best, best_d = i, d + if best is not None: + return best + self.points.append((float(p[0]), float(p[1]))) + self._grid.setdefault((cx, cy), []).append(len(self.points) - 1) + return len(self.points) - 1 + + +def _choose_layers(snap: Snapshot, layers) -> tuple[str, ...]: + known = _layer_names(snap) + if layers is None: + chosen = default_layers(snap) + if not chosen: + raise ValueError( + "layers: no layer is classified piping or electrical with confidence >= 0.9; " + f"name the layers to read. The drawing has: {', '.join(known)}" + ) + return chosen + names = [str(name) for name in layers] + if not names: + raise ValueError("layers: an empty list reads nothing; omit it for the default") + missing = [name for name in names if name not in known] + if missing: + raise ValueError( + f"layers: {', '.join(missing)} not in the drawing; it has: {', '.join(known)}" + ) + return tuple(dict.fromkeys(names)) + + +def _arc_segment(rec: EntityRecord): + if not rec.points or rec.radius is None or rec.angles is None: + return None + (cx, cy), r = rec.points[0], rec.radius + start, end = rec.angles + sweep = (end - start) % 360.0 + if sweep == 0.0: + return None + a = (cx + r * math.cos(math.radians(start)), cy + r * math.sin(math.radians(start))) + b = (cx + r * math.cos(math.radians(end)), cy + r * math.sin(math.radians(end))) + return a, b, math.tan(math.radians(sweep) / 4.0) + + +def _segments(snap: Snapshot, layers) -> list[tuple[Pt, Pt, float, str, str]]: + wanted = set(layers) + out = [] + for rec in snap.records: + if rec.space != MODEL or rec.layer not in wanted or rec.type not in LINE_TYPES: + continue + if rec.type == "LINE": + if len(rec.points) >= 2: + out.append((rec.points[0], rec.points[1], 0.0, rec.handle, rec.layer)) + elif rec.type == "ARC": + arc = _arc_segment(rec) + if arc is not None: + out.append((arc[0], arc[1], arc[2], rec.handle, rec.layer)) + else: + pts = list(rec.points) + bulges = list(rec.bulges) + [0.0] * (len(pts) - len(rec.bulges)) + count = len(pts) if rec.closed else len(pts) - 1 + for i in range(max(count, 0)): + out.append( + (pts[i], pts[(i + 1) % len(pts)], float(bulges[i]), rec.handle, rec.layer) + ) + return out + + +class _PieceIndex: + """Grid of pieces for 'nearest piece within `search`' queries.""" + + def __init__(self, pieces: list[dict], points: list[Pt], search: float) -> None: + lengths = [p["length"] for p in pieces] or [search] + self.cell = max(search, statistics.median(lengths), _EPS) + self.search = search + self.cells: dict[tuple[int, int], list[int]] = {} + for k, piece in enumerate(pieces): + a, b = points[piece["u"]], points[piece["v"]] + x0 = math.floor((min(a[0], b[0]) - search) / self.cell) + x1 = math.floor((max(a[0], b[0]) + search) / self.cell) + y0 = math.floor((min(a[1], b[1]) - search) / self.cell) + y1 = math.floor((max(a[1], b[1]) + search) / self.cell) + for cx in range(x0, x1 + 1): + for cy in range(y0, y1 + 1): + self.cells.setdefault((cx, cy), []).append(k) + + def candidates(self, p: Pt) -> list[int]: + return self.cells.get((math.floor(p[0] / self.cell), math.floor(p[1] / self.cell)), []) + + +def build_network(snap: Snapshot, *, layers=None, tol=1.0, label_search=None) -> dict: + """Runs of the line network on `layers` - see the module docstring for every rule. + + Returns ``{"runs": [Run], "unassigned": [...], "attachments": {run id: [{"at", "tag"}]}, + "equipment": {tag: [{"box", "handle", "at"}]}, "stats": {...}}``. Refused by + name before anything is read: a non-positive `tol` or `label_search`, a + named layer the drawing does not have (with the list it has), an empty + `layers`, and - when `layers` is omitted - a drawing with no layer filed + under piping or electrical at confidence >= 0.9. + """ + tol = _positive(tol, "tol") + search = ( + label_search_default(snap, tol=tol) + if label_search is None + else _positive(label_search, "label_search") + ) + chosen = _choose_layers(snap, layers) + raw = _segments(snap, chosen) + verts = _Vertices(tol) + snapped = [] + for a, b, bulge, handle, layer in raw: + u, v = verts.index(a), verts.index(b) + if u != v: + snapped.append((u, v, bulge, handle, layer)) + pts = verts.points + + # split straight segments at every vertex lying on them (T-junctions, overlaps) + order = sorted(range(len(pts)), key=lambda i: pts[i][0]) + xs = [pts[i][0] for i in order] + pieces_by_key: dict[tuple[int, int, float], dict] = {} + overlap_count, overlap_length = 0, 0.0 + for u, v, bulge, handle, layer in snapped: + chain = [u, v] + if bulge == 0.0: + pa, pb = pts[u], pts[v] + length = math.dist(pa, pb) + lo = bisect_left(xs, min(pa[0], pb[0]) - tol) + hi = bisect_right(xs, max(pa[0], pb[0]) + tol) + inner = [] + for k in order[lo:hi]: + if k in (u, v): + continue + p = pts[k] + if not min(pa[1], pb[1]) - tol <= p[1] <= max(pa[1], pb[1]) + tol: + continue + s, off = _along(p, pa, pb) + if off <= tol and tol < s < length - tol: + inner.append((s, k)) + chain = [u] + [k for _s, k in sorted(inner)] + [v] + for p, q in zip(chain, chain[1:], strict=False): + if p == q: + continue + b_signed = bulge if p < q else -bulge + key = (min(p, q), max(p, q), round(b_signed, 9)) + piece_length = _piece_length(pts[p], pts[q], bulge) + known = pieces_by_key.get(key) + if known is not None: + overlap_count += 1 + overlap_length += piece_length + known["handles"].add(handle) + known["layers"].add(layer) + continue + pieces_by_key[key] = { + "u": key[0], + "v": key[1], + "bulge": key[2], + "length": piece_length, + "layer": layer, + "layers": {layer}, + "handles": {handle}, + } + pieces = [pieces_by_key[key] for key in sorted(pieces_by_key)] + + at_vertex: dict[int, list[int]] = {} + for k, piece in enumerate(pieces): + at_vertex.setdefault(piece["u"], []).append(k) + at_vertex.setdefault(piece["v"], []).append(k) + + # equipment first: an INSERT a tag labels is never a fitting + occurrences = tag_occurrences(snap) + equipment = equipment_boxes( + snap, occurrences, exclude_layers=chosen, search=EQUIPMENT_SEARCH_SHARE * search + ) + equipment_handles = {box["handle"] for boxes in equipment.values() for box in boxes} + + free = [vertex for vertex, ks in at_vertex.items() if len(ks) == 1] + bridges: list[tuple[int, int, bool, str]] = [] + fittings, reducers, skipped = 0, 0, [] + for rec in snap.records: + if ( + rec.space != MODEL + or rec.type != "INSERT" + or rec.bbox is None + or rec.handle in equipment_handles + ): + continue + inside = [vertex for vertex in free if _in_box(pts[vertex], rec.bbox, tol)] + if len(inside) < 2: + continue + if len(inside) > MAX_FITTING_ENDS: + skipped.append(rec.handle) + continue + reducer = is_reducer(rec.block) + fittings += 1 + reducers += int(reducer) + for vertex in inside[1:]: + bridges.append((inside[0], vertex, reducer, rec.handle)) + + # runs: union-find over pieces and bridges + parent = list(range(len(pts))) + + def find(i: int) -> int: + while parent[i] != i: + parent[i] = parent[parent[i]] + i = parent[i] + return i + + for piece in pieces: + parent[find(piece["u"])] = find(piece["v"]) + for u, v, _reducer, _handle in bridges: + parent[find(u)] = find(v) + groups: dict[int, list[int]] = {} + for k, piece in enumerate(pieces): + groups.setdefault(find(piece["u"]), []).append(k) + + def first_point(ks: list[int]) -> tuple[float, float]: + return min(min(pts[pieces[k]["u"]], pts[pieces[k]["v"]]) for k in ks) + + ordered = sorted(groups.values(), key=first_point) + run_of: dict[int, int] = {} + for r, ks in enumerate(ordered): + for k in ks: + run_of[k] = r + + # neighbours for continuity: shared vertices, and bridges that are not reducers + neighbours: dict[int, set[tuple[int, bool]]] = {k: set() for k in range(len(pieces))} + for ks in at_vertex.values(): + for k in ks: + neighbours[k].update((j, False) for j in ks if j != k) + for u, v, reducer, _handle in bridges: + for k in at_vertex.get(u, ()): + neighbours[k].update((j, reducer) for j in at_vertex.get(v, ())) + for k in at_vertex.get(v, ()): + neighbours[k].update((j, reducer) for j in at_vertex.get(u, ())) + + # labels: diameters and supply / return words go to the nearest piece + index = _PieceIndex(pieces, pts, search) + by_handle: dict[str, list[int]] = {} + for k, piece in enumerate(pieces): + for handle in piece["handles"]: + by_handle.setdefault(handle, []).append(k) + + def nearest(p: Pt) -> tuple[int, float] | None: + found = [] + for k in index.candidates(p): + piece = pieces[k] + d = _segment_distance(p, pts[piece["u"]], pts[piece["v"]]) + if d <= search: + found.append((d, k)) + if not found: + return None + d, k = min(found) + return k, d + + reaching: dict[int, list[tuple[float, str, str]]] = {} + votes: dict[int, list[tuple[float, str, str | None, str]]] = {} + labels_skipped = [] + for rec in snap.records: + if rec.space != MODEL or rec.type not in TEXT_TYPES or not rec.text: + continue + where = _centre(rec) + if where is None: + continue + tokens = parse_diameters(rec.text) + direction = supply_return(rec.text) + if not tokens and direction is None: + continue + hit = nearest(where) + if hit is None: + continue + k, dist = hit + if len(tokens) > 1: + labels_skipped.append( + {"handle": rec.handle, "text": rec.text, "reason": "more than one diameter"} + ) + elif tokens: + for handle in pieces[k]["handles"]: + for j in by_handle[handle]: + q = pieces[j] + reaching.setdefault(j, []).append( + (_segment_distance(where, pts[q["u"]], pts[q["v"]]), tokens[0], rec.handle) + ) + if direction is not None: + family = _family(classify_layer(rec.text)["service"]) + votes.setdefault(run_of[k], []).append((dist, direction, family, rec.handle)) + + direct: dict[int, tuple[str, str]] = {} + diameter_conflicts = [] + for k, found in sorted(reaching.items()): + found.sort() + direct[k] = (found[0][1], found[0][2]) + others = sorted({token for _d, token, _h in found} - {found[0][1]}) + if others: + diameter_conflicts.append({"piece": k, "kept": found[0][1], "also": others}) + + best: dict[int, list] = {k: [0.0, {token}] for k, (token, _h) in direct.items()} + heap = [(0.0, k, token) for k, (token, _h) in direct.items()] + heapq.heapify(heap) + while heap: + d, k, token = heapq.heappop(heap) + current = best[k] + if d > current[0] + _EPS * max(1.0, d) or token not in current[1]: + continue + for j, via_reducer in neighbours[k]: + if via_reducer or j in direct: + continue + nd = d + (pieces[k]["length"] + pieces[j]["length"]) / 2.0 + known = best.get(j) + if known is None or nd < known[0] - _EPS * max(1.0, nd): + best[j] = [nd, {token}] + heapq.heappush(heap, (nd, j, token)) + elif abs(nd - known[0]) <= _EPS * max(1.0, nd) and token not in known[1]: + known[1].add(token) + heapq.heappush(heap, (nd, j, token)) + + runs, unassigned, attachments, service_conflicts = [], [], {}, [] + for r, ks in enumerate(ordered): + run_id = f"R{r + 1}" + service = _run_service([pieces[k] for k in ks], votes.get(r, []), run_id, service_conflicts) + edges = [] + for n, k in enumerate(sorted(ks), start=1): + piece = pieces[k] + if k in direct: + diameter, source, label = direct[k][0], "direct", direct[k][1] + elif k in best and len(best[k][1]) == 1: + diameter, source, label = next(iter(best[k][1])), "continuity", None + else: + diameter, source, label = None, "unassigned", None + unassigned.append( + { + "run": run_id, + "edge": f"{run_id}.{n}", + "length": piece["length"], + "reason": "two different diameters equally near" + if k in best + else "no diameter label within label_search and none carried along the run", + } + ) + edges.append( + { + "id": f"{run_id}.{n}", + "a": pts[piece["u"]], + "b": pts[piece["v"]], + "bulge": piece["bulge"], + "length": piece["length"], + "layer": piece["layer"], + "layers": tuple(sorted(piece["layers"])), + "handles": tuple(sorted(piece["handles"])), + "service": service, + "diameter": diameter, + "diameter_source": source, + "label": label, + } + ) + degree: dict[int, int] = {} + for k in ks: + for vertex in (pieces[k]["u"], pieces[k]["v"]): + degree[vertex] = degree.get(vertex, 0) + 1 + for u, v, _reducer, _handle in bridges: + if u in degree: + degree[u] += 1 + degree[v] += 1 + end_points = sorted(pts[vertex] for vertex, count in degree.items() if count == 1) + attached = [{"at": p, "tag": _attach(p, equipment, tol)} for p in end_points] + attachments[run_id] = attached + runs.append( + Run( + id=run_id, + edges=tuple(edges), + service=service, + ends=tuple(end_points), + tags=tuple(sorted({a["tag"] for a in attached if a["tag"] is not None})), + ) + ) + + return { + "runs": runs, + "unassigned": unassigned, + "attachments": attachments, + "equipment": equipment, + "stats": { + "layers": list(chosen), + "tol": tol, + "label_search": search, + "segments": len(raw), + "pieces": len(pieces), + "overlap_removed": overlap_count, + "overlap_length": overlap_length, + "fittings": fittings, + "reducers": reducers, + "fittings_skipped": skipped, + "runs": len(runs), + "labels_skipped": labels_skipped, + "diameter_conflicts": diameter_conflicts, + "service_conflicts": service_conflicts, + }, + } + + +def _family(service: str | None) -> str | None: + for family, pair in _PAIRS.items(): + if service in pair: + return family + return None + + +def _run_service(pieces: list[dict], votes: list, run_id: str, conflicts: list) -> str: + weight: dict[str, float] = {} + for piece in pieces: + service = classify_layer(piece["layer"])["service"] + weight[service] = weight.get(service, 0.0) + piece["length"] + known = {s: w for s, w in weight.items() if s != "unknown"} or weight + base = min(known, key=lambda s: (-known[s], s)) + if not votes: + return base + votes = sorted(votes) + direction = votes[0][1] + if len({vote[1] for vote in votes}) > 1: + conflicts.append({"run": run_id, "kept": direction, "reason": "supply and return words"}) + family = _family(base) or next((vote[2] for vote in votes if vote[2]), None) + if family is None: + conflicts.append( + {"run": run_id, "kept": base, "reason": f"{direction} word on a line with no pair"} + ) + return base + return _PAIRS[family][0 if direction == "supply" else 1] + + +def _attach(p: Pt, equipment: dict[str, list[dict]], tol: float) -> str | None: + hits = [ + (_box_area(box["box"]), tag) + for tag, boxes in equipment.items() + for box in boxes + if _in_box(p, box["box"], tol) + ] + return min(hits)[1] if hits else None +``` + +- [ ] **Step 4: Run it to verify it passes** + +Run: `uv run --frozen pytest tests/test_understand_network.py -q` +Expected: PASS — `38 passed`. If a vocabulary guard fails (`the vocabulary must file CIP_LINES as CIP`), Task 1's `classify_layer` spells the CIP keyword differently: change the test's *layer name* to one `classify_layer` files under CIP with confidence >= 0.9, keep every assertion, and record it. If a fixture test fails, read the generator's truth and the run it names before touching anything: the truth is the specification here (Global Constraints: fix a plan test to the verified truth, keep the behaviour, never weaken a test). + +- [ ] **Step 5: Gates** + +``` +uv run --frozen ruff check engineering/understand/network.py tests/test_understand_network.py +uv run --frozen ruff format engineering/understand/network.py tests/test_understand_network.py +uv run --frozen pytest -q --deselect tests/test_release_consistency.py::test_document_numbers_match_the_tree +``` + +Expected: ruff clean; the suite green apart from the deselected known red (record it). In a fresh worktree venv use `uv run --frozen --all-extras …` (Global Constraints). + +- [ ] **Step 6: Commit** + +``` +git add engineering/understand/network.py tests/test_understand_network.py +git -c user.name="Umut CAN" -c user.email="umutcan.edizaslan@gmail.com" commit -m "$(cat <<'MSG' +feat(understand): the line-network engine - runs, services, diameters, equipment + +build_network reads the segments of a foreign P&ID's piping and electrical +layers into runs: ends merged within tol, a branch ending on a header split +into a tee, a crossing without an end left unconnected, and a line drawn +twice - or a line over a polyline - cut into identical pieces that are kept +once, the doubled length reported as overlap_length. An INSERT holding two to +four free ends is an inline fitting joined through; a reducer word in its +block name (reducer, perekhod, reduksiyon, verloop) stops diameter continuity. + +Each diameter label goes to the nearest piece and holds for its drawn entity +(direct); unlabelled pieces take the nearest direct diameter along the run +(continuity) or stay unassigned - two sizes equally near are never guessed. +Labels stay as drawn. A supply or return word near a run overrides its +layer's service for the whole run. Run ends attach to the tag whose labelled +geometry holds them; a shape holding two tags (a room, a skid) is not +equipment. Tests on hand-built drawings and on the synthetic plant pair. + +Co-Authored-By: Claude Opus 5.5 (1M context) +Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd +MSG +)" +``` + +--- + +### Task 6: Orthogonal lengths and pipe takeoff rows + +**Files:** +- Create: `engineering/understand/route.py` +- Create: `engineering/understand/takeoff.py` +- Test: `tests/test_understand_route.py` +- Test: `tests/test_understand_takeoff.py` + +**Interfaces:** +- Consumes: Task 5 — `build_network(...) -> dict` (its `runs`, `attachments` and `stats.tol` / `stats.label_search` / `stats.overlap_length`), `Run`, `tag_occurrences(snap)`, `MODEL`, `TEXT_TYPES`; Task 1 — `Pt`, `Snapshot`, `read_snapshot`, `vocab.SERVICES`, `vocab.classify_layer`, `vocab.room_label(text) -> dict | None`; track F — `engineering.arch.faces.planar_faces`, `face_containing`, `Face`, `DEFAULT_TOL` (reused, not re-implemented); Task 2 — `build_plant_pair` truth keys `pid`, `layout`, `scale_verdict`, `layout_insunits`, `layout_true_unit`, `pipe_runs`, `rooms`. A `scale` argument in the shape of group U's `scale_check` result (`verdict`, `factor`, `median`, `iqr`, `within_10pct`, `pairs`, `matched`, `ambiguous`, `groups`) is consumed when a caller passes one. +- Produces: in `engineering/understand/route.py` — `manhattan(a: Pt, b: Pt) -> float`, `rmst(points: Sequence[Pt]) -> tuple[float, tuple[tuple[int, int], ...]]`, `route_legs(a: Pt, b: Pt) -> tuple[tuple[Pt, Pt], ...]`. In `engineering/understand/takeoff.py` — `pipe_rows(network, pid: Snapshot, layout: Snapshot | None, *, scale=None, services=DEFAULT_SERVICES, allowance=0.20, vertical_allowance=0.0, length_source="auto", force=False) -> dict` (keys `kind`, `rows`, `runs`, `control`, `summary{by_service, by_diameter, by_room}`, `totals{routed_m, unroutable_m, net_m, allowance, allowance_m, total_m, flagged_runs}`, `length_source`, `scale`, `scale_verified`, `units{pid, layout}`, `warnings`, `method`); `DEFAULT_SERVICES = ("product", "cip_supply", "cip_return")`, `LENGTH_SOURCES`, `UNIT_METRES`; `check_pipe_options(*, services, allowance, vertical_allowance, length_source) -> dict`; `check_allowance(allowance) -> float`; `unit_of(snap) -> dict` (`declared`, `inferred`, `m_per_unit`, `warning`, `evidence`); `layout_tags(snap, wanted=()) -> dict` (`positions`, `ambiguous`, `cluster`, `clusters`); `scale_stand_in(pid, layout, *, pid_tags=None, layout_found=None, layout_unit=None) -> dict` (the subset of group U's `scale_check` shape a takeoff reads: `verdict`, `factor` (median, only when `to_scale`), `median`, `q1`, `q3`, `iqr`, `within_10pct`, `pair_count`, `pairs[{tags, a_mm, b_mm, ratio}]`, `matched[{tag, a, b}]`, `ambiguous`, `groups: []`, `notes`); `room_regions(snap, cluster=None) -> dict` (`faces: [(key, Face)]`, `labels: [(key, Pt)]`); `room_at(regions, p) -> str`; `apportion(legs, regions) -> dict[str, float]`. + +**Why this module carries stand-ins.** Group N is cut from `main` beside groups U and D (skeleton, Execution), so `engineering/understand/{units,clusters,scale}.py` do not exist on this branch. `takeoff.py` therefore carries the narrow form of each that a takeoff needs — `unit_of`, `layout_tags` with its plan-copy clusters, and `scale_stand_in` (spec §6's statistics without per-room groups) — and `pipe_rows` uses a `scale` result it is given before its own. Tag positions on the layout always come from `layout_tags`, which applies Review Focus item 3's rule (only tags unique inside the chosen cluster; the rest `ambiguous`, never picked). The stand-in returns group U's result shape — the ratio is P&ID millimetres over layout millimetres, each drawing in its inferred unit, and `factor` is set only when the verdict is `to_scale` — so after the merge a caller can pass `scale_check(pid, layout)` into `pipe_rows(scale=...)` without touching a row — Task 12's benchmark task, correctness checks, smoke and field-test script do exactly that. No task of this plan swaps `unit_of` / `room_regions` for group U's `infer_units` / `find_rooms`; the stand-ins stay in `takeoff.py`, and the `pipe_takeoff` tool keeps computing its own `scale_stand_in`. The unit bands (text 0.5 mm – 1 m, span 0.1 m – 5 km) and the 2 % cluster gap are this module's heuristics, stated in its docstring — not standards values. Review Focus item 5 is owned here: `test_review_focus_5_inches_declared_over_millimetres` (hand-built) and `test_review_focus_5_the_synthetic_layout_is_read_in_millimetres` (the Task 2 pair). + +- [ ] **Step 1: Write the failing route test** + +Create `tests/test_understand_route.py`: + +```python +"""Orthogonal lengths: Manhattan distance, the rectilinear MST, the x-then-y route. + +Every expected value is worked by hand in the comment beside it. +""" + +from __future__ import annotations + +from engineering.understand.route import manhattan, rmst, route_legs + +EPS = 1e-9 + + +def test_manhattan_is_the_sum_of_the_axis_distances(): + assert manhattan((0.0, 0.0), (3.0, -4.0)) == 7.0 # 3 + 4, not the diagonal 5 + + +def test_fewer_than_two_points_have_no_tree(): + assert rmst([]) == (0.0, ()) + assert rmst([(5.0, 5.0)]) == (0.0, ()) + + +def test_a_two_ended_run_is_dx_plus_dy(): + length, edges = rmst([(1000.0, 3000.0), (29000.0, 15000.0)]) + assert abs(length - 40000.0) < EPS # 28000 + 12000 + assert edges == ((0, 1),) + + +def test_a_square_of_four_tags_is_three_sides(): + # Prim from point 0: (0,1) at 10 (tie with 3 goes to the lower index), + # then (1,2) at 10, then (0,3) at 10 -> 30, never the 40 of the full ring + length, edges = rmst([(0.0, 0.0), (10.0, 0.0), (10.0, 10.0), (0.0, 10.0)]) + assert abs(length - 30.0) < EPS + assert edges == ((0, 1), (1, 2), (0, 3)) + + +def test_three_tags_in_a_row_chain_instead_of_a_star(): + length, edges = rmst([(0.0, 0.0), (100.0, 0.0), (200.0, 0.0)]) + assert abs(length - 200.0) < EPS # 100 + 100, not 100 + 200 + assert edges == ((0, 1), (1, 2)) + + +def test_the_route_runs_along_x_first_then_y(): + assert route_legs((0.0, 0.0), (30.0, 40.0)) == ( + ((0.0, 0.0), (30.0, 0.0)), + ((30.0, 0.0), (30.0, 40.0)), + ) + assert route_legs((0.0, 0.0), (0.0, 40.0)) == (((0.0, 0.0), (0.0, 40.0)),) + assert route_legs((1.0, 1.0), (1.0, 1.0)) == () +``` + +- [ ] **Step 2: Run it to verify it fails** + +Run: `uv run --frozen pytest tests/test_understand_route.py -q` +Expected: FAIL — collection error `ModuleNotFoundError: No module named 'engineering.understand.route'`. + +- [ ] **Step 3: Write the orthogonal lengths** + +Create `engineering/understand/route.py`: + +```python +"""Orthogonal lengths on a layout: Manhattan distance and the rectilinear MST. + +A pipe or cable on a plant floor runs along the building grid, so its length +between two points is estimated as |dx| + |dy| - never the diagonal. A run that +joins more than two pieces of equipment is estimated as the minimum spanning +tree of their positions under that distance (Prim's algorithm); a two-ended run +is simply |dx| + |dy|. This is the tree on the given points only - no Steiner +points are added - so it is the length a drafter gets by routing each +connection separately, and it is exact for the points given. + +A route is drawn along x first, then y (`route_legs`): deterministic, and +stated on every method sheet that uses it. +""" + +from __future__ import annotations + +import math +from collections.abc import Sequence + +from engineering.understand.snapshot import Pt + +__all__ = ["manhattan", "rmst", "route_legs"] + + +def manhattan(a: Pt, b: Pt) -> float: + """|dx| + |dy| between two points.""" + return abs(a[0] - b[0]) + abs(a[1] - b[1]) + + +def rmst(points: Sequence[Pt]) -> tuple[float, tuple[tuple[int, int], ...]]: + """Rectilinear minimum spanning tree: (total length, edges as (from, to) indices). + + Prim's algorithm from point 0; each edge is (the tree vertex it grows from, + the new vertex), in the order the tree grows. Ties go to the lower index, + so the answer is deterministic. Fewer than two points: (0.0, ()). + """ + n = len(points) + if n < 2: + return 0.0, () + in_tree = [False] * n + best = [math.inf] * n + parent = [-1] * n + best[0] = 0.0 + total = 0.0 + edges: list[tuple[int, int]] = [] + for _ in range(n): + k = min((i for i in range(n) if not in_tree[i]), key=lambda i: (best[i], i)) + in_tree[k] = True + if parent[k] >= 0: + edges.append((parent[k], k)) + total += best[k] + for j in range(n): + if not in_tree[j]: + d = manhattan(points[k], points[j]) + if d < best[j]: + best[j], parent[j] = d, k + return total, tuple(edges) + + +def route_legs(a: Pt, b: Pt) -> tuple[tuple[Pt, Pt], ...]: + """The orthogonal route from a to b: along x first, then y; zero-length legs dropped.""" + corner = (b[0], a[1]) + return tuple(leg for leg in ((a, corner), (corner, b)) if leg[0] != leg[1]) +``` + +- [ ] **Step 4: Run it to verify it passes** + +Run: `uv run --frozen pytest tests/test_understand_route.py -q` +Expected: PASS — `6 passed`. + +- [ ] **Step 5: Write the failing takeoff test** + +Create `tests/test_understand_takeoff.py`: + +```python +"""Pipe takeoff rows: layout lengths, statuses, rooms, units and the scale rule. + +The hand-built pair: a P&ID (mm) with three tanks and two runs, and a layout +whose INSUNITS says inches over millimetre geometry - two rooms split by a +wall at x = 20000, the tags at T4100 (2000, 3000), T4101 (30000, 3000) and +T4102 (30000, 15000). Every expected length is worked in the comment beside it. +The last four tests read the synthetic plant pair of Task 2 and compare every +run with the generator's own hand-computed rectilinear MST. +""" + +from __future__ import annotations + +import pytest + +from engineering.understand.network import build_network +from engineering.understand.snapshot import EntityRecord, Snapshot, read_snapshot +from engineering.understand.takeoff import ( + DEFAULT_SERVICES, + layout_tags, + pipe_rows, + scale_stand_in, + unit_of, +) +from engineering.understand.vocab import classify_layer, room_label +from tests.fixtures.plant_pair import build_plant_pair + +EPS = 1e-9 +PIPE = "PRODUCT" +WALLS = "WALLS" + + +def rec(handle, kind, layer, points=(), **extra): + return EntityRecord( + handle=handle, type=kind, layer=layer, space="Model", points=tuple(points), **extra + ) + + +def text(handle, value, at, height, layer="TEXT"): + x, y = at + return rec( + handle, + "TEXT", + layer, + (at,), + text=value, + height=height, + bbox=(x, y, x + height * len(value) * 0.6, y + height), + ) + + +def snap(records, insunits, source): + layers = {r.layer: {"color": 7, "linetype": "Continuous"} for r in records} + return Snapshot( + source=source, + insunits=insunits, + extmin=None, + extmax=None, + layers=layers, + layouts=(), + records=tuple(records), + ) + + +def tank(handle, tag, x0, y0): + box = (x0, y0, x0 + 500.0, y0 + 500.0) + return [ + rec( + handle, + "LWPOLYLINE", + "EQUIPMENT", + ((x0, y0), (x0 + 500.0, y0), (x0 + 500.0, y0 + 500.0), (x0, y0 + 500.0)), + closed=True, + bbox=box, + ), + text(handle + "T", tag, (x0 + 150.0, y0 + 225.0), 50.0), + ] + + +def pid_records(): + return [ + *tank("A", "T4100", 0.0, 0.0), + *tank("B", "T4101", 3000.0, 0.0), + *tank("C", "T4102", 3000.0, 2000.0), + rec("L1", "LINE", PIPE, ((500.0, 250.0), (3000.0, 250.0))), + rec("L2", "LINE", PIPE, ((3250.0, 500.0), (3250.0, 2000.0))), + text("D1", "Ø51", (1500.0, 260.0), 50.0), + ] + + +def layout_records(dx=0.0, extra=()): + def at(x, y): + return (x + dx, y) + + return [ + rec( + f"W{dx}", + "LWPOLYLINE", + WALLS, + (at(0, 0), at(40000, 0), at(40000, 20000), at(0, 20000)), + closed=True, + bbox=(dx, 0.0, dx + 40000.0, 20000.0), + ), + rec(f"X{dx}", "LINE", WALLS, (at(20000, 0), at(20000, 20000))), + text(f"R1{dx}", "ROOM 101", at(5000, 10000), 250.0), + text(f"R2{dx}", "ROOM 102", at(30000, 10000), 250.0), + text(f"T0{dx}", "T4100", at(2000, 3000), 250.0), + text(f"T1{dx}", "T4101", at(30000, 3000), 250.0), + text(f"T2{dx}", "T4102", at(30000, 15000), 250.0), + *extra, + ] + + +def room_key(label): + found = room_label(label) + assert found is not None, f"the vocabulary must read {label!r} as a room label" + return str(found.get("number") or found.get("name")) + + +@pytest.fixture +def pair(): + assert classify_layer(PIPE)["service"] == "product", "PRODUCT must be a product layer" + assert classify_layer(WALLS)["discipline"] == "architecture", "WALLS must be a wall layer" + pid = snap(pid_records(), 4, "pid.dxf") + layout = snap(layout_records(), 1, "layout.dxf") # INSUNITS 1 = inches, geometry in mm + return pid, layout, build_network(pid, layers=[PIPE]) + + +def run_by_tags(result, *tags): + (run,) = [r for r in result["runs"] if r["tags"] == list(tags)] + return run + + +def test_layout_lengths_are_manhattan_between_the_tags(pair): + pid, layout, network = pair + result = pipe_rows(network, pid, layout, length_source="layout") + a = run_by_tags(result, "T4100", "T4101") + b = run_by_tags(result, "T4101", "T4102") + assert abs(a["layout_m"] - 28.0) < EPS # |30000 - 2000| + 0 mm + assert abs(b["layout_m"] - 12.0) < EPS # 0 + |15000 - 3000| mm + assert (a["status"], b["status"]) == ("A", "B") # b has no diameter label + assert result["length_source"] == "layout" + + +def test_review_focus_5_inches_declared_over_millimetres(pair): + pid, layout, network = pair + result = pipe_rows(network, pid, layout, length_source="layout") + units = result["units"]["layout"] + assert (units["declared"], units["inferred"], units["m_per_unit"]) == ("in", "mm", 0.001) + a = run_by_tags(result, "T4100", "T4101") + assert abs(a["layout_m"] - 28.0) < EPS # not 28000 x 0.0254 = 711.2 + assert any("INSUNITS declares in" in w and "as mm" in w for w in result["warnings"]) + assert unit_of(pid)["warning"] is None # the P&ID's mm is what its geometry says + + +def test_rooms_share_the_route_x_first(pair): + pid, layout, network = pair + result = pipe_rows(network, pid, layout, length_source="layout") + a = run_by_tags(result, "T4100", "T4101") + # (2000, 3000) -> (30000, 3000): 18000 mm left of the wall at x = 20000, 10000 right + assert a["rooms"] == pytest.approx( + {room_key("ROOM 101"): 18.0, room_key("ROOM 102"): 10.0}, abs=EPS + ) + rows = {(r["room"], r["diameter"]): r for r in result["rows"]} + assert rows[(room_key("ROOM 101"), "Ø51")]["net_m"] == pytest.approx(18.0, abs=EPS) + assert rows[(room_key("ROOM 101"), "Ø51")]["direct_m"] == pytest.approx(18.0, abs=EPS) + assert rows[(room_key("ROOM 102"), "unassigned")]["unassigned_m"] == pytest.approx( + 12.0, abs=EPS + ) + + +def test_allowance_is_its_own_column_and_totals_add_up(pair): + pid, layout, network = pair + result = pipe_rows(network, pid, layout, length_source="layout", allowance=0.25) + for row in result["rows"]: + assert row["allowance"] == 0.25 + assert row["allowance_m"] == pytest.approx(row["net_m"] * 0.25, abs=EPS) + assert row["total_m"] == pytest.approx(row["net_m"] * 1.25, abs=EPS) + totals = result["totals"] + assert totals["net_m"] == pytest.approx(40.0, abs=EPS) # 28 + 12 + assert totals["total_m"] == pytest.approx(50.0, abs=EPS) + assert totals["flagged_runs"] == 0 + + +def test_vertical_allowance_is_added_per_connection(pair): + pid, layout, network = pair + result = pipe_rows(network, pid, layout, length_source="layout", vertical_allowance=0.5) + a = run_by_tags(result, "T4100", "T4101") + assert a["vertical_m"] == 0.5 # one tree edge + assert a["length_m"] == pytest.approx(28.5, abs=EPS) + + +def test_the_schematic_length_is_kept_beside_the_answer(pair): + pid, layout, network = pair + result = pipe_rows(network, pid, layout, length_source="layout") + a = run_by_tags(result, "T4100", "T4101") + assert a["schematic_m"] == pytest.approx(2.5, abs=EPS) # the 2500 mm line on the P&ID + + +def test_a_tag_missing_from_the_layout_makes_the_run_c(pair): + pid, _layout, network = pair + layout = snap([r for r in layout_records() if r.text != "T4102"], 1, "layout.dxf") + result = pipe_rows(network, pid, layout, length_source="layout") + b = run_by_tags(result, "T4101", "T4102") + assert (b["status"], b["reason"]) == ("C", "tag_not_on_layout") + (row,) = [c for c in result["control"] if c["kind"] == "unroutable"] + assert row["schematic_m"] == pytest.approx(1.5, abs=EPS) # the 1500 mm line, drawn length only + assert result["totals"]["unroutable_m"] == pytest.approx(1.5, abs=EPS) + assert result["totals"]["flagged_runs"] == 1 + + +def test_a_tag_written_twice_in_the_plan_is_ambiguous_not_picked(pair): + pid, _layout, network = pair + extra = [text("dup", "T4101", (35000.0, 18000.0), 250.0)] + layout = snap(layout_records(extra=extra), 1, "layout.dxf") + result = pipe_rows(network, pid, layout, length_source="layout") + assert {r["reason"] for r in result["runs"]} == {"tag_ambiguous"} + + +def test_of_two_plan_copies_one_is_chosen_and_nothing_is_ambiguous(pair): + pid, _layout, network = pair + layout = snap(layout_records() + layout_records(dx=100000.0), 1, "layout.dxf") + found = layout_tags(layout, ["T4100", "T4101", "T4102"]) + assert found["clusters"] == 2 + assert found["cluster"][2] < 100000.0 # the left copy: ties go to the leftmost + assert found["ambiguous"] == [] + result = pipe_rows(network, pid, layout, length_source="layout") + assert run_by_tags(result, "T4100", "T4101")["layout_m"] == pytest.approx(28.0, abs=EPS) + + +def test_a_run_with_a_free_end_is_c(pair): + pid, layout, _network = pair + records = pid_records() + [rec("L3", "LINE", PIPE, ((0.0, 500.0), (0.0, 1500.0)))] + drawing = snap(records, 4, "pid.dxf") + result = pipe_rows( + build_network(drawing, layers=[PIPE]), drawing, layout, length_source="layout" + ) + reasons = {r["reason"] for r in result["runs"] if r["status"] == "C"} + assert reasons == {"end_not_on_equipment"} + + +def test_other_services_are_left_out_unless_asked(pair): + pid, layout, network = pair + assert DEFAULT_SERVICES == ("product", "cip_supply", "cip_return") + result = pipe_rows(network, pid, layout, length_source="layout", services=["steam"]) + assert result["rows"] == [] + assert result["method"]["excluded_services"] == {"product": 2} + + +# -- the scale rule ---------------------------------------------------------- + +SCHEMATIC = { + "verdict": "schematic", + "within_10pct": 0.25, + "pair_count": 12, + "median": 1.3, + "iqr": 0.8, + "factor": None, + "matched": [], + "ambiguous": [], +} + + +def test_auto_measures_a_schematic_pid_on_the_layout(pair): + pid, layout, network = pair + result = pipe_rows(network, pid, layout, scale=SCHEMATIC) + assert result["length_source"] == "layout" + assert result["method"]["reason"] == "auto_schematic" + assert result["scale_verified"] is True + + +def test_pid_on_a_schematic_pid_is_refused_with_the_statistics(pair): + pid, layout, network = pair + with pytest.raises(ValueError) as excinfo: + pipe_rows(network, pid, layout, scale=SCHEMATIC, length_source="pid") + message = str(excinfo.value) + assert message.startswith("length_source='pid': the scale check calls this P&ID schematic") + assert "25% of 12 tag pairs" in message and "median ratio 1.3" in message + assert "force=True" in message + + +def test_force_measures_on_the_pid_and_says_it_is_unverified(pair): + pid, layout, network = pair + result = pipe_rows(network, pid, layout, scale=SCHEMATIC, length_source="pid", force=True) + assert result["length_source"] == "pid" + assert result["method"]["reason"] == "forced" + assert result["scale_verified"] is False + assert run_by_tags(result, "T4100", "T4101")["length_m"] == pytest.approx(2.5, abs=EPS) + assert any(w.startswith("scale_verified: false") for w in result["warnings"]) + + +def test_a_to_scale_pid_is_measured_through_its_factor(pair): + pid, layout, network = pair + to_scale = dict(SCHEMATIC, verdict="to_scale", within_10pct=0.9, factor=0.1) + result = pipe_rows(network, pid, layout, scale=to_scale) + assert result["length_source"] == "pid" + # 2.5 m drawn on the P&ID / 0.1 P&ID mm per layout mm = 25 m + assert run_by_tags(result, "T4100", "T4101")["length_m"] == pytest.approx(25.0, abs=EPS) + + +def test_without_a_layout_auto_measures_the_pid_unverified(pair): + pid, _layout, network = pair + result = pipe_rows(network, pid, None) + assert (result["length_source"], result["scale_verified"]) == ("pid", False) + assert result["method"]["reason"] == "no_layout" + with pytest.raises(ValueError, match="^length_source='layout' needs a layout drawing"): + pipe_rows(network, pid, None, length_source="layout") + + +def test_the_stand_in_recognises_a_uniform_scale_and_a_rearranged_one(): + places = { + "T4100": (2000.0, 3000.0), + "T4101": (30000.0, 3000.0), + "T4102": (30000.0, 15000.0), + "T4103": (8000.0, 17000.0), + } + layout = snap( + [text(t, t, p, 250.0) for t, p in places.items()] + + [rec("W", "LINE", WALLS, ((0.0, 0.0), (40000.0, 20000.0)))], + 4, + "l", + ) + half = snap([text(t, t, (x * 0.5, y * 0.5), 50.0) for t, (x, y) in places.items()], 4, "p") + uniform = scale_stand_in(half, layout) + assert uniform["verdict"] == "to_scale" + assert uniform["factor"] == pytest.approx(0.5, abs=EPS) # P&ID mm per layout mm + assert uniform["pair_count"] == 6 # four tags, every pair at least 2 m apart + shuffled = { + "T4100": (30000.0, 15000.0), + "T4101": (2000.0, 3000.0), + "T4102": (8000.0, 17000.0), + "T4103": (30000.0, 3000.0), + } + other = snap([text(t, t, p, 50.0) for t, p in shuffled.items()], 4, "p") + assert scale_stand_in(other, layout)["verdict"] == "schematic" + + +@pytest.mark.parametrize( + ("kwargs", "message"), + [ + ({"allowance": 1.5}, "allowance: must lie between 0 and 1"), + ({"allowance": -0.1}, "allowance: must lie between 0 and 1"), + ({"vertical_allowance": -1.0}, "vertical_allowance: must be zero or more metres"), + ({"length_source": "tape"}, "length_source: 'tape' unknown"), + ({"services": ["lava"]}, "services: lava unknown"), + ({"services": []}, "services: an empty list takes nothing off"), + ], +) +def test_refusals_name_the_option(pair, kwargs, message): + pid, layout, network = pair + with pytest.raises(ValueError, match="^" + message): + pipe_rows(network, pid, layout, **kwargs) + + +# -- the synthetic plant pair (Task 2) ------------------------------------------- + + +@pytest.fixture(scope="module") +def plant(tmp_path_factory): + truth = build_plant_pair(tmp_path_factory.mktemp("plant")) + pid, layout = read_snapshot(truth["pid"]), read_snapshot(truth["layout"]) + return truth, pid, layout, build_network(pid) + + +def truth_services(truth): + return sorted({run["service"] for run in truth["pipe_runs"]}) + + +def test_every_synthetic_layout_length_is_the_hand_computed_rmst(plant): + truth, pid, layout, network = plant + result = pipe_rows(network, pid, layout, services=truth_services(truth), length_source="layout") + runs = {(tuple(sorted(r["tags"])), r["service"]): r for r in result["runs"]} + for expected in truth["pipe_runs"]: + run = runs[(tuple(sorted(expected["tags"])), expected["service"])] + assert run["status"] == expected["status"], run["id"] + if expected["status"] != "C": + assert run["layout_m"] * 1000.0 == pytest.approx(expected["layout_length_mm"], abs=1e-6) + assert sum(run["rooms"].values()) == pytest.approx(run["length_m"], abs=1e-9) + + +def test_auto_calls_the_synthetic_pid_schematic_and_measures_the_layout(plant): + truth, pid, layout, network = plant + result = pipe_rows(network, pid, layout, services=truth_services(truth)) + assert result["scale"]["verdict"] == truth["scale_verdict"] == "schematic" + assert result["length_source"] == "layout" + with pytest.raises(ValueError, match="the scale check calls this P&ID schematic"): + pipe_rows(network, pid, layout, services=truth_services(truth), length_source="pid") + + +def test_review_focus_5_the_synthetic_layout_is_read_in_millimetres(plant): + truth, pid, layout, network = plant + assert layout.insunits == truth["layout_insunits"] == 1 + result = pipe_rows(network, pid, layout, services=truth_services(truth), length_source="layout") + assert result["units"]["layout"]["inferred"] == truth["layout_true_unit"] == "mm" + assert result["units"]["layout"]["m_per_unit"] == 0.001 + assert any("INSUNITS declares in" in w for w in result["warnings"]) + + +def test_routed_lengths_land_in_the_synthetic_rooms(plant): + truth, pid, layout, network = plant + result = pipe_rows(network, pid, layout, services=truth_services(truth), length_source="layout") + known = {str(room["number"] or room["name"]) for room in truth["rooms"]} | {""} + for run in result["runs"]: + assert set(run["rooms"]) <= known, run["id"] +``` + +- [ ] **Step 6: Run it to verify it fails** + +Run: `uv run --frozen pytest tests/test_understand_takeoff.py -q` +Expected: FAIL — collection error `ModuleNotFoundError: No module named 'engineering.understand.takeoff'`. + +- [ ] **Step 7: Write the pipe rows** + +Create `engineering/understand/takeoff.py`: + +```python +"""Pipe takeoff rows: topology from the P&ID, lengths from the layout (spec §8). + +A P&ID says which equipment a line joins, what it carries and how big it is; +it does not say how long the pipe is unless the scale check calls it to scale. +So each run found by `engineering.understand.network` is measured one of two +ways, and the rows say which: + +* **layout** - the positions of the tags the run joins, on the layout, joined + by the rectilinear minimum spanning tree (`route.rmst`), plus + `vertical_allowance` metres per tree edge. The P&ID's drawn length is kept + beside it as `schematic_m`, a comparison column, never the answer. +* **pid** - the drawn length on the P&ID: divided by the scale factor when the + scale check calls it `to_scale`, taken in the P&ID's own unit when there is + no layout (every row then carries `scale_verified: false`), and on a + schematic P&ID only with `force=True`. + +This module also carries the takeoff's own stand-ins for three readers another +branch of track H owns - the unit inference, the plan-copy clusters and the +scale check (group U's `units`, `clusters` and `scale`). They follow the +spec's rules (§5, §6) in the narrow form a takeoff needs; `pipe_rows` uses a +`scale` result it is given before its own, so the full scale check replaces +the stand-in wherever a caller passes it. + +Every length leaves this module in metres, converted from the unit the +geometry implies - never blindly from `INSUNITS`. Nothing here touches a +drawing. +""" + +from __future__ import annotations + +import math +import statistics +from collections.abc import Iterable + +from engineering.arch.faces import DEFAULT_TOL, Face, face_containing, planar_faces +from engineering.understand.network import MODEL, TEXT_TYPES, tag_occurrences +from engineering.understand.route import rmst, route_legs +from engineering.understand.snapshot import Pt, Snapshot +from engineering.understand.vocab import SERVICES, classify_layer, room_label + +__all__ = [ + "DEFAULT_SERVICES", + "LENGTH_SOURCES", + "UNIT_METRES", + "apportion", + "check_allowance", + "check_pipe_options", + "layout_tags", + "pipe_rows", + "room_at", + "room_regions", + "scale_stand_in", + "unit_of", +] + +#: The user's own takeoff scope; utilities only on request (spec §8). +DEFAULT_SERVICES = ("product", "cip_supply", "cip_return") +LENGTH_SOURCES = ("auto", "pid", "layout") +UNASSIGNED = "unassigned" + +#: INSUNITS codes this module reads, and metres per unit of each. +INSUNITS_NAMES = {1: "in", 2: "ft", 4: "mm", 5: "cm", 6: "m"} +UNIT_METRES = {"mm": 0.001, "cm": 0.01, "m": 1.0, "in": 0.0254, "ft": 0.3048} +#: This module's heuristic, not a standard: the model-space text of any plant +#: drawing - a P&ID sheet at 1:1 or a plan at 1:200 - stands between 0.5 mm and +#: 1 m tall, and the drawing spans between 0.1 m and 5 km. A declared unit that +#: puts the geometry outside either band is not believed. +TEXT_BAND_M = (0.0005, 1.0) +SPAN_BAND_M = (0.1, 5000.0) +#: Among the units that fit both bands, the one nearest (in log) a 50 m plant +#: with 0.25 m text is chosen. +TYPICAL_SPAN_M = 50.0 +TYPICAL_TEXT_M = 0.25 + +#: Plan copies are told apart when the gap between them exceeds 2 % of the +#: drawing's robust span (5th-95th percentile). +CLUSTER_GAP_SHARE = 0.02 +#: Spec §6: pairs of matched tags at least 2 m apart, a ±10 % band, 80 % / 50 %. +MIN_PAIR_MM = 2000.0 +SCALE_BAND = 0.10 +TO_SCALE_SHARE = 0.80 +SCHEMATIC_SHARE = 0.50 +#: Fewer matched tags than this and the verdict is `insufficient`. +MIN_TAGS = 3 +#: A wall layer is one `classify_layer` files under architecture this surely. +WALL_CONFIDENCE = 0.9 + + +# -- options ----------------------------------------------------------------- + + +def _number(value, name: str) -> float: + try: + number = float(value) + except (TypeError, ValueError): + raise ValueError(f"{name}: expected a number, got {value!r}") from None + if not math.isfinite(number): + raise ValueError(f"{name}: must be finite, got {value!r}") + return number + + +def check_allowance(allowance) -> float: + number = _number(allowance, "allowance") + if not 0.0 <= number <= 1.0: + raise ValueError(f"allowance: must lie between 0 and 1 (0.20 is 20 %), got {number:g}") + return number + + +def check_pipe_options(*, services, allowance, vertical_allowance, length_source) -> dict: + """Refuse bad takeoff options by name, before a drawing is read.""" + chosen = DEFAULT_SERVICES if services is None else tuple(dict.fromkeys(services)) + if not chosen: + raise ValueError("services: an empty list takes nothing off; omit it for the default") + unknown = [s for s in chosen if s not in SERVICES] + if unknown: + raise ValueError( + f"services: {', '.join(map(str, unknown))} unknown; choose from {', '.join(SERVICES)}" + ) + vertical = _number(vertical_allowance, "vertical_allowance") + if vertical < 0.0: + raise ValueError(f"vertical_allowance: must be zero or more metres, got {vertical:g}") + if length_source not in LENGTH_SOURCES: + raise ValueError( + f"length_source: {length_source!r} unknown; choose from {', '.join(LENGTH_SOURCES)}" + ) + return { + "services": chosen, + "allowance": check_allowance(allowance), + "vertical_allowance": vertical, + "length_source": length_source, + } + + +# -- units ------------------------------------------------------------------- + + +def _model_points(snap: Snapshot) -> list[Pt]: + out: list[Pt] = [] + for rec in snap.records: + if rec.space != MODEL: + continue + out.extend(rec.points) + if rec.bbox is not None: + out.extend(((rec.bbox[0], rec.bbox[1]), (rec.bbox[2], rec.bbox[3]))) + return out + + +def _robust_box(points: list[Pt]) -> tuple[float, float, float, float] | None: + """The 5th-95th percentile box of the points: one stray entity does not stretch it.""" + if len(points) < 2: + return None + qx = statistics.quantiles([p[0] for p in points], n=20, method="inclusive") + qy = statistics.quantiles([p[1] for p in points], n=20, method="inclusive") + return (qx[0], qy[0], qx[-1], qy[-1]) + + +def _text_heights(snap: Snapshot) -> list[float]: + return [ + rec.height + for rec in snap.records + if rec.space == MODEL and rec.type in TEXT_TYPES and rec.height and rec.height > 0.0 + ] + + +def unit_of(snap: Snapshot) -> dict: + """The unit a drawing's geometry implies, against the one INSUNITS declares. + + ``{"declared", "inferred", "m_per_unit", "warning", "evidence"}``. The + declared unit (mm when INSUNITS declares none) stands when it puts the + median model-space text height between 0.5 mm and 1 m and the robust span + (5th-95th percentile) between 0.1 m and 5 km - this module's heuristic. + Otherwise the unit of mm / cm / m / in / ft that fits both bands and lies + nearest a 50 m drawing with 0.25 m text is taken. A disagreement is a + warning with the numbers - never a silent correction. + """ + declared = INSUNITS_NAMES.get(snap.insunits) + box = _robust_box(_model_points(snap)) + span = max(box[2] - box[0], box[3] - box[1]) if box else 0.0 + heights = _text_heights(snap) + height = statistics.median(heights) if heights else 0.0 + + def fits(unit: str) -> bool: + m = UNIT_METRES[unit] + return (span <= 0.0 or SPAN_BAND_M[0] <= span * m <= SPAN_BAND_M[1]) and ( + height <= 0.0 or TEXT_BAND_M[0] <= height * m <= TEXT_BAND_M[1] + ) + + def score(unit: str) -> float: + m = UNIT_METRES[unit] + total = abs(math.log(span * m / TYPICAL_SPAN_M)) if span > 0.0 else 0.0 + return total + (abs(math.log(height * m / TYPICAL_TEXT_M)) if height > 0.0 else 0.0) + + base = declared or "mm" + inferred = base + if not fits(base): + candidates = [unit for unit in UNIT_METRES if fits(unit)] + if candidates: + inferred = min(candidates, key=lambda unit: (score(unit), unit)) + warning = None + if inferred != declared: + warning = ( + f"{snap.source}: INSUNITS declares {declared or 'no unit'} but the geometry reads " + f"as {inferred} (median text height {height:g}, robust span {span:g} drawing " + f"units); lengths are taken in {inferred}" + ) + return { + "declared": declared, + "inferred": inferred, + "m_per_unit": UNIT_METRES[inferred], + "warning": warning, + "evidence": {"robust_span": span, "median_text_height": height}, + } + + +# -- plan copies and tag positions --------------------------------------------- + + +def _mark_segment(cells: set, a: Pt, b: Pt, gap: float, limit: float) -> None: + length = math.dist(a, b) + if length > limit: + steps = 0 # a stray line across the world marks only its ends + else: + steps = int(length / (gap / 2.0)) + for i in range(steps + 1): + t = i / steps if steps else 0.0 + x, y = a[0] + (b[0] - a[0]) * t, a[1] + (b[1] - a[1]) * t + cells.add((math.floor(x / gap), math.floor(y / gap))) + cells.add((math.floor(b[0] / gap), math.floor(b[1] / gap))) + + +def _clusters(snap: Snapshot) -> tuple[float, list[tuple[float, float, float, float]]]: + """(cell size, cluster boxes): separate bodies of model-space content. + + Every record's geometry marks the cells of a grid whose cell is 2 % of the + robust span; 8-connected cells form bodies; bodies whose boxes overlap are + one cluster (a label in the middle of a room belongs to the plan around it). + """ + points = _model_points(snap) + box = _robust_box(points) + if box is None: + return 1.0, [] + span = max(box[2] - box[0], box[3] - box[1], 1.0) + gap = CLUSTER_GAP_SHARE * span + limit = 10.0 * span + cells: set[tuple[int, int]] = set() + for rec in snap.records: + if rec.space != MODEL: + continue + pts = list(rec.points) + if rec.type in ("LINE", "LWPOLYLINE", "POLYLINE") and len(pts) >= 2: + ring = pts + ([pts[0]] if rec.closed else []) + for a, b in zip(ring, ring[1:], strict=False): + _mark_segment(cells, a, b, gap, limit) + elif rec.bbox is not None: + x0, y0, x1, y1 = rec.bbox + corners = [(x0, y0), (x1, y0), (x1, y1), (x0, y1), (x0, y0)] + for a, b in zip(corners, corners[1:], strict=False): + _mark_segment(cells, a, b, gap, limit) + for p in pts[:1]: + cells.add((math.floor(p[0] / gap), math.floor(p[1] / gap))) + seen: set[tuple[int, int]] = set() + bodies: list[list[float]] = [] + for start in sorted(cells): + if start in seen: + continue + seen.add(start) + stack, body = [start], [start[0], start[1], start[0], start[1]] + while stack: + cx, cy = stack.pop() + body = [min(body[0], cx), min(body[1], cy), max(body[2], cx), max(body[3], cy)] + for dx in (-1, 0, 1): + for dy in (-1, 0, 1): + n = (cx + dx, cy + dy) + if n in cells and n not in seen: + seen.add(n) + stack.append(n) + bodies.append(body) + merged = True + while merged: + merged = False + for i in range(len(bodies)): + for j in range(i + 1, len(bodies)): + a, b = bodies[i], bodies[j] + if a[0] <= b[2] and b[0] <= a[2] and a[1] <= b[3] and b[1] <= a[3]: + bodies[i] = [min(a[0], b[0]), min(a[1], b[1]), max(a[2], b[2]), max(a[3], b[3])] + del bodies[j] + merged = True + break + if merged: + break + boxes = [(b[0] * gap, b[1] * gap, (b[2] + 1) * gap, (b[3] + 1) * gap) for b in sorted(bodies)] + return gap, boxes + + +def layout_tags(snap: Snapshot, wanted: Iterable[str] = ()) -> dict: + """Tag positions on a drawing, using only tags that are unique where they are read. + + When no tag is written twice the whole model space is read. Otherwise the + drawing is split into clusters (plan copies, sheets, a far outlier) and the + cluster holding the most `wanted` tags is chosen (ties: the one furthest + left, then lowest); a tag written more than once inside it is `ambiguous`, + never picked arbitrarily. ``{"positions": {tag: Pt}, "ambiguous": [...], + "cluster": box | None, "clusters": n}``. + """ + occurrences = tag_occurrences(snap) + wanted = set(wanted) + if all(len(places) == 1 for places in occurrences.values()): + return { + "positions": {tag: places[0]["at"] for tag, places in occurrences.items()}, + "ambiguous": [], + "cluster": None, + "clusters": 1, + } + _gap, boxes = _clusters(snap) + + def inside(p: Pt, box) -> bool: + return box[0] <= p[0] <= box[2] and box[1] <= p[1] <= box[3] + + def weight(box) -> tuple: + count = sum( + 1 + for tag, places in occurrences.items() + if tag in wanted and any(inside(place["at"], box) for place in places) + ) + return (-count, box[0], box[1]) + + chosen = min(boxes, key=weight) if boxes else None + positions, ambiguous = {}, [] + for tag, places in sorted(occurrences.items()): + here = [place for place in places if chosen is None or inside(place["at"], chosen)] + if len(here) == 1: + positions[tag] = here[0]["at"] + elif len(here) > 1: + ambiguous.append(tag) + return { + "positions": positions, + "ambiguous": ambiguous, + "cluster": chosen, + "clusters": len(boxes), + } + + +def scale_stand_in( + pid: Snapshot, layout: Snapshot, *, pid_tags=None, layout_found=None, layout_unit=None +) -> dict: + """Spec §6 on the tags unique in both drawings - the takeoff's stand-in for scale_check. + + The result has `scale_check`'s shape (group U, Task 4): for every pair of + matched tags at least 2 m apart on the layout, ``ratio`` is their P&ID + distance over their layout distance, both in millimetres of each drawing's + inferred unit; ``median``, ``q1``, ``q3``, ``iqr`` and ``within_10pct`` (the + share within ±10 % of the median) describe them. Verdict ``to_scale`` at + >= 80 %, ``schematic`` at <= 50 %, ``partly`` between, ``insufficient`` + with fewer than three matched tags or no pair; ``factor`` is the median + only when ``to_scale``. There are no per-room ``groups`` here. Tag labels + are not equipment centres: the band and the 2 m floor absorb that. + """ + pid_occ = pid_tags if pid_tags is not None else tag_occurrences(pid) + pid_positions = {tag: places[0]["at"] for tag, places in pid_occ.items() if len(places) == 1} + found = layout_found if layout_found is not None else layout_tags(layout, pid_positions) + unit = layout_unit if layout_unit is not None else unit_of(layout) + mm_layout = unit["m_per_unit"] * 1000.0 + mm_pid = unit_of(pid)["m_per_unit"] * 1000.0 + tags = [tag for tag in sorted(pid_positions) if tag in found["positions"]] + matched = {tag: (pid_positions[tag], found["positions"][tag]) for tag in tags} + ambiguous = sorted( + {tag for tag, places in pid_occ.items() if len(places) > 1} | set(found["ambiguous"]) + ) + pairs = [] + for i, a in enumerate(tags): + for b in tags[i + 1 :]: + d_layout = math.dist(matched[a][1], matched[b][1]) * mm_layout + if d_layout >= MIN_PAIR_MM: + d_pid = math.dist(matched[a][0], matched[b][0]) * mm_pid + pairs.append( + {"tags": [a, b], "a_mm": d_pid, "b_mm": d_layout, "ratio": d_pid / d_layout} + ) + out = { + "verdict": "insufficient", + "factor": None, + "median": None, + "q1": None, + "q3": None, + "iqr": None, + "within_10pct": None, + "pair_count": len(pairs), + "pairs": pairs, + "matched": [ + {"tag": tag, "a": list(matched[tag][0]), "b": list(matched[tag][1])} for tag in tags + ], + "ambiguous": ambiguous, + "groups": [], + "notes": ["takeoff stand-in for scale_check (spec §6): no per-room groups"], + } + ratios = [pair["ratio"] for pair in pairs] + if len(tags) < MIN_TAGS or not ratios: + return out + median = statistics.median(ratios) + q1, q3 = ( + statistics.quantiles(ratios, n=4, method="inclusive")[::2] + if len(ratios) >= 2 + else (ratios[0], ratios[0]) + ) + within = sum(1 for r in ratios if abs(r - median) <= SCALE_BAND * median) / len(ratios) + if within >= TO_SCALE_SHARE: + verdict = "to_scale" + elif within <= SCHEMATIC_SHARE: + verdict = "schematic" + else: + verdict = "partly" + out.update( + verdict=verdict, + factor=median if verdict == "to_scale" else None, + median=median, + q1=q1, + q3=q3, + iqr=q3 - q1, + within_10pct=within, + ) + return out + + +# -- rooms ------------------------------------------------------------------- + + +def room_regions(snap: Snapshot, cluster=None) -> dict: + """Room polygons and room labels of a drawing. + + Labels are texts `room_label` reads; polygons are the faces of the wall + layers (`classify_layer` architecture, confidence >= 0.9, straight + segments) that hold a label, found by track F's planar-face finder. Inside + `cluster` only, when one is given. ``{"faces": [(key, Face)], "labels": + [(key, Pt)]}``; the key is the room number, else its name. + """ + + def keep(p: Pt) -> bool: + return cluster is None or ( + cluster[0] <= p[0] <= cluster[2] and cluster[1] <= p[1] <= cluster[3] + ) + + labels = [] + for rec in snap.records: + if rec.space != MODEL or rec.type not in TEXT_TYPES or not rec.text or not rec.points: + continue + found = room_label(rec.text) + if found and keep(rec.points[0]): + key = found.get("number") or found.get("name") or rec.text + labels.append((str(key), rec.points[0])) + walls = set() + for rec in snap.records: + if rec.layer in walls: + continue + info = classify_layer(rec.layer) + if info["discipline"] == "architecture" and info["confidence"] >= WALL_CONFIDENCE: + walls.add(rec.layer) + segments = [] + for rec in snap.records: + if rec.space != MODEL or rec.layer not in walls: + continue + if rec.type not in ("LINE", "LWPOLYLINE", "POLYLINE") or len(rec.points) < 2: + continue + pts = list(rec.points) + ([rec.points[0]] if rec.closed else []) + bulges = list(rec.bulges) + [0.0] * len(pts) + for i, (a, b) in enumerate(zip(pts, pts[1:], strict=False)): + if bulges[i] == 0.0 and keep(a) and keep(b): + segments.append((a, b)) + faces = planar_faces(segments, tol=DEFAULT_TOL) if segments else () + rooms: list[tuple[str, Face]] = [] + taken: set[int] = set() + for key, at in sorted(labels): + face = face_containing(faces, at) + if face is not None and id(face) not in taken: + taken.add(id(face)) + rooms.append((key, face)) + return {"faces": rooms, "labels": sorted(labels)} + + +def _rooms_mode(regions: dict) -> str: + if regions["faces"]: + return "polygons" + return "nearest_label" if regions["labels"] else "none" + + +def room_at(regions: dict, p: Pt) -> str: + """The room a point stands in; else the nearest room label; else ''.""" + face = face_containing([face for _key, face in regions["faces"]], p) + if face is not None: + return next(key for key, f in regions["faces"] if f is face) + if regions["labels"]: + return min(regions["labels"], key=lambda item: (math.dist(item[1], p), item[0]))[0] + return "" + + +def apportion(legs, regions: dict) -> dict[str, float]: + """Length of each orthogonal leg inside each room polygon (drawing units). + + A leg is cut at every crossing with a room outline or hole; each piece goes + to the room its midpoint stands in, or to '' outside every room. Nothing is + counted twice: the pieces add up to the leg. + """ + faces = [face for _key, face in regions["faces"]] + key_of = {id(face): key for key, face in regions["faces"]} + out: dict[str, float] = {} + for a, b in legs: + length = math.dist(a, b) + if length == 0.0: + continue + cuts = {0.0, 1.0} + for face in faces: + for ring in (face.loop, *face.holes): + for i in range(len(ring)): + t = _cut(a, b, ring[i], ring[(i + 1) % len(ring)]) + if t is not None: + cuts.add(t) + ts = sorted(cuts) + for t0, t1 in zip(ts, ts[1:], strict=False): + if t1 - t0 <= 0.0: + continue + tm = (t0 + t1) / 2.0 + mid = (a[0] + (b[0] - a[0]) * tm, a[1] + (b[1] - a[1]) * tm) + face = face_containing(faces, mid) + key = key_of[id(face)] if face is not None else "" + out[key] = out.get(key, 0.0) + (t1 - t0) * length + return out + + +def _cut(a: Pt, b: Pt, c: Pt, d: Pt) -> float | None: + """Parameter along a->b where it crosses segment c-d, or None.""" + rx, ry = b[0] - a[0], b[1] - a[1] + sx, sy = d[0] - c[0], d[1] - c[1] + denom = rx * sy - ry * sx + if denom == 0.0: + return None + qx, qy = c[0] - a[0], c[1] - a[1] + t = (qx * sy - qy * sx) / denom + u = (qx * ry - qy * rx) / denom + if 0.0 < t < 1.0 and 0.0 <= u <= 1.0: + return t + return None + + +# -- pipe rows --------------------------------------------------------------- + + +def _choose_source(requested: str, layout, scale: dict, force: bool) -> tuple[str, str]: + verdict = scale.get("verdict") + if requested == "layout": + if layout is None: + raise ValueError("length_source='layout' needs a layout drawing; none was given") + return "layout", "requested" + if requested == "pid": + if layout is not None and verdict == "schematic" and not force: + raise ValueError( + "length_source='pid': the scale check calls this P&ID schematic - " + f"{scale['within_10pct']:.0%} of {scale['pair_count']} tag pairs lie within " + f"±10 % of the median ratio {scale['median']:.6g} (IQR {scale['iqr']:.6g}); " + "a length measured on it is drawn line, not pipe. Pass force=True to measure " + "on it anyway, or use length_source='layout'." + ) + return "pid", "forced" if layout is not None and verdict == "schematic" else "requested" + if layout is None: + return "pid", "no_layout" + if verdict == "to_scale": + return "pid", "auto_to_scale" + return "layout", f"auto_{verdict}" + + +def pipe_rows( + network, + pid: Snapshot, + layout: Snapshot | None, + *, + scale=None, + services=DEFAULT_SERVICES, + allowance=0.20, + vertical_allowance=0.0, + length_source="auto", + force=False, +) -> dict: + """Takeoff rows by room x service x diameter, with the runs, checks and method behind them. + + `network` is `build_network(pid)`'s result. `scale` is a scale-check + result (`scale_check`'s shape); omitted, the stand-in runs when a layout + is given. Refused by name before anything is measured: an unknown service + or length source, an allowance outside 0-1, a negative vertical + allowance, `length_source='layout'` without a layout, and + `length_source='pid'` on a P&ID the scale check calls schematic unless + `force=True` (the refusal quotes the statistics). + + Status per run: **A** every end on a tag placed on the layout and every + piece's diameter read directly; **B** a diameter carried by continuity or + unassigned; **C** not routable - an end on no equipment, fewer than two + tags, a tag missing from (or written twice on) the layout. A C run goes to + `control` with its schematic length only, is counted in the totals and is + flagged. A run holding several diameters shares its length among them in + proportion to their drawn length on the P&ID. + """ + options = check_pipe_options( + services=services, + allowance=allowance, + vertical_allowance=vertical_allowance, + length_source=length_source, + ) + allowance = options["allowance"] + vertical = options["vertical_allowance"] + pid_unit = unit_of(pid) + layout_unit = unit_of(layout) if layout is not None else None + wanted = {tag for run in network["runs"] for tag in run.tags} + found = layout_tags(layout, wanted) if layout is not None else None + if scale is None: + scale = ( + scale_stand_in(pid, layout, layout_found=found, layout_unit=layout_unit) + if layout is not None + else {"verdict": "not_checked", "pair_count": 0, "matched": [], "ambiguous": []} + ) + source, reason = _choose_source(length_source, layout, scale, bool(force)) + scale_verified = source == "layout" or scale.get("verdict") == "to_scale" + positions = found["positions"] if found else {} + ambiguous = set(found["ambiguous"]) if found else set() + regions = ( + room_regions(layout, found["cluster"]) + if found is not None and source == "layout" + else room_regions(pid) + ) + m_pid = pid_unit["m_per_unit"] + m_layout = layout_unit["m_per_unit"] if layout_unit else None + + rows: dict[tuple[str, str, str], dict] = {} + runs_out, control, excluded = [], [], {} + for run in network["runs"]: + if run.service not in options["services"]: + excluded[run.service] = excluded.get(run.service, 0) + 1 + continue + ends = network["attachments"].get(run.id, []) + drawn = sum(edge["length"] for edge in run.edges) + schematic_m = drawn * m_pid + parts: dict[str, dict] = {} + for edge in run.edges: + part = parts.setdefault( + edge["diameter"] or UNASSIGNED, + {"drawn": 0.0, "direct": 0.0, "continuity": 0.0, "unassigned": 0.0}, + ) + part["drawn"] += edge["length"] + part[edge["diameter_source"]] += edge["length"] + why = None + if any(end["tag"] is None for end in ends) or not ends: + why = "end_not_on_equipment" + elif len(run.tags) < 2: + why = "single_tag" + elif source == "layout": + missing = [tag for tag in run.tags if tag not in positions] + if missing: + why = ( + "tag_ambiguous" if any(t in ambiguous for t in missing) else "tag_not_on_layout" + ) + detail = { + "id": run.id, + "service": run.service, + "tags": list(run.tags), + "diameters": {d: p["drawn"] * m_pid for d, p in sorted(parts.items())}, + "schematic_m": schematic_m, + "layout_m": None, + "vertical_m": 0.0, + "length_m": None, + "rooms": {}, + "status": "C", + "reason": why, + } + runs_out.append(detail) + for diameter, part in sorted(parts.items()): + if diameter == UNASSIGNED: + control.append( + { + "run": run.id, + "kind": "unassigned_diameter", + "service": run.service, + "tags": list(run.tags), + "schematic_m": part["drawn"] * m_pid, + "reason": "no_diameter_label", + } + ) + if why is not None: + control.append( + { + "run": run.id, + "kind": "unroutable", + "service": run.service, + "tags": list(run.tags), + "schematic_m": schematic_m, + "reason": why, + } + ) + continue + if source == "layout": + points = [positions[tag] for tag in run.tags] + length_units, tree = rmst(points) + if regions["faces"]: + legs = [leg for i, j in tree for leg in route_legs(points[i], points[j])] + rooms = {key: value * m_layout for key, value in apportion(legs, regions).items()} + else: + centre = ( + sum(p[0] for p in points) / len(points), + sum(p[1] for p in points) / len(points), + ) + rooms = {room_at(regions, centre): length_units * m_layout} + for _i, j in tree: + key = room_at(regions, points[j]) + rooms[key] = rooms.get(key, 0.0) + vertical + detail["layout_m"] = length_units * m_layout + detail["vertical_m"] = vertical * len(tree) + length_m = detail["layout_m"] + detail["vertical_m"] + else: + if layout is not None and scale.get("verdict") == "to_scale": + length_m = schematic_m / scale["factor"] + else: + length_m = schematic_m + centre = ( + sum(end["at"][0] for end in ends) / len(ends), + sum(end["at"][1] for end in ends) / len(ends), + ) + rooms = {room_at(regions, centre): length_m} + detail["length_m"] = length_m + detail["rooms"] = rooms + status = ( + "A" + if all(p["continuity"] == 0.0 and p["unassigned"] == 0.0 for p in parts.values()) + else "B" + ) + detail["status"] = status + for diameter, part in parts.items(): + share = part["drawn"] / drawn if drawn else 0.0 + for room, room_m in rooms.items(): + x = room_m * share + row = rows.setdefault( + (room, run.service, diameter), + { + "room": room, + "service": run.service, + "diameter": diameter, + "direct_m": 0.0, + "continuity_m": 0.0, + "unassigned_m": 0.0, + "net_m": 0.0, + "schematic_m": 0.0, + "status": "A", + "runs": [], + }, + ) + row["net_m"] += x + row["direct_m"] += x * part["direct"] / part["drawn"] + row["continuity_m"] += x * part["continuity"] / part["drawn"] + row["unassigned_m"] += x * part["unassigned"] / part["drawn"] + row["schematic_m"] += schematic_m * share * (room_m / length_m if length_m else 0.0) + if part["continuity"] or part["unassigned"]: + row["status"] = "B" + if run.id not in row["runs"]: + row["runs"].append(run.id) + + order = {service: i for i, service in enumerate(SERVICES)} + table = [] + for key in sorted(rows, key=lambda k: (k[0], order.get(k[1], len(order)), k[2])): + row = rows[key] + row["allowance"] = allowance + row["allowance_m"] = row["net_m"] * allowance + row["total_m"] = row["net_m"] + row["allowance_m"] + table.append(row) + + routed = sum(row["net_m"] for row in table) + unroutable = sum(c["schematic_m"] for c in control if c["kind"] == "unroutable") + net_total = routed + unroutable + summary = { + name: [ + { + "key": key, + "net_m": sum(r["net_m"] for r in table if r[field] == key), + "total_m": sum(r["total_m"] for r in table if r[field] == key), + } + for key in sorted({r[field] for r in table}) + ] + for name, field in ( + ("by_service", "service"), + ("by_diameter", "diameter"), + ("by_room", "room"), + ) + } + warnings = [u["warning"] for u in (pid_unit, layout_unit) if u and u["warning"]] + if not scale_verified: + warnings.append( + "scale_verified: false - lengths are drawn lengths on a P&ID that no scale check " + "has confirmed; they are not physical pipe lengths" + ) + return { + "kind": "pipe", + "rows": table, + "runs": runs_out, + "control": control, + "summary": summary, + "totals": { + "routed_m": routed, + "unroutable_m": unroutable, + "net_m": net_total, + "allowance": allowance, + "allowance_m": net_total * allowance, + "total_m": net_total * (1.0 + allowance), + "flagged_runs": sum(1 for r in runs_out if r["status"] == "C"), + }, + "length_source": source, + "scale": scale, + "scale_verified": scale_verified, + "units": {"pid": pid_unit, "layout": layout_unit}, + "warnings": warnings, + "method": { + "sources": {"pid": pid.source, "layout": layout.source if layout else None}, + "length_source": source, + "requested": length_source, + "reason": reason, + "allowance": allowance, + "vertical_allowance": vertical, + "services": list(options["services"]), + "excluded_services": excluded, + "tol": network["stats"]["tol"], + "label_search": network["stats"]["label_search"], + "overlap_length_m": network["stats"]["overlap_length"] * m_pid, + "route": "rectilinear MST of the tag positions; each connection x first, then y", + "rooms": _rooms_mode(regions), + }, + } +``` + +- [ ] **Step 8: Run it to verify it passes** + +Run: `uv run --frozen pytest tests/test_understand_route.py tests/test_understand_takeoff.py tests/test_understand_network.py -q` +Expected: PASS — `71 passed` (6 + 27 + 38). The hand-built tests guard their vocabulary assumptions (`PRODUCT must be a product layer`, `WALLS must be a wall layer`, `the vocabulary must read 'ROOM 101' as a room label`); if Task 1 spells those differently, change the test's input text, never the assertion, and record it. + +- [ ] **Step 9: Gates** + +``` +uv run --frozen ruff check engineering/understand/route.py engineering/understand/takeoff.py tests/test_understand_route.py tests/test_understand_takeoff.py +uv run --frozen ruff format engineering/understand/route.py engineering/understand/takeoff.py tests/test_understand_route.py tests/test_understand_takeoff.py +uv run --frozen pytest -q --deselect tests/test_release_consistency.py::test_document_numbers_match_the_tree +``` + +Expected: ruff clean; the suite green apart from the deselected known red (record it). + +- [ ] **Step 10: Commit** + +``` +git add engineering/understand/route.py engineering/understand/takeoff.py tests/test_understand_route.py tests/test_understand_takeoff.py +git -c user.name="Umut CAN" -c user.email="umutcan.edizaslan@gmail.com" commit -m "$(cat <<'MSG' +feat(understand): orthogonal lengths and pipe takeoff rows + +route.py: Manhattan distance, the rectilinear minimum spanning tree (Prim +over |dx| + |dy|, exact on the points given) and the x-then-y route. + +pipe_rows turns the P&ID's runs into rows by room x service x diameter. +length_source auto measures on the P&ID only when the scale check calls it +to scale; otherwise each run is the RMST of its tags' layout positions plus +vertical_allowance per connection, with the drawn length kept beside it as +schematic_m. length_source=pid on a schematic P&ID is refused with the +statistics unless force=True. Status A / B / C; a C run goes to control +with its schematic length, counted and flagged. Routed lengths are shared +among the layout rooms their route crosses (track F's face finder); the +allowance is its own column. Lengths are metres from the unit the geometry +implies: a layout declaring inches over millimetres is read in mm and warned +about, never multiplied by 25.4. + +The takeoff carries narrow stand-ins for the unit inference, the plan-copy +clusters and the scale check (group U's modules are on another branch); a +scale result passed in is used first. Tags written twice in the chosen +cluster are ambiguous, never picked. + +Co-Authored-By: Claude Opus 5.5 (1M context) +Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd +MSG +)" +``` + +--- + +### Task 7: The takeoff workbook and the pipe_takeoff tool + +**Files:** +- Create: `engineering/understand/report.py` +- Create: `engineering/understand/draw.py` +- Modify: `server.py` (open `# ── SECTION 27: Plant Takeoffs (1 tool)` immediately above the `# ── RESOURCES ──` banner pair, with `pipe_takeoff`) +- Modify: `discovery/aliases.py` (the `pipe_takeoff` record) +- Modify: `tests/test_tool_registry.py` (group snapshot 241 -> 242, `analysis` 12 -> 13) +- Modify: `docs/tool-inventory.json` (regenerated) +- Modify: `README.md` (release-snapshot tool counts 241 -> 242, advertised 236 -> 237) +- Test: `tests/test_understand_report.py` +- Test: `tests/test_understand_tools.py` + +`pyproject.toml` and `uv.lock` stay untouched: Task 1 added the `office` extra. + +**Interfaces:** +- Consumes: Task 6 — `pipe_rows(...) -> dict` and its keys, `check_pipe_options(...)`; Task 5 — `build_network(...)`; Task 1 — `take_snapshot(backend, path: str | None = None) -> Snapshot` (async; a missing, unreadable or oversize file raises `ValueError` or `OSError`, the oversize refusal naming `MAX_DXF_BYTES`); `engineering.sheet.extract.xlsx_available() -> bool`; `backends.capability.UnsupportedCapabilityError(capability, error).to_dict()`; `security.validate_path`; server helpers `cad_tool`, `_backend`, `ToolError`. +- Produces: in `engineering/understand/report.py` — `write_workbook(result: dict, *, kind: str, lang: str, path: str) -> dict` (`{"xlsx": path | None, "csv": [paths], "refused": {...} | None}` — the skeleton's shape), `LANGS = ("tr", "en", "ru")`, `SHEETS = ("Metraj", "Özet", "Kontrol", "Metodoloji")`, `CSV_SLUGS`, `LABELS` (`{key: (tr, en, ru)}`), `TEXTS`, `check_lang(lang) -> str`, and the private builder registry `_BUILDERS = {"pipe": _pipe_sheets}` Task 8 extends. In `engineering/understand/draw.py` — `async run_pipe_takeoff(backend, *, pid_path=None, layout_path=None, output=None, lang="en", services=None, allowance=0.20, vertical_allowance=0.0, length_source="auto", force=False, layers=None, tol=1.0) -> dict` (`pipe_rows`'s dict plus `files`). The MCP tool `pipe_takeoff`. + +The sheet names stay those of the user's own workbook in every language; the headers, service names and method sentences follow `lang`. CSV is one file per sheet (`_metraj.csv`, `_ozet`, `_kontrol`, `_metodoloji`), UTF-8 with a byte-order mark, a point as decimal separator. Numbers are written rounded to 1 mm. + +- [ ] **Step 1: Write the failing workbook test** + +Create `tests/test_understand_report.py`: + +```python +"""The takeoff workbook: four sheets, three languages, CSV always, XLSX with openpyxl. + +The result written is `pipe_rows` over a hand-built pair: a P&ID with two +runs (T4100-T4101 labelled Ø51, T4101-T4102 unlabelled) and a layout whose +INSUNITS says inches over millimetre geometry, measured on the layout (28 m +and 12 m). +""" + +from __future__ import annotations + +import csv +from pathlib import Path + +import pytest + +import engineering.understand.report as report +from engineering.understand.network import build_network +from engineering.understand.report import LANGS, SHEETS, write_workbook +from engineering.understand.snapshot import EntityRecord, Snapshot +from engineering.understand.takeoff import pipe_rows + +PIPE = "PRODUCT" +WALLS = "WALLS" +SCHEMATIC = { + "verdict": "schematic", + "within_10pct": 0.25, + "pair_count": 12, + "median": 1.3, + "iqr": 0.8, + "factor": None, + "matched": [], + "ambiguous": [], +} + + +def rec(handle, kind, layer, points=(), **extra): + return EntityRecord( + handle=handle, type=kind, layer=layer, space="Model", points=tuple(points), **extra + ) + + +def text(handle, value, at, height, layer="TEXT"): + x, y = at + return rec( + handle, + "TEXT", + layer, + (at,), + text=value, + height=height, + bbox=(x, y, x + height * len(value) * 0.6, y + height), + ) + + +def snap(records, insunits, source): + layers = {r.layer: {"color": 7, "linetype": "Continuous"} for r in records} + return Snapshot( + source=source, + insunits=insunits, + extmin=None, + extmax=None, + layers=layers, + layouts=(), + records=tuple(records), + ) + + +def tank(handle, tag, x0, y0): + return [ + rec( + handle, + "LWPOLYLINE", + "EQUIPMENT", + ((x0, y0), (x0 + 500.0, y0), (x0 + 500.0, y0 + 500.0), (x0, y0 + 500.0)), + closed=True, + bbox=(x0, y0, x0 + 500.0, y0 + 500.0), + ), + text(handle + "T", tag, (x0 + 150.0, y0 + 225.0), 50.0), + ] + + +@pytest.fixture +def result(): + pid = snap( + [ + *tank("A", "T4100", 0.0, 0.0), + *tank("B", "T4101", 3000.0, 0.0), + *tank("C", "T4102", 3000.0, 2000.0), + rec("L1", "LINE", PIPE, ((500.0, 250.0), (3000.0, 250.0))), + rec("L2", "LINE", PIPE, ((3250.0, 500.0), (3250.0, 2000.0))), + text("D1", "Ø51", (1500.0, 260.0), 50.0), + ], + 4, + "pid.dxf", + ) + layout = snap( + [ + rec( + "W", + "LWPOLYLINE", + WALLS, + ((0, 0), (40000, 0), (40000, 20000), (0, 20000)), + closed=True, + bbox=(0.0, 0.0, 40000.0, 20000.0), + ), + rec("X", "LINE", WALLS, ((20000, 0), (20000, 20000))), + text("R1", "ROOM 101", (5000.0, 10000.0), 250.0), + text("R2", "ROOM 102", (30000.0, 10000.0), 250.0), + text("T0", "T4100", (2000.0, 3000.0), 250.0), + text("T1", "T4101", (30000.0, 3000.0), 250.0), + text("T2", "T4102", (30000.0, 15000.0), 250.0), + ], + 1, + "layout.dxf", + ) + return pipe_rows(build_network(pid, layers=[PIPE]), pid, layout, scale=SCHEMATIC) + + +METRAJ_HEADERS = { + "tr": [ + "Mahal", + "Hizmet", + "Çap", + "Doğrudan (m)", + "Süreklilik (m)", + "Atanmamış (m)", + "Net (m)", + "Pay oranı", + "Pay (m)", + "Paylı toplam (m)", + "Şematik (m)", + "Durum", + "Hatlar", + ], + "en": [ + "Room", + "Service", + "Diameter", + "Direct (m)", + "Continuity (m)", + "Unassigned (m)", + "Net (m)", + "Allowance", + "Allowance (m)", + "Total incl. allowance (m)", + "Schematic (m)", + "Status", + "Runs", + ], + "ru": [ + "Помещение", + "Среда", + "Диаметр", + "Прямо (м)", + "По непрерывности (м)", + "Не назначено (м)", + "Нетто (м)", + "Запас", + "Запас (м)", + "Итого с запасом (м)", + "По схеме (м)", + "Статус", + "Линии", + ], +} +TOTAL = {"tr": "TOPLAM", "en": "TOTAL", "ru": "ИТОГО"} + + +def read_csv(path): + with open(path, encoding="utf-8-sig", newline="") as handle: + return list(csv.reader(handle)) + + +@pytest.mark.parametrize("lang", LANGS) +def test_csv_sheets_carry_the_headers_of_each_language(result, tmp_path, lang): + files = write_workbook(result, kind="pipe", lang=lang, path=str(tmp_path / "takeoff.xlsx")) + names = [Path(p).name for p in files["csv"]] + assert names == [ + "takeoff_metraj.csv", + "takeoff_ozet.csv", + "takeoff_kontrol.csv", + "takeoff_metodoloji.csv", + ] + metraj = read_csv(files["csv"][0]) + assert metraj[0] == METRAJ_HEADERS[lang] + total = metraj[-1] + assert total[0] == TOTAL[lang] + assert float(total[6]) == pytest.approx(40.0) # 28 m + 12 m net + assert float(total[9]) == pytest.approx(48.0) # with the 20 % allowance + + +def test_the_workbook_has_the_four_sheets_and_numbers_stay_numbers(result, tmp_path): + openpyxl = pytest.importorskip("openpyxl") + files = write_workbook(result, kind="pipe", lang="tr", path=str(tmp_path / "takeoff.xlsx")) + assert files["refused"] is None + book = openpyxl.load_workbook(files["xlsx"]) + assert book.sheetnames == list(SHEETS) == ["Metraj", "Özet", "Kontrol", "Metodoloji"] + rows = list(book["Metraj"].iter_rows(values_only=True)) + assert list(rows[0]) == METRAJ_HEADERS["tr"] + assert rows[-1][0] == "TOPLAM" + assert rows[-1][6] == pytest.approx(40.0) + assert isinstance(rows[1][6], (int, float)) # a number cell, not the text '18.0' + assert rows[1][6] == pytest.approx(18.0) # room 101's share of the Ø51 run + + +def test_the_method_sheet_is_generated_from_the_result(result, tmp_path): + files = write_workbook(result, kind="pipe", lang="en", path=str(tmp_path / "t.xlsx")) + method = dict((row[0], row[1]) for row in read_csv(files["csv"][3])[1:] if len(row) == 2) + assert method["Length source"] == "layout" + assert method["Scale check: verdict"] == "schematic" + assert method["Scale check: pairs"] == "12" + assert method["Scale verified"] == "yes" + assert method["Units: layout"] == "INSUNITS in → mm" + assert "x first, then y" in method["Rule: route"] + warnings = [row[1] for row in read_csv(files["csv"][3]) if row[0] == "Warning"] + assert any("INSUNITS declares in" in w for w in warnings) + + +def test_kontrol_lists_the_unassigned_diameter(result, tmp_path): + files = write_workbook(result, kind="pipe", lang="en", path=str(tmp_path / "t.xlsx")) + kontrol = read_csv(files["csv"][2]) + assert kontrol[0] == ["Run", "Kind", "Service", "Tags", "Schematic (m)", "Reason"] + (row,) = kontrol[1:] + assert row[1] == "Diameter unassigned" and row[3] == "T4101, T4102" + assert row[5] == "No diameter label" + + +def test_without_openpyxl_the_csv_is_written_and_xlsx_is_refused(result, tmp_path, monkeypatch): + monkeypatch.setattr(report, "xlsx_available", lambda: False) + files = write_workbook(result, kind="pipe", lang="tr", path=str(tmp_path / "takeoff.xlsx")) + assert files["xlsx"] is None + assert not (tmp_path / "takeoff.xlsx").exists() + assert all(Path(p).exists() for p in files["csv"]) + assert files["refused"]["capability"] == "xlsx_write" + assert files["refused"]["ok"] is False + assert '".[office]"' in files["refused"]["error"] + + +@pytest.mark.parametrize( + ("kwargs", "message"), + [ + ({"kind": "steel"}, "kind: 'steel' unknown; choose from pipe"), + ({"lang": "de"}, "lang: 'de' unknown; choose from tr, en, ru"), + ({"path": "takeoff.ods"}, "path: the workbook is written as .xlsx"), + ], +) +def test_refusals_come_before_any_file(result, tmp_path, kwargs, message): + arguments = {"kind": "pipe", "lang": "en", "path": str(tmp_path / "t.xlsx"), **kwargs} + if "path" in kwargs: + arguments["path"] = str(tmp_path / kwargs["path"]) + with pytest.raises(ValueError, match="^" + message.replace(".", r"\.")): + write_workbook(result, **arguments) + assert list(tmp_path.iterdir()) == [] +``` + +- [ ] **Step 2: Run it to verify it fails** + +Run: `uv run --frozen pytest tests/test_understand_report.py -q` +Expected: FAIL — collection error `ModuleNotFoundError: No module named 'engineering.understand.report'`. + +- [ ] **Step 3: Write the workbook** + +Create `engineering/understand/report.py`: + +```python +"""The takeoff workbook: Metraj, Özet, Kontrol and Metodoloji, as XLSX and as CSV. + +The sheets follow the user's own takeoff workbook (spec §8): *Metraj* holds +the rows, *Özet* the sums, *Kontrol* what needs a person's eye and +*Metodoloji* how every number was reached - generated from the result, never +typed. The sheet names stay as the user's workbook names them; the headers, +the service names and the method text follow `lang` (tr / en / ru). + +CSV is always written, one file per sheet beside the workbook +(``_metraj.csv`` ...), UTF-8 with a byte-order mark so a spreadsheet +opens the Cyrillic and Turkish letters, a point as the decimal separator. XLSX +needs openpyxl (the `office` extra); without it the CSV files are still +written and the result carries the `xlsx_write` refusal instead of a path. +Numbers are written rounded to 1 mm (three decimals of a metre). +""" + +from __future__ import annotations + +import csv +from pathlib import Path + +from backends.capability import UnsupportedCapabilityError +from engineering.sheet.extract import xlsx_available + +__all__ = ["CSV_SLUGS", "LABELS", "LANGS", "SHEETS", "check_lang", "write_workbook"] + +LANGS = ("tr", "en", "ru") +SHEETS = ("Metraj", "Özet", "Kontrol", "Metodoloji") +CSV_SLUGS = {"Metraj": "metraj", "Özet": "ozet", "Kontrol": "kontrol", "Metodoloji": "metodoloji"} +_INDEX = {lang: i for i, lang in enumerate(LANGS)} + +#: Every word the workbook prints, as (tr, en, ru). +LABELS: dict[str, tuple[str, str, str]] = { + # Metraj + "room": ("Mahal", "Room", "Помещение"), + "service": ("Hizmet", "Service", "Среда"), + "diameter": ("Çap", "Diameter", "Диаметр"), + "direct_m": ("Doğrudan (m)", "Direct (m)", "Прямо (м)"), + "continuity_m": ("Süreklilik (m)", "Continuity (m)", "По непрерывности (м)"), + "unassigned_m": ("Atanmamış (m)", "Unassigned (m)", "Не назначено (м)"), + "net_m": ("Net (m)", "Net (m)", "Нетто (м)"), + "allowance": ("Pay oranı", "Allowance", "Запас"), + "allowance_m": ("Pay (m)", "Allowance (m)", "Запас (м)"), + "total_m": ("Paylı toplam (m)", "Total incl. allowance (m)", "Итого с запасом (м)"), + "schematic_m": ("Şematik (m)", "Schematic (m)", "По схеме (м)"), + "status": ("Durum", "Status", "Статус"), + "runs": ("Hatlar", "Runs", "Линии"), + # Özet, Kontrol, Metodoloji + "group": ("Grup", "Group", "Группа"), + "key": ("Anahtar", "Key", "Ключ"), + "run": ("Hat", "Run", "Линия"), + "kind": ("Tür", "Kind", "Вид"), + "tags": ("Etiketler", "Tags", "Теги"), + "reason": ("Neden", "Reason", "Причина"), + "topic": ("Konu", "Topic", "Раздел"), + "value": ("Değer", "Value", "Значение"), + "by_service": ("Hizmete göre", "By service", "По среде"), + "by_diameter": ("Çapa göre", "By diameter", "По диаметру"), + "by_room": ("Mahale göre", "By room", "По помещению"), + "total": ("TOPLAM", "TOTAL", "ИТОГО"), + "routed": ("Yönlendirilen", "Routed", "Трассировано"), + "unroutable_total": ("Yönlendirilemeyen (C)", "Unroutable (C)", "Без трассы (C)"), + "yes": ("evet", "yes", "да"), + "no": ("hayır", "no", "нет"), + # services + "product": ("Ürün", "Product", "Продукт"), + "cip_supply": ("CIP besleme", "CIP supply", "CIP подача"), + "cip_return": ("CIP dönüş", "CIP return", "CIP возврат"), + "glycol_supply": ("Glikol besleme", "Glycol supply", "Гликоль подача"), + "glycol_return": ("Glikol dönüş", "Glycol return", "Гликоль возврат"), + "steam": ("Buhar", "Steam", "Пар"), + "condensate": ("Kondens", "Condensate", "Конденсат"), + "cold_water": ("Soğuk su", "Cold water", "Холодная вода"), + "hot_water": ("Sıcak su", "Hot water", "Горячая вода"), + "ice_water": ("Buzlu su", "Ice water", "Ледяная вода"), + "compressed_air": ("Basınçlı hava", "Compressed air", "Сжатый воздух"), + "drain": ("Drenaj", "Drain", "Дренаж"), + "electrical": ("Elektrik", "Electrical", "Электрика"), + "unknown": ("Bilinmeyen", "Unknown", "Неизвестно"), + # Kontrol kinds and reasons + "unassigned_diameter": ("Çap atanmamış", "Diameter unassigned", "Диаметр не назначен"), + "unroutable": ("Yönlendirilemez", "Unroutable", "Трассировка невозможна"), + "end_not_on_equipment": ( + "Bir uç ekipmanda değil", + "An end is on no equipment", + "Конец не на оборудовании", + ), + "single_tag": ("Tek etiket", "Only one tag", "Только один тег"), + "tag_not_on_layout": ( + "Etiket yerleşimde yok", + "Tag not on the layout", + "Тега нет на планировке", + ), + "tag_ambiguous": ( + "Etiket yerleşimde birden çok kez yazılı", + "Tag written more than once on the layout", + "Тег повторяется на планировке", + ), + "no_diameter_label": ("Çap etiketi yok", "No diameter label", "Нет обозначения диаметра"), + # Metodoloji topics + "source_pid": ("Kaynak: P&ID", "Source: P&ID", "Источник: P&ID"), + "source_layout": ("Kaynak: yerleşim", "Source: layout", "Источник: планировка"), + "length_source": ("Uzunluk kaynağı", "Length source", "Источник длины"), + "choice": ("Seçim gerekçesi", "Why this source", "Почему этот источник"), + "scale_verdict": ("Ölçek kontrolü: sonuç", "Scale check: verdict", "Проверка масштаба: вывод"), + "scale_factor": ( + "Ölçek kontrolü: oran (P&ID mm / yerleşim mm)", + "Scale check: factor (P&ID mm per layout mm)", + "Проверка масштаба: коэффициент (мм P&ID на мм планировки)", + ), + "scale_iqr": ("Ölçek kontrolü: IQR", "Scale check: IQR", "Проверка масштаба: IQR"), + "scale_within": ( + "Ölçek kontrolü: ±%10 içindeki çiftler", + "Scale check: pairs within ±10 %", + "Проверка масштаба: пары в пределах ±10 %", + ), + "scale_pairs": ("Ölçek kontrolü: çift sayısı", "Scale check: pairs", "Проверка масштаба: пар"), + "scale_matched": ( + "Ölçek kontrolü: eşleşen etiketler", + "Scale check: matched tags", + "Проверка масштаба: совпавшие теги", + ), + "scale_ambiguous": ( + "Ölçek kontrolü: belirsiz etiketler", + "Scale check: ambiguous tags", + "Проверка масштаба: неоднозначные теги", + ), + "scale_note": ("Ölçek kontrolü: not", "Scale check: note", "Проверка масштаба: примечание"), + "scale_verified": ("Ölçek doğrulandı", "Scale verified", "Масштаб подтверждён"), + "units_pid": ("Birim: P&ID", "Units: P&ID", "Единицы: P&ID"), + "units_layout": ("Birim: yerleşim", "Units: layout", "Единицы: планировка"), + "rule_route": ("Kural: güzergâh", "Rule: route", "Правило: трасса"), + "rule_vertical": ( + "Kural: bağlantı başına düşey pay (m)", + "Rule: vertical allowance per connection (m)", + "Правило: вертикальный запас на соединение (м)", + ), + "rule_allowance": ("Kural: pay oranı", "Rule: allowance", "Правило: запас"), + "rule_diameter": ("Kural: çap", "Rule: diameter", "Правило: диаметр"), + "rule_service": ("Kural: hizmet", "Rule: service", "Правило: среда"), + "rule_status": ("Kural: durum", "Rule: status", "Правило: статус"), + "rule_rooms": ("Kural: mahaller", "Rule: rooms", "Правило: помещения"), + "rule_multi": ( + "Kural: çok çaplı hat", + "Rule: several diameters", + "Правило: несколько диаметров", + ), + "tol": ("Tolerans (çizim birimi)", "Tolerance (drawing units)", "Допуск (ед. чертежа)"), + "label_search": ( + "Etiket arama mesafesi (çizim birimi)", + "Label search distance (drawing units)", + "Радиус поиска обозначений (ед. чертежа)", + ), + "overlap": ( + "Çift çizilmiş, bir kez sayılan boru (m)", + "Double-drawn pipe counted once (m)", + "Дважды начерченная труба, учтена один раз (м)", + ), + "services": ("Kapsanan hizmetler", "Services taken off", "Учтённые среды"), + "warning": ("Uyarı", "Warning", "Предупреждение"), +} + +#: The method sheet's sentences, as (tr, en, ru). +TEXTS: dict[str, tuple[str, str, str]] = { + "requested": ("çağıran seçti", "chosen by the caller", "выбран вызывающим"), + "forced": ( + "ölçek kontrolü P&ID'yi şematik bulduğu halde istek üzerine (force=True) P&ID'den ölçüldü", + "measured on the P&ID on request (force=True) although the scale check calls it schematic", + "измерено по P&ID по запросу (force=True), хотя проверка масштаба считает его схемой", + ), + "no_layout": ( + "yerleşim verilmedi: P&ID'den ölçüldü, ölçek doğrulanmadı", + "no layout given: measured on the P&ID, scale not verified", + "планировка не задана: измерено по P&ID, масштаб не подтверждён", + ), + "auto_to_scale": ( + "ölçek kontrolü P&ID'yi ölçekli buldu: oran üzerinden P&ID'den ölçüldü", + "the scale check calls the P&ID to scale: measured on it through the factor", + "проверка масштаба считает P&ID масштабным: измерено по нему через коэффициент", + ), + "auto_layout": ( + "ölçek kontrolü P&ID'yi ölçekli bulmadı: yerleşimden ölçüldü", + "the scale check does not call the P&ID to scale: measured on the layout", + "проверка масштаба не считает P&ID масштабным: измерено по планировке", + ), + "scale_note": ( + "Etiketler ekipman merkezi değildir: ±%10 bant ve 2 m alt sınır bunu karşılar.", + "Tag labels are not equipment centres: the ±10 % band and the 2 m floor absorb that.", + "Теги — не центры оборудования: полоса ±10 % и порог 2 м это учитывают.", + ), + "rule_route": ( + "Yerleşim uzunluğu = hattın bağladığı etiket konumlarının dik açılı (Manhattan) en küçük " + "kapsayan ağacı; her bağlantı önce x, sonra y boyunca.", + "Layout length = the rectilinear (Manhattan) minimum spanning tree of the tag positions " + "the run joins; each connection along x first, then y.", + "Длина по планировке = прямоугольное (манхэттенское) минимальное остовное дерево " + "позиций тегов, которые соединяет линия; каждое соединение сначала по x, затем по y.", + ), + "rule_diameter": ( + "Çap etiketleri çizildiği gibi alınır (Ø51, SMS51, DN20 farklı ölçülerdir). Doğrudan: " + "arama mesafesindeki etiket; süreklilik: etiketli parçadan hat boyunca taşınır, " + "redüksiyondan geçmez; atanmamış: ikisi de değil (Kontrol sayfasında).", + "Diameter labels are kept as drawn (Ø51, SMS51 and DN20 are different sizes). Direct: a " + "label within the search distance; continuity: carried along the run from a labelled " + "piece, never across a reducer; unassigned: neither (listed on Kontrol).", + "Обозначения диаметров сохраняются как на чертеже (Ø51, SMS51 и DN20 — разные " + "размеры). Прямо: обозначение в радиусе поиска; по непрерывности: перенос вдоль линии " + "от обозначенного участка, но не через переход; не назначено: ни то, ни другое (лист " + "Kontrol).", + ), + "rule_service": ( + "Hizmet katmandan alınır; hattın yakınındaki besleme / dönüş sözcüğü (ПОДАЧА, ВОЗВРАТ, " + "BESLEME, DÖNÜŞ, SUPPLY, RETURN, AANVOER, RETOUR) tüm hat için onu geçersiz kılar.", + "Service comes from the layer; a supply / return word near the run (ПОДАЧА, ВОЗВРАТ, " + "BESLEME, DÖNÜŞ, SUPPLY, RETURN, AANVOER, RETOUR) overrides it for the whole run.", + "Среда определяется по слою; слово подачи / возврата рядом с линией (ПОДАЧА, ВОЗВРАТ, " + "BESLEME, DÖNÜŞ, SUPPLY, RETURN, AANVOER, RETOUR) переопределяет её для всей линии.", + ), + "rule_status": ( + "A: her uç yerleşimde bulunan bir etikette ve her çap doğrudan. B: süreklilikle ya da " + "atanmamış çap. C: yönlendirilemez - yalnız şematik uzunluk, Kontrol sayfasında, " + "toplama katılır ve işaretlenir.", + "A: every end on a tag placed on the layout and every diameter direct. B: a diameter by " + "continuity or unassigned. C: not routable - schematic length only, on Kontrol, counted " + "in the total and flagged.", + "A: каждый конец на теге, найденном на планировке, и каждый диаметр прямо. B: диаметр " + "по непрерывности или не назначен. C: трассировка невозможна — только длина по схеме, " + "на листе Kontrol, входит в итог и помечена.", + ), + "rule_rooms": ( + "Yönlendirilen uzunluk, güzergâhın geçtiği yerleşim mahallerine paylaştırılır (duvar " + "yüzleri ve mahal etiketlerinden çokgenler); çokgen yoksa hatta en yakın mahal etiketine.", + "A routed length is shared among the layout rooms its route crosses (room polygons from " + "the wall faces and room labels); without polygons, the room label nearest the run.", + "Трассированная длина распределяется по помещениям планировки, через которые проходит " + "трасса (контуры из граней стен и обозначений помещений); без контуров — ближайшему " + "обозначению помещения.", + ), + "rule_multi": ( + "Birden çok çaplı bir hat, uzunluğunu P&ID üzerindeki çizili uzunlukları oranında " + "paylaştırır.", + "A run holding several diameters shares its length among them in proportion to their " + "drawn length on the P&ID.", + "Линия с несколькими диаметрами делит длину между ними пропорционально их длине на " + "схеме P&ID.", + ), +} + + +def check_lang(lang: str) -> str: + """Refuse a language the workbook does not speak, naming the three it does.""" + if lang not in LANGS: + raise ValueError(f"lang: {lang!r} unknown; choose from {', '.join(LANGS)}") + return lang + + +def _t(key: str, lang: str) -> str: + """A label in `lang`; a key with no label (a diameter, a room number) prints as itself.""" + entry = LABELS.get(key) + return entry[_INDEX[lang]] if entry else key + + +def _say(key: str, lang: str) -> str: + """One of the method sheet's sentences in `lang`.""" + return TEXTS[key][_INDEX[lang]] + + +def _num(value): + return None if value is None else round(float(value), 3) + + +def _yes(value, lang: str): + return None if value is None else _t("yes" if value else "no", lang) + + +def _pipe_sheets(result: dict, lang: str) -> dict[str, tuple[list, list]]: + columns = ( + "room", + "service", + "diameter", + "direct_m", + "continuity_m", + "unassigned_m", + "net_m", + "allowance", + "allowance_m", + "total_m", + "schematic_m", + "status", + "runs", + ) + rows = [ + [ + row["room"], + _t(row["service"], lang), + row["diameter"], + _num(row["direct_m"]), + _num(row["continuity_m"]), + _num(row["unassigned_m"]), + _num(row["net_m"]), + row["allowance"], + _num(row["allowance_m"]), + _num(row["total_m"]), + _num(row["schematic_m"]), + row["status"], + ", ".join(row["runs"]), + ] + for row in result["rows"] + ] + totals = result["totals"] + a = totals["allowance"] + rows.append( + [ + _t("routed", lang), + None, + None, + _num(sum(r["direct_m"] for r in result["rows"])), + _num(sum(r["continuity_m"] for r in result["rows"])), + _num(sum(r["unassigned_m"] for r in result["rows"])), + _num(totals["routed_m"]), + a, + _num(totals["routed_m"] * a), + _num(totals["routed_m"] * (1.0 + a)), + _num(sum(r["schematic_m"] for r in result["rows"])), + None, + None, + ] + ) + rows.append( + [ + _t("unroutable_total", lang), + None, + None, + None, + None, + None, + _num(totals["unroutable_m"]), + a, + _num(totals["unroutable_m"] * a), + _num(totals["unroutable_m"] * (1.0 + a)), + _num(totals["unroutable_m"]), + "C", + str(totals["flagged_runs"]), + ] + ) + rows.append( + [ + _t("total", lang), + None, + None, + None, + None, + None, + _num(totals["net_m"]), + a, + _num(totals["allowance_m"]), + _num(totals["total_m"]), + None, + None, + None, + ] + ) + + summary = [] + for group, field in ( + ("by_service", "service"), + ("by_diameter", "diameter"), + ("by_room", "room"), + ): + for item in result["summary"][group]: + key = _t(item["key"], lang) if field == "service" else item["key"] + summary.append([_t(group, lang), key, _num(item["net_m"]), _num(item["total_m"])]) + summary.append([_t("total", lang), None, _num(totals["net_m"]), _num(totals["total_m"])]) + + control = [ + [ + c["run"], + _t(c["kind"], lang), + _t(c["service"], lang), + ", ".join(c["tags"]), + _num(c["schematic_m"]), + _t(c["reason"], lang), + ] + for c in result["control"] + ] + return { + "Metraj": ([_t(c, lang) for c in columns], rows), + "Özet": ([_t(c, lang) for c in ("group", "key", "net_m", "total_m")], summary), + "Kontrol": ( + [_t(c, lang) for c in ("run", "kind", "service", "tags", "schematic_m", "reason")], + control, + ), + "Metodoloji": ([_t("topic", lang), _t("value", lang)], _pipe_method(result, lang)), + } + + +def _unit_text(unit: dict | None) -> str: + if not unit: + return "—" + return f"INSUNITS {unit['declared'] or '—'} → {unit['inferred']}" + + +def _pipe_method(result: dict, lang: str) -> list[list]: + method, scale = result["method"], result["scale"] + reason = method["reason"] + if reason.startswith("auto_") and reason != "auto_to_scale": + reason = "auto_layout" + rows = [ + ["source_pid", method["sources"]["pid"]], + ["source_layout", method["sources"]["layout"] or "—"], + ["length_source", method["length_source"]], + ["choice", _say(reason, lang)], + ["scale_verdict", scale.get("verdict")], + ["scale_factor", None if scale.get("factor") is None else round(float(scale["factor"]), 6)], + ["scale_iqr", None if scale.get("iqr") is None else round(float(scale["iqr"]), 6)], + [ + "scale_within", + None if scale.get("within_10pct") is None else round(float(scale["within_10pct"]), 4), + ], + ["scale_pairs", scale.get("pair_count")], + ["scale_matched", len(scale.get("matched", {}))], + ["scale_ambiguous", ", ".join(scale.get("ambiguous", [])) or "—"], + ["scale_note", _say("scale_note", lang)], + ["scale_verified", _yes(result["scale_verified"], lang)], + ["units_pid", _unit_text(result["units"]["pid"])], + ["units_layout", _unit_text(result["units"]["layout"])], + ["services", ", ".join(_t(s, lang) for s in method["services"])], + ["rule_route", _say("rule_route", lang)], + ["rule_vertical", method["vertical_allowance"]], + ["rule_allowance", method["allowance"]], + ["rule_diameter", _say("rule_diameter", lang)], + ["rule_service", _say("rule_service", lang)], + ["rule_status", _say("rule_status", lang)], + ["rule_rooms", _say("rule_rooms", lang)], + ["rule_multi", _say("rule_multi", lang)], + ["tol", method["tol"]], + ["label_search", round(float(method["label_search"]), 6)], + ["overlap", _num(method["overlap_length_m"])], + ] + rows += [["warning", warning] for warning in result["warnings"]] + return [[_t(topic, lang), value] for topic, value in rows] + + +_BUILDERS = {"pipe": _pipe_sheets} + + +def write_workbook(result: dict, *, kind: str, lang: str, path: str) -> dict: + """Write the takeoff: CSV always, XLSX when openpyxl is installed. + + ``{"xlsx": path | None, "csv": [paths], "refused": {...} | None}``. Refused + before a file is written: a `kind` it builds no sheets for, a `lang` other + than tr / en / ru, and a `path` with a suffix other than .xlsx. Without + openpyxl the CSV files are written and `refused` carries the + `xlsx_write` capability refusal naming the `office` extra. + """ + if kind not in _BUILDERS: + raise ValueError(f"kind: {kind!r} unknown; choose from {', '.join(_BUILDERS)}") + check_lang(lang) + target = Path(path) + if target.suffix.lower() not in ("", ".xlsx"): + raise ValueError( + f"path: the workbook is written as .xlsx with CSV files beside it; got {target.suffix!r}" + ) + sheets = _BUILDERS[kind](result, lang) + xlsx = target.with_suffix(".xlsx") + xlsx.parent.mkdir(parents=True, exist_ok=True) + written = [] + for name, (header, rows) in sheets.items(): + csv_path = xlsx.with_name(f"{xlsx.stem}_{CSV_SLUGS[name]}.csv") + with open(csv_path, "w", newline="", encoding="utf-8-sig") as handle: + writer = csv.writer(handle) + writer.writerow(header) + writer.writerows([["" if cell is None else cell for cell in row] for row in rows]) + written.append(str(csv_path)) + if not xlsx_available(): + refusal = UnsupportedCapabilityError( + "xlsx_write", + "The takeoff workbook needs openpyxl for XLSX, which is not installed. Install it " + 'with: pip install -e ".[office]" (or pip install openpyxl). The same sheets were ' + f"written as CSV: {', '.join(written)}", + ) + return {"xlsx": None, "csv": written, "refused": refusal.to_dict()} + from openpyxl import Workbook + + book = Workbook() + for i, (name, (header, rows)) in enumerate(sheets.items()): + sheet = book.active if i == 0 else book.create_sheet() + sheet.title = name + sheet.append(header) + for row in rows: + sheet.append(row) + book.save(xlsx) + return {"xlsx": str(xlsx), "csv": written, "refused": None} +``` + +- [ ] **Step 4: Run it to verify it passes** + +Run: `uv run --frozen pytest tests/test_understand_report.py -q` +Expected: PASS — `10 passed` (Task 1 installed the `office` extra, so the XLSX read-back runs; in a venv without openpyxl it reports `9 passed, 1 skipped` — record which). + +- [ ] **Step 5: Write the failing tool test** + +Create `tests/test_understand_tools.py`: + +```python +"""pipe_takeoff over the MCP wire, on the synthetic plant pair - never on a client drawing.""" + +from __future__ import annotations + +from pathlib import Path + +import pytest +import pytest_asyncio +from fastmcp import Client + +import server +from tests.fixtures.plant_pair import build_plant_pair + +pytestmark = pytest.mark.asyncio + +CSV_NAMES = [ + "takeoff_metraj.csv", + "takeoff_ozet.csv", + "takeoff_kontrol.csv", + "takeoff_metodoloji.csv", +] + + +@pytest_asyncio.fixture +async def client(monkeypatch): + """In-memory client on the headless engine - never the operator's AutoCAD.""" + monkeypatch.setenv("AUTOCAD_MCP_BACKEND", "ezdxf") + async with Client(server.mcp) as connected: + status = (await connected.call_tool("system_status", {})).structured_content or {} + assert status.get("backend") == "ezdxf", "these tests must never touch live AutoCAD" + yield connected + + +@pytest.fixture +def plant(tmp_path): + return build_plant_pair(tmp_path) + + +async def test_pipe_takeoff_measures_on_the_layout_and_writes_the_workbook(client, plant, tmp_path): + output = tmp_path / "takeoff.xlsx" + arguments = { + "pid_path": plant["pid"], + "layout_path": plant["layout"], + "output": str(output), + "lang": "tr", + } + result = (await client.call_tool("pipe_takeoff", arguments)).structured_content + assert result["length_source"] == "layout" + assert result["scale"]["verdict"] == plant["scale_verdict"] + files = result["files"] + assert [Path(p).name for p in files["csv"]] == CSV_NAMES + assert all(Path(p).exists() for p in files["csv"]) + if files["xlsx"] is None: + assert files["refused"]["capability"] == "xlsx_write" + else: + assert Path(files["xlsx"]) == output + assert output.exists() + + +async def test_pid_lengths_on_the_schematic_pid_are_refused_without_force(client, plant): + arguments = {"pid_path": plant["pid"], "layout_path": plant["layout"], "length_source": "pid"} + result = await client.call_tool("pipe_takeoff", arguments, raise_on_error=False) + assert result.is_error is True + text = result.content[0].text + assert "the scale check calls this P&ID schematic" in text + assert "force=True" in text + + +async def test_force_measures_on_the_pid_and_marks_it_unverified(client, plant): + arguments = { + "pid_path": plant["pid"], + "layout_path": plant["layout"], + "length_source": "pid", + "force": True, + } + result = (await client.call_tool("pipe_takeoff", arguments)).structured_content + assert (result["length_source"], result["scale_verified"]) == ("pid", False) + assert result["files"] is None # no output named, nothing written + + +async def test_an_unknown_language_is_refused_before_anything_is_read(client, tmp_path): + arguments = {"pid_path": str(tmp_path / "never_read.dxf"), "lang": "de"} + result = await client.call_tool("pipe_takeoff", arguments, raise_on_error=False) + assert result.is_error is True + assert "lang: 'de' unknown; choose from tr, en, ru" in result.content[0].text + + +async def test_the_drawings_are_never_modified(client, plant, tmp_path): + before = {key: Path(plant[key]).read_bytes() for key in ("pid", "layout")} + arguments = { + "pid_path": plant["pid"], + "layout_path": plant["layout"], + "output": str(tmp_path / "t.xlsx"), + } + await client.call_tool("pipe_takeoff", arguments) + assert {key: Path(plant[key]).read_bytes() for key in ("pid", "layout")} == before + + +async def test_the_tool_says_it_writes_a_file_and_destroys_nothing(client): + tools = {tool.name: tool for tool in await client.list_tools()} + annotations = tools["pipe_takeoff"].annotations + assert annotations.readOnlyHint is False + assert annotations.destructiveHint is False +``` + +- [ ] **Step 6: Run it to verify it fails** + +Run: `uv run --frozen pytest tests/test_understand_tools.py -q` +Expected: FAIL — `6 failed`: the `call_tool` tests with `fastmcp.exceptions.ToolError: Not found: Unknown tool: 'pipe_takeoff'` (or, with `raise_on_error=False`, an assertion on that text), the annotations test with `KeyError: 'pipe_takeoff'`. + +- [ ] **Step 7: Write the async layer** + +Create `engineering/understand/draw.py`: + +```python +"""The async layer of track H's takeoffs: read the drawings, run the pure engine, write the workbook. + +The only module of the takeoffs that touches a backend, and it only reads: +`take_snapshot` gives one bulk read of each drawing (a file, or the current +document), the pure modules compute in a worker thread so a 35,000-entity +P&ID does not stall the server, and the workbook is a file beside the path the +caller names. The drawings themselves are never modified. +""" + +from __future__ import annotations + +import asyncio +import math + +from engineering.understand.network import build_network +from engineering.understand.report import check_lang, write_workbook +from engineering.understand.snapshot import take_snapshot +from engineering.understand.takeoff import check_pipe_options, pipe_rows + +__all__ = ["run_pipe_takeoff"] + + +def _check_tol(tol) -> float: + try: + value = float(tol) + except (TypeError, ValueError): + raise ValueError(f"tol: expected a number, got {tol!r}") from None + if not math.isfinite(value) or value <= 0.0: + raise ValueError(f"tol: must be a finite number greater than zero, got {tol!r}") + return value + + +async def run_pipe_takeoff( + backend, + *, + pid_path: str | None = None, + layout_path: str | None = None, + output: str | None = None, + lang: str = "en", + services=None, + allowance: float = 0.20, + vertical_allowance: float = 0.0, + length_source: str = "auto", + force: bool = False, + layers=None, + tol: float = 1.0, +) -> dict: + """`pipe_rows` over a P&ID and an optional layout, plus the workbook when `output` is named. + + Every option is checked before a drawing is read. `pid_path=None` reads the + current document as the P&ID. The result is `pipe_rows`'s dict with + ``files`` added: `write_workbook`'s answer, or None when no `output` was named. + """ + check_lang(lang) + options = check_pipe_options( + services=services, + allowance=allowance, + vertical_allowance=vertical_allowance, + length_source=length_source, + ) + tol = _check_tol(tol) + if length_source == "layout" and not layout_path: + raise ValueError("length_source='layout' needs a layout drawing; none was given") + pid = await take_snapshot(backend, pid_path) + layout = await take_snapshot(backend, layout_path) if layout_path else None + + def compute() -> dict: + network = build_network(pid, layers=layers, tol=tol) + return pipe_rows( + network, + pid, + layout, + services=options["services"], + allowance=options["allowance"], + vertical_allowance=options["vertical_allowance"], + length_source=options["length_source"], + force=bool(force), + ) + + result = await asyncio.to_thread(compute) + result["files"] = ( + await asyncio.to_thread(write_workbook, result, kind="pipe", lang=lang, path=output) + if output + else None + ) + return result +``` + +The skeleton names `draw.py` as track H's async layer; this task creates it with the takeoffs' function only (group D keeps its diff wiring in its own modules), so the merge meets no second copy. + +- [ ] **Step 8: Register pipe_takeoff (SECTION 27)** + +In `server.py`, find the end of SECTION 25 — `arch_plan_from_spec`'s last lines ` except ValueError as exc:` / ` raise ToolError(str(exc)) from exc` — and the three-line banner that follows it: + +``` +# --------------------------------------------------------------------------- +# ── RESOURCES ─────────────────────────────────────────────────────────────── +# --------------------------------------------------------------------------- +``` + +Insert the block below between them, leaving two blank lines on each side. SECTION 26 does not exist on this branch (groups U and D open it); the merge puts it above this block (skeleton, Merge notes: the RESOURCES seam). The banner says `(1 tool)` because `tests/test_release_consistency.py::test_section_header_counts_match_decorators` counts the decorators under it; Task 8 makes it `(2 tools)`. + +```python +# --------------------------------------------------------------------------- +# ── SECTION 27: Plant Takeoffs (1 tool) ───────────────────────────────────── +# --------------------------------------------------------------------------- + + +@cad_tool( + summary="Pipe takeoff from a P&ID and its layout: rows by room, service and diameter.", + cost="safe", +) +@mcp.tool( + annotations={ + "title": "Plant: Pipe Takeoff", + "readOnlyHint": False, + "destructiveHint": False, + }, + tags={"plant", "analysis"}, +) +async def pipe_takeoff( + pid_path: Annotated[ + str | None, + Field( + default=None, + description="The P&ID (DXF headlessly; DXF or DWG on a live seat). Omit to read " + "the current drawing as the P&ID.", + ), + ] = None, + layout_path: Annotated[ + str | None, + Field( + default=None, + description="The layout of the same plant. Omitted, every length is the P&ID's " + "drawn length and is marked scale_verified: false.", + ), + ] = None, + output: Annotated[ + str | None, + Field( + default=None, + description="Workbook to write (.xlsx); the four sheets are also written as CSV " + "beside it. Omit to return the rows only.", + ), + ] = None, + lang: Annotated[ + str, Field(default="en", description="Headers and method text: tr | en | ru.") + ] = "en", + services: Annotated[ + list[str] | None, + Field( + default=None, + description="Services to take off; default product, cip_supply, cip_return. Also: " + "glycol_supply, glycol_return, steam, condensate, cold_water, hot_water, " + "ice_water, compressed_air, drain, electrical, unknown.", + ), + ] = None, + allowance: Annotated[ + float, + Field( + default=0.20, + ge=0.0, + le=1.0, + description="Allowance as a share of the net length (0.20 = 20 %); its own column.", + ), + ] = 0.20, + vertical_allowance: Annotated[ + float, + Field( + default=0.0, + ge=0.0, + description="Metres added per connection for rises and drops (layout lengths).", + ), + ] = 0.0, + length_source: Annotated[ + str, + Field( + default="auto", + description="auto | pid | layout. auto measures on the P&ID only when the scale " + "check calls it to scale, else on the layout.", + ), + ] = "auto", + force: Annotated[ + bool, + Field( + default=False, + description="With length_source='pid', measure on a P&ID the scale check calls " + "schematic anyway.", + ), + ] = False, + layers: Annotated[ + list[str] | None, + Field( + default=None, + description="P&ID layers holding the lines; default every layer the vocabulary " + "files under piping or electrical with confidence >= 0.9.", + ), + ] = None, + tol: Annotated[ + float, + Field( + default=1.0, + gt=0.0, + description="Line ends closer than this (P&ID drawing units) are one junction.", + ), + ] = 1.0, + ctx: Context = None, +) -> dict: + """A pipe takeoff: topology from the P&ID, lengths from the layout, rows by room x service x diameter. + + The P&ID gives the runs (junctions merged, tees split, a line drawn twice + counted once, valves and reducers bridged), each run's service (the layer, + overridden by a supply / return word near the line), its diameters exactly + as labelled (direct, carried by continuity - never across a reducer - or + unassigned) and the equipment tags at its ends. The length comes from the + layout unless the scale check calls the P&ID to scale: the rectilinear + minimum spanning tree of the run's tag positions on the layout, each + connection along x first then y, plus `vertical_allowance` per connection. + The P&ID's drawn length stays beside it as `schematic_m`, never as the + answer. Lengths are metres from the unit the geometry implies; an INSUNITS + that disagrees is a warning, never a factor. Status A / B / C per run: a C + run (an end on no equipment, a tag missing from or written twice on the + layout) is on the Kontrol sheet with its schematic length, counted in the + total and flagged. + + Read-only on both drawings. With `output` it writes one workbook - Metraj, + Özet, Kontrol, Metodoloji - as XLSX and, always, as CSV beside it. + + Refused before a drawing is read: `lang` outside tr / en / ru, an unknown + service or length source, an allowance outside 0-1, a negative vertical + allowance, a non-positive tolerance, and `length_source='layout'` without + a layout. A missing, unreadable or oversize drawing is refused by the + reader, which names MAX_DXF_BYTES. Refused after the scale check: + `length_source='pid'` on a P&ID it calls schematic, unless `force=True` + (the refusal quotes the statistics). Without openpyxl the XLSX is not + written and `files.refused` carries `capability: "xlsx_write"` naming the + `office` extra; the CSV files are written all the same. + """ + from engineering.understand.draw import run_pipe_takeoff + + pid = str(validate_path(pid_path)) if pid_path else None + layout = str(validate_path(layout_path)) if layout_path else None + target = str(validate_path(output, allow_write=True)) if output else None + await ctx.info(f"pipe_takeoff: {pid or 'current drawing'} / {layout or 'no layout'}") + try: + return await run_pipe_takeoff( + _backend(ctx), + pid_path=pid, + layout_path=layout, + output=target, + lang=lang, + services=services, + allowance=allowance, + vertical_allowance=vertical_allowance, + length_source=length_source, + force=force, + layers=layers, + tol=tol, + ) + except (ValueError, OSError) as exc: + raise ToolError(str(exc)) from exc +``` + +The tags are `plant` and `analysis`. `plant` is deliberately **not** added to `_GROUP_TAG_PRIORITY`, `_GROUP_TAG_LABELS`, `TOOL_PACK_NAMES` or `PACK_TOOL_NAMES` here: the `plant` pack, its group and the lean list are Task 11's (skeleton, File Structure), and editing those lists on three branches would conflict three ways — the track F precedent. Until then `pipe_takeoff` files under `analysis` and is core-pack. + +- [ ] **Step 9: The alias record** + +In `discovery/aliases.py`, add this record at the end of `TOOL_ALIASES`, directly after the `"arch_plan_from_spec"` record and before the dictionary's closing `}` (additive at the merge). `acad` stays empty: AutoCAD has no takeoff command, and the corpus rule is that a wrong command misroutes a search. `pipe list` is deliberately absent — it is `pid_line_list`'s synonym. + +```python + "pipe_takeoff": ToolAliases( + acad=(), + synonyms=( + "pipe takeoff", + "piping takeoff", + "takeoff", + "pipe quantities", + "how much pipe", + "pipe lengths from a p&id", + "bill of quantities", + "boq", + "metraj", + "boru metrajı", + "boru metraji", + "metre cetveli", + "boru listesi", + "спецификация", + "ведомость", + "спецификация трубопроводов", + "ведомость трубопроводов", + ), + ), +``` + +- [ ] **Step 10: Run the tool and alias tests to verify they pass** + +Run: `uv run --frozen pytest tests/test_understand_tools.py tests/test_discovery_aliases.py -q` +Expected: PASS — the 6 tool tests and every alias gate (`every registered tool has an alias record`, lowercase trimmed synonyms, no AutoCAD command claimed twice). + +- [ ] **Step 11: Move the registration snapshot** + +In `tests/test_tool_registry.py`, `test_tool_group_sizes_are_unchanged`: + +- the docstring's first line `"""Frozen snapshot of the surface (241 tools, 26 groups).` becomes `"""Frozen snapshot of the surface (242 tools, 26 groups).`; +- directly after the docstring line ` pre-track-F value.` (the last line before the closing `"""`), add: + +``` + + `analysis` moved 12 -> 13 when track H's SECTION 27 opened with + `pipe_takeoff` on the network branch. It is tagged `plant` and + `analysis`, and `plant` is not in `_GROUP_TAG_PRIORITY` on that + branch, so it files under `analysis`. This snapshot is branch-local: + the track H integration task gives `plant` its pack and its group + once every branch has landed. +``` + +- `"analysis": 12,` becomes `"analysis": 13,`; +- `assert sum(sizes.values()) == 241` becomes `assert sum(sizes.values()) == 242`. + +`tests/test_tool_packs.py` needs no change: the group count stays 26 and the pack lists are untouched. + +- [ ] **Step 12: Regenerate the inventory and move the README count** + +Run: `uv run --frozen python scripts/generate_tool_inventory.py` +Expected: `wrote docs/tool-inventory.json: 242 tools, 8 resources, 5 prompts, 26 groups`. + +Then in `README.md`: + +- `> **v1.5 release snapshot:** 241 tools · 8 resources` → `> **v1.5 release snapshot:** 242 tools · 8 resources` (leave the collected-tests figure: it is the known red, and Task 11 retakes it on main); +- `> 241 is the **registered** count; a default install advertises 236 over `tools/list`,` → `> 242 is the **registered** count; a default install advertises 237 over `tools/list`,`; +- `241 tools in 26 groups;` → `242 tools in 26 groups;`. + +The alias-corpus totals sentence (`205 AutoCAD command names and 1318 synonym phrases … covering all 241 tools`) is a measurement all three branches move; Task 11 recomputes it after the merge. + +- [ ] **Step 13: Gates** + +``` +uv run --frozen python scripts/generate_tool_inventory.py --check +uv run --frozen ruff check engineering/understand/report.py engineering/understand/draw.py server.py discovery/aliases.py tests/test_understand_report.py tests/test_understand_tools.py tests/test_tool_registry.py +uv run --frozen ruff format engineering/understand/report.py engineering/understand/draw.py server.py discovery/aliases.py tests/test_understand_report.py tests/test_understand_tools.py tests/test_tool_registry.py +uv run --frozen pytest tests/test_understand_network.py tests/test_understand_route.py tests/test_understand_takeoff.py tests/test_understand_report.py tests/test_understand_tools.py tests/test_tool_registry.py tests/test_tool_packs.py tests/test_tool_inventory.py tests/test_discovery_aliases.py tests/test_discovery_transform.py tests/test_release_consistency.py -q --deselect tests/test_release_consistency.py::test_document_numbers_match_the_tree +uv run --frozen pytest -q --deselect tests/test_release_consistency.py::test_document_numbers_match_the_tree +``` + +Expected: `ok: docs/tool-inventory.json matches the live surface`; ruff clean; every listed test passes — `test_section_header_counts_match_decorators` sees SECTION 27 declare 1 and hold 1, `test_readme_snapshot_counts_match_registrations` sees 242/8/5 on both sides, the discovery golden set holds its measured bars; the full suite green apart from the deselected known red (record it). If a golden discovery case moves, the new synonyms outrank an existing tool: drop the offending synonym, never the case. + +- [ ] **Step 14: Commit** + +``` +git add engineering/understand/report.py engineering/understand/draw.py server.py discovery/aliases.py tests/test_understand_report.py tests/test_understand_tools.py tests/test_tool_registry.py docs/tool-inventory.json README.md +git -c user.name="Umut CAN" -c user.email="umutcan.edizaslan@gmail.com" commit -m "$(cat <<'MSG' +feat(understand): the takeoff workbook and pipe_takeoff (SECTION 27 opens) + +write_workbook builds the user's takeoff workbook from a result: Metraj (rows +with direct, continuity and unassigned lengths split, the allowance as its +own column, routed / unroutable / grand totals), Ozet (by service, diameter +and room), Kontrol (unassigned diameters and unroutable runs with the +reason) and Metodoloji, generated from the result: sources, the length +source and why, the scale-check verdict with its statistics, the units, +every rule and tolerance, and every warning. Headers and sentences in +Turkish, English or Russian. CSV always, one file per sheet; XLSX with +openpyxl, otherwise the xlsx_write refusal naming the office extra while +the CSV files are still written. + +pipe_takeoff reads the P&ID and the layout once each (take_snapshot), +computes in a worker thread and never modifies either drawing. Tagged plant +and analysis; the plant pack is the integration task's. Alias record in +English, Turkish and Russian. Inventory regenerated; README snapshot 242 +tools (237 advertised); registry snapshot analysis: 13. + +Co-Authored-By: Claude Opus 5.5 (1M context) +Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd +MSG +)" +``` + +--- + +### Task 8: cable_takeoff + +**Files:** +- Modify: `engineering/understand/takeoff.py` (`cable_rows`, `check_section_rules`, `section_for`, `roundup_m`) +- Modify: `engineering/understand/report.py` (the cable workbook shape) +- Modify: `engineering/understand/draw.py` (`run_cable_takeoff`) +- Modify: `server.py` (`cable_takeoff` under SECTION 27, banner `(2 tools)`) +- Modify: `discovery/aliases.py` (the `cable_takeoff` record) +- Modify: `tests/test_tool_registry.py` (group snapshot 242 -> 243, `analysis` 13 -> 14) +- Modify: `docs/tool-inventory.json` (regenerated) +- Modify: `README.md` (release-snapshot tool counts 242 -> 243, advertised 237 -> 238) +- Test: `tests/test_understand_cable.py` +- Test: `tests/test_understand_tools.py` (four cable tests appended) + +**Interfaces:** +- Consumes: Task 6 — `check_allowance`, `unit_of`, `layout_tags`, `room_regions`, `room_at`, `_number` (module-private, same file); Task 5 — `tag_occurrences`, `label_search_default`, `MODEL`, `TEXT_TYPES`; Task 6 route — `manhattan`; Task 1 — `labels.parse_electrical(text) -> {"kw": float | None, "phases": int | None, "voltage": int | None, "neutral": bool, "vfd": bool}`, `labels.wiring_target(text) -> str | None`, `labels.normalize_panel(text) -> str | None`; Task 7 — `report.LABELS`, `report.TEXTS`, `report._BUILDERS`, `_t`, `_say`, `_num`, `_yes`, `_unit_text`, `write_workbook`, `check_lang`; Task 7's `engineering/understand/draw.py`; Task 2 — truth keys `cable_rows[{tag, panel, kw, length_mm, roundup_m}]` and `rooms[{number, name, polygon}]`. +- Produces: `cable_rows(pid: Snapshot, layout: Snapshot | None, *, allowance=0.20, section_rules=None) -> dict` (the skeleton's signature; keys `kind`, `rows[{tag, room, at, kw, phases, voltage, neutral, vfd, panel, panel_source, length_m, allowance, cable_m, section, package}]`, `open_items[{tag, item, detail}]`, `summary{by_room, by_panel}`, `totals{loads, known_kw, cable_m, open_items, packages}`, `units`, `warnings`, `method`); `check_section_rules(section_rules) -> tuple[dict, ...]`; `section_for(kw, phases, rules) -> str | None`; `roundup_m(length_m, allowance) -> int`; `CABLE_SEARCH_SHARE = 2.0`; `_BUILDERS["cable"]`; `async run_cable_takeoff(backend, *, pid_path=None, layout_path=None, output=None, lang="en", allowance=0.20, section_rules=None) -> dict`; the MCP tool `cable_takeoff`. + +No cable-section table exists anywhere in this task: a section is only ever the caller's `section_rules` row, and without rules the column is blank. **Power is never invented** — `test_power_is_never_invented` below. The panel-package rule (a load wired to a panel that states its own power belongs to that package; totals count the package's power, not its children's) is this repository's choice between the two ways of counting once, and the method sheet states it. `roundup_m` rounds to 1e-6 m before the ceiling so binary noise never buys a metre (50 m × 1.1 is 55.00000000000001). + +- [ ] **Step 1: Write the failing test** + +Create `tests/test_understand_cable.py`: + +```python +"""Cable takeoff rows: loads from the P&ID, panels and Manhattan lengths from the layout. + +The hand-built pair. On the P&ID: M82 with 'P = 18,5 кВт' and '3 ф. 400 В + +N + PE' under it, M87 with only '3 ф. 400 В + N + PE' (no power stated), the +package panel CP-M88 with 'P = 15 кВт' and its child motor M88 with +'P = 4 кВт'. On the layout (mm): two rooms split at x = 20000, the panel CP-1 +at (15000, 1000), M82 (2000, 3000), M87 (4000, 3000), CP-M88 (25000, 5000), +M88 (26000, 8000), each with a 'Wiring to ...' callout 300 mm below it. +Every expected length is worked in the comment beside it. The last two tests +read the synthetic plant pair of Task 2 and compare every cable row with the +generator's own Manhattan lengths. +""" + +from __future__ import annotations + +import csv + +import pytest + +from engineering.understand.labels import normalize_panel +from engineering.understand.report import write_workbook +from engineering.understand.snapshot import EntityRecord, Snapshot, read_snapshot +from engineering.understand.takeoff import cable_rows, roundup_m +from engineering.understand.vocab import classify_layer, room_label +from tests.fixtures.plant_pair import build_plant_pair + +EPS = 1e-9 +WALLS = "WALLS" + + +def rec(handle, kind, layer, points=(), **extra): + return EntityRecord( + handle=handle, type=kind, layer=layer, space="Model", points=tuple(points), **extra + ) + + +def text(handle, value, at, height, layer="TEXT"): + x, y = at + return rec( + handle, + "TEXT", + layer, + (at,), + text=value, + height=height, + bbox=(x, y, x + height * len(value) * 0.6, y + height), + ) + + +def snap(records, insunits, source): + layers = {r.layer: {"color": 7, "linetype": "Continuous"} for r in records} + return Snapshot( + source=source, + insunits=insunits, + extmin=None, + extmax=None, + layers=layers, + layouts=(), + records=tuple(records), + ) + + +def pid_records(): + return [ + text("a", "M82", (0.0, 0.0), 50.0), + text("a1", "P = 18,5 кВт", (0.0, -80.0), 50.0), + text("a2", "3 ф. 400 В + N + PE", (0.0, -160.0), 50.0), + text("b", "M87", (1000.0, 0.0), 50.0), + text("b1", "3 ф. 400 В + N + PE", (1000.0, -80.0), 50.0), + text("c", "CP-M88", (2000.0, 0.0), 50.0), + text("c1", "P = 15 кВт", (2000.0, -80.0), 50.0), + text("d", "M88", (3000.0, 0.0), 50.0), + text("d1", "P = 4 кВт", (3000.0, -80.0), 50.0), + ] + + +def layout_records(skip=()): + places = { + "M82": (2000.0, 3000.0), + "M87": (4000.0, 3000.0), + "CP-M88": (25000.0, 5000.0), + "M88": (26000.0, 8000.0), + } + panels = {"M82": "CP1", "M87": "CP 1", "CP-M88": "CP-1", "M88": "CP-M88"} + records = [ + rec( + "W", + "LWPOLYLINE", + WALLS, + ((0, 0), (40000, 0), (40000, 20000), (0, 20000)), + closed=True, + bbox=(0.0, 0.0, 40000.0, 20000.0), + ), + rec("X", "LINE", WALLS, ((20000, 0), (20000, 20000))), + text("R1", "ROOM 101", (5000.0, 10000.0), 250.0), + text("R2", "ROOM 102", (30000.0, 10000.0), 250.0), + text("P1", "CP-1", (15000.0, 1000.0), 250.0), + ] + for tag, (x, y) in places.items(): + if tag in skip: + continue + records.append(text("t" + tag, tag, (x, y), 250.0)) + records.append(text("w" + tag, f"Wiring to {panels[tag]}", (x, y - 300.0), 250.0)) + return records + + +def room_key(label): + found = room_label(label) + assert found is not None, f"the vocabulary must read {label!r} as a room label" + return str(found.get("number") or found.get("name")) + + +@pytest.fixture +def pair(): + assert classify_layer(WALLS)["discipline"] == "architecture", "WALLS must be a wall layer" + return snap(pid_records(), 4, "pid.dxf"), snap(layout_records(), 4, "layout.dxf") + + +def by_tag(result): + return {row["tag"]: row for row in result["rows"]} + + +def test_every_load_gets_its_panel_length_and_rounded_cable(pair): + pid, layout = pair + rows = by_tag(cable_rows(pid, layout)) + assert set(rows) == {"M82", "M87", "CP-M88", "M88"} + expected = { + # tag: (panel, Manhattan mm, ROUNDUP(m x 1.2)) + "M82": ("CP-1", 15000.0, 18), # |15000-2000| + |1000-3000| = 13000 + 2000 + "M87": ("CP-1", 13000.0, 16), # 11000 + 2000 -> 15.6 -> 16 + "CP-M88": ("CP-1", 14000.0, 17), # 10000 + 4000 -> 16.8 -> 17 + "M88": ("CP-M88", 4000.0, 5), # 1000 + 3000 -> 4.8 -> 5 + } + for tag, (panel, mm, cable) in expected.items(): + row = rows[tag] + assert row["panel"] == panel # 'CP1', 'CP 1' and 'CP-1' are one panel + assert row["panel_source"] == "layout" + assert row["length_m"] * 1000.0 == pytest.approx(mm, abs=1e-6) + assert row["cable_m"] == cable + + +def test_power_and_supply_are_read_from_the_pid_texts(pair): + pid, layout = pair + m82 = by_tag(cable_rows(pid, layout))["M82"] + assert m82["kw"] == pytest.approx(18.5) + assert (m82["phases"], m82["voltage"], m82["neutral"]) == (3, 400, True) + + +def test_power_is_never_invented(pair): + pid, layout = pair + result = cable_rows(pid, layout) + m87 = by_tag(result)["M87"] + assert m87["kw"] is None # the P&ID states phases and voltage, no kW + assert m87["cable_m"] == 16 # the cable is still measured + assert { + "tag": "M87", + "item": "missing_power", + "detail": "no power is stated near the tag on the P&ID", + } in result["open_items"] + assert all(item["tag"] == "M87" for item in result["open_items"]) + + +def test_room_totals_count_a_package_and_not_its_children(pair): + pid, layout = pair + result = cable_rows(pid, layout) + rows = by_tag(result) + assert rows["M88"]["package"] == "CP-M88" + assert rows["M82"]["room"] == room_key("ROOM 101") + assert rows["CP-M88"]["room"] == room_key("ROOM 102") + rooms = {item["key"]: item for item in result["summary"]["by_room"]} + assert rooms[room_key("ROOM 101")]["kw"] == pytest.approx(18.5) # M82; M87 has none + assert rooms[room_key("ROOM 102")]["kw"] == pytest.approx(15.0) # CP-M88, not + M88's 4 + assert rooms[room_key("ROOM 102")]["loads"] == 2 + assert result["totals"]["known_kw"] == pytest.approx(33.5) + assert result["totals"]["cable_m"] == 18 + 16 + 17 + 5 + + +def test_roundup_does_not_buy_a_metre_of_binary_noise(): + assert 50.0 * 1.1 > 55.0 # 55.00000000000001 in floating point + assert roundup_m(50.0, 0.10) == 55 + assert roundup_m(15.0, 0.20) == 18 + assert roundup_m(15.001, 0.20) == 19 # 18.0012 -> 19 + + +def test_without_a_layout_lengths_stay_empty_with_an_open_item(pair): + pid, _layout = pair + result = cable_rows(pid, None) + assert all(row["length_m"] is None and row["cable_m"] is None for row in result["rows"]) + no_layout = [item["tag"] for item in result["open_items"] if item["item"] == "no_layout"] + assert sorted(no_layout) == ["CP-M88", "M82", "M87", "M88"] + # no callout on the P&ID either: the panel is open, never guessed + assert all(row["panel"] is None for row in result["rows"]) + + +def test_a_tag_missing_from_the_layout_is_an_open_item(pair): + pid, _layout = pair + layout = snap(layout_records(skip=("M87",)), 4, "layout.dxf") + result = cable_rows(pid, layout) + items = {(item["tag"], item["item"]) for item in result["open_items"]} + assert ("M87", "tag_not_on_layout") in items + assert ("M87", "missing_panel") in items # its callout went with it + assert by_tag(result)["M87"]["cable_m"] is None + + +def test_sections_come_only_from_the_callers_rules(pair): + pid, layout = pair + assert all(row["section"] is None for row in cable_rows(pid, layout)["rows"]) + rules = [{"max_kw": 40, "section": "5x10"}, {"max_kw": 5, "section": "5x2.5"}] + rows = by_tag(cable_rows(pid, layout, section_rules=rules)) + assert rows["M82"]["section"] == "5x10" # 18.5 kW + assert rows["M88"]["section"] == "5x2.5" # 4 kW: the smaller rule is tried first + assert rows["M87"]["section"] is None # no power, no section + + +def test_a_load_beyond_every_rule_is_an_open_item(pair): + pid, layout = pair + result = cable_rows(pid, layout, section_rules=[{"max_kw": 10, "section": "5x4"}]) + assert {"tag": "M82", "item": "no_section_rule", "detail": "no rule covers 18.5 kW"} in ( + result["open_items"] + ) + + +@pytest.mark.parametrize( + ("kwargs", "message"), + [ + ({"allowance": 2.0}, "allowance: must lie between 0 and 1"), + ({"section_rules": {"max_kw": 5}}, "section_rules: expected a list"), + ( + {"section_rules": [{"max_kw": 0, "section": "x"}]}, + r"section_rules\[0\]\.max_kw: must be", + ), + ({"section_rules": [{"max_kw": 5}]}, r"section_rules\[0\]\.section: expected a non-empty"), + ( + {"section_rules": [{"max_kw": 5, "section": "x", "phases": 2}]}, + r"section_rules\[0\]\.phases: 1 or 3", + ), + ( + {"section_rules": [{"max_kw": 5, "section": "x", "iec": True}]}, + r"section_rules\[0\]: unknown key\(s\) iec", + ), + ], +) +def test_refusals_name_the_option(pair, kwargs, message): + pid, layout = pair + with pytest.raises(ValueError, match="^" + message): + cable_rows(pid, layout, **kwargs) + + +CABLE_HEADERS = { + "tr": [ + "Etiket", + "Mahal", + "Güç (kW)", + "Faz", + "Gerilim (V)", + "Nötr", + "Sürücü (VFD)", + "Pano", + "Mesafe (m)", + "Pay oranı", + "Kablo (m)", + "Kesit", + "Paket", + ], + "en": [ + "Tag", + "Room", + "Power (kW)", + "Phases", + "Voltage (V)", + "Neutral", + "VFD", + "Panel", + "Distance (m)", + "Allowance", + "Cable (m)", + "Section", + "Package", + ], + "ru": [ + "Тег", + "Помещение", + "Мощность (кВт)", + "Фазы", + "Напряжение (В)", + "Нейтраль", + "ЧРП", + "Шкаф", + "Расстояние (м)", + "Запас", + "Кабель (м)", + "Сечение", + "Комплект", + ], +} + + +@pytest.mark.parametrize("lang", ["tr", "en", "ru"]) +def test_the_cable_workbook_speaks_three_languages(pair, tmp_path, lang): + pid, layout = pair + files = write_workbook( + cable_rows(pid, layout), kind="cable", lang=lang, path=str(tmp_path / "cables.xlsx") + ) + with open(files["csv"][0], encoding="utf-8-sig", newline="") as handle: + metraj = list(csv.reader(handle)) + assert metraj[0] == CABLE_HEADERS[lang] + assert metraj[-1][10] == str(18 + 16 + 17 + 5) + with open(files["csv"][2], encoding="utf-8-sig", newline="") as handle: + kontrol = list(csv.reader(handle)) + assert [row[0] for row in kontrol[1:]] == ["M87"] # the one open item: no power + + +def test_the_method_sheet_states_the_package_rule_and_that_power_is_never_invented(pair, tmp_path): + pid, layout = pair + files = write_workbook( + cable_rows(pid, layout), kind="cable", lang="en", path=str(tmp_path / "cables.xlsx") + ) + with open(files["csv"][3], encoding="utf-8-sig", newline="") as handle: + method = {row[0]: row[1] for row in list(csv.reader(handle))[1:]} + assert "never invented" in method["Rule: power"] + assert "counted twice" in method["Rule: panel packages"] + assert method["Section rules (the caller's)"] == "—" + + +# -- the synthetic plant pair (Task 2) ------------------------------------------- + + +@pytest.fixture(scope="module") +def plant(tmp_path_factory): + truth = build_plant_pair(tmp_path_factory.mktemp("plant")) + return truth, read_snapshot(truth["pid"]), read_snapshot(truth["layout"]) + + +def inside(point, polygon): + """Even-odd ray cast over the generator's own room polygon.""" + x, y = point + hit = False + for (x1, y1), (x2, y2) in zip(polygon, polygon[1:] + polygon[:1], strict=False): + if (y1 > y) != (y2 > y) and x < x1 + (y - y1) * (x2 - x1) / (y2 - y1): + hit = not hit + return hit + + +def test_every_synthetic_cable_row_equals_the_generators_truth(plant): + truth, pid, layout = plant + result = cable_rows(pid, layout) + rows = by_tag(result) + assert set(rows) == {expected["tag"] for expected in truth["cable_rows"]} + for expected in truth["cable_rows"]: + row = rows[expected["tag"]] + assert row["panel"] == expected["panel"], expected["tag"] + if expected["kw"] is None: + assert row["kw"] is None, expected["tag"] + assert { + "tag": expected["tag"], + "item": "missing_power", + "detail": "no power is stated near the tag on the P&ID", + } in result["open_items"] + else: + assert row["kw"] == pytest.approx(expected["kw"]), expected["tag"] + assert row["length_m"] * 1000.0 == pytest.approx(expected["length_mm"], abs=1e-6) + assert row["cable_m"] == expected["roundup_m"], expected["tag"] + + +def test_the_synthetic_room_totals_count_each_power_once(plant): + truth, pid, layout = plant + result = cable_rows(pid, layout) + rooms = [ + (str(room["number"] or room["name"]), [tuple(p) for p in room["polygon"]]) + for room in truth["rooms"] + ] + for row in result["rows"]: + homes = [key for key, polygon in rooms if row["at"] and inside(row["at"], polygon)] + if homes: + assert row["room"] == homes[0], row["tag"] + stated = { + normalize_panel(r["tag"]) or r["tag"] for r in truth["cable_rows"] if r["kw"] is not None + } + + def child(r): + return r["panel"] in stated and (normalize_panel(r["tag"]) or r["tag"]) != r["panel"] + + expected_kw = sum(r["kw"] for r in truth["cable_rows"] if r["kw"] is not None and not child(r)) + assert result["totals"]["known_kw"] == pytest.approx(expected_kw) + by_room = sum(item["kw"] for item in result["summary"]["by_room"]) + assert by_room == pytest.approx(expected_kw) +``` + +- [ ] **Step 2: Run it to verify it fails** + +Run: `uv run --frozen pytest tests/test_understand_cable.py -q` +Expected: FAIL — collection error `ImportError: cannot import name 'cable_rows' from 'engineering.understand.takeoff'`. + +- [ ] **Step 3: Add the cable rows to takeoff.py** + +In `engineering/understand/takeoff.py`, replace the import block + +```python +from engineering.arch.faces import DEFAULT_TOL, Face, face_containing, planar_faces +from engineering.understand.network import MODEL, TEXT_TYPES, tag_occurrences +from engineering.understand.route import rmst, route_legs +``` + +with + +```python +from engineering.arch.faces import DEFAULT_TOL, Face, face_containing, planar_faces +from engineering.understand.labels import normalize_panel, parse_electrical, wiring_target +from engineering.understand.network import ( + MODEL, + TEXT_TYPES, + label_search_default, + tag_occurrences, +) +from engineering.understand.route import manhattan, rmst, route_legs +``` + +replace the `__all__` list + +```python +__all__ = [ + "DEFAULT_SERVICES", + "LENGTH_SOURCES", + "UNIT_METRES", + "apportion", + "check_allowance", + "check_pipe_options", + "layout_tags", + "pipe_rows", + "room_at", + "room_regions", + "scale_stand_in", + "unit_of", +] +``` + +with + +```python +__all__ = [ + "CABLE_SEARCH_SHARE", + "DEFAULT_SERVICES", + "LENGTH_SOURCES", + "UNIT_METRES", + "apportion", + "cable_rows", + "check_allowance", + "check_pipe_options", + "check_section_rules", + "layout_tags", + "pipe_rows", + "room_at", + "room_regions", + "roundup_m", + "scale_stand_in", + "section_for", + "unit_of", +] +``` + +and append, after `pipe_rows` (the end of the file), leaving two blank lines before it: + +```python +# -- cable rows ---------------------------------------------------------------- + +#: A load's power block stands under its symbol rather than on a line, so its +#: electrical texts and its wiring callout are searched twice as far as a +#: diameter label: 10 x the median text height. +CABLE_SEARCH_SHARE = 2.0 +SECTION_RULE_KEYS = ("max_kw", "section", "phases") + + +def check_section_rules(section_rules) -> tuple[dict, ...]: + """The caller's own cable-section rule table, validated and sorted by max_kw. + + Each rule is ``{"max_kw": > 0, "section": text, "phases": 1 | 3 (optional)}``; + a load takes the first rule whose max_kw it does not exceed and whose + phases, when the rule names them, equal the load's. No standard table exists + in this repository: without rules no section is proposed. + """ + if section_rules is None: + return () + if not isinstance(section_rules, (list, tuple)): + raise ValueError("section_rules: expected a list of {max_kw, section[, phases]}") + rules = [] + for i, rule in enumerate(section_rules): + where = f"section_rules[{i}]" + if not isinstance(rule, dict): + raise ValueError(f"{where}: expected {{max_kw, section[, phases]}}, got {rule!r}") + unknown = sorted(set(rule) - set(SECTION_RULE_KEYS)) + if unknown: + raise ValueError(f"{where}: unknown key(s) {', '.join(map(str, unknown))}") + max_kw = _number(rule.get("max_kw"), f"{where}.max_kw") + if max_kw <= 0.0: + raise ValueError(f"{where}.max_kw: must be greater than zero, got {max_kw:g}") + section = rule.get("section") + if not isinstance(section, str) or not section.strip(): + raise ValueError( + f"{where}.section: expected a non-empty text such as '5x2.5', got {section!r}" + ) + phases = rule.get("phases") + if phases is not None and phases not in (1, 3): + raise ValueError(f"{where}.phases: 1 or 3, got {phases!r}") + rules.append({"max_kw": max_kw, "section": section.strip(), "phases": phases}) + return tuple(sorted(rules, key=lambda r: (r["max_kw"], r["phases"] or 0))) + + +def section_for(kw: float | None, phases: int | None, rules) -> str | None: + """The first rule the load fits, or None - never a guessed section.""" + if kw is None: + return None + for rule in rules: + if kw <= rule["max_kw"] and (rule["phases"] is None or rule["phases"] == phases): + return rule["section"] + return None + + +def roundup_m(length_m: float, allowance: float) -> int: + """ROUNDUP(length x (1 + allowance)) to the metre, on the value rounded to 1e-6 m first. + + The rounding keeps binary noise from buying a metre of cable: 50 m with a + 10 % allowance is 55.00000000000001 in floating point, and 55 m on the sheet. + """ + return math.ceil(round(length_m * (1.0 + allowance), 6)) + + +def _nearest_tag(at: Pt, places: dict[str, list[Pt]], search: float) -> str | None: + found = [ + (math.dist(at, p), tag) + for tag, points in places.items() + for p in points + if math.dist(at, p) <= search + ] + return min(found)[1] if found else None + + +def _inside_box(p: Pt, box) -> bool: + return box is None or (box[0] <= p[0] <= box[2] and box[1] <= p[1] <= box[3]) + + +def cable_rows( + pid: Snapshot, layout: Snapshot | None, *, allowance=0.20, section_rules=None +) -> dict: + """One row per electrical load, the cable method made repeatable (spec §9). + + Loads are the P&ID tags with electrical texts near them (`parse_electrical`: + kW, phases, voltage, + N, VFD) and the tags a wiring callout names. The + panel comes from the layout's callouts (`wiring_target`: 'Wiring to CP1' + -> CP-1), else the P&ID's. The length is the Manhattan distance on the + layout from the load's tag to its panel's tag, and the cable is + `roundup_m(length, allowance)`. **Power is never invented**: a load with no + stated kW keeps an empty cell and an open item. With no layout, lengths + stay empty and every row gets an open item - the P&ID is not a length + source for cables. A load wired to a panel that itself states a power is + part of that package: room and panel totals count the package's power and + not its children's. Sections come only from `section_rules`. + + Refused by name before anything is read: an allowance outside 0-1 and a + malformed section rule. + """ + allowance = check_allowance(allowance) + rules = check_section_rules(section_rules) + pid_places = {tag: [p["at"] for p in places] for tag, places in tag_occurrences(pid).items()} + pid_search = CABLE_SEARCH_SHARE * label_search_default(pid) + + loads: dict[str, list] = {} + orphans: list[str] = [] + pid_callouts: dict[str, str] = {} + for rec in pid.records: + if rec.space != MODEL or rec.type not in TEXT_TYPES or not rec.text or not rec.points: + continue + at = rec.points[0] + target = wiring_target(rec.text) + if target is not None: + tag = _nearest_tag(at, pid_places, pid_search) + if tag is not None: + pid_callouts.setdefault(tag, target) + continue + found = parse_electrical(rec.text) + if found["kw"] is None and not (found["phases"] or found["voltage"] or found["vfd"]): + continue + tag = _nearest_tag(at, pid_places, pid_search) + if tag is None: + orphans.append(rec.handle) + continue + distance = min(math.dist(at, p) for p in pid_places[tag]) + loads.setdefault(tag, []).append((distance, rec.handle, found)) + + positions: dict[str, Pt] = {} + ambiguous: set[str] = set() + panels: dict[str, tuple[str, str]] = {} + layout_unit = unit_of(layout) if layout is not None else None + cluster = None + layout_search = None + if layout is not None: + placed = layout_tags(layout, set(loads) | set(pid_callouts)) + positions, ambiguous, cluster = ( + placed["positions"], + set(placed["ambiguous"]), + placed["cluster"], + ) + layout_search = CABLE_SEARCH_SHARE * label_search_default(layout) + layout_places = {tag: [p] for tag, p in positions.items()} + for rec in layout.records: + if rec.space != MODEL or rec.type not in TEXT_TYPES or not rec.text or not rec.points: + continue + target = wiring_target(rec.text) + if target is None or not _inside_box(rec.points[0], cluster): + continue + tag = _nearest_tag(rec.points[0], layout_places, layout_search) + if tag is not None: + panels.setdefault(tag, (target, "layout")) + for tag, target in pid_callouts.items(): + panels.setdefault(tag, (target, "pid")) + panel_at = {} + for tag, p in positions.items(): + name = normalize_panel(tag) + if name is not None: + panel_at[name] = p + regions = room_regions(layout, cluster) if layout is not None else room_regions(pid) + + rows, items = [], [] + for tag in sorted(set(loads) | set(panels)): + texts = sorted(loads.get(tag, []), key=lambda item: item[:2]) + powers = [(d, f["kw"]) for d, _h, f in texts if f["kw"] is not None] + kw = powers[0][1] if powers else None + if len({value for _d, value in powers}) > 1: + items.append( + { + "tag": tag, + "item": "conflicting_power", + "detail": ", ".join(f"{value:g} kW" for _d, value in powers), + } + ) + phases = next((f["phases"] for _d, _h, f in texts if f["phases"]), None) + voltage = next((f["voltage"] for _d, _h, f in texts if f["voltage"]), None) + panel, panel_source = panels.get(tag, (None, None)) + row = { + "tag": tag, + "room": "", + "at": None, + "kw": kw, + "phases": phases, + "voltage": voltage, + "neutral": any(f["neutral"] for _d, _h, f in texts), + "vfd": any(f["vfd"] for _d, _h, f in texts), + "panel": panel, + "panel_source": panel_source, + "length_m": None, + "allowance": allowance, + "cable_m": None, + "section": section_for(kw, phases, rules), + "package": None, + } + if kw is None: + items.append( + { + "tag": tag, + "item": "missing_power", + "detail": "no power is stated near the tag on the P&ID", + } + ) + if panel is None: + items.append( + { + "tag": tag, + "item": "missing_panel", + "detail": "no wiring callout names a panel for this tag", + } + ) + if rules and kw is not None and row["section"] is None: + items.append( + {"tag": tag, "item": "no_section_rule", "detail": f"no rule covers {kw:g} kW"} + ) + if layout is None: + items.append( + { + "tag": tag, + "item": "no_layout", + "detail": "no layout given: the P&ID is not a length source for cables", + } + ) + if pid_places.get(tag): + row["room"] = room_at(regions, pid_places[tag][0]) + else: + here = positions.get(tag) + if here is None: + items.append( + { + "tag": tag, + "item": "tag_ambiguous" if tag in ambiguous else "tag_not_on_layout", + "detail": "the load's tag cannot be placed on the layout", + } + ) + else: + row["at"] = here + row["room"] = room_at(regions, here) + if panel is not None and panel not in panel_at: + items.append( + { + "tag": tag, + "item": "panel_not_on_layout", + "detail": f"{panel} is not written once on the layout", + } + ) + if here is not None and panel in panel_at: + row["length_m"] = manhattan(here, panel_at[panel]) * layout_unit["m_per_unit"] + row["cable_m"] = roundup_m(row["length_m"], allowance) + rows.append(row) + + stated = { + (normalize_panel(r["tag"]) or r["tag"]): r["tag"] for r in rows if r["kw"] is not None + } + for row in rows: + owner = stated.get(row["panel"]) if row["panel"] else None + if owner is not None and owner != row["tag"]: + row["package"] = owner + + def counted(row: dict) -> float: + return row["kw"] if row["kw"] is not None and row["package"] is None else 0.0 + + summary = {} + for name, field in (("by_room", "room"), ("by_panel", "panel")): + summary[name] = [ + { + "key": key, + "loads": sum(1 for r in rows if (r[field] or "") == key), + "kw": sum(counted(r) for r in rows if (r[field] or "") == key), + "cable_m": sum(r["cable_m"] or 0 for r in rows if (r[field] or "") == key), + } + for key in sorted({row[field] or "" for row in rows}) + ] + units = {"pid": unit_of(pid), "layout": layout_unit} + warnings = [unit["warning"] for unit in units.values() if unit and unit["warning"]] + if orphans: + warnings.append(f"electrical text(s) near no tag, not used: {', '.join(orphans)}") + return { + "kind": "cable", + "rows": rows, + "open_items": items, + "summary": summary, + "totals": { + "loads": len(rows), + "known_kw": sum(counted(r) for r in rows), + "cable_m": sum(r["cable_m"] or 0 for r in rows), + "open_items": len(items), + "packages": sum(1 for r in rows if r["package"]), + }, + "units": units, + "warnings": warnings, + "method": { + "sources": {"pid": pid.source, "layout": layout.source if layout else None}, + "allowance": allowance, + "search": {"pid": pid_search, "layout": layout_search}, + "section_rules": [dict(rule) for rule in rules], + "rooms": _rooms_mode(regions), + }, + } +``` + +- [ ] **Step 4: Run it to see the rows pass and the workbook still fail** + +Run: `uv run --frozen pytest tests/test_understand_cable.py -q` +Expected: FAIL — `4 failed, 17 passed`: the three `test_the_cable_workbook_speaks_three_languages[...]` cases and `test_the_method_sheet_states_the_package_rule_...` with `ValueError: kind: 'cable' unknown; choose from pipe`. + +- [ ] **Step 5: Add the cable workbook to report.py** + +In `engineering/understand/report.py`, replace the end of `LABELS` + +```python + "warning": ("Uyarı", "Warning", "Предупреждение"), +} +``` + +with + +```python + "warning": ("Uyarı", "Warning", "Предупреждение"), + # cable takeoff + "tag": ("Etiket", "Tag", "Тег"), + "kw": ("Güç (kW)", "Power (kW)", "Мощность (кВт)"), + "phases": ("Faz", "Phases", "Фазы"), + "voltage": ("Gerilim (V)", "Voltage (V)", "Напряжение (В)"), + "neutral": ("Nötr", "Neutral", "Нейтраль"), + "vfd": ("Sürücü (VFD)", "VFD", "ЧРП"), + "panel": ("Pano", "Panel", "Шкаф"), + "length_m": ("Mesafe (m)", "Distance (m)", "Расстояние (м)"), + "cable_m": ("Kablo (m)", "Cable (m)", "Кабель (м)"), + "section": ("Kesit", "Section", "Сечение"), + "package": ("Paket", "Package", "Комплект"), + "loads": ("Yük sayısı", "Loads", "Нагрузок"), + "item": ("Madde", "Item", "Пункт"), + "detail": ("Ayrıntı", "Detail", "Подробности"), + "by_panel": ("Panoya göre", "By panel", "По шкафу"), + "missing_power": ("Güç yazılmamış", "No power stated", "Мощность не указана"), + "missing_panel": ("Pano belirtilmemiş", "No panel named", "Шкаф не указан"), + "no_layout": ("Yerleşim yok", "No layout", "Нет планировки"), + "panel_not_on_layout": ( + "Pano yerleşimde yok", + "Panel not on the layout", + "Шкафа нет на планировке", + ), + "no_section_rule": ("Kesit kuralı yok", "No section rule covers it", "Нет правила сечения"), + "conflicting_power": ( + "Çelişen güç yazıları", + "Conflicting power texts", + "Противоречивая мощность", + ), + "rule_cable_route": ("Kural: kablo güzergâhı", "Rule: cable route", "Правило: трасса кабеля"), + "rule_rounding": ("Kural: yuvarlama", "Rule: rounding", "Правило: округление"), + "rule_power": ("Kural: güç", "Rule: power", "Правило: мощность"), + "rule_panel": ("Kural: pano", "Rule: panel", "Правило: шкаф"), + "rule_package": ("Kural: pano paketleri", "Rule: panel packages", "Правило: комплектные шкафы"), + "rule_section": ("Kural: kesit", "Rule: section", "Правило: сечение"), + "section_rules": ( + "Kesit kuralları (çağıranın)", + "Section rules (the caller's)", + "Правила сечений (вызывающего)", + ), +} +``` + +replace the last entry and the end of `TEXTS` + +```python + "rule_multi": ( + "Birden çok çaplı bir hat, uzunluğunu P&ID üzerindeki çizili uzunlukları oranında " + "paylaştırır.", + "A run holding several diameters shares its length among them in proportion to their " + "drawn length on the P&ID.", + "Линия с несколькими диаметрами делит длину между ними пропорционально их длине на " + "схеме P&ID.", + ), +} +``` + +with + +```python + "rule_multi": ( + "Birden çok çaplı bir hat, uzunluğunu P&ID üzerindeki çizili uzunlukları oranında " + "paylaştırır.", + "A run holding several diameters shares its length among them in proportion to their " + "drawn length on the P&ID.", + "Линия с несколькими диаметрами делит длину между ними пропорционально их длине на " + "схеме P&ID.", + ), + "rule_cable_route": ( + "Uzunluk = yerleşimde yükün etiketinden panosunun etiketine Manhattan mesafesi.", + "Length = the Manhattan distance on the layout from the load's tag to its panel's tag.", + "Длина = манхэттенское расстояние на планировке от тега нагрузки до тега её шкафа.", + ), + "rule_rounding": ( + "Kablo = YUKARIYUVARLA(uzunluk × (1 + pay)) tam metreye; uzunluk önce 1e-6 m'ye " + "yuvarlanır.", + "Cable = ROUNDUP(length × (1 + allowance)) to the whole metre; the length is rounded to " + "1e-6 m first.", + "Кабель = ОКРВВЕРХ(длина × (1 + запас)) до целого метра; длина сначала округляется до " + "1e-6 м.", + ), + "rule_power": ( + "Güç, P&ID'de her etiketin yakınındaki elektrik yazılarından okunur; gücü yazılmamış " + "yükün hücresi boş kalır ve açık madde olur - güç asla uydurulmaz.", + "Power is read from the P&ID's electrical texts near each tag; a load with no stated " + "power keeps an empty cell and an open item - power is never invented.", + "Мощность берётся из электрических надписей P&ID рядом с каждым тегом; у нагрузки без " + "указанной мощности ячейка пуста и заводится открытый вопрос — мощность никогда не " + "выдумывается.", + ), + "rule_panel": ( + "Pano, yerleşimdeki bağlantı notlarından ('Wiring to CP1' gibi), yoksa P&ID'den alınır; " + "adlar normalleştirilir (CP 1 → CP-1).", + "The panel comes from the layout's wiring callouts ('Wiring to CP1' and its variants), " + "else from the P&ID; names are normalised (CP 1 → CP-1).", + "Шкаф берётся из выносок подключения на планировке ('Wiring to CP1' и варианты), иначе " + "из P&ID; имена нормализуются (CP 1 → CP-1).", + ), + "rule_package": ( + "Pano paketleri: kendisi güç belirten bir panoya bağlı yük o paketin parçasıdır; mahal ve " + "pano toplamları paketin gücünü sayar, alt yüklerini saymaz - hiçbir güç iki kez " + "sayılmaz.", + "Panel packages: a load wired to a panel that itself states a power is part of that " + "package; room and panel totals count the package's power and not its child loads, so " + "no power is counted twice.", + "Комплектные шкафы: нагрузка, подключённая к шкафу с собственной указанной мощностью, " + "входит в его комплект; итоги по помещениям и шкафам учитывают мощность комплекта, а не " + "его дочерних нагрузок, — ничего не учитывается дважды.", + ), + "rule_section": ( + "Kablo kesitleri yalnız çağıranın section_rules tablosundan gelir; tablo yoksa sütun " + "boş kalır. Hiçbir standart tablo uygulanmaz.", + "Cable sections come only from the caller's section_rules; without them the column " + "stays blank. No standard table is applied.", + "Сечения кабелей — только из таблицы section_rules вызывающего; без неё столбец пуст. " + "Никакая стандартная таблица не применяется.", + ), +} +``` + +and replace the builder registry line + +```python +_BUILDERS = {"pipe": _pipe_sheets} +``` + +with + +```python +def _cable_sheets(result: dict, lang: str) -> dict[str, tuple[list, list]]: + columns = ( + "tag", + "room", + "kw", + "phases", + "voltage", + "neutral", + "vfd", + "panel", + "length_m", + "allowance", + "cable_m", + "section", + "package", + ) + rows = [ + [ + row["tag"], + row["room"], + row["kw"], + row["phases"], + row["voltage"], + _yes(row["neutral"], lang), + _yes(row["vfd"], lang), + row["panel"], + _num(row["length_m"]), + row["allowance"], + row["cable_m"], + row["section"], + row["package"], + ] + for row in result["rows"] + ] + totals = result["totals"] + rows.append( + [ + _t("total", lang), + None, + _num(totals["known_kw"]), + None, + None, + None, + None, + None, + None, + None, + totals["cable_m"], + None, + None, + ] + ) + summary = [] + for group in ("by_room", "by_panel"): + for item in result["summary"][group]: + summary.append( + [_t(group, lang), item["key"], item["loads"], _num(item["kw"]), item["cable_m"]] + ) + summary.append( + [_t("total", lang), None, totals["loads"], _num(totals["known_kw"]), totals["cable_m"]] + ) + control = [ + [item["tag"], _t(item["item"], lang), item["detail"]] for item in result["open_items"] + ] + return { + "Metraj": ([_t(c, lang) for c in columns], rows), + "Özet": ([_t(c, lang) for c in ("group", "key", "loads", "kw", "cable_m")], summary), + "Kontrol": ([_t(c, lang) for c in ("tag", "item", "detail")], control), + "Metodoloji": ([_t("topic", lang), _t("value", lang)], _cable_method(result, lang)), + } + + +def _cable_method(result: dict, lang: str) -> list[list]: + method = result["method"] + rules = method["section_rules"] + rows = [ + ["source_pid", method["sources"]["pid"]], + ["source_layout", method["sources"]["layout"] or "—"], + ["units_pid", _unit_text(result["units"]["pid"])], + ["units_layout", _unit_text(result["units"]["layout"])], + ["rule_cable_route", _say("rule_cable_route", lang)], + ["rule_allowance", method["allowance"]], + ["rule_rounding", _say("rule_rounding", lang)], + ["rule_power", _say("rule_power", lang)], + ["rule_panel", _say("rule_panel", lang)], + ["rule_package", _say("rule_package", lang)], + ["rule_section", _say("rule_section", lang)], + ["section_rules", "; ".join(_rule_text(rule) for rule in rules) or "—"], + ["label_search", round(float(method["search"]["pid"]), 6)], + ] + rows += [["warning", warning] for warning in result["warnings"]] + return [[_t(topic, lang), value] for topic, value in rows] + + +def _rule_text(rule: dict) -> str: + phases = "" if rule["phases"] is None else f" / {rule['phases']} ph" + return f"<= {rule['max_kw']:g} kW{phases}: {rule['section']}" + + +_BUILDERS = {"pipe": _pipe_sheets, "cable": _cable_sheets} +``` + +- [ ] **Step 6: Run it to verify it passes** + +Run: `uv run --frozen pytest tests/test_understand_cable.py tests/test_understand_report.py tests/test_understand_takeoff.py -q` +Expected: PASS — `58 passed` (21 + 10 + 27; `57 passed, 1 skipped` without openpyxl). The report test's kind refusal still matches: `choose from pipe, cable` begins with `choose from pipe`. + +- [ ] **Step 7: Append the failing tool tests** + +In `tests/test_understand_tools.py`, replace the module docstring line `"""pipe_takeoff over the MCP wire, on the synthetic plant pair - never on a client drawing."""` with + +```python +"""pipe_takeoff and cable_takeoff over the MCP wire, on the synthetic plant pair - never on a +client drawing.""" +``` + +and append at the end of the file, after two blank lines: + +```python +async def test_cable_takeoff_gives_every_synthetic_load_its_rounded_cable(client, plant, tmp_path): + arguments = { + "pid_path": plant["pid"], + "layout_path": plant["layout"], + "output": str(tmp_path / "cables.xlsx"), + "lang": "ru", + } + result = (await client.call_tool("cable_takeoff", arguments)).structured_content + rows = {row["tag"]: row for row in result["rows"]} + for expected in plant["cable_rows"]: + assert rows[expected["tag"]]["cable_m"] == expected["roundup_m"], expected["tag"] + assert Path(result["files"]["csv"][0]).name == "cables_metraj.csv" + + +async def test_cable_takeoff_without_a_layout_leaves_every_length_empty(client, plant): + result = ( + await client.call_tool("cable_takeoff", {"pid_path": plant["pid"]}) + ).structured_content + assert result["rows"], "the synthetic P&ID carries electrical loads" + assert all(row["cable_m"] is None for row in result["rows"]) + no_layout = {item["tag"] for item in result["open_items"] if item["item"] == "no_layout"} + assert no_layout == {row["tag"] for row in result["rows"]} + + +async def test_a_malformed_section_rule_is_refused_by_path(client, plant): + arguments = {"pid_path": plant["pid"], "section_rules": [{"max_kw": 5}]} + result = await client.call_tool("cable_takeoff", arguments, raise_on_error=False) + assert result.is_error is True + assert "section_rules[0].section: expected a non-empty text" in result.content[0].text + + +async def test_the_cable_tool_says_it_writes_a_file_and_destroys_nothing(client): + tools = {tool.name: tool for tool in await client.list_tools()} + annotations = tools["cable_takeoff"].annotations + assert annotations.readOnlyHint is False + assert annotations.destructiveHint is False +``` + +- [ ] **Step 8: Run them to verify they fail** + +Run: `uv run --frozen pytest tests/test_understand_tools.py -q` +Expected: FAIL — `4 failed, 6 passed`: the cable tests with `Not found: Unknown tool: 'cable_takeoff'` (or `KeyError: 'cable_takeoff'` in the annotations test). + +- [ ] **Step 9: Add run_cable_takeoff to draw.py** + +In `engineering/understand/draw.py`, replace + +```python +from engineering.understand.takeoff import check_pipe_options, pipe_rows + +__all__ = ["run_pipe_takeoff"] +``` + +with + +```python +from engineering.understand.takeoff import ( + cable_rows, + check_allowance, + check_pipe_options, + check_section_rules, + pipe_rows, +) + +__all__ = ["run_cable_takeoff", "run_pipe_takeoff"] +``` + +and append at the end of the file, after two blank lines: + +```python +async def run_cable_takeoff( + backend, + *, + pid_path: str | None = None, + layout_path: str | None = None, + output: str | None = None, + lang: str = "en", + allowance: float = 0.20, + section_rules=None, +) -> dict: + """`cable_rows` over a P&ID and an optional layout, plus the workbook when `output` is named.""" + check_lang(lang) + allowance = check_allowance(allowance) + rules = check_section_rules(section_rules) + pid = await take_snapshot(backend, pid_path) + layout = await take_snapshot(backend, layout_path) if layout_path else None + result = await asyncio.to_thread( + cable_rows, pid, layout, allowance=allowance, section_rules=list(rules) or None + ) + result["files"] = ( + await asyncio.to_thread(write_workbook, result, kind="cable", lang=lang, path=output) + if output + else None + ) + return result +``` + +- [ ] **Step 10: Register cable_takeoff under SECTION 27** + +In `server.py`, change the banner + +``` +# ── SECTION 27: Plant Takeoffs (1 tool) ───────────────────────────────────── +``` + +to + +``` +# ── SECTION 27: Plant Takeoffs (2 tools) ──────────────────────────────────── +``` + +and insert the block below directly after `pipe_takeoff` (whose last lines are ` except (ValueError, OSError) as exc:` / ` raise ToolError(str(exc)) from exc`) and before the `# ── RESOURCES ──` banner pair, with two blank lines on each side: + +```python +@cad_tool( + summary="Cable takeoff: one row per load, Manhattan length to its panel, rounded up to the metre.", + cost="safe", +) +@mcp.tool( + annotations={ + "title": "Plant: Cable Takeoff", + "readOnlyHint": False, + "destructiveHint": False, + }, + tags={"plant", "analysis"}, +) +async def cable_takeoff( + pid_path: Annotated[ + str | None, + Field( + default=None, + description="The P&ID carrying the electrical texts (kW, phases, voltage, VFD). " + "Omit to read the current drawing.", + ), + ] = None, + layout_path: Annotated[ + str | None, + Field( + default=None, + description="The layout with the tags, the panels and the 'Wiring to CPn' " + "callouts. Omitted, every length stays empty with an open item.", + ), + ] = None, + output: Annotated[ + str | None, + Field( + default=None, + description="Workbook to write (.xlsx); the four sheets are also written as CSV " + "beside it. Omit to return the rows only.", + ), + ] = None, + lang: Annotated[ + str, Field(default="en", description="Headers and method text: tr | en | ru.") + ] = "en", + allowance: Annotated[ + float, + Field( + default=0.20, + ge=0.0, + le=1.0, + description="Allowance before rounding up to the metre (0.20 = 20 %).", + ), + ] = 0.20, + section_rules: Annotated[ + list[dict] | None, + Field( + default=None, + description="Your own section table: [{max_kw, section, phases (1|3, optional)}], " + "e.g. [{'max_kw': 5.5, 'section': '5x2.5'}]. Omitted, no section is proposed.", + ), + ] = None, + ctx: Context = None, +) -> dict: + """A cable takeoff: one row per electrical load, the shop's cable method made repeatable. + + Power, phases, voltage, neutral and VFD come from the P&ID's electrical + texts near each tag (Russian, Turkish and English spellings). **Power is + never invented**: a load with no stated kW keeps an empty cell and an open + item. The panel comes from the layout's wiring callouts ('Wiring to CP1' + and its variants, normalised to CP-1), else from the P&ID. The length is the + Manhattan distance on the layout from the load's tag to its panel's tag, + and the cable is ROUNDUP(length x (1 + allowance)) to the metre. Room and + panel totals count known power once: a load wired to a panel that states + its own power is part of that package, and the package's power is counted + instead of its children's - the rule is written on the method sheet. + Sections only from `section_rules`; no standard table is applied. Open + items: missing power, missing panel, a tag or panel not on the layout, no + layout at all, a load no section rule covers. + + Read-only on both drawings. With `output` it writes one workbook - Metraj, + Özet, Kontrol, Metodoloji - as XLSX and, always, as CSV beside it. + + Refused before a drawing is read: `lang` outside tr / en / ru, an allowance + outside 0-1 and a malformed section rule (named by its path, + `section_rules[0].section: ...`). A missing, unreadable or oversize drawing + is refused by the reader, which names MAX_DXF_BYTES. Without openpyxl the + XLSX is not written and `files.refused` carries `capability: "xlsx_write"` + naming the `office` extra; the CSV files are written all the same. + """ + from engineering.understand.draw import run_cable_takeoff + + pid = str(validate_path(pid_path)) if pid_path else None + layout = str(validate_path(layout_path)) if layout_path else None + target = str(validate_path(output, allow_write=True)) if output else None + await ctx.info(f"cable_takeoff: {pid or 'current drawing'} / {layout or 'no layout'}") + try: + return await run_cable_takeoff( + _backend(ctx), + pid_path=pid, + layout_path=layout, + output=target, + lang=lang, + allowance=allowance, + section_rules=section_rules, + ) + except (ValueError, OSError) as exc: + raise ToolError(str(exc)) from exc +``` + +As in Task 7, `plant` stays out of the priority, label and pack lists; `cable_takeoff` files under `analysis` until Task 11. + +- [ ] **Step 11: The alias record** + +In `discovery/aliases.py`, add this record directly after the `"pipe_takeoff"` record, before the closing `}` of `TOOL_ALIASES`: + +```python + "cable_takeoff": ToolAliases( + acad=(), + synonyms=( + "cable takeoff", + "cable schedule", + "cable lengths", + "cable list", + "how much cable", + "motor cable lengths", + "kablo metrajı", + "kablo metraji", + "kablo listesi", + "kablo cetveli", + "кабельный журнал", + "ведомость кабелей", + ), + ), +``` + +- [ ] **Step 12: Run the tool and alias tests to verify they pass** + +Run: `uv run --frozen pytest tests/test_understand_tools.py tests/test_discovery_aliases.py -q` +Expected: PASS — the 10 tool tests and every alias gate. + +- [ ] **Step 13: Move the registration snapshot** + +In `tests/test_tool_registry.py`, `test_tool_group_sizes_are_unchanged`: + +- the docstring's first line becomes `"""Frozen snapshot of the surface (243 tools, 26 groups).`; +- directly after Task 7's paragraph (its last line ` once every branch has landed.`), add: + +``` + + It moved 13 -> 14 when `cable_takeoff` joined the same section, for + the same branch-local reason. +``` + +- `"analysis": 13,` becomes `"analysis": 14,`; +- `assert sum(sizes.values()) == 242` becomes `assert sum(sizes.values()) == 243`. + +- [ ] **Step 14: Regenerate the inventory and move the README count** + +Run: `uv run --frozen python scripts/generate_tool_inventory.py` +Expected: `wrote docs/tool-inventory.json: 243 tools, 8 resources, 5 prompts, 26 groups`. + +Then in `README.md`: + +- `> **v1.5 release snapshot:** 242 tools · 8 resources` → `> **v1.5 release snapshot:** 243 tools · 8 resources` (the collected-tests figure stays Task 11's); +- `> 242 is the **registered** count; a default install advertises 237 over `tools/list`,` → `> 243 is the **registered** count; a default install advertises 238 over `tools/list`,`; +- `242 tools in 26 groups;` → `243 tools in 26 groups;`. + +- [ ] **Step 15: Gates** + +``` +uv run --frozen python scripts/generate_tool_inventory.py --check +uv run --frozen ruff check engineering/understand/takeoff.py engineering/understand/report.py engineering/understand/draw.py server.py discovery/aliases.py tests/test_understand_cable.py tests/test_understand_tools.py tests/test_tool_registry.py +uv run --frozen ruff format engineering/understand/takeoff.py engineering/understand/report.py engineering/understand/draw.py server.py discovery/aliases.py tests/test_understand_cable.py tests/test_understand_tools.py tests/test_tool_registry.py +uv run --frozen pytest tests/test_understand_network.py tests/test_understand_route.py tests/test_understand_takeoff.py tests/test_understand_report.py tests/test_understand_cable.py tests/test_understand_tools.py tests/test_tool_registry.py tests/test_tool_packs.py tests/test_tool_inventory.py tests/test_discovery_aliases.py tests/test_discovery_transform.py tests/test_release_consistency.py -q --deselect tests/test_release_consistency.py::test_document_numbers_match_the_tree +uv run --frozen pytest -q --deselect tests/test_release_consistency.py::test_document_numbers_match_the_tree +``` + +Expected: `ok: docs/tool-inventory.json matches the live surface`; ruff clean; every listed test passes — SECTION 27 declares 2 and holds 2, the README snapshot reads 243/8/5 on both sides; the full suite green apart from the deselected known red (record it). A golden discovery case that moves is fixed by dropping the offending synonym, never the case. + +- [ ] **Step 16: Commit** + +``` +git add engineering/understand/takeoff.py engineering/understand/report.py engineering/understand/draw.py server.py discovery/aliases.py tests/test_understand_cable.py tests/test_understand_tools.py tests/test_tool_registry.py docs/tool-inventory.json README.md +git -c user.name="Umut CAN" -c user.email="umutcan.edizaslan@gmail.com" commit -m "$(cat <<'MSG' +feat(understand): cable_takeoff - one row per load, rounded to the metre + +cable_rows reads each load's power, phases, voltage, neutral and VFD from the +P&ID's electrical texts near its tag, and its panel from the layout's wiring +callouts (else the P&ID's), names normalised CP 1 -> CP-1. The length is the +Manhattan distance on the layout from the tag to its panel, and the cable +ROUNDUP(length x (1 + allowance)) to the metre, on the length rounded to +1e-6 m so binary noise never buys a metre. Power is never invented: a load +with no stated kW keeps an empty cell and an open item. Without a layout the +lengths stay empty with an open item per row. Room and panel totals count a +panel package's power once, never its children's as well; the rule is on the +method sheet. Sections only from the caller's section_rules - no standard +table exists here. + +The workbook gains the cable shape (Metraj, Ozet by room and panel, Kontrol +as the open-items list, Metodoloji) in the same three languages; +cable_takeoff joins SECTION 27. Inventory regenerated; README snapshot 243 +tools (238 advertised); registry snapshot analysis: 14. + +Co-Authored-By: Claude Opus 5.5 (1M context) +Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd +MSG +)" +``` + +--- + +### Task 9: drawing_diff — two revisions by geometric signature + +**Files:** +- Create: `engineering/understand/diff.py` +- Create: `tests/fixtures/revisions.py` (the edited revision pair, built from Task 2's plant P&ID) +- Modify: `server.py` (group D's copy of the SECTION 26 banner, `(1 tool)`, with `_diff_revision`, `_diff_markup` and `drawing_diff`, directly above the RESOURCES banner) +- Modify: `discovery/aliases.py` (the `drawing_diff` record) +- Modify: `docs/tool-inventory.json` (regenerated), `README.md` (release-snapshot tool count 241 → 242, advertised 236 → 237, caption 241 → 242) +- Modify: `tests/test_tool_registry.py` (frozen group sizes: `analysis` 12 → 13, total 242) +- Test: `tests/test_understand_diff.py`, `tests/test_understand_diff_tool.py` + +**Interfaces:** +- Consumes: `EntityRecord`, `Snapshot`, `Pt`, `records_from_doc(doc, *, source: str) -> Snapshot` and `async take_snapshot(backend, path: str | None = None) -> Snapshot` from `engineering/understand/snapshot.py` (Task 1; `EntityRecord.points[0]` is a CIRCLE's centre, as for every other one-point record); `build_plant_pair(directory) -> dict` from `tests/fixtures/plant_pair.py` (Task 2; this task reads only its `"pid"` path); `engineering.sheet.revision.add_revision(backend, *, rev, description, clouds, at, cloud_segment, ...) -> dict` and `REV_RE`; `engineering.sheet.bom.LIST_WIDTH` / `ROW_HEIGHT`; `config.settings.max_dxf_bytes`. +- Produces (the skeleton's pinned signature first): `diff_snapshots(old: Snapshot, new: Snapshot, *, tol=0.01, move_tol=None) -> dict` with keys `tol`, `move_tol`, `handles_stable`, `summary` (`old_entities`, `new_entities`, `unchanged`, `changed`, `added`, `removed`), `by_layer` / `by_type` (`{name: {"changed", "added", "removed"}}`), `changed` (`[{"old_handle", "new_handle", "type", "layer", "space", "matched_by": "handle" | "position", "at", "changes"}]`), `added` / `removed` (`[{"handle", "type", "layer", "space", "at"}]`), `clusters` (`[{"space", "box": [x0, y0, x1, y1], "count"}]`); `signature(rec, tol=0.01) -> tuple`; `anchor(rec) -> Pt | None`; `robust_box(*snaps) -> tuple[float, float, float, float] | None`; `default_move_tol(old, new, tol=0.01) -> float`; `cap_report(result: dict, limit: int = 200) -> dict` (adds `truncated`); `block_signatures(doc, *, tol=0.01) -> dict[str, str]`; `block_changes(old: dict | None, new: dict | None) -> dict`; `read_revision(path: str, *, tol=0.01) -> tuple[Snapshot, dict[str, str]]`. Test helper: `tests.fixtures.revisions.build_revision_pair(directory, *, edits: bool = True) -> dict`. MCP tool `drawing_diff(old_path, new_path=None, tol=0.01, move_tol=None, limit=200, markup=False, rev="", description="")`. + +- [ ] **Step 1: Write the failing test** + +Precondition: the group branch `track-h/diff` sits at main's HEAD with Tasks 1 and 2 landed — `git checkout -b track-h/diff` (or `git checkout track-h/diff` in an existing worktree), then `git merge --ff-only main`, and `engineering/understand/snapshot.py`, `engineering/understand/vocab.py` and `tests/fixtures/plant_pair.py` exist. If the branch cannot fast-forward, stop and report. + +First create `tests/fixtures/revisions.py`. It edits the synthetic plant P&ID five ways and renumbers every model-space handle; the expected answer is read off the older revision's own records, never typed in: + +```python +"""Two revisions of the synthetic plant P&ID with known edits, for ``drawing_diff``. + +Built from ``build_plant_pair`` (``tests/fixtures/plant_pair.py``) with +ezdxf. The older revision is the P&ID plus one tagged block reference; the +newer one moves a line by half the move tolerance, rewrites a text, changes +the block's attribute, deletes one line, adds a circle - and then gives every +model-space entity a new handle, the way a save through another program +renumbers them. What was edited is +picked from the older revision's own records (entities whose signature is +unique, so a double-drawn line can never be the one edited) and returned with +the paths: the expected answer is read off the fixture, never typed in. +""" + +from __future__ import annotations + +from collections import Counter +from pathlib import Path + +import ezdxf + +from engineering.understand.diff import diff_snapshots, read_revision, robust_box, signature +from engineering.understand.snapshot import records_from_doc +from tests.fixtures.plant_pair import build_plant_pair + +TAG_BLOCK = "DIFF_TAG" +TAG_LAYER = "DIFF-TAGS" +ADDED_LAYER = "DIFF-ADDED" +NEW_TEXT = "REV-B NOTE" +TAG_BEFORE = "X-1" +TAG_AFTER = "X-2" + + +def renumber_modelspace(doc) -> None: + """Every model-space entity replaced by a copy under a fresh handle.""" + msp = doc.modelspace() + for entity in list(msp): + clone = entity.copy() + msp.add_entity(clone) + msp.delete_entity(entity) + + +def build_revision_pair(directory, *, edits: bool = True) -> dict: + directory = Path(directory) + truth = build_plant_pair(directory / "plant") + base = ezdxf.readfile(truth["pid"]) + box = robust_box(records_from_doc(base, source="base")) + centre = ((box[0] + box[2]) / 2.0, (box[1] + box[3]) / 2.0) + block = base.blocks.new(TAG_BLOCK) + block.add_circle((0.0, 0.0), 5.0) + block.add_attdef("TAG", (0.0, -8.0), dxfattribs={"height": 2.5}) + base.layers.add(TAG_LAYER) + tag = base.modelspace().add_blockref(TAG_BLOCK, centre, dxfattribs={"layer": TAG_LAYER}) + tag.add_auto_attribs({"TAG": TAG_BEFORE}) + old_path = directory / "rev_a.dxf" + base.saveas(old_path) + + old_snap, _blocks = read_revision(str(old_path)) + move_tol = diff_snapshots(old_snap, old_snap)["move_tol"] + counts = Counter(signature(r) for r in old_snap.records) + unique = [r for r in old_snap.records if r.space == "Model" and counts[signature(r)] == 1] + moved = next(r for r in unique if r.type == "LINE") + text = next(r for r in unique if r.type in ("TEXT", "MTEXT")) + insert = next(r for r in old_snap.records if r.block == TAG_BLOCK) + deleted = next( + r + for r in unique + if r.type in ("LINE", "LWPOLYLINE") and (r.type, r.layer) != (moved.type, moved.layer) + ) + + new = ezdxf.readfile(old_path) + msp = new.modelspace() + if edits: + new.entitydb[moved.handle].translate(move_tol / 2.0, 0.0, 0.0) + target = new.entitydb[text.handle] + if target.dxftype() == "MTEXT": + target.text = NEW_TEXT + else: + target.dxf.text = NEW_TEXT + new.entitydb[insert.handle].get_attrib("TAG").dxf.text = TAG_AFTER + msp.delete_entity(new.entitydb[deleted.handle]) + new.layers.add(ADDED_LAYER) + msp.add_circle(centre, move_tol / 10.0, dxfattribs={"layer": ADDED_LAYER}) + renumber_modelspace(new) + new_path = directory / "rev_b.dxf" + new.saveas(new_path) + return { + "old": str(old_path), + "new": str(new_path), + "move_tol": move_tol, + "old_count": len(old_snap.records), + "moved": moved.handle, + "text": text.handle, + "text_before": text.text, + "insert": insert.handle, + "deleted": deleted.handle, + } +``` + +Then create `tests/test_understand_diff.py`: + +```python +"""drawing_diff's engine: signatures, the three matching stages, the report. + +Pure tests build ``EntityRecord``s by hand, so every coordinate is the +answer. The known-edits tests build two revisions of the synthetic plant P&ID +(``tests/fixtures/revisions.py``) and assert exactly the edits made - a move, +a text change, an attribute change, an add, a delete, under renumbered +handles - and nothing else. +""" + +from __future__ import annotations + +import ezdxf +import pytest + +import config +from engineering.understand.diff import ( + block_changes, + block_signatures, + cap_report, + diff_snapshots, + read_revision, + robust_box, + signature, +) +from engineering.understand.snapshot import EntityRecord, Snapshot +from tests.fixtures.revisions import ( + ADDED_LAYER, + NEW_TEXT, + TAG_AFTER, + TAG_BEFORE, + build_revision_pair, +) + + +def rec(handle, type_, layer="PIPE", points=(), **kw): + space = kw.pop("space", "Model") + return EntityRecord( + handle=handle, type=type_, layer=layer, space=space, points=tuple(points), **kw + ) + + +def snap(*records): + return Snapshot( + source="test", + insunits=4, + extmin=None, + extmax=None, + layers={}, + layouts=(), + records=tuple(records), + ) + + +def line(handle, x1, y1, x2, y2, layer="PIPE"): + return rec(handle, "LINE", layer, ((x1, y1), (x2, y2))) + + +STILL = [line("A", 0, 0, 100, 0), line("B", 0, 50, 100, 50), line("C", 0, 100, 100, 100)] +RENUMBERED = [line("1A", 0, 0, 100, 0), line("1B", 0, 50, 100, 50), line("1C", 0, 100, 100, 100)] + + +# -- signatures ------------------------------------------------------------------------ + + +def test_a_signature_is_rounded_to_tol_and_ignores_a_lines_direction(): + a = line("A", 0.0, 0.0, 10.0, 0.0) + b = line("B", 10.001, 0.004, 0.0, 0.0) + assert signature(a, 0.01) == signature(b, 0.01) + assert signature(a, 0.01) != signature(line("C", 0.0, 0.0, 10.02, 0.0), 0.01) + + +def test_identical_revisions_have_no_change_and_no_cluster(): + result = diff_snapshots(snap(*STILL), snap(*STILL)) + assert result["summary"] == { + "old_entities": 3, + "new_entities": 3, + "unchanged": 3, + "changed": 0, + "added": 0, + "removed": 0, + } + assert result["clusters"] == [] and result["handles_stable"] is True + + +# -- the three stages ------------------------------------------------------------------ + + +def test_a_move_under_renumbered_handles_is_matched_by_position(): + old = snap(*STILL, line("D", 0, 200, 100, 200)) + new = snap(*RENUMBERED, line("1D", 3, 200, 103, 200)) + result = diff_snapshots(old, new, move_tol=10.0) + assert result["handles_stable"] is False + (row,) = result["changed"] + assert (row["old_handle"], row["new_handle"], row["matched_by"]) == ("D", "1D", "position") + assert row["changes"] == {"moved_by": [pytest.approx(3.0), pytest.approx(0.0)]} + assert result["summary"]["unchanged"] == 3 + + +def test_a_text_change_is_reported_from_and_to(): + old = snap(*STILL, rec("T", "TEXT", "NOTES", ((10, 10),), text="OLD", height=2.5)) + new = snap(*STILL, rec("T", "TEXT", "NOTES", ((10, 10),), text="NEW", height=2.5)) + (row,) = diff_snapshots(old, new)["changed"] + assert row["matched_by"] == "handle" + assert row["changes"] == {"text": {"from": "OLD", "to": "NEW"}} + + +def test_an_attribute_change_is_reported_by_tag(): + before = (("TAG", "A-1"), ("SERVICE", "CIP")) + after = (("TAG", "A-2"), ("SERVICE", "CIP")) + old = snap(*STILL, rec("I", "INSERT", "TAGS", ((500, 500),), block="TB", attribs=before)) + new = snap(*STILL, rec("I", "INSERT", "TAGS", ((500, 500),), block="TB", attribs=after)) + (row,) = diff_snapshots(old, new)["changed"] + assert row["changes"] == {"attribs": [{"tag": "TAG", "from": "A-1", "to": "A-2"}]} + + +def test_a_radius_change_is_reported_from_and_to(): + old = snap(*STILL, rec("R", "CIRCLE", "PIPE", ((0, 0),), radius=10.0)) + new = snap(*STILL, rec("R", "CIRCLE", "PIPE", ((0, 0),), radius=12.0)) + (row,) = diff_snapshots(old, new)["changed"] + assert row["changes"] == {"radius": {"from": 10.0, "to": 12.0}} + + +def test_the_handle_stage_catches_a_layer_change_the_position_stage_cannot(): + old = snap(*STILL, line("L", 0, 300, 100, 300, layer="P1")) + new = snap(*STILL, line("L", 0, 300, 100, 300, layer="P2")) + (row,) = diff_snapshots(old, new)["changed"] + assert row["matched_by"] == "handle" + assert row["changes"] == {"layer": {"from": "P1", "to": "P2"}} + + +def test_stable_handles_pair_a_far_move_by_handle(): + old = snap(*STILL, rec("20", "CIRCLE", "PIPE", ((0, 0),), radius=5.0)) + new = snap(*STILL, rec("20", "CIRCLE", "PIPE", ((5000, 5000),), radius=5.0)) + (row,) = diff_snapshots(old, new, move_tol=10.0)["changed"] + assert row["matched_by"] == "handle" + assert row["changes"] == {"moved_by": [pytest.approx(5000.0), pytest.approx(5000.0)]} + + +def test_renumbered_handles_never_pair_strangers_that_share_a_handle(): + old = snap(*STILL, rec("20", "CIRCLE", "PIPE", ((0, 0),), radius=5.0)) + new = snap(*RENUMBERED, rec("20", "CIRCLE", "PIPE", ((5000, 5000),), radius=5.0)) + result = diff_snapshots(old, new, move_tol=10.0) + assert result["handles_stable"] is False + assert result["changed"] == [] + assert [r["handle"] for r in result["removed"]] == ["20"] + assert [r["at"] for r in result["added"]] == [[5000.0, 5000.0]] + + +def test_beyond_move_tol_a_line_is_removed_and_another_added(): + old = snap(*STILL, line("D", 0, 200, 100, 200)) + new = snap(*STILL, line("E", 0, 700, 100, 700)) + result = diff_snapshots(old, new, move_tol=10.0) + assert (result["summary"]["added"], result["summary"]["removed"]) == (1, 1) + assert result["changed"] == [] + + +# -- the move tolerance ---------------------------------------------------------------- + + +def test_the_default_move_tol_is_2_percent_of_the_robust_diagonal(): + # 100 points on the diagonal of a 2970 x 3960 box, and one stray 10 km away: + # the 1st-99th percentile box is (30, 40)-(2970, 3960), diagonal 4900. + points = [rec(f"P{i}", "POINT", "PTS", ((i * 30.0, i * 40.0),)) for i in range(100)] + stray = rec("FAR", "POINT", "PTS", ((1.0e7, 1.0e7),)) + drawing = snap(*points, stray) + assert robust_box(drawing) == (30.0, 40.0, 2970.0, 3960.0) + assert diff_snapshots(drawing, drawing)["move_tol"] == pytest.approx(98.0) + + +def test_a_non_positive_tolerance_is_refused_by_name(): + with pytest.raises(ValueError, match=r"^tol must be a positive number; got 0"): + diff_snapshots(snap(), snap(), tol=0) + with pytest.raises(ValueError, match=r"^move_tol must be a positive number; got -1"): + diff_snapshots(snap(), snap(), move_tol=-1) + + +# -- the report ------------------------------------------------------------------------ + + +def test_changes_are_counted_by_layer_and_type_and_clustered(): + old = snap( + line("U", 0, -500, 100, -500), + line("D", 0, 200, 100, 200), + rec("T", "TEXT", "NOTES", ((10, 10),), text="OLD", height=2.5), + ) + new = snap( + line("U", 0, -500, 100, -500), + line("D", 3, 200, 103, 200), + rec("T", "TEXT", "NOTES", ((10, 10),), text="NEW", height=2.5), + rec("N1", "CIRCLE", "PIPE", ((5000, 5000),), radius=5.0, bbox=(4995, 4995, 5005, 5005)), + ) + result = diff_snapshots(old, new, move_tol=10.0) + assert result["by_layer"] == { + "NOTES": {"changed": 1, "added": 0, "removed": 0}, + "PIPE": {"changed": 1, "added": 1, "removed": 0}, + } + assert result["by_type"] == { + "CIRCLE": {"changed": 0, "added": 1, "removed": 0}, + "LINE": {"changed": 1, "added": 0, "removed": 0}, + "TEXT": {"changed": 1, "added": 0, "removed": 0}, + } + # pad = max(0.25 x 10, 10 x 0.01) = 2.5 around each cluster + assert result["clusters"] == [ + {"space": "Model", "box": [-2.5, 197.5, 105.5, 202.5], "count": 1}, + {"space": "Model", "box": [7.5, 7.5, 12.5, 12.5], "count": 1}, + {"space": "Model", "box": [4992.5, 4992.5, 5007.5, 5007.5], "count": 1}, + ] + + +def test_cap_report_cuts_every_list_and_says_how_much(): + old = snap(*STILL) + new = snap(*STILL, *(line(f"N{i}", 0, 1000 * (i + 1), 100, 1000 * (i + 1)) for i in range(5))) + capped = cap_report(diff_snapshots(old, new, move_tol=10.0), limit=2) + assert len(capped["added"]) == 2 + assert capped["truncated"] == {"changed": 0, "added": 3, "removed": 0} + assert capped["summary"]["added"] == 5 + + +# -- block definitions ----------------------------------------------------------------- + + +def _blocks(radius: float, extra: bool): + doc = ezdxf.new() + doc.blocks.new("B1").add_circle((0, 0), radius) + doc.blocks.new("B2").add_line((0, 0), (10, 0)) + if extra: + doc.blocks.new("B3").add_line((0, 0), (0, 10)) + doc.blocks.new_anonymous_block().add_line((0, 0), (1, 1)) + return doc + + +def test_block_definitions_are_compared_by_content(): + old = block_signatures(_blocks(5.0, extra=False)) + new = block_signatures(_blocks(6.0, extra=True)) + assert sorted(old) == ["B1", "B2"] # the anonymous *U block is left out + assert block_changes(old, new) == { + "compared": True, + "added": ["B3"], + "removed": [], + "changed": ["B1"], + } + assert block_changes(None, new)["compared"] is False + + +# -- reading a revision ---------------------------------------------------------------- + + +def test_read_revision_refuses_a_file_over_the_size_limit(tmp_path, monkeypatch): + path = tmp_path / "big.dxf" + ezdxf.new().saveas(path) + monkeypatch.setattr(config.settings, "max_dxf_bytes", 10) + with pytest.raises(ValueError, match=r"over the MAX_DXF_BYTES limit of 10"): + read_revision(str(path)) + + +def test_read_revision_refuses_a_file_that_is_not_a_dxf(tmp_path): + path = tmp_path / "notes.dxf" + path.write_text("not a drawing\n", encoding="utf-8") + with pytest.raises(ValueError, match=r"not a readable DXF"): + read_revision(str(path)) + + +# -- the known edits on the synthetic plant -------------------------------------------- + + +def test_the_known_edits_and_nothing_else(tmp_path): + pair = build_revision_pair(tmp_path) + old, old_blocks = read_revision(pair["old"]) + new, new_blocks = read_revision(pair["new"]) + result = diff_snapshots(old, new, move_tol=pair["move_tol"]) + total = pair["old_count"] + assert result["summary"] == { + "old_entities": total, + "new_entities": total, + "unchanged": total - 4, + "changed": 3, + "added": 1, + "removed": 1, + } + changed = {row["old_handle"]: row for row in result["changed"]} + assert set(changed) == {pair["moved"], pair["text"], pair["insert"]} + assert {row["matched_by"] for row in result["changed"]} == {"position"} + moved = changed[pair["moved"]]["changes"] + assert list(moved) == ["moved_by"] + assert moved["moved_by"][0] == pytest.approx(pair["move_tol"] / 2.0, abs=1e-9) + assert moved["moved_by"][1] == pytest.approx(0.0, abs=1e-9) + assert changed[pair["text"]]["changes"] == { + "text": {"from": pair["text_before"], "to": NEW_TEXT} + } + assert changed[pair["insert"]]["changes"] == { + "attribs": [{"tag": "TAG", "from": TAG_BEFORE, "to": TAG_AFTER}] + } + assert [row["handle"] for row in result["removed"]] == [pair["deleted"]] + assert [(row["type"], row["layer"]) for row in result["added"]] == [("CIRCLE", ADDED_LAYER)] + assert block_changes(old_blocks, new_blocks) == { + "compared": True, + "added": [], + "removed": [], + "changed": [], + } + + +def test_renumbered_handles_alone_are_no_change(tmp_path): + pair = build_revision_pair(tmp_path, edits=False) + old, _ = read_revision(pair["old"]) + new, _ = read_revision(pair["new"]) + summary = diff_snapshots(old, new)["summary"] + assert (summary["changed"], summary["added"], summary["removed"]) == (0, 0, 0) + assert summary["unchanged"] == pair["old_count"] +``` + +- [ ] **Step 2: Run it to verify it fails** + +Run: `uv run --frozen pytest tests/test_understand_diff.py -q` +Expected: FAIL — `ERROR collecting tests/test_understand_diff.py` with `ModuleNotFoundError: No module named 'engineering.understand.diff'` (1 error, nothing collected). + +- [ ] **Step 3: Write `engineering/understand/diff.py`** + +```python +"""``drawing_diff``: two revisions of one drawing, matched by geometric signature. + +Spec §10. Each revision is read once into ``EntityRecord``s +(``engineering.understand.snapshot``) and the two are compared in memory. +Nothing in this module writes to a drawing; the revision clouds of +``markup=True`` are drawn by the tool through +``engineering.sheet.revision.add_revision``. + +Matching runs in three stages, in this order, and a record leaves the pool the +moment it is matched: + +1. **Exact.** Every record has a signature: its type, layer and space, its + geometry rounded to ``tol`` (points, bulges, closed flag, radius, arc + angles; the bounding box when it has no points), its text and text height, + its block name, rotation, scale and attributes. Equal signatures are the + same entity whatever their handles; where several old records share a + signature the one with the same handle is taken first. A LINE's two ends + are sorted, so a line drawn backwards is the same line. +2. **Handle.** A record left over on both sides with the same handle and the + same type is the same entity, edited. A save that renumbers handles would + make this stage pair strangers, so it runs only while handles look stable: + when at least half of the exact matches kept their handle. The report says + which way it went (``handles_stable``). +3. **Position.** Within one (type, layer, space), a leftover new record is + paired with the nearest leftover old record whose anchor (the mean of its + points, else the centre of its box) lies within ``move_tol``; the closest + pairs are taken first. ``move_tol`` defaults to 2 % of the drawing's robust + diagonal - the diagonal of the 1st-99th percentile box of every model-space + anchor of both revisions - so one stray entity kilometres away cannot + inflate it. + +What is left is added (new side) or removed (old side). A matched pair that +is not identical is *changed*, and the row says what changed: ``moved_by`` +(dx, dy) when the geometry is the old geometry translated, else ``geometry`` +from -> to; ``text``, ``attribs`` (by tag), ``radius``, ``angles``, +``height``, ``rotation``, ``scale``, ``block``, ``layer``, ``space``, +``bulges``, ``closed`` from -> to. + +The changes are grouped into clusters for the revision clouds: every changed, +added and removed record contributes its box (a changed one its old box and +its new box), boxes within ``move_tol`` of each other join one cluster, and a +cluster's box is padded by a quarter of ``move_tol`` (never less than ten +``tol``). ``count`` is the number of distinct changes in the cluster. + +Block definitions are compared apart (``block_signatures`` / +``block_changes``): the entity records carry INSERTs, not the blocks they +reference, so a redrawn definition changes every drawn copy while every INSERT +record stays identical. + +The 2 % move tolerance, the 1st/99th percentiles, the half-of-the-matches +handle rule and the cluster padding are this module's declared defaults, not +values from a standard. +""" + +from __future__ import annotations + +import hashlib +import math +import os +from collections import defaultdict +from collections.abc import Iterable, Sequence + +import config +from engineering.understand.snapshot import EntityRecord, Pt, Snapshot, records_from_doc + +__all__ = [ + "CATEGORIES", + "DEFAULT_LIMIT", + "DEFAULT_TOL", + "MOVE_FRACTION", + "PAD_FRACTION", + "ROBUST_QUANTILE", + "anchor", + "block_changes", + "block_signatures", + "cap_report", + "default_move_tol", + "diff_snapshots", + "read_revision", + "robust_box", + "signature", +] + +DEFAULT_TOL = 0.01 +#: The default move tolerance, as a share of the robust diagonal. +MOVE_FRACTION = 0.02 +#: The robust box runs from this quantile of the anchors to its complement. +ROBUST_QUANTILE = 0.01 +#: A cluster's box is padded by this share of ``move_tol``. +PAD_FRACTION = 0.25 +DEFAULT_LIMIT = 200 +CATEGORIES = ("changed", "added", "removed") + +Box = tuple[float, float, float, float] + + +def _positive(name: str, value) -> float: + try: + number = float(value) + except (TypeError, ValueError): + raise ValueError(f"{name} must be a positive number; got {value!r}") from None + if not math.isfinite(number) or number <= 0: + raise ValueError(f"{name} must be a positive number; got {value!r}") + return number + + +def _q(value, tol: float): + """``value`` on the ``tol`` grid; a non-finite value is kept as its repr.""" + number = float(value) + if not math.isfinite(number): + return repr(number) + return int(round(number / tol)) + + +def _qp(point, tol: float) -> tuple: + return (_q(point[0], tol), _q(point[1], tol)) + + +def signature(rec: EntityRecord, tol: float = DEFAULT_TOL) -> tuple: + """Everything that makes two records the same entity, rounded to ``tol``.""" + points = tuple(_qp(p, tol) for p in rec.points) + if rec.type == "LINE" and len(points) == 2: + points = tuple(sorted(points)) + if points: + geometry = points + elif rec.bbox is not None: + geometry = tuple(_q(v, tol) for v in rec.bbox) + else: + geometry = () + return ( + rec.type, + rec.layer, + rec.space, + geometry, + tuple(_q(b, tol) for b in rec.bulges), + bool(rec.closed), + None if rec.radius is None else _q(rec.radius, tol), + None + if rec.angles is None + else (_q(rec.angles[0] % 360.0, tol), _q(rec.angles[1] % 360.0, tol)), + rec.text, + None if rec.height is None else _q(rec.height, tol), + rec.block, + tuple(sorted(rec.attribs)), + _q(rec.rotation % 360.0, tol), + (_q(rec.scale[0], tol), _q(rec.scale[1], tol)), + ) + + +def anchor(rec: EntityRecord) -> Pt | None: + """The mean of the record's points, else the centre of its box, else None.""" + if rec.points: + n = len(rec.points) + point = (sum(p[0] for p in rec.points) / n, sum(p[1] for p in rec.points) / n) + elif rec.bbox is not None: + x0, y0, x1, y1 = rec.bbox + point = ((x0 + x1) / 2.0, (y0 + y1) / 2.0) + else: + return None + if not (math.isfinite(point[0]) and math.isfinite(point[1])): + return None + return point + + +def _box(rec: EntityRecord) -> Box | None: + if rec.bbox is not None: + return tuple(float(v) for v in rec.bbox) + if rec.points: + xs = [p[0] for p in rec.points] + ys = [p[1] for p in rec.points] + return (min(xs), min(ys), max(xs), max(ys)) + return None + + +def _quantile_pair(values: list[float], q: float) -> tuple[float, float]: + values = sorted(values) + last = len(values) - 1 + low = int(math.floor(q * last)) + return values[low], values[last - low] + + +def robust_box(*snaps: Snapshot) -> Box | None: + """The 1st-99th percentile box of the model-space anchors of ``snaps`` + (every space when model space is empty); None when there is no anchor.""" + records = [r for s in snaps for r in s.records] + anchors = [a for r in records if r.space == "Model" and (a := anchor(r)) is not None] + if not anchors: + anchors = [a for r in records if (a := anchor(r)) is not None] + if not anchors: + return None + x0, x1 = _quantile_pair([a[0] for a in anchors], ROBUST_QUANTILE) + y0, y1 = _quantile_pair([a[1] for a in anchors], ROBUST_QUANTILE) + return (x0, y0, x1, y1) + + +def default_move_tol(old: Snapshot, new: Snapshot, tol: float = DEFAULT_TOL) -> float: + """2 % of the robust diagonal of both revisions, never less than ``tol``.""" + box = robust_box(old, new) + diagonal = 0.0 if box is None else math.hypot(box[2] - box[0], box[3] - box[1]) + return max(MOVE_FRACTION * diagonal, tol) + + +def _nearest_pairs( + olds: Sequence[EntityRecord], + news: Sequence[EntityRecord], + left_old: Iterable[int], + left_new: Iterable[int], + move_tol: float, +) -> list[tuple[int, int]]: + """Stage 3: closest pairs first, within one (type, layer, space).""" + groups: dict[tuple, tuple[list[int], list[int]]] = defaultdict(lambda: ([], [])) + for i in left_old: + groups[(olds[i].type, olds[i].layer, olds[i].space)][0].append(i) + for j in left_new: + groups[(news[j].type, news[j].layer, news[j].space)][1].append(j) + candidates: list[tuple[float, str, str, int, int]] = [] + for old_ids, new_ids in groups.values(): + if not old_ids or not new_ids: + continue + grid: dict[tuple[int, int], list[tuple[int, Pt]]] = defaultdict(list) + for i in old_ids: + a = anchor(olds[i]) + if a is not None: + cell = (math.floor(a[0] / move_tol), math.floor(a[1] / move_tol)) + grid[cell].append((i, a)) + for j in new_ids: + b = anchor(news[j]) + if b is None: + continue + cx, cy = math.floor(b[0] / move_tol), math.floor(b[1] / move_tol) + for dx in (-1, 0, 1): + for dy in (-1, 0, 1): + for i, a in grid.get((cx + dx, cy + dy), ()): + distance = math.dist(a, b) + if distance <= move_tol: + candidates.append((distance, olds[i].handle, news[j].handle, i, j)) + candidates.sort() + used_old: set[int] = set() + used_new: set[int] = set() + pairs: list[tuple[int, int]] = [] + for _distance, _old_handle, _new_handle, i, j in candidates: + if i in used_old or j in used_new: + continue + used_old.add(i) + used_new.add(j) + pairs.append((i, j)) + return pairs + + +def _number_changed(a, b, tol: float, *, modulo: float | None = None) -> bool: + if a is None or b is None: + return (a is None) != (b is None) + if modulo is not None: + a, b = a % modulo, b % modulo + return _q(a, tol) != _q(b, tol) + + +def _movement(o: EntityRecord, n: EntityRecord, tol: float) -> dict: + """``moved_by`` for a pure translation, ``geometry`` for anything else.""" + if o.points and n.points: + before, after = list(o.points), list(n.points) + elif not o.points and not n.points: + if o.bbox is None or n.bbox is None: + if (o.bbox is None) != (n.bbox is None): + return {"geometry": {"from": o.bbox, "to": n.bbox}} + return {} + before = [(o.bbox[0], o.bbox[1]), (o.bbox[2], o.bbox[3])] + after = [(n.bbox[0], n.bbox[1]), (n.bbox[2], n.bbox[3])] + else: + return { + "geometry": {"from": [list(p) for p in o.points], "to": [list(p) for p in n.points]} + } + a, b = anchor(o), anchor(n) + if a is None or b is None: + return {"geometry": {"from": [list(p) for p in before], "to": [list(p) for p in after]}} + dx, dy = b[0] - a[0], b[1] - a[1] + half = tol / 2.0 + translated = len(before) == len(after) and all( + abs(p[0] + dx - q[0]) <= half and abs(p[1] + dy - q[1]) <= half + for p, q in zip(before, after, strict=True) + ) + if not translated: + return {"geometry": {"from": [list(p) for p in before], "to": [list(p) for p in after]}} + if abs(dx) > half or abs(dy) > half: + return {"moved_by": [dx, dy]} + return {} + + +def _attrib_changes(o: EntityRecord, n: EntityRecord) -> list[dict]: + before, after = dict(o.attribs), dict(n.attribs) + return [ + {"tag": tag, "from": before.get(tag), "to": after.get(tag)} + for tag in sorted(set(before) | set(after)) + if before.get(tag) != after.get(tag) + ] + + +def _changes(o: EntityRecord, n: EntityRecord, tol: float) -> dict: + out: dict = {} + if o.layer != n.layer: + out["layer"] = {"from": o.layer, "to": n.layer} + if o.space != n.space: + out["space"] = {"from": o.space, "to": n.space} + out.update(_movement(o, n, tol)) + if tuple(_q(b, tol) for b in o.bulges) != tuple(_q(b, tol) for b in n.bulges): + out["bulges"] = {"from": list(o.bulges), "to": list(n.bulges)} + if bool(o.closed) != bool(n.closed): + out["closed"] = {"from": bool(o.closed), "to": bool(n.closed)} + if _number_changed(o.radius, n.radius, tol): + out["radius"] = {"from": o.radius, "to": n.radius} + if (o.angles is None) != (n.angles is None) or ( + o.angles is not None + and n.angles is not None + and any( + _number_changed(a, b, tol, modulo=360.0) + for a, b in zip(o.angles, n.angles, strict=True) + ) + ): + out["angles"] = { + "from": None if o.angles is None else list(o.angles), + "to": None if n.angles is None else list(n.angles), + } + if o.text != n.text: + out["text"] = {"from": o.text, "to": n.text} + if _number_changed(o.height, n.height, tol): + out["height"] = {"from": o.height, "to": n.height} + if o.block != n.block: + out["block"] = {"from": o.block, "to": n.block} + attribs = _attrib_changes(o, n) + if attribs: + out["attribs"] = attribs + if _number_changed(o.rotation, n.rotation, tol, modulo=360.0): + out["rotation"] = {"from": o.rotation, "to": n.rotation} + if any(_number_changed(a, b, tol) for a, b in zip(o.scale, n.scale, strict=True)): + out["scale"] = {"from": list(o.scale), "to": list(n.scale)} + return out + + +def _row(rec: EntityRecord) -> dict: + at = anchor(rec) + return { + "handle": rec.handle, + "type": rec.type, + "layer": rec.layer, + "space": rec.space, + "at": None if at is None else [at[0], at[1]], + } + + +def _tally(changed: list[dict], added: list[dict], removed: list[dict]) -> tuple[dict, dict]: + by_layer: dict[str, dict[str, int]] = {} + by_type: dict[str, dict[str, int]] = {} + for category, rows in (("changed", changed), ("added", added), ("removed", removed)): + for row in rows: + for table, key in ((by_layer, row["layer"]), (by_type, row["type"])): + counts = table.setdefault(key, {"changed": 0, "added": 0, "removed": 0}) + counts[category] += 1 + return dict(sorted(by_layer.items())), dict(sorted(by_type.items())) + + +def _box_gap(a: Box, b: Box) -> float: + dx = max(0.0, b[0] - a[2], a[0] - b[2]) + dy = max(0.0, b[1] - a[3], a[1] - b[3]) + return math.hypot(dx, dy) + + +def _clusters(items: list[tuple[str, Box, int]], *, link: float, pad: float) -> list[dict]: + """Single-linkage clusters of change boxes, per space, swept along x.""" + parent = list(range(len(items))) + + def find(k: int) -> int: + while parent[k] != k: + parent[k] = parent[parent[k]] + k = parent[k] + return k + + by_space: dict[str, list[int]] = defaultdict(list) + for k, (space, _box_value, _key) in enumerate(items): + by_space[space].append(k) + for members in by_space.values(): + members.sort(key=lambda k: items[k][1][0]) + active: list[int] = [] + for k in members: + box = items[k][1] + active = [m for m in active if items[m][1][2] + link >= box[0]] + for m in active: + if _box_gap(items[m][1], box) <= link: + parent[find(k)] = find(m) + active.append(k) + grouped: dict[int, list[int]] = defaultdict(list) + for k in range(len(items)): + grouped[find(k)].append(k) + clusters = [] + for members in grouped.values(): + boxes = [items[k][1] for k in members] + clusters.append( + { + "space": items[members[0]][0], + "box": [ + min(b[0] for b in boxes) - pad, + min(b[1] for b in boxes) - pad, + max(b[2] for b in boxes) + pad, + max(b[3] for b in boxes) + pad, + ], + "count": len({items[k][2] for k in members}), + } + ) + clusters.sort(key=lambda c: (c["space"], c["box"])) + return clusters + + +def diff_snapshots( + old: Snapshot, new: Snapshot, *, tol: float = DEFAULT_TOL, move_tol: float | None = None +) -> dict: + """Match ``new`` against ``old`` and report every change (spec §10). + + Refuses a non-positive ``tol`` or ``move_tol`` before any work. + """ + tol = _positive("tol", tol) + move_tol = ( + default_move_tol(old, new, tol) if move_tol is None else _positive("move_tol", move_tol) + ) + olds = list(old.records) + news = list(new.records) + + # 1. exact + pools: dict[tuple, list[int]] = defaultdict(list) + for i, rec in enumerate(olds): + pools[signature(rec, tol)].append(i) + used_old: set[int] = set() + left_new: list[int] = [] + exact = kept_handle = 0 + for j, rec in enumerate(news): + pool = pools.get(signature(rec, tol)) + if not pool: + left_new.append(j) + continue + at = next((k for k, i in enumerate(pool) if olds[i].handle == rec.handle), 0) + i = pool.pop(at) + used_old.add(i) + exact += 1 + kept_handle += int(olds[i].handle == rec.handle) + left_old = [i for i in range(len(olds)) if i not in used_old] + handles_stable = exact == 0 or 2 * kept_handle >= exact + + pairs: list[tuple[int, int, str]] = [] + # 2. handle + if handles_stable: + by_handle = {olds[i].handle: i for i in left_old} + still_new: list[int] = [] + for j in left_new: + i = by_handle.get(news[j].handle) + if i is not None and olds[i].type == news[j].type: + pairs.append((i, j, "handle")) + del by_handle[news[j].handle] + else: + still_new.append(j) + left_new = still_new + taken = {i for i, _j, _how in pairs} + left_old = [i for i in left_old if i not in taken] + + # 3. position + pairs.extend( + (i, j, "position") for i, j in _nearest_pairs(olds, news, left_old, left_new, move_tol) + ) + taken_old = {i for i, _j, _how in pairs} + taken_new = {j for _i, j, _how in pairs} + + changed: list[dict] = [] + items: list[tuple[str, Box, int]] = [] + for key, (i, j, how) in enumerate( + sorted(pairs, key=lambda p: (olds[p[0]].handle, news[p[1]].handle)) + ): + o, n = olds[i], news[j] + at = anchor(n) + changed.append( + { + "old_handle": o.handle, + "new_handle": n.handle, + "type": n.type, + "layer": n.layer, + "space": n.space, + "matched_by": how, + "at": None if at is None else [at[0], at[1]], + "changes": _changes(o, n, tol), + } + ) + for rec in (o, n): + box = _box(rec) + if box is not None: + items.append((rec.space, box, key)) + base = len(changed) + removed_recs = sorted((olds[i] for i in left_old if i not in taken_old), key=lambda r: r.handle) + added_recs = sorted((news[j] for j in left_new if j not in taken_new), key=lambda r: r.handle) + for offset, rec in enumerate(removed_recs + added_recs): + box = _box(rec) + if box is not None: + items.append((rec.space, box, base + offset)) + removed = [_row(rec) for rec in removed_recs] + added = [_row(rec) for rec in added_recs] + by_layer, by_type = _tally(changed, added, removed) + pad = max(PAD_FRACTION * move_tol, 10.0 * tol) + return { + "tol": tol, + "move_tol": move_tol, + "handles_stable": handles_stable, + "summary": { + "old_entities": len(olds), + "new_entities": len(news), + "unchanged": exact, + "changed": len(changed), + "added": len(added), + "removed": len(removed), + }, + "by_layer": by_layer, + "by_type": by_type, + "changed": changed, + "added": added, + "removed": removed, + "clusters": _clusters(items, link=move_tol, pad=pad), + } + + +def cap_report(result: dict, limit: int = DEFAULT_LIMIT) -> dict: + """``result`` with each list of ``CATEGORIES`` cut to ``limit`` rows and + ``truncated`` saying how many rows each cut dropped.""" + if int(limit) < 1: + raise ValueError(f"limit must be at least 1; got {limit!r}") + capped = dict(result) + capped["truncated"] = {} + for category in CATEGORIES: + rows = list(result.get(category) or []) + capped[category] = rows[: int(limit)] + capped["truncated"][category] = max(0, len(rows) - int(limit)) + return capped + + +# -- block definitions ----------------------------------------------------------------- + +_BLOCK_ATTRS = ( + "start", + "end", + "center", + "insert", + "radius", + "start_angle", + "end_angle", + "rotation", + "height", + "text", + "tag", + "name", + "xscale", + "yscale", + "major_axis", + "ratio", +) + + +def _q_value(value, tol: float): + if isinstance(value, bool): + return value + if isinstance(value, (int, float)): + return _q(value, tol) + if isinstance(value, str): + return value + try: + return tuple(_q(v, tol) for v in value) + except TypeError: + return repr(value) + + +def _block_entity_key(entity, tol: float) -> str: + key: list = [entity.dxftype(), entity.dxf.get("layer", "0")] + for attr in _BLOCK_ATTRS: + if entity.dxf.hasattr(attr): + key.append((attr, _q_value(entity.dxf.get(attr), tol))) + kind = entity.dxftype() + if kind == "LWPOLYLINE": + key.append(("points", tuple(_q_value(p, tol) for p in entity.get_points("xyb")))) + key.append(("closed", bool(entity.closed))) + elif kind == "POLYLINE": + key.append(("points", tuple(_q_value(v.dxf.location, tol) for v in entity.vertices))) + elif kind == "MTEXT": + key.append(("mtext", entity.text)) + return repr(tuple(key)) + + +def block_signatures(doc, *, tol: float = DEFAULT_TOL) -> dict[str, str]: + """``{block name: digest}`` for every named block definition of an ezdxf + document: its base point and, per entity, its type, layer and main DXF + attributes rounded to ``tol``. Layout blocks and anonymous blocks + (``*U``, ``*D`` ...) are left out - their names are not stable across saves.""" + tol = _positive("tol", tol) + out: dict[str, str] = {} + for block in doc.blocks: + name = block.name + if block.block_record.is_any_layout or name.startswith("*"): + continue + base = _q_value(block.block.dxf.base_point, tol) + rows = sorted(_block_entity_key(entity, tol) for entity in block) + out[name] = hashlib.sha256(repr((base, rows)).encode("utf-8")).hexdigest() + return dict(sorted(out.items())) + + +def block_changes(old: dict[str, str] | None, new: dict[str, str] | None) -> dict: + """Block definitions added, removed and redrawn between two revisions.""" + if old is None or new is None: + return { + "compared": False, + "reason": "block definitions are compared only when both revisions are DXF files " + "or the current document of the headless engine", + } + return { + "compared": True, + "added": sorted(set(new) - set(old)), + "removed": sorted(set(old) - set(new)), + "changed": sorted(name for name in set(old) & set(new) if old[name] != new[name]), + } + + +def read_revision(path: str, *, tol: float = DEFAULT_TOL) -> tuple[Snapshot, dict[str, str]]: + """One read of a DXF revision: its records and its block signatures. + + Refused before reading: a file that cannot be stat'ed and one over + ``MAX_DXF_BYTES`` (the size and the variable named); a file ezdxf cannot + parse is refused with ezdxf's reason. + """ + import ezdxf + + limit = int(config.settings.max_dxf_bytes) + try: + size = os.path.getsize(path) + except OSError as exc: + raise ValueError(f"{path}: cannot be read ({exc.strerror or exc})") from exc + if limit > 0 and size > limit: + raise ValueError( + f"{path} is {size} bytes, over the MAX_DXF_BYTES limit of {limit}; " + "raise MAX_DXF_BYTES to read it" + ) + try: + doc = ezdxf.readfile(path) + except (OSError, ezdxf.DXFError) as exc: + raise ValueError(f"{path}: not a readable DXF ({exc})") from exc + return records_from_doc(doc, source=str(path)), block_signatures(doc, tol=tol) +``` + +- [ ] **Step 4: Run the engine tests to verify they pass** + +Run: `uv run --frozen pytest tests/test_understand_diff.py -q` +Expected: `19 passed`. If `test_the_known_edits_and_nothing_else` fails because Task 1's `records_from_doc` stores a record differently from what this module assumes (for example a CIRCLE without its centre in `points`), fix the module to Task 1's truth, keep the assertion, and record the deviation — never weaken the test. + +- [ ] **Step 5: Write the failing wire tests** + +Create `tests/test_understand_diff_tool.py`: + +```python +"""drawing_diff over the MCP wire, on the headless engine, against the synthetic revisions.""" + +from __future__ import annotations + +import pytest +import pytest_asyncio +from fastmcp import Client + +import server +from engineering.sheet.frames import SHEET_LAYER +from tests.fixtures.revisions import build_revision_pair + +pytestmark = pytest.mark.asyncio + + +@pytest_asyncio.fixture +async def client(monkeypatch): + """In-memory client on a fresh headless document - never the operator's AutoCAD.""" + monkeypatch.setenv("AUTOCAD_MCP_BACKEND", "ezdxf") + async with Client(server.mcp) as connected: + status = (await connected.call_tool("system_status", {})).structured_content or {} + assert status.get("backend") == "ezdxf", "these tests must never touch live AutoCAD" + await connected.call_tool("drawing_new", {}) + yield connected + + +async def _total(client) -> int: + stats = (await client.call_tool("analysis_entity_stats", {})).structured_content + return stats["total_entities"] + + +async def test_the_current_document_against_its_older_revision(client, tmp_path): + pair = build_revision_pair(tmp_path) + await client.call_tool("drawing_open", {"path": pair["new"]}) + before = await _total(client) + result = ( + await client.call_tool( + "drawing_diff", {"old_path": pair["old"], "move_tol": pair["move_tol"]} + ) + ).structured_content + assert (result["summary"]["changed"], result["summary"]["added"]) == (3, 1) + assert result["summary"]["removed"] == 1 + assert {row["old_handle"] for row in result["changed"]} == { + pair["moved"], + pair["text"], + pair["insert"], + } + assert result["blocks"]["compared"] is True and result["blocks"]["changed"] == [] + assert result["truncated"] == {"changed": 0, "added": 0, "removed": 0} + assert await _total(client) == before, "a diff without markup writes nothing" + + +async def test_two_paths_and_the_list_cap(client, tmp_path): + pair = build_revision_pair(tmp_path) + result = ( + await client.call_tool( + "drawing_diff", + { + "old_path": pair["old"], + "new_path": pair["new"], + "move_tol": pair["move_tol"], + "limit": 1, + }, + ) + ).structured_content + assert len(result["changed"]) == 1 + assert result["truncated"]["changed"] == 2 + assert result["summary"]["changed"] == 3 + + +async def test_markup_clouds_every_model_space_cluster(client, tmp_path): + pair = build_revision_pair(tmp_path) + await client.call_tool("drawing_open", {"path": pair["new"]}) + result = ( + await client.call_tool( + "drawing_diff", + {"old_path": pair["old"], "move_tol": pair["move_tol"], "markup": True, "rev": "b"}, + ) + ).structured_content + model = [c for c in result["clusters"] if c["space"] == "Model"] + markup = result["markup"] + assert markup["created"] is True and markup["rev"] == "B" + assert len(markup["clouds"]) == len(model) >= 1 + for handle in markup["clouds"]: + cloud = (await client.call_tool("entity_get", {"handle": handle})).structured_content + assert (cloud["type"], cloud["layer"]) == ("LWPOLYLINE", SHEET_LAYER) + + +@pytest.mark.parametrize( + ("arguments", "fragment"), + [ + ( + {"markup": True, "rev": "B", "new_path": "NEW"}, + "markup draws revision clouds on the current", + ), + ({"markup": True}, "markup needs `rev`"), + ({"markup": True, "rev": "TOO-LONG"}, "markup needs `rev`"), + ], +) +async def test_markup_refusals_come_before_any_read(client, tmp_path, arguments, fragment): + pair = build_revision_pair(tmp_path) + if arguments.get("new_path") == "NEW": + arguments = {**arguments, "new_path": pair["new"]} + result = await client.call_tool( + "drawing_diff", {"old_path": pair["old"], **arguments}, raise_on_error=False + ) + assert result.is_error is True + assert fragment in result.content[0].text + + +async def test_a_missing_revision_is_refused_by_parameter(client, tmp_path): + result = await client.call_tool( + "drawing_diff", {"old_path": str(tmp_path / "nowhere.dxf")}, raise_on_error=False + ) + assert result.is_error is True + assert "old_path:" in result.content[0].text and "does not exist" in result.content[0].text + + +async def test_the_diff_is_advertised_as_non_destructive(client): + tools = {tool.name: tool for tool in await client.list_tools()} + assert tools["drawing_diff"].annotations.destructiveHint is False +``` + +- [ ] **Step 6: Run them to verify they fail** + +Run: `uv run --frozen pytest tests/test_understand_diff_tool.py -q` +Expected: FAIL — `8 failed`: the three calling tests with `fastmcp.exceptions.ToolError: Not found: Unknown tool: 'drawing_diff'`, the four refusal tests on their `assert fragment in ...` / `does not exist` assertions (the error text is the unknown-tool message), and `test_the_diff_is_advertised_as_non_destructive` with `KeyError: 'drawing_diff'`. + +- [ ] **Step 7: Register `drawing_diff` in `server.py`** + +Insert this block directly above the three-line RESOURCES banner (`# ----…` / `# ── RESOURCES ───…` / `# ----…`) at the end of SECTION 25, leaving two blank lines before the banner. It is group D's own copy of the SECTION 26 banner (the banner protocol: U's branch opens the same banner with its own tools; the merge keeps one banner at 4 with U's block, then D's): + +```python +# --------------------------------------------------------------------------- +# ── SECTION 26: Understanding & QA (1 tool) ───────────────────────────────── +# --------------------------------------------------------------------------- + + +async def _diff_revision(backend, path: str | None, *, tol: float, role: str): + """One revision for ``drawing_diff``: its records and, where they can be + read, its block signatures (None otherwise). + + A DXF path is read once, here, for both. The current document goes through + ``take_snapshot``; its block table is read only on the headless engine, + under the backend's own lock. Any other path (a DWG) goes through the + snapshot too, which opens it read-only on the live engine. + """ + import asyncio + + from engineering.understand.diff import block_signatures, read_revision + from engineering.understand.snapshot import take_snapshot + + if path is None: + snap = await take_snapshot(backend, None) + blocks = None + run = getattr(backend, "_async", None) + if backend.name == "ezdxf" and run is not None: + blocks = await run(lambda: block_signatures(backend._doc, tol=tol)) + return snap, blocks + validated = validate_path(path) + if not validated.is_file(): + raise ToolError(f"{role}: {validated} does not exist or is not a file") + if validated.suffix.lower() == ".dxf": + try: + return await asyncio.to_thread(read_revision, str(validated), tol=tol) + except ValueError as exc: + raise ToolError(f"{role}: {exc}") from exc + return await take_snapshot(backend, str(validated)), None + + +async def _diff_markup(backend, result: dict, new_snap, *, rev: str, description: str) -> dict: + """Revision clouds around every model-space cluster of change, filed under + ``rev`` through ``engineering.sheet.revision.add_revision`` - the only + write ``drawing_diff`` ever makes.""" + from engineering.sheet.bom import LIST_WIDTH, ROW_HEIGHT + from engineering.sheet.revision import add_revision + from engineering.understand.diff import robust_box + + clusters = [c for c in result["clusters"] if c["space"] == "Model"] + paper = len(result["clusters"]) - len(clusters) + if not clusters: + return { + "created": False, + "clouds": [], + "reason": "no change in model space to cloud", + "paper_space_clusters": paper, + } + boxes = [c["box"] for c in clusters] + shortest = min(min(b[2] - b[0], b[3] - b[1]) for b in boxes) + box = robust_box(new_snap) or ( + min(b[0] for b in boxes), + min(b[1] for b in boxes), + max(b[2] for b in boxes), + max(b[3] for b in boxes), + ) + summary = result["summary"] + text = description or ( + f"drawing_diff: {summary['changed']} changed, {summary['added']} added, " + f"{summary['removed']} removed" + ) + drawn = await add_revision( + backend, + rev=rev, + description=text, + clouds=boxes, + at=(box[2] - LIST_WIDTH, box[3] + 3.0 * ROW_HEIGHT), + cloud_segment=shortest / 4.0, + ) + drawn["paper_space_clusters"] = paper + return drawn + + +@cad_tool( + summary="Compare two revisions of a drawing: what moved, changed, was added or removed.", + cost="mutate", +) +@mcp.tool( + annotations={"title": "Drawing: Diff Two Revisions", "destructiveHint": False}, + tags={"analysis", "query"}, +) +async def drawing_diff( + old_path: Annotated[ + str, + Field( + description="The OLDER revision: a DXF path (a DWG only on the live engine, which " + "exports it read-only)." + ), + ], + new_path: Annotated[ + str | None, + Field( + default=None, + description="The NEWER revision: a DXF path, or omitted for the current document.", + ), + ] = None, + tol: Annotated[ + float, + Field( + default=0.01, + gt=0, + description="Geometry is compared rounded to this many drawing units.", + ), + ] = 0.01, + move_tol: Annotated[ + float | None, + Field( + default=None, + gt=0, + description="An unmatched entity is paired with the nearest one of its type and layer " + "within this distance and reported as changed; default 2 % of the drawing's robust " + "diagonal.", + ), + ] = None, + limit: Annotated[ + int, + Field( + default=200, ge=1, le=5000, description="Rows kept per list (changed, added, removed)." + ), + ] = 200, + markup: Annotated[ + bool, + Field( + default=False, + description="Draw revision clouds around every cluster of change on the current " + "document (the newer revision) and file them under `rev`.", + ), + ] = False, + rev: Annotated[ + str, + Field( + default="", + description="Revision code for markup=True: one to three capitals or digits.", + ), + ] = "", + description: Annotated[ + str, + Field(default="", description="Revision-row text for markup=True; default: the counts."), + ] = "", + ctx: Context = None, +) -> dict: + """Compare two revisions of one drawing by geometric signature (spec §10). + + Entities are matched exactly first (type, layer, geometry rounded to `tol`, + text, block, attributes - handles do not matter), then by handle while + handles look stable, then by type + layer + nearest position within + `move_tol`; what is left is added or removed. The report counts changes by + layer and type, lists each changed entity with what changed (moved by, text + from -> to, attribute from -> to, radius from -> to, ...), compares the + block definitions when both revisions are DXF files or the headless current + document, and groups the changes into clusters. + + Read-only unless `markup=True`: then revision clouds are drawn around the + model-space clusters on the current document, which must be the newer + revision, and filed as one revision row - the only write this tool makes. + + Refused before anything is read: `markup=True` with `new_path` (the clouds go + on the current document), `markup=True` without a valid `rev`, `markup=True` + on an engine without the revcloud capability (the live engine - + `capability: "revcloud"`), a path that does not exist, a DXF over + MAX_DXF_BYTES (the size and the variable named), a file that is not a + readable DXF, and a non-positive `tol` or `move_tol`. + """ + from backends.base import UnsupportedCapabilityError + from engineering.sheet.revision import REV_RE + from engineering.understand.diff import block_changes, cap_report, diff_snapshots + + backend = _backend(ctx) + if markup: + if new_path: + raise ToolError( + "markup draws revision clouds on the current document, which must be the newer " + "revision: open it with drawing_open and call again without new_path" + ) + if not REV_RE.fullmatch(rev.strip().upper()): + raise ToolError( + f"markup needs `rev`, the revision code the clouds are filed under - one to " + f"three capitals or digits (A, B, 01); got {rev!r}" + ) + feature = backend.capabilities().features.get("revcloud") + if feature is None or not feature.supported: + raise UnsupportedCapabilityError( + "revcloud", + f"drawing_diff(markup=True): the {backend.name} backend cannot draw revision " + "clouds (no ActiveX member; REVCLOUD is command-line only). Run the diff " + "without markup, or on the ezdxf engine.", + ) + await ctx.info(f"drawing_diff: {old_path} -> {new_path or 'the current document'}") + old_snap, old_blocks = await _diff_revision(backend, old_path, tol=tol, role="old_path") + new_snap, new_blocks = await _diff_revision(backend, new_path, tol=tol, role="new_path") + try: + result = diff_snapshots(old_snap, new_snap, tol=tol, move_tol=move_tol) + except ValueError as exc: + raise ToolError(str(exc)) from exc + result["old"] = old_snap.source + result["new"] = new_snap.source + result["blocks"] = block_changes(old_blocks, new_blocks) + report = cap_report(result, limit) + if markup: + report["markup"] = await _diff_markup( + backend, result, new_snap, rev=rev.strip().upper(), description=description + ) + return report +``` + +`validate_path`, `ToolError`, `Annotated`, `Field`, `Context`, `cad_tool`, `mcp` and `_backend` are already imported or defined in `server.py`; `asyncio` is imported inside `_diff_revision` because `server.py` does not import it at module level. `UnsupportedCapabilityError(key, reason)` is the same refusal `add_revision` raises, fired here before either revision is read. + +- [ ] **Step 8: Add the alias record** + +In `discovery/aliases.py`, insert directly after the `"arch_plan_from_spec"` record, before the closing `}` of `TOOL_ALIASES`: + +```python + # ── Understanding & QA (SECTION 26, track H group D) ──────────────────── + "drawing_diff": ToolAliases( + acad=("COMPARE",), + synonyms=( + "compare two drawings", + "compare two revisions", + "what changed between revisions", + "drawing differences", + "cloud the changes", + "revizyonlar arası değişiklik", + "revizyonlar arasi degisiklik", + ), + ), +``` + +`COMPARE` is AutoCAD's DWG Compare command and no other record claims it, so `SHARED_ACAD_COMMANDS` is untouched. `DWGCOMPARE` and the five-language words of spec §12 (fark, karşılaştır, revizyon farkı, compare …) are Task 11's integration vocabulary and are deliberately not added here, so Task 11 cannot collide with a duplicate synonym. + +- [ ] **Step 9: Run the wire tests to verify they pass** + +Run: `uv run --frozen pytest tests/test_understand_diff.py tests/test_understand_diff_tool.py -q` +Expected: `27 passed`. + +- [ ] **Step 10: Regenerate the inventory and bump the documented counts** + +Run: `uv run --frozen python scripts/generate_tool_inventory.py` +Expected output: `wrote docs/tool-inventory.json: 242 tools, 8 resources, 5 prompts, 26 groups` + +In `README.md`, in the release-snapshot lines, change `241 tools ·` to `242 tools ·` and `> 241 is the **registered** count; a default install advertises 236 over` to `> 242 is the **registered** count; a default install advertises 237 over`; in the caption under the "What you get" table change `241 tools in 26 groups;` to `242 tools in 26 groups;`. Leave the collected-tests figure alone (Task 11 retakes it on main) and leave the "… command names and … synonym phrases covering all 241 tools" sentence alone: every group branch adds alias records, so that sentence is recomputed once after the merge. + +- [ ] **Step 11: Bump the frozen group-size snapshot** + +In `tests/test_tool_registry.py::test_tool_group_sizes_are_unchanged`: + +- the docstring's first line `"""Frozen snapshot of the surface (241 tools, 26 groups).` becomes `"""Frozen snapshot of the surface (242 tools, 26 groups).`; +- in the asserted dict, `"analysis": 12,` becomes `"analysis": 13,`; +- `assert sum(sizes.values()) == 241` becomes `assert sum(sizes.values()) == 242`; +- replace the docstring's closing lines + +```python + had grown while the tag was unranked on them returned to its + pre-track-F value. + """ +``` + +with + +```python + had grown while the tag was unranked on them returned to its + pre-track-F value. + + Track H group D opened its copy of SECTION 26 (Understanding & QA) with + `drawing_diff`, tagged `analysis` and `query`; `analysis` outranks `query` + in `_GROUP_TAG_PRIORITY`, so it files under `analysis` (12 -> 13). This + snapshot is branch-local: the track H merge recomputes it once the + understand, network and diff branches have landed. + """ +``` + +- [ ] **Step 12: Gates** + +Run each and require it clean: + +```bash +uv run --frozen ruff check engineering/understand/diff.py tests/fixtures/revisions.py tests/test_understand_diff.py tests/test_understand_diff_tool.py server.py discovery/aliases.py tests/test_tool_registry.py +uv run --frozen ruff format engineering/understand/diff.py tests/fixtures/revisions.py tests/test_understand_diff.py tests/test_understand_diff_tool.py +uv run --frozen ruff format --check server.py discovery/aliases.py tests/test_tool_registry.py +uv run --frozen python scripts/generate_tool_inventory.py --check +uv run --frozen pytest tests/test_understand_diff.py tests/test_understand_diff_tool.py -q +uv run --frozen pytest tests/test_tool_registry.py tests/test_tool_inventory.py tests/test_tool_packs.py tests/test_discovery_aliases.py tests/test_discovery_transform.py tests/test_capability_contract.py -q +uv run --frozen pytest tests/test_release_consistency.py -q --deselect tests/test_release_consistency.py::test_document_numbers_match_the_tree +uv run --frozen pytest -q --deselect tests/test_release_consistency.py::test_document_numbers_match_the_tree +``` + +`generate_tool_inventory.py --check` prints `ok: docs/tool-inventory.json matches the live surface`. `test_section_header_counts_match_decorators` passes: the banner says `(1 tool)` and one `@mcp.tool` decorator sits under it (the two helpers are plain functions). `test_a_read_only_tool_is_costed_read` is not engaged: `drawing_diff` carries no `readOnlyHint` (markup writes), and its cost is `mutate`. No capability key is added — `revcloud` already exists in both maps — so `tests/test_capability_contract.py` passes untouched. The deselect is the known-red collected-tests drift of the Global Constraints; say so in the task report. If `tests/test_discovery_transform.py`'s golden ranks move, tune only the new record's synonyms, re-run, and record it. + +- [ ] **Step 13: Commit** + +```bash +git add engineering/understand/diff.py tests/fixtures/revisions.py tests/test_understand_diff.py tests/test_understand_diff_tool.py server.py discovery/aliases.py docs/tool-inventory.json README.md tests/test_tool_registry.py +git -c user.name="Umut CAN" -c user.email="umutcan.edizaslan@gmail.com" commit -m "$(cat <<'MSG' +feat(understand): drawing_diff - two revisions matched by geometric signature + +engineering/understand/diff.py reads each revision once into entity +records and matches them in three stages: exact signature (type, layer, +geometry rounded to tol, text, block, rotation, scale, attributes; +handles ignored), then handle while at least half of the exact matches +kept theirs, then type + layer + nearest anchor within move_tol (default +2 % of the 1st-99th percentile diagonal). The report counts changes by +layer and type, says what changed per entity (moved by, text, attribute, +radius ... from -> to), compares named block definitions by content and +groups the changes into clusters. + +drawing_diff opens group D's copy of SECTION 26. It is read-only unless +markup=True, which files revision clouds around the model-space clusters +on the current document through add_revision - refused before any read +with new_path, without a valid rev, or on an engine without revcloud. + +Tests: the synthetic plant P&ID edited five ways (a move, a text, an +attribute, an add, a delete) under renumbered handles reports exactly +those five edits; renumbering alone reports nothing. + +Co-Authored-By: Claude Opus 5.5 (1M context) +Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd +MSG +)" +``` + +--- + +### Task 10: drawing_topology_check and the three topo_* focuses + +**Files:** +- Create: `engineering/understand/topology.py` +- Create: `engineering/understand/critique.py` +- Modify: `engineering/plan_spec.py` (`CritiqueFocus` and `ALL_CRITIQUE_FOCUSES` gain the three `topo_*` focuses) +- Modify: `engineering/critique.py` (`TOPO_DISPATCH` registered after `ARCH_DISPATCH`) +- Modify: `server.py` (SECTION 26 banner `(1 tool)` → `(2 tools)`, `drawing_topology_check` after `drawing_diff`; the `drawing_critique` focus description and the `drawing_finalize` docstring name the new focuses) +- Modify: `discovery/aliases.py` (the `drawing_topology_check` record) +- Modify: `docs/tool-inventory.json` (regenerated), `README.md` (242 → 243, advertised 237 → 238, caption 242 → 243) +- Modify: `tests/test_tool_registry.py` (`analysis` 13 → 14, total 243) +- Test: `tests/test_understand_topology.py`, `tests/test_understand_critique.py`, `tests/test_understand_topology_tool.py` + +**Interfaces:** +- Consumes: `EntityRecord`, `Snapshot`, `Pt`, `read_snapshot(path: str) -> Snapshot` and `async take_snapshot(backend, path: str | None = None) -> Snapshot` (Task 1; `EntityRecord.points[0]` is the centre of an ARC or a CIRCLE, `angles` its start/end in degrees CCW, `bulges` one per polyline vertex); `vocab.classify_layer(name: str) -> dict` with `discipline` / `service` / `confidence` / `keyword` (Task 1); `build_plant_pair(directory) -> dict` and its `layer_services` truth (Task 2); `engineering.plan_spec.Issue`; `engineering.layers.PID_LAYERS`, `engineering.pid.lines.PID_LINE_LAYERS`, `engineering.arch.layers.ARCH_LAYERS` / `ARCH_LAYER_ROLES`; `backends.quarantine.DocumentQuarantineError`; the `pid_from_spec` / `arch_plan_from_spec` engines `engineering.pid.spec.run_spec(backend, EXAMPLE_SPEC)` and `engineering.arch.spec.draw_plan_from_spec(backend, EXAMPLE_SPEC)`. +- Produces (the skeleton's pinned signature first): `topology_findings(snap: Snapshot, *, layers, gap=None, tol=0.01) -> dict` with keys `layers`, `gap`, `tol`, `dangling` (`[{"handle", "layer", "end": "start" | "end", "at", "space"}]`), `near_miss` (`[{"handles": [h, other], "layers", "kind": "end_end" | "end_segment", "gap", "at", "to", "space"}]`), `crossing` (`[{"handles", "layer", "at", "space"}]`), `stats` (`entities`, `ends`, `skipped`); `network_layers(snap) -> list[dict]` (`layer`, `discipline`, `service`, `confidence`, `keyword`); `entity_pieces(rec) -> tuple[list[Piece], tuple[Pt, Pt] | None] | None`; `default_gap(points, tol) -> float`; `cap_findings(findings, limit) -> dict` (adds `counts`, `truncated`); `Piece` (frozen: `owner`, `layer`, `index`, `a`, `b`, `centre`, `radius`, `start`, `sweep`; `.box()`, `.nearest(p)`). In `critique.py`: `TOPO_FOCUSES`, `SEMANTIC_LAYERS: frozenset[str]`, `focus_layers(snap) -> list[str]`, `async build_index(backend) -> dict` (`layers`, `findings`, `error`), `issues_for(focus, index) -> list[Issue]`, `TOPO_DISPATCH`. MCP tool `drawing_topology_check(layers=None, path=None, gap=None, tol=0.01, limit=200)`. + +- [ ] **Step 1: Write the failing test** + +Create `tests/test_understand_topology.py`. Every record is built by hand, so every coordinate in an assertion is the answer: + +```python +"""The topology engine: ends, joins, near misses, crossings - by hand-computed geometry. + +Every record is built by hand, so every coordinate in an assertion is the +answer. The network layer is called PIPE here; which layers are network layers +by default is ``network_layers``' job and is tested on the synthetic plant. +""" + +from __future__ import annotations + +import pytest + +from engineering.understand.snapshot import EntityRecord, Snapshot +from engineering.understand.topology import ( + Piece, + cap_findings, + entity_pieces, + network_layers, + topology_findings, +) +from engineering.understand.vocab import classify_layer +from tests.fixtures.plant_pair import build_plant_pair + +L = "PIPE" + + +def rec(handle, type_, layer=L, points=(), **kw): + space = kw.pop("space", "Model") + return EntityRecord( + handle=handle, type=type_, layer=layer, space=space, points=tuple(points), **kw + ) + + +def line(handle, x1, y1, x2, y2, layer=L, space="Model"): + return rec(handle, "LINE", layer, ((x1, y1), (x2, y2)), space=space) + + +def snap(*records): + return Snapshot( + source="test", + insunits=4, + extmin=None, + extmax=None, + layers={r.layer: {"color": 7, "linetype": "Continuous"} for r in records}, + layouts=("Layout1",), + records=tuple(records), + ) + + +def run(*records, gap=5.0, layers=(L,)): + return topology_findings(snap(*records), layers=list(layers), gap=gap) + + +def ends(rows): + return {(row["handle"], row["end"]) for row in rows} + + +# -- pieces ---------------------------------------------------------------------------- + + +def test_a_bulge_of_one_is_the_semicircle_below_its_chord(): + # bulge +1 from (0,0) to (200,0): counter-clockwise, centre (100,0), radius 100, + # through (100,-100) + pieces, entity_ends = entity_pieces( + rec("PL", "LWPOLYLINE", L, ((0.0, 0.0), (200.0, 0.0)), bulges=(1.0, 0.0)) + ) + (arc,) = pieces + assert arc.centre == (pytest.approx(100.0), pytest.approx(0.0, abs=1e-9)) + assert arc.radius == pytest.approx(100.0) + foot = arc.nearest((100.0, -150.0)) + assert foot == (pytest.approx(100.0), pytest.approx(-100.0)) + assert entity_ends == ((0.0, 0.0), (200.0, 0.0)) + + +def test_a_closed_polyline_has_no_ends_and_a_closing_piece(): + pieces, entity_ends = entity_pieces( + rec("SQ", "LWPOLYLINE", L, ((0, 0), (10, 0), (10, 10), (0, 10)), closed=True) + ) + assert entity_ends is None and len(pieces) == 4 + assert (pieces[3].a, pieces[3].b) == ((0.0, 10.0), (0.0, 0.0)) + + +def test_a_straight_piece_nearest_point_is_clamped_to_its_ends(): + piece = Piece("A", L, 0, (0.0, 0.0), (10.0, 0.0)) + assert piece.nearest((5.0, 3.0)) == (5.0, 0.0) + assert piece.nearest((-4.0, 3.0)) == (0.0, 0.0) + + +# -- dangling and joined --------------------------------------------------------------- + + +def test_a_lone_line_dangles_at_both_ends(): + result = run(line("A", 0, 0, 1000, 0)) + assert ends(result["dangling"]) == {("A", "start"), ("A", "end")} + assert result["near_miss"] == [] and result["crossing"] == [] + + +def test_a_t_junction_joins_the_stem(): + result = run(line("A", 0, 0, 1000, 0), line("B", 500, 0, 500, 500)) + assert ends(result["dangling"]) == {("A", "start"), ("A", "end"), ("B", "end")} + + +def test_a_closed_square_of_lines_has_no_finding(): + square = [ + line("A", 0, 0, 1000, 0), + line("B", 1000, 0, 1000, 1000), + line("C", 1000, 1000, 0, 1000), + line("D", 0, 1000, 0, 0), + ] + result = run(*square) + assert (result["dangling"], result["near_miss"], result["crossing"]) == ([], [], []) + + +def test_an_end_on_an_arc_and_on_an_arc_body_is_joined(): + # ARC centre (0,0) r 100 from 0 to 90 degrees: ends (100,0) and (0,100) + arc = rec("ARC1", "ARC", L, ((0.0, 0.0),), radius=100.0, angles=(0.0, 90.0)) + on_end = line("L1", 100, 0, 500, 0) + on_body = line("L2", 70.71067811865476, 70.71067811865476, 300, 300) + result = run(arc, on_end, on_body) + assert ends(result["dangling"]) == {("ARC1", "end"), ("L1", "end"), ("L2", "end")} + + +def test_an_end_on_a_bulged_segment_is_joined(): + arc = rec("PL", "LWPOLYLINE", L, ((0.0, 0.0), (200.0, 0.0)), bulges=(1.0, 0.0)) + result = run(arc, line("L", 100, -100, 100, -500)) + assert ends(result["dangling"]) == {("PL", "start"), ("PL", "end"), ("L", "end")} + + +def test_an_end_inside_an_insert_box_is_attached(): + valve = rec("V1", "INSERT", "VALVES", ((1000, 0),), block="VALVE", bbox=(995, -5, 1005, 5)) + result = run(line("A", 0, 0, 995, 0), valve) + assert ends(result["dangling"]) == {("A", "start")} + + +def test_an_end_on_a_curve_of_another_layer_is_attached(): + tank = rec( + "TK", + "LWPOLYLINE", + "EQUIPMENT", + ((1000, -500), (2000, -500), (2000, 500), (1000, 500)), + closed=True, + ) + result = run(line("A", 0, 0, 1000, 0), tank) + assert ends(result["dangling"]) == {("A", "start")} + + +def test_model_space_and_a_layout_never_join(): + result = run(line("A", 0, 0, 100, 0), line("B", 100, 0, 200, 0, space="Layout1")) + assert ends(result["dangling"]) == {("A", "start"), ("A", "end"), ("B", "start"), ("B", "end")} + + +# -- lines drawn twice ----------------------------------------------------------------- + + +def test_a_pipe_drawn_twice_still_dangles_at_all_four_ends(): + result = run(line("A", 0, 0, 1000, 0), line("A2", 0, 0, 1000, 0)) + assert ends(result["dangling"]) == { + ("A", "start"), + ("A", "end"), + ("A2", "start"), + ("A2", "end"), + } + assert result["near_miss"] == [] and result["crossing"] == [] + + +def test_a_line_over_a_polyline_joins_only_where_the_polyline_turns(): + polyline = rec("PL", "LWPOLYLINE", L, ((0, 0), (1000, 0), (1000, 1000))) + result = run(polyline, line("OV", 0, 0, 1000, 0)) + assert ends(result["dangling"]) == {("OV", "start"), ("PL", "end"), ("PL", "start")} + + +def test_a_side_drawn_twice_inside_a_closed_loop_is_harmless(): + square = [ + line("A", 0, 0, 1000, 0), + line("A2", 0, 0, 1000, 0), + line("B", 1000, 0, 1000, 1000), + line("C", 1000, 1000, 0, 1000), + line("D", 0, 1000, 0, 0), + ] + result = run(*square) + assert (result["dangling"], result["near_miss"], result["crossing"]) == ([], [], []) + + +# -- near misses ----------------------------------------------------------------------- + + +def test_two_ends_3_apart_are_one_end_end_near_miss(): + result = run(line("A", 0, 0, 1000, 0), line("B", 1003, 0, 2000, 0)) + (row,) = result["near_miss"] + assert row["handles"] == ["A", "B"] and row["kind"] == "end_end" + assert row["gap"] == pytest.approx(3.0) + assert (row["at"], row["to"]) == ([1000.0, 0.0], [1003.0, 0.0]) + assert ends(result["dangling"]) == {("A", "start"), ("B", "end")} + + +def test_an_end_4_above_a_line_is_an_end_segment_near_miss(): + result = run(line("A", 0, 0, 1000, 0), line("B", 500, 4, 500, 500)) + (row,) = result["near_miss"] + assert row["handles"] == ["B", "A"] and row["kind"] == "end_segment" + assert row["gap"] == pytest.approx(4.0) and row["to"] == [500.0, 0.0] + + +def test_an_end_beyond_the_gap_dangles_instead(): + result = run(line("A", 0, 0, 1000, 0), line("B", 500, 6, 500, 500)) + assert result["near_miss"] == [] + assert ("B", "start") in ends(result["dangling"]) + + +def test_a_polyline_that_nearly_closes_on_itself_is_one_near_miss(): + ring = rec("RING", "LWPOLYLINE", L, ((0, 0), (1000, 0), (1000, 1000), (0, 1000), (0, 3))) + result = run(ring) + (row,) = result["near_miss"] + assert row["handles"] == ["RING", "RING"] and row["kind"] == "end_end" + assert row["gap"] == pytest.approx(3.0) + assert result["dangling"] == [] + + +# -- crossings ------------------------------------------------------------------------- + + +def test_two_lines_of_one_layer_crossing_mid_span_are_a_crossing(): + result = run(line("A", 0, 0, 1000, 0), line("B", 500, -500, 500, 500)) + assert result["crossing"] == [ + {"handles": ["A", "B"], "layer": L, "at": [500.0, 0.0], "space": "Model"} + ] + + +def test_two_layers_crossing_and_a_t_junction_are_not_crossings(): + other = line("B", 500, -500, 500, 500, layer="CABLE") + assert run(line("A", 0, 0, 1000, 0), other, layers=(L, "CABLE"))["crossing"] == [] + assert run(line("A", 0, 0, 1000, 0), line("B", 500, 0, 500, 500))["crossing"] == [] + + +def test_a_bow_tie_polyline_crosses_itself(): + bow = rec("BOW", "LWPOLYLINE", L, ((0, 0), (100, 100), (100, 0), (0, 100))) + (row,) = run(bow)["crossing"] + assert row["handles"] == ["BOW", "BOW"] and row["at"] == [50.0, 50.0] + + +# -- defaults and refusals ------------------------------------------------------------- + + +def test_the_default_gap_is_0_2_percent_of_the_network_diagonal(): + # box 0..3000 x 0..4000: diagonal 5000, gap 10 + result = topology_findings(snap(line("A", 0, 0, 3000, 0), line("B", 0, 0, 0, 4000)), layers=[L]) + assert result["gap"] == pytest.approx(10.0) + assert result["stats"] == {"entities": 2, "ends": 4, "skipped": {}} + + +def test_an_arc_without_a_centre_is_skipped_and_counted(): + result = run(line("A", 0, 0, 10, 0), rec("ARC9", "ARC", L, (), radius=5.0, angles=(0, 90))) + assert result["stats"]["skipped"] == {"ARC": 1} + + +@pytest.mark.parametrize( + ("kwargs", "message"), + [ + ({"layers": []}, r"^layers: name at least one layer to check"), + ({"layers": [L], "tol": 0}, r"^tol must be a positive number; got 0"), + ({"layers": [L], "gap": -1}, r"^gap must be a positive number; got -1"), + ({"layers": [L], "gap": 0.005}, r"^gap \(0\.005\) must exceed tol \(0\.01\)"), + ], +) +def test_refusals_come_before_any_work(kwargs, message): + with pytest.raises(ValueError, match=message): + topology_findings(snap(line("A", 0, 0, 1, 0)), **kwargs) + + +def test_cap_findings_counts_everything_and_cuts_the_lists(): + lines = [line(f"L{i}", 0, 1000 * i, 100, 1000 * i) for i in range(3)] + capped = cap_findings(run(*lines), limit=2) + assert capped["counts"] == {"dangling": 6, "near_miss": 0, "crossing": 0} + assert capped["truncated"] == {"dangling": 4, "near_miss": 0, "crossing": 0} + assert len(capped["dangling"]) == 2 + + +# -- the default layers on the synthetic plant ----------------------------------------- + + +def test_the_plant_piping_layers_are_network_layers(tmp_path): + from engineering.understand.snapshot import read_snapshot + + truth = build_plant_pair(tmp_path) + pid = read_snapshot(truth["pid"]) + found = {row["layer"] for row in network_layers(pid)} + for layer, service in truth["layer_services"].items(): + if layer in pid.layers and service in ("product", "cip_supply", "cip_return"): + assert layer in found, (layer, classify_layer(layer)) + assert all(row["confidence"] >= 0.9 for row in network_layers(pid)) +``` + +- [ ] **Step 2: Run it to verify it fails** + +Run: `uv run --frozen pytest tests/test_understand_topology.py -q` +Expected: FAIL — `ERROR collecting tests/test_understand_topology.py` with `ModuleNotFoundError: No module named 'engineering.understand.topology'`. + +- [ ] **Step 3: Write `engineering/understand/topology.py`** + +```python +"""The topology of a line network: dangling ends, near misses, interior crossings. + +Spec §11. Pure: a ``Snapshot`` in, three lists out; nothing touches a drawing. + +What is analysed: the entities on ``layers`` that draw a network - LINE, +LWPOLYLINE, POLYLINE (a bulged segment is a true arc) and ARC. Their *ends* +are the ends of the entity: a polyline's first and last vertex, never an +interior vertex, and a closed polyline has none. Everything else in the same +space is context: every LINE / LWPOLYLINE / POLYLINE / ARC / CIRCLE on another +layer (a tank outline, a wall), every CIRCLE on the analysed layers, and the +box of every INSERT on any layer (a valve, a pump, a reducer). + +Each end is exactly one of, tested in this order: + +* **joined** (no finding) - within ``tol`` of another network end, or of a + network segment other than its own segment and that segment's neighbours. + A polyline of three or more segments may join its own other end (a ring + drawn as an open polyline). A segment lying along the end's own segment and + running into it - a line drawn twice, a line over a polyline - is not a + join, so a pipe drawn twice dangles exactly like a pipe drawn once; +* **attached** (no finding) - within ``tol`` of a context curve, or inside or + within ``tol`` of an INSERT box: a line ending on equipment or on a fitting + is connected, whatever layer the equipment is on; +* **near miss** - the nearest network end or segment (the candidates of + *joined*) is within ``gap``: ``end_end`` when an end is nearest (an end + wins a tie), else ``end_segment``. One row per pair; +* **dangling** - none of the above. + +An **interior crossing** is two straight network segments on the same layer +that cross at a point more than ``tol`` from every end of both segments: a +junction nobody drew, or a crossing drawn without a break. A crossing through +a polyline vertex reads as a touch and is not reported, arcs are not tested +for crossings, and two different layers crossing is how every plan is drawn (a +cable over a pipe), so it is not a finding. Spaces are analysed apart: model +space and each paper layout never join each other. + +``gap`` defaults to 0.2 % of the diagonal of the analysed network's box, never +less than ten ``tol``. The 0.2 %, the ten-``tol`` floor and the same-layer +rule for crossings are this module's declared defaults, not standards values. + +``EntityRecord.points[0]`` is the centre of an ARC or a CIRCLE (the reader in +``engineering.understand.snapshot``); an ARC or CIRCLE without one is skipped +and counted in ``stats["skipped"]``. Spatial lookups go through a uniform +grid, so a large drawing is read in one pass rather than in pairs of +entities. +""" + +from __future__ import annotations + +import math +from collections import Counter, defaultdict +from collections.abc import Iterable +from dataclasses import dataclass + +from engineering.understand.snapshot import EntityRecord, Pt, Snapshot +from engineering.understand.vocab import classify_layer + +__all__ = [ + "DEFAULT_TOL", + "GAP_FLOOR_TOLS", + "GAP_FRACTION", + "NETWORK_CONFIDENCE", + "NETWORK_DISCIPLINES", + "Piece", + "cap_findings", + "default_gap", + "entity_pieces", + "network_layers", + "topology_findings", +] + +DEFAULT_TOL = 0.01 +#: The default gap, as a share of the analysed network's diagonal. +GAP_FRACTION = 0.002 +#: ... and never less than this many ``tol``. +GAP_FLOOR_TOLS = 10.0 +#: What makes a layer a network layer by default (spec §11). +NETWORK_DISCIPLINES = ("piping", "electrical") +NETWORK_CONFIDENCE = 0.9 +FINDING_KINDS = ("dangling", "near_miss", "crossing") + +_NETWORK_TYPES = frozenset({"LINE", "LWPOLYLINE", "POLYLINE", "ARC"}) +_CONTEXT_TYPES = _NETWORK_TYPES | {"CIRCLE"} +_TWO_PI = 2.0 * math.pi +#: A piece whose box covers more grid cells than this is kept on a side list. +_MAX_CELLS = 4096 + + +@dataclass(frozen=True) +class Piece: + """One segment of an entity: straight, or an arc when ``centre`` is set. + + ``a`` and ``b`` are its ends as drawn. An arc runs counter-clockwise from + the angle ``start`` (radians) through ``sweep`` (radians, > 0). + """ + + owner: str + layer: str + index: int + a: Pt + b: Pt + centre: Pt | None = None + radius: float = 0.0 + start: float = 0.0 + sweep: float = 0.0 + + def box(self) -> tuple[float, float, float, float]: + if self.centre is not None: + cx, cy = self.centre + r = self.radius + return (cx - r, cy - r, cx + r, cy + r) + return ( + min(self.a[0], self.b[0]), + min(self.a[1], self.b[1]), + max(self.a[0], self.b[0]), + max(self.a[1], self.b[1]), + ) + + def nearest(self, p: Pt) -> Pt: + """The point of this piece nearest to ``p``.""" + if self.centre is None: + ax, ay = self.a + dx, dy = self.b[0] - ax, self.b[1] - ay + length2 = dx * dx + dy * dy + if length2 == 0.0: + return self.a + t = ((p[0] - ax) * dx + (p[1] - ay) * dy) / length2 + t = min(1.0, max(0.0, t)) + return (ax + t * dx, ay + t * dy) + cx, cy = self.centre + rel = (math.atan2(p[1] - cy, p[0] - cx) - self.start) % _TWO_PI + distance = math.hypot(p[0] - cx, p[1] - cy) + if rel <= self.sweep + 1e-12 and distance > 0.0: + return ( + cx + self.radius * (p[0] - cx) / distance, + cy + self.radius * (p[1] - cy) / distance, + ) + return self.a if math.dist(p, self.a) <= math.dist(p, self.b) else self.b + + +def _arc_from_bulge(owner: str, layer: str, index: int, a: Pt, b: Pt, bulge: float) -> Piece: + """A polyline segment with a bulge: the arc it stands for. + + The bulge is tan(sweep / 4), positive counter-clockwise (the DXF rule). + """ + dx, dy = b[0] - a[0], b[1] - a[1] + chord = math.hypot(dx, dy) + theta = 4.0 * math.atan(bulge) + if chord == 0.0 or theta == 0.0: + return Piece(owner, layer, index, a, b) + offset = chord / (2.0 * math.tan(theta / 2.0)) + nx, ny = -dy / chord, dx / chord + centre = ((a[0] + b[0]) / 2.0 + nx * offset, (a[1] + b[1]) / 2.0 + ny * offset) + radius = abs(chord / (2.0 * math.sin(theta / 2.0))) + first = a if theta > 0 else b + start = math.atan2(first[1] - centre[1], first[0] - centre[0]) + return Piece(owner, layer, index, a, b, centre, radius, start, abs(theta)) + + +def _arc_piece(rec: EntityRecord, *, full: bool) -> Piece | None: + if not rec.points or rec.radius is None: + return None + centre = (float(rec.points[0][0]), float(rec.points[0][1])) + r = float(rec.radius) + if full or rec.angles is None: + a = b = (centre[0] + r, centre[1]) + return Piece(rec.handle, rec.layer, 0, a, b, centre, r, 0.0, _TWO_PI) + s = math.radians(float(rec.angles[0])) + e = math.radians(float(rec.angles[1])) + sweep = (e - s) % _TWO_PI or _TWO_PI + a = (centre[0] + r * math.cos(s), centre[1] + r * math.sin(s)) + b = (centre[0] + r * math.cos(e), centre[1] + r * math.sin(e)) + return Piece(rec.handle, rec.layer, 0, a, b, centre, r, s, sweep) + + +def entity_pieces(rec: EntityRecord) -> tuple[list[Piece], tuple[Pt, Pt] | None] | None: + """The pieces of one record and its two ends (None when it has none). + + None for a record that draws no curve or lacks what it needs. + """ + kind = rec.type + if kind == "LINE": + if len(rec.points) < 2: + return None + a, b = tuple(rec.points[0]), tuple(rec.points[1]) + return [Piece(rec.handle, rec.layer, 0, a, b)], (a, b) + if kind in ("LWPOLYLINE", "POLYLINE"): + points = [(float(p[0]), float(p[1])) for p in rec.points] + if len(points) < 2: + return None + bulges = list(rec.bulges) + [0.0] * (len(points) - len(rec.bulges)) + count = len(points) if rec.closed else len(points) - 1 + pieces = [ + _arc_from_bulge( + rec.handle, + rec.layer, + i, + points[i], + points[(i + 1) % len(points)], + float(bulges[i]), + ) + for i in range(count) + ] + return pieces, None if rec.closed else (points[0], points[-1]) + if kind == "ARC": + piece = _arc_piece(rec, full=False) + return None if piece is None else ([piece], (piece.a, piece.b)) + if kind == "CIRCLE": + piece = _arc_piece(rec, full=True) + return None if piece is None else ([piece], None) + return None + + +def network_layers(snap: Snapshot) -> list[dict]: + """The layers ``vocab.classify_layer`` calls piping or electrical with + confidence >= 0.9, with the classification as evidence.""" + names = set(snap.layers) | {rec.layer for rec in snap.records} + rows = [] + for name in sorted(names): + meaning = classify_layer(name) + confidence = float(meaning.get("confidence") or 0.0) + if meaning.get("discipline") in NETWORK_DISCIPLINES and confidence >= NETWORK_CONFIDENCE: + rows.append( + { + "layer": name, + "discipline": meaning.get("discipline"), + "service": meaning.get("service"), + "confidence": confidence, + "keyword": meaning.get("keyword"), + } + ) + return rows + + +def default_gap(points: Iterable[Pt], tol: float) -> float: + """0.2 % of the diagonal of the points' box, never less than ten ``tol``.""" + xs: list[float] = [] + ys: list[float] = [] + for x, y in points: + xs.append(x) + ys.append(y) + diagonal = math.hypot(max(xs) - min(xs), max(ys) - min(ys)) if xs else 0.0 + return max(GAP_FRACTION * diagonal, GAP_FLOOR_TOLS * tol) + + +def _positive(name: str, value) -> float: + try: + number = float(value) + except (TypeError, ValueError): + raise ValueError(f"{name} must be a positive number; got {value!r}") from None + if not math.isfinite(number) or number <= 0: + raise ValueError(f"{name} must be a positive number; got {value!r}") + return number + + +def _layer_list(layers) -> set[str]: + if isinstance(layers, str): + layers = [layers] + names = {str(name) for name in (layers or []) if str(name)} + if not names: + raise ValueError("layers: name at least one layer to check") + return names + + +class _Grid: + """A uniform grid of item indices by box; oversized boxes on a side list.""" + + def __init__(self, cell: float) -> None: + self.cell = cell + self.cells: dict[tuple[int, int], list[int]] = defaultdict(list) + self.oversize: list[int] = [] + + def _span(self, box) -> tuple[int, int, int, int]: + c = self.cell + return ( + math.floor(box[0] / c), + math.floor(box[1] / c), + math.floor(box[2] / c), + math.floor(box[3] / c), + ) + + def add(self, item: int, box) -> None: + x0, y0, x1, y1 = self._span(box) + if (x1 - x0 + 1) * (y1 - y0 + 1) > _MAX_CELLS: + self.oversize.append(item) + return + for ix in range(x0, x1 + 1): + for iy in range(y0, y1 + 1): + self.cells[(ix, iy)].append(item) + + def near(self, box) -> list[int]: + x0, y0, x1, y1 = self._span(box) + found: set[int] = set(self.oversize) + if (x1 - x0 + 1) * (y1 - y0 + 1) > _MAX_CELLS: + for items in self.cells.values(): + found.update(items) + return sorted(found) + for ix in range(x0, x1 + 1): + for iy in range(y0, y1 + 1): + found.update(self.cells.get((ix, iy), ())) + return sorted(found) + + +def _around(p: Pt, r: float) -> tuple[float, float, float, float]: + return (p[0] - r, p[1] - r, p[0] + r, p[1] + r) + + +def _box_distance(p: Pt, box) -> float: + dx = max(box[0] - p[0], 0.0, p[0] - box[2]) + dy = max(box[1] - p[1], 0.0, p[1] - box[3]) + return math.hypot(dx, dy) + + +def _cross(ax: float, ay: float, bx: float, by: float) -> float: + return ax * by - ay * bx + + +def _crossing(p: Piece, q: Piece, tol: float) -> Pt | None: + """Where two straight pieces cross strictly inside both, away from their ends.""" + rx, ry = p.b[0] - p.a[0], p.b[1] - p.a[1] + sx, sy = q.b[0] - q.a[0], q.b[1] - q.a[1] + denominator = _cross(rx, ry, sx, sy) + if denominator == 0.0: + return None + qx, qy = q.a[0] - p.a[0], q.a[1] - p.a[1] + t = _cross(qx, qy, sx, sy) / denominator + u = _cross(qx, qy, rx, ry) / denominator + if not (0.0 < t < 1.0 and 0.0 < u < 1.0): + return None + point = (p.a[0] + t * rx, p.a[1] + t * ry) + if min(math.dist(point, e) for e in (p.a, p.b, q.a, q.b)) <= tol: + return None + return point + + +def _overlaps(own: Piece, point: Pt, other: Piece, tol: float) -> bool: + """True when ``other`` lies along ``own`` and runs from ``point`` into it: + a line drawn twice, or a line over a polyline. Such a piece is not a join - + otherwise a pipe drawn twice would join itself and never dangle. A + collinear piece that continues the other way (a butt joint) is a join.""" + if own.centre is not None or other.centre is not None: + return False + far = own.b if math.dist(point, own.a) <= math.dist(point, own.b) else own.a + length = math.dist(point, far) + if length == 0.0: + return False + ux, uy = (far[0] - point[0]) / length, (far[1] - point[1]) / length + along = [] + for p in (other.a, other.b): + dx, dy = p[0] - point[0], p[1] - point[1] + if abs(dy * ux - dx * uy) > tol: + return False + along.append(dx * ux + dy * uy) + return max(along) > tol + + +def _analyse_space( + space: str, + records: list[EntityRecord], + wanted: set[str], + gap: float, + tol: float, + out: dict, + skipped: Counter, +) -> tuple[int, int]: + net: list[Piece] = [] + counts: dict[str, int] = {} + closed: set[str] = set() + # handle, layer, "start" | "end", the point, the index in `net` of the piece it ends + ends: list[tuple[str, str, str, Pt, int]] = [] + context: list[Piece] = [] + boxes: list[tuple[float, float, float, float]] = [] + for rec in records: + if rec.type == "INSERT": + if rec.bbox is not None: + boxes.append(tuple(float(v) for v in rec.bbox)) + continue + if rec.type not in _CONTEXT_TYPES: + continue + built = entity_pieces(rec) + if built is None: + skipped[rec.type] += 1 + continue + pieces, entity_ends = built + if rec.layer in wanted and rec.type in _NETWORK_TYPES: + first = len(net) + net.extend(pieces) + counts[rec.handle] = len(pieces) + if entity_ends is None: + closed.add(rec.handle) + else: + ends.append((rec.handle, rec.layer, "start", entity_ends[0], first)) + ends.append((rec.handle, rec.layer, "end", entity_ends[1], len(net) - 1)) + else: + context.extend(pieces) + if not net: + return 0, 0 + + cell = max(8.0 * gap, 100.0 * tol) + net_grid = _Grid(cell) + for k, piece in enumerate(net): + net_grid.add(k, piece.box()) + end_grid = _Grid(cell) + for k, (_h, _l, _w, point, _piece) in enumerate(ends): + end_grid.add(k, _around(point, 0.0)) + context_grid = _Grid(cell) + for k, piece in enumerate(context): + context_grid.add(k, piece.box()) + box_grid = _Grid(cell) + for k, box in enumerate(boxes): + box_grid.add(k, box) + + seen_pairs: set = set() + for handle, layer, which, point, own_k in sorted(ends, key=lambda e: (e[0], e[2])): + own = net[own_k] + pieces_of_owner = counts[handle] + own_index = 0 if which == "start" else pieces_of_owner - 1 + reach = _around(point, gap) + best_end: tuple[float, int] | None = None + for k in end_grid.near(reach): + other_handle, _ol, other_which, other_point, other_k = ends[k] + if other_handle == handle and (other_which == which or pieces_of_owner < 3): + continue + if _overlaps(own, point, net[other_k], tol): + continue + distance = math.dist(point, other_point) + if best_end is None or distance < best_end[0]: + best_end = (distance, k) + best_piece: tuple[float, int, Pt] | None = None + for k in net_grid.near(reach): + piece = net[k] + if piece.owner == handle and abs(piece.index - own_index) <= 1: + continue + if _overlaps(own, point, piece, tol): + continue + foot = piece.nearest(point) + distance = math.dist(point, foot) + if best_piece is None or distance < best_piece[0]: + best_piece = (distance, k, foot) + nearest = min( + best_end[0] if best_end is not None else math.inf, + best_piece[0] if best_piece is not None else math.inf, + ) + if nearest <= tol: + continue # joined + touch = _around(point, tol) + if any( + math.dist(point, context[k].nearest(point)) <= tol for k in context_grid.near(touch) + ) or any(_box_distance(point, boxes[k]) <= tol for k in box_grid.near(touch)): + continue # attached + if nearest <= gap: + if best_end is not None and (best_piece is None or best_end[0] <= best_piece[0]): + other_handle, other_layer, other_which, other_point, _k = ends[best_end[1]] + key = frozenset({(handle, which), (other_handle, other_which)}) + kind, distance, to = "end_end", best_end[0], other_point + else: + piece = net[best_piece[1]] + other_handle, other_layer = piece.owner, piece.layer + key = (handle, which, other_handle) + kind, distance, to = "end_segment", best_piece[0], best_piece[2] + if key in seen_pairs: + continue + seen_pairs.add(key) + out["near_miss"].append( + { + "handles": [handle, other_handle], + "layers": [layer, other_layer], + "kind": kind, + "gap": distance, + "at": [point[0], point[1]], + "to": [to[0], to[1]], + "space": space, + } + ) + continue + out["dangling"].append( + { + "handle": handle, + "layer": layer, + "end": which, + "at": [point[0], point[1]], + "space": space, + } + ) + + for k, piece in enumerate(net): + if piece.centre is not None: + continue + for m in net_grid.near(piece.box()): + if m <= k: + continue + other = net[m] + if other.centre is not None or other.layer != piece.layer: + continue + if other.owner == piece.owner: + apart = abs(other.index - piece.index) + wraps = piece.owner in closed and apart == counts[piece.owner] - 1 + if apart <= 1 or wraps: + continue + point = _crossing(piece, other, tol) + if point is None: + continue + out["crossing"].append( + { + "handles": [piece.owner, other.owner], + "layer": piece.layer, + "at": [point[0], point[1]], + "space": space, + } + ) + return len(counts), len(ends) + + +def topology_findings( + snap: Snapshot, *, layers, gap: float | None = None, tol: float = DEFAULT_TOL +) -> dict: + """Dangling ends, near misses and interior crossings on ``layers`` (spec §11). + + Refused before any work: an empty ``layers``, a non-positive ``tol`` or + ``gap``, and a ``gap`` that does not exceed ``tol``. + """ + tol = _positive("tol", tol) + wanted = _layer_list(layers) + if gap is not None: + gap = _positive("gap", gap) + if gap <= tol: + raise ValueError( + f"gap ({gap:g}) must exceed tol ({tol:g}): an end within tol is joined, and a " + "near miss is a distance between the two" + ) + by_space: dict[str, list[EntityRecord]] = defaultdict(list) + for rec in snap.records: + by_space[rec.space].append(rec) + if gap is None: + points: list[Pt] = [] + for rec in snap.records: + if rec.layer in wanted and rec.type in _NETWORK_TYPES: + built = entity_pieces(rec) + if built is not None: + for piece in built[0]: + points.extend((piece.a, piece.b)) + gap = default_gap(points, tol) + out: dict = {kind: [] for kind in FINDING_KINDS} + skipped: Counter = Counter() + entities = end_count = 0 + for space in sorted(by_space): + n_entities, n_ends = _analyse_space(space, by_space[space], wanted, gap, tol, out, skipped) + entities += n_entities + end_count += n_ends + out["dangling"].sort(key=lambda r: (r["space"], r["handle"], r["end"])) + out["near_miss"].sort(key=lambda r: (r["space"], r["handles"], r["at"])) + out["crossing"].sort(key=lambda r: (r["space"], r["handles"], r["at"])) + return { + "layers": sorted(wanted), + "gap": gap, + "tol": tol, + **out, + "stats": { + "entities": entities, + "ends": end_count, + "skipped": dict(sorted(skipped.items())), + }, + } + + +def cap_findings(findings: dict, limit: int) -> dict: + """``findings`` with each list cut to ``limit`` rows, plus ``counts`` and + ``truncated`` per kind.""" + if int(limit) < 1: + raise ValueError(f"limit must be at least 1; got {limit!r}") + capped = dict(findings) + capped["counts"] = {kind: len(findings.get(kind) or []) for kind in FINDING_KINDS} + capped["truncated"] = { + kind: max(0, capped["counts"][kind] - int(limit)) for kind in FINDING_KINDS + } + for kind in FINDING_KINDS: + capped[kind] = list(findings.get(kind) or [])[: int(limit)] + return capped +``` + +- [ ] **Step 4: Run the engine tests to verify they pass** + +Run: `uv run --frozen pytest tests/test_understand_topology.py -q` +Expected: `29 passed`. `test_the_plant_piping_layers_are_network_layers` is the one test that reads Task 1's vocabulary through Task 2's layer names; if it fails, the assertion message names the layer and its classification — fix whichever side is wrong against spec §5 (a direct keyword is confidence 0.9), record it, and never drop the layer from the assertion. + +- [ ] **Step 5: Write the failing critique tests** + +Create `tests/test_understand_critique.py`. The network layer is the plant fixture's product-piping layer, read from the truth dict, so the focuses are exercised on a layer the vocabulary classifies the way a foreign drawing's would be: + +```python +"""The three topo_* focuses: a positive and a negative fixture each, and silence +on every drawing this server makes. + +The network layer is the synthetic plant's product-piping layer, read from the +fixture's truth dict (``tests/fixtures/plant_pair.py``), so the test draws on a +layer the vocabulary classifies the way a foreign drawing's layer would be +classified. +""" + +from __future__ import annotations + +import copy + +import pytest + +from engineering.arch.layers import ARCH_ROLE_LAYER +from engineering.critique import run_critique +from engineering.pid.lines import PID_LINE_LAYERS +from engineering.plan_spec import ALL_CRITIQUE_FOCUSES +from engineering.understand.critique import ( + SEMANTIC_LAYERS, + TOPO_DISPATCH, + TOPO_FOCUSES, + build_index, + focus_layers, + issues_for, +) +from engineering.understand.snapshot import EntityRecord, Snapshot +from engineering.understand.vocab import classify_layer +from tests.fixtures.plant_pair import build_plant_pair + +pytestmark = pytest.mark.asyncio + + +@pytest.fixture(scope="module") +def pipe_layer(tmp_path_factory) -> str: + truth = build_plant_pair(tmp_path_factory.mktemp("plant")) + layer = next( + name for name, service in sorted(truth["layer_services"].items()) if service == "product" + ) + meaning = classify_layer(layer) + assert meaning["discipline"] == "piping" and meaning["confidence"] >= 0.9, meaning + return layer + + +async def _lines(backend, layer, *segments): + handles = [] + for x1, y1, x2, y2 in segments: + entity = await backend.entity_create_line(x1, y1, x2, y2, layer=layer) + handles.append(entity.handle) + return handles + + +async def _issues(backend, focus): + return issues_for(focus, await build_index(backend)) + + +# -- the closed enum and the exclusions ------------------------------------------------ + + +async def test_the_three_focuses_are_the_spec_ones_and_are_in_the_closed_enum(): + assert TOPO_FOCUSES == ("topo_dangling_endpoint", "topo_near_miss", "topo_interior_crossing") + for focus in TOPO_FOCUSES: + assert focus in ALL_CRITIQUE_FOCUSES + assert TOPO_DISPATCH[focus].needs_shared is True + + +async def test_pid_line_layers_and_arch_layers_are_excluded(): + assert set(PID_LINE_LAYERS) <= SEMANTIC_LAYERS + assert ARCH_ROLE_LAYER["wall"] in SEMANTIC_LAYERS + assert "0" not in SEMANTIC_LAYERS + snap = Snapshot( + source="test", + insunits=4, + extmin=None, + extmax=None, + layers={"PROCESS-PIPING-MAIN": {}, "A-WALL-E-N": {}}, + layouts=(), + records=( + EntityRecord("1", "LINE", "PROCESS-PIPING-MAIN", "Model", ((0, 0), (10, 0))), + EntityRecord("2", "LINE", "A-WALL-E-N", "Model", ((0, 5), (10, 5))), + ), + ) + assert focus_layers(snap) == [] + + +# -- topo_dangling_endpoint ------------------------------------------------------------ + + +async def test_dangling_fires_on_a_lone_pipe(backend, pipe_layer): + (handle,) = await _lines(backend, pipe_layer, (0, 0, 1000, 0)) + issues = await _issues(backend, "topo_dangling_endpoint") + assert [(i.severity, i.handles, i.detail["end"]) for i in issues] == [ + ("warning", [handle], "end"), + ("warning", [handle], "start"), + ] + + +async def test_dangling_is_quiet_on_a_closed_loop(backend, pipe_layer): + await _lines( + backend, + pipe_layer, + (0, 0, 1000, 0), + (1000, 0, 1000, 1000), + (1000, 1000, 0, 1000), + (0, 1000, 0, 0), + ) + assert await _issues(backend, "topo_dangling_endpoint") == [] + + +# -- topo_near_miss -------------------------------------------------------------------- + + +async def test_near_miss_fires_on_a_3_unit_gap(backend, pipe_layer): + # box 0..2003 wide: default gap 0.002 x 2003 = 4.006 > 3 + first, second = await _lines(backend, pipe_layer, (0, 0, 1000, 0), (1003, 0, 2003, 0)) + (issue,) = await _issues(backend, "topo_near_miss") + assert issue.severity == "error" + assert issue.handles == sorted([first, second]) + assert issue.detail["kind"] == "end_end" + assert issue.detail["gap"] == pytest.approx(3.0) + assert "stops 3 short of the end of" in issue.message + + +async def test_near_miss_is_quiet_when_the_lines_join(backend, pipe_layer): + await _lines(backend, pipe_layer, (0, 0, 1000, 0), (1000, 0, 2000, 0)) + assert await _issues(backend, "topo_near_miss") == [] + + +# -- topo_interior_crossing ------------------------------------------------------------ + + +async def test_crossing_fires_on_two_pipes_crossing_mid_span(backend, pipe_layer): + await _lines(backend, pipe_layer, (0, 0, 1000, 0), (500, -500, 500, 500)) + (issue,) = await _issues(backend, "topo_interior_crossing") + assert issue.severity == "info" + assert issue.detail["at"] == [500.0, 0.0] + + +async def test_crossing_is_quiet_on_a_t_junction(backend, pipe_layer): + await _lines(backend, pipe_layer, (0, 0, 1000, 0), (500, 0, 500, 500)) + assert await _issues(backend, "topo_interior_crossing") == [] + + +# -- silence on this server's own drawings --------------------------------------------- + + +async def test_a_pid_from_spec_sheet_gets_no_topo_issue(backend): + from engineering.pid.spec import EXAMPLE_SPEC, run_spec + + await run_spec(backend, copy.deepcopy(EXAMPLE_SPEC)) + assert await run_critique(backend, list(TOPO_FOCUSES)) == [] + everything = await run_critique(backend, None) + assert not [i for i in everything if i.focus.startswith("topo_")] + + +async def test_an_arch_plan_from_spec_gets_no_topo_issue(backend): + from engineering.arch.spec import EXAMPLE_SPEC, draw_plan_from_spec + + await draw_plan_from_spec(backend, copy.deepcopy(EXAMPLE_SPEC)) + assert await run_critique(backend, list(TOPO_FOCUSES)) == [] + + +async def test_a_drawing_without_a_network_layer_is_never_snapshotted(backend, monkeypatch): + import engineering.understand.snapshot as snapshot + + async def refuse(*_args, **_kwargs): + raise AssertionError("the snapshot was taken") + + monkeypatch.setattr(snapshot, "take_snapshot", refuse) + await backend.entity_create_line(0, 0, 100, 0, layer="GEOMETRY") + assert await build_index(backend) == {"layers": [], "findings": None, "error": None} + + +async def test_a_snapshot_that_fails_is_reported_not_passed(backend, pipe_layer, monkeypatch): + import engineering.understand.snapshot as snapshot + + async def broken(*_args, **_kwargs): + raise RuntimeError("export failed") + + monkeypatch.setattr(snapshot, "take_snapshot", broken) + await _lines(backend, pipe_layer, (0, 0, 1000, 0)) + (issue,) = await _issues(backend, "topo_near_miss") + assert issue.severity == "info" and "export failed" in issue.message + + +# -- the dispatch ---------------------------------------------------------------------- + + +async def test_the_three_focuses_share_one_index_per_run(backend, pipe_layer): + await _lines(backend, pipe_layer, (0, 0, 1000, 0)) + shared: dict = {} + for focus in TOPO_FOCUSES: + await TOPO_DISPATCH[focus](backend, shared) + assert list(shared) == ["topo_index"] + + +async def test_run_critique_reaches_the_topo_focuses(backend, pipe_layer): + await _lines(backend, pipe_layer, (0, 0, 1000, 0), (1003, 0, 2003, 0)) + (issue,) = await run_critique(backend, ["topo_near_miss"]) + assert issue.focus == "topo_near_miss" and issue.severity == "error" +``` + +- [ ] **Step 6: Run them to verify they fail** + +Run: `uv run --frozen pytest tests/test_understand_critique.py -q` +Expected: FAIL — `ERROR collecting tests/test_understand_critique.py` with `ModuleNotFoundError: No module named 'engineering.understand.critique'`. + +- [ ] **Step 7: Write `engineering/understand/critique.py`** + +```python +"""The three topology critique focuses (spec §11). + +``topo_dangling_endpoint`` (warning), ``topo_near_miss`` (error) and +``topo_interior_crossing`` (info) run ``topology.topology_findings`` - the +engine ``drawing_topology_check`` runs - once per critique run, shared through +``run_critique``'s context the way the P&ID, mechanical and architectural +focuses share theirs. + +Which layers: the network layers ``vocab.classify_layer`` calls piping or +electrical with confidence >= 0.9, **minus every layer a semantic focus +already reads**: the P&ID layer set and line layers (``pid_*``) and the +architectural layer set (``arch_*``). One defect is never scored twice, and a +drawing made entirely by this server - a ``pid_from_spec`` sheet, an +``arch_plan_from_spec`` plan, a mechanical part - gets none of these issues. + +Cost: the layer table is read first; a drawing with no candidate layer (every +drawing this server makes) is never snapshotted, so ``focus=None`` stays cheap. +A snapshot that cannot be taken is reported as one ``info`` issue per focus, +never as a clean result. + +Severities are this module's decision: a near miss is an ``error`` (a network +that looks connected and is not - a takeoff counts two runs), a dangling end a +``warning`` (it may be a real open end), a crossing ``info`` (often drawn on +purpose). +""" + +from __future__ import annotations + +from typing import TYPE_CHECKING + +from backends.quarantine import DocumentQuarantineError +from engineering.arch.layers import ARCH_LAYER_ROLES, ARCH_LAYERS +from engineering.layers import PID_LAYERS +from engineering.pid.lines import PID_LINE_LAYERS +from engineering.plan_spec import Issue +from engineering.understand.topology import ( + NETWORK_CONFIDENCE, + NETWORK_DISCIPLINES, + network_layers, + topology_findings, +) +from engineering.understand.vocab import classify_layer + +if TYPE_CHECKING: + from backends.base import AutoCADBackend + +__all__ = [ + "SEMANTIC_LAYERS", + "TOPO_DISPATCH", + "TOPO_FOCUSES", + "build_index", + "focus_layers", + "issues_for", +] + +TOPO_FOCUSES = ("topo_dangling_endpoint", "topo_near_miss", "topo_interior_crossing") +_KEY = "topo_index" + +#: Layers the pid_* and arch_* focuses already read. Layer "0" is nobody's. +SEMANTIC_LAYERS: frozenset[str] = frozenset( + {name for name, *_rest in PID_LAYERS} + | set(PID_LINE_LAYERS) + | {name for name, *_rest in ARCH_LAYERS} + | set(ARCH_LAYER_ROLES.values()) +) - {"0"} + + +def _is_candidate(name: str) -> bool: + if name in SEMANTIC_LAYERS: + return False + meaning = classify_layer(name) + return ( + meaning.get("discipline") in NETWORK_DISCIPLINES + and float(meaning.get("confidence") or 0.0) >= NETWORK_CONFIDENCE + ) + + +def focus_layers(snap) -> list[str]: + """The network layers of ``snap`` no semantic focus covers.""" + return [row["layer"] for row in network_layers(snap) if row["layer"] not in SEMANTIC_LAYERS] + + +async def build_index(backend: AutoCADBackend) -> dict: + """One read of the drawing for the three focuses: + ``{"layers": [...], "findings": dict | None, "error": str | None}``.""" + try: + names = [layer.name for layer in await backend.layer_list()] + except DocumentQuarantineError: + raise + except Exception: + names = [] + if not any(_is_candidate(name) for name in names): + return {"layers": [], "findings": None, "error": None} + from engineering.understand.snapshot import take_snapshot + + try: + snap = await take_snapshot(backend, None) + except DocumentQuarantineError: + raise + except Exception as exc: + return { + "layers": sorted(n for n in names if _is_candidate(n)), + "findings": None, + "error": str(exc) or type(exc).__name__, + } + layers = focus_layers(snap) + if not layers: + return {"layers": [], "findings": None, "error": None} + return {"layers": layers, "findings": topology_findings(snap, layers=layers), "error": None} + + +def _dangling(findings: dict) -> list[Issue]: + return [ + Issue( + "warning", + "topo_dangling_endpoint", + f"{row['handle']} on layer {row['layer']} ends at ({row['at'][0]:.3f}, " + f"{row['at'][1]:.3f}) touching nothing - no line, fitting or equipment is at that " + "end. Connect it, or confirm it is a real open end.", + [row["handle"]], + {"layer": row["layer"], "end": row["end"], "at": row["at"], "space": row["space"]}, + ) + for row in findings["dangling"] + ] + + +def _near_miss(findings: dict) -> list[Issue]: + out = [] + for row in findings["near_miss"]: + first, second = row["handles"] + what = "the end of" if row["kind"] == "end_end" else "the body of" + out.append( + Issue( + "error", + "topo_near_miss", + f"{first} on layer {row['layers'][0]} stops {row['gap']:.4g} short of {what} " + f"{second} without joining it: the network reads as two pieces there. Extend or " + "trim it onto the other line.", + sorted({first, second}), + { + "kind": row["kind"], + "gap": row["gap"], + "at": row["at"], + "to": row["to"], + "layers": row["layers"], + "space": row["space"], + }, + ) + ) + return out + + +def _crossing(findings: dict) -> list[Issue]: + return [ + Issue( + "info", + "topo_interior_crossing", + f"{row['handles'][0]} and {row['handles'][1]} on layer {row['layer']} cross at " + f"({row['at'][0]:.3f}, {row['at'][1]:.3f}) away from their ends with no junction. " + "If they connect, split both at the crossing; if not, draw the crossing with a break.", + sorted(set(row["handles"])), + {"layer": row["layer"], "at": row["at"], "space": row["space"]}, + ) + for row in findings["crossing"] + ] + + +_CHECKS = { + "topo_dangling_endpoint": _dangling, + "topo_near_miss": _near_miss, + "topo_interior_crossing": _crossing, +} + + +def issues_for(focus: str, index: dict) -> list[Issue]: + if index.get("error"): + return [ + Issue( + "info", + focus, + f"Topology not checked on {', '.join(index['layers'])}: the drawing could not be " + f"read ({index['error']}).", + [], + {"layers": list(index["layers"])}, + ) + ] + findings = index.get("findings") + if not findings: + return [] + return _CHECKS[focus](findings) + + +def _make(focus: str): + async def check(backend: AutoCADBackend, shared: dict) -> list[Issue]: + index = shared.get(_KEY) + if index is None: + index = shared[_KEY] = await build_index(backend) + return issues_for(focus, index) + + check.needs_shared = True # type: ignore[attr-defined] + check.__name__ = f"check_{focus}" + return check + + +TOPO_DISPATCH = {focus: _make(focus) for focus in TOPO_FOCUSES} +``` + +- [ ] **Step 8: Run the critique tests to verify only the registration is missing** + +Run: `uv run --frozen pytest tests/test_understand_critique.py -q` +Expected: FAIL — `4 failed, 10 passed`: `test_the_three_focuses_are_the_spec_ones_and_are_in_the_closed_enum` (`AssertionError: assert 'topo_dangling_endpoint' in ('iso128', …)`), `test_a_pid_from_spec_sheet_gets_no_topo_issue` and `test_an_arch_plan_from_spec_gets_no_topo_issue` (three `Unknown critique focus 'topo_…' (closed enum).` warnings instead of `[]`), and `test_run_critique_reaches_the_topo_focuses` (`'warning' == 'error'`). + +- [ ] **Step 9: Register the three focuses** + +In `engineering/plan_spec.py`, the end of the `CritiqueFocus` Literal + +```python + "arch_opening_clash", +] +``` + +becomes + +```python + "arch_opening_clash", + "topo_dangling_endpoint", + "topo_near_miss", + "topo_interior_crossing", +] +``` + +and the end of `ALL_CRITIQUE_FOCUSES` + +```python + "arch_opening_clash", +) +``` + +becomes + +```python + "arch_opening_clash", + "topo_dangling_endpoint", + "topo_near_miss", + "topo_interior_crossing", +) +``` + +In `engineering/critique.py`, the arch registration + +```python +from .arch.critique import ARCH_DISPATCH # noqa: E402 (same reason: after the base focuses) + +_FOCUS_DISPATCH.update(ARCH_DISPATCH) +``` + +becomes + +```python +from .arch.critique import ARCH_DISPATCH # noqa: E402 (same reason: after the base focuses) + +_FOCUS_DISPATCH.update(ARCH_DISPATCH) + +from .understand.critique import TOPO_DISPATCH # noqa: E402 (same reason: after the base focuses) + +_FOCUS_DISPATCH.update(TOPO_DISPATCH) +``` + +- [ ] **Step 10: Run the critique tests to verify they pass** + +Run: `uv run --frozen pytest tests/test_understand_critique.py tests/test_pid_critique.py tests/test_mech_critique.py tests/test_arch_critique.py -q` +Expected: all pass (`14 passed` in the new file). The three older critique files prove the new focuses change nothing on the drawings they build — every P&ID, mechanical and architectural fixture sits on layers the exclusion covers or the vocabulary does not call a network. + +- [ ] **Step 11: Write the failing wire tests** + +Create `tests/test_understand_topology_tool.py`: + +```python +"""drawing_topology_check over the MCP wire, on the headless engine.""" + +from __future__ import annotations + +import ezdxf +import pytest +import pytest_asyncio +from fastmcp import Client + +import server +from tests.fixtures.plant_pair import build_plant_pair + +pytestmark = pytest.mark.asyncio + + +@pytest.fixture(scope="module") +def pipe_layer(tmp_path_factory) -> str: + truth = build_plant_pair(tmp_path_factory.mktemp("plant")) + return next( + name for name, service in sorted(truth["layer_services"].items()) if service == "product" + ) + + +@pytest_asyncio.fixture +async def client(monkeypatch): + """In-memory client on a fresh headless document - never the operator's AutoCAD.""" + monkeypatch.setenv("AUTOCAD_MCP_BACKEND", "ezdxf") + async with Client(server.mcp) as connected: + status = (await connected.call_tool("system_status", {})).structured_content or {} + assert status.get("backend") == "ezdxf", "these tests must never touch live AutoCAD" + await connected.call_tool("drawing_new", {}) + yield connected + + +async def _line(client, layer, x1, y1, x2, y2): + await client.call_tool( + "entity_create_line", {"x1": x1, "y1": y1, "x2": x2, "y2": y2, "layer": layer} + ) + + +async def _total(client) -> int: + stats = (await client.call_tool("analysis_entity_stats", {})).structured_content + return stats["total_entities"] + + +async def test_the_default_layers_are_the_classified_network_layers(client, pipe_layer): + await _line(client, pipe_layer, 0, 0, 1000, 0) + await _line(client, pipe_layer, 1003, 0, 2003, 0) + before = await _total(client) + result = (await client.call_tool("drawing_topology_check", {})).structured_content + assert result["layers"] == [pipe_layer] + assert [row["layer"] for row in result["classification"]] == [pipe_layer] + assert result["counts"] == {"dangling": 2, "near_miss": 1, "crossing": 0} + assert result["near_miss"][0]["gap"] == pytest.approx(3.0) + assert await _total(client) == before, "the check never writes" + + +async def test_a_dxf_path_is_read_without_opening_it(client, pipe_layer, tmp_path): + doc = ezdxf.new() + doc.layers.add(pipe_layer) + msp = doc.modelspace() + msp.add_line((0, 0), (1000, 0), dxfattribs={"layer": pipe_layer}) + msp.add_line((500, -500), (500, 500), dxfattribs={"layer": pipe_layer}) + path = tmp_path / "crossing.dxf" + doc.saveas(path) + result = ( + await client.call_tool("drawing_topology_check", {"path": str(path), "gap": 5.0}) + ).structured_content + assert result["counts"]["crossing"] == 1 + assert result["crossing"][0]["at"] == [500.0, 0.0] + assert await _total(client) == 0, "the current document is untouched" + + +async def test_an_unknown_layer_is_refused_with_the_drawings_layers(client, pipe_layer): + await _line(client, pipe_layer, 0, 0, 10, 0) + result = await client.call_tool( + "drawing_topology_check", {"layers": ["NOPE"]}, raise_on_error=False + ) + assert result.is_error is True + assert "layers: 'NOPE' is not a layer of this drawing" in result.content[0].text + + +async def test_a_drawing_with_no_network_layer_is_refused_not_passed(client): + await _line(client, "GEOMETRY", 0, 0, 10, 0) + result = await client.call_tool("drawing_topology_check", {}, raise_on_error=False) + assert result.is_error is True + assert "no layer of this drawing is classified piping or electrical" in result.content[0].text + + +async def test_gap_must_exceed_tol(client, pipe_layer): + await _line(client, pipe_layer, 0, 0, 10, 0) + result = await client.call_tool( + "drawing_topology_check", {"gap": 0.005, "tol": 0.01}, raise_on_error=False + ) + assert result.is_error is True + assert "gap (0.005) must exceed tol (0.01)" in result.content[0].text + + +async def test_the_check_is_advertised_read_only(client): + tools = {tool.name: tool for tool in await client.list_tools()} + assert tools["drawing_topology_check"].annotations.readOnlyHint is True +``` + +- [ ] **Step 12: Run them to verify they fail** + +Run: `uv run --frozen pytest tests/test_understand_topology_tool.py -q` +Expected: FAIL — `6 failed`: the calling tests with `fastmcp.exceptions.ToolError: Not found: Unknown tool: 'drawing_topology_check'`, the refusal tests on their text assertions, and `test_the_check_is_advertised_read_only` with `KeyError: 'drawing_topology_check'`. + +- [ ] **Step 13: Register `drawing_topology_check` in `server.py`** + +First change group D's SECTION 26 banner line + +```python +# ── SECTION 26: Understanding & QA (1 tool) ───────────────────────────────── +``` + +to + +```python +# ── SECTION 26: Understanding & QA (2 tools) ──────────────────────────────── +``` + +Then insert this block directly after `drawing_diff` (after its final `return report`, two blank lines before and after), still above the RESOURCES banner: + +```python +@cad_tool( + summary="Find dangling ends, near misses and unjoined crossings in a pipe or cable network.", + cost="read", +) +@mcp.tool( + annotations={"title": "Drawing: Topology Check", "readOnlyHint": True}, + tags={"analysis", "query"}, +) +async def drawing_topology_check( + layers: Annotated[ + list[str] | None, + Field( + default=None, + description="Layers to check; default: every layer the vocabulary classifies as " + "piping or electrical with confidence >= 0.9.", + ), + ] = None, + path: Annotated[ + str | None, + Field( + default=None, + description="A DXF to check instead of the current document; it is read, never " + "opened or modified.", + ), + ] = None, + gap: Annotated[ + float | None, + Field( + default=None, + gt=0, + description="Two ends, or an end and a line, closer than this without touching are a " + "near miss; default 0.2 % of the network's diagonal (never under 10 x tol).", + ), + ] = None, + tol: Annotated[ + float, + Field(default=0.01, gt=0, description="Within this distance two ends are joined."), + ] = 0.01, + limit: Annotated[ + int, + Field(default=200, ge=1, le=5000, description="Rows kept per list; counts are complete."), + ] = 200, + ctx: Context = None, +) -> dict: + """Topology of a line network (spec §11): dangling ends, near misses (end to end and + end to line within `gap`) and interior crossings (two lines of one layer crossing + away from their ends). Never modifies the drawing. + + An end that touches another network line, any curve on another layer (a tank + outline) or the box of any block reference (a valve, a pump) is connected. Arcs + and bulged polyline segments are measured as true arcs; arcs are not tested for + crossings. The default layers come with their classification as evidence. + + Refused before any work: a `path` that does not exist or cannot be read, a + `layers` entry that is not a layer of the drawing (the drawing's layers named), + no layer classified as a network layer when `layers` is omitted (name them), + a non-positive `tol` or `gap`, and a `gap` that does not exceed `tol`. + """ + from engineering.understand.snapshot import take_snapshot + from engineering.understand.topology import cap_findings, network_layers, topology_findings + + backend = _backend(ctx) + source = None + if path: + validated = validate_path(path) + if not validated.is_file(): + raise ToolError(f"path: {validated} does not exist or is not a file") + source = str(validated) + try: + snap = await take_snapshot(backend, source) + except (OSError, ValueError) as exc: + raise ToolError(f"path: {exc}") from exc + known = set(snap.layers) | {rec.layer for rec in snap.records} + classification = network_layers(snap) + if layers: + missing = [name for name in layers if name not in known] + if missing: + listed = ", ".join(sorted(known)[:20]) + (" ..." if len(known) > 20 else "") + raise ToolError( + f"layers: {', '.join(repr(m) for m in missing)} is not a layer of this drawing; " + f"its layers are {listed}" + ) + chosen = list(layers) + else: + chosen = [row["layer"] for row in classification] + if not chosen: + raise ToolError( + "no layer of this drawing is classified piping or electrical with confidence " + ">= 0.9 - name the layers to check with `layers` (drawing_understand shows what " + "each layer carries)" + ) + await ctx.info(f"drawing_topology_check: {len(chosen)} layer(s) of {snap.source}") + try: + findings = topology_findings(snap, layers=chosen, gap=gap, tol=tol) + except ValueError as exc: + raise ToolError(str(exc)) from exc + report = cap_findings(findings, limit) + report["source"] = snap.source + report["classification"] = [row for row in classification if row["layer"] in set(chosen)] + return report +``` + +Then make the two focus lists name the new focuses. In `drawing_finalize`'s docstring, + +```python + architectural focuses arch_room_unlabelled, arch_wall_gap, arch_opening_clash — + silent on a sheet with no ACADMCP_ARCH record), + then saves to disk, exports a screenshot, and returns the DWG path. +``` + +becomes + +```python + architectural focuses arch_room_unlabelled, arch_wall_gap, arch_opening_clash — + silent on a sheet with no ACADMCP_ARCH record, plus the topology focuses + topo_dangling_endpoint, topo_near_miss, topo_interior_crossing — on network + layers no P&ID or architectural focus reads, silent on every sheet this server draws), + then saves to disk, exports a screenshot, and returns the DWG path. +``` + +and in `drawing_critique`'s `focus` description, + +```python + "the architectural focuses arch_room_unlabelled, arch_wall_gap, " + "arch_opening_clash (silent on a drawing with no ACADMCP_ARCH record). " + "None = run all.", +``` + +becomes + +```python + "the architectural focuses arch_room_unlabelled, arch_wall_gap, " + "arch_opening_clash (silent on a drawing with no ACADMCP_ARCH record), and " + "the topology focuses topo_dangling_endpoint, topo_near_miss, " + "topo_interior_crossing (network layers no P&ID or architectural focus reads). " + "None = run all.", +``` + +- [ ] **Step 14: Add the alias record** + +In `discovery/aliases.py`, insert directly after the `"drawing_diff"` record (Task 9), before the closing `}` of `TOOL_ALIASES`: + +```python + "drawing_topology_check": ToolAliases( + synonyms=( + "dangling pipe ends", + "pipes that do not connect", + "gaps in the pipe network", + "unconnected cable ends", + "network connectivity check", + "açık uçlar", + "acik uclar", + "kopuk hatlar", + ), + ), +``` + +`acad` stays empty: vanilla AutoCAD has no command that reports network topology (the Map 3D cleanup tool edits the drawing, which this tool never does). The synonyms avoid `lines`, `cross` and `overshoot`, the tokens of the golden `entity_trim` case. + +- [ ] **Step 15: Run the wire tests to verify they pass** + +Run: `uv run --frozen pytest tests/test_understand_topology.py tests/test_understand_critique.py tests/test_understand_topology_tool.py -q` +Expected: `49 passed`. + +- [ ] **Step 16: Regenerate the inventory and bump the documented counts** + +Run: `uv run --frozen python scripts/generate_tool_inventory.py` +Expected output: `wrote docs/tool-inventory.json: 243 tools, 8 resources, 5 prompts, 26 groups` + +In `README.md` change `242 tools ·` to `243 tools ·`, `> 242 is the **registered** count; a default install advertises 237 over` to `> 243 is the **registered** count; a default install advertises 238 over`, and `242 tools in 26 groups;` to `243 tools in 26 groups;`. The collected-tests figure and the alias-corpus sentence stay as they are (Task 11). + +- [ ] **Step 17: Bump the frozen group-size snapshot** + +In `tests/test_tool_registry.py::test_tool_group_sizes_are_unchanged`: the docstring's first line becomes `"""Frozen snapshot of the surface (243 tools, 26 groups).`; `"analysis": 13,` becomes `"analysis": 14,`; `assert sum(sizes.values()) == 242` becomes `assert sum(sizes.values()) == 243`; and the paragraph Task 9 added at the end of the docstring is replaced, so the closing lines + +```python + had grown while the tag was unranked on them returned to its + pre-track-F value. + + Track H group D opened its copy of SECTION 26 (Understanding & QA) with + `drawing_diff`, tagged `analysis` and `query`; `analysis` outranks `query` + in `_GROUP_TAG_PRIORITY`, so it files under `analysis` (12 -> 13). This + snapshot is branch-local: the track H merge recomputes it once the + understand, network and diff branches have landed. + """ +``` + +become + +```python + had grown while the tag was unranked on them returned to its + pre-track-F value. + + Track H group D opened its copy of SECTION 26 (Understanding & QA) with + `drawing_diff` and `drawing_topology_check`, both tagged `analysis` and + `query`; `analysis` outranks `query` in `_GROUP_TAG_PRIORITY`, so both file + under `analysis` (12 -> 14). This snapshot is branch-local: the track H + merge recomputes it once the understand, network and diff branches have + landed. + """ +``` + +- [ ] **Step 18: Gates** + +Run each and require it clean: + +```bash +uv run --frozen ruff check engineering/understand/topology.py engineering/understand/critique.py engineering/plan_spec.py engineering/critique.py server.py discovery/aliases.py tests/test_understand_topology.py tests/test_understand_critique.py tests/test_understand_topology_tool.py tests/test_tool_registry.py +uv run --frozen ruff format engineering/understand/topology.py engineering/understand/critique.py tests/test_understand_topology.py tests/test_understand_critique.py tests/test_understand_topology_tool.py +uv run --frozen ruff format --check engineering/plan_spec.py engineering/critique.py server.py discovery/aliases.py tests/test_tool_registry.py +uv run --frozen python scripts/generate_tool_inventory.py --check +uv run --frozen pytest tests/test_understand_topology.py tests/test_understand_critique.py tests/test_understand_topology_tool.py tests/test_understand_diff.py tests/test_understand_diff_tool.py -q +uv run --frozen pytest tests/test_pid_critique.py tests/test_mech_critique.py tests/test_arch_critique.py tests/test_meta_tools.py tests/test_benchmark_v5.py tests/test_benchmark_v6.py -q +uv run --frozen pytest tests/test_tool_registry.py tests/test_tool_inventory.py tests/test_tool_packs.py tests/test_discovery_aliases.py tests/test_discovery_transform.py tests/test_capability_contract.py -q +uv run --frozen pytest tests/test_release_consistency.py -q --deselect tests/test_release_consistency.py::test_document_numbers_match_the_tree +uv run --frozen pytest -q --deselect tests/test_release_consistency.py::test_document_numbers_match_the_tree +``` + +`test_section_header_counts_match_decorators` passes: the banner says `(2 tools)` and two `@mcp.tool` decorators sit under it. The second pytest line is the regression net for `focus=None`: the benchmark gates require zero critique issues on the v5 and v6 sheets, and they must stay at zero with three more focuses in the closed enum. No capability key is added (both tools and the focuses compose generic members), so `tests/test_capability_contract.py` passes untouched. The deselect is the known-red collected-tests drift of the Global Constraints; say so in the task report. + +- [ ] **Step 19: Commit** + +```bash +git add engineering/understand/topology.py engineering/understand/critique.py engineering/plan_spec.py engineering/critique.py server.py discovery/aliases.py docs/tool-inventory.json README.md tests/test_tool_registry.py tests/test_understand_topology.py tests/test_understand_critique.py tests/test_understand_topology_tool.py +git -c user.name="Umut CAN" -c user.email="umutcan.edizaslan@gmail.com" commit -m "$(cat <<'MSG' +feat(understand): drawing_topology_check and the three topo_* critique focuses + +engineering/understand/topology.py reads a line network (LINE, polylines +with true bulge arcs, ARC) on the chosen layers and reports dangling +ends, near misses (end-end and end-segment within gap, one row per pair) +and interior crossings of two straight segments of one layer. An end on +another network line is joined; an end on any curve of another layer or +inside a block reference's box is attached; a line drawn twice is not a +join, so a doubled pipe still dangles. gap defaults to 0.2 % of the +network diagonal, never under ten tol. + +drawing_topology_check joins drawing_diff in SECTION 26: default layers +are those classify_layer calls piping or electrical at >= 0.9, reported +with their classification; no such layer is a refusal, not a pass. + +topo_dangling_endpoint (warning), topo_near_miss (error) and +topo_interior_crossing (info) join the closed enum and run under +focus=None, excluding the P&ID and architectural layer sets, so a +pid_from_spec sheet and an arch_plan_from_spec plan get none of them; +a drawing with no candidate layer is never snapshotted. + +Co-Authored-By: Claude Opus 5.5 (1M context) +Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd +MSG +)" +``` + +--- + +### Task 11: Integration on main — packs, lean, aliases, golden queries, counts + +**Files:** +- Modify: `server.py` (`TOOL_PACK_NAMES` gains `"plant"`; `PACK_TOOL_NAMES["plant"]`; `"drawing_understand"` into `LEAN_TOOL_NAMES` → 65; `"understand"` and `"plant"` ranked after `"arch"` in `_GROUP_TAG_PRIORITY` with labels `understanding` / `plant`; SECTION 26 / 27 tags checked) +- Modify: `discovery/aliases.py` (spec §12's five-language vocabulary folded into the six track-H records; `SHARED_ACAD_COMMANDS` reconciled) +- Modify: `tests/data/discovery_golden.py` (eight tuning cases; the docstring count) +- Modify: `tests/test_discovery_aliases.py` (one test: the §12 vocabulary is in the records) +- Modify: `tests/test_tool_packs.py` (replaced: the `plant` pack, lean 65, 28 groups) +- Modify: `tests/test_token_suite.py` (the two single-vertical rows now also exclude `plant`) +- Modify: `tests/test_tool_registry.py` (frozen group-size snapshot: 247 tools, 28 groups) +- Modify: `docs/tool-inventory.json` (regenerated, never hand-edited) +- Modify: `README.md` (release snapshot tools / advertised / collected tests, the alias-corpus sentence, the Discovery row, the `` pack line, the `TOOL_PROFILE` and `TOOL_PACKS` configuration rows) +- Test: `tests/test_tool_packs.py`, `tests/test_discovery_aliases.py`, `tests/test_discovery_transform.py`, `tests/test_tool_registry.py`, `tests/test_token_suite.py` + +**Interfaces:** +- Consumes: the six tools the groups registered — SECTION 26 `drawing_understand`, `drawing_scale_check` (group U, Tasks 3-4, tagged `{"analysis", "query"}`), `drawing_diff`, `drawing_topology_check` (group D, Tasks 9-10, tagged `{"analysis", "query"}`); SECTION 27 `pipe_takeoff`, `cable_takeoff` (group N, Tasks 7-8, tagged `{"plant", "analysis"}`) — each with its alias record in `discovery/aliases.py` (every one `acad=()` except `drawing_diff`'s `("COMPARE",)`), and each branch's paragraph in the `test_tool_group_sizes_are_unchanged` docstring saying the six filed under `analysis` while `understand` / `plant` were unranked; `server._registered_tools()`, `server._tool_groups()`, `server._apply_tool_profile(profile) -> {"disabled_tools", "enabled_count", "tool_packs"}`; `discovery.aliases.TOOL_ALIASES`, `ToolAliases(acad=(), synonyms=())`, `SHARED_ACAD_COMMANDS`. +- Produces: `server.TOOL_PACK_NAMES == ("core", "pid", "settings", "mech", "arch", "plant")`, `server.PACK_TOOL_NAMES["plant"] == frozenset({"pipe_takeoff", "cable_takeoff"})`, `len(server.LEAN_TOOL_NAMES) == 65` (55 with `TOOL_PACKS=core`), groups `understanding` (the four SECTION 26 tools, core pack) and `plant` (exactly `PACK_TOOL_NAMES["plant"]`) — 247 tools in 28 groups; `discovery/aliases.py::_TRACK_H_VOCABULARY` and `_fold_in(records, extra)`; eight golden tuning cases. + +- [ ] **Step 1: Write the failing test for the new surface — replace `tests/test_tool_packs.py`** + +```python +# tests/test_tool_packs.py +"""TOOL_PACKS: advertise only the vertical packs a client uses; core is always on. + +v1.6 track E adds the `settings` pack: the 22 SECTION 20 environment tools. +Styles (SECTION 18) and page setup / templates (SECTION 19) are drafting +essentials and stay in `core`, so `TOOL_PACKS=core` hides the environment +surface and nothing else. + +v1.6 tracks B+G add the `mech` pack: SECTION 21's six part-model tools and +SECTION 22's three standard-parts tools and SECTION 23's five annotation +symbols - 14 in all. SECTION 24 (the sheet: frames, title blocks, revisions, +the parts list, balloons, xrefs, images, DWG) stays in `core` - universal +drafting, not a vertical - and files under its own `sheet` group. + +v1.6 track F adds the `arch` pack: SECTION 25's twelve architectural tools - +the wall network, openings, stairs, rooms and their reader, schedules, the +grid and symbols, the furniture and sanitary catalogue, the exterior chains +and the whole plan from one spec. Lean gains four of them. + +v1.6 track H adds the `plant` pack: SECTION 27's two takeoffs (`pipe_takeoff`, +`cable_takeoff`) - quantities for a plant job, the vertical track D's Plant 3D +readers join later. SECTION 26's four readers (`drawing_understand`, +`drawing_scale_check`, `drawing_diff`, `drawing_topology_check`) stay in +`core`: understanding, comparing and checking a drawing is not a vertical. +Lean gains `drawing_understand` and reaches 65. +""" + +from __future__ import annotations + +import logging + +import pytest +import pytest_asyncio + +import config +import server + +pytestmark = pytest.mark.asyncio + +PID_TOOLS = { + "pid_symbol_list", + "pid_symbol_insert", + "pid_line_draw", + "pid_tag_parse", + "pid_graph", + "pid_instrument_index", + "pid_line_list", + "pid_equipment_list", + "pid_from_spec", +} + +STYLE_TOOLS = { + "dimstyle_list", + "dimstyle_create", + "dimstyle_modify", + "dimstyle_set_current", + "textstyle_list", + "textstyle_create", + "textstyle_set_current", + "mleaderstyle_list", + "mleaderstyle_create", + "drawing_apply_standard", +} + +PAGE_SETUP_TOOLS = { + "page_setup_list", + "page_setup_apply", + "plot_style_list", + "batch_plot", + "drawing_template_list", + "drawing_template_save", +} + +SETTINGS_TOOLS = { + "document_list", + "document_activate", + "document_close", + "layer_state_save", + "layer_state_restore", + "layer_state_list", + "layer_state_delete", + "view_named_save", + "view_named_restore", + "view_named_list", + "ucs_list", + "ucs_set", + "ucs_restore", + "system_launch", + "system_preferences_get", + "system_preferences_set", + "drawing_properties_get", + "drawing_properties_set", + "system_variable_describe", + "user_pick_point", + "user_select", + "system_prompt_message", +} + +MECH_TOOLS = { + "mech_part_draw", + "mech_view_add", + "mech_dimension_part", + "mech_hole_pattern", + "mech_part_from_spec", + "mech_part_inspect", + "std_part_list", + "std_part_insert", + "std_feature_draw", + "surface_texture", + "weld_symbol", + "centre_marks", + "section_line", + "hatch_material", +} + +SHEET_TOOLS = { + "sheet_frame", + "titleblock_apply", + "revision_add", + "bom_extract", + "bom_table", + "balloon_add", + "data_extract", + "xref_attach", + "xref_manage", + "image_attach", + "drawing_export_dwg", +} + +ARCH_TOOLS = { + "arch_wall", + "arch_opening", + "arch_stair", + "arch_room", + "arch_rooms_detect", + "arch_schedule", + "arch_grid", + "arch_symbol", + "arch_catalogue_list", + "arch_catalogue_insert", + "arch_dimension_chains", + "arch_plan_from_spec", +} + +UNDERSTAND_TOOLS = { + "drawing_understand", + "drawing_scale_check", + "drawing_diff", + "drawing_topology_check", +} + +PLANT_TOOLS = { + "pipe_takeoff", + "cable_takeoff", +} + +LEAN_UNDERSTAND_ESSENTIALS = {"drawing_understand"} + +LEAN_ARCH_ESSENTIALS = { + "arch_wall", + "arch_opening", + "arch_room", + "arch_plan_from_spec", +} + +LEAN_MECH_ESSENTIALS = { + "mech_part_draw", + "mech_view_add", + "mech_dimension_part", + "sheet_frame", + "titleblock_apply", +} + +LEAN_SETTINGS_ESSENTIALS = { + "drawing_apply_standard", + "page_setup_apply", + "batch_plot", + "dimstyle_set_current", + "textstyle_set_current", +} + +#: Declaration order - what `tool_packs["available"]` reports. +ALL_PACKS = ["core", "pid", "settings", "mech", "arch", "plant"] +#: `tool_packs["enabled"]` is sorted, which is no longer the same list. +ALL_PACKS_ENABLED = sorted(ALL_PACKS) + + +@pytest_asyncio.fixture(autouse=True) +async def _restore(monkeypatch): + yield + monkeypatch.setattr(config.settings, "tool_packs", "all") + await server._apply_tool_profile("full") + + +def test_pack_registry_names_real_tools_and_only_them(): + assert server.TOOL_PACK_NAMES == ("core", "pid", "settings", "mech", "arch", "plant") + assert server.PACK_TOOL_NAMES["pid"] == frozenset(PID_TOOLS) + assert server.PACK_TOOL_NAMES["mech"] == frozenset(MECH_TOOLS) + assert server.PACK_TOOL_NAMES["settings"] == frozenset(SETTINGS_TOOLS) + assert server.PACK_TOOL_NAMES["arch"] == frozenset(ARCH_TOOLS) + assert server.PACK_TOOL_NAMES["plant"] == frozenset(PLANT_TOOLS) + assert len(SETTINGS_TOOLS) == 22 + assert len(MECH_TOOLS) == 14 + assert len(ARCH_TOOLS) == 12 + assert len(PLANT_TOOLS) == 2 + assert not (server.PACK_TOOL_NAMES["settings"] & server.PACK_TOOL_NAMES["pid"]) + assert not (server.PACK_TOOL_NAMES["mech"] & server.PACK_TOOL_NAMES["pid"]) + assert not (server.PACK_TOOL_NAMES["mech"] & server.PACK_TOOL_NAMES["settings"]) + for pack in ("pid", "settings", "mech"): + assert not (server.PACK_TOOL_NAMES["arch"] & server.PACK_TOOL_NAMES[pack]), pack + for pack in ("pid", "settings", "mech", "arch"): + assert not (server.PACK_TOOL_NAMES["plant"] & server.PACK_TOOL_NAMES[pack]), pack + assert not (UNDERSTAND_TOOLS & set().union(*server.PACK_TOOL_NAMES.values())), ( + "SECTION 26 reads any drawing: core, not a vertical pack" + ) + assert not (SHEET_TOOLS & set().union(*server.PACK_TOOL_NAMES.values())), ( + "SECTION 24 is core drafting, not a vertical pack" + ) + + +async def test_every_pack_member_is_a_registered_tool(): + registered = {tool.name for tool in await server._registered_tools()} + for pack, names in server.PACK_TOOL_NAMES.items(): + missing = names - registered + assert not missing, f"pack {pack!r} names unregistered tools: {sorted(missing)}" + + +async def test_core_only_hides_pid_and_settings_but_keeps_styles_and_page_setup(monkeypatch): + monkeypatch.setattr(config.settings, "tool_packs", "core") + info = await server._apply_tool_profile("full") + disabled = set(info["disabled_tools"]) + assert PID_TOOLS <= disabled + assert SETTINGS_TOOLS <= disabled + assert MECH_TOOLS <= disabled + assert ARCH_TOOLS <= disabled + assert PLANT_TOOLS <= disabled + assert not (SHEET_TOOLS & disabled) + assert not (UNDERSTAND_TOOLS & disabled), "the readers of SECTION 26 are core" + assert not (STYLE_TOOLS & disabled), "styles are core drafting essentials" + assert not (PAGE_SETUP_TOOLS & disabled), "page setup and templates are core" + assert "block_define" not in disabled and "entity_set_xdata" not in disabled + assert info["tool_packs"] == {"enabled": ["core"], "available": ALL_PACKS, "ignored": []} + + +async def test_settings_pack_alone_keeps_pid_hidden(monkeypatch): + monkeypatch.setattr(config.settings, "tool_packs", "settings") + info = await server._apply_tool_profile("full") + disabled = set(info["disabled_tools"]) + assert PID_TOOLS <= disabled + assert not (SETTINGS_TOOLS & disabled) + assert info["tool_packs"]["enabled"] == ["core", "settings"] + + +async def test_the_plant_pack_alone_adds_the_takeoffs_to_core(monkeypatch): + monkeypatch.setattr(config.settings, "tool_packs", "plant") + info = await server._apply_tool_profile("full") + disabled = set(info["disabled_tools"]) + assert not (PLANT_TOOLS & disabled) + assert not (UNDERSTAND_TOOLS & disabled) + assert PID_TOOLS <= disabled and ARCH_TOOLS <= disabled + assert info["tool_packs"]["enabled"] == ["core", "plant"] + + +async def test_all_is_the_default_and_enables_every_pack(monkeypatch): + monkeypatch.setattr(config.settings, "tool_packs", "all") + info = await server._apply_tool_profile("full") + disabled = set(info["disabled_tools"]) + assert not ((PID_TOOLS | SETTINGS_TOOLS | MECH_TOOLS | ARCH_TOOLS | PLANT_TOOLS) & disabled) + assert info["tool_packs"]["enabled"] == ALL_PACKS_ENABLED + + +async def test_unknown_pack_is_ignored_with_a_warning_and_core_cannot_be_dropped( + monkeypatch, caplog +): + monkeypatch.setattr(config.settings, "tool_packs", "pid,mech3000") + with caplog.at_level(logging.WARNING, logger="autocad_mcp"): + info = await server._apply_tool_profile("full") + assert info["tool_packs"] == { + "enabled": ["core", "pid"], + "available": ALL_PACKS, + "ignored": ["mech3000"], + } + assert "mech3000" in caplog.text + + +async def test_lean_intersects_with_packs(monkeypatch): + assert {"pid_symbol_insert", "pid_line_draw", "pid_graph"} <= server.LEAN_TOOL_NAMES + monkeypatch.setattr(config.settings, "tool_packs", "core") + info = await server._apply_tool_profile("lean") + assert "pid_graph" in info["disabled_tools"] + assert "entity_create_line" not in info["disabled_tools"] + + +async def test_lean_carries_the_five_settings_essentials_and_nothing_from_the_settings_pack( + monkeypatch, +): + """Spec §8.2: lean = 65 (55 + the five mechanical/sheet essentials + track + F's four architectural ones + track H's `drawing_understand`). The five + settings essentials and `drawing_understand` are core-pack tools, so + `TOOL_PACKS=core` leaves them on a lean surface; no environment tool and no + takeoff is lean.""" + assert LEAN_SETTINGS_ESSENTIALS <= server.LEAN_TOOL_NAMES + assert LEAN_MECH_ESSENTIALS <= server.LEAN_TOOL_NAMES + assert LEAN_ARCH_ESSENTIALS <= server.LEAN_TOOL_NAMES + assert server.LEAN_TOOL_NAMES & ARCH_TOOLS == LEAN_ARCH_ESSENTIALS + assert server.LEAN_TOOL_NAMES & UNDERSTAND_TOOLS == LEAN_UNDERSTAND_ESSENTIALS + assert not (PLANT_TOOLS & server.LEAN_TOOL_NAMES) + assert not (SETTINGS_TOOLS & server.LEAN_TOOL_NAMES) + assert len(server.LEAN_TOOL_NAMES) == 65 + monkeypatch.setattr(config.settings, "tool_packs", "core") + info = await server._apply_tool_profile("lean") + disabled = set(info["disabled_tools"]) + assert not (LEAN_SETTINGS_ESSENTIALS & disabled) + assert not ({"sheet_frame", "titleblock_apply"} & disabled), "sheet tools are core" + assert "drawing_understand" not in disabled, "the one-call report is core" + assert info["enabled_count"] == 65 - 10, ( + "lean minus the three pid, three mech and four arch tools" + ) + + +async def test_the_three_track_e_sections_file_under_their_own_groups(): + """Pack, group and SECTION are the same list, so `system_about` and + `docs/tool-inventory.json` describe the surface the packs gate.""" + groups = await server._tool_groups() + assert set(groups["styles"]) == STYLE_TOOLS + assert set(groups["page_setup"]) == PAGE_SETUP_TOOLS + assert set(groups["environment"]) == SETTINGS_TOOLS + assert set(groups["mechanical"]) == MECH_TOOLS + assert set(groups["sheet"]) == SHEET_TOOLS + assert set(groups["architecture"]) == ARCH_TOOLS + assert set(groups["understanding"]) == UNDERSTAND_TOOLS + assert set(groups["plant"]) == PLANT_TOOLS + assert len(groups) == 28 + + +async def test_system_about_reports_packs(): + from fastmcp import Client + + async with Client(server.mcp) as client: + about = (await client.call_tool("system_about", {})).structured_content + assert about["tool_packs"]["available"] == ALL_PACKS +``` + +- [ ] **Step 2: Run it to verify it fails** + +Run: `uv run --frozen pytest tests/test_tool_packs.py -q` +Expected: FAIL — `test_pack_registry_names_real_tools_and_only_them` with `AssertionError: assert ('core', 'pid', 'settings', 'mech', 'arch') == ('core', 'pid', 'settings', 'mech', 'arch', 'plant')`; `test_the_plant_pack_alone_adds_the_takeoffs_to_core` with `assert ['core'] == ['core', 'plant']` (`plant` is still an unknown pack, ignored with a warning); the lean test with `assert 64 == 65`; the groups test with `KeyError: 'understanding'` (or `'plant'`, whichever the branches' secondary tags left unfiled); `test_system_about_reports_packs` on the `available` list. + +- [ ] **Step 3: The `plant` pack, the lean name and the two groups in `server.py`** + +Replace + +```python +TOOL_PACK_NAMES = ("core", "pid", "settings", "mech", "arch") +``` + +with + +```python +TOOL_PACK_NAMES = ("core", "pid", "settings", "mech", "arch", "plant") +``` + +In `PACK_TOOL_NAMES`, replace the end of the `"arch"` entry and the dict's closing brace + +```python + "arch_plan_from_spec", + } + ), +} +``` + +with + +```python + "arch_plan_from_spec", + } + ), + # SECTION 27 - the plant takeoffs: pipe and cable quantities whose topology + # comes from a P&ID and whose lengths come from its layout (track H spec + # §8-9, §12). SECTION 26's four readers stay in `core`: understanding, + # comparing and checking a drawing is not a vertical. Track D's Plant 3D + # readers join this pack. + "plant": frozenset({"pipe_takeoff", "cable_takeoff"}), +} +``` + +In `LEAN_TOOL_NAMES`, replace the end of the set + +```python + "arch_room", + "arch_plan_from_spec", + } +) +``` + +with + +```python + "arch_room", + "arch_plan_from_spec", + # Understanding (track H) - the one read a lean client needs before it + # touches a drawing somebody else made: units, extents, clusters, + # layers and their services, tags and rooms in one call. A core-pack + # tool, so TOOL_PACKS=core keeps it; with the four architectural ones + # above, lean is 65. The takeoffs are `plant`-pack, full-profile work. + "drawing_understand", + } +) +``` + +In `_GROUP_TAG_PRIORITY`, replace + +```python + "arch", + "sheet", +``` + +with + +```python + "arch", + "understand", + "plant", + "sheet", +``` + +and in `_GROUP_TAG_LABELS`, replace + +```python + "arch": "architecture", +``` + +with + +```python + "arch": "architecture", + "understand": "understanding", + "plant": "plant", +``` + +(If a group branch already added either tag to these two tables, keep exactly one of each, in this position: `uv run --frozen python -c "import server; p = server._GROUP_TAG_PRIORITY; print(p.count('understand'), p.count('plant'), p.index('understand') == p.index('arch') + 1, server._GROUP_TAG_LABELS.get('understand'), server._GROUP_TAG_LABELS.get('plant'))"` must print `1 1 True understanding plant`.) + +- [ ] **Step 4: Every SECTION 26 tool carries `understand`, every SECTION 27 tool `plant`** + +A tool without its section tag files under its secondary one (`query`, `analysis`, `validation`) and the groups test fails. Check rather than trust the branches: + +``` +uv run --frozen python -c "import asyncio, server; tools = asyncio.run(server._registered_tools()); s26 = {'drawing_understand', 'drawing_scale_check', 'drawing_diff', 'drawing_topology_check'}; print('untagged 26:', sorted(t.name for t in tools if t.name in s26 and 'understand' not in (t.tags or set()))); print('untagged 27:', sorted(t.name for t in tools if t.name in server.PACK_TOOL_NAMES['plant'] and 'plant' not in (t.tags or set())))" +``` + +Expected, as the three group plans register them: `untagged 26: ['drawing_diff', 'drawing_scale_check', 'drawing_topology_check', 'drawing_understand']` and `untagged 27: []` (groups U and D tag their tools `{"analysis", "query"}`; group N tags the takeoffs `{"plant", "analysis"}`). In each of the four SECTION 26 registrations in `server.py`, change + +```python + tags={"analysis", "query"}, +``` + +to + +```python + tags={"understand", "analysis", "query"}, +``` + +(only inside the `@mcp.tool(...)` of those four tools; other tools carry the same pair and keep it), and rerun the check until it prints two empty lists. For any other name it prints, add the section tag the same way. Then confirm the two banners count their decorators: + +Run: `uv run --frozen pytest tests/test_release_consistency.py::test_section_header_counts_match_decorators -q` +Expected: pass — `# ── SECTION 26: Understanding & QA (4 tools)` over four registrations (the merge kept one banner, U's block then D's) and `# ── SECTION 27: Plant Takeoffs (2 tools)` over two. If the merge left two SECTION 26 banners, keep the first, delete the second, and set its count to 4. Groups N and D both inserted their blocks directly above the RESOURCES banner and N merges first, so D's `_diff_revision`, `_diff_markup`, `drawing_diff` and `drawing_topology_check` may have landed *below* `# ── SECTION 27: Plant Takeoffs (2 tools)`; move that whole block up to directly after U's `drawing_scale_check`, above the SECTION 27 banner, so SECTION 26 holds U's block then D's and SECTION 27 holds exactly the two takeoffs. + +- [ ] **Step 5: Run the pack and profile gates** + +Run: `uv run --frozen pytest tests/test_tool_packs.py tests/test_tool_profiles.py tests/test_styles.py -q` +Expected: all pass. If `enabled_count` is not `65 - 10`, print it — `uv run --frozen python -c "import asyncio, config, server; config.settings.tool_packs='core'; print(asyncio.run(server._apply_tool_profile('lean'))['enabled_count'])"` — and find which lean name a pack does or does not claim; the count is data, not a target. + +- [ ] **Step 6: The token suite's single-vertical rows exclude the new pack too** + +`TOOL_PACKS=core,mech` and `TOOL_PACKS=core,arch` now also hide the two takeoffs, so the formula in `tests/test_token_suite.py` is short by `plant`. Replace the whole `test_the_idle_lane_covers_the_five_advertised_surfaces` function with: + +```python +def test_the_idle_lane_covers_the_five_advertised_surfaces(report): + """Default, lean, search - and the two single-vertical clients: + TOOL_PACKS=core,mech (the default surface minus exactly the pid, settings, + arch and plant packs) and TOOL_PACKS=core,arch (minus pid, settings, mech + and plant).""" + variants = {row["variant"]: row for row in report["idle"]} + assert set(variants) == {"default", "lean", "packs_core_mech", "packs_core_arch", "search"} + assert variants["default"]["advertised_tools"] > 100 + assert variants["lean"]["advertised_tools"] == len(server.LEAN_TOOL_NAMES) + assert variants["search"]["advertised_tools"] == 2 + packs = server.PACK_TOOL_NAMES + assert variants["packs_core_mech"]["advertised_tools"] == ( + variants["default"]["advertised_tools"] + - len(packs["pid"]) + - len(packs["settings"]) + - len(packs["arch"]) + - len(packs["plant"]) + ) + assert variants["packs_core_arch"]["advertised_tools"] == ( + variants["default"]["advertised_tools"] + - len(packs["pid"]) + - len(packs["settings"]) + - len(packs["mech"]) + - len(packs["plant"]) + ) + for narrowed in ("packs_core_mech", "packs_core_arch"): + assert ( + variants["default"]["chars"] + > variants[narrowed]["chars"] + > variants["lean"]["chars"] + > variants["search"]["chars"] + ) +``` + +Run: `uv run --frozen pytest tests/test_token_suite.py -q` +Expected: all pass (`packs_core_mech` 197 and `packs_core_arch` 195 of a 242-tool default; Task 12 remeasures the README table). + +- [ ] **Step 7: Write the failing test for spec §12's vocabulary** + +Append to `tests/test_discovery_aliases.py` (it already imports `TOOL_ALIASES` from `discovery.aliases`): + +```python +# ── track H: spec §12's five-language vocabulary ─────────────────────────── +# +# One row per term the spec names, in the record(s) a plant engineer means by +# it. Turkish comes in both spellings (diacritic and ASCII-folded): the search +# tokenizer splits on a non-ASCII letter instead of folding it. + +TRACK_H_TERMS = { + "pipe_takeoff": ( + "takeoff", + "boq", + "metraj", + "metre cetveli", + "boru listesi", + "boru metrajı", + "boru metraji", + "boru metrajı çıkar", + "boru metraji cikar", + "спецификация", # specification: a bill of quantities + "ведомость", # statement: a quantity schedule + ), + "cable_takeoff": ( + "takeoff", + "boq", + "metraj", + "metre cetveli", + "kablo listesi", + "kablo metrajı", + "kablo metraji", + ), + "drawing_diff": ( + "compare", + "dwgcompare", + "fark", + "karşılaştır", + "karsilastir", + "revizyon farkı", + "revizyon farki", + "iki revizyonu karşılaştır", + "iki revizyonu karsilastir", + ), + "drawing_scale_check": ( + "scale check", + "ölçek", + "olcek", + "bu çizim ölçekli mi", + "bu cizim olcekli mi", + ), + "drawing_understand": ("birim", "çizim birimi", "cizim birimi"), +} + + +def test_track_h_records_carry_the_five_language_vocabulary(): + missing = { + tool: [term for term in terms if term not in TOOL_ALIASES[tool].synonyms] + for tool, terms in TRACK_H_TERMS.items() + } + missing = {tool: terms for tool, terms in missing.items() if terms} + assert not missing, f"track-H records miss spec §12 vocabulary: {missing}" + assert "COMPARE" in TOOL_ALIASES["drawing_diff"].acad +``` + +- [ ] **Step 8: Run it to verify it fails** + +Run: `uv run --frozen pytest tests/test_discovery_aliases.py -q -k five_language` +Expected: FAIL — `AssertionError: track-H records miss spec §12 vocabulary: {...}`. Against the records the three group plans write, the dict lists `pipe_takeoff: ['boru metrajı çıkar', 'boru metraji cikar']`, `cable_takeoff: ['takeoff', 'boq', 'metraj', 'metre cetveli']`, `drawing_diff: ['compare', 'dwgcompare', 'fark', 'karşılaştır', 'karsilastir', 'revizyon farkı', 'revizyon farki', 'iki revizyonu karşılaştır', 'iki revizyonu karsilastir']`, `drawing_scale_check: ['bu çizim ölçekli mi', 'bu cizim olcekli mi']` and `drawing_understand: ['çizim birimi', 'cizim birimi']`; a term a branch already carries is simply not listed, and the fold below never writes a phrase twice. + +- [ ] **Step 9: Fold the vocabulary into the six records** + +In `discovery/aliases.py`, insert directly after the closing `}` of `TOOL_ALIASES` and before `def aliases_for(`: + +```python +# ── Track H: the five-language vocabulary of spec §12 ────────────────────── +# +# The six track-H records were written by three branches (SECTION 26 by groups +# U and D, SECTION 27 by group N). Spec §12's vocabulary - the words a plant +# engineer types when asking for a takeoff, a comparison or a scale check - is +# folded into them here, after the merge, so every branch's own phrases stay +# and nothing is written twice. A record missing here is a KeyError at import: +# a track-H tool without its branch's record is a merge that lost something. +# +# Turkish carries both spellings, diacritic and ASCII-folded, because the +# search tokenizer splits on a non-ASCII letter instead of folding it (the +# room records above say the same). The Russian terms are data for a client +# that searches this corpus itself: the BM25 tokenizer keeps only [a-z0-9], so +# it cannot index Cyrillic yet, and no golden case pretends it can. +# +# COMPARE is AutoCAD's drawing-compare command (the "DWG Compare" feature); +# "dwg compare" / "dwgcompare" are how drafters name the feature, so they are +# synonyms, not a command token. +_TRACK_H_VOCABULARY: dict[str, tuple[tuple[str, ...], tuple[str, ...]]] = { + "drawing_understand": ( + (), + ( + "what is in this drawing", + "what is in this drawing somebody sent me", + "understand a foreign drawing", + "read a drawing somebody else made", + "which units is this drawing in", + "drawing units check", + "birim", + "birim kontrolü", + "birim kontrolu", + "çizim birimi", + "cizim birimi", + "çizimi anla", + "cizimi anla", + "yabancı çizim", + "yabanci cizim", + ), + ), + "drawing_scale_check": ( + (), + ( + "scale check", + "is this drawing to scale", + "is the p&id to scale", + "schematic or to scale", + "compare distances between two drawings", + "ölçek", + "olcek", + "ölçek kontrolü", + "ölçek kontrolu", + "olcek kontrolu", + "ölçekli mi", + "olcekli mi", + "bu çizim ölçekli mi", + "bu cizim olcekli mi", + ), + ), + "drawing_diff": ( + ("COMPARE",), + ( + "compare", + "compare two revisions", + "compare two revisions of this drawing", + "what changed between revisions", + "drawing diff", + "dwg compare", + "dwgcompare", + "fark", + "revizyon farkı", + "revizyon farki", + "karşılaştır", + "karsilastir", + "iki revizyonu karşılaştır", + "iki revizyonu karsilastir", + "сравнить чертежи", # compare drawings + ), + ), + "drawing_topology_check": ( + (), + ( + "find dangling line ends", + "find dangling line ends and gaps", + "uçları açık çizgiler", + "uclari acik cizgiler", + "kopuk hat", + ), + ), + "pipe_takeoff": ( + (), + ( + "takeoff", + "pipe takeoff", + "pipe takeoff by diameter from the p&id", + "pipe quantities", + "bill of quantities", + "boq", + "metraj", + "metre cetveli", + "boru listesi", + "boru metrajı", + "boru metraji", + "boru metrajı çıkar", + "boru metraji cikar", + "спецификация", # specification: a bill of quantities + "ведомость", # statement: a quantity schedule + ), + ), + "cable_takeoff": ( + (), + ( + "takeoff", + "cable takeoff", + "cable schedule", + "cable lengths to the panel", + "bill of quantities", + "boq", + "metraj", + "metre cetveli", + "kablo listesi", + "kablo metrajı", + "kablo metraji", + "kablo metrajı çıkar", + "kablo metraji cikar", + "кабельная ведомость", # cable schedule + "спецификация", # specification: a bill of quantities + ), + ), +} + + +def _fold_in(records: dict[str, ToolAliases], extra) -> None: + """Append ``extra``'s commands and phrases to existing records, never twice.""" + for name, (acad, synonyms) in extra.items(): + record = records[name] + records[name] = ToolAliases( + acad=record.acad + tuple(c for c in acad if c not in record.acad), + synonyms=record.synonyms + tuple(s for s in synonyms if s not in record.synonyms), + ) + + +_fold_in(TOOL_ALIASES, _TRACK_H_VOCABULARY) +``` + +`COMPARE` is AutoCAD's own drawing-compare command; group D's `drawing_diff` record already claims it and no other record does, so the fold adds nothing to `acad` and nothing to the allowlist. The topology phrases avoid `lines` and `cross`, the tokens of the golden `entity_trim` case — the same care group D's record took. Nothing here is a standards value: these are words users type. + +- [ ] **Step 10: Run it to verify it passes** + +Run: `uv run --frozen pytest tests/test_discovery_aliases.py -q` +Expected: all pass, including `test_track_h_records_carry_the_five_language_vocabulary`, `test_synonyms_are_unique_within_a_record` (the fold skips phrases already present) and `test_each_acad_command_routes_to_a_single_tool`. + +- [ ] **Step 11: Add the eight golden tuning cases** + +In `tests/data/discovery_golden.py`, append to `TUNING_CASES`, after `GoldenCase("oda alanlarını hesapla", "arch_rooms_detect", "synonym"),` and before the closing `)`: + +```python + # v1.6 track H - reading drawings somebody else made. Written before the + # merged alias records were measured against them; a miss is fixed by + # adding vocabulary to that tool's record in discovery/aliases.py, never by + # moving the expectation. Half of them Turkish: a takeoff is asked for in + # the language of the shop that prices the job. + GoldenCase("boru metrajı çıkar", "pipe_takeoff", "synonym"), + GoldenCase("kablo metrajı", "cable_takeoff", "synonym"), + GoldenCase("iki revizyonu karşılaştır", "drawing_diff", "synonym"), + GoldenCase("bu çizim ölçekli mi", "drawing_scale_check", "synonym"), + GoldenCase("what is in this drawing somebody sent me", "drawing_understand", "paraphrase"), + GoldenCase("pipe takeoff by diameter from the p&id", "pipe_takeoff", "paraphrase"), + GoldenCase("compare two revisions of this drawing", "drawing_diff", "paraphrase"), + GoldenCase("find dangling line ends and gaps", "drawing_topology_check", "paraphrase"), +``` + +and change the module docstring's first line `Ninety-nine cases (84 tuning + 15 holdout) split into a **tuning** set and a` to `One hundred and seven cases (92 tuning + 15 holdout) split into a **tuning** set and a`. + +- [ ] **Step 12: Make each golden query reachable, then measure** + +The corpus is what BM25 indexes, and the tokenizer keeps only `[a-z0-9]` runs (`çıkar` indexes as `kar`, `ölçekli` as `ekli`), which is why every Turkish phrase above is in its record verbatim in both spellings. Check each case's distinctive words: + +``` +uv run --frozen python -c " +from discovery.aliases import TOOL_ALIASES +from tests.data.discovery_golden import TUNING_CASES +new = {'pipe_takeoff', 'cable_takeoff', 'drawing_diff', 'drawing_scale_check', 'drawing_understand', 'drawing_topology_check'} +for c in TUNING_CASES: + if c.expect not in new: + continue + rec = TOOL_ALIASES[c.expect] + print(c.expect, '|', c.query, '->', 'phrase OK' if c.query in rec.synonyms else 'PHRASE MISSING') +" +``` + +Expected: eight `phrase OK` lines (Step 9 carries every query verbatim). Then: + +Run: `uv run --frozen pytest tests/test_discovery_transform.py tests/test_discovery_aliases.py -q` +Expected: every tuning case passes and the bars hold (`TUNING_TOP1_BAR = 0.85`, `TUNING_TOP3_BAR = 0.94`, `HOLDOUT_TOP3_BAR = 0.80`). Measured for this plan on a copy of main with six stand-in tools carrying the three group plans' alias records plus Step 9's fold: all eight new cases rank first, `entity_trim`'s "the two lines overshoot, cut them back to where they cross" still ranks first, and `tests/test_discovery_transform.py` + `tests/test_discovery_aliases.py` pass (205 passed, 1 xfailed — the known corpus gap). A case that misses against the real records is fixed with more vocabulary in its record (a descriptive phrase from the tool's docstring usually suffices), never by editing the expectation; a holdout case that slips is fixed by a new tuning case of the same class, never by editing the holdout tuple. + +- [ ] **Step 13: Reconcile the AutoCAD commands the merged records now share** + +The three branches wrote their own records, so a command may now be claimed twice — for instance `DATAEXTRACTION` (owned by `data_extract`) given to a takeoff, `AUDIT` (owned by `drawing_audit`) given to the topology check, or `COMPARE` given to a SECTION 26 record besides `drawing_diff`. Compute the real list: + +``` +uv run --frozen python -c " +from discovery.aliases import TOOL_ALIASES, SHARED_ACAD_COMMANDS +own = {} +for tool, rec in TOOL_ALIASES.items(): + for cmd in rec.acad: own.setdefault(cmd, []).append(tool) +shared = {c: sorted(n) for c, n in own.items() if len(n) > 1} +print('now shared:', {c: shared[c] for c in sorted(shared)}) +print('to add :', sorted(set(shared) - SHARED_ACAD_COMMANDS)) +print('stale :', sorted(SHARED_ACAD_COMMANDS - set(shared))) +" +``` + +Expected: `to add : []` and `stale : []` — the three group plans give every takeoff and reader `acad=()` except `drawing_diff`'s `COMPARE`, which nothing else claims. For every name in `to add`, decide: a command that genuinely means either tool goes into `SHARED_ACAD_COMMANDS` in alphabetical position with a reason line in the comment block above the set, in the shape of the existing ones — + +``` +# DATAEXTRACTION -- one wizard extracts attribute data to a table or file +# (data_extract) or, on a plant job, is what a drafter reaches for to +# count what a drawing holds (pipe_takeoff); a drafter typing it could +# mean either. +``` + +— and a command that belongs to one tool only is removed from the other record instead (`AUDIT` is AutoCAD's database repair, `drawing_audit`; a topology record claiming it loses it). A `stale` name stopped being shared and comes out of the set. + +Run: `uv run --frozen pytest tests/test_discovery_aliases.py -q` +Expected: all pass, including `test_each_acad_command_routes_to_a_single_tool` and `test_shared_command_allowlist_has_no_stale_entries`. + +- [ ] **Step 14: Regenerate the inventory and retake the frozen registry snapshot** + +``` +uv run --frozen python scripts/generate_tool_inventory.py +uv run --frozen python -c " +import asyncio, server +sizes = {k: len(v) for k, v in asyncio.run(server._tool_groups()).items()} +print(sizes); print('total', sum(sizes.values()), 'groups', len(sizes)) +" +``` + +Expected: `total 247 groups 28` — `understanding: 4`, `plant: 2`, and `analysis` back at 12 (the branches had grown it 12 -> 18 while the tags were unranked), every other group at its pre-track-H value (measured for this plan by registering six stand-ins with these tags on a copy of main: the dict below). In `tests/test_tool_registry.py::test_tool_group_sizes_are_unchanged`, change the docstring's first line to `Frozen snapshot of the surface (247 tools, 28 groups).`, delete the branch-local track-H paragraphs the merge left at the end of the docstring (they begin `Track H group U opened SECTION 26`, `Track H group D opened its copy of SECTION 26`, `` `analysis` moved 12 -> 13 when track H's SECTION 27 `` and ``It moved 13 -> 14 when `cable_takeoff` ``), and append after the paragraph ending `pre-track-F value.`: + +``` + `understanding` and `plant` appeared with v1.6's track H: `understanding` + (4, SECTION 26: `drawing_understand`, `drawing_scale_check`, + `drawing_diff`, `drawing_topology_check` - the readers of drawings + somebody else made, core pack) and `plant` (2, SECTION 27: `pipe_takeoff` + and `cable_takeoff`, exactly `PACK_TOOL_NAMES["plant"]`). The three + track-H branches tagged the six `analysis` and left the priority list + alone, so `analysis` grew 12 -> 18 on them; the integration tagged + SECTION 26 `understand` and ranked `understand` and `plant` directly + after `arch` in `_GROUP_TAG_PRIORITY`, and `analysis` returned to 12. +``` + +and replace the asserted dict and total with: + +```python + assert sizes == { + "analysis": 12, + "architecture": 12, + "batch": 3, + "blocks": 9, + "corner_ops": 4, + "dimensions": 5, + "drawing": 11, + "engineering": 10, + "entity_creation": 18, + "entity_modification": 16, + "entity_query": 9, + "environment": 22, + "layers": 14, + "layouts": 12, + "mechanical": 14, + "page_setup": 6, + "pid": 9, + "plant": 2, + "premium": 12, + "sheet": 11, + "solids": 5, + "styles": 10, + "system": 7, + "templates": 2, + "transactions": 3, + "understanding": 4, + "validation": 1, + "view": 4, + } + assert sum(sizes.values()) == 247 +``` + +If the printed dict differs anywhere but by a tool that lacks its section tag (Step 4), stop and find out why before writing it: a group that moved without a tool being added is a regression, not a snapshot. + +Run: `uv run --frozen pytest tests/test_tool_registry.py tests/test_tool_inventory.py tests/test_release_consistency.py::test_section_header_counts_match_decorators -q` +Expected: all pass. + +- [ ] **Step 15: Retake the README figures the tree derives** + +``` +uv run --frozen python scripts/check_doc_numbers.py --list +uv run --frozen pytest --collect-only -q | tail -1 +uv run --frozen python -c " +from discovery.aliases import TOOL_ALIASES +acad = {c for r in TOOL_ALIASES.values() for c in r.acad} +syn = [s for r in TOOL_ALIASES.values() for s in r.synonyms] +print(len(TOOL_ALIASES), 'records', len(acad), 'commands', len(syn), 'synonyms') +" +``` + +Update `README.md` from the printed values: + +- the release-snapshot line `> **v1.5 release snapshot:** tools · 8 resources · 5 prompt templates · collected tests.` (the merge leaves a branch's count there) — `247 tools` and the collected-tests figure `check_doc_numbers --list` printed; +- the line under it — `> 247 is the **registered** count; a default install advertises 242 over \`tools/list\`,` (247 minus the five `ENABLE_3D` solids); +- the alias sentence — `an authored corpus of ** AutoCAD command names and synonym phrases** covering all 247 tools`, with the two numbers the last command printed; +- the Discovery row of "What you get" — `each rank **#1** of the 242-tool advertised catalog`; +- the first sentence of the `` line under "What you get": + +``` +247 tools in 28 groups; TOOL_PACKS=core hides the nine pid_* tools, the 22 environment tools, the 14 mechanical tools, the 12 architectural tools and the two plant takeoffs (TOOL_PACKS=core,mech keeps the mechanical ones, TOOL_PACKS=core,arch the architectural ones, TOOL_PACKS=core,plant the takeoffs) from a client that needs none of them. +``` + + (the rest of that line — the eight resources and five prompt templates — is unchanged); +- the configuration rows `| \`TOOL_PROFILE\` | \`full\` | \`lean\` (64 curated tools, 54 with \`TOOL_PACKS=core\`) or \`full\` |` → `(65 curated tools, 55 with \`TOOL_PACKS=core\`)`, and `| \`TOOL_PACKS\` | \`all\` | Vertical packs to advertise: \`core,pid,settings\` (\`core\` always on) |` → `Vertical packs to advertise: \`core,pid,settings,mech,arch,plant\` (\`core\` always on)`. + +Then: + +Run: `uv run --frozen python scripts/check_doc_numbers.py` +Expected: exit 0. This task changes no benchmark figure — `correctness_suite.py` still holds the 43 checks the published A/B report was run over — so the only numbers that moved are the ones just rewritten. A `DRIFT` line names the file and the figure; fix that figure, never the checker. (The README token table is remeasured by Task 12.) + +- [ ] **Step 16: Gates** + +``` +uv run --frozen ruff format server.py discovery/aliases.py tests/test_tool_packs.py tests/test_tool_registry.py tests/test_token_suite.py tests/test_discovery_aliases.py tests/data/discovery_golden.py +uv run --frozen ruff check server.py discovery tests +uv run --frozen python scripts/generate_tool_inventory.py --check +uv run --frozen pytest -q +``` + +Expected: ruff clean; no inventory drift; the whole suite green, `tests/test_release_consistency.py` in full included — the collected-tests figure was retaken in Step 15, so the test every earlier task deselected passes again on main. + +- [ ] **Step 17: Commit** + +``` +git add server.py discovery/aliases.py docs/tool-inventory.json README.md tests/test_tool_packs.py tests/test_tool_registry.py tests/test_token_suite.py tests/test_discovery_aliases.py tests/data/discovery_golden.py +git -c user.name="Umut CAN" -c user.email="umutcan.edizaslan@gmail.com" commit -m "$(cat <<'MSG' +feat(understand): the plant pack, lean 65, five-language aliases, eight golden queries and the counts + +The three track-H branches are merged; this is the integration that makes the +six tools one surface. + +- TOOL_PACK_NAMES gains "plant" (pipe_takeoff, cable_takeoff); SECTION 26's + four readers stay core. Lean gains drawing_understand and reaches 65 (55 + with TOOL_PACKS=core). The `understand` and `plant` tags rank after `arch`, + so the sections file as `understanding` (4) and `plant` (2): 247 tools in + 28 groups. +- discovery/aliases.py folds spec section 12's vocabulary into the six records + the branches wrote - metraj, metre cetveli, boru / kablo listesi, fark, + karsilastir, revizyon farki, olcek, birim in both Turkish spellings, BOQ, + takeoff, scale check, COMPARE - without writing a phrase twice. The Russian + terms are data only: the BM25 tokenizer keeps [a-z0-9], and no golden case + pretends otherwise. +- Eight golden tuning cases, four of them Turkish; the shared-command + allowlist reconciled with what the merged records claim; the token-suite + formula excludes the new pack; the inventory regenerated, the registry + snapshot retaken and the README figures taken from check_doc_numbers. + +Co-Authored-By: Claude Opus 5.5 (1M context) +Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd +MSG +)" +``` + +--- + +### Task 12: Evidence — benchmark v7, correctness checks, live smoke, field-test script, docs, full gate + +**Files:** +- Create: `benchmarks/tasks_v7.py` +- Create: `tests/test_benchmark_v7.py` +- Create: `scripts/field_test_takeoff.py` +- Create: `tests/test_field_test_takeoff.py` +- Create: `scripts/smoke_understand_com.py` +- Modify: `benchmarks/adapters/autocad_mcp_pro.py` (import from `tasks_v7`; `_task_takeoff_roundtrip`, `_task_understand_foreign`) +- Modify: `benchmarks/render_matrix_chart.py`, `benchmarks/run_competitors.py` (import the v7 matrix) +- Modify: `benchmarks/correctness_suite.py` (43 → 48: `scale_check_detects_schematic`, `pipe_takeoff_rmst_exact`, `cable_takeoff_roundup`, `diff_detects_known_edits`, `topology_known_defects`) +- Modify: `tests/test_benchmark_v4.py`, `tests/test_benchmark_v5.py`, `tests/test_benchmark_v6.py` (their `len(CHECKS) == 43` pins become 48) +- Modify: `scripts/check_doc_numbers.py` (derive the task count from `TASKS_V7`) +- Modify: `benchmarks/results/published/ab-v1.5.1-vs-v1.6.0-dev.json` (regenerated over 48 checks) +- Modify: `CLAUDE.md`, `README.md`, `benchmarks/README.md`, `CHANGELOG.md` +- Modify: `docs/superpowers/specs/2026-09-24-v1.6-understand-design.md` (the status line) +- Test: `tests/test_benchmark_v7.py`, `tests/test_field_test_takeoff.py` + +**Interfaces:** +- Consumes (pinned by the skeleton): `engineering/understand/snapshot.py::read_snapshot(path) -> Snapshot`, `take_snapshot(backend, path=None) -> Snapshot`, `Snapshot.records[i].space / .type / .layer / .handle`; `describe(snap) -> dict`; `scale_check(a, b, *, min_pair_mm=2000.0) -> {"verdict", "factor", "within_10pct", ...}`; `build_network(snap, *, layers=None, tol=1.0, label_search=None) -> {"runs": [Run], ...}`; `pipe_rows(network, pid, layout, *, scale=None, services=DEFAULT_SERVICES, allowance=0.20, vertical_allowance=0.0, length_source="auto", force=False) -> dict`; `cable_rows(pid, layout, *, allowance=0.20, section_rules=None) -> dict`; `write_workbook(result, *, kind, lang, path) -> {"xlsx", "csv", "refused"}`; `diff_snapshots(old, new, *, tol=0.01, move_tol=None) -> dict`; `topology_findings(snap, *, layers, gap=None, tol=0.01) -> {"dangling", "near_miss", "crossing"}`; from Task 2 — `tests/fixtures/plant_pair.py::build_plant_pair(directory) -> dict` with the keys listed in Task 2 (`pipe_runs[i]["id" | "service" | "tags" | "status" | "layout_length_mm" | "diameter" | "diameters"]`, `cable_rows[i]["tag" | "panel" | "kw" | "length_mm" | "roundup_m"]`, `layer_services`, `outlier_handle`, `cluster_count`, `layout_true_unit`, `scale_verdict`); `benchmarks/tasks_v6.py::TASKS_V6, TaskSpec`; `benchmarks/tasks_v2..v5`; the adapter's `setup(artifact_dir)`, `_reset()`, `cleanup()`, `self.artifact_dir`; the COM backend's `drawing_open`, `drawing_save_as(path, fmt)`, `document_list()`, `document_close(name, discard=True)`, `entity_move(handle, dx, dy)`, `system_status()`. +- Consumes (result keys as the group plans pin them; Step 1 prints the merged ones): Task 3 — `describe(snap)["units"]["inferred"]` (`"mm"`) and `["units"]["agree"]`, `["warnings"]` (a list of strings; a disagreement starts `INSUNITS declares …`), `["extents"]` (its `outliers` carry the handle), `["clusters"]`, `["layers"][i]["name" | "service"]`; Task 4 — `scale_check(a, b)["verdict" | "factor" | "within_10pct"]`, `b` the reference, `factor` the distance on `a` over the distance on `b` (0.5 for `a` at half scale); Task 6 — `pipe_rows(...)["runs"][i]["id" | "tags" | "service" | "diameters" | "layout_m" | "status"]` (`diameters` maps each size to its schematic metres; `layout_m` is **metres**, `None` for status C), `["rows"]`, `["length_source"]`; Task 7 — `write_workbook(result, kind="pipe" | "cable", lang, path)`; Task 8 — `cable_rows(...)["rows"][i]["tag" | "panel" | "kw" | "length_m" | "cable_m"]` (`length_m` metres, `cable_m` the rounded-up metres); Task 9 — `diff_snapshots(...)["added" | "removed"]` (items with `handle`) and `["changed"]` (items with `old_handle` / `new_handle`); Task 10 — `topology_findings(...)["dangling"]` (items with `handle`), `["near_miss"]` and `["crossing"]` (items with `handles`). Handles are always searched in a JSON dump of a result, so an item's other fields never matter. +- Produces: `benchmarks/tasks_v7.py::NEW_TASKS_V7, TASKS_V7, MATRICES, DEFAULT_MATRIX = "v7", task_by_id(task_id, matrix=DEFAULT_MATRIX)`; `AutoCADMCPProAdapter._task_takeoff_roundtrip()`, `_task_understand_foreign()` (each `-> (passed, metrics, artifacts)`); five `CHECKS` rows (category `"Understanding"`); `scripts/field_test_takeoff.py::main(argv=None) -> int` (0 done, 2 refused); `scripts/smoke_understand_com.py::main(keep_document=False) -> int` (0 / 1 / 2). + +- [ ] **Step 1: Confirm the result shapes this task reads** + +``` +uv run --frozen python - <<'PY' +import tempfile +from tests.fixtures.plant_pair import build_plant_pair +from engineering.understand.describe import describe +from engineering.understand.diff import diff_snapshots +from engineering.understand.network import build_network +from engineering.understand.scale import scale_check +from engineering.understand.snapshot import read_snapshot +from engineering.understand.takeoff import cable_rows, pipe_rows +from engineering.understand.topology import topology_findings + +t = build_plant_pair(tempfile.mkdtemp()) +pid, layout = read_snapshot(t["pid"]), read_snapshot(t["layout"]) +report = describe(layout) +print("describe", sorted(report), "units", report["units"]) +print("describe.layers[0]", sorted(describe(pid)["layers"][0])) +scale = scale_check(pid, layout) +print("scale_check", sorted(scale)) +pipe = pipe_rows(build_network(pid), pid, layout, scale=scale) +print("pipe_rows", sorted(pipe), "runs[0]", sorted(pipe["runs"][0]), "rows[0]", sorted(pipe["rows"][0])) +cable = cable_rows(pid, layout) +print("cable_rows", sorted(cable), "rows[0]", sorted(cable["rows"][0])) +print("diff", sorted(diff_snapshots(layout, layout))) +print("topology", sorted(topology_findings(pid, layers=["P_product piping"]))) +PY +``` + +Expected (the names the group plans pin): `describe` carries `clusters`, `extents`, `layers`, `units` (with `agree`, `declared`, `inferred`) and `warnings`, and a layer row `name` and `service`; `scale_check` carries `factor`, `verdict`, `within_10pct`; `pipe_rows` carries `runs` (with `diameters`, `id`, `layout_m`, `service`, `status`, `tags`), `rows` and `length_source`; `cable_rows` carries `rows` (with `cable_m`, `kw`, `length_m`, `panel`, `tag`); `diff` carries `added`, `changed`, `removed`; `topology` carries `crossing`, `dangling`, `near_miss`. **Where the merged code names a key differently, change only the line of this task that reads it** (in the adapter, a check, the smoke or the field script) to the merged name and record it in the task report; never change a gate's number, and never rename the merged code's key to fit this plan. + +- [ ] **Step 2: Write the failing test for the v7 matrix, the two gates and the five checks** + +Create `tests/test_benchmark_v7.py`: + +```python +"""The v7 task matrix: v6 plus what track H added. + +``takeoff_roundtrip`` and ``understand_foreign`` are track H's evidence, both +gated on the synthetic plant pair (tests/fixtures/plant_pair.py) whose truth +is computed from the positions it placed. Each gate is shown to fail on a +broken input - a gate that cannot fail is not evidence. The five correctness +checks run here once too, so a regression fails the suite and not only the +A/B report. +""" + +from __future__ import annotations + +import ezdxf +import pytest + +from benchmarks.tasks_v6 import TASKS_V6 +from benchmarks.tasks_v7 import DEFAULT_MATRIX, MATRICES, NEW_TASKS_V7, TASKS_V7, task_by_id +from tests.fixtures import plant_pair + +UNDERSTANDING_CHECKS = ( + "scale_check_detects_schematic", + "pipe_takeoff_rmst_exact", + "cable_takeoff_roundup", + "diff_detects_known_edits", + "topology_known_defects", +) + + +def test_v7_extends_v6_without_touching_it(): + assert TASKS_V7[: len(TASKS_V6)] == TASKS_V6 + assert [t.task_id for t in NEW_TASKS_V7] == ["takeoff_roundtrip", "understand_foreign"] + assert len(TASKS_V7) == 21 + assert MATRICES["v6"] == TASKS_V6 and MATRICES["v7"] == TASKS_V7 + assert DEFAULT_MATRIX == "v7" + assert task_by_id("takeoff_roundtrip").category == "plant" + assert task_by_id("understand_foreign").category == "understand" + + +def test_the_adapter_implements_every_v7_task(): + from benchmarks.adapters.autocad_mcp_pro import AutoCADMCPProAdapter + + for task in TASKS_V7: + assert hasattr(AutoCADMCPProAdapter, f"_task_{task.task_id}"), task.task_id + + +def test_the_suite_carries_the_five_understanding_checks(): + from benchmarks.correctness_suite import CHECKS + + for name in UNDERSTANDING_CHECKS: + assert CHECKS[name][1] == "Understanding" + assert len(CHECKS) == 48 + + +async def _run(task_id, tmp_path): + from benchmarks.adapters.autocad_mcp_pro import AutoCADMCPProAdapter + + adapter = AutoCADMCPProAdapter(backend="ezdxf") + await adapter.setup(tmp_path) + try: + await adapter._reset() + return await getattr(adapter, f"_task_{task_id}")() + finally: + await adapter.cleanup() + + +def _broken(edit): + """build_plant_pair, then ``edit(truth)`` on the files it wrote.""" + real = plant_pair.build_plant_pair + + def build(directory): + truth = real(directory) + edit(truth) + return truth + + return build + + +@pytest.mark.asyncio +async def test_takeoff_roundtrip_passes_headlessly(tmp_path): + passed, metrics, _artifacts = await _run("takeoff_roundtrip", tmp_path) + assert passed, metrics + assert metrics["scale_verdict"] == "schematic" + assert metrics["runs_checked"] == 9 and metrics["loads_checked"] == 4 + + +@pytest.mark.asyncio +async def test_takeoff_roundtrip_fails_loudly_without_the_wiring_callouts(tmp_path, monkeypatch): + """Every callout gone from the layout: no load has a panel any more, so no + cable length can be measured - and the gate names all four loads, while the + pipe half, untouched, still agrees.""" + + def drop_callouts(truth): + doc = ezdxf.readfile(truth["layout"]) + msp = doc.modelspace() + for entity in list(msp.query("MULTILEADER")): + msp.delete_entity(entity) + doc.saveas(truth["layout"]) + + monkeypatch.setattr(plant_pair, "build_plant_pair", _broken(drop_callouts)) + passed, metrics, _artifacts = await _run("takeoff_roundtrip", tmp_path) + assert passed is False + assert sorted(row["tag"] for row in metrics["cable_mismatch"]) == ["M11", "M12", "T102", "T202"] + assert metrics["pipe_mismatch"] == [] + + +@pytest.mark.asyncio +async def test_understand_foreign_passes_headlessly(tmp_path): + passed, metrics, _artifacts = await _run("understand_foreign", tmp_path) + assert passed, metrics + assert metrics["clusters"] == 2 and metrics["misread_layers"] == {} + + +@pytest.mark.asyncio +async def test_understand_foreign_fails_loudly_when_the_units_tell_the_truth(tmp_path, monkeypatch): + """The same layout declaring millimetres: there is no disagreement left to + flag, so the units gate fails - and only that gate.""" + + def honest_units(truth): + doc = ezdxf.readfile(truth["layout"]) + doc.header["$INSUNITS"] = 4 + doc.saveas(truth["layout"]) + + monkeypatch.setattr(plant_pair, "build_plant_pair", _broken(honest_units)) + passed, metrics, _artifacts = await _run("understand_foreign", tmp_path) + assert passed is False + assert metrics["units_flagged"] is False + assert metrics["outlier_found"] and metrics["copies_separated"] + + +@pytest.mark.asyncio +@pytest.mark.parametrize("name", UNDERSTANDING_CHECKS) +async def test_each_understanding_check_passes(name): + from benchmarks.correctness_suite import CHECKS + + check, _category = CHECKS[name] + assert await check() is True +``` + +- [ ] **Step 3: Run it to verify it fails** + +Run: `uv run --frozen pytest tests/test_benchmark_v7.py -q` +Expected: FAIL at collection — `ModuleNotFoundError: No module named 'benchmarks.tasks_v7'`. + +- [ ] **Step 4: Create `benchmarks/tasks_v7.py` and point every matrix consumer at it** + +`benchmarks/tasks_v7.py`: + +```python +"""The v7 task matrix: v6 plus what track H added. + +Track H teaches the server to read drawings somebody else made, so its two +tasks are gated on a drawing pair whose answers are known in advance - the +synthetic plant pair of ``tests/fixtures/plant_pair.py``, which reproduces +every field finding of the design spec with invented names: + +* ``takeoff_roundtrip`` - the scale check calls the P&ID ``schematic``; every + routable pipe run's layout length equals the rectilinear minimum spanning + tree of its tags exactly; its size, service and status match the generator's + truth; the cable rows equal the Manhattan lengths from each load to its panel + with the allowance rounded up to the metre. +* ``understand_foreign`` - one ``describe`` of the layout flags inches declared + over millimetre geometry, finds the stray entity by handle and separates the + two plan copies; one ``describe`` of the P&ID classifies its English and + Dutch service layers. + +The published competitor reports were never asked these questions; the chart +shows ``not_run`` for them, not zero. +""" + +from __future__ import annotations + +from benchmarks.tasks_v2 import TASKS_V2 +from benchmarks.tasks_v3 import TASKS_V3 +from benchmarks.tasks_v4 import TASKS_V4 +from benchmarks.tasks_v5 import TASKS_V5 +from benchmarks.tasks_v6 import TASKS_V6, TaskSpec + +#: Added in v1.6.0 by track H. +NEW_TASKS_V7: tuple[TaskSpec, ...] = ( + TaskSpec( + "takeoff_roundtrip", + "plant", + "Take pipe and cable quantities off a schematic P&ID and its layout: the P&ID " + "called schematic, every run's layout length equal to the rectilinear MST of " + "its tags, sizes and services as drawn, cable metres rounded up after the " + "allowance", + 1.0, + ), + TaskSpec( + "understand_foreign", + "understand", + "Read a foreign layout in one call: inches declared over millimetres flagged, " + "the stray entity found, both plan copies separated, the service layers of its " + "P&ID classified", + 1.0, + ), +) + +TASKS_V7: tuple[TaskSpec, ...] = TASKS_V6 + NEW_TASKS_V7 + +#: Runner-selectable task sets. v2 to v6 stay addressable so an earlier report +#: can be reproduced exactly rather than only described. +MATRICES: dict[str, tuple[TaskSpec, ...]] = { + "v2": TASKS_V2, + "v3": TASKS_V3, + "v4": TASKS_V4, + "v5": TASKS_V5, + "v6": TASKS_V6, + "v7": TASKS_V7, +} + +DEFAULT_MATRIX = "v7" + + +def task_by_id(task_id: str, matrix: str = DEFAULT_MATRIX) -> TaskSpec: + for task in MATRICES[matrix]: + if task.task_id == task_id: + return task + raise KeyError(f"{task_id} is not in matrix {matrix}") +``` + +In `benchmarks/adapters/autocad_mcp_pro.py`, change `from benchmarks.tasks_v6 import TaskSpec` to `from benchmarks.tasks_v7 import TaskSpec` and the module docstring's first line to `"""Reference adapter for this repository, covering the v7 task matrix.`. In `benchmarks/render_matrix_chart.py`, change `from benchmarks.tasks_v6 import TASKS_V6` to `from benchmarks.tasks_v7 import TASKS_V7` and both `TASKS_V6` uses in its body to `TASKS_V7`. In `benchmarks/run_competitors.py`, change `from benchmarks.tasks_v6 import DEFAULT_MATRIX, MATRICES, TaskSpec, task_by_id` to `from benchmarks.tasks_v7 import DEFAULT_MATRIX, MATRICES, TaskSpec, task_by_id`. Then confirm nothing else takes v6 as the current matrix: + +``` +git grep -l "tasks_v6" -- "*.py" +``` + +Expected hits only: `benchmarks/tasks_v7.py` (deliberate — v7 extends v6), `tests/test_benchmark_v6.py`, `tests/test_benchmark_v7.py` and `scripts/check_doc_numbers.py` (Step 16 repoints it). The published task-matrix SVG is not re-rendered: the competitor reports it draws were never asked the v3-v7 tasks. + +- [ ] **Step 5: Run it to verify the matrix half passes and the rest fails for the right reasons** + +Run: `uv run --frozen pytest tests/test_benchmark_v7.py -q` +Expected: `test_v7_extends_v6_without_touching_it` passes; `test_the_adapter_implements_every_v7_task` FAILS with `AssertionError: takeoff_roundtrip`; `test_the_suite_carries_the_five_understanding_checks` FAILS with `KeyError: 'scale_check_detects_schematic'`; the four gate tests FAIL with `AttributeError: 'AutoCADMCPProAdapter' object has no attribute '_task_takeoff_roundtrip'` (or `_task_understand_foreign`); the five parametrized check tests FAIL with `KeyError`. + +- [ ] **Step 6: Add the two task methods to the adapter** + +In `benchmarks/adapters/autocad_mcp_pro.py`, insert into `AutoCADMCPProAdapter`, directly before `async def _task_auditable_delivery(self):`: + +```python + async def _task_takeoff_roundtrip(self): + """Pipe and cable quantities off the synthetic plant pair, against its truth. + + Track H's evidence (spec §15). The pair is written fresh into the + artifact folder by ``tests/fixtures/plant_pair.py``, whose truth is + computed from the positions it placed. Then only the two DXFs are read: + + * ``scale_check(pid, layout)`` must say ``schematic`` - room 1 is the + layout stretched 1.3x, room 2 is rearranged, and 21.8 % of the 55 tag + pairs lie within ±10 % of the median ratio; + * ``pipe_rows(..., length_source="auto")`` must report every run of the + P&ID (the ice-water utility included, because ``services`` names it) + with the truth's status; a routable run's layout length must equal the + rectilinear MST of its tags exactly, its sizes (both across the + reducer) and its service must be the drawn ones (the CIP return drawn + on the supply layer included); the run ending on T105, which the + layout lacks, is status C with no layout length; + * ``cable_rows(pid, layout)`` must give every load its panel from the + layout's wiring callouts, its stated power (the heater's stays empty), + the Manhattan length to the panel and the metres rounded up after the + 20 % allowance. + + Any one slipping fails the task, and the metrics name what slipped. + """ + from engineering.understand.network import build_network + from engineering.understand.scale import scale_check + from engineering.understand.snapshot import read_snapshot + from engineering.understand.takeoff import cable_rows, pipe_rows + from tests.fixtures.plant_pair import build_plant_pair + + truth = build_plant_pair(self.artifact_dir / "plant_pair") + pid = read_snapshot(truth["pid"]) + layout = read_snapshot(truth["layout"]) + scale = scale_check(pid, layout) + services = tuple(sorted({run["service"] for run in truth["pipe_runs"]})) + pipe = pipe_rows( + build_network(pid), pid, layout, scale=scale, services=services, length_source="auto" + ) + found = {(run["service"], tuple(sorted(run["tags"]))): run for run in pipe["runs"]} + pipe_mismatch = [] + for run in truth["pipe_runs"]: + got = found.get((run["service"], tuple(run["tags"]))) + if got is None: + pipe_mismatch.append({"run": run["id"], "reason": "no run with these tags"}) + continue + reasons = [] + if got["status"] != run["status"]: + reasons.append(f"status {got['status']} != {run['status']}") + want, have = run["layout_length_mm"], got.get("layout_m") + if want is None and have is not None: + reasons.append(f"a layout length ({have} m) for an unroutable run") + elif want is not None and (have is None or abs(float(have) * 1000.0 - want) > 1e-6): + reasons.append(f"layout length {have} m != {want} mm") + if set(got["diameters"]) != set(run["diameters"]): + reasons.append(f"sizes {sorted(got['diameters'])} != {sorted(run['diameters'])}") + if reasons: + pipe_mismatch.append({"run": run["id"], "reason": "; ".join(reasons)}) + + cable = {row["tag"]: row for row in cable_rows(pid, layout)["rows"]} + cable_mismatch = [] + for want in truth["cable_rows"]: + got = cable.get(want["tag"]) + if got is None: + cable_mismatch.append({"tag": want["tag"], "wrong": ["missing"]}) + continue + wrong = [key for key in ("panel", "kw") if got.get(key) != want[key]] + if got.get("cable_m") != want["roundup_m"]: + wrong.append("cable_m") + length = got.get("length_m") + if length is None or abs(float(length) * 1000.0 - want["length_mm"]) > 1e-6: + wrong.append("length_m") + if wrong: + cable_mismatch.append({"tag": want["tag"], "wrong": wrong}) + + passed = scale["verdict"] == truth["scale_verdict"] and not pipe_mismatch + passed = passed and not cable_mismatch + return ( + passed, + { + "scale_verdict": scale["verdict"], + "within_10pct": scale.get("within_10pct"), + "length_source": pipe.get("length_source"), + "runs_checked": len(truth["pipe_runs"]), + "pipe_mismatch": pipe_mismatch, + "loads_checked": len(truth["cable_rows"]), + "cable_mismatch": cable_mismatch, + }, + [truth["pid"], truth["layout"]], + ) + + async def _task_understand_foreign(self): + """One call on a foreign layout reports what is wrong with it, by evidence. + + Track H's evidence (spec §5, §15), on the synthetic plant pair: the + layout declares inches (``$INSUNITS = 1``) over millimetre geometry, a + stray LINE 5,000 km away stretches its declared extents, and the plan is + drawn twice, 100 m apart. ``describe`` of the layout must infer + millimetres, report that they disagree with the declared unit and warn + about ``INSUNITS``, name the stray entity's handle in its extents, and + report exactly two clusters; ``describe`` of the P&ID must classify its + four service layers and the electrical layer as the generator names them + (English and Dutch names). A warning or an extents report is searched as + JSON, so the gate holds whatever shape the evidence takes. + """ + import json + + from engineering.understand.describe import describe + from engineering.understand.snapshot import read_snapshot + from tests.fixtures.plant_pair import build_plant_pair + + truth = build_plant_pair(self.artifact_dir / "plant_pair") + layout = describe(read_snapshot(truth["layout"])) + pid = describe(read_snapshot(truth["pid"])) + units = layout["units"] + units_flagged = ( + str(units.get("inferred")) == truth["layout_true_unit"] + and units.get("agree") is False + and "INSUNITS" in json.dumps(layout["warnings"], ensure_ascii=False) + ) + outlier_found = f'"{truth["outlier_handle"]}"' in json.dumps(layout["extents"]) + copies_separated = len(layout["clusters"]) == truth["cluster_count"] + read = {row["name"]: row["service"] for row in pid["layers"]} + misread = { + name: read.get(name) + for name, service in truth["layer_services"].items() + if read.get(name) != service + } + passed = units_flagged and outlier_found and copies_separated and not misread + return ( + passed, + { + "units": units, + "units_flagged": units_flagged, + "outlier_found": outlier_found, + "clusters": len(layout["clusters"]), + "copies_separated": copies_separated, + "misread_layers": misread, + }, + [truth["layout"], truth["pid"]], + ) +``` + +Both read the synthetic pair and nothing else; the pair is written into the task's own artifact folder, so a run leaves its evidence beside its report. + +- [ ] **Step 7: Run the matrix half** + +Run: `uv run --frozen pytest tests/test_benchmark_v7.py -q -k "extends or adapter or roundtrip or foreign"` +Expected: all seven pass — including `test_takeoff_roundtrip_fails_loudly_without_the_wiring_callouts` (the cable half names M11, M12, T102, T202 while `pipe_mismatch` stays `[]`) and `test_understand_foreign_fails_loudly_when_the_units_tell_the_truth` (`units_flagged` false, the outlier and the copies still found). If a passing gate fails, its metrics name what slipped — `pipe_mismatch` (run id and reason), `cable_mismatch` (tag and the wrong fields), `misread_layers`, `units`: fix the *track H code*, not the gate. The numbers are the criterion. + +- [ ] **Step 8: Add the five correctness checks** + +In `benchmarks/correctness_suite.py`, insert directly before `CHECKS = {`: + +```python +# ── Understanding foreign drawings (track H) ─────────────────────────────── + + +def _plant_pair() -> dict: + """The synthetic plant pair, written to a fresh temporary folder.""" + from tests.fixtures.plant_pair import build_plant_pair + + return build_plant_pair(tempfile.mkdtemp(prefix="acadmcp_plant_")) + + +def _quoted(handle: str) -> str: + """A handle as it appears in a JSON dump - quoted, so '2A' never matches '12A3'.""" + return f'"{handle}"' + + +async def scale_check_detects_schematic(): + """The synthetic P&ID is schematic against its layout; a uniform 2x copy is not. + + The pair: room 1 of the P&ID is the layout stretched 1.3x, room 2 is + rearranged, and of the 55 pairs of the 11 shared tags 21.8 % lie within + ±10 % of the median ratio - at most 50 % is ``schematic`` (spec §6). + The control: six tags at (0, 0), (4000, 0), (8000, 0), (0, 3000), + (4000, 3000), (8000, 6000) and the same tags at twice those coordinates - + the nearest pair is 3000 mm apart, every ratio is exactly 2.0, so the + doubled drawing is ``to_scale`` against the first with factor 2. + """ + import ezdxf + + from engineering.understand.scale import scale_check + from engineering.understand.snapshot import read_snapshot + + truth = _plant_pair() + pair = scale_check(read_snapshot(truth["pid"]), read_snapshot(truth["layout"])) + folder = tempfile.mkdtemp(prefix="acadmcp_scale_") + positions = [(0, 0), (4000, 0), (8000, 0), (0, 3000), (4000, 3000), (8000, 6000)] + paths = [] + for factor in (1.0, 2.0): + doc = ezdxf.new("R2018") + msp = doc.modelspace() + for number, (x, y) in enumerate(positions, start=1): + msp.add_circle((x * factor, y * factor), 500.0) + msp.add_text(f"T{number}", dxfattribs={"insert": (x * factor, y * factor)}) + path = os.path.join(folder, f"x{factor:g}.dxf") + doc.saveas(path) + paths.append(path) + control = scale_check(read_snapshot(paths[1]), read_snapshot(paths[0])) + return ( + pair["verdict"] == "schematic" + and control["verdict"] == "to_scale" + and abs(float(control["factor"]) - 2.0) < 1e-9 + ) + + +async def pipe_takeoff_rmst_exact(): + """Every routable pipe run's layout length is the rectilinear MST of its tags. + + Measured on the synthetic pair with ``length_source="layout"``. The truth + is the generator's own Manhattan / Prim arithmetic over the tag positions it + placed, hand-checked in tests/test_plant_pair.py - PR1 5500 (5000 + 500), + PR2 13200 (the three distances 6000 / 7200 / 13200, spanning tree 6000 + + 7200), CS1 20700 (20500 + 200). All eight routable runs must agree to a + micron; the run ending on T105, which the layout lacks, is not routable. + """ + from engineering.understand.network import build_network + from engineering.understand.snapshot import read_snapshot + from engineering.understand.takeoff import pipe_rows + + truth = _plant_pair() + pid, layout = read_snapshot(truth["pid"]), read_snapshot(truth["layout"]) + services = tuple(sorted({run["service"] for run in truth["pipe_runs"]})) + result = pipe_rows(build_network(pid), pid, layout, services=services, length_source="layout") + found = {(r["service"], tuple(sorted(r["tags"]))): r for r in result["runs"]} + routable = [run for run in truth["pipe_runs"] if run["layout_length_mm"] is not None] + for run in routable: + got = found.get((run["service"], tuple(run["tags"]))) + if got is None or got.get("layout_m") is None: + return False + if abs(float(got["layout_m"]) * 1000.0 - run["layout_length_mm"]) > 1e-6: + return False + return len(routable) == 8 + + +async def cable_takeoff_roundup(): + """Cable rows: the panel from the callout, Manhattan length, metres rounded up. + + The truth rows of the synthetic pair, by hand: M11 (5500, 8200) to CP-1 + (1200, 10800) is 6900 mm, 6.9 m x 1.2 = 8.28 -> 9; T102 (8000, 3500) to CP-1 + is 14100 mm -> 16.92 -> 17; M12 (26300, 8400) to CP-2 (35000, 1300) is + 15800 mm -> 18.96 -> 19; T202 (28700, 3000) to CP-2 is 8000 mm -> 9.6 -> 10. + The heater on T202 states no power, and its row must keep an empty one. + """ + from engineering.understand.snapshot import read_snapshot + from engineering.understand.takeoff import cable_rows + + truth = _plant_pair() + rows = { + row["tag"]: row + for row in cable_rows(read_snapshot(truth["pid"]), read_snapshot(truth["layout"]))["rows"] + } + expected = {"M11": ("CP-1", 9), "T102": ("CP-1", 17), "M12": ("CP-2", 19), "T202": ("CP-2", 10)} + for tag, (panel, metres) in expected.items(): + row = rows.get(tag) + if row is None or row.get("panel") != panel or row.get("cable_m") != metres: + return False + return rows["T202"].get("kw") is None and rows["M11"].get("kw") == 5.5 + + +async def diff_detects_known_edits(): + """Two revisions differing by exactly five edits - and the diff says so. + + A move (a LINE shifted 10 mm), a text change (PUMP A -> PUMP B), an + attribute change (TAG T101 -> T102), an add (a new LINE) and a delete (a + LINE removed); a CIRCLE stays untouched. The edits are made on a copy read + back from the first file, so every surviving entity keeps its handle. The + diff must list the added line as added, the removed line as removed, the + three edited entities as changed - and nothing else, the circle included. + """ + import json + + import ezdxf + + from engineering.understand.diff import diff_snapshots + from engineering.understand.snapshot import read_snapshot + + folder = tempfile.mkdtemp(prefix="acadmcp_diff_") + old_path, new_path = os.path.join(folder, "rev_a.dxf"), os.path.join(folder, "rev_b.dxf") + doc = ezdxf.new("R2018") + msp = doc.modelspace() + moved = msp.add_line((0, 0), (100, 0), dxfattribs={"layer": "PIPE"}) + text = msp.add_text("PUMP A", dxfattribs={"insert": (0, 50), "height": 2.5}) + block = doc.blocks.new("TAGGED") + block.add_circle((0, 0), 10) + block.add_attdef("TAG", (0, 0), dxfattribs={"height": 2.5}) + insert = msp.add_blockref("TAGGED", (200, 0)) + insert.add_auto_attribs({"TAG": "T101"}) + gone = msp.add_line((0, 100), (100, 100), dxfattribs={"layer": "PIPE"}) + kept = msp.add_circle((300, 300), 20) + doc.saveas(old_path) + + revision = ezdxf.readfile(old_path) + revision.entitydb[moved.dxf.handle].translate(10, 0, 0) + revision.entitydb[text.dxf.handle].dxf.text = "PUMP B" + revision.entitydb[insert.dxf.handle].get_attrib("TAG").dxf.text = "T102" + revision.modelspace().delete_entity(revision.entitydb[gone.dxf.handle]) + added = revision.modelspace().add_line((500, 500), (600, 500), dxfattribs={"layer": "PIPE"}) + revision.saveas(new_path) + + diff = diff_snapshots(read_snapshot(old_path), read_snapshot(new_path)) + changed = json.dumps(diff["changed"]) + dumped = changed + json.dumps(diff["added"]) + json.dumps(diff["removed"]) + return ( + len(diff["added"]) == 1 + and len(diff["removed"]) == 1 + and len(diff["changed"]) == 3 + and _quoted(added.dxf.handle) in json.dumps(diff["added"]) + and _quoted(gone.dxf.handle) in json.dumps(diff["removed"]) + and all(_quoted(e.dxf.handle) in changed for e in (moved, text, insert)) + and _quoted(kept.dxf.handle) not in dumped + ) + + +async def topology_known_defects(): + """Dangling ends, a near miss and a crossing, placed on purpose - and nothing else. + + On layer PIPE: a closed 1000 x 1000 square (no free end, no defect); a LINE + from (500, -500) that stops at (500, -5), 5 mm short of the square's bottom + edge (a near miss at gap 10, and a dangling end); two LINEs crossing at + (2500, 500), away from all four of their ends (one interior crossing); and a + clean T - a LINE ending on the middle of another at (4500, 0), which is a + junction, not a crossing and not a near miss. + """ + import json + + import ezdxf + + from engineering.understand.snapshot import read_snapshot + from engineering.understand.topology import topology_findings + + doc = ezdxf.new("R2018") + msp = doc.modelspace() + pipe = {"layer": "PIPE"} + ring = msp.add_lwpolyline( + [(0, 0), (1000, 0), (1000, 1000), (0, 1000)], close=True, dxfattribs=pipe + ) + short = msp.add_line((500, -500), (500, -5), dxfattribs=pipe) + cross_a = msp.add_line((2000, 0), (3000, 1000), dxfattribs=pipe) + cross_b = msp.add_line((2000, 1000), (3000, 0), dxfattribs=pipe) + bar = msp.add_line((4000, 0), (5000, 0), dxfattribs=pipe) + stem = msp.add_line((4500, 0), (4500, 500), dxfattribs=pipe) + path = os.path.join(tempfile.mkdtemp(prefix="acadmcp_topo_"), "topology.dxf") + doc.saveas(path) + + found = topology_findings(read_snapshot(path), layers=["PIPE"], gap=10.0) + near = json.dumps(found["near_miss"]) + crossing = json.dumps(found["crossing"]) + dangling = json.dumps(found["dangling"]) + return ( + _quoted(short.dxf.handle) in near + and _quoted(short.dxf.handle) in dangling + and _quoted(ring.dxf.handle) not in dangling + and _quoted(ring.dxf.handle) not in crossing + and len(found["crossing"]) == 1 + and _quoted(cross_a.dxf.handle) in crossing + and _quoted(cross_b.dxf.handle) in crossing + and all(_quoted(e.dxf.handle) not in crossing + near for e in (bar, stem)) + ) +``` + +and register them at the end of `CHECKS`, after `"arch_rooms_detect_foreign": (arch_rooms_detect_foreign, "Architecture"),`: + +```python + "scale_check_detects_schematic": (scale_check_detects_schematic, "Understanding"), + "pipe_takeoff_rmst_exact": (pipe_takeoff_rmst_exact, "Understanding"), + "cable_takeoff_roundup": (cable_takeoff_roundup, "Understanding"), + "diff_detects_known_edits": (diff_detects_known_edits, "Understanding"), + "topology_known_defects": (topology_known_defects, "Understanding"), +``` + +`os` and `tempfile` are already imported at the top of the module; `ezdxf` and `json` are imported inside the checks that use them, so the suite stays importable on a baseline without them. On `v1.5.1` every one of the five raises `ImportError` (no `engineering.understand`, no `tests.fixtures`) — a `miss`, exactly as the A/B runner counts it. + +- [ ] **Step 9: Move the three older suite-size pins, then run the checks one at a time** + +In `tests/test_benchmark_v4.py`, `tests/test_benchmark_v5.py` and `tests/test_benchmark_v6.py`, change `assert len(CHECKS) == 43` to `assert len(CHECKS) == 48`. + +``` +uv run --frozen python benchmarks/correctness_suite.py --list +uv run --frozen python benchmarks/correctness_suite.py scale_check_detects_schematic +uv run --frozen python benchmarks/correctness_suite.py pipe_takeoff_rmst_exact +uv run --frozen python benchmarks/correctness_suite.py cable_takeoff_roundup +uv run --frozen python benchmarks/correctness_suite.py diff_detects_known_edits +uv run --frozen python benchmarks/correctness_suite.py topology_known_defects +uv run --frozen pytest tests/test_benchmark_v4.py tests/test_benchmark_v5.py tests/test_benchmark_v6.py tests/test_benchmark_v7.py -q +``` + +Expected: 48 rows listed, five `PASS` lines, the four test files green. A `FAIL` is investigated by running that one check alone and fixing the code it exercises; the hand-computed numbers in the docstrings are the one thing never edited to fit. `scale_check_detects_schematic`'s control passes the doubled drawing first: Task 4's `factor` is the distance on `a` over the distance on its reference `b`, so 2.0. + +- [ ] **Step 10: Write the failing test for the field-test script** + +Create `tests/test_field_test_takeoff.py`: + +```python +"""scripts/field_test_takeoff.py never writes into the repository and never prints a quantity. + +The script exists for client drawings, which never enter this repository +(spec §14). It is exercised here on the synthetic plant pair only - the field +test itself is never run in CI. +""" + +from __future__ import annotations + +import importlib.util +import json +from pathlib import Path + +from tests.fixtures.plant_pair import build_plant_pair + +ROOT = Path(__file__).resolve().parents[1] + + +def _script(): + spec = importlib.util.spec_from_file_location( + "field_test_takeoff", ROOT / "scripts" / "field_test_takeoff.py" + ) + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + +def test_an_output_folder_inside_the_repository_is_refused_before_anything_runs(tmp_path, capsys): + truth = build_plant_pair(tmp_path) + target = ROOT / "field_test_output_must_not_exist" + code = _script().main( + ["--pid", truth["pid"], "--layout", truth["layout"], "--out", str(target)] + ) + assert code == 2 + assert not target.exists() + assert "refused" in capsys.readouterr().out + + +def test_a_missing_drawing_is_refused(tmp_path, capsys): + code = _script().main( + ["--pid", str(tmp_path / "none.dxf"), "--layout", str(tmp_path / "none.dxf")] + ) + assert code == 2 + assert "does not exist" in capsys.readouterr().out + + +def test_the_report_lands_next_to_the_pid_and_the_console_carries_no_quantity(tmp_path, capsys): + truth = build_plant_pair(tmp_path) + code = _script().main(["--pid", truth["pid"], "--layout", truth["layout"]]) + assert code == 0 + summary = json.loads(capsys.readouterr().out) + assert summary["scale_verdict"] == "schematic" + assert Path(summary["report"]).parent == tmp_path + assert all(Path(path).parent == tmp_path for path in summary["workbooks"]) + printed = json.dumps({k: v for k, v in summary.items() if k not in ("report", "workbooks")}) + for run in truth["pipe_runs"]: + if run["layout_length_mm"] is not None: + assert f"{run['layout_length_mm']:g}" not in printed, run["id"] + for row in truth["cable_rows"]: + assert f"{row['length_mm']:g}" not in printed, row["tag"] + report = json.loads(Path(summary["report"]).read_text(encoding="utf-8")) + assert report["scale"]["verdict"] == "schematic" + + +def test_show_prints_the_rows_on_request(tmp_path, capsys): + truth = build_plant_pair(tmp_path) + code = _script().main(["--pid", truth["pid"], "--layout", truth["layout"], "--show"]) + assert code == 0 + assert '"cable"' in capsys.readouterr().out +``` + +- [ ] **Step 11: Run it to verify it fails** + +Run: `uv run --frozen pytest tests/test_field_test_takeoff.py -q` +Expected: FAIL — `FileNotFoundError` from `spec.loader.exec_module` for `scripts/field_test_takeoff.py`, in all four tests. + +- [ ] **Step 12: Write `scripts/field_test_takeoff.py`** + +```python +"""Run both takeoffs on a real P&ID and layout, and write the report next to them. + +LOCAL ONLY. This is the field test of track H (spec §15): the user points it at +a client's P&ID and layout, compares what comes out with the hand-made +workbooks, and keeps the result on their own disk. Nothing it reads or writes +may enter this repository (spec §14), so: + +* the report is written next to the P&ID (or into ``--out``) and **the script + refuses any output folder inside this repository**, before reading a byte; +* the console gets file names, row counts and the scale verdict - **never a + quantity**, unless ``--show`` is given on the user's own terminal; +* it is never run in CI and never by a test on a real drawing; the test next + to it (tests/test_field_test_takeoff.py) runs it on the synthetic plant pair. + +What it writes: ``pipe_takeoff.xlsx`` / ``cable_takeoff.xlsx`` (CSV beside +them always, XLSX when the ``office`` extra is installed) through the same +``write_workbook`` the tools use, and ``field_test_report.json`` with the scale +check and both results in full. + + uv run --frozen python scripts/field_test_takeoff.py --pid D:/job/pid.dxf --layout D:/job/layout.dxf + uv run --frozen python scripts/field_test_takeoff.py --pid ... --layout ... --lang en --show +""" + +from __future__ import annotations + +import argparse +import json +import sys +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(ROOT)) + +#: Exit code for a refused run (output inside the repository, a missing file). +REFUSED = 2 + + +def _inside_repository(folder: Path) -> bool: + resolved = folder.resolve() + return resolved == ROOT or ROOT in resolved.parents + + +def _parse(argv) -> argparse.Namespace: + parser = argparse.ArgumentParser( + description="Pipe and cable takeoffs on a real P&ID + layout (local only)." + ) + parser.add_argument("--pid", required=True, help="the P&ID, a .dxf") + parser.add_argument("--layout", required=True, help="the layout, a .dxf") + parser.add_argument("--out", default=None, help="report folder (default: the P&ID's folder)") + parser.add_argument("--lang", default="tr", choices=("tr", "en", "ru")) + parser.add_argument("--allowance", type=float, default=0.20) + parser.add_argument( + "--show", action="store_true", help="also print the totals (quantities) to the console" + ) + return parser.parse_args(argv) + + +def main(argv=None) -> int: + args = _parse(argv) + pid_path, layout_path = Path(args.pid), Path(args.layout) + out = Path(args.out) if args.out else pid_path.resolve().parent + if _inside_repository(out): + print( + f"refused: {out} is inside the repository ({ROOT}); client drawings and " + "anything derived from them never enter it. Pass --out outside it." + ) + return REFUSED + for path in (pid_path, layout_path): + if not path.is_file(): + print(f"refused: {path} does not exist") + return REFUSED + + from engineering.understand.network import build_network + from engineering.understand.report import write_workbook + from engineering.understand.scale import scale_check + from engineering.understand.snapshot import read_snapshot + from engineering.understand.takeoff import cable_rows, pipe_rows + + out.mkdir(parents=True, exist_ok=True) + pid = read_snapshot(str(pid_path)) + layout = read_snapshot(str(layout_path)) + scale = scale_check(pid, layout) + pipe = pipe_rows(build_network(pid), pid, layout, scale=scale, allowance=args.allowance) + cable = cable_rows(pid, layout, allowance=args.allowance) + written = [ + write_workbook(pipe, kind="pipe", lang=args.lang, path=str(out / "pipe_takeoff.xlsx")), + write_workbook(cable, kind="cable", lang=args.lang, path=str(out / "cable_takeoff.xlsx")), + ] + report = out / "field_test_report.json" + report.write_text( + json.dumps( + {"scale": scale, "pipe": pipe, "cable": cable, "written": written}, + indent=2, + ensure_ascii=False, + default=str, + ), + encoding="utf-8", + ) + print( + json.dumps( + { + "scale_verdict": scale["verdict"], + "pipe_rows": len(pipe["rows"]), + "cable_rows": len(cable["rows"]), + "report": str(report), + "workbooks": [ + path + for item in written + for path in ([item["xlsx"]] if item["xlsx"] else []) + list(item["csv"]) + ], + }, + indent=2, + ) + ) + if args.show: + print( + json.dumps( + {"pipe": pipe["rows"], "cable": cable["rows"]}, + indent=2, + ensure_ascii=False, + default=str, + ) + ) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) +``` + +- [ ] **Step 13: Run it to verify it passes** + +Run: `uv run --frozen pytest tests/test_field_test_takeoff.py -q` +Expected: `4 passed`. The script is never pointed at a real drawing by a test or by CI; the user runs it on their own pair, and its output stays next to their files. + +- [ ] **Step 14: Write `scripts/smoke_understand_com.py`** + +```python +"""Exercise track H once against the live AutoCAD on this machine. + +Builds the synthetic plant pair (tests/fixtures/plant_pair.py) in a temporary +folder and reads it through the live engine - never an operator's drawing: + + open pid.dxf -> take_snapshot(backend) (Document.Export, "DXF") + -> drawing_save_as(pid.dwg), close it + -> take_snapshot(backend, pid.dwg) (opened read-only, exported, closed) + open layout.dxf -> take_snapshot(backend) + pipe_rows / cable_rows on the two live snapshots, against the pair's truth + entity_move one tag text by 10 mm -> take_snapshot -> diff_snapshots + against the snapshot taken before the move + +and prints what came back as JSON. Exit codes: + + 0 every live snapshot holds the same model-space entities (by type) as the + file it came from, the takeoffs match the truth, and the diff reports + exactly the one moved entity + 1 it ran, but something disagreed (the JSON says what) + 2 no live CAD application reachable (pywin32 missing, or the ProgID + cannot be created) -- the COM paths then remain fake-tested only + +Every document this run opens is closed on the way out, also when a step +fails, unless ``--keep`` is given. + + AUTOCAD_MCP_BACKEND=com uv run --frozen python scripts/smoke_understand_com.py + uv run --frozen python scripts/smoke_understand_com.py --keep # leave them open +""" + +from __future__ import annotations + +import asyncio +import dataclasses +import json +import sys +import tempfile +import time +from collections import Counter +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(ROOT)) + +#: ``RPC_E_CALL_REJECTED``: AutoCAD's message filter refused the call because +#: it was busy (a document still initialising after ``Documents.Open``). A +#: refused call was not executed. +RPC_E_CALL_REJECTED = -2147418111 + +#: How long a refused call is retried before the smoke gives up on it. +REJECTED_PATIENCE_S = 90.0 + +#: Names of the documents this run opened and has not closed yet. +_OPENED: list[str] = [] + + +def _patient(backend) -> None: + """Retry any backend call AutoCAD refused with ``RPC_E_CALL_REJECTED``.""" + run = backend._run + + async def patient(func, *args, **kwargs): + deadline = time.monotonic() + REJECTED_PATIENCE_S + pause = 0.5 + while True: + try: + return await run(func, *args, **kwargs) + except RuntimeError as exc: + cause = exc.__cause__ + hr = cause.args[0] if cause is not None and cause.args else None + if hr != RPC_E_CALL_REJECTED or time.monotonic() + pause > deadline: + raise + await asyncio.sleep(pause) + pause = min(pause * 2.0, 5.0) + + backend._run = patient + + +async def _names(backend) -> set[str]: + return {row["name"] for row in await backend.document_list()} + + +async def _open(backend, path: str) -> str: + """``drawing_open``, recording every document it added (normally one).""" + before = await _names(backend) + await backend.drawing_open(path) + added = sorted((await _names(backend)) - before) + _OPENED.extend(name for name in added if name not in _OPENED) + return added[-1] if added else Path(path).name + + +async def _close(backend, name: str) -> None: + if name in await _names(backend): + await backend.document_close(name, discard=True) + if name in _OPENED: + _OPENED.remove(name) + + +async def _close_leftovers(backend) -> None: + open_names = await _names(backend) + for name in list(_OPENED): + if name in open_names: + try: + await backend.document_close(name, discard=True) + except Exception as exc: # best effort on the way out + print(f"could not close {name}: {exc}", file=sys.stderr) + _OPENED.remove(name) + + +def _model(snapshot): + """The snapshot with its model-space records only (AutoCAD adds a paper-space + viewport when it initialises a layout; the takeoffs read model space).""" + return dataclasses.replace( + snapshot, records=tuple(r for r in snapshot.records if r.space == "Model") + ) + + +def _types(snapshot) -> dict[str, int]: + return dict(sorted(Counter(r.type for r in _model(snapshot).records).items())) + + +def _check_takeoffs(truth, pid, layout) -> dict: + from engineering.understand.network import build_network + from engineering.understand.scale import scale_check + from engineering.understand.takeoff import cable_rows, pipe_rows + + scale = scale_check(pid, layout) + services = tuple(sorted({run["service"] for run in truth["pipe_runs"]})) + pipe = pipe_rows(build_network(pid), pid, layout, scale=scale, services=services) + found = {(r["service"], tuple(sorted(r["tags"]))): r for r in pipe["runs"]} + pipe_mismatch = [] + for run in truth["pipe_runs"]: + got = found.get((run["service"], tuple(run["tags"]))) + want = run["layout_length_mm"] + have = None if got is None else got.get("layout_m") + agrees = got is not None and got["status"] == run["status"] + if want is None: + agrees = agrees and have is None + else: + agrees = agrees and have is not None and abs(float(have) * 1000.0 - want) <= 1e-6 + if not agrees: + pipe_mismatch.append({"run": run["id"], "want": want, "have": have}) + rows = {row["tag"]: row for row in cable_rows(pid, layout)["rows"]} + cable_mismatch = [ + want["tag"] + for want in truth["cable_rows"] + if rows.get(want["tag"], {}).get("cable_m") != want["roundup_m"] + or rows.get(want["tag"], {}).get("panel") != want["panel"] + ] + return { + "scale_verdict": scale["verdict"], + "runs_checked": len(truth["pipe_runs"]), + "pipe_mismatch": pipe_mismatch, + "loads_checked": len(truth["cable_rows"]), + "cable_mismatch": cable_mismatch, + } + + +async def _smoke(backend) -> dict: + from engineering.understand.diff import diff_snapshots + from engineering.understand.snapshot import read_snapshot, take_snapshot + from tests.fixtures.plant_pair import build_plant_pair + + folder = Path(tempfile.mkdtemp(prefix="acadmcp_understand_smoke_")) + truth = build_plant_pair(folder) + + pid_name = await _open(backend, truth["pid"]) + started = time.perf_counter() + live_pid = await take_snapshot(backend) + export_s = time.perf_counter() - started + dwg = folder / "pid.dwg" + await backend.drawing_save_as(str(dwg), "dwg") + for name in (pid_name, dwg.name): + await _close(backend, name) + dwg_pid = await take_snapshot(backend, str(dwg)) + + layout_name = await _open(backend, truth["layout"]) + live_layout = await take_snapshot(backend) + + takeoffs = _check_takeoffs(truth, _model(live_pid), _model(live_layout)) + + tag = next( + r + for r in live_layout.records + if r.type == "TEXT" and r.layer == "TAGS" and r.space == "Model" + ) + await backend.entity_move(tag.handle, 10.0, 0.0) + after = await take_snapshot(backend) + diff = diff_snapshots(_model(live_layout), _model(after)) + await _close(backend, layout_name) + + file_pid, file_layout = read_snapshot(truth["pid"]), read_snapshot(truth["layout"]) + return { + "export_s": round(export_s, 3), + "live_pid_source": live_pid.source, + "dwg_pid_source": dwg_pid.source, + "model_space": { + "pid_file": _types(file_pid), + "pid_live": _types(live_pid), + "pid_dwg": _types(dwg_pid), + "layout_file": _types(file_layout), + "layout_live": _types(live_layout), + }, + "paper_space_layouts": sorted({r.space for r in live_layout.records} - {"Model"}), + "takeoffs": takeoffs, + "diff": { + "moved": tag.handle, + "added": len(diff["added"]), + "removed": len(diff["removed"]), + "changed": len(diff["changed"]), + "moved_is_the_change": f'"{tag.handle}"' in json.dumps(diff["changed"]), + }, + } + + +async def main(keep_document: bool = False) -> int: + try: + from backends.com_backend import ComBackend + except ImportError as exc: + print(f"pywin32 not available: {exc}") + return 2 + backend = ComBackend() + _patient(backend) + try: + await backend.connect() + except Exception as exc: + print(f"no live AutoCAD reachable: {exc}") + return 2 + try: + status = await backend.system_status() + if not status.get("connected"): + print(f"no live AutoCAD reachable: {status.get('error', status)}") + return 2 + report = { + "autocad": status.get("autocad_version") or status.get("cad_progid"), + **await _smoke(backend), + } + print(json.dumps(report, indent=2, ensure_ascii=False)) + spaces = report["model_space"] + ok = ( + spaces["pid_live"] == spaces["pid_file"] == spaces["pid_dwg"] + and spaces["layout_live"] == spaces["layout_file"] + and report["takeoffs"]["scale_verdict"] == "schematic" + and not report["takeoffs"]["pipe_mismatch"] + and not report["takeoffs"]["cable_mismatch"] + and report["diff"]["changed"] == 1 + and report["diff"]["added"] == report["diff"]["removed"] == 0 + and report["diff"]["moved_is_the_change"] + ) + return 0 if ok else 1 + finally: + if not keep_document: + await _close_leftovers(backend) + await backend.disconnect() + + +if __name__ == "__main__": + raise SystemExit(asyncio.run(main(keep_document="--keep" in sys.argv[1:]))) +``` + +- [ ] **Step 15: Run the smoke against the live seat and keep its output** + +Open AutoCAD 2026 first (a cold start can exceed `COM_CALL_TIMEOUT`), then: + +``` +AUTOCAD_MCP_BACKEND=com uv run --frozen python scripts/smoke_understand_com.py +``` + +Expected: exit 0 and a JSON report whose `model_space` counts agree pairwise (`pid_live` = `pid_file` = `pid_dwg`, `layout_live` = `layout_file`), whose `takeoffs` show `schematic` with empty `pipe_mismatch` and `cable_mismatch`, and whose `diff` is `changed` 1, `added` 0, `removed` 0 with `moved_is_the_change` true. `paper_space_layouts` records which layouts the live export carried (Task 1 measured Layout1 reaching the snapshot). Save the printed JSON verbatim — Step 20 quotes it in the CHANGELOG. If a live defect shows (an entity AutoCAD rewrites on open, a snapshot that differs from its file), fix the *code* to the measured behaviour with a test that pins it, never the smoke's expectation, and record it under **Fixed**. If AutoCAD is not reachable (exit 2), say so in the task report and do not write a smoke record the run did not produce. + +- [ ] **Step 16: Point the document-number checker at the v7 matrix** + +In `scripts/check_doc_numbers.py`, change + +```python + from benchmarks.tasks_v6 import TASKS_V6 +``` + +to + +```python + from benchmarks.tasks_v7 import TASKS_V7 +``` + +and + +```python + "matrix_tasks": len(TASKS_V6), +``` + +to + +```python + "matrix_tasks": len(TASKS_V7), +``` + +- [ ] **Step 17: Regenerate the published A/B report over the grown suite** + +`scripts/check_doc_numbers.py` refuses while the suite holds more checks than the published report was run over — the report is re-run, never re-typed: + +``` +uv run --frozen python benchmarks/compare_versions.py v1.5.1 --json benchmarks/results/published/ab-v1.5.1-vs-v1.6.0-dev.json +uv run --frozen python -c "import json; print(json.load(open('benchmarks/results/published/ab-v1.5.1-vs-v1.6.0-dev.json'))['summary'])" +``` + +Expected: `total` 48, `old_pass` 26, `new_pass` 48, `old_pct` 54.2, `fixed` 22, `regressed` 0 — the five new checks are `miss` on v1.5.1 and `pass` now. **A non-zero `regressed` stops the task**: find the regression and fix the code, never the report. + +- [ ] **Step 18: Remeasure the README token table** + +The surface moved (247 registered, 242 advertised, lean 65, a new pack), so every row of the README's "Advertised surface" table is remeasured — the suite rows and the two single-pack rows it does not produce: + +``` +uv run --frozen pytest tests/test_token_suite.py -q +uv run --frozen python -m benchmarks.token_suite --tokenizer ratio +uv run --frozen python -c "import json; r = json.load(open('benchmarks/results/token-cost.json', encoding='utf-8')); [print(x['variant'], x['advertised_tools'], x['tokens']['value']) for x in r['idle']]" +TOOL_PACKS=core,settings uv run --frozen python -m benchmarks.token_suite --tokenizer ratio --out benchmarks/results/token-core-settings.json +TOOL_PACKS=core uv run --frozen python -m benchmarks.token_suite --tokenizer ratio --out benchmarks/results/token-core.json +uv run --frozen python -c "import json; [print(f, [(x['advertised_tools'], x['tokens']['value']) for x in json.load(open(f, encoding='utf-8'))['idle'] if x['variant'] == 'default']) for f in ('benchmarks/results/token-core-settings.json', 'benchmarks/results/token-core.json')]" +``` + +Expected: the token tests green; `default` 242 tools, `packs_core_mech` 197, `packs_core_arch` 195, `lean` 65, `search` 2; `core,settings` 205 and `core` 183. Keep every printed `advertised_tools` / `tokens` pair — Step 20 writes them into the README. (`benchmarks/results/*` other than `published/` is git-ignored: these are measurements, not artefacts.) + +- [ ] **Step 19: Update `CLAUDE.md`** + +Append to the numbered tool-section list, after item 25: + +``` +26. Understanding & QA (4 tools, pack `core`): `drawing_understand` (one read-only report of a drawing somebody else made — declared vs inferred units with the evidence, declared vs robust extents and the outlier handles behind them, clusters (sheets, plan copies), every layer's likely discipline and service from an EN / TR / RU / NL / DE vocabulary with a confidence and the deciding keyword, equipment tags, rooms, text languages, blocks, layouts, title-block attributes), `drawing_scale_check` (a drawing against a reference: tags unique within a cluster are matched — a tag seen twice is `ambiguous`, never picked — pairs at least 2 m apart, the median ratio, IQR and the share within ±10 %: `to_scale` at 80 % or more, with its factor; `schematic` at 50 % or less; `partly` between), `drawing_diff` (two revisions matched by signature, then handle, then type + layer + nearest position; `markup=True` draws revision clouds on the newer one — its only write) and `drawing_topology_check` (dangling ends, near misses, interior crossings; the three `topo_*` critique focuses run the same engine under `focus=None` on layers no semantic focus covers). Every one reads one snapshot per drawing (`engineering/understand/snapshot.py`: `ezdxf` directly, or on a live seat `Document.Export(/snap, "DXF", ...)`, which never renames the document and carries model space, paper space and block definitions — measured). +27. Plant takeoffs (2 tools, pack `plant`): `pipe_takeoff` (topology from the P&ID — segments merged at their ends, split at T-junctions, continued through inline fittings, a double-drawn segment counted once; service from the layer overridden by supply / return words; sizes kept as drawn — `Ø51`, `SMS51`, `DN20` are never equated — direct, by continuity or `unassigned`; lengths from the layout: the rectilinear MST of the tags a run connects, labelled an estimate, the P&ID length kept beside it as a comparison; statuses A / B / C) and `cable_takeoff` (one row per load: power from the P&ID's electrical texts, never invented; the panel from the layout's `Wiring to CPn` callouts; the Manhattan length plus the allowance rounded up to the metre; a section only from a caller's `section_rules`). Both write a workbook (Metraj / Özet / Kontrol / Metodoloji, `lang` tr / en / ru, XLSX with the `office` extra, CSV always) and never the drawing. `scripts/field_test_takeoff.py` runs both on a user's own pair and writes next to it, never into this repository. The COM paths were executed live once by `scripts/smoke_understand_com.py` (AutoCAD 2026). +``` + +(If Step 15 exited 2 — no live seat — end item 27 at "never into this repository." instead: the smoke sentence is written only for a run that happened.) + +In the **Installation** block, after the `pip install -e ".[pdf]"` line and its comment, add: + +``` +# XLSX workbooks (takeoffs, data_extract) via openpyxl +pip install -e ".[office]" +``` + +In the **Tool profiles** paragraph, replace `` `lean` = 64 curated drafting tools (54 with `TOOL_PACKS=core`, which drops the three `pid_*`, the three `mech_*` and the four `arch_*` lean members; `` with `` `lean` = 65 curated drafting tools (55 with `TOOL_PACKS=core`, which drops the three `pid_*`, the three `mech_*` and the four `arch_*` lean members, and keeps `drawing_understand`; ``. + +In the **Tool packs** paragraph, replace `` `TOOL_PACKS=all|core,pid,settings,mech,arch` `` with `` `TOOL_PACKS=all|core,pid,settings,mech,arch,plant` ``, `` `arch` the 12 SECTION 25 architectural tools; SECTION 18/19/24 are core; `` with `` `arch` the 12 SECTION 25 architectural tools, `plant` the two SECTION 27 takeoffs (track D's Plant 3D readers join it); SECTION 18/19/24/26 are core; ``. + +In the **Benchmark matrix** paragraph, replace `` `benchmarks/tasks_v6.py` is the current set (19 tasks; `--matrix v5` / `v4` / `v3` / `v2` reproduce the earlier sets) `` with `` `benchmarks/tasks_v7.py` is the current set (21 tasks; `--matrix v6` / `v5` / `v4` / `v3` / `v2` reproduce the earlier sets) ``; replace `` gated like `mech_assembly`) — the chart shows them `not run` `` with `` gated like `mech_assembly`) or the v7 `takeoff_roundtrip` / `understand_foreign` (the synthetic plant pair of `tests/fixtures/plant_pair.py` read without its truth: the P&ID called schematic, every routable run's layout length equal to the rectilinear MST of its tags, the cable metres rounded up after the allowance; units, the outlier, both plan copies and the service layers found by one `describe`) — the chart shows them `not run` ``; and `` `benchmarks/correctness_suite.py` (43 checks) `` with `` `benchmarks/correctness_suite.py` (48 checks) ``. + +In **Key Conventions**, replace `` - **`TOOL_PACKS=all|core,pid,settings,mech,arch`** advertises `` with `` - **`TOOL_PACKS=all|core,pid,settings,mech,arch,plant`** advertises ``, and append after the bullet that begins `- **Nominal, not standard**`: + +``` +- **Read-only by default (track H)**: `drawing_understand`, `drawing_scale_check`, `drawing_topology_check` and both takeoffs never modify the drawing; `drawing_diff` writes only with `markup=True`, and a takeoff writes a workbook file, never the drawing. +- **Lengths come from the inferred unit**: a reader measures in the unit the geometry implies (text heights, wall and door sizes, overall size) and reports metres; a disagreement with `INSUNITS` is a warning with the numbers — a layout declaring inches over millimetre geometry is measured in millimetres, never ×25.4. +- **Scale check before a P&ID length**: a P&ID is a length source only when `drawing_scale_check` calls it `to_scale`; otherwise a run's length is the orthogonal estimate on the layout and the drawn length stays a comparison column (`length_source="pid"` on a schematic P&ID is refused unless `force=True`, quoting the statistics). +- **Nothing is invented (track H)**: a power the P&ID does not state stays empty and becomes an open item, an unreadable size stays `unassigned`, and no cable-section table is authored — a section comes only from the caller's `section_rules`. +- **Client drawings never enter the repository**: every track-H fixture is the synthetic plant pair (`tests/fixtures/plant_pair.py`), whose truth is computed from the positions it placed; `scripts/field_test_takeoff.py` refuses an output folder inside the repository and prints no quantity unless `--show` is given. +``` + +- [ ] **Step 20: Update `README.md`, `benchmarks/README.md`, `CHANGELOG.md` and the spec status line** + +`README.md`: + +- add a "What you get" row after the `Architecture` row: + +``` +| Understanding & takeoffs | `drawing_understand` reads a drawing somebody else made in one call — declared vs inferred units, robust extents and the outlier behind them, plan copies, each layer's discipline and service in five languages, tags, rooms; `drawing_scale_check` says whether a P&ID is to scale against its layout; `pipe_takeoff` / `cable_takeoff` take the topology from the P&ID and the lengths from the layout (rectilinear MST, statuses A/B/C, power never invented) into a Turkish / English / Russian workbook; `drawing_diff` between revisions; `drawing_topology_check` for dangling ends, near misses and crossings | +``` + +- replace the token table's rows with the ones Step 18 measured. The tool counts are the ones Step 18 printed (242 / 205 / 197 / 195 / 183 / 65 / 2 when the merge added exactly six tools); the token column is that run's `tokens` value for the row, written with thousands separators like the existing rows: + +``` +| `TOOL_PROFILE=full` (default) | 242 | tokens | +| `TOOL_PACKS=core,settings` (full profile) | 205 | tokens | +| `TOOL_PACKS=core,mech` (full profile) | 197 | tokens | +| `TOOL_PACKS=core,arch` (full profile) | 195 | tokens | +| `TOOL_PACKS=core` (full profile) | 183 | tokens | +| `TOOL_PROFILE=lean` | 65 | tokens | +| `DISCOVERY_MODE=search` | 2 | ** tokens** | +``` + + (the angle-bracketed cells are Step 18's output, copied, never estimated), and take the sentence above the table (`The full catalog costs a client **N tokens** … costs **M**.`) and the `token_budget` row of the task-matrix table (`N → M tokens`) from the `default` and `search` rows of the same run; +- in the task-matrix section, change `1.6 adds four more — one the P&ID reader can fail on its own, one where the PDF file, not the setter, is the witness, one where a whole mechanical sheet has to come out clean, and one where a floor plan's rooms are read back off the drawing.` to `1.6 adds six more — one the P&ID reader can fail on its own, one where the PDF file, not the setter, is the witness, one where a whole mechanical sheet has to come out clean, one where a floor plan's rooms are read back off the drawing, and two where a foreign plant's P&ID and layout are read: the takeoffs against lengths computed in advance, and the one-call report against the defects planted in it.` and add after the `arch_roundtrip` row: + +``` +| `takeoff_roundtrip` | the synthetic plant pair — a P&ID stretched 1.3× in one room and rearranged in the other, Russian supply / return words, a return drawn on the supply layer, a reducer, a segment drawn twice — read by the scale check (**schematic**: 21.8 % of 55 pairs within ±10 %), `pipe_takeoff` (every routable run's layout length equal to the rectilinear MST of its tags, sizes and services as drawn, the run to a tag the layout lacks status C) and `cable_takeoff` (9 / 17 / 19 / 10 m after the 20 % allowance, the heater's power left empty); a test removes the wiring callouts and watches the cable half fail by name | +| `understand_foreign` | the pair's layout — inches declared over millimetres, a stray line 5,000 km out, two plan copies — comes back as millimetres with an `INSUNITS` warning, the stray line's handle and two clusters; the P&ID's `P_product piping` / `P_cipsupplyline` / `P_cipreturnline` / `P_ijswater` / `E_power` classify as product / CIP supply / CIP return / ice water / electrical; a test makes the layout declare millimetres and watches the units gate fail | +``` + +- in the A/B section, change `43 deterministic headless checks` to `48 deterministic headless checks`, replace the two table rows with the regenerated report's numbers — + +``` +| v1.5.1 *(baseline)* | 26 / 48 | 54.2 % | — | — | +| **v1.6.0-dev** *(this branch)* | **48 / 48** | **100 %** | 22 | **0** | +``` + + (if Step 17's `summary` printed other numbers, write those) — and change the prose opening `The seventeen fixed rows are new capability (\`miss → pass\`). Track F adds four:` to: + +``` +The twenty-two fixed rows are new capability (`miss → pass`). Track H adds five: `scale_check_detects_schematic` (the synthetic P&ID reads **schematic** against its layout, and a uniform 2× copy of six tags reads **to scale** with factor 2), `pipe_takeoff_rmst_exact` (all eight routable runs of the pair measure exactly the rectilinear MST of their tags), `cable_takeoff_roundup` (9, 17, 19 and 10 m after the 20 % allowance, and the heater's unstated power stays empty), `diff_detects_known_edits` (a move, a text change, an attribute change, an add and a delete — and nothing else) and `topology_known_defects` (a 5 mm near miss, one interior crossing, and a clean T that is neither). Track F adds four: +``` + +`benchmarks/README.md`: + +- after the last paragraph of `### Matrix v6 (v1.6) — the architectural plan` (the one ending `The competitor reports carry no result for it.`), add: + +``` +### Matrix v7 (v1.6) — understanding foreign drawings + +`tasks_v7.py` keeps the v6 nineteen unchanged and adds track H's evidence, both on the synthetic plant pair of `tests/fixtures/plant_pair.py` (invented names; its truth is computed from the positions it placed, never typed): + +| Task | Category | Verified against | +|---|---|---| +| `takeoff_roundtrip` | plant | `scale_check(pid, layout)` must say **schematic** (room 1 is a 1.3× stretch, room 2 is rearranged: 21.8 % of the 55 tag pairs within ±10 % of the median ratio); `pipe_rows(..., length_source="auto")` must report all nine runs with the truth's status, each routable run's layout length equal to the rectilinear MST of its tags to a micron (PR1 5500, PR2 13200, CS1 20700 mm, …), its size and service as drawn — the CIP return drawn on the supply layer included — and both sizes across the reducer; `cable_rows` must give M11 / T102 / M12 / T202 their panels, 6900 / 14100 / 15800 / 8000 mm and 9 / 17 / 19 / 10 m after the 20 % allowance, the heater's power empty. `tests/test_benchmark_v7.py` deletes the wiring callouts and the gate names all four loads | +| `understand_foreign` | understand | `describe(layout)` must infer millimetres and warn about `INSUNITS` (the file says inches), name the stray line's handle in its extents and report two clusters (the plan is drawn twice, 100 m apart); `describe(pid)` must classify `P_product piping`, `P_cipsupplyline`, `P_cipreturnline`, `P_ijswater` and `E_power` as product, CIP supply, CIP return, ice water and electrical. `tests/test_benchmark_v7.py` makes the layout declare millimetres and the units gate fails alone | + +The competitor reports carry no result for them. +``` + +- in `### Result — this branch (1.6.0-dev) vs v1.5.1 (release gate)`, change `43 checks, ezdxf backend` to `48 checks, ezdxf backend`, replace the two table rows with + +``` +| **v1.5.1** (baseline) | 26 / 48 | 54.2 % | — | — | +| **v1.6.0-dev** (this branch) | 48 / 48 | 100 % | 22 | **0** | +``` + + and change `The seventeen are all \`miss → pass\` — v1.5.1 has none of the methods. Four are track F:` to `The twenty-two are all \`miss → pass\` — v1.5.1 has none of the methods. Five are track H: \`scale_check_detects_schematic\` (the synthetic P&ID is schematic against its layout; a uniform 2× copy is to scale with factor 2), \`pipe_takeoff_rmst_exact\` (eight routable runs, each exactly the rectilinear MST of its tags), \`cable_takeoff_roundup\` (9 / 17 / 19 / 10 m, the unstated power left empty), \`diff_detects_known_edits\` (five known edits and nothing else) and \`topology_known_defects\` (one near miss, one crossing, a clean T). Four are track F:`. + +`CHANGELOG.md` — add to `## [Unreleased]`, after the track F paragraph (the one ending `` `docs/superpowers/specs/2026-09-23-v1.6-arch-design.md`. ``): + +``` +Track H of 1.6: understanding drawings somebody else made. Six tools in two +new sections — understanding and QA (26: a one-call report, a scale check +between a P&ID and its layout, a revision diff, topology checks) and plant +takeoffs (27: pipe and cable quantities whose topology comes from the P&ID +and whose lengths come from the layout) — one DXF snapshot per drawing (a +live document through `Document.Export`, never renamed), a vocabulary in five +languages, three `topo_*` critique focuses, a new `plant` pack and the +`office` extra. Read-only by default; nothing invented. No new contract +member. Spec: `docs/superpowers/specs/2026-09-24-v1.6-understand-design.md`. +``` + +under `### Added`, directly after the track F bullet that begins `- **Live COM smoke.** \`scripts/smoke_arch_com.py\``: + +``` +- **Understanding foreign drawings — 4 tools, pack `core`; plant takeoffs — 2 + tools, pack `plant`.** `drawing_understand`, `drawing_scale_check`, + `drawing_diff`, `drawing_topology_check`, `pipe_takeoff`, `cable_takeoff`. + Every one reads a snapshot of frozen WCS records + (`engineering/understand/snapshot.py`); labels are read by one parser + (`%%c`, `\U+2205`, the U+2300 sign, `ø` and MTEXT format runs decoded, + Cyrillic look-alike tags folded to Latin) and layer names and texts by one + EN / TR / RU / NL / DE vocabulary. The takeoff workbook is written as CSV + always and as XLSX with the new `office` extra (`openpyxl`, also in `full`). + Lean gains `drawing_understand` and reaches 65. +- **Benchmark evidence for track H.** `benchmarks/tasks_v7.py` (21 tasks, the + default matrix; v6 to v2 stay addressable) adds `takeoff_roundtrip` and + `understand_foreign`, both on the synthetic plant pair of + `tests/fixtures/plant_pair.py`, whose truth is computed from the positions + it placed; each gate is shown to fail on a broken input. The correctness + suite grows to 48 with `scale_check_detects_schematic`, + `pipe_takeoff_rmst_exact`, `cable_takeoff_roundup`, + `diff_detects_known_edits` and `topology_known_defects`. A/B against + `v1.5.1`: **26 / 48 → 48 / 48, twenty-two `miss → pass`, zero regressed**. +- **`scripts/field_test_takeoff.py`** runs both takeoffs on a user's own P&ID + and layout and writes the report next to them; it refuses an output folder + inside the repository and prints no quantity unless `--show` is given. +- **Live COM smoke.** `scripts/smoke_understand_com.py` executed once on + AutoCAD 2026 (``): the live snapshots matched + their files type for type in model space, the DWG route read the same + P&ID, both takeoffs matched the truth and the diff reported exactly the + one entity moved (record below). +``` + +(write the `autocad` version string exactly as Step 15's JSON printed it), and under `### Changed`, as its first bullet: + +``` +- **`MAX_DXF_BYTES` defaults to 512 MB** (was 50 MB): a real plant layout read + for track H was a 188 MB DXF. The snapshot's refusal names the file's size, + the limit and the variable; `0` still disables the check. +``` + +and a new section directly after the `### Live COM smoke — architecture (track F)` section, before the next `## ` heading: + +```` +### Live COM smoke — understanding (track H) + +`scripts/smoke_understand_com.py`, AutoCAD 2026, Windows 11, on the synthetic +plant pair only — `take_snapshot` of `pid.dxf` opened live, the same P&ID +saved as DWG and read back through the read-only DWG route, `take_snapshot` +of `layout.dxf`, `pipe_rows` / `cable_rows` on the live snapshots against the +pair's truth, then one tag moved 10 mm and `diff_snapshots` of the layout +before and after, all through the COM engine, exit 0, every document closed: + +```json + +``` +```` + +(this slot and the token cells of the README table are the only values in this plan that cannot be written in advance: they are the live seat's and the token run's own output, pasted, never retyped or estimated). + +`docs/superpowers/specs/2026-09-24-v1.6-understand-design.md` — change `**Status:** design approved in conversation` to `**Status:** shipped (v1.6.0-dev, track H)`. + +- [ ] **Step 21: The full gate** + +``` +uv run --frozen ruff format benchmarks scripts tests +uv run --frozen ruff check . +uv run --frozen python scripts/generate_tool_inventory.py --check +uv run --frozen python scripts/check_doc_numbers.py --list +uv run --frozen python scripts/check_doc_numbers.py +uv run --frozen pytest -q +uv run --frozen pytest tests/test_capability_contract.py tests/test_tool_registry.py tests/test_discovery_aliases.py tests/test_tool_packs.py tests/test_release_consistency.py tests/test_benchmark_v7.py tests/test_field_test_takeoff.py tests/test_plant_pair.py -q +git grep -n -i -E "client|customer" -- tests/fixtures benchmarks/results/published +``` + +Expected: ruff clean; no inventory drift; `check_doc_numbers` exits 0 with every figure derived (`matrix_tasks` 21, `suite_checks` = `ab_total` = 48, `ab_fixed` 22); the whole suite green, `tests/test_release_consistency.py::test_document_numbers_match_the_tree` included. This task added test files, so the collected-tests figure moved after Task 11: if `check_doc_numbers` reports it, retake it from its own `--list` output into the README release-snapshot line. The last `git grep` prints only the two fixture docstrings that say the pair is never a client drawing — no client name, file name or quantity is in a fixture or a published result (spec §14). + +- [ ] **Step 22: Commit** + +``` +git add benchmarks/tasks_v7.py benchmarks/adapters/autocad_mcp_pro.py benchmarks/render_matrix_chart.py benchmarks/run_competitors.py benchmarks/correctness_suite.py benchmarks/results/published/ab-v1.5.1-vs-v1.6.0-dev.json scripts/smoke_understand_com.py scripts/field_test_takeoff.py scripts/check_doc_numbers.py tests/test_benchmark_v4.py tests/test_benchmark_v5.py tests/test_benchmark_v6.py tests/test_benchmark_v7.py tests/test_field_test_takeoff.py CLAUDE.md README.md benchmarks/README.md CHANGELOG.md docs/superpowers/specs/2026-09-24-v1.6-understand-design.md +git -c user.name="Umut CAN" -c user.email="umutcan.edizaslan@gmail.com" commit -m "$(cat <<'MSG' +bench: matrix v7 adds takeoff_roundtrip and understand_foreign; five understanding checks; the live smoke and the field-test script + +The evidence for track H, in the shape tracks A, E, B + G and F used. + +- benchmarks/tasks_v7.py: 21 tasks, DEFAULT_MATRIX = "v7"; v6 to v2 stay + addressable. Both new tasks read the synthetic plant pair and nothing + else. takeoff_roundtrip: the P&ID called schematic, every routable run's + layout length equal to the rectilinear MST of its tags, sizes and services + as drawn (a return drawn on the supply layer included), the cable metres + rounded up after the allowance, an unstated power left empty. + understand_foreign: inches over millimetres flagged, the stray entity found + by handle, two plan copies separated, five layers classified. A test breaks + each input and watches its gate fail by name. +- Five correctness checks, every number worked out by hand in its docstring; + the published A/B report is re-run over 48 checks: 26 / 48 -> 48 / 48, + zero regressed. +- scripts/smoke_understand_com.py reads the pair through the live engine - + snapshots, the DWG route, both takeoffs and a diff - and closes every + document it opens; the CHANGELOG records what it returned. + scripts/field_test_takeoff.py runs the takeoffs on a user's own files, + refuses to write inside the repository and prints no quantity by default. +- The README token table is remeasured; check_doc_numbers derives the task + count from the v7 matrix; CLAUDE.md, both READMEs, the CHANGELOG and the + spec status line say what shipped. + +Co-Authored-By: Claude Opus 5.5 (1M context) +Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd +MSG +)" +```