Umut CANandClaude Opus 5.5
ba41428c88
The evidence for track H, in the shape tracks A, E, B + G and F used.
- benchmarks/tasks_v7.py: 21 tasks, DEFAULT_MATRIX = "v7"; v6 to v2 stay
addressable. Both new tasks read the synthetic plant pair and nothing
else. takeoff_roundtrip: the P&ID called schematic, every routable run's
layout length equal to the rectilinear MST of its tags, sizes and services
as drawn (a return drawn on the supply layer included), the cable metres
rounded up after the allowance, an unstated power left empty.
understand_foreign: inches over millimetres flagged, the stray entity found
by handle, two plan copies separated, five layers classified. A test breaks
each input and watches its gate fail by name.
- Five correctness checks, every number worked out by hand in its docstring;
the published A/B report is re-run over 48 checks: 26 / 48 -> 48 / 48,
zero regressed.
- scripts/smoke_understand_com.py reads the pair through the live engine -
snapshots, the DWG route, both takeoffs and a diff - and closes every
document it opens; the CHANGELOG records what it returned.
scripts/field_test_takeoff.py runs the takeoffs on a user's own files,
refuses to write inside the repository and prints no quantity by default.
- The README token table is remeasured; check_doc_numbers derives the task
count from the v7 matrix; CLAUDE.md, both READMEs, the CHANGELOG and the
spec status line say what shipped.
One reader defect found on the way: describe.tag_occurrences read a wiring
callout ("Wiring to CP-2") as a place its panel is written, so the scale
check called CP-2 ambiguous and read "Wiring to CP1" as a phantom tag CP1 -
10 tags and 45 pairs (13.3 % within 10 %) where the generator placed 11 and
55 (21.8 %). A callout is no longer a tag occurrence, the rule
network.tag_occurrences already kept; two tests failed before the fix.
The smoke works only in documents it opened from its own temporary folder,
known by full path; it attaches with GetActiveObject only (the backend's
Dispatch fallback would launch AutoCAD), checks in the same COM-thread call
that the active document is its own before each snapshot, the DWG save and
the move, closes its documents with Close(False) and re-activates the
document that was active when it started. No new ActiveX member: Documents,
Item, FullName, ActiveDocument, Activate and Close are the ones the backend
already calls. Measured on AutoCAD 2026 (25.1s): exit 0, model-space types
equal file / live / DWG, export 0.104 s, no document of the run left open.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd
307 lines
17 KiB
Markdown
307 lines
17 KiB
Markdown
# v1.6 Track H — understanding foreign drawings, takeoffs, diff and topology
|
||
|
||
**Date:** 2026-09-24 · **Status:** shipped (v1.6.0-dev, track H) ·
|
||
**Parent:** `2026-09-15-v1.6-roadmap-design.md` (track H; track D stays separate
|
||
and follows) · **Baseline:** main after track F (`0abbe55`, 241 tools)
|
||
|
||
## 1. Summary
|
||
|
||
Every track so far taught the server to *draw*. This one teaches it to *read* a
|
||
drawing somebody else made and turn it into a deliverable — the job a plant
|
||
engineer does by hand when a client sends a P&ID and a layout and asks how much
|
||
pipe and cable the job needs.
|
||
|
||
The design is grounded in a real client project from the user's archive, read
|
||
on 2026-09-24 (the files are confidential and never enter this repository; see
|
||
§14): a process plant with a 35,590-entity P&ID (96 layers, labels in
|
||
Russian, layer names in English and Dutch, 296 block names, most anonymous),
|
||
a 22,399-entity layout, a hand-made pipe takeoff and a hand-made cable
|
||
takeoff. Four findings shape everything below:
|
||
|
||
1. **Today's reader cannot read it.** `pid_graph` found 441 nodes on that P&ID —
|
||
427 of them `unknown_block` at confidence 0.3 — and 663 edges, every one of
|
||
line class `unknown`, in 73 s. Its confidences were honest; its answer was
|
||
not useful. It cannot tell a CIP supply line from a product line, read a
|
||
diameter label, or recognise a Russian service word.
|
||
2. **The P&ID is not to scale.** Matching equipment tags between the P&ID and
|
||
the layout, distances between the same pairs of tanks differ by factors from
|
||
0.1 to 5.6; inside one room the P&ID is a consistent ~1.28× stretch of the
|
||
plan, inside another it is rearranged outright. A pipe length measured on
|
||
that P&ID is a length of drawn line, not of pipe — and the hand-made pipe
|
||
takeoff was measured exactly that way, while the hand-made cable takeoff of
|
||
the same project states in its own method sheet that the P&ID is unscaled.
|
||
No tool warned anybody. This track makes that check a tool.
|
||
3. **The layout lies about its units.** `INSUNITS` says inches; the geometry is
|
||
millimetres (a 5.9 km wall total, a 150–280 m building). One stray entity
|
||
drags the extents to −6,450 km, and two copies of the plan sit side by side
|
||
in model space.
|
||
4. **Real drawings are big.** The two DXFs are 55 MB and 188 MB; `drawing_open`
|
||
refuses anything over 50 MB, and reading 35,000 entities one COM call at a
|
||
time takes minutes where a DXF snapshot takes half a second.
|
||
|
||
## 2. Scope
|
||
|
||
**In:** six tools (§12), three `topo_*` critique focuses, the `office` extra
|
||
(`openpyxl`), a DXF-snapshot reader for the live engine, the raised size limit,
|
||
synthetic fixtures that reproduce the field findings, two benchmark tasks,
|
||
correctness checks and a local field-test script.
|
||
|
||
**Out of 1.6 (recorded):**
|
||
|
||
- Plant 3D `.dcf` reading (track D, next).
|
||
- Cable cross-section *standards*: no IEC 60364-5-52 table is authored. A
|
||
section is chosen only from a rule table the caller supplies (the shop's own
|
||
rule); without one, no section is proposed. Voltage-drop calculation is out.
|
||
- Routing pipes around walls and equipment. A layout length is an orthogonal
|
||
estimate between connected equipment (§9) — the method the user's cable
|
||
takeoff already uses — and is labelled an estimate.
|
||
- OCR of raster underlays and text inside PDF-imported hatch geometry.
|
||
|
||
## 3. Architecture
|
||
|
||
```
|
||
engineering/understand/
|
||
__init__.py
|
||
snapshot.py one bulk read of a drawing: ezdxf directly, or a DXF snapshot of
|
||
the live document through Document.Export (§4)
|
||
vocab.py multilingual keyword tables (EN / TR / RU / NL / DE): disciplines,
|
||
services, equipment kinds, room words, supply/return words
|
||
labels.py parsers: diameters (Ø51, Ø25,4, SMS51, DN20, 20x27, 1,5"),
|
||
equipment tags, electrical data (kW, phases, voltage, +N, VFD)
|
||
units.py declared vs inferred units, robust extents, outliers
|
||
clusters.py spatial clusters of model-space content (sheets, plan copies)
|
||
describe.py drawing_understand: the report assembled from the above
|
||
scale.py drawing_scale_check: tag matching and ratio statistics
|
||
network.py line networks: junction merge, T-split, fittings bridged, runs,
|
||
service and diameter per edge, equipment attachment
|
||
route.py orthogonal lengths: Manhattan distance, rectilinear MST
|
||
takeoff.py pipe_takeoff and cable_takeoff rows, statuses, totals
|
||
report.py the takeoff workbook (Metraj / Özet / Kontrol / Metodoloji), CSV
|
||
diff.py entity signatures, matching, the change report
|
||
topology.py dangling endpoints, near misses, interior crossings
|
||
critique.py the three topo_* focuses
|
||
draw.py the async layer: snapshot, markup (revision clouds), exports
|
||
scripts/field_test_takeoff.py local only: runs the takeoffs on the user's files
|
||
```
|
||
|
||
Everything except `snapshot.py` and `draw.py` is pure: entity records in,
|
||
results out. The entity record is a small frozen dataclass (type, layer,
|
||
handle, geometry, text, block name, attributes) filled once per drawing, so a
|
||
35,000-entity drawing is read once and every analysis runs in memory.
|
||
|
||
## 4. Reading a drawing (the snapshot)
|
||
|
||
- **Headless:** `ezdxf.readfile` on the path, or the document already open.
|
||
- **Live:** `Document.Export(<path without extension>, "DXF", <selection set of
|
||
all>)` writes a DXF copy **without renaming the document** — measured on
|
||
AutoCAD 2026 on 2026-09-24: the document kept its name, an unsaved document
|
||
exported too, and ezdxf read the result. The extension argument must be
|
||
`"DXF"`; `".dxf"` fails with *Invalid argument* (measured). The snapshot goes
|
||
to a temporary folder and is deleted afterwards. Whether paper-space content
|
||
reaches the snapshot is measured by the plan's first live task and stated in
|
||
the tool docstrings. An xref's own entities are not read in 1.6 on either
|
||
engine - the host drawing is - and `drawing_understand`'s docstring says so.
|
||
- `MAX_DXF_BYTES` default rises from 50 MB to 512 MB, and the refusal names the
|
||
size and the variable (a real layout here was 188 MB).
|
||
|
||
Every tool in this track is read-only unless it says otherwise (only
|
||
`drawing_diff(markup=True)` and the takeoff export write anything, and the
|
||
export writes a file, never the drawing).
|
||
|
||
## 5. `drawing_understand`
|
||
|
||
One call, one report, every claim with its evidence and a confidence:
|
||
|
||
- **Units:** declared `INSUNITS` versus the unit the geometry implies (text
|
||
heights, typical wall / door sizes, dimension values, overall size). A
|
||
disagreement is a warning with the numbers, never a silent correction.
|
||
- **Extents:** declared versus robust (quantile) extents; the outlier entities
|
||
that stretch them, by handle.
|
||
- **Clusters:** separate bodies of content in model space — sheets, plan copies,
|
||
details — each with its box, its entity count and its most telling labels.
|
||
- **Layers:** every layer with its entity count and drawn length, and the
|
||
discipline / service it most likely carries (piping · product, CIP supply,
|
||
CIP return, glycol, steam, condensate, water, air, drain; electrical; walls;
|
||
equipment; annotation), from the multilingual vocabulary and the texts found
|
||
on it; confidence 0.9 for a direct keyword, lower for weaker evidence.
|
||
- **Equipment tags**, **rooms** (label words in five languages — *room*, *oda*,
|
||
*mahal*, *помещение*, *ruimte* — plus faces of wall layers through track F's
|
||
finder), **text languages**, **blocks** (named / anonymous, kinds by name),
|
||
**layouts** and **title-block attributes**.
|
||
|
||
## 6. `drawing_scale_check`
|
||
|
||
Compares a drawing with a reference drawing of the same plant. Equipment tags
|
||
found once in each are matched; for every pair of matched tags at least 2 m
|
||
apart the ratio of their distances is taken. The report gives the matched tags,
|
||
the median ratio, the interquartile range, the share of pairs within ±10 % of
|
||
the median, the same statistics per room or cluster, and a verdict:
|
||
`to_scale` (≥ 80 % of pairs within ±10 %; the factor is reported, so a drawing
|
||
at a different but uniform scale is recognised), `schematic` (≤ 50 %), or
|
||
`partly` in between. Tag labels are not equipment centres; the ±10 % band and
|
||
the 2 m floor are there to absorb that, and the report says so.
|
||
|
||
## 7. Line networks (the shared engine)
|
||
|
||
Segments (LINE, LWPOLYLINE, POLYLINE, ARC) on the chosen layers become a graph:
|
||
endpoints merged within a tolerance, segments split where another ends on them
|
||
(T-junctions), and a segment that ends at an inline fitting (an INSERT whose
|
||
box it touches — reducer, valve, coupling) continued through it. A connected
|
||
component is a **run**.
|
||
|
||
- **Service** of an edge: the layer's vocabulary class, overridden by supply /
|
||
return words within a search distance (`ПОДАЧА`, `ВОЗВРАТ`, `BESLEME`,
|
||
`DÖNÜŞ`, `SUPPLY`, `RETURN`, `AANVOER`, `RETOUR`), carried along the run. Layer
|
||
alone is not trusted to split CIP supply from return — the field P&ID proved
|
||
why.
|
||
- **Diameter** of an edge: the diameter label nearest to it within the search
|
||
distance (*direct*), else carried along the run from a labelled edge until a
|
||
reducer or a conflicting label (*continuity*), else *unassigned*. Labels are
|
||
kept exactly as drawn: `Ø51`, `SMS51`, `DN20`, `Ø25,4` and `Ø25` are different
|
||
sizes unless the drawing equates them.
|
||
- **Equipment attachment:** a run end inside (or within tolerance of) the box of
|
||
the geometry a tag labels is attached to that tag.
|
||
|
||
## 8. `pipe_takeoff`
|
||
|
||
Rows by room × service × diameter, in the layout of the user's own takeoff
|
||
workbook:
|
||
|
||
- **Lengths.** `length_source="auto"` (default) runs the scale check between
|
||
the P&ID and the layout. If the P&ID is `to_scale`, runs are measured on it.
|
||
If not, the length of each run is estimated on the layout: the positions of
|
||
the tags it connects, joined by a **rectilinear minimum spanning tree**
|
||
(Manhattan distance; a two-ended run is |Δx| + |Δy|), plus
|
||
`vertical_allowance` per connection (default 0, stated in the report). The
|
||
schematic length drawn on the P&ID is kept alongside as a comparison column,
|
||
never as the answer. `"pid"` and `"layout"` choose a source and the report
|
||
says which was chosen; `"pid"` on a P&ID the scale check calls `schematic` is
|
||
refused unless `force=True`. With no layout given, `auto` measures on the P&ID
|
||
and every row carries `scale_verified: false`, stated on the method sheet.
|
||
An orthogonal route runs along x first, then y — deterministic, and stated.
|
||
- **Status:** **A** — every end on a tag found on the layout and the diameter
|
||
direct; **B** — diameter by continuity; **C** — not routable (an end not on
|
||
equipment, a tag missing from the layout): reported on the *Kontrol* sheet
|
||
with its schematic length only, counted in the total and flagged.
|
||
- **Rooms:** a routed length is apportioned to the layout rooms its orthogonal
|
||
route crosses (room polygons from the layout's wall faces and room labels);
|
||
without room polygons, to the room label nearest the run.
|
||
- **Scope:** `services` defaults to product + CIP supply + CIP return (the
|
||
user's takeoff); utilities are included only on request.
|
||
- **Output:** a workbook — *Metraj* (rows with direct and continuity lengths
|
||
split, status, allowance, allowance-included), *Özet* (by service, by
|
||
diameter, by room), *Kontrol* (unassigned diameters and unroutable runs with
|
||
the reason), *Metodoloji* (generated: sources, the scale-check result, the
|
||
rules applied, the tolerances) — with Turkish, English or Russian headers
|
||
(`lang`), written as XLSX when `openpyxl` is installed (the new `office`
|
||
extra) and as CSV always. `allowance` defaults to 0.20 and is its own column,
|
||
never mixed into the net length.
|
||
|
||
## 9. `cable_takeoff`
|
||
|
||
One row per electrical load, the user's cable method made repeatable:
|
||
|
||
- **Power and supply** from the P&ID's electrical texts near each tag (`P =
|
||
18,5 кВт`, `3 ф. 400 В + N + PE`, `VFD`, in Russian, Turkish and English
|
||
spellings). A load with no stated power keeps an empty power cell and an open
|
||
item — **power is never invented**.
|
||
- **Panel** from the layout's electrical callouts (`Wiring to CP1` and its
|
||
variants) or the P&ID; panel names normalised (`CP 1` → `CP-1`).
|
||
- **Length:** Manhattan distance on the layout from the tag to its panel, plus
|
||
the allowance, rounded up to the metre (`ROUNDUP(length × (1 + allowance))`,
|
||
as the user's sheet does). With no layout given, lengths stay empty and each
|
||
row gets an open item — the P&ID is not a length source for cables.
|
||
- **Room totals** of known power, with the rule for panel packages versus their
|
||
child motors stated in the method sheet so no load is counted twice.
|
||
- **Section:** only from a caller-supplied `section_rules` table; otherwise
|
||
blank.
|
||
- Same workbook shape and languages as §8, with an *Open items* list.
|
||
|
||
## 10. `drawing_diff`
|
||
|
||
Two revisions of one drawing (paths, or the current document against a path).
|
||
Entity signatures (type, layer, geometry rounded to a tolerance; text content
|
||
and position; block name, insertion, rotation, scale and attributes) are
|
||
matched exactly first, then by handle, then by type + layer + nearest position
|
||
within a move tolerance; what is left is added or removed. The report gives
|
||
counts by layer and type, the changed entities with what changed (moved by, text
|
||
from → to, attribute from → to, radius from → to), block-definition changes and
|
||
a capped list per category. `markup=True` draws revision clouds around the
|
||
clusters of change on the newer drawing with track G's revision tools — the
|
||
only write in the tool, and only on request.
|
||
|
||
## 11. `drawing_topology_check` and the `topo_*` focuses
|
||
|
||
Dangling endpoints, near misses (two ends, or an end and a segment, within a
|
||
gap tolerance but not joined) and interior crossings (two segments crossing away
|
||
from their ends), on the layers given or, by default, on the network layers
|
||
`drawing_understand` classifies with confidence ≥ 0.9. The three critique
|
||
focuses (`topo_dangling_endpoint`, `topo_near_miss`, `topo_interior_crossing`)
|
||
run the same engine under `focus=None`, **excluding layers a semantic focus
|
||
already covers** (P&ID line layers → `pid_*`, architectural walls → `arch_*`),
|
||
so one defect is never scored twice; on a drawing made entirely by this server
|
||
they are silent.
|
||
|
||
## 12. Tool surface, packs, lean
|
||
|
||
| Tool | Pack | Writes? |
|
||
|---|---|---|
|
||
| `drawing_understand` | core (lean) | no |
|
||
| `drawing_scale_check` | core | no |
|
||
| `drawing_diff` | core | only `markup=True` |
|
||
| `drawing_topology_check` | core | no |
|
||
| `pipe_takeoff` | `plant` | a workbook file |
|
||
| `cable_takeoff` | `plant` | a workbook file |
|
||
|
||
**+6 → 247 tools.** New pack `plant` (track D's readers join it later). Lean
|
||
gains `drawing_understand` (→ 65). Aliases in five languages (metraj, metre
|
||
cetveli, boru listesi, kablo listesi, спецификация, ведомость, takeoff, BOQ,
|
||
fark, karşılaştır, revizyon farkı, compare, DWGCOMPARE, birim, ölçek, scale
|
||
check).
|
||
|
||
## 13. Refusals
|
||
|
||
Before any work: a missing or unreadable file, a DXF over the size limit (with
|
||
the variable named), `lang` outside tr/en/ru, a non-positive tolerance or
|
||
allowance outside 0–1, `length_source="pid"` with a P&ID the scale check calls
|
||
schematic *without* `force=True` (the refusal quotes the statistics), an XLSX
|
||
export without `openpyxl` (the `xlsx_write` capability, naming the extra; CSV
|
||
still written).
|
||
|
||
## 14. Confidentiality
|
||
|
||
Client drawings, spreadsheets and anything derived from them that identifies
|
||
the client or its business — file names, client and site names, takeoff
|
||
quantities — never enter the repository, its tests, its
|
||
benchmark results or its commit messages. The fixtures are **synthetic**
|
||
(§15). `scripts/field_test_takeoff.py` runs the tools on files the user points
|
||
it at and writes its report next to them, outside the repository; its output is
|
||
never committed.
|
||
|
||
## 15. Evidence
|
||
|
||
- **Synthetic plant pair** (`tests/fixtures/plant_pair.py`): a generator that
|
||
builds a schematic P&ID DXF and a layout DXF of one small plant with known
|
||
answers — equipment tags present in both, the P&ID deliberately stretched
|
||
1.3× in one room and rearranged in another, service layers with English /
|
||
Dutch names, Russian supply / return words, `Ø` / `SMS` / `DN` / `a x b`
|
||
labels, a reducer, an inline valve, electrical texts in Russian, `Wiring to
|
||
CPn` callouts, a layout with `INSUNITS` set to inches over millimetre
|
||
geometry, one far outlier entity and two plan copies.
|
||
- **Benchmark tasks** (matrix v7, 21 tasks): `takeoff_roundtrip` — the scale
|
||
check calls the synthetic P&ID `schematic`, every pipe run's layout length
|
||
equals the hand-computed rectilinear MST exactly, diameters and services match
|
||
the generator's truth, and the cable rows equal the generator's Manhattan
|
||
lengths with the allowance rounded up; `understand_foreign` — units flagged,
|
||
the outlier found, both plan copies separated, the service layers classified.
|
||
- **Correctness checks** (+5): `scale_check_detects_schematic`,
|
||
`pipe_takeoff_rmst_exact`, `cable_takeoff_roundup`, `diff_detects_known_edits`
|
||
(a move, a text change, an attribute change, an add and a delete, and nothing
|
||
else), `topology_known_defects`.
|
||
- **Field test** (local, not committed): `pipe_takeoff` and `cable_takeoff` on
|
||
the user's real pair, compared with their hand-made workbooks; the difference
|
||
is expected where the hand-made pipe takeoff measured the unscaled P&ID, and
|
||
the report explains each row.
|
||
- **Live smoke** `scripts/smoke_understand_com.py`: the snapshot, the takeoffs
|
||
and the diff on AutoCAD 2026, every document closed.
|