docs: track H (understanding foreign drawings) design spec
Read a drawing somebody else made and turn it into a deliverable: a one-call drawing report (units that lie, stray extents, plan copies, multilingual layer meaning), a scale check between a P&ID and its layout, pipe and cable takeoffs whose topology comes from the P&ID and whose lengths come from the layout, a revision diff by geometric signature, and topology checks. Grounded in a real client project read locally (never committed): its P&ID proved to be unscaled against its layout, which no tool had flagged. Six tools (+ pack plant), three topo_* focuses, the office extra, a live DXF snapshot through Document.Export (measured: the document is not renamed), and synthetic fixtures that reproduce the field findings. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd
This commit is contained in:
Umut CAN
co-authored by
Claude Opus 5.5
Claude Opus 5.5
parent
0abbe55041
commit
18995bf854
@@ -0,0 +1,305 @@
|
||||
# v1.6 Track H — understanding foreign drawings, takeoffs, diff and topology
|
||||
|
||||
**Date:** 2026-09-24 · **Status:** design approved in conversation ·
|
||||
**Parent:** `2026-09-15-v1.6-roadmap-design.md` (track H; track D stays separate
|
||||
and follows) · **Baseline:** main after track F (`0abbe55`, 241 tools)
|
||||
|
||||
## 1. Summary
|
||||
|
||||
Every track so far taught the server to *draw*. This one teaches it to *read* a
|
||||
drawing somebody else made and turn it into a deliverable — the job a plant
|
||||
engineer does by hand when a client sends a P&ID and a layout and asks how much
|
||||
pipe and cable the job needs.
|
||||
|
||||
The design is grounded in a real client project from the user's archive, read
|
||||
on 2026-09-24 (the files are confidential and never enter this repository; see
|
||||
§14): a process plant with a 35,590-entity P&ID (96 layers, labels in
|
||||
Russian, layer names in English and Dutch, 296 block names, most anonymous),
|
||||
a 22,399-entity layout, a hand-made pipe takeoff and a hand-made cable
|
||||
takeoff. Four findings shape everything below:
|
||||
|
||||
1. **Today's reader cannot read it.** `pid_graph` found 441 nodes on that P&ID —
|
||||
427 of them `unknown_block` at confidence 0.3 — and 663 edges, every one of
|
||||
line class `unknown`, in 73 s. Its confidences were honest; its answer was
|
||||
not useful. It cannot tell a CIP supply line from a product line, read a
|
||||
diameter label, or recognise a Russian service word.
|
||||
2. **The P&ID is not to scale.** Matching equipment tags between the P&ID and
|
||||
the layout, distances between the same pairs of tanks differ by factors from
|
||||
0.1 to 5.6; inside one room the P&ID is a consistent ~1.28× stretch of the
|
||||
plan, inside another it is rearranged outright. A pipe length measured on
|
||||
that P&ID is a length of drawn line, not of pipe — and the hand-made pipe
|
||||
takeoff was measured exactly that way, while the hand-made cable takeoff of
|
||||
the same project states in its own method sheet that the P&ID is unscaled.
|
||||
No tool warned anybody. This track makes that check a tool.
|
||||
3. **The layout lies about its units.** `INSUNITS` says inches; the geometry is
|
||||
millimetres (a 5.9 km wall total, a 150–280 m building). One stray entity
|
||||
drags the extents to −6,450 km, and two copies of the plan sit side by side
|
||||
in model space.
|
||||
4. **Real drawings are big.** The two DXFs are 55 MB and 188 MB; `drawing_open`
|
||||
refuses anything over 50 MB, and reading 35,000 entities one COM call at a
|
||||
time takes minutes where a DXF snapshot takes half a second.
|
||||
|
||||
## 2. Scope
|
||||
|
||||
**In:** six tools (§12), three `topo_*` critique focuses, the `office` extra
|
||||
(`openpyxl`), a DXF-snapshot reader for the live engine, the raised size limit,
|
||||
synthetic fixtures that reproduce the field findings, two benchmark tasks,
|
||||
correctness checks and a local field-test script.
|
||||
|
||||
**Out of 1.6 (recorded):**
|
||||
|
||||
- Plant 3D `.dcf` reading (track D, next).
|
||||
- Cable cross-section *standards*: no IEC 60364-5-52 table is authored. A
|
||||
section is chosen only from a rule table the caller supplies (the shop's own
|
||||
rule); without one, no section is proposed. Voltage-drop calculation is out.
|
||||
- Routing pipes around walls and equipment. A layout length is an orthogonal
|
||||
estimate between connected equipment (§9) — the method the user's cable
|
||||
takeoff already uses — and is labelled an estimate.
|
||||
- OCR of raster underlays and text inside PDF-imported hatch geometry.
|
||||
|
||||
## 3. Architecture
|
||||
|
||||
```
|
||||
engineering/understand/
|
||||
__init__.py
|
||||
snapshot.py one bulk read of a drawing: ezdxf directly, or a DXF snapshot of
|
||||
the live document through Document.Export (§4)
|
||||
vocab.py multilingual keyword tables (EN / TR / RU / NL / DE): disciplines,
|
||||
services, equipment kinds, room words, supply/return words
|
||||
labels.py parsers: diameters (Ø51, Ø25,4, SMS51, DN20, 20x27, 1,5"),
|
||||
equipment tags, electrical data (kW, phases, voltage, +N, VFD)
|
||||
units.py declared vs inferred units, robust extents, outliers
|
||||
clusters.py spatial clusters of model-space content (sheets, plan copies)
|
||||
describe.py drawing_understand: the report assembled from the above
|
||||
scale.py drawing_scale_check: tag matching and ratio statistics
|
||||
network.py line networks: junction merge, T-split, fittings bridged, runs,
|
||||
service and diameter per edge, equipment attachment
|
||||
route.py orthogonal lengths: Manhattan distance, rectilinear MST
|
||||
takeoff.py pipe_takeoff and cable_takeoff rows, statuses, totals
|
||||
report.py the takeoff workbook (Metraj / Özet / Kontrol / Metodoloji), CSV
|
||||
diff.py entity signatures, matching, the change report
|
||||
topology.py dangling endpoints, near misses, interior crossings
|
||||
critique.py the three topo_* focuses
|
||||
draw.py the async layer: snapshot, markup (revision clouds), exports
|
||||
scripts/field_test_takeoff.py local only: runs the takeoffs on the user's files
|
||||
```
|
||||
|
||||
Everything except `snapshot.py` and `draw.py` is pure: entity records in,
|
||||
results out. The entity record is a small frozen dataclass (type, layer,
|
||||
handle, geometry, text, block name, attributes) filled once per drawing, so a
|
||||
35,000-entity drawing is read once and every analysis runs in memory.
|
||||
|
||||
## 4. Reading a drawing (the snapshot)
|
||||
|
||||
- **Headless:** `ezdxf.readfile` on the path, or the document already open.
|
||||
- **Live:** `Document.Export(<path without extension>, "DXF", <selection set of
|
||||
all>)` writes a DXF copy **without renaming the document** — measured on
|
||||
AutoCAD 2026 on 2026-09-24: the document kept its name, an unsaved document
|
||||
exported too, and ezdxf read the result. The extension argument must be
|
||||
`"DXF"`; `".dxf"` fails with *Invalid argument* (measured). The snapshot goes
|
||||
to a temporary folder and is deleted afterwards. Whether paper-space content
|
||||
and xrefs reach the snapshot is measured by the plan's first live task and
|
||||
stated in the tool docstrings.
|
||||
- `MAX_DXF_BYTES` default rises from 50 MB to 512 MB, and the refusal names the
|
||||
size and the variable (a real layout here was 188 MB).
|
||||
|
||||
Every tool in this track is read-only unless it says otherwise (only
|
||||
`drawing_diff(markup=True)` and the takeoff export write anything, and the
|
||||
export writes a file, never the drawing).
|
||||
|
||||
## 5. `drawing_understand`
|
||||
|
||||
One call, one report, every claim with its evidence and a confidence:
|
||||
|
||||
- **Units:** declared `INSUNITS` versus the unit the geometry implies (text
|
||||
heights, typical wall / door sizes, dimension values, overall size). A
|
||||
disagreement is a warning with the numbers, never a silent correction.
|
||||
- **Extents:** declared versus robust (quantile) extents; the outlier entities
|
||||
that stretch them, by handle.
|
||||
- **Clusters:** separate bodies of content in model space — sheets, plan copies,
|
||||
details — each with its box, its entity count and its most telling labels.
|
||||
- **Layers:** every layer with its entity count and drawn length, and the
|
||||
discipline / service it most likely carries (piping · product, CIP supply,
|
||||
CIP return, glycol, steam, condensate, water, air, drain; electrical; walls;
|
||||
equipment; annotation), from the multilingual vocabulary and the texts found
|
||||
on it; confidence 0.9 for a direct keyword, lower for weaker evidence.
|
||||
- **Equipment tags**, **rooms** (label words in five languages — *room*, *oda*,
|
||||
*mahal*, *помещение*, *ruimte* — plus faces of wall layers through track F's
|
||||
finder), **text languages**, **blocks** (named / anonymous, kinds by name),
|
||||
**layouts** and **title-block attributes**.
|
||||
|
||||
## 6. `drawing_scale_check`
|
||||
|
||||
Compares a drawing with a reference drawing of the same plant. Equipment tags
|
||||
found once in each are matched; for every pair of matched tags at least 2 m
|
||||
apart the ratio of their distances is taken. The report gives the matched tags,
|
||||
the median ratio, the interquartile range, the share of pairs within ±10 % of
|
||||
the median, the same statistics per room or cluster, and a verdict:
|
||||
`to_scale` (≥ 80 % of pairs within ±10 %; the factor is reported, so a drawing
|
||||
at a different but uniform scale is recognised), `schematic` (≤ 50 %), or
|
||||
`partly` in between. Tag labels are not equipment centres; the ±10 % band and
|
||||
the 2 m floor are there to absorb that, and the report says so.
|
||||
|
||||
## 7. Line networks (the shared engine)
|
||||
|
||||
Segments (LINE, LWPOLYLINE, POLYLINE, ARC) on the chosen layers become a graph:
|
||||
endpoints merged within a tolerance, segments split where another ends on them
|
||||
(T-junctions), and a segment that ends at an inline fitting (an INSERT whose
|
||||
box it touches — reducer, valve, coupling) continued through it. A connected
|
||||
component is a **run**.
|
||||
|
||||
- **Service** of an edge: the layer's vocabulary class, overridden by supply /
|
||||
return words within a search distance (`ПОДАЧА`, `ВОЗВРАТ`, `BESLEME`,
|
||||
`DÖNÜŞ`, `SUPPLY`, `RETURN`, `AANVOER`, `RETOUR`), carried along the run. Layer
|
||||
alone is not trusted to split CIP supply from return — the field P&ID proved
|
||||
why.
|
||||
- **Diameter** of an edge: the diameter label nearest to it within the search
|
||||
distance (*direct*), else carried along the run from a labelled edge until a
|
||||
reducer or a conflicting label (*continuity*), else *unassigned*. Labels are
|
||||
kept exactly as drawn: `Ø51`, `SMS51`, `DN20`, `Ø25,4` and `Ø25` are different
|
||||
sizes unless the drawing equates them.
|
||||
- **Equipment attachment:** a run end inside (or within tolerance of) the box of
|
||||
the geometry a tag labels is attached to that tag.
|
||||
|
||||
## 8. `pipe_takeoff`
|
||||
|
||||
Rows by room × service × diameter, in the layout of the user's own takeoff
|
||||
workbook:
|
||||
|
||||
- **Lengths.** `length_source="auto"` (default) runs the scale check between
|
||||
the P&ID and the layout. If the P&ID is `to_scale`, runs are measured on it.
|
||||
If not, the length of each run is estimated on the layout: the positions of
|
||||
the tags it connects, joined by a **rectilinear minimum spanning tree**
|
||||
(Manhattan distance; a two-ended run is |Δx| + |Δy|), plus
|
||||
`vertical_allowance` per connection (default 0, stated in the report). The
|
||||
schematic length drawn on the P&ID is kept alongside as a comparison column,
|
||||
never as the answer. `"pid"` and `"layout"` choose a source and the report
|
||||
says which was chosen; `"pid"` on a P&ID the scale check calls `schematic` is
|
||||
refused unless `force=True`. With no layout given, `auto` measures on the P&ID
|
||||
and every row carries `scale_verified: false`, stated on the method sheet.
|
||||
An orthogonal route runs along x first, then y — deterministic, and stated.
|
||||
- **Status:** **A** — every end on a tag found on the layout and the diameter
|
||||
direct; **B** — diameter by continuity; **C** — not routable (an end not on
|
||||
equipment, a tag missing from the layout): reported on the *Kontrol* sheet
|
||||
with its schematic length only, counted in the total and flagged.
|
||||
- **Rooms:** a routed length is apportioned to the layout rooms its orthogonal
|
||||
route crosses (room polygons from the layout's wall faces and room labels);
|
||||
without room polygons, to the room label nearest the run.
|
||||
- **Scope:** `services` defaults to product + CIP supply + CIP return (the
|
||||
user's takeoff); utilities are included only on request.
|
||||
- **Output:** a workbook — *Metraj* (rows with direct and continuity lengths
|
||||
split, status, allowance, allowance-included), *Özet* (by service, by
|
||||
diameter, by room), *Kontrol* (unassigned diameters and unroutable runs with
|
||||
the reason), *Metodoloji* (generated: sources, the scale-check result, the
|
||||
rules applied, the tolerances) — with Turkish, English or Russian headers
|
||||
(`lang`), written as XLSX when `openpyxl` is installed (the new `office`
|
||||
extra) and as CSV always. `allowance` defaults to 0.20 and is its own column,
|
||||
never mixed into the net length.
|
||||
|
||||
## 9. `cable_takeoff`
|
||||
|
||||
One row per electrical load, the user's cable method made repeatable:
|
||||
|
||||
- **Power and supply** from the P&ID's electrical texts near each tag (`P =
|
||||
18,5 кВт`, `3 ф. 400 В + N + PE`, `VFD`, in Russian, Turkish and English
|
||||
spellings). A load with no stated power keeps an empty power cell and an open
|
||||
item — **power is never invented**.
|
||||
- **Panel** from the layout's electrical callouts (`Wiring to CP1` and its
|
||||
variants) or the P&ID; panel names normalised (`CP 1` → `CP-1`).
|
||||
- **Length:** Manhattan distance on the layout from the tag to its panel, plus
|
||||
the allowance, rounded up to the metre (`ROUNDUP(length × (1 + allowance))`,
|
||||
as the user's sheet does). With no layout given, lengths stay empty and each
|
||||
row gets an open item — the P&ID is not a length source for cables.
|
||||
- **Room totals** of known power, with the rule for panel packages versus their
|
||||
child motors stated in the method sheet so no load is counted twice.
|
||||
- **Section:** only from a caller-supplied `section_rules` table; otherwise
|
||||
blank.
|
||||
- Same workbook shape and languages as §8, with an *Open items* list.
|
||||
|
||||
## 10. `drawing_diff`
|
||||
|
||||
Two revisions of one drawing (paths, or the current document against a path).
|
||||
Entity signatures (type, layer, geometry rounded to a tolerance; text content
|
||||
and position; block name, insertion, rotation, scale and attributes) are
|
||||
matched exactly first, then by handle, then by type + layer + nearest position
|
||||
within a move tolerance; what is left is added or removed. The report gives
|
||||
counts by layer and type, the changed entities with what changed (moved by, text
|
||||
from → to, attribute from → to, radius from → to), block-definition changes and
|
||||
a capped list per category. `markup=True` draws revision clouds around the
|
||||
clusters of change on the newer drawing with track G's revision tools — the
|
||||
only write in the tool, and only on request.
|
||||
|
||||
## 11. `drawing_topology_check` and the `topo_*` focuses
|
||||
|
||||
Dangling endpoints, near misses (two ends, or an end and a segment, within a
|
||||
gap tolerance but not joined) and interior crossings (two segments crossing away
|
||||
from their ends), on the layers given or, by default, on the network layers
|
||||
`drawing_understand` classifies with confidence ≥ 0.9. The three critique
|
||||
focuses (`topo_dangling_endpoint`, `topo_near_miss`, `topo_interior_crossing`)
|
||||
run the same engine under `focus=None`, **excluding layers a semantic focus
|
||||
already covers** (P&ID line layers → `pid_*`, architectural walls → `arch_*`),
|
||||
so one defect is never scored twice; on a drawing made entirely by this server
|
||||
they are silent.
|
||||
|
||||
## 12. Tool surface, packs, lean
|
||||
|
||||
| Tool | Pack | Writes? |
|
||||
|---|---|---|
|
||||
| `drawing_understand` | core (lean) | no |
|
||||
| `drawing_scale_check` | core | no |
|
||||
| `drawing_diff` | core | only `markup=True` |
|
||||
| `drawing_topology_check` | core | no |
|
||||
| `pipe_takeoff` | `plant` | a workbook file |
|
||||
| `cable_takeoff` | `plant` | a workbook file |
|
||||
|
||||
**+6 → 247 tools.** New pack `plant` (track D's readers join it later). Lean
|
||||
gains `drawing_understand` (→ 65). Aliases in five languages (metraj, metre
|
||||
cetveli, boru listesi, kablo listesi, спецификация, ведомость, takeoff, BOQ,
|
||||
fark, karşılaştır, revizyon farkı, compare, DWGCOMPARE, birim, ölçek, scale
|
||||
check).
|
||||
|
||||
## 13. Refusals
|
||||
|
||||
Before any work: a missing or unreadable file, a DXF over the size limit (with
|
||||
the variable named), `lang` outside tr/en/ru, a non-positive tolerance or
|
||||
allowance outside 0–1, `length_source="pid"` with a P&ID the scale check calls
|
||||
schematic *without* `force=True` (the refusal quotes the statistics), an XLSX
|
||||
export without `openpyxl` (the `xlsx_write` capability, naming the extra; CSV
|
||||
still written).
|
||||
|
||||
## 14. Confidentiality
|
||||
|
||||
Client drawings, spreadsheets and anything derived from them that identifies
|
||||
the client or its business — file names, client and site names, takeoff
|
||||
quantities — never enter the repository, its tests, its
|
||||
benchmark results or its commit messages. The fixtures are **synthetic**
|
||||
(§15). `scripts/field_test_takeoff.py` runs the tools on files the user points
|
||||
it at and writes its report next to them, outside the repository; its output is
|
||||
never committed.
|
||||
|
||||
## 15. Evidence
|
||||
|
||||
- **Synthetic plant pair** (`tests/fixtures/plant_pair.py`): a generator that
|
||||
builds a schematic P&ID DXF and a layout DXF of one small plant with known
|
||||
answers — equipment tags present in both, the P&ID deliberately stretched
|
||||
1.3× in one room and rearranged in another, service layers with English /
|
||||
Dutch names, Russian supply / return words, `Ø` / `SMS` / `DN` / `a x b`
|
||||
labels, a reducer, an inline valve, electrical texts in Russian, `Wiring to
|
||||
CPn` callouts, a layout with `INSUNITS` set to inches over millimetre
|
||||
geometry, one far outlier entity and two plan copies.
|
||||
- **Benchmark tasks** (matrix v7, 21 tasks): `takeoff_roundtrip` — the scale
|
||||
check calls the synthetic P&ID `schematic`, every pipe run's layout length
|
||||
equals the hand-computed rectilinear MST exactly, diameters and services match
|
||||
the generator's truth, and the cable rows equal the generator's Manhattan
|
||||
lengths with the allowance rounded up; `understand_foreign` — units flagged,
|
||||
the outlier found, both plan copies separated, the service layers classified.
|
||||
- **Correctness checks** (+5): `scale_check_detects_schematic`,
|
||||
`pipe_takeoff_rmst_exact`, `cable_takeoff_roundup`, `diff_detects_known_edits`
|
||||
(a move, a text change, an attribute change, an add and a delete, and nothing
|
||||
else), `topology_known_defects`.
|
||||
- **Field test** (local, not committed): `pipe_takeoff` and `cable_takeoff` on
|
||||
the user's real pair, compared with their hand-made workbooks; the difference
|
||||
is expected where the hand-made pipe takeoff measured the unscaled P&ID, and
|
||||
the report explains each row.
|
||||
- **Live smoke** `scripts/smoke_understand_com.py`: the snapshot, the takeoffs
|
||||
and the diff on AutoCAD 2026, every document closed.
|
||||
Reference in New Issue
Block a user