Umut CANandClaude Opus 5.5
ba41428c88
The evidence for track H, in the shape tracks A, E, B + G and F used.
- benchmarks/tasks_v7.py: 21 tasks, DEFAULT_MATRIX = "v7"; v6 to v2 stay
addressable. Both new tasks read the synthetic plant pair and nothing
else. takeoff_roundtrip: the P&ID called schematic, every routable run's
layout length equal to the rectilinear MST of its tags, sizes and services
as drawn (a return drawn on the supply layer included), the cable metres
rounded up after the allowance, an unstated power left empty.
understand_foreign: inches over millimetres flagged, the stray entity found
by handle, two plan copies separated, five layers classified. A test breaks
each input and watches its gate fail by name.
- Five correctness checks, every number worked out by hand in its docstring;
the published A/B report is re-run over 48 checks: 26 / 48 -> 48 / 48,
zero regressed.
- scripts/smoke_understand_com.py reads the pair through the live engine -
snapshots, the DWG route, both takeoffs and a diff - and closes every
document it opens; the CHANGELOG records what it returned.
scripts/field_test_takeoff.py runs the takeoffs on a user's own files,
refuses to write inside the repository and prints no quantity by default.
- The README token table is remeasured; check_doc_numbers derives the task
count from the v7 matrix; CLAUDE.md, both READMEs, the CHANGELOG and the
spec status line say what shipped.
One reader defect found on the way: describe.tag_occurrences read a wiring
callout ("Wiring to CP-2") as a place its panel is written, so the scale
check called CP-2 ambiguous and read "Wiring to CP1" as a phantom tag CP1 -
10 tags and 45 pairs (13.3 % within 10 %) where the generator placed 11 and
55 (21.8 %). A callout is no longer a tag occurrence, the rule
network.tag_occurrences already kept; two tests failed before the fix.
The smoke works only in documents it opened from its own temporary folder,
known by full path; it attaches with GetActiveObject only (the backend's
Dispatch fallback would launch AutoCAD), checks in the same COM-thread call
that the active document is its own before each snapshot, the DWG save and
the move, closes its documents with Close(False) and re-activates the
document that was active when it started. No new ActiveX member: Documents,
Item, FullName, ActiveDocument, Activate and Close are the ones the backend
already calls. Measured on AutoCAD 2026 (25.1s): exit 0, model-space types
equal file / live / DWG, export 0.104 s, no document of the run left open.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd
72 lines
2.6 KiB
Python
72 lines
2.6 KiB
Python
"""The v7 task matrix: v6 plus what track H added.
|
|
|
|
Track H teaches the server to read drawings somebody else made, so its two
|
|
tasks are gated on a drawing pair whose answers are known in advance - the
|
|
synthetic plant pair of ``tests/fixtures/plant_pair.py``, which reproduces
|
|
every field finding of the design spec with invented names:
|
|
|
|
* ``takeoff_roundtrip`` - the scale check calls the P&ID ``schematic``; every
|
|
routable pipe run's layout length equals the rectilinear minimum spanning
|
|
tree of its tags exactly; its size, service and status match the generator's
|
|
truth; the cable rows equal the Manhattan lengths from each load to its panel
|
|
with the allowance rounded up to the metre.
|
|
* ``understand_foreign`` - one ``describe`` of the layout flags inches declared
|
|
over millimetre geometry, finds the stray entity by handle and separates the
|
|
two plan copies; one ``describe`` of the P&ID classifies its English and
|
|
Dutch service layers.
|
|
|
|
The published competitor reports were never asked these questions; the chart
|
|
shows ``not_run`` for them, not zero.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from benchmarks.tasks_v2 import TASKS_V2
|
|
from benchmarks.tasks_v3 import TASKS_V3
|
|
from benchmarks.tasks_v4 import TASKS_V4
|
|
from benchmarks.tasks_v5 import TASKS_V5
|
|
from benchmarks.tasks_v6 import TASKS_V6, TaskSpec
|
|
|
|
#: Added in v1.6.0 by track H.
|
|
NEW_TASKS_V7: tuple[TaskSpec, ...] = (
|
|
TaskSpec(
|
|
"takeoff_roundtrip",
|
|
"plant",
|
|
"Take pipe and cable quantities off a schematic P&ID and its layout: the P&ID "
|
|
"called schematic, every run's layout length equal to the rectilinear MST of "
|
|
"its tags, sizes and services as drawn, cable metres rounded up after the "
|
|
"allowance",
|
|
1.0,
|
|
),
|
|
TaskSpec(
|
|
"understand_foreign",
|
|
"understand",
|
|
"Read a foreign layout in one call: inches declared over millimetres flagged, "
|
|
"the stray entity found, both plan copies separated, the service layers of its "
|
|
"P&ID classified",
|
|
1.0,
|
|
),
|
|
)
|
|
|
|
TASKS_V7: tuple[TaskSpec, ...] = TASKS_V6 + NEW_TASKS_V7
|
|
|
|
#: Runner-selectable task sets. v2 to v6 stay addressable so an earlier report
|
|
#: can be reproduced exactly rather than only described.
|
|
MATRICES: dict[str, tuple[TaskSpec, ...]] = {
|
|
"v2": TASKS_V2,
|
|
"v3": TASKS_V3,
|
|
"v4": TASKS_V4,
|
|
"v5": TASKS_V5,
|
|
"v6": TASKS_V6,
|
|
"v7": TASKS_V7,
|
|
}
|
|
|
|
DEFAULT_MATRIX = "v7"
|
|
|
|
|
|
def task_by_id(task_id: str, matrix: str = DEFAULT_MATRIX) -> TaskSpec:
|
|
for task in MATRICES[matrix]:
|
|
if task.task_id == task_id:
|
|
return task
|
|
raise KeyError(f"{task_id} is not in matrix {matrix}")
|