bench: matrix v6 adds arch_roundtrip; four architectural correctness checks; the live arch smoke

The evidence for track F, in the shape tracks A, E and B + G used.

- benchmarks/tasks_v6.py: 19 tasks, DEFAULT_MATRIX = "v6"; v5 to v2 stay
  addressable. arch_roundtrip draws the example plan with
  arch_plan_from_spec and reads it back without the spec: rooms_detect must
  find each labelled room within 0.1 % of its label and of the net floor
  computed by hand (4825 x 5750 and 3825 x 5750), the schedules must list
  D1, D2, W1, W2, and the plan is gated on a zero-issue critique and a
  finalize score of at least 90. A test removes one room label and the gate
  fails naming arch_room_unlabelled.
- Four correctness checks, every number worked out by hand in the docstring:
  L / T / X junction corners and wall areas, a 4 x 5 m room labelled 20.00 m2
  rather than its 21.84 m2 axis area, a door interrupting both faces, and a
  plan of plain lines read as 20 and 15 m2 at confidence 0.6. The published
  A/B report is re-run over 43 checks: 26 / 43 -> 43 / 43, zero regressed.
- scripts/smoke_arch_com.py runs the whole plan once on a live seat and
  closes every document it opens; the CHANGELOG records what it returned.
- The token suite measures a TOOL_PACKS=core,arch row; check_doc_numbers
  derives the task count from the v6 matrix; CLAUDE.md, both READMEs and
  the spec status line say what shipped.
- fix(com): the first live smoke exited 1 - the room reader found 0 wall
  segments. Measured on AutoCAD 2026 (2026-09-23): with a makepy cache,
  ModelSpace.Item(i) returns the gen_py IAcadEntity wrapper
  (type(item).__name__ == "IAcadEntity") and item.StartPoint on an
  AcDbLine raises AttributeError, so every entity_list row lost its
  type-specific properties (28 wall LINEs, 0 with start/end). _entity_info
  now re-dispatches exactly that wrapper through _com_unnarrow; after the
  fix 38 of 53 rows carry start/end and the smoke exits 0 with both rooms
  at 27 743 750 / 21 993 750 mm2 and 0 critique issues. Re-dispatch
  measured at ~1.7 ms per entity (dynamic vs gencache alike).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd

benchmarks/README.md

@@ -117,6 +117,16 @@ the live seat a missing `MECH` layer and a parts list that could not see a
 `BLOCKREFERENCE`; each is fixed with a test. The competitor reports carry no
 result for it.
 
+### Matrix v6 (v1.6) — the architectural plan
+
+`tasks_v6.py` keeps the v5 eighteen unchanged and adds track F's evidence:
+
+| Task | Category | Verified against |
+|---|---|---|
+| `arch_roundtrip` | arch | the plan of `engineering/arch/spec.py::EXAMPLE_SPEC` — a 9 × 6 m brick ring at 250 mm split by a 100 mm AAC wall, an entrance door, an interior door, two windows, a straight stair, two labelled rooms, the exterior chains and the door / window / room schedules — drawn in one transaction by `draw_plan_from_spec`, then read back without the spec: `rooms_detect` on the wall layer must find each labelled room within 0.1 % of the area its label carries **and** of the net floor computed by hand (4825 × 5750 = 27 743 750 and 3825 × 5750 = 21 993 750 mm²), `schedule_rows` must list exactly D1, D2 and W1, W2, and the saved plan is gated like `mech_assembly`: `run_critique(focus=None)` returns **zero** issues and `combine(validator, critique)` scores at least **90**. `tests/test_benchmark_v6.py` removes one room label to prove the gate fails and names `arch_room_unlabelled` |
+
+The competitor reports carry no result for it.
+
 ## Live competitor lane (v1.4)
 
 Two competitor adapters now execute the exact same task matrix, black-box over
@@ -234,15 +244,15 @@ python benchmarks/compare_versions.py --json results.json
 
 ### Result — this branch (1.6.0-dev) vs v1.5.1 (release gate)
 
-39 checks, ezdxf backend, one subprocess per check. Machine-readable report:
+43 checks, ezdxf backend, one subprocess per check. Machine-readable report:
 [`results/published/ab-v1.5.1-vs-v1.6.0-dev.json`](results/published/ab-v1.5.1-vs-v1.6.0-dev.json).
 
 | Version | Checks passing | Pass rate | Fixed | Regressed |
 |---------|----------------|-----------|-------|-----------|
-| **v1.5.1** (baseline)     | 26 / 39 | 66.7 % | — | — |
-| **v1.6.0-dev** (this branch) | 39 / 39 | 100 % | 13 | **0** |
+| **v1.5.1** (baseline)     | 26 / 43 | 60.5 % | — | — |
+| **v1.6.0-dev** (this branch) | 43 / 43 | 100 % | 17 | **0** |
 
-The thirteen are all `miss → pass` — v1.5.1 has none of the methods. Seven
+The seventeen are all `miss → pass` — v1.5.1 has none of the methods. Four are track F: `arch_junction_l_t_x` (L, T and X junction outlines against hand-computed corners and wall areas), `arch_room_area_net` (a 4 × 5 m room between 200 mm walls is labelled 20.00 m², not its 21.84 m² axis area), `arch_opening_cuts_wall` (both faces interrupted across a door, two jambs) and `arch_rooms_detect_foreign` (plain lines on a WALLS layer read as 20 and 15 m² at confidence 0.6, nothing written). Seven
 are tracks B and G: `mech_part_roundtrip` (a part read back from its own
 `ACADMCP_MECH` XDATA equals the part drawn), `mech_section_hatch_area` (a
 sleeve 60 long, ⌀40 outside, ⌀20 bore: a full section cuts two 60 × 10 faces,

benchmarks/adapters/autocad_mcp_pro.py

@@ -1,4 +1,4 @@
-"""Reference adapter for this repository, covering the v5 task matrix.
+"""Reference adapter for this repository, covering the v6 task matrix.
 
 Every task verifies against a value worked out independently of the code under
 test — a closed-form area, a count of entities placed on purpose, a token
@@ -12,7 +12,7 @@ import math
 from pathlib import Path
 
 from benchmarks.adapters.base import BenchmarkAdapter, TaskResult
-from benchmarks.tasks_v5 import TaskSpec
+from benchmarks.tasks_v6 import TaskSpec
 from engineering.delivery import deliver_drawing
 from engineering.layers import ensure_engineering_layers, ensure_standard_linetypes
 from engineering.refiner import refine_drawing
@@ -40,6 +40,15 @@ DISCOVERY_TOP_N = 3
 #: rather than derived from the measurement, so the measurement can miss it.
 TOKEN_CEILING = 2_000
 
+#: The net floor of the two rooms of ``engineering/arch/spec.py::EXAMPLE_SPEC``,
+#: computed by hand: the 250 mm ring on axis (0,0)-(9000,6000) has its inner
+#: faces at x = 125 / 8875 and y = 125 / 5875, the 100 mm wall on x = 5000 its
+#: faces at 4950 / 5050. Room 01: 4825 x 5750; room 02: 3825 x 5750.
+ROUNDTRIP_AREAS_MM2: dict[str, float] = {
+    "01": 4825.0 * 5750.0,
+    "02": 3825.0 * 5750.0,
+}
+
 
 class AutoCADMCPProAdapter(BenchmarkAdapter):
     name = "autocad-mcp-pro"
@@ -682,6 +691,94 @@ class AutoCADMCPProAdapter(BenchmarkAdapter):
             [str(sheet)],
         )
 
+    async def _task_arch_roundtrip(self):
+        """A two-room plan drawn from one spec and read back off the drawing.
+
+        Track F's evidence (spec §10). ``arch_plan_from_spec`` draws the plan of
+        ``engineering/arch/spec.py::EXAMPLE_SPEC`` - a 9 x 6 m brick ring at 250
+        mm split by a 100 mm AAC wall, an entrance door, an interior door, two
+        windows, a straight stair, two labelled rooms, the exterior chains and
+        the three schedules. Then the drawing is read, never the spec:
+
+        * ``rooms_detect`` on the wall layer must find each labelled room, and
+          its area must agree with the area the label's record carries within
+          0.1 %, and with the net floor computed by hand in
+          ``ROUNDTRIP_AREAS_MM2`` - so a label that typed the axis area fails;
+        * the door and window schedules read off the records must list exactly
+          the drawn tags;
+        * ``run_critique(focus=None)`` returns zero issues and the finalize
+          score - ``combine(validator.summary, critique)`` over the saved plan -
+          is at least 90.
+
+        Any one slipping fails the task, and the metrics name what slipped.
+        """
+        from engineering import DrawingValidator
+        from engineering.arch.critique import point_in_loop
+        from engineering.arch.draw import read_plan
+        from engineering.arch.layers import ARCH_ROLE_LAYER
+        from engineering.arch.rooms import rooms_detect
+        from engineering.arch.schedule import schedule_rows
+        from engineering.arch.spec import EXAMPLE_SPEC, draw_plan_from_spec
+        from engineering.critique import run_critique
+        from engineering.scoring import combine
+
+        drawn = await draw_plan_from_spec(self.backend, EXAMPLE_SPEC)
+        plan = await read_plan(self.backend)
+        detected = await rooms_detect(self.backend, layers=[ARCH_ROLE_LAYER["wall"]])
+        faces = list(detected.get("rooms") or [])
+
+        areas: dict[str, dict] = {}
+        for room in plan["rooms"]:
+            face = next((f for f in faces if point_in_loop(room.at, f["loop"])), None)
+            detected_area = float(face["area"]) if face is not None else 0.0
+            hand = ROUNDTRIP_AREAS_MM2.get(str(room.number))
+            areas[str(room.number)] = {
+                "label_mm2": float(room.area),
+                "detected_mm2": detected_area,
+                "hand_mm2": hand,
+                "agrees": face is not None
+                and hand is not None
+                and abs(detected_area - float(room.area)) <= 1e-3 * float(room.area)
+                and abs(detected_area - hand) <= 1e-3 * hand,
+            }
+        rooms_ok = (
+            set(areas) == set(ROUNDTRIP_AREAS_MM2)
+            and len(faces) == len(ROUNDTRIP_AREAS_MM2)
+            and all(row["agrees"] for row in areas.values())
+        )
+
+        door_tags = sorted(str(row["tag"]) for row in schedule_rows("doors", plan, lang="en"))
+        window_tags = sorted(str(row["tag"]) for row in schedule_rows("windows", plan, lang="en"))
+        tags_ok = door_tags == ["D1", "D2"] and window_tags == ["W1", "W2"]
+
+        sheet = self.artifact_dir / "arch_roundtrip.dxf"
+        await self.backend.drawing_save_as(str(sheet))
+        issues = await run_critique(self.backend, None)
+        validation = await DrawingValidator().run(self.backend)
+        summary = {"error": 0, "warning": 0, "info": 0}
+        for issue in issues:
+            summary[issue.severity] = summary.get(issue.severity, 0) + 1
+        score = combine(validation.summary, summary)
+        passed = rooms_ok and tags_ok and len(issues) == 0 and float(score["score"]) >= 90.0
+        return (
+            passed,
+            {
+                "rooms": areas,
+                "faces_detected": len(faces),
+                "confidence_min": detected.get("confidence_min"),
+                "door_tags": door_tags,
+                "window_tags": window_tags,
+                "omitted": drawn.get("omitted"),
+                "critique_issues": len(issues),
+                "critique_focuses": sorted({issue.focus for issue in issues}),
+                "score": float(score["score"]),
+                "grade": score["grade"],
+                "invalidity_ratio": score["invalidity_ratio"],
+                "validator_findings": sorted(f.code for f in validation.findings),
+            },
+            [str(sheet)],
+        )
+
     async def _task_auditable_delivery(self):
         await ensure_engineering_layers(self.backend)
         await self.backend.entity_create_line(0, 0, 10, 0, layer="GEOMETRY")

benchmarks/correctness_suite.py

@@ -642,6 +642,182 @@ async def std_part_iso4014_m12():
     return payload["kind"] == "std_part" and "M12" in str(payload["designation"])
 
 
+# ── Architecture (track F) ──────────────────────────────────────────────────
+
+
+def _arch_points(geometry) -> set:
+    """Every endpoint of a drawn face or jamb, rounded to a micron."""
+    return {
+        (round(float(x), 6), round(float(y), 6))
+        for line in (*geometry.faces, *geometry.jambs)
+        for x, y in (line.p1, line.p2)
+    }
+
+
+def _arch_passes_through(geometry, point) -> bool:
+    """Does a drawn face run *through* ``point`` (not merely end on it)?"""
+    px, py = point
+    for line in geometry.faces:
+        (x1, y1), (x2, y2) = line.p1, line.p2
+        length = math.hypot(x2 - x1, y2 - y1)
+        cross = (x2 - x1) * (py - y1) - (y2 - y1) * (px - x1)
+        along = ((px - x1) * (x2 - x1) + (py - y1) * (y2 - y1)) / (length * length)
+        if abs(cross) <= 1e-6 * length and 1e-9 < along < 1.0 - 1e-9:
+            return True
+    return False
+
+
+def _arch_region_area(geometry) -> float:
+    from engineering.measure import polygon_area_perimeter
+
+    return sum(
+        polygon_area_perimeter([(x, y, 0.0) for x, y in loop], True)[0] for loop in geometry.regions
+    )
+
+
+async def arch_junction_l_t_x():
+    """The three junction kinds, against coordinates worked out by hand.
+
+    L: one 200 mm centre wall turning at (4000, 0) - the outer corner is
+    (4100, -100), the inner (3900, 100), and a mitred corner keeps the area at
+    axis length x thickness: 7000 x 200 = 1 400 000 mm².
+    T: a 100 mm stem on x = 3000 ending on the axis of a 200 mm wall - its faces
+    stop on the near face at (2950, 100) and (3050, 100), the near face is open
+    between them, the far face runs through; 6000 x 200 + 2900 x 100 = 1 490 000.
+    X: two 200 mm walls crossing at (3000, 0) - the first stays whole, the second
+    stops on its faces, all four faces are cut at x = 2900 / 3100 and y = +-100;
+    6000 x 200 + 2 x 2900 x 200 = 2 360 000.
+    """
+    from engineering.arch.model import wall_from_dict
+    from engineering.arch.walls import wall_geometry
+
+    def wall(wall_id, axis, thickness):
+        return wall_from_dict({"id": wall_id, "axis": axis, "thickness": thickness})
+
+    l_geo = wall_geometry([wall("L", [[0, 0], [4000, 0], [4000, 3000]], 200)], poche=False)
+    t_geo = wall_geometry(
+        [wall("A", [[0, 0], [6000, 0]], 200), wall("B", [[3000, 0], [3000, 3000]], 100)],
+        poche=False,
+    )
+    x_geo = wall_geometry(
+        [wall("A", [[0, 0], [6000, 0]], 200), wall("B", [[3000, -3000], [3000, 3000]], 200)],
+        poche=False,
+    )
+    l_ok = (
+        {(4100.0, -100.0), (3900.0, 100.0)} <= _arch_points(l_geo)
+        and not {(4100.0, 100.0), (3900.0, -100.0)} & _arch_points(l_geo)
+        and abs(_arch_region_area(l_geo) - 1_400_000.0) < 1e-6
+    )
+    t_ok = (
+        {(2950.0, 100.0), (3050.0, 100.0)} <= _arch_points(t_geo)
+        and not _arch_passes_through(t_geo, (3000.0, 100.0))
+        and _arch_passes_through(t_geo, (3000.0, -100.0))
+        and abs(_arch_region_area(t_geo) - 1_490_000.0) < 1e-6
+    )
+    x_ok = (
+        {(2900.0, 100.0), (3100.0, 100.0), (2900.0, -100.0), (3100.0, -100.0)}
+        <= _arch_points(x_geo)
+        and not any(
+            _arch_passes_through(x_geo, p)
+            for p in ((3000.0, 100.0), (3000.0, -100.0), (2900.0, 0.0), (3100.0, 0.0))
+        )
+        and abs(_arch_region_area(x_geo) - 2_360_000.0) < 1e-6
+    )
+    return l_ok and t_ok and x_ok
+
+
+async def arch_room_area_net():
+    """A 4 x 5 m room between 200 mm walls measures 20.00 m², not its axis area.
+
+    The ring's axis runs (-100, -100) to (4100, 5100), so its inner faces are
+    x = 0 / 4000 and y = 0 / 5000: the floor is 4000 x 5000 = 20 000 000 mm²,
+    while the axis encloses 4200 x 5200 = 21 840 000. The number is read back
+    out of the room record the label carries, and the label text must say it.
+    """
+    from engineering.arch.draw import draw_walls, read_plan
+    from engineering.arch.lang import fmt_area_m2
+    from engineering.arch.model import wall_from_dict
+    from engineering.arch.rooms import label_room
+
+    b = await _b()
+    ring = wall_from_dict(
+        {
+            "id": "W1",
+            "axis": [[-100, -100], [4100, -100], [4100, 5100], [-100, 5100]],
+            "thickness": 200,
+            "closed": True,
+        }
+    )
+    await draw_walls(b, [ring])
+    await label_room(b, {"id": "R1", "name": "ROOM", "number": "01", "at": [2000, 2500]})
+    (room,) = (await read_plan(b))["rooms"]
+    texts = [e.properties.get("text", "") for e in await b.entity_list(type_filter="TEXT")]
+    return (
+        abs(float(room.area) - 20_000_000.0) < 0.5
+        and fmt_area_m2(room.area, "en") == "20.00 m²"
+        and any("20.00 m²" in str(text) for text in texts)
+    )
+
+
+async def arch_opening_cuts_wall():
+    """A 900 door at 1000 along a 5 m, 200 mm wall interrupts both faces.
+
+    No face runs through x = 1450 (the middle of the door) on either face line
+    y = +-100 while one still runs through x = 500, and two jambs close the cut
+    at x = 1000 and x = 1900.
+    """
+    from engineering.arch.model import opening_from_dict, wall_from_dict
+    from engineering.arch.walls import wall_geometry
+
+    wall = wall_from_dict({"id": "W1", "axis": [[0, 0], [5000, 0]], "thickness": 200})
+    door = opening_from_dict(
+        {"id": "D1", "wall": "W1", "kind": "door", "offset": 1000, "width": 900}
+    )
+    geo = wall_geometry([wall], [door], poche=False)
+    jambs = {
+        round(float(line.p1[0]), 6)
+        for line in geo.jambs
+        if abs(line.p1[0] - line.p2[0]) < 1e-9 and abs(abs(line.p1[1] - line.p2[1]) - 200.0) < 1e-9
+    }
+    return (
+        not _arch_passes_through(geo, (1450.0, 100.0))
+        and not _arch_passes_through(geo, (1450.0, -100.0))
+        and _arch_passes_through(geo, (500.0, 100.0))
+        and {1000.0, 1900.0} <= jambs
+        and tuple(geo.omitted) == ()
+    )
+
+
+async def arch_rooms_detect_foreign():
+    """A plan of plain lines - no record of ours - yields the net areas.
+
+    Two rooms drawn as the inner outlines of 200 mm walls on a layer named
+    WALLS: 4000 x 5000 and 3000 x 5000, inside an outer outline (-200, -200)
+    to (7400, 5200). The reader must find exactly those two faces - 20 000 000
+    and 15 000 000 mm² - at the foreign-plan confidence 0.6, and must not have
+    written anything.
+    """
+    from engineering.arch.rooms import rooms_detect
+
+    b = await _b()
+    for x0, y0, x1, y1 in (
+        (-200.0, -200.0, 7400.0, 5200.0),
+        (0.0, 0.0, 4000.0, 5000.0),
+        (4200.0, 0.0, 7200.0, 5000.0),
+    ):
+        corners = [(x0, y0), (x1, y0), (x1, y1), (x0, y1)]
+        for a, c in zip(corners, corners[1:] + corners[:1], strict=True):
+            await b.entity_create_line(a[0], a[1], c[0], c[1], layer="WALLS")
+    before = len(await b.entity_list(limit=1000))
+    found = await rooms_detect(b)
+    areas = sorted(round(float(room["area"]), 3) for room in found["rooms"])
+    return (
+        areas == [15_000_000.0, 20_000_000.0]
+        and all(float(room["confidence"]) == 0.6 for room in found["rooms"])
+        and len(await b.entity_list(limit=1000)) == before
+    )
+
+
 CHECKS = {
     "core_line_length": (core_line_length, "Core"),
     "core_circle_radius": (core_circle_radius, "Core"),
@@ -682,6 +858,10 @@ CHECKS = {
     "std_part_iso4014_m12": (std_part_iso4014_m12, "Mechanical"),
     "sheet_frame_iso5457_a3": (sheet_frame_iso5457_a3, "Sheet"),
     "bom_balloon_link": (bom_balloon_link, "Sheet"),
+    "arch_junction_l_t_x": (arch_junction_l_t_x, "Architecture"),
+    "arch_room_area_net": (arch_room_area_net, "Architecture"),
+    "arch_opening_cuts_wall": (arch_opening_cuts_wall, "Architecture"),
+    "arch_rooms_detect_foreign": (arch_rooms_detect_foreign, "Architecture"),
 }
 
 

benchmarks/render_matrix_chart.py

@@ -13,7 +13,7 @@ import json
 from pathlib import Path
 from typing import Any
 
-from benchmarks.tasks_v5 import TASKS_V5
+from benchmarks.tasks_v6 import TASKS_V6
 
 ROOT = Path(__file__).resolve().parents[1]
 DEFAULT_INPUT_DIR = ROOT / "benchmarks" / "results" / "published"
@@ -66,8 +66,8 @@ def render_chart(reports: list[dict[str, Any]], output: Path) -> None:
 
     matplotlib.rcParams["svg.hashsalt"] = "autocad-mcp-taskmatrix-v1"
 
-    task_ids = [task.task_id for task in TASKS_V5]
-    task_labels = [f"{task.task_id}  ({task.category})" for task in TASKS_V5]
+    task_ids = [task.task_id for task in TASKS_V6]
+    task_labels = [f"{task.task_id}  ({task.category})" for task in TASKS_V6]
     servers = [report["adapter"] for report in reports]
     status_by_server = {
         report["adapter"]: {item["task_id"]: item["status"] for item in report["results"]}

benchmarks/results/published/ab-v1.5.1-vs-v1.6.0-dev.json

@@ -1,12 +1,12 @@
 {
   "summary": {
     "baseline_ref": "v1.5.1",
-    "total": 39,
+    "total": 43,
     "old_pass": 26,
-    "new_pass": 39,
-    "old_pct": 66.7,
+    "new_pass": 43,
+    "old_pct": 60.5,
     "new_pct": 100.0,
-    "fixed": 13,
+    "fixed": 17,
     "regressed": 0
   },
   "rows": [
@@ -243,6 +243,30 @@
       "category": "Sheet",
       "old": "miss",
       "new": "pass"
+    },
+    {
+      "check": "arch_junction_l_t_x",
+      "category": "Architecture",
+      "old": "miss",
+      "new": "pass"
+    },
+    {
+      "check": "arch_room_area_net",
+      "category": "Architecture",
+      "old": "miss",
+      "new": "pass"
+    },
+    {
+      "check": "arch_opening_cuts_wall",
+      "category": "Architecture",
+      "old": "miss",
+      "new": "pass"
+    },
+    {
+      "check": "arch_rooms_detect_foreign",
+      "category": "Architecture",
+      "old": "miss",
+      "new": "pass"
     }
   ]
 }

benchmarks/run_competitors.py

@@ -16,7 +16,7 @@ from pathlib import Path
 
 from backends.capability import UnsupportedCapabilityError
 from benchmarks.adapters.base import BenchmarkAdapter, TaskResult
-from benchmarks.tasks_v5 import DEFAULT_MATRIX, MATRICES, TaskSpec, task_by_id
+from benchmarks.tasks_v6 import DEFAULT_MATRIX, MATRICES, TaskSpec, task_by_id
 
 
 def _load_registry() -> dict[str, dict]:

benchmarks/tasks_v6.py

@@ -0,0 +1,54 @@
+"""The v6 task matrix: v5 plus what track F added.
+
+``arch_roundtrip`` (track F) is the roadmap evidence for the plan model: a
+two-room plan - an entrance door, an interior door, two windows and a stair -
+drawn by ``arch_plan_from_spec``, then read back by ``arch_rooms_detect``,
+which never sees the spec. Each room's detected area has to agree with the area
+its label carries within 0.1 %, and both with the net floor computed by hand;
+the door and window schedules have to list the drawn tags; and the sheet is
+gated on the same two numbers as ``mech_assembly``: ``drawing_critique
+(focus=None)`` returns zero issues and the finalize score is at least 90.
+
+The published competitor reports were never asked this question; the chart
+shows ``not_run`` for them, not zero.
+"""
+
+from __future__ import annotations
+
+from benchmarks.tasks_v2 import TASKS_V2
+from benchmarks.tasks_v3 import TASKS_V3
+from benchmarks.tasks_v4 import TASKS_V4
+from benchmarks.tasks_v5 import TASKS_V5, TaskSpec
+
+#: Added in v1.6.0 by track F.
+NEW_TASKS_V6: tuple[TaskSpec, ...] = (
+    TaskSpec(
+        "arch_roundtrip",
+        "arch",
+        "Draw a two-room plan from one spec, read its rooms back from the drawing "
+        "within 0.1 %, schedule its tags, with a clean critique and a finalize score "
+        "of at least 90",
+        1.0,
+    ),
+)
+
+TASKS_V6: tuple[TaskSpec, ...] = TASKS_V5 + NEW_TASKS_V6
+
+#: Runner-selectable task sets. v2 to v5 stay addressable so an earlier report
+#: can be reproduced exactly rather than only described.
+MATRICES: dict[str, tuple[TaskSpec, ...]] = {
+    "v2": TASKS_V2,
+    "v3": TASKS_V3,
+    "v4": TASKS_V4,
+    "v5": TASKS_V5,
+    "v6": TASKS_V6,
+}
+
+DEFAULT_MATRIX = "v6"
+
+
+def task_by_id(task_id: str, matrix: str = DEFAULT_MATRIX) -> TaskSpec:
+    for task in MATRICES[matrix]:
+        if task.task_id == task_id:
+            return task
+    raise KeyError(f"{task_id} is not in matrix {matrix}")

benchmarks/token_suite.py

@@ -521,6 +521,16 @@ async def idle_lane(counter: Any, turns: int = SESSION_TURNS) -> list[dict[str,
                 turns,
             )
         )
+        config.settings.tool_packs = "core,arch"
+        rows.append(
+            await _idle_row(
+                "packs_core_arch",
+                "TOOL_PACKS=core,arch - an architectural client that never opens a "
+                "P&ID, a machine part or a live seat.",
+                counter,
+                turns,
+            )
+        )
         config.settings.tool_packs = previous_packs
         srv._apply_discovery_mode("search")
         rows.append(