fix(task-25): page_setup_truth and settings_pdf_mediabox_a3 move the sheet to ANSI B before proving A3

A fresh Layout1 is already ISO A3 landscape (ezdxf R2018 default: 420 x 297,
plot_rotation 0), so an A3-only /MediaBox read-back passed with
page_setup_apply stubbed to a no-op: the A/B gate row and the v4 matrix task
could not fail on the setter. Both now apply ANSI B (432 x 279, a size no
fresh document has; spec section 9 names it) and read that PDF first, then
apply ISO A3, read its PDF, and require the setter's changed.size_mm to have
moved from B to A3. The stub fails at the first PDF.

The moved-sheet comparison is +/-0.5 mm, the spec's tolerance, not exact:
measured live on AutoCAD 2026 (scratch Documents.Add(), closed unsaved), the
COM engine reports ANSI B as 431.8 x 279.4 and the DWG To PDF.pc3 file's
MediaBox as 431.8 x 279.4 then 420.158 x 297.039 for A3; the task passes
there with changed.size_mm [[431.8, 279.4], [420.0, 297.0]]. Headless it is
[[432.0, 279.0], [420.0, 297.0]].

tests/test_benchmark_v4.py gains the mutation test the finding describes
(setter stubbed -> both rows must fail; both passed on the old code) and a
run of the gate row itself; the headless task test asserts the B stage.
README rows and the tasks_v4 docstring say what the task now does; the README
collected-tests figure follows the two new tests (3282 -> 3284). The
regenerated A/B report against v1.5.1 is byte-identical (26 -> 32/32).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd
This commit is contained in:
Umut CAN
2026-09-22 13:27:17 +03:00
co-authored by Claude Opus 5
parent c2942d3c94
commit 18c7c69da6
6 changed files with 134 additions and 14 deletions
+1 -1
View File
@@ -95,7 +95,7 @@ fail on their own:
| Task | Category | Verified against |
|---|---|---|
| `pid_roundtrip` | pid | the example spec drawn through `pid_from_spec`'s code, then read back by the graph builder, which never sees the spec: 5 nodes, 4 edges, 0 dangling ends, `confidence_min` 1.0, zero critique issues, an instrument index of exactly `FIC-101` wired to `FCV-101`, and the two authored line numbers back on the two process lines while the two unnumbered lines carry sequence-built numbers from XDATA |
| `page_setup_truth` | pagesetup | `page_setup_apply("Layout1", ISO_A3 landscape 1:1)` then `batch_plot`; the PDF's own `/MediaBox` must parse to 420 × 297 mm (±0.5), the page-setup list must say `ISO_A3` / `landscape` / `[420, 297]`, and the ctb must be a known one |
| `page_setup_truth` | pagesetup | `page_setup_apply("Layout1", ANSI_B landscape 1:1)` then `batch_plot` — the PDF's own `/MediaBox` must parse to 432 × 279 mm (±0.5), a size no fresh document has; then `page_setup_apply("Layout1", ISO_A3 landscape 1:1)` and `batch_plot` again — 420 × 297, the setter's `changed.size_mm` must read `[[432, 279], [420, 297]]`, the page-setup list must say `ISO_A3` / `landscape` / `[420, 297]`, and the ctb must be a known one. A setter that writes nothing fails at the first PDF (`tests/test_benchmark_v4.py` stubs it to prove that) |
The competitor reports carry no result for either, for the same reason they
carry none for the v3 five. `--matrix v3` reproduces the v1.5 set exactly.
+40 -5
View File
@@ -438,10 +438,29 @@ class AutoCADMCPProAdapter(BenchmarkAdapter):
setter's ``applied`` block is what we asked for, the page-setup list is
what the LAYOUT now says, and the PDF is what a printer would get. All
three have to agree with ISO 216's 420 × 297.
A fresh document's Layout1 is already A3 landscape, so that read-back
alone would pass with the setter stubbed out. The task first moves the
sheet to ANSI B — 432 × 279, a size no fresh document has — and reads
that PDF, then applies A3 and requires the setter to report the layout
moved from B to A3. The stub fails at the first PDF.
"""
from engineering.standards.papers import resolve_page_setup
from engineering.standards.plot import batch_plot
out_dir = self.artifact_dir / "page_setup_truth"
out_dir.mkdir(parents=True, exist_ok=True)
# Stage 1: a non-default sheet, proven by its PDF.
await self.backend.page_setup_apply(
"Layout1",
resolve_page_setup("ANSI_B", orientation="landscape", scale="1:1"),
)
before = await batch_plot(
self.backend, ["Layout1"], str(out_dir), "{drawing}-{layout}-ansi_b.pdf"
)
row_before = before["sheets"][0]
width_before, height_before = (float(v) for v in row_before["mediabox_mm"])
# Stage 2: the sheet under test.
setup = resolve_page_setup(
"ISO_A3",
orientation="landscape",
@@ -450,15 +469,29 @@ class AutoCADMCPProAdapter(BenchmarkAdapter):
)
applied = await self.backend.page_setup_apply("Layout1", setup)
listed = (await self.backend.page_setup_list("Layout1"))[0]
out_dir = self.artifact_dir / "page_setup_truth"
out_dir.mkdir(parents=True, exist_ok=True)
report = await batch_plot(self.backend, ["Layout1"], str(out_dir), "{drawing}-{layout}.pdf")
report = await batch_plot(
self.backend, ["Layout1"], str(out_dir), "{drawing}-{layout}-iso_a3.pdf"
)
row = report["sheets"][0]
width_mm, height_mm = (float(v) for v in row["mediabox_mm"])
size_listed = tuple(float(v) for v in listed["size_mm"])
size_changed = applied["changed"].get("size_mm")
# ±0.5 mm, the spec's own tolerance: the live engine measures ANSI B
# as 431.8 × 279.4 (11 × 17 in), the headless one as 432 × 279.
moved = size_changed or [[0.0, 0.0], [0.0, 0.0]]
moved_from_b = (
abs(float(moved[0][0]) - 432.0) < 0.5 and abs(float(moved[0][1]) - 279.0) < 0.5
)
moved_to_a3 = (
abs(float(moved[1][0]) - 420.0) < 0.5 and abs(float(moved[1][1]) - 297.0) < 0.5
)
passed = (
applied["ok"] is True
abs(width_before - 432.0) < 0.5
and abs(height_before - 279.0) < 0.5
and applied["ok"] is True
and applied["plot_style_known"] is True
and moved_from_b
and moved_to_a3
and listed["paper"] == "ISO_A3"
and listed["orientation"] == "landscape"
and size_listed == (420.0, 297.0)
@@ -473,10 +506,12 @@ class AutoCADMCPProAdapter(BenchmarkAdapter):
"orientation": listed["orientation"],
"size_mm_listed": list(size_listed),
"mediabox_mm": [width_mm, height_mm],
"mediabox_mm_before": [width_before, height_before],
"size_mm_changed": size_changed,
"pdf_bytes": row["bytes"],
"plot_style_known": applied["plot_style_known"],
},
[str(row["path"])],
[str(row_before["path"]), str(row["path"])],
)
async def _task_auditable_delivery(self):
+33 -5
View File
@@ -376,17 +376,45 @@ async def settings_layer_state_roundtrip():
)
def _sheet_near(size_mm, width_mm: float, height_mm: float) -> bool:
"""Within the spec's ±0.5 mm: COM measures ANSI B as 431.8 × 279.4."""
try:
w, h = (float(v) for v in size_mm)
except (TypeError, ValueError):
return False
return abs(w - width_mm) < 0.5 and abs(h - height_mm) < 0.5
async def settings_pdf_mediabox_a3():
"""Page setup evidence is the PDF's own /MediaBox (spec §8.3)."""
"""Page setup evidence is the PDF's own /MediaBox (spec §8.3).
A fresh document's Layout1 is already ISO A3 landscape, so an A3-only
read-back would pass with the setter stubbed out. The sheet is therefore
moved to ANSI B first — 432 × 279, a size no fresh document has (spec §9
names it) — and that PDF is read; then ISO A3 is applied and read again.
The A3 apply must also report the layout moved from B to A3, so the PDF
and the setter's own diff have to agree on where the sheet was.
"""
from engineering.standards.papers import resolve_page_setup
from engineering.standards.plot import batch_plot
b = await _b()
await b.page_setup_apply("Layout1", resolve_page_setup("ISO_A3"))
await b.page_setup_apply("Layout1", resolve_page_setup("ANSI_B"))
out_dir = tempfile.mkdtemp()
report = await batch_plot(b, ["Layout1"], out_dir, "{drawing}-{layout}.pdf")
width_mm, height_mm = report["sheets"][0]["mediabox_mm"]
return abs(width_mm - 420.0) < 0.5 and abs(height_mm - 297.0) < 0.5
report_b = await batch_plot(b, ["Layout1"], out_dir, "{drawing}-{layout}-ansi_b.pdf")
width_b, height_b = report_b["sheets"][0]["mediabox_mm"]
applied = await b.page_setup_apply("Layout1", resolve_page_setup("ISO_A3"))
report_a3 = await batch_plot(b, ["Layout1"], out_dir, "{drawing}-{layout}-iso_a3.pdf")
width_a3, height_a3 = report_a3["sheets"][0]["mediabox_mm"]
moved = applied["changed"].get("size_mm") or [[0.0, 0.0], [0.0, 0.0]]
return (
abs(width_b - 432.0) < 0.5
and abs(height_b - 279.0) < 0.5
and _sheet_near(moved[0], 432.0, 279.0)
and _sheet_near(moved[1], 420.0, 297.0)
and abs(width_a3 - 420.0) < 0.5
and abs(height_a3 - 297.0) < 0.5
)
CHECKS = {
+3 -1
View File
@@ -7,7 +7,9 @@ and the instrument index are each derived independently of the drawing code.
``page_setup_truth`` (track E) applies an ISO A3 landscape page setup to a
layout, plots it through ``batch_plot`` and reads the sheet size back from
the PDF's own ``/MediaBox`` — the setup is verified by the file, not by the
setter's return value.
setter's return value. A fresh Layout1 is already A3 landscape, so the task
moves the sheet to ANSI B (432 × 279) and reads that PDF first; a setter that
writes nothing fails there, not at the A3 read-back.
The published competitor reports were never asked either question; the chart
shows ``not_run`` for them, not zero.