bench: matrix v5 adds mech_assembly; the correctness suite grows with the mechanical and sheet checks

The evidence for tracks B and G, in the shape tracks A and E used.

- benchmarks/tasks_v5.py: 18 tasks, DEFAULT_MATRIX = "v5"; v4, v3 and v2 stay
  addressable so an earlier report reproduces exactly rather than being only
  described. mech_assembly is roadmap criterion 3 - a flange-coupling sheet
  built from the part model and the sheet standard, through the code the
  tools call (apply_titleblock, draw_part/add_view/dimension_part,
  insert_std_part, extract_records -> draw_bom_table, add_balloon), gated on
  two numbers that cannot be negotiated: drawing_critique(focus=None) returns
  zero issues and the finalize score is at least 90 (it scores 100, 18/18 on
  the v5 matrix). The adapter returns the focuses that fired, so a
  regression names itself, and a test drops one balloon to prove the gate
  fails and names mech_bom_balloon_mismatch.
- Seven correctness checks (32 -> 39). mech_section_hatch_area is the one
  that cannot be faked: a sleeve of length 60, outside diameter 40 and bore
  20 cuts two faces totalling 1200 mm^2 exactly, read back out of the drawing
  through entity_measure. sheet_frame_iso5457_a3 pins the A3 frame (20,10 to
  410,287 inside the 420 x 297 trimmed sheet); mech_iso286_on_dimension pins
  40 H7 at +0.025/0 through the layout; std_part_iso4014_m12 pins s = 18 and
  k = 7.5 by value; mech_thread_unrepresented_is_caught is the seventh: a
  quality gate that never fires is not a gate.
- scripts/smoke_mech_com.py executed every new COM path once on AutoCAD 2026
  (25.1s): frame, title block, a sectioned shaft with 12 dimensions, an ISO
  4014 block with 1 ACADMCP_MECH chunk, a weld symbol, the parts list read off
  the drawing, a linked balloon, an xref attached / listed / detached, a
  31687-byte .dwg (AC1032) through ActiveX SaveAs, critique 0 issues, exit 0,
  every scratch document closed. It retries RPC_E_CALL_REJECTED (measured:
  the first call after Documents.Add is refused) and tracks its documents by
  the document list, since SaveAs renames them.
- Defects the evidence found, each fixed with a test that fails on the old
  code:
  * ezdxf dimension_diameter put the text beyond the SECOND chord point;
    ActiveX AddDimDiametric puts it beyond the first (measured live: a d50
    from (x,170) to (x,220), leader 10, text at (x,160)). Headlessly every
    front-view diameter stood on its own chain dimension. ezdxf now follows
    ActiveX.
  * draw_part's anchor goes on MECH; ActiveX refuses entity.Layer on a
    missing layer (measured -0x7ffdfff7 'Key not found'). draw.py ensures
    every target layer before the first entity.
  * bom.extract_records filtered on INSERT; ActiveX names it BLOCKREFERENCE
    (ObjectName AcDbBlockReference) - the live parts list was empty.
  * COM _entity_info reported no dimension text_position: ModelSpace.Item
    narrows to IAcadEntity, which has no TextPosition (measured
    AttributeError); read through _com_unnarrow (TextPosition measured on
    AcDbDiametricDimension, WCS). dim_overlap compared bbox centres and a
    diameter collided with its own bore.
  * pid graph raised KeyError 'start' mid-critique on a LINE whose point read
    AutoCAD refused while busy; such a line is left out.
  * the validator read "ISO 4014 - M12x60" as fake dimension text.
- The published A/B report is re-run over the grown suite rather than
  re-typed (26/39 -> 39/39, 13 fixed, 0 regressed), check_doc_numbers derives
  the task count from the v5 matrix, the token table is remeasured with a
  TOOL_PACKS=core,mech row (193 tools / 61,673 tokens), and the README's
  collected-tests figure is retaken (3990).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd
This commit is contained in:
Umut CAN
2026-09-23 13:30:02 +03:00
co-authored by Claude Opus 5.5
parent 3b033033f9
commit 898e775f93
30 changed files with 1355 additions and 50 deletions
+62
View File
@@ -492,3 +492,65 @@ async def test_a_transaction_that_cannot_open_is_a_value_error(backend, monkeypa
monkeypatch.setattr(backend, "transaction_begin", busy)
with pytest.raises(ValueError, match="could not open a transaction"):
await draw_from_spec(backend, {"parts": [{"part": PLATE, "at": [0.0, 0.0]}]})
async def test_a_single_step_disk_dimensions_without_a_text_collision(backend):
"""The largest diameter's text used to land on its own segment's chain text.
The front-view diameters start at the upper chord end, and ActiveX puts a
diameter's text beyond that FIRST point - above the view, clear of the
chain rows (measured on AutoCAD 2026). The headless engine put it beyond
the second point instead, so ⌀90 landed exactly on the "15" of a 15 mm
disk and ``dim_overlap`` fired on every single-step part the default
``mech_part_draw`` produced headlessly. Found by the v5 ``mech_assembly``
benchmark and the live smoke together.
"""
from engineering.critique import run_critique
disk = build_part(
{
"kind": "revolved",
"name": "FLANGE",
"material": "cast_iron",
"segments": [{"length": 15.0, "d_outer": 90.0, "d_inner": 40.0}],
}
)
await draw_part(backend, disk, at=(0.0, 0.0), views=("front",), dimension=True)
assert await run_critique(backend, ["dim_overlap"]) == []
async def test_every_layer_exists_before_an_entity_is_put_on_it(backend, monkeypatch):
"""ActiveX will not create a layer on assignment - the headless engine will.
Measured on AutoCAD 2026 by scripts/smoke_mech_com.py: ``entity.Layer =
"MECH"`` raised ``-0x7ffdfff7 ('Key not found')`` after every view of the
part had been drawn, so ``mech_part_draw`` could not finish on a live
seat. This backend is made as strict as ActiveX, and the whole part -
views, anchor, dimensions - has to draw on it.
"""
creators = (
"entity_create_point",
"entity_create_line",
"entity_create_circle",
"entity_create_arc",
"entity_create_polyline",
"entity_create_text",
"entity_create_hatch",
)
for name in creators:
real = getattr(backend, name)
async def strict(*args, _real=real, **kwargs):
layer = kwargs.get("layer")
known = {row.name.lower() for row in await backend.layer_list()}
if layer and layer.lower() not in known:
raise RuntimeError(f"Key not found: layer {layer!r}")
return await _real(*args, **kwargs)
monkeypatch.setattr(backend, name, strict)
drawn = await draw_part(
backend, build_part(SHAFT), at=(0.0, 0.0), views=("front", "side"), dimension=True
)
assert drawn["part_id"]
assert ANCHOR_LAYER.lower() in {row.name.lower() for row in await backend.layer_list()}