db5b4d3ef8f8b0911e71af9cbb3944639f8ee41b
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Umut CANandClaude Opus 5.5
|
ba41428c88 |
bench: matrix v7 adds takeoff_roundtrip and understand_foreign; five understanding checks; the live smoke and the field-test script
The evidence for track H, in the shape tracks A, E, B + G and F used.
- benchmarks/tasks_v7.py: 21 tasks, DEFAULT_MATRIX = "v7"; v6 to v2 stay
addressable. Both new tasks read the synthetic plant pair and nothing
else. takeoff_roundtrip: the P&ID called schematic, every routable run's
layout length equal to the rectilinear MST of its tags, sizes and services
as drawn (a return drawn on the supply layer included), the cable metres
rounded up after the allowance, an unstated power left empty.
understand_foreign: inches over millimetres flagged, the stray entity found
by handle, two plan copies separated, five layers classified. A test breaks
each input and watches its gate fail by name.
- Five correctness checks, every number worked out by hand in its docstring;
the published A/B report is re-run over 48 checks: 26 / 48 -> 48 / 48,
zero regressed.
- scripts/smoke_understand_com.py reads the pair through the live engine -
snapshots, the DWG route, both takeoffs and a diff - and closes every
document it opens; the CHANGELOG records what it returned.
scripts/field_test_takeoff.py runs the takeoffs on a user's own files,
refuses to write inside the repository and prints no quantity by default.
- The README token table is remeasured; check_doc_numbers derives the task
count from the v7 matrix; CLAUDE.md, both READMEs, the CHANGELOG and the
spec status line say what shipped.
One reader defect found on the way: describe.tag_occurrences read a wiring
callout ("Wiring to CP-2") as a place its panel is written, so the scale
check called CP-2 ambiguous and read "Wiring to CP1" as a phantom tag CP1 -
10 tags and 45 pairs (13.3 % within 10 %) where the generator placed 11 and
55 (21.8 %). A callout is no longer a tag occurrence, the rule
network.tag_occurrences already kept; two tests failed before the fix.
The smoke works only in documents it opened from its own temporary folder,
known by full path; it attaches with GetActiveObject only (the backend's
Dispatch fallback would launch AutoCAD), checks in the same COM-thread call
that the active document is its own before each snapshot, the DWG save and
the move, closes its documents with Close(False) and re-activates the
document that was active when it started. No new ActiveX member: Documents,
Item, FullName, ActiveDocument, Activate and Close are the ones the backend
already calls. Measured on AutoCAD 2026 (25.1s): exit 0, model-space types
equal file / live / DWG, export 0.104 s, no document of the run left open.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd
|
||
|
Umut CANandClaude Opus 5.5
|
0abbe55041 |
bench: matrix v6 adds arch_roundtrip; four architectural correctness checks; the live arch smoke
The evidence for track F, in the shape tracks A, E and B + G used. - benchmarks/tasks_v6.py: 19 tasks, DEFAULT_MATRIX = "v6"; v5 to v2 stay addressable. arch_roundtrip draws the example plan with arch_plan_from_spec and reads it back without the spec: rooms_detect must find each labelled room within 0.1 % of its label and of the net floor computed by hand (4825 x 5750 and 3825 x 5750), the schedules must list D1, D2, W1, W2, and the plan is gated on a zero-issue critique and a finalize score of at least 90. A test removes one room label and the gate fails naming arch_room_unlabelled. - Four correctness checks, every number worked out by hand in the docstring: L / T / X junction corners and wall areas, a 4 x 5 m room labelled 20.00 m2 rather than its 21.84 m2 axis area, a door interrupting both faces, and a plan of plain lines read as 20 and 15 m2 at confidence 0.6. The published A/B report is re-run over 43 checks: 26 / 43 -> 43 / 43, zero regressed. - scripts/smoke_arch_com.py runs the whole plan once on a live seat and closes every document it opens; the CHANGELOG records what it returned. - The token suite measures a TOOL_PACKS=core,arch row; check_doc_numbers derives the task count from the v6 matrix; CLAUDE.md, both READMEs and the spec status line say what shipped. - fix(com): the first live smoke exited 1 - the room reader found 0 wall segments. Measured on AutoCAD 2026 (2026-09-23): with a makepy cache, ModelSpace.Item(i) returns the gen_py IAcadEntity wrapper (type(item).__name__ == "IAcadEntity") and item.StartPoint on an AcDbLine raises AttributeError, so every entity_list row lost its type-specific properties (28 wall LINEs, 0 with start/end). _entity_info now re-dispatches exactly that wrapper through _com_unnarrow; after the fix 38 of 53 rows carry start/end and the smoke exits 0 with both rooms at 27 743 750 / 21 993 750 mm2 and 0 critique issues. Re-dispatch measured at ~1.7 ms per entity (dynamic vs gencache alike). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd |
||
|
Umut CANandClaude Opus 5.5
|
898e775f93 |
bench: matrix v5 adds mech_assembly; the correctness suite grows with the mechanical and sheet checks
The evidence for tracks B and G, in the shape tracks A and E used.
- benchmarks/tasks_v5.py: 18 tasks, DEFAULT_MATRIX = "v5"; v4, v3 and v2 stay
addressable so an earlier report reproduces exactly rather than being only
described. mech_assembly is roadmap criterion 3 - a flange-coupling sheet
built from the part model and the sheet standard, through the code the
tools call (apply_titleblock, draw_part/add_view/dimension_part,
insert_std_part, extract_records -> draw_bom_table, add_balloon), gated on
two numbers that cannot be negotiated: drawing_critique(focus=None) returns
zero issues and the finalize score is at least 90 (it scores 100, 18/18 on
the v5 matrix). The adapter returns the focuses that fired, so a
regression names itself, and a test drops one balloon to prove the gate
fails and names mech_bom_balloon_mismatch.
- Seven correctness checks (32 -> 39). mech_section_hatch_area is the one
that cannot be faked: a sleeve of length 60, outside diameter 40 and bore
20 cuts two faces totalling 1200 mm^2 exactly, read back out of the drawing
through entity_measure. sheet_frame_iso5457_a3 pins the A3 frame (20,10 to
410,287 inside the 420 x 297 trimmed sheet); mech_iso286_on_dimension pins
40 H7 at +0.025/0 through the layout; std_part_iso4014_m12 pins s = 18 and
k = 7.5 by value; mech_thread_unrepresented_is_caught is the seventh: a
quality gate that never fires is not a gate.
- scripts/smoke_mech_com.py executed every new COM path once on AutoCAD 2026
(25.1s): frame, title block, a sectioned shaft with 12 dimensions, an ISO
4014 block with 1 ACADMCP_MECH chunk, a weld symbol, the parts list read off
the drawing, a linked balloon, an xref attached / listed / detached, a
31687-byte .dwg (AC1032) through ActiveX SaveAs, critique 0 issues, exit 0,
every scratch document closed. It retries RPC_E_CALL_REJECTED (measured:
the first call after Documents.Add is refused) and tracks its documents by
the document list, since SaveAs renames them.
- Defects the evidence found, each fixed with a test that fails on the old
code:
* ezdxf dimension_diameter put the text beyond the SECOND chord point;
ActiveX AddDimDiametric puts it beyond the first (measured live: a d50
from (x,170) to (x,220), leader 10, text at (x,160)). Headlessly every
front-view diameter stood on its own chain dimension. ezdxf now follows
ActiveX.
* draw_part's anchor goes on MECH; ActiveX refuses entity.Layer on a
missing layer (measured -0x7ffdfff7 'Key not found'). draw.py ensures
every target layer before the first entity.
* bom.extract_records filtered on INSERT; ActiveX names it BLOCKREFERENCE
(ObjectName AcDbBlockReference) - the live parts list was empty.
* COM _entity_info reported no dimension text_position: ModelSpace.Item
narrows to IAcadEntity, which has no TextPosition (measured
AttributeError); read through _com_unnarrow (TextPosition measured on
AcDbDiametricDimension, WCS). dim_overlap compared bbox centres and a
diameter collided with its own bore.
* pid graph raised KeyError 'start' mid-critique on a LINE whose point read
AutoCAD refused while busy; such a line is left out.
* the validator read "ISO 4014 - M12x60" as fake dimension text.
- The published A/B report is re-run over the grown suite rather than
re-typed (26/39 -> 39/39, 13 fixed, 0 regressed), check_doc_numbers derives
the task count from the v5 matrix, the token table is remeasured with a
TOOL_PACKS=core,mech row (193 tools / 61,673 tokens), and the README's
collected-tests figure is retaken (3990).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd
|
||
|
Umut CANandClaude Opus 5
|
18c7c69da6 |
fix(task-25): page_setup_truth and settings_pdf_mediabox_a3 move the sheet to ANSI B before proving A3
A fresh Layout1 is already ISO A3 landscape (ezdxf R2018 default: 420 x 297, plot_rotation 0), so an A3-only /MediaBox read-back passed with page_setup_apply stubbed to a no-op: the A/B gate row and the v4 matrix task could not fail on the setter. Both now apply ANSI B (432 x 279, a size no fresh document has; spec section 9 names it) and read that PDF first, then apply ISO A3, read its PDF, and require the setter's changed.size_mm to have moved from B to A3. The stub fails at the first PDF. The moved-sheet comparison is +/-0.5 mm, the spec's tolerance, not exact: measured live on AutoCAD 2026 (scratch Documents.Add(), closed unsaved), the COM engine reports ANSI B as 431.8 x 279.4 and the DWG To PDF.pc3 file's MediaBox as 431.8 x 279.4 then 420.158 x 297.039 for A3; the task passes there with changed.size_mm [[431.8, 279.4], [420.0, 297.0]]. Headless it is [[432.0, 279.0], [420.0, 297.0]]. tests/test_benchmark_v4.py gains the mutation test the finding describes (setter stubbed -> both rows must fail; both passed on the old code) and a run of the gate row itself; the headless task test asserts the B stage. README rows and the tasks_v4 docstring say what the task now does; the README collected-tests figure follows the two new tests (3282 -> 3284). The regenerated A/B report against v1.5.1 is byte-identical (26 -> 32/32). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd |
||
|
Umut CANandClaude Opus 5
|
c2942d3c94 |
bench: matrix v4 adds page_setup_truth; correctness suite grows to 32 with three settings checks (A/B vs v1.5.1: 26 -> 32 / 32)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd |
||
|
Umut CANandClaude Opus 5
|
8d9c444b76 |
bench: matrix v4 adds pid_roundtrip; correctness suite grows to 29 with three P&ID checks
tasks_v4.py extends v3 with one task the reader can fail on its own: the example spec is drawn through pid_from_spec's code and read back by the graph builder, which never sees the spec. The runner, the chart and the doc-number gate now default to v4; --matrix v3 / v2 reproduce the earlier sets, and tests/test_benchmark_v3.py labels its full-matrix run "v3" explicitly now that the default has moved. The plan's adapter check expected the two unnumbered lines to come back with no line number; draw_line builds one from the default number_format (bare sequence when size/service/spec/insulation are empty), so the task asserts the two authored numbers on the two process lines and sequence- built XDATA numbers on the other two instead. The published A/B report still covers 26 checks; Task 21 re-runs it against v1.5.1, which is what clears the remaining doc-number drift. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd |