Umut CANandClaude Opus 5
2be5e4180d
scripts/smoke_pid_com.py runs the whole track against the live AutoCAD in a new document. ComBackend.connect() is lazy by design, so the script reads reachability from system_status()["connected"] rather than from connect() as the plan sketched, and closes its document without saving unless --keep is passed. On this machine (AutoCAD 2026, "25.1s (LMS Tech)") it exits 0: 5 nodes, 4 edges, 0 dangling, confidence_min 1.0, 0 critique issues, 2 XDATA chunks — identical to the headless engine on the same spec. The first attempt exited 2 on a stale win32com gen_py cache (a package folder holding only __pycache__); an environment fault, no backend change needed. A/B against v1.5.1 published as ab-v1.5.1-vs-v1.6.0-dev.json: 26/29 -> 29/29, three miss -> pass, zero regressed. check_doc_numbers.py now names the report and the two row labels in one place (AB_REPORT, AB_*_LABEL), and README.md / benchmarks/README.md carry the new rows; the v1.5.0 and v1.4.0 tables stay as history. README: P&ID row, the showcase and catalogue images, the token table re-measured (161 / 44,930 by default, 152 / 41,420 with TOOL_PACKS=core, lean 50 / 14,007), every stale count reconciled (154 -> 166 registered, 149 -> 161 advertised, corpus 158/672 -> 162/731 by the same count, 19 -> 20 groups, 6 -> 8 resources, six-task matrix with pid_roundtrip), a "Known limitations of the P&ID track" subsection, and the snapshot line at this machine's 2423 collected tests (CI has not run on this tree; noted in the changelog with the pre-existing 1369 vs 1363 drift). CLAUDE.md: section 17, sections 5 and 7 counts, TOOL_PACKS, the two 1.6 contracts and the reader rules. Spec status -> implemented. The plan and spec markdown were reformatted by ruff (fenced Python only) because `ruff format --check .` was already red on them at HEAD and the gate requires the whole tree clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd
188 lines
3.7 KiB
JSON
188 lines
3.7 KiB
JSON
{
|
|
"summary": {
|
|
"baseline_ref": "v1.5.1",
|
|
"total": 29,
|
|
"old_pass": 26,
|
|
"new_pass": 29,
|
|
"old_pct": 89.7,
|
|
"new_pct": 100.0,
|
|
"fixed": 3,
|
|
"regressed": 0
|
|
},
|
|
"rows": [
|
|
{
|
|
"check": "core_line_length",
|
|
"category": "Core",
|
|
"old": "pass",
|
|
"new": "pass"
|
|
},
|
|
{
|
|
"check": "core_circle_radius",
|
|
"category": "Core",
|
|
"old": "pass",
|
|
"new": "pass"
|
|
},
|
|
{
|
|
"check": "core_layer_create",
|
|
"category": "Core",
|
|
"old": "pass",
|
|
"new": "pass"
|
|
},
|
|
{
|
|
"check": "core_polyline_closed",
|
|
"category": "Core",
|
|
"old": "pass",
|
|
"new": "pass"
|
|
},
|
|
{
|
|
"check": "core_linear_dim",
|
|
"category": "Core",
|
|
"old": "pass",
|
|
"new": "pass"
|
|
},
|
|
{
|
|
"check": "core_save_dxf_roundtrip",
|
|
"category": "Core",
|
|
"old": "pass",
|
|
"new": "pass"
|
|
},
|
|
{
|
|
"check": "dim_aligned_no_error",
|
|
"category": "Dimensions",
|
|
"old": "pass",
|
|
"new": "pass"
|
|
},
|
|
{
|
|
"check": "dim_angular_no_error",
|
|
"category": "Dimensions",
|
|
"old": "pass",
|
|
"new": "pass"
|
|
},
|
|
{
|
|
"check": "array_polar_360_distinct",
|
|
"category": "Modify",
|
|
"old": "pass",
|
|
"new": "pass"
|
|
},
|
|
{
|
|
"check": "point_intersection_line_line",
|
|
"category": "Geometry",
|
|
"old": "pass",
|
|
"new": "pass"
|
|
},
|
|
{
|
|
"check": "point_tangent_external",
|
|
"category": "Geometry",
|
|
"old": "pass",
|
|
"new": "pass"
|
|
},
|
|
{
|
|
"check": "arc_has_length",
|
|
"category": "Query",
|
|
"old": "pass",
|
|
"new": "pass"
|
|
},
|
|
{
|
|
"check": "arc_select_by_length",
|
|
"category": "Query",
|
|
"old": "pass",
|
|
"new": "pass"
|
|
},
|
|
{
|
|
"check": "ezdxf_bounding_box",
|
|
"category": "Query",
|
|
"old": "pass",
|
|
"new": "pass"
|
|
},
|
|
{
|
|
"check": "mtext_rotation_roundtrip",
|
|
"category": "Entities",
|
|
"old": "pass",
|
|
"new": "pass"
|
|
},
|
|
{
|
|
"check": "screenshot_png",
|
|
"category": "Render",
|
|
"old": "pass",
|
|
"new": "pass"
|
|
},
|
|
{
|
|
"check": "dim_overlap_critique_fires",
|
|
"category": "Quality gate",
|
|
"old": "pass",
|
|
"new": "pass"
|
|
},
|
|
{
|
|
"check": "iso13567_dim_layer",
|
|
"category": "Quality gate",
|
|
"old": "pass",
|
|
"new": "pass"
|
|
},
|
|
{
|
|
"check": "construction_left_iso_caught",
|
|
"category": "Quality gate",
|
|
"old": "pass",
|
|
"new": "pass"
|
|
},
|
|
{
|
|
"check": "gear_no_self_overlap",
|
|
"category": "Engineering",
|
|
"old": "pass",
|
|
"new": "pass"
|
|
},
|
|
{
|
|
"check": "center_linetype_applied",
|
|
"category": "Entities",
|
|
"old": "pass",
|
|
"new": "pass"
|
|
},
|
|
{
|
|
"check": "hatch_area_subtracts_its_island",
|
|
"category": "Measurement",
|
|
"old": "pass",
|
|
"new": "pass"
|
|
},
|
|
{
|
|
"check": "boundary_area_agrees_with_measure",
|
|
"category": "Measurement",
|
|
"old": "pass",
|
|
"new": "pass"
|
|
},
|
|
{
|
|
"check": "two_vertex_circle_has_area",
|
|
"category": "Measurement",
|
|
"old": "pass",
|
|
"new": "pass"
|
|
},
|
|
{
|
|
"check": "diameter_dim_measures_the_diameter",
|
|
"category": "Dimensions",
|
|
"old": "pass",
|
|
"new": "pass"
|
|
},
|
|
{
|
|
"check": "radius_dim_ignores_the_leader_length",
|
|
"category": "Dimensions",
|
|
"old": "pass",
|
|
"new": "pass"
|
|
},
|
|
{
|
|
"check": "pid_block_define_attdef_roundtrip",
|
|
"category": "P&ID",
|
|
"old": "miss",
|
|
"new": "pass"
|
|
},
|
|
{
|
|
"check": "pid_tag_parse_fic",
|
|
"category": "P&ID",
|
|
"old": "miss",
|
|
"new": "pass"
|
|
},
|
|
{
|
|
"check": "pid_graph_edge_count",
|
|
"category": "P&ID",
|
|
"old": "miss",
|
|
"new": "pass"
|
|
}
|
|
]
|
|
} |