anatoly/autocad-mcp-pro
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

docs(pid): track A shipped — CLAUDE.md, README, changelog with the live COM smoke record

Browse Source 
scripts/smoke_pid_com.py runs the whole track against the live AutoCAD in
a new document. ComBackend.connect() is lazy by design, so the script reads
reachability from system_status()["connected"] rather than from connect()
as the plan sketched, and closes its document without saving unless --keep
is passed. On this machine (AutoCAD 2026, "25.1s (LMS Tech)") it exits 0:
5 nodes, 4 edges, 0 dangling, confidence_min 1.0, 0 critique issues,
2 XDATA chunks — identical to the headless engine on the same spec. The
first attempt exited 2 on a stale win32com gen_py cache (a package folder
holding only __pycache__); an environment fault, no backend change needed.

A/B against v1.5.1 published as ab-v1.5.1-vs-v1.6.0-dev.json: 26/29 ->
29/29, three miss -> pass, zero regressed. check_doc_numbers.py now names
the report and the two row labels in one place (AB_REPORT, AB_*_LABEL),
and README.md / benchmarks/README.md carry the new rows; the v1.5.0 and
v1.4.0 tables stay as history.

README: P&ID row, the showcase and catalogue images, the token table
re-measured (161 / 44,930 by default, 152 / 41,420 with TOOL_PACKS=core,
lean 50 / 14,007), every stale count reconciled (154 -> 166 registered,
149 -> 161 advertised, corpus 158/672 -> 162/731 by the same count, 19 ->
20 groups, 6 -> 8 resources, six-task matrix with pid_roundtrip), a
"Known limitations of the P&ID track" subsection, and the snapshot line at
this machine's 2423 collected tests (CI has not run on this tree; noted in
the changelog with the pre-existing 1369 vs 1363 drift).

CLAUDE.md: section 17, sections 5 and 7 counts, TOOL_PACKS, the two 1.6
contracts and the reader rules. Spec status -> implemented. The plan and
spec markdown were reformatted by ruff (fenced Python only) because
`ruff format --check .` was already red on them at HEAD and the gate
requires the whole tree clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd
This commit is contained in:
Umut CAN
2026-09-16 13:34:06 +03:00
co-authored by Claude Opus 5
parent 8d9c444b76
commit 2be5e4180d
9 changed files with 2604 additions and 789 deletions
Download Patch File Download Diff File Expand all files Collapse all files
benchmarks/README.md
+23 -7
View File
@@ -214,29 +214,45 @@ python benchmarks/compare_versions.py v1.0.0 # vs a tag/ref
python benchmarks/compare_versions.py --json results.json
```
### Result — v1.5.1 vs v1.5.0 (release gate)
### Result — this branch (1.6.0-dev) vs v1.5.1 (release gate)
29 checks, ezdxf backend, one subprocess per check. Machine-readable report:
[`results/published/ab-v1.5.1-vs-v1.6.0-dev.json`](results/published/ab-v1.5.1-vs-v1.6.0-dev.json).
| Version | Checks passing | Pass rate | Fixed | Regressed |
|---------|----------------|-----------|-------|-----------|
| **v1.5.1** (baseline) | 26 / 29 | 89.7 % | — | — |
| **v1.6.0-dev** (this branch) | 29 / 29 | 100 % | 3 | **0** |
The three are the P&ID checks Track A added — `pid_block_define_attdef_roundtrip`,
`pid_tag_parse_fic`, `pid_graph_edge_count` — all `miss → pass`: v1.5.1 has
none of the methods. Every one of the 26 checks it was released on still
passes, so the track added capability without moving a number it had already
earned.
### Result — v1.5.1 vs v1.5.0
26 checks at the time, ezdxf backend, one subprocess per check. Machine-readable report:
[`results/published/ab-v1.5.0-vs-v1.5.1.json`](results/published/ab-v1.5.0-vs-v1.5.1.json).
| Version | Checks passing | Pass rate | Fixed | Regressed |
|---------|----------------|-----------|-------|-----------|
| **v1.5.0** (baseline) | 24 / 26 | 92.3 % | — | — |
| **v1.5.1** (this release) | 26 / 26 | 100 % | 2 | **0** |
| v1.5.0 (baseline) | 24 / 26 | 92.3 % | — | — |
| v1.5.1 (that release) | 26 / 26 | 100 % | 2 | **0** |
v1.5.0 fails `diameter_dim_measures_the_diameter` and
`radius_dim_ignores_the_leader_length`: it has both methods and gets both
wrong. That is the defect 1.5.1 exists to fix and the reason 1.5.0 is yanked.
### Result — this release vs v1.4.0
### Result — v1.5.1 vs v1.4.0
Same suite, older baseline:
Same 26-check suite, older baseline:
[`results/published/ab-v1.4.0-vs-v1.5.1.json`](results/published/ab-v1.4.0-vs-v1.5.1.json).
| Version | Checks passing | Pass rate | Fixed | Regressed |
|---------|----------------|-----------|-------|-----------|
| **v1.4.0** (baseline) | 21 / 26 | 80.8 % | — | — |
| **v1.5.1** (this release) | 26 / 26 | 100 % | 5 | **0** |
| v1.4.0 (baseline) | 21 / 26 | 80.8 % | — | — |
| v1.5.1 (that release) | 26 / 26 | 100 % | 5 | **0** |
> This row used to read `v1.5.0 | 24 / 26 | 92.3 % | 5`, which was impossible on
> its face — 21 passing plus 5 fixed is 26, not 24 — and it contradicted the
benchmarks/results/published/ab-v1.5.1-vs-v1.6.0-dev.json
+188
View File
@@ -0,0 +1,188 @@
{
"summary": {
"baseline_ref": "v1.5.1",
"total": 29,
"old_pass": 26,
"new_pass": 29,
"old_pct": 89.7,
"new_pct": 100.0,
"fixed": 3,
"regressed": 0
},
"rows": [
{
"check": "core_line_length",
"category": "Core",
"old": "pass",
"new": "pass"
},
{
"check": "core_circle_radius",
"category": "Core",
"old": "pass",
"new": "pass"
},
{
"check": "core_layer_create",
"category": "Core",
"old": "pass",
"new": "pass"
},
{
"check": "core_polyline_closed",
"category": "Core",
"old": "pass",
"new": "pass"
},
{
"check": "core_linear_dim",
"category": "Core",
"old": "pass",
"new": "pass"
},
{
"check": "core_save_dxf_roundtrip",
"category": "Core",
"old": "pass",
"new": "pass"
},
{
"check": "dim_aligned_no_error",
"category": "Dimensions",
"old": "pass",
"new": "pass"
},
{
"check": "dim_angular_no_error",
"category": "Dimensions",
"old": "pass",
"new": "pass"
},
{
"check": "array_polar_360_distinct",
"category": "Modify",
"old": "pass",
"new": "pass"
},
{
"check": "point_intersection_line_line",
"category": "Geometry",
"old": "pass",
"new": "pass"
},
{
"check": "point_tangent_external",
"category": "Geometry",
"old": "pass",
"new": "pass"
},
{
"check": "arc_has_length",
"category": "Query",
"old": "pass",
"new": "pass"
},
{
"check": "arc_select_by_length",
"category": "Query",
"old": "pass",
"new": "pass"
},
{
"check": "ezdxf_bounding_box",
"category": "Query",
"old": "pass",
"new": "pass"
},
{
"check": "mtext_rotation_roundtrip",
"category": "Entities",
"old": "pass",
"new": "pass"
},
{
"check": "screenshot_png",
"category": "Render",
"old": "pass",
"new": "pass"
},
{
"check": "dim_overlap_critique_fires",
"category": "Quality gate",
"old": "pass",
"new": "pass"
},
{
"check": "iso13567_dim_layer",
"category": "Quality gate",
"old": "pass",
"new": "pass"
},
{
"check": "construction_left_iso_caught",
"category": "Quality gate",
"old": "pass",
"new": "pass"
},
{
"check": "gear_no_self_overlap",
"category": "Engineering",
"old": "pass",
"new": "pass"
},
{
"check": "center_linetype_applied",
"category": "Entities",
"old": "pass",
"new": "pass"
},
{
"check": "hatch_area_subtracts_its_island",
"category": "Measurement",
"old": "pass",
"new": "pass"
},
{
"check": "boundary_area_agrees_with_measure",
"category": "Measurement",
"old": "pass",
"new": "pass"
},
{
"check": "two_vertex_circle_has_area",
"category": "Measurement",
"old": "pass",
"new": "pass"
},
{
"check": "diameter_dim_measures_the_diameter",
"category": "Dimensions",
"old": "pass",
"new": "pass"
},
{
"check": "radius_dim_ignores_the_leader_length",
"category": "Dimensions",
"old": "pass",
"new": "pass"
},
{
"check": "pid_block_define_attdef_roundtrip",
"category": "P&ID",
"old": "miss",
"new": "pass"
},
{
"check": "pid_tag_parse_fic",
"category": "P&ID",
"old": "miss",
"new": "pass"
},
{
"check": "pid_graph_edge_count",
"category": "P&ID",
"old": "miss",
"new": "pass"
}
]
}