anatoly/autocad-mcp-pro
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

release: v1.5.0 — discovery layer, honesty debt closed, Wave A

Browse Source 
Tool count 131 -> 154; 474 -> 1242 collected tests.

The release exists for two measured reasons. A 149-tool catalog costs a client
~40,300 tokens before it has asked for anything, and a drafter searching the way
drafters think -- FILLET, BPOLY, QSELECT, WBLOCK -- scored df=0 against the stock
index, because those words were simply not in the data.

Discovery and batching
- DISCOVERY_MODE=search replaces the catalog with search_tools + call_tool,
  ranked over an AutoCAD command and synonym corpus (discovery/aliases.py) on
  top of fastmcp's BM25 index. Measured: 40,305 -> 356 tokens advertised.
- cad_batch runs a step list in one round trip; fields= projects result-heavy
  responses. The `core` profile is gone -- discovery replaced it.

Honesty debt (Tier 0)
- OCS/ECS: `extrusion` appeared nowhere in this codebase, so mirrored and
  rotated geometry reported coordinates that were quietly wrong. backends/ocs.py
  translates at the backend boundary; every tool coordinate is WCS on both
  engines, and a tilted plane refuses rather than answering with a number that
  looks fine.
- drawing_save_as(fmt="dwg") wrote DXF bytes to a .dwg path. drawing_audit never
  repaired. analysis_measure_area shoelaced points the caller typed in.
- ezdxf calls gained a per-call timeout, so one hung call can no longer wedge a
  server whose document lock is a single asyncio.Lock.

Architecture
- The 1854-line backend ABC is 19 contract modules; @capability(key, reason=)
  makes optional methods declarative and a gate fails on undeclared holes.
- 8 layout/viewport tools, CHSPACE, CAD_PROGID escape hatch.
- Wave A: selection filters, boundary tracing, hatch depth, annotation objects,
  measurement -- 13 tools of a planned 39. The REGION and 2D-boolean family was
  cut on evidence: add_region() yields zero ACIS bytes and the greiner_hormann
  substitute loses 28.2% of the area on a square with one semicircular edge.

Verified against a live AutoCAD 2026, which found four COM defects no amount of
reading would have caught -- AcadPViewport has no ViewCenter, and the line that
set it had been swallowed by a bare except since v1.4.

Benchmark v3 (15 tasks) and what it caught
- v2 scored this server 10/10, which carried no information: every task in it
  exercised something the server was built around. Five tasks were added because
  they can fail, and three did while being written -- a hatch could not be
  measured though the refusal listed HATCH as measurable, the boundary tools and
  the measure tool disagreed about the same shape (139.2177 vs 139.2699), and a
  circle stored as two bulged vertices measured 0.0.
- Competitor reports are NOT re-scored. They are pinned v1.4 runs against v2 and
  were never asked these five questions; the chart shows those rows `not run`.
- Correctness A/B: 21/24 -> 24/24, zero regressions.

Performance, stated plainly
The published v1.4 perf report was recorded on CPython 3.14 and the new one on
3.11.15, so the apparent 2-6x speedup is the interpreter, not this release.
Re-measured on one interpreter, v1.5.0 is 28% slower creating entities and 22%
slower on the 10k roundtrip. Fully attributed: EZDXF_CALL_TIMEOUT=0 restores
v1.4.0's exact numbers. That is the deadlock fix being paid for, the knob is
documented, and winning it back is on the 1.6 roadmap.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Umut CAN
2026-08-06 09:45:20 +03:00
co-authored by Claude Opus 5
parent d5a0ff5109
commit b03a7615b1
107 changed files with 32640 additions and 4810 deletions
Download Patch File Download Diff File Expand all files Collapse all files
benchmarks/results/published/ab-v1.4.0-vs-v1.5.0.json
+158
View File
@@ -0,0 +1,158 @@
{
"summary": {
"baseline_ref": "v1.4.0",
"total": 24,
"old_pass": 21,
"new_pass": 24,
"old_pct": 87.5,
"new_pct": 100.0,
"fixed": 3,
"regressed": 0
},
"rows": [
{
"check": "core_line_length",
"category": "Core",
"old": "pass",
"new": "pass"
},
{
"check": "core_circle_radius",
"category": "Core",
"old": "pass",
"new": "pass"
},
{
"check": "core_layer_create",
"category": "Core",
"old": "pass",
"new": "pass"
},
{
"check": "core_polyline_closed",
"category": "Core",
"old": "pass",
"new": "pass"
},
{
"check": "core_linear_dim",
"category": "Core",
"old": "pass",
"new": "pass"
},
{
"check": "core_save_dxf_roundtrip",
"category": "Core",
"old": "pass",
"new": "pass"
},
{
"check": "dim_aligned_no_error",
"category": "Dimensions",
"old": "pass",
"new": "pass"
},
{
"check": "dim_angular_no_error",
"category": "Dimensions",
"old": "pass",
"new": "pass"
},
{
"check": "array_polar_360_distinct",
"category": "Modify",
"old": "pass",
"new": "pass"
},
{
"check": "point_intersection_line_line",
"category": "Geometry",
"old": "pass",
"new": "pass"
},
{
"check": "point_tangent_external",
"category": "Geometry",
"old": "pass",
"new": "pass"
},
{
"check": "arc_has_length",
"category": "Query",
"old": "pass",
"new": "pass"
},
{
"check": "arc_select_by_length",
"category": "Query",
"old": "pass",
"new": "pass"
},
{
"check": "ezdxf_bounding_box",
"category": "Query",
"old": "pass",
"new": "pass"
},
{
"check": "mtext_rotation_roundtrip",
"category": "Entities",
"old": "pass",
"new": "pass"
},
{
"check": "screenshot_png",
"category": "Render",
"old": "pass",
"new": "pass"
},
{
"check": "dim_overlap_critique_fires",
"category": "Quality gate",
"old": "pass",
"new": "pass"
},
{
"check": "iso13567_dim_layer",
"category": "Quality gate",
"old": "pass",
"new": "pass"
},
{
"check": "construction_left_iso_caught",
"category": "Quality gate",
"old": "pass",
"new": "pass"
},
{
"check": "gear_no_self_overlap",
"category": "Engineering",
"old": "pass",
"new": "pass"
},
{
"check": "center_linetype_applied",
"category": "Entities",
"old": "pass",
"new": "pass"
},
{
"check": "hatch_area_subtracts_its_island",
"category": "Measurement",
"old": "miss",
"new": "pass"
},
{
"check": "boundary_area_agrees_with_measure",
"category": "Measurement",
"old": "miss",
"new": "pass"
},
{
"check": "two_vertex_circle_has_area",
"category": "Measurement",
"old": "miss",
"new": "pass"
}
]
}
benchmarks/results/published/autocad-mcp-pro.json
+188 -26
View File
@@ -5,6 +5,21 @@
"capabilities": {
"backend": "ezdxf",
"features": {
"audit_detail": {
"mode": "native",
"reason": null,
"supported": true
},
"boundary_trace": {
"mode": "computed",
"reason": "straight_edges_split_at_intersections;curved_edges_not_split",
"supported": true
},
"chspace": {
"mode": "computed",
"reason": "top_view_untwisted_viewports_only;dimensions_refused_unless_frozen;acis_proxy_table_refused;viewport_clipping_reported_not_applied",
"supported": true
},
"delivery": {
"mode": "shared",
"reason": null,
@@ -15,26 +30,61 @@
"reason": null,
"supported": true
},
"dwg": {
"mode": null,
"reason": "no_dwg_writer_requires_live_autocad",
"supported": false
},
"dxf": {
"mode": "native",
"reason": null,
"supported": true
},
"handle_overlay": {
"mode": "rendered",
"reason": null,
"supported": true
},
"hatch_edge_paths": {
"mode": "native",
"reason": null,
"supported": true
},
"lisp": {
"mode": null,
"reason": "live_com_only",
"supported": false
},
"measure_area_acis": {
"mode": null,
"reason": "acis_evaluation_requires_live_autocad",
"supported": false
},
"mleader": {
"mode": "composite",
"reason": null,
"supported": true
},
"paper_space": {
"mtext_background_color": {
"mode": "native",
"reason": null,
"supported": true
},
"ocs_normalized": {
"mode": "wcs",
"reason": "reported_xy_is_wcs;z_component_not_reported;text_rotation_stays_in_entity_frame;hatch_boundaries_not_reported",
"supported": true
},
"ocs_tilted_plane": {
"mode": null,
"reason": "planned_1.4",
"reason": "2d_xy_cannot_address_a_tilted_plane",
"supported": false
},
"paper_space": {
"mode": "native",
"reason": null,
"supported": true
},
"pdf": {
"mode": "rendered",
"reason": null,
@@ -55,9 +105,14 @@
"reason": null,
"supported": true
},
"revcloud": {
"mode": "native",
"reason": "ezdxf_revcloud_helper",
"supported": true
},
"solid_3d": {
"mode": null,
"reason": "planned_1.5",
"reason": "acis_generation_requires_live_autocad",
"supported": false
},
"table": {
@@ -69,6 +124,21 @@
"mode": "snapshot",
"reason": null,
"supported": true
},
"undo_history": {
"mode": null,
"reason": "set_EZDXF_UNDO_DEPTH_to_enable",
"supported": false
},
"viewport_render": {
"mode": "rendered",
"reason": "viewport_borders_not_drawn",
"supported": true
},
"wipeout": {
"mode": "native",
"reason": null,
"supported": true
}
}
}
@@ -76,15 +146,16 @@
"environment": {
"implementation": "CPython",
"machine": "AMD64",
"platform": "Windows-11-10.0.26200-SP0",
"python": "3.14.0"
"platform": "Windows-10-10.0.26200-SP0",
"python": "3.11.15"
},
"generated_at": "2026-07-23T11:39:04.277175+00:00",
"git_sha": "4a2e9d658f051cb756f92cb8fae945c64b8cc7d4",
"generated_at": "2026-08-06T06:27:59.080169+00:00",
"git_sha": "d5a0ff5109ddbd37d88371c10a1ed087201de4b5",
"matrix": "v3",
"results": [
{
"artifacts": [],
"duration_ms": 8.14,
"duration_ms": 2.49,
"message": "",
"metrics": {
"line_length": 5.0
@@ -97,7 +168,7 @@
},
{
"artifacts": [],
"duration_ms": 3.39,
"duration_ms": 1.41,
"message": "",
"metrics": {},
"score": 100.0,
@@ -108,7 +179,7 @@
},
{
"artifacts": [],
"duration_ms": 8.11,
"duration_ms": 3.31,
"message": "",
"metrics": {},
"score": 100.0,
@@ -119,7 +190,7 @@
},
{
"artifacts": [],
"duration_ms": 348.74,
"duration_ms": 17.55,
"message": "",
"metrics": {
"entity_type": "DIMENSION"
@@ -132,7 +203,7 @@
},
{
"artifacts": [],
"duration_ms": 15.74,
"duration_ms": 3.7,
"message": "",
"metrics": {
"representations": [
@@ -148,7 +219,7 @@
},
{
"artifacts": [],
"duration_ms": 28.95,
"duration_ms": 29.09,
"message": "",
"metrics": {
"entity_count": 0
@@ -161,7 +232,7 @@
},
{
"artifacts": [],
"duration_ms": 2.5,
"duration_ms": 1.2,
"message": "",
"metrics": {},
"score": 100.0,
@@ -172,7 +243,7 @@
},
{
"artifacts": [],
"duration_ms": 29.19,
"duration_ms": 8.42,
"message": "",
"metrics": {
"final_score": 100.0,
@@ -215,11 +286,11 @@
"artifacts": [
{
"filename": "roundtrip.dxf",
"sha256": "6413204a5baec19a5fbf551418eb56e30b1488ca5bb755e6a8709ac8e8ab0b22",
"size_bytes": 18442
"sha256": "d11dfce5eddaf849075c62854a8526efaaa3bbd779d374e4c287de77f361caf7",
"size_bytes": 18440
}
],
"duration_ms": 16.66,
"duration_ms": 32.29,
"message": "",
"metrics": {
"entity_count": 1
@@ -234,11 +305,11 @@
"artifacts": [
{
"filename": "manifest.json",
"sha256": "38c2ee819b49b9c66b2311b952e856eff9e3a7565194cce2edea60a3211d233b",
"size_bytes": 3581
"sha256": "a74b2e96dbe7ea373717f26a8d26e58fe079a61370eb5d646f04a09f1b7cca2f",
"size_bytes": 5662
}
],
"duration_ms": 4380.8,
"duration_ms": 305.81,
"message": "",
"metrics": {
"score": {
@@ -255,16 +326,107 @@
"stderr_summary": "",
"stdout_summary": "",
"task_id": "auditable_delivery"
},
{
"artifacts": [],
"duration_ms": 1149.77,
"message": "",
"metrics": {
"ranks": {
"BPOLY": 1,
"CHSPACE": 1,
"FILLET": 1,
"OVERKILL": 1,
"QSELECT": 1,
"WBLOCK": 1
},
"top_n": 3
},
"score": 100.0,
"status": "pass",
"stderr_summary": "",
"stdout_summary": "",
"task_id": "tool_discovery"
},
{
"artifacts": [],
"duration_ms": 53.97,
"message": "",
"metrics": {
"advertised_tools": 2,
"catalog_tokens": 40305,
"catalog_tools": 149,
"ceiling": 2000,
"discovery_tokens": 356,
"reduction_x": 113.2,
"tokenizer": "ratio/v1 (estimate)"
},
"score": 100.0,
"status": "pass",
"stderr_summary": "",
"stdout_summary": "",
"task_id": "token_budget"
},
{
"artifacts": [],
"duration_ms": 10.6,
"message": "",
"metrics": {
"area_ignoring_islands": 400.0,
"expected_filled": 300.0,
"filled_area": 300.0,
"hatch_style": "outer",
"path_count": 2
},
"score": 100.0,
"status": "pass",
"stderr_summary": "",
"stdout_summary": "",
"task_id": "hatch_islands"
},
{
"artifacts": [],
"duration_ms": 8.0,
"message": "",
"metrics": {
"bounding_box": 3,
"by_layer": 1,
"crossing": 2,
"polygon": 1,
"window": 1
},
"score": 100.0,
"status": "pass",
"stderr_summary": "",
"stdout_summary": "",
"task_id": "selection_filter"
},
{
"artifacts": [],
"duration_ms": 5.36,
"message": "",
"metrics": {
"boundary_agrees": true,
"fraction_lost_by_reading_points_back": 0.282,
"measured_area": 139.269908,
"true_area": 139.269908,
"vertex_shoelace_area": 100.0
},
"score": 100.0,
"status": "pass",
"stderr_summary": "",
"stdout_summary": "",
"task_id": "measure_from_handle"
}
],
"schema_version": "2.0",
"summary": {
"attempted": 10,
"attempted": 15,
"coverage_percent": 100.0,
"passed": 10,
"passed": 15,
"score": 100.0,
"supported": 10,
"supported": 15,
"unsupported": 0
},
"timeout_seconds": 30.0
}
"timeout_seconds": 180.0
}
benchmarks/results/published/perf-ezdxf.json
+12 -12
View File
@@ -3,8 +3,8 @@
"environment": {
"implementation": "CPython",
"machine": "AMD64",
"platform": "Windows-11-10.0.26200-SP0",
"python": "3.14.0"
"platform": "Windows-10-10.0.26200-SP0",
"python": "3.11.15"
},
"schema_version": "perf/1.0",
"workloads": [
@@ -14,22 +14,22 @@
"entity_count": 2000
},
"metric": "entities_per_second",
"metric_value": 3148.0,
"wall_ms": 635.3,
"metric_value": 6343.1,
"wall_ms": 315.3,
"workload_id": "create_lines_2k"
},
{
"description": "Create 10,000 lines, export DXF, reopen and verify",
"detail": {
"build_ms": 10884.9,
"build_ms": 1574.1,
"entity_count": 10000,
"export_ms": 1439.2,
"file_bytes": 1369662,
"reopen_ms": 1607.5
"export_ms": 261.8,
"file_bytes": 1369663,
"reopen_ms": 320.2
},
"metric": "entities_per_second",
"metric_value": 717.8,
"wall_ms": 13931.6,
"metric_value": 4637.9,
"wall_ms": 2156.1,
"workload_id": "roundtrip_10k"
},
{
@@ -39,7 +39,7 @@
},
"metric": "entities_matched",
"metric_value": 2500.0,
"wall_ms": 3434.2,
"wall_ms": 890.8,
"workload_id": "region_query_10k"
},
{
@@ -49,7 +49,7 @@
},
"metric": "critique_issues",
"metric_value": 0.0,
"wall_ms": 315.9,
"wall_ms": 176.5,
"workload_id": "premium_pass"
}
]