415de73137300f6ed800981e282c2a585bbc220f
4
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Umut CANandClaude Opus 5
|
6b9860e68e |
docs: apply 27 corrections from the adversarial triage, and rework the perf lane
The triage over the 43 unaddressed verdicts returned 4 code and 35 doc items. The code ones shipped in e24b46d; these are the documents. Twenty-one applied mechanically after a validation pass confirmed each locator still matched exactly once and each replacement was supported by the tree. Ten were held back because a re-verify pass found them mutually exclusive -- three patches rewriting the same benchmarks/README block, three rewriting the same OCS paragraph -- and applying them in sequence would have corrupted the files. Those were resolved by hand into one coherent edit each. One was rejected. The corrections worth naming: - **The A/B table contradicted itself in three files.** benchmarks/README.md and CLAUDE.md said 24 checks / 21-24 / 87.5% / "fixed 3" while the suite has 26 and the published JSON says 21/26, 80.8%, five fixed. All three now agree, and the five are split into the three new-capability `miss -> pass` and the two repaired-defect `fail -> pass`, which is the distinction the lane exists to make. - **"Every number below is produced by a script"** was false twice over: section 5's nine scores are literals a human assigned, and section 4 now quotes hand-collected timings. The claim is narrowed to the charts, which really are all rendered from committed data, and both exceptions are named. - **The OCS paragraph over-quantified.** TEXT rotation was given as the single exception; z components and HATCH boundaries are also entity-frame, and the live backend records `2d_polyline_coordinates_not_verified_against_live_autocad` in the same reason string. It now points at `ocs_normalized` as the live list. - **Transactions were shown as bare parity.** The server itself grades them differently -- `FeatureCapability(True, "undo_mark")` on COM against `(True, "snapshot")` headless -- and COM rollback sends `_UNDO B` without confirming it landed. The row says so. - **"Raw commands: Opt-in"** named a switch that does not exist. Both tools are registered unconditionally; `DANGEROUS_COMMANDS_ENABLED` only disables the sanitizer. - **"Wave A shipped 13 of a planned 39"** hardened an approximation: the only source writes `~39`. - **CAD_PROGID** said nothing beyond the connection was tested against BricsCAD/ZWCAD/GstarCAD, implying the connection was. It was not. Perf lane reworked against a fresh paired measurement. The old text claimed "28% slower than v1.4.0, median of three runs" and that `EZDXF_CALL_TIMEOUT=0` restored v1.4.0's numbers "exactly". Three problems: `perf_suite` runs each workload once per invocation and cannot produce a median, "exactly" was refuted by the figures in its own sentence, and the number moved when the timeout wrapper was rebuilt. It now reports the guard's cost -- +20.8% and +19.9%, three paired runs, median taken by hand -- says the figures have no published artifact, and says the wrapper rework recovered about a third of the v1.5.0 overhead rather than closing it. perf-ezdxf.json and the perf chart regenerated against the shipped code, and the README table rebuilt from that artifact so the two cannot drift. 1267 passed, 1 xfailed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Umut CANandClaude Opus 5
|
b03a7615b1 |
release: v1.5.0 — discovery layer, honesty debt closed, Wave A
Tool count 131 -> 154; 474 -> 1242 collected tests. The release exists for two measured reasons. A 149-tool catalog costs a client ~40,300 tokens before it has asked for anything, and a drafter searching the way drafters think -- FILLET, BPOLY, QSELECT, WBLOCK -- scored df=0 against the stock index, because those words were simply not in the data. Discovery and batching - DISCOVERY_MODE=search replaces the catalog with search_tools + call_tool, ranked over an AutoCAD command and synonym corpus (discovery/aliases.py) on top of fastmcp's BM25 index. Measured: 40,305 -> 356 tokens advertised. - cad_batch runs a step list in one round trip; fields= projects result-heavy responses. The `core` profile is gone -- discovery replaced it. Honesty debt (Tier 0) - OCS/ECS: `extrusion` appeared nowhere in this codebase, so mirrored and rotated geometry reported coordinates that were quietly wrong. backends/ocs.py translates at the backend boundary; every tool coordinate is WCS on both engines, and a tilted plane refuses rather than answering with a number that looks fine. - drawing_save_as(fmt="dwg") wrote DXF bytes to a .dwg path. drawing_audit never repaired. analysis_measure_area shoelaced points the caller typed in. - ezdxf calls gained a per-call timeout, so one hung call can no longer wedge a server whose document lock is a single asyncio.Lock. Architecture - The 1854-line backend ABC is 19 contract modules; @capability(key, reason=) makes optional methods declarative and a gate fails on undeclared holes. - 8 layout/viewport tools, CHSPACE, CAD_PROGID escape hatch. - Wave A: selection filters, boundary tracing, hatch depth, annotation objects, measurement -- 13 tools of a planned 39. The REGION and 2D-boolean family was cut on evidence: add_region() yields zero ACIS bytes and the greiner_hormann substitute loses 28.2% of the area on a square with one semicircular edge. Verified against a live AutoCAD 2026, which found four COM defects no amount of reading would have caught -- AcadPViewport has no ViewCenter, and the line that set it had been swallowed by a bare except since v1.4. Benchmark v3 (15 tasks) and what it caught - v2 scored this server 10/10, which carried no information: every task in it exercised something the server was built around. Five tasks were added because they can fail, and three did while being written -- a hatch could not be measured though the refusal listed HATCH as measurable, the boundary tools and the measure tool disagreed about the same shape (139.2177 vs 139.2699), and a circle stored as two bulged vertices measured 0.0. - Competitor reports are NOT re-scored. They are pinned v1.4 runs against v2 and were never asked these five questions; the chart shows those rows `not run`. - Correctness A/B: 21/24 -> 24/24, zero regressions. Performance, stated plainly The published v1.4 perf report was recorded on CPython 3.14 and the new one on 3.11.15, so the apparent 2-6x speedup is the interpreter, not this release. Re-measured on one interpreter, v1.5.0 is 28% slower creating entities and 22% slower on the 10k roundtrip. Fully attributed: EZDXF_CALL_TIMEOUT=0 restores v1.4.0's exact numbers. That is the deadlock fix being paid for, the knob is documented, and winning it back is on the 1.6 roadmap. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Umutcan EdizsalanandCursor
|
bae0750d4c |
docs(bench): dark leaderboard styling for all benchmark charts
Leaderboard restyled to the LLM-benchmark look: dark background, rank-numbered labels, per-rank color palette, 'NN / 100 - grade' annotations. The live-run, task-matrix and performance charts share the same dark theme for a consistent README identity. Charts regenerated deterministically from their data files. Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Umutcan EdizsalanandCursor
|
21cbc62df5 |
feat(bench): performance lane + task-matrix chart; docs: README rebuilt around four evidence lanes
- benchmarks/perf_suite.py: fixed headless workloads (2k creates, 10k build/export/reopen, 10k region query, premium quality pass) timed through the same backend methods the tools call; machine fingerprint recorded; self-measurement only - benchmarks/render_perf_chart.py -> docs/assets/autocad-mcp-perf.svg (log-scale wall time) - benchmarks/render_matrix_chart.py -> docs/assets/autocad-mcp-taskmatrix.svg (tasks x servers status heatmap from published reports) - render_live_chart now tolerates other lanes sharing results/published/ - README: PyPI-first install, four-lane benchmark section with Python-generated visuals, v1.4 feature table, updated client/config/roadmap - published perf report: benchmarks/results/published/perf-ezdxf.json Co-authored-by: Cursor <cursoragent@cursor.com> |
