cdb10638963898b3ea9b10cdd96a2c9bc495f184
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Umut CANandClaude Opus 5.5
|
17bf3a39ce |
release: 1.6.0
The v1.6 roadmap in five tracks: 154 -> 247 registered tools in 28 groups. Evidence re-measured for the release: - correctness A/B against v1.5.1: 26/48 -> 48/48, 22 fixed, 0 regressed (ab-v1.5.1-vs-v1.6.0.json, replacing the -dev report); - the reference adapter on matrix v7: 21/21 (autocad-mcp-pro.json, which was the v3 run of 1.5); the published-report test follows it to v7; - the 100-point rubric re-scored per category, every withheld point named: 93 (the competitor rows are their July snapshots, not re-reviewed); - the perf lane as medians of five alternating runs, beside a v1.4.0 comparison on one interpreter (perf-v1.4.0-vs-v1.6.0.json). Two roadmap criteria are not green, and the changelog, the README and the roadmap say so: criterion 5 is withdrawn (track D, Plant 3D, dropped) and criterion 7 is not met (creation 1.25x / 1.38x v1.4.0's time, still 1.17x / 1.20x with the timeout off). Carried to 1.7. README rewritten for 1.6: what is new, the mechanical and architectural sheets, the rubric and task-matrix charts, the measured limits (including the four feature tables that ship empty), and the 1.7 roadmap. server.json describes the 1.6 surface and declares TOOL_PACKS. CLAUDE.md corrects the ISO 286 letter set and the empty feature tables. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd |
||
|
Umut CANandClaude Opus 5
|
8f405bddf3 |
fix(dimensions): diameter and radius callouts measured the leader as geometry
`add_diameter_dim` / `add_radius_dim` measure to the point they are handed; `leader_length` is a text placement. The ezdxf backend passed `centre + (radius + leader)` as that point, so the leader became part of the measurement: a true 40 mm bore was dimensioned 60 at the default `leader_length=10`, and 108 at 34. Radius callouts were `leader_length` too large the same way. This is the worst class of defect for this project -- a wrong number, at default settings, on the one part of a drawing that exists to carry numbers -- and it shipped in v1.4.0 and earlier. The size now comes from `radius` and `location` only places the text. The live COM backend was never affected: ActiveX's AddDimDiametric takes the two chord points and the leader length as separate arguments. The two engines had been disagreeing by 2 x leader_length on the same call. Found while building the drawing for the new README, whose own bore callout was wrong. Two checks added to correctness_suite.py; both report fail -> pass against v1.4.0, not miss -> pass, which is what separates a repaired defect from new capability in that lane. A/B is now 21/26 -> 26/26, five fixed, zero regressed. README rewritten from scratch - Hero is a real ISO A3 sheet the server drew: m6 z24 involute spur gear, DIN 6885 keyed bore, section A-A, ISO 129 dimensions, ISO 286 H7 fit, ISO 7200 title block, 56 entities, drawing_critique clean. New scripts/render_readme_showcase.py rebuilds it, and the tool-call sequence that produces it is printed in the page. - Star history chart, badge row, capability matrix, five evidence lanes, and a "what this release is bad at" section carrying the 28% throughput regression. - Every image and repo link is now an absolute URL: README.md is the PyPI long description, where relative paths 404. Corrections this work forced, all measured - The df=0 example was wrong. FILLET is already in entity_fillet's own text; the commands that genuinely scored zero are WBLOCK, PEDIT, OVERKILL, LAYTRANS and MATCHPROP. - 40,305 tokens is the tools/list wire payload; ~25,143 is the part forwarded to the model. The old wording said "of schema", which is neither. - `--tokenizer anthropic` needs a package and credentials this repo does not lock, and refuses rather than falling back. Said so. - Discovery ranks over the 149-tool advertised catalog, not the 154 registered. - Five middlewares, not four; the fifth is what makes refusals machine-readable. - Five of the 27 capability keys are environment-dependent, so the matrix carries footnotes instead of bare ticks. - "Every coordinate is WCS" has one deliberate exception: TEXT rotation. - render_live_chart's axis still said "fixed 10-task matrix" while our bar came from v3's fifteen. Each bar now carries its own matrix. - .env.example said MAX_UNDO_STACK=50 (code: 5) and omitted four real variables, two of them the remote-HTTP controls. - server.json described an empty ALLOWED_PATHS as blocking access. It disables the allowlist. - server.py called the lean profile ~46 tools; it is 47. - backends/capability.py cited a test file that does not exist. 1249 passed, 1 xfailed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Umut CANandClaude Opus 5
|
b03a7615b1 |
release: v1.5.0 — discovery layer, honesty debt closed, Wave A
Tool count 131 -> 154; 474 -> 1242 collected tests. The release exists for two measured reasons. A 149-tool catalog costs a client ~40,300 tokens before it has asked for anything, and a drafter searching the way drafters think -- FILLET, BPOLY, QSELECT, WBLOCK -- scored df=0 against the stock index, because those words were simply not in the data. Discovery and batching - DISCOVERY_MODE=search replaces the catalog with search_tools + call_tool, ranked over an AutoCAD command and synonym corpus (discovery/aliases.py) on top of fastmcp's BM25 index. Measured: 40,305 -> 356 tokens advertised. - cad_batch runs a step list in one round trip; fields= projects result-heavy responses. The `core` profile is gone -- discovery replaced it. Honesty debt (Tier 0) - OCS/ECS: `extrusion` appeared nowhere in this codebase, so mirrored and rotated geometry reported coordinates that were quietly wrong. backends/ocs.py translates at the backend boundary; every tool coordinate is WCS on both engines, and a tilted plane refuses rather than answering with a number that looks fine. - drawing_save_as(fmt="dwg") wrote DXF bytes to a .dwg path. drawing_audit never repaired. analysis_measure_area shoelaced points the caller typed in. - ezdxf calls gained a per-call timeout, so one hung call can no longer wedge a server whose document lock is a single asyncio.Lock. Architecture - The 1854-line backend ABC is 19 contract modules; @capability(key, reason=) makes optional methods declarative and a gate fails on undeclared holes. - 8 layout/viewport tools, CHSPACE, CAD_PROGID escape hatch. - Wave A: selection filters, boundary tracing, hatch depth, annotation objects, measurement -- 13 tools of a planned 39. The REGION and 2D-boolean family was cut on evidence: add_region() yields zero ACIS bytes and the greiner_hormann substitute loses 28.2% of the area on a square with one semicircular edge. Verified against a live AutoCAD 2026, which found four COM defects no amount of reading would have caught -- AcadPViewport has no ViewCenter, and the line that set it had been swallowed by a bare except since v1.4. Benchmark v3 (15 tasks) and what it caught - v2 scored this server 10/10, which carried no information: every task in it exercised something the server was built around. Five tasks were added because they can fail, and three did while being written -- a hatch could not be measured though the refusal listed HATCH as measurable, the boundary tools and the measure tool disagreed about the same shape (139.2177 vs 139.2699), and a circle stored as two bulged vertices measured 0.0. - Competitor reports are NOT re-scored. They are pinned v1.4 runs against v2 and were never asked these five questions; the chart shows those rows `not run`. - Correctness A/B: 21/24 -> 24/24, zero regressions. Performance, stated plainly The published v1.4 perf report was recorded on CPython 3.14 and the new one on 3.11.15, so the apparent 2-6x speedup is the interpreter, not this release. Re-measured on one interpreter, v1.5.0 is 28% slower creating entities and 22% slower on the 10k roundtrip. Fully attributed: EZDXF_CALL_TIMEOUT=0 restores v1.4.0's exact numbers. That is the deadlock fix being paid for, the knob is documented, and winning it back is on the 1.6 roadmap. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Umutcan EdizsalanandCursor
|
bae0750d4c |
docs(bench): dark leaderboard styling for all benchmark charts
Leaderboard restyled to the LLM-benchmark look: dark background, rank-numbered labels, per-rank color palette, 'NN / 100 - grade' annotations. The live-run, task-matrix and performance charts share the same dark theme for a consistent README identity. Charts regenerated deterministically from their data files. Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Umutcan EdizsalanandCursor
|
6d7c39e012 |
feat(bench): live competitor lane - black-box MCP stdio adapters, pinned checkouts, published results
- adapters/mcp_stdio.py: drives competitors over MCP stdio like a real host; payload-level error detection, fastmcp output-wrapper normalization, cold-start warmup, artifact-based verification (harness re-opens every exported DXF with ezdxf) - competitors_env.py: reproducible pinned-SHA checkout + isolated venv per competitor - adapters for puran-water/autocad-mcp (95476a3) and beiming183-cloud/AutoCAD-MCP (11f7c47) mapped to their own documented tool contracts; unmapped tasks report 'unsupported' with reasons - run_competitors.py: registry-driven --server from competitors.yaml - published reports (pro 100.0 / beiming 50.0 / puran 45.0) + render_live_chart.py -> docs/assets/autocad-mcp-livebench.svg - benchmark.yml: weekly + manual dispatch matrix Co-authored-by: Cursor <cursoragent@cursor.com> |
