Commit Graph
5 Commits
Author SHA1 Message Date
Umut CANandClaude Opus 5.5 17bf3a39ce release: 1.6.0
The v1.6 roadmap in five tracks: 154 -> 247 registered tools in 28 groups.

Evidence re-measured for the release:
- correctness A/B against v1.5.1: 26/48 -> 48/48, 22 fixed, 0 regressed
  (ab-v1.5.1-vs-v1.6.0.json, replacing the -dev report);
- the reference adapter on matrix v7: 21/21 (autocad-mcp-pro.json, which
  was the v3 run of 1.5); the published-report test follows it to v7;
- the 100-point rubric re-scored per category, every withheld point named:
  93 (the competitor rows are their July snapshots, not re-reviewed);
- the perf lane as medians of five alternating runs, beside a v1.4.0
  comparison on one interpreter (perf-v1.4.0-vs-v1.6.0.json).

Two roadmap criteria are not green, and the changelog, the README and the
roadmap say so: criterion 5 is withdrawn (track D, Plant 3D, dropped) and
criterion 7 is not met (creation 1.25x / 1.38x v1.4.0's time, still
1.17x / 1.20x with the timeout off). Carried to 1.7.

README rewritten for 1.6: what is new, the mechanical and architectural
sheets, the rubric and task-matrix charts, the measured limits (including
the four feature tables that ship empty), and the 1.7 roadmap. server.json
describes the 1.6 surface and declares TOOL_PACKS. CLAUDE.md corrects the
ISO 286 letter set and the empty feature tables.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd
2026-09-25 00:17:17 +03:00
Umut CANandClaude Opus 5 8f405bddf3 fix(dimensions): diameter and radius callouts measured the leader as geometry
`add_diameter_dim` / `add_radius_dim` measure to the point they are handed;
`leader_length` is a text placement. The ezdxf backend passed
`centre + (radius + leader)` as that point, so the leader became part of the
measurement: a true 40 mm bore was dimensioned 60 at the default
`leader_length=10`, and 108 at 34. Radius callouts were `leader_length` too
large the same way.

This is the worst class of defect for this project -- a wrong number, at
default settings, on the one part of a drawing that exists to carry numbers --
and it shipped in v1.4.0 and earlier. The size now comes from `radius` and
`location` only places the text.

The live COM backend was never affected: ActiveX's AddDimDiametric takes the
two chord points and the leader length as separate arguments. The two engines
had been disagreeing by 2 x leader_length on the same call.

Found while building the drawing for the new README, whose own bore callout was
wrong. Two checks added to correctness_suite.py; both report fail -> pass
against v1.4.0, not miss -> pass, which is what separates a repaired defect
from new capability in that lane. A/B is now 21/26 -> 26/26, five fixed, zero
regressed.

README rewritten from scratch
- Hero is a real ISO A3 sheet the server drew: m6 z24 involute spur gear, DIN
  6885 keyed bore, section A-A, ISO 129 dimensions, ISO 286 H7 fit, ISO 7200
  title block, 56 entities, drawing_critique clean. New
  scripts/render_readme_showcase.py rebuilds it, and the tool-call sequence
  that produces it is printed in the page.
- Star history chart, badge row, capability matrix, five evidence lanes, and a
  "what this release is bad at" section carrying the 28% throughput regression.
- Every image and repo link is now an absolute URL: README.md is the PyPI long
  description, where relative paths 404.

Corrections this work forced, all measured
- The df=0 example was wrong. FILLET is already in entity_fillet's own text;
  the commands that genuinely scored zero are WBLOCK, PEDIT, OVERKILL,
  LAYTRANS and MATCHPROP.
- 40,305 tokens is the tools/list wire payload; ~25,143 is the part forwarded
  to the model. The old wording said "of schema", which is neither.
- `--tokenizer anthropic` needs a package and credentials this repo does not
  lock, and refuses rather than falling back. Said so.
- Discovery ranks over the 149-tool advertised catalog, not the 154 registered.
- Five middlewares, not four; the fifth is what makes refusals machine-readable.
- Five of the 27 capability keys are environment-dependent, so the matrix
  carries footnotes instead of bare ticks.
- "Every coordinate is WCS" has one deliberate exception: TEXT rotation.
- render_live_chart's axis still said "fixed 10-task matrix" while our bar came
  from v3's fifteen. Each bar now carries its own matrix.
- .env.example said MAX_UNDO_STACK=50 (code: 5) and omitted four real
  variables, two of them the remote-HTTP controls.
- server.json described an empty ALLOWED_PATHS as blocking access. It disables
  the allowlist.
- server.py called the lean profile ~46 tools; it is 47.
- backends/capability.py cited a test file that does not exist.

1249 passed, 1 xfailed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 11:24:44 +03:00
Umutcan EdizsalanandCursor bae0750d4c docs(bench): dark leaderboard styling for all benchmark charts
Leaderboard restyled to the LLM-benchmark look: dark background, rank-numbered labels, per-rank color palette, 'NN / 100 - grade' annotations. The live-run, task-matrix and performance charts share the same dark theme for a consistent README identity. Charts regenerated deterministically from their data files.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-23 16:14:04 +03:00
Umutcan EdizsalanandCursor 21cbc62df5 feat(bench): performance lane + task-matrix chart; docs: README rebuilt around four evidence lanes
- benchmarks/perf_suite.py: fixed headless workloads (2k creates, 10k build/export/reopen, 10k region query, premium quality pass) timed through the same backend methods the tools call; machine fingerprint recorded; self-measurement only
- benchmarks/render_perf_chart.py -> docs/assets/autocad-mcp-perf.svg (log-scale wall time)
- benchmarks/render_matrix_chart.py -> docs/assets/autocad-mcp-taskmatrix.svg (tasks x servers status heatmap from published reports)
- render_live_chart now tolerates other lanes sharing results/published/
- README: PyPI-first install, four-lane benchmark section with Python-generated visuals, v1.4 feature table, updated client/config/roadmap
- published perf report: benchmarks/results/published/perf-ezdxf.json

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-23 15:53:04 +03:00
Umutcan EdizsalanandCursor 6d7c39e012 feat(bench): live competitor lane - black-box MCP stdio adapters, pinned checkouts, published results
- adapters/mcp_stdio.py: drives competitors over MCP stdio like a real host; payload-level error detection, fastmcp output-wrapper normalization, cold-start warmup, artifact-based verification (harness re-opens every exported DXF with ezdxf)
- competitors_env.py: reproducible pinned-SHA checkout + isolated venv per competitor
- adapters for puran-water/autocad-mcp (95476a3) and beiming183-cloud/AutoCAD-MCP (11f7c47) mapped to their own documented tool contracts; unmapped tasks report 'unsupported' with reasons
- run_competitors.py: registry-driven --server from competitors.yaml
- published reports (pro 100.0 / beiming 50.0 / puran 45.0) + render_live_chart.py -> docs/assets/autocad-mcp-livebench.svg
- benchmark.yml: weekly + manual dispatch matrix

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-23 15:15:27 +03:00