Umut CANandClaude Opus 5.5
17bf3a39ce
The v1.6 roadmap in five tracks: 154 -> 247 registered tools in 28 groups. Evidence re-measured for the release: - correctness A/B against v1.5.1: 26/48 -> 48/48, 22 fixed, 0 regressed (ab-v1.5.1-vs-v1.6.0.json, replacing the -dev report); - the reference adapter on matrix v7: 21/21 (autocad-mcp-pro.json, which was the v3 run of 1.5); the published-report test follows it to v7; - the 100-point rubric re-scored per category, every withheld point named: 93 (the competitor rows are their July snapshots, not re-reviewed); - the perf lane as medians of five alternating runs, beside a v1.4.0 comparison on one interpreter (perf-v1.4.0-vs-v1.6.0.json). Two roadmap criteria are not green, and the changelog, the README and the roadmap say so: criterion 5 is withdrawn (track D, Plant 3D, dropped) and criterion 7 is not met (creation 1.25x / 1.38x v1.4.0's time, still 1.17x / 1.20x with the timeout off). Carried to 1.7. README rewritten for 1.6: what is new, the mechanical and architectural sheets, the rubric and task-matrix charts, the measured limits (including the four feature tables that ship empty), and the 1.7 roadmap. server.json describes the 1.6 surface and declares TOOL_PACKS. CLAUDE.md corrects the ISO 286 letter set and the empty feature tables. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FKFStjEFzpGyLq5bMpWxTd
64 lines
2.1 KiB
Python
64 lines
2.1 KiB
Python
"""Render the mechanical sheet the README shows, using only the server's own tools.
|
|
|
|
The sheet is the one the ``mech_assembly`` benchmark task gates on - a flange
|
|
coupling: a hub with a full section, a flange with its bolt-circle end view, two
|
|
ISO 4014 bolts with ISO 4032 nuts, an ISO 7573 parts list read off the drawing,
|
|
linked ISO 6433 balloons, an ISO 5457 frame and an ISO 7200 title block. It is
|
|
built by the benchmark's own task, so the picture and the gate cannot drift
|
|
apart: if the view engine or the sheet standard regresses, both change.
|
|
|
|
Reproduce:
|
|
|
|
AUTOCAD_MCP_BACKEND=ezdxf uv run --frozen python scripts/render_readme_mech.py
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import argparse
|
|
import asyncio
|
|
import sys
|
|
import tempfile
|
|
from pathlib import Path
|
|
|
|
ROOT = Path(__file__).resolve().parents[1]
|
|
sys.path.insert(0, str(ROOT))
|
|
|
|
DEFAULT_OUTPUT = ROOT / "docs" / "assets" / "mech-showcase.png"
|
|
|
|
|
|
async def render(output: Path) -> dict:
|
|
from benchmarks.adapters.autocad_mcp_pro import AutoCADMCPProAdapter
|
|
|
|
adapter = AutoCADMCPProAdapter(backend="ezdxf")
|
|
with tempfile.TemporaryDirectory(prefix="readme_mech_") as folder:
|
|
await adapter.setup(Path(folder))
|
|
try:
|
|
await adapter.backend.drawing_new() # the harness opens one before every task
|
|
passed, metrics = (await adapter._task_mech_assembly())[:2]
|
|
png = await adapter.backend.view_screenshot()
|
|
finally:
|
|
await adapter.cleanup()
|
|
if not png:
|
|
raise RuntimeError("view_screenshot returned no PNG (is matplotlib installed?)")
|
|
output.parent.mkdir(parents=True, exist_ok=True)
|
|
output.write_bytes(png)
|
|
return {
|
|
"output": str(output),
|
|
"bytes": len(png),
|
|
"gate_passed": passed,
|
|
"critique_issues": metrics["critique_issues"],
|
|
"score": metrics["score"],
|
|
}
|
|
|
|
|
|
def main() -> int:
|
|
parser = argparse.ArgumentParser(description=__doc__)
|
|
parser.add_argument("--output", type=Path, default=DEFAULT_OUTPUT)
|
|
args = parser.parse_args()
|
|
print(asyncio.run(render(args.output)))
|
|
return 0
|
|
|
|
|
|
if __name__ == "__main__":
|
|
raise SystemExit(main())
|