# Repository Cleanup, README, and Benchmark Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Remove private/generated repository noise and publish a concise English README with an honest, reproducible Matplotlib benchmark graphic naming major AutoCAD MCP alternatives. **Architecture:** Keep benchmark evidence, rendering, and presentation separate: a schema-versioned JSON snapshot is the source of truth, a focused Python module validates and renders it, and README/benchmark documentation explains the evidence boundary. Repository cleanup uses explicit, workspace-confined PowerShell targets so current version 1.3 source work remains untouched. **Tech Stack:** Python 3.11+, JSON, Matplotlib SVG backend, pytest, Ruff, PowerShell, Markdown. --- ## File Map - Modify `.gitignore`: remove private-draft names; add general cache/test/output rules. - Delete untracked `b.md`, `new.md`, and `p.md` only when present. - Delete ignored/generated workspace artifacts listed in Task 1; no tracked product code. - Create `benchmarks/source_review.json`: dated source-review snapshot and rubric metadata. - Create `benchmarks/render_chart.py`: validate the snapshot and render deterministic SVG. - Create `tests/test_benchmark_chart.py`: data-contract and SVG-rendering coverage. - Create `docs/assets/autocad-mcp-benchmark.svg`: generated benchmark image. - Modify `benchmarks/README.md`: explain runtime versus source-review evidence and reproduction. - Replace `README.md`: concise public product narrative and embedded benchmark. ### Task 1: Clean the workspace without touching product work **Files:** - Modify: `.gitignore:1-44` - Delete if present: `b.md` - Delete if present: `new.md` - Delete if present: `p.md` - Delete local artifacts: `.codex-pytest-temp/`, `.pytest_cache/`, `.release-*/`, `.tdd-*/`, `.ruff_cache/`, `**/__pycache__/`, `dist/`, `out/`, `.coverage`, and an empty `.firecrawl/` - [ ] **Step 1: Capture the pre-cleanup product status** Run: ```powershell git status --porcelain=v1 -uall ``` Expected: existing version 1.3 modifications and untracked product files are visible. Keep this output available for comparison; do not stage or reset any of them. - [ ] **Step 2: Replace draft-specific ignores with general generated-output rules** Apply this exact `.gitignore` structure while preserving the existing environment, IDE, and OS sections: ```gitignore # Python __pycache__/ *.py[cod] *$py.class *.so *.egg-info/ *.egg dist/ build/ .eggs/ # Virtual environments .venv/ venv/ env/ # IDE .vscode/ .idea/ *.swp *.swo *~ # Testing and tooling .pytest_cache/ .ruff_cache/ .codex-pytest-temp/ .release-*/ .tdd-*/ .coverage coverage.xml htmlcov/ # Environment .env # OS .DS_Store Thumbs.db desktop.ini # Generated CAD and benchmark output out/ benchmarks/results/* !benchmarks/results/.gitkeep *.dxf *.dwg *.bak ``` Expected: `b.md` and `new.md` no longer appear in `.gitignore`; generated artifacts remain ignored by category. - [ ] **Step 3: Resolve and print every deletion target before deleting** Run this PowerShell from the repository root: ```powershell $repoRoot = (Resolve-Path -LiteralPath '.').Path.TrimEnd('\') $rootNames = @( '.codex-pytest-temp', '.pytest_cache', '.ruff_cache', 'dist', 'out' ) $targets = @() foreach ($name in $rootNames) { $candidate = Join-Path $repoRoot $name if (Test-Path -LiteralPath $candidate) { $targets += Get-Item -Force -LiteralPath $candidate } } $targets += Get-ChildItem -Force -Directory -LiteralPath $repoRoot | Where-Object { $_.Name -like '.release-*' -or $_.Name -like '.tdd-*' } $targets += Get-ChildItem -Force -Directory -Recurse -LiteralPath $repoRoot -Filter '__pycache__' -ErrorAction SilentlyContinue $targets = $targets | Sort-Object FullName -Unique foreach ($target in $targets) { $full = [IO.Path]::GetFullPath($target.FullName) $insideRepo = $full.StartsWith( $repoRoot + [IO.Path]::DirectorySeparatorChar, [StringComparison]::OrdinalIgnoreCase ) if (-not $insideRepo -or $full -eq $repoRoot) { throw "Unsafe deletion target: $full" } [pscustomobject]@{ Type = 'directory'; Path = $full } } foreach ($name in @('b.md', 'new.md', 'p.md', '.coverage')) { $candidate = Join-Path $repoRoot $name if (Test-Path -LiteralPath $candidate) { $full = [IO.Path]::GetFullPath($candidate) [pscustomobject]@{ Type = 'file'; Path = $full } } } $firecrawl = Join-Path $repoRoot '.firecrawl' if (Test-Path -LiteralPath $firecrawl) { $children = @(Get-ChildItem -Force -LiteralPath $firecrawl) if ($children.Count -eq 0) { [pscustomobject]@{ Type = 'empty-directory'; Path = $firecrawl } } else { Write-Warning '.firecrawl is not empty and will be preserved.' } } ``` Expected: every printed path begins with the resolved repository root; no `backends`, `engineering`, `benchmarks`, `tests`, or source file is listed. - [ ] **Step 4: Delete only the verified targets** In the same PowerShell session, after Step 3 has printed a safe list: ```powershell $targets | Sort-Object { $_.FullName.Length } -Descending | ForEach-Object { Remove-Item -Force -Recurse -LiteralPath $_.FullName } foreach ($name in @('b.md', 'new.md', 'p.md', '.coverage')) { $candidate = Join-Path $repoRoot $name if (Test-Path -LiteralPath $candidate) { Remove-Item -Force -LiteralPath $candidate } } $firecrawl = Join-Path $repoRoot '.firecrawl' if (Test-Path -LiteralPath $firecrawl) { $children = @(Get-ChildItem -Force -LiteralPath $firecrawl) if ($children.Count -eq 0) { Remove-Item -Force -LiteralPath $firecrawl } } ``` Expected: all intended local artifacts are absent; a non-empty `.firecrawl/` is preserved. - [ ] **Step 5: Verify hygiene and preservation** Run: ```powershell git status --porcelain=v1 -uall git check-ignore -v -- .ruff_cache/sample .codex-pytest-temp/sample .release-check/sample .tdd-check/sample out/sample.dxf rg -n '^(b\.md|new\.md|p\.md)$' .gitignore ``` Expected: pre-existing product changes remain; the first five generated paths report matching ignore rules; `rg` returns exit code 1 because no private draft name is ignored. - [ ] **Step 6: Commit only repository hygiene** ```powershell git add -- .gitignore git commit -m "chore: remove local artifact noise" ``` Expected: the commit contains only `.gitignore`. The ignored/untracked deletions do not accidentally stage product files. ### Task 2: Define the benchmark contract with failing tests **Files:** - Create: `tests/test_benchmark_chart.py` - Test: `tests/test_benchmark_chart.py` - [ ] **Step 1: Write data-contract and renderer tests** Create `tests/test_benchmark_chart.py` with: ```python import json from pathlib import Path import pytest from benchmarks.render_chart import load_benchmark, render_chart ROOT = Path(__file__).resolve().parents[1] DATA_PATH = ROOT / "benchmarks" / "source_review.json" def test_source_review_contract_is_complete() -> None: raw = json.loads(DATA_PATH.read_text(encoding="utf-8")) data = load_benchmark(DATA_PATH) assert data == raw assert data["schema_version"] == "1.0" assert sum(item["weight"] for item in data["rubric"]) == 100 assert data["evidence_boundary"] == "source_review_not_shared_live_run" projects = data["projects"] repositories = [item["repository"] for item in projects] assert len(repositories) == len(set(repositories)) assert "U-C4N/Autocad-MCP" in repositories assert "varavista/autocad-mcp" in repositories assert "beiming183-cloud/AutoCAD-MCP" in repositories assert "AnCode666/multiCAD-mcp" in repositories assert "puran-water/autocad-mcp" in repositories assert "NCO-1986/AutoCAD_mcp" in repositories assert all(0 <= item["score"] <= 100 for item in projects) assert all(item["evidence_grade"] in {"A", "B", "C"} for item in projects) def test_load_benchmark_rejects_duplicate_repositories(tmp_path: Path) -> None: payload = json.loads(DATA_PATH.read_text(encoding="utf-8")) payload["projects"].append(payload["projects"][0].copy()) invalid = tmp_path / "duplicate.json" invalid.write_text(json.dumps(payload), encoding="utf-8") with pytest.raises(ValueError, match="duplicate repository"): load_benchmark(invalid) def test_render_chart_writes_readable_svg(tmp_path: Path) -> None: pytest.importorskip("matplotlib") data = load_benchmark(DATA_PATH) output = tmp_path / "benchmark.svg" render_chart(data, output) svg = output.read_text(encoding="utf-8") assert svg.lstrip().startswith(" dict[str, Any]: """Load and validate the benchmark snapshot used by docs and tests.""" data = json.loads(path.read_text(encoding="utf-8")) if data.get("schema_version") != "1.0": raise ValueError("unsupported benchmark schema_version") if data.get("evidence_boundary") != "source_review_not_shared_live_run": raise ValueError("benchmark evidence boundary must be explicit") rubric = data.get("rubric") if not isinstance(rubric, list) or sum(item.get("weight", 0) for item in rubric) != 100: raise ValueError("benchmark rubric weights must total 100") projects = data.get("projects") if not isinstance(projects, list) or not projects: raise ValueError("benchmark must contain projects") seen: set[str] = set() for project in projects: missing = REQUIRED_PROJECT_KEYS - project.keys() if missing: raise ValueError(f"project is missing keys: {sorted(missing)}") repository = project["repository"] if repository in seen: raise ValueError(f"duplicate repository: {repository}") seen.add(repository) if not 0 <= project["score"] <= 100: raise ValueError(f"score outside 0..100: {repository}") if project["evidence_grade"] not in {"A", "B", "C"}: raise ValueError(f"invalid evidence grade: {repository}") return data def render_chart(data: dict[str, Any], output: Path) -> None: """Render a GitHub-readable fixed-background horizontal bar chart.""" try: import matplotlib matplotlib.use("Agg") import matplotlib.pyplot as plt except ImportError as exc: raise SystemExit( 'Matplotlib is required. Install it with: pip install -e ".[pdf]"' ) from exc matplotlib.rcParams["svg.hashsalt"] = "autocad-mcp-source-review-v1" projects = sorted(data["projects"], key=lambda item: item["score"]) labels = [item["repository"] for item in projects] scores = [item["score"] for item in projects] grades = [item["evidence_grade"] for item in projects] colors = [ "#0969da" if label == "U-C4N/Autocad-MCP" else "#6e7781" for label in labels ] fig, ax = plt.subplots(figsize=(12, 6.4), constrained_layout=True) fig.patch.set_facecolor("#f6f8fa") ax.set_facecolor("#f6f8fa") bars = ax.barh(labels, scores, color=colors, height=0.62) ax.set_xlim(0, 100) ax.set_xlabel("Weighted capability score / 100", color="#24292f", labelpad=10) ax.set_title( "Source-reviewed capability benchmark", loc="left", fontsize=18, fontweight="bold", color="#24292f", pad=28, ) ax.text( 0, 1.02, f"Fixed rubric · reviewed {data['reviewed_at']} · evidence grade shown per project", transform=ax.transAxes, color="#57606a", fontsize=10, ) for bar, score, grade in zip(bars, scores, grades, strict=True): ax.text( min(score + 1.2, 96), bar.get_y() + bar.get_height() / 2, f"{score} · {grade}", va="center", color="#24292f", fontweight="bold", ) ax.text( 0, -0.17, "Shared live-run results are not implied. Scores reflect public-source and recorded test evidence.", transform=ax.transAxes, color="#57606a", fontsize=9, ) ax.grid(axis="x", color="#d0d7de", linewidth=0.8, alpha=0.8) ax.set_axisbelow(True) ax.tick_params(axis="x", colors="#57606a") ax.tick_params(axis="y", colors="#24292f", length=0, pad=8) for spine in ax.spines.values(): spine.set_visible(False) output.parent.mkdir(parents=True, exist_ok=True) fig.savefig( output, format="svg", facecolor=fig.get_facecolor(), metadata={"Creator": "AutoCAD MCP Pro benchmark renderer", "Date": None}, ) plt.close(fig) def build_parser() -> argparse.ArgumentParser: parser = argparse.ArgumentParser(description=__doc__) parser.add_argument("--input", type=Path, default=DEFAULT_INPUT) parser.add_argument("--output", type=Path, default=DEFAULT_OUTPUT) return parser def main() -> int: args = build_parser().parse_args() render_chart(load_benchmark(args.input), args.output) print(f"Rendered benchmark chart: {args.output}") return 0 if __name__ == "__main__": raise SystemExit(main()) ``` - [ ] **Step 3: Run the focused tests** Run: ```powershell python -m pytest tests/test_benchmark_chart.py -q ``` Expected: `3 passed` when Matplotlib is installed; otherwise the render test is skipped and the two contract tests pass. - [ ] **Step 4: Render the committed SVG** Run: ```powershell python -m benchmarks.render_chart ``` Expected: `Rendered benchmark chart:` followed by `docs\assets\autocad-mcp-benchmark.svg`; the file begins with an XML declaration and contains all six project labels. - [ ] **Step 5: Commit the benchmark data, renderer, and asset** ```powershell git add -- benchmarks/source_review.json benchmarks/render_chart.py docs/assets/autocad-mcp-benchmark.svg git commit -m "feat: add reproducible benchmark visualization" ``` Expected: only the dataset, renderer, and SVG are committed. ### Task 4: Document both benchmark evidence layers **Files:** - Modify: `benchmarks/README.md:1-67` - Test: `tests/test_benchmark_chart.py` - [ ] **Step 1: Add a source-review section before the runtime runner section** Add this content after the `# Benchmarks` heading: ```markdown ## Two evidence layers This repository publishes two deliberately separate forms of evidence: 1. **Source-reviewed capability benchmark.** Six named AutoCAD MCP projects are scored with a fixed 100-point rubric covering CAD breadth, correctness and delivery, backend reach, engineering production, tests, and security. The dated data lives in [`source_review.json`](source_review.json), and the README graphic is generated—not hand-edited—from that file. 2. **Fixed-task runtime benchmark.** Adapters execute the same ten tasks and return `pass`, `partial`, `unsupported`, `fail`, `timeout`, or `not_run`. AutoCAD MCP Pro currently has the reference adapter; other projects do not receive runtime scores until an adapter actually runs their public interface. Regenerate the source-review chart: ```bash pip install -e ".[pdf]" python -m benchmarks.render_chart ``` The source-review chart is not presented as a shared live AutoCAD run. Review dates, evidence grades, project URLs, and the boundary are stored with the data. ``` Keep the existing fixed-task and correctness-suite commands below it. Change the existing `## Fixed-task competitor runner v2` heading to `## Fixed-task runtime runner v2` so the distinction is visible in navigation. - [ ] **Step 2: Verify benchmark documentation references** Run: ```powershell rg -n 'source_review\.json|benchmarks\.render_chart|runtime benchmark|not presented as a shared live' benchmarks/README.md ``` Expected: all four concepts appear in the updated benchmark documentation. - [ ] **Step 3: Commit benchmark documentation** ```powershell git add -- benchmarks/README.md git commit -m "docs: explain benchmark evidence boundaries" ``` Expected: the commit contains only `benchmarks/README.md`. ### Task 5: Replace the public README with concise, evidence-led copy **Files:** - Modify: `README.md:1-585` - Reference: `.env.example` - Reference: `pyproject.toml` - Reference: `version.py` - Reference: `server.py` - [ ] **Step 1: Collect authoritative local claims before writing** Run: ```powershell python -m pytest --collect-only -q rg -c '^@mcp\.tool' server.py rg -c '^@mcp\.resource' server.py rg -c '^@mcp\.prompt' server.py Get-Content -LiteralPath pyproject.toml -Raw Get-Content -LiteralPath .env.example -Raw ``` Expected: fresh test collection and decorator counts are available. Use exact counts only in prose tied to this release; describe `system_about` as runtime-authoritative. - [ ] **Step 2: Replace the hero, quickstart, and benchmark opening** Use this exact opening structure and copy, substituting only the fresh test count if it is mentioned later: ```markdown # AutoCAD MCP Pro Production-grade AutoCAD automation for AI agents—live through COM on Windows, or headless through ezdxf on any platform. [![License: MIT](https://img.shields.io/badge/license-MIT-3fb950)](LICENSE) [![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-1f6feb?logo=python&logoColor=white)](pyproject.toml) [![FastMCP 3](https://img.shields.io/badge/MCP-FastMCP%203-8957e5)](https://github.com/jlowin/fastmcp) [![Stars](https://img.shields.io/github/stars/U-C4N/Autocad-MCP?style=flat&color=e3b341)](https://github.com/U-C4N/Autocad-MCP/stargazers) One typed MCP contract controls two execution engines. Build and edit drawings, apply engineering standards, inspect exact geometry, refine quality issues inside transactions, and deliver hashed artifacts with validation evidence. ## Start in 60 seconds ```bash git clone https://github.com/U-C4N/Autocad-MCP.git cd Autocad-MCP python -m venv .venv # Windows: .venv\Scripts\activate # macOS/Linux: source .venv/bin/activate pip install -e . python server.py ``` That starts the headless-capable stdio server. For live AutoCAD control on Windows, install the COM extra and select the backend: ```powershell pip install -e ".[com]" $env:AUTOCAD_MCP_BACKEND = "com" python server.py ``` ## Evidence, not adjectives ![Source-reviewed AutoCAD MCP capability benchmark](docs/assets/autocad-mcp-benchmark.svg) The chart is generated with Matplotlib from [`benchmarks/source_review.json`](benchmarks/source_review.json). It applies the same public 100-point rubric to six named projects. This is a dated source review, not a claim that every project completed the shared live AutoCAD task runner. ```bash pip install -e ".[pdf]" python -m benchmarks.render_chart python -m benchmarks.run_competitors --server autocad-mcp-pro --backend ezdxf --json ``` The second command regenerates the graphic. The third runs AutoCAD MCP Pro's ten-task reference adapter; cross-project runtime scores will appear only after equivalent adapters execute the same contract. See the full [`benchmarks/` methodology](benchmarks/README.md). ``` - [ ] **Step 3: Add four proof-led differentiators and the backend table** Use four compact subsections—`One contract, two engines`, `Geometry agents do not have to guess`, `Quality is a loop`, and `Delivery carries evidence`. Name concrete tools such as `point_from_snap`, `drawing_critique`, `drawing_refine`, `drawing_finalize`, and `drawing_deliver`; avoid comparative superlatives. Include this backend table: ```markdown | Capability | COM backend | ezdxf backend | |---|:---:|:---:| | Live AutoCAD document control | ✓ | — | | Headless DXF creation and editing | — | ✓ | | Cross-platform execution | — | ✓ | | Deterministic entity/query contract | ✓ | ✓ | | Transactions and rollback | ✓ | ✓ | | TABLE and MLEADER semantics | Native | Portable composite | | Screenshots | AutoCAD window | Matplotlib render | | Raw AutoCAD commands / AutoLISP | Opt-in | — | ``` Follow it with: `Use system_capabilities at runtime instead of assuming backend support.` - [ ] **Step 4: Add installation, MCP configuration, workflow, security, and development sections** Keep the existing valid installation extras from `pyproject.toml`. Include one Claude Desktop JSON example using the absolute path placeholder `C:\\path\\to\\Autocad-MCP\\server.py`, and state that Cursor/Cline/Continue/Goose use the same stdio command/args shape. Include one representative workflow, not an exhaustive catalog: ```text drawing_preflight → drawing_plan → drawing_apply_iso_layers → deterministic create/edit tools → dimension_auto → drawing_critique → drawing_refine → drawing_finalize → drawing_deliver ``` Explain that `system_about` and `system_capabilities` provide the authoritative runtime inventory. Preserve concise configuration and security facts backed by `.env.example`, `config.py`, and `security.py`: allowed paths, HTTP bearer auth, non-loopback bind guard, dangerous command opt-in, AutoLISP allowlist, audit logging, and COM timeouts. Use these development commands: ```bash pip install -e ".[full]" python -m pytest python -m ruff check . python -m ruff format --check . python -m build ``` End with compact `Contributing`, `Author`, and `License` sections. Link Umutcan Edizsalan and Anka-Makine only where an existing public URL is known; otherwise use plain text. Link the MIT license to `LICENSE`. - [ ] **Step 5: Remove README anti-patterns** Run: ```powershell rg -ni 'the best|best AutoCAD|only one|without a single misstep|table of contents|public benchmark suite is on the way' README.md ``` Expected: no matches. Also verify the README does not reproduce the long private source-review report or claim live cross-project execution. - [ ] **Step 6: Commit the README rewrite** ```powershell git add -- README.md git commit -m "docs: rebuild readme around verified evidence" ``` Expected: the commit contains only `README.md`. ### Task 6: Verify generated evidence, documentation, code quality, and workspace hygiene **Files:** - Modify if verification exposes a defect: `benchmarks/source_review.json` - Modify if verification exposes a defect: `benchmarks/render_chart.py` - Modify if verification exposes a defect: `benchmarks/README.md` - Modify if verification exposes a defect: `README.md` - Test: `tests/test_benchmark_chart.py` - [ ] **Step 1: Prove chart generation is deterministic** Run: ```powershell $first = Join-Path $env:TEMP 'autocad-mcp-benchmark-first.svg' $second = Join-Path $env:TEMP 'autocad-mcp-benchmark-second.svg' python -m benchmarks.render_chart --output $first python -m benchmarks.render_chart --output $second $hash1 = (Get-FileHash -Algorithm SHA256 -LiteralPath $first).Hash $hash2 = (Get-FileHash -Algorithm SHA256 -LiteralPath $second).Hash if ($hash1 -ne $hash2) { throw "Benchmark SVG is not deterministic." } Compare-Object (Get-Content -LiteralPath $first) (Get-Content -LiteralPath 'docs\assets\autocad-mcp-benchmark.svg') ``` Expected: hashes match and `Compare-Object` produces no output. - [ ] **Step 2: Verify all local README links resolve** Run: ```powershell @' import re from pathlib import Path root = Path.cwd() missing = [] for document in (root / "README.md", root / "benchmarks" / "README.md"): text = document.read_text(encoding="utf-8") for target in re.findall(r"\[[^]]*\]\(([^)]+)\)", text): if target.startswith(("http://", "https://", "#", "mailto:")): continue path_text = target.split("#", 1)[0] if path_text and not (document.parent / path_text).resolve().exists(): missing.append(f"{document.relative_to(root)} -> {target}") if missing: raise SystemExit("Missing local links:\n" + "\n".join(missing)) print("All local README links resolve.") '@ | python - ``` Expected: `All local README links resolve.` - [ ] **Step 3: Run focused and full verification** Run: ```powershell python -m pytest tests/test_benchmark_chart.py -q python -m pytest -q python -m ruff check . python -m ruff format --check . python -m build ``` Expected: focused tests pass; the full suite passes; Ruff check and format check exit 0; sdist and wheel build successfully. If the environment lacks the `build` module, install the development build tool and rerun rather than claiming a package build passed. - [ ] **Step 4: Verify the final diff contains no generated noise or lost product work** Run: ```powershell git status --short --ignored git diff --check git log --oneline -6 ``` Expected: ignored entries contain no lingering cache/output directories; the current version 1.3 product modifications visible before Task 1 remain present unless separately committed by their owner; no whitespace errors are reported; recent commits correspond to the plan's focused changes. - [ ] **Step 5: Commit verification-only fixes if needed** If Steps 1–4 require changes, stage only the affected benchmark/README files and commit them: ```powershell git add -- README.md benchmarks/README.md benchmarks/source_review.json benchmarks/render_chart.py docs/assets/autocad-mcp-benchmark.svg tests/test_benchmark_chart.py git commit -m "fix: align benchmark docs with verification" ``` Expected: no commit is created when verification needed no fixes; otherwise the commit contains only the listed task files.