# skill-cli

CLI for managing and testing [Anthropic Agent Skills](https://agentskills.io) (e.g. [anthropics/skills](https://github.com/anthropics/skills)).

## Install (npm)

```bash
# Global
npm install -g ancoder-skill-cli

# Or run without installing
npx ancoder-skill-cli --help
```

The npm package is self-contained and includes prebuilt binaries for:

- macOS arm64
- macOS x64
- Linux arm64
- Linux x64
- Windows x64

After install, the wrapper selects the correct bundled binary for the current platform automatically.

## One-line install (Gitee or intranet)

For users who cannot access GitHub, publish the release-only bundle to Gitee or copy these files to an intranet static file server:

- `scripts/install.ps1`
- `scripts/install.sh`
- `bin/targets/skill-cli-win32-x64.exe`
- `bin/targets/skill-cli-linux-x64`
- `bin/targets/skill-cli-linux-arm64`
- `bin/targets/skill-cli-darwin-arm64`
- `bin/targets/skill-cli-darwin-x64`

Windows users can install the latest Gitee release with PowerShell:

```powershell
irm https://gitee.com/marvin-dev/skill-cli-release/raw/main/scripts/install.ps1 | iex
```

macOS/Linux users can install the latest Gitee release with curl:

```bash
curl -fsSL https://gitee.com/marvin-dev/skill-cli-release/raw/main/scripts/install.sh | bash
```

The installer downloads the matching prebuilt binary, installs it as `skill-cli`, runs `skill-cli install`, then runs a lite `doctor` check. No Go toolchain is required.

The `raw/main` URL is the moving latest channel. Each GitHub release updates Gitee `main` with the newest install bundle. If you need a reproducible install, pin a specific release tag instead:

```powershell
irm https://gitee.com/marvin-dev/skill-cli-release/raw/v0.13.32/scripts/install.ps1 | iex
```

```bash
curl -fsSL https://gitee.com/marvin-dev/skill-cli-release/raw/v0.13.32/scripts/install.sh | bash
```

Before sharing the Gitee command externally, make sure both the Gitee `main` branch and the release tag have been updated. The installer in `main` is generated from the latest release bundle.

Gitee release checklist:

1. Run `go test ./...`, `bash scripts/build-all.sh`, `node scripts/check-bin.js`, and `npm pack --dry-run`.
2. Commit `bin/targets/`, `scripts/install.ps1`, `scripts/install.sh`, `scripts/make-install-bundle.js`, `scripts/check-install-host.js`, `scripts/check-bin.js`, `scripts/build-all.sh`, `package.json`, `package-lock.json`, `.gitattributes`, and `README.md`.
3. Push the release tag to GitHub. The GitHub Actions release workflow builds the binaries and syncs a dist-only bundle to `git@gitee.com:marvin-dev/skill-cli-release.git`.
4. Verify these URLs return `200` before sending the install command:
   - `https://gitee.com/marvin-dev/skill-cli-release/raw/main/scripts/install.ps1`
   - `https://gitee.com/marvin-dev/skill-cli-release/raw/main/scripts/install.sh`
   - `https://gitee.com/marvin-dev/skill-cli-release/raw/main/bin/targets/skill-cli-win32-x64.exe`
   - `https://gitee.com/marvin-dev/skill-cli-release/raw/v0.13.32/scripts/install.ps1`
   - `https://gitee.com/marvin-dev/skill-cli-release/raw/v0.13.32/bin/targets/skill-cli-win32-x64.exe`
   Or run:

   ```bash
   node scripts/check-install-host.js \
     --script-root https://gitee.com/marvin-dev/skill-cli-release/raw/main \
     --base-url https://gitee.com/marvin-dev/skill-cli-release/raw/main/bin/targets
   ```

5. Share the `raw/main` install command only after the check above passes.

GitHub Actions Gitee sync setup:

1. Create a dedicated SSH key for the Gitee release-only repo:

   ```bash
   ssh-keygen -t ed25519 -C "github-actions skill-cli-release" -f ~/.ssh/skill-cli-release-gitee
   ```

2. Add the public key `~/.ssh/skill-cli-release-gitee.pub` to Gitee with write access to `git@gitee.com:marvin-dev/skill-cli-release.git`.
3. Add the private key content from `~/.ssh/skill-cli-release-gitee` to the GitHub repository secret `GITEE_SSH_PRIVATE_KEY`.

The workflow only pushes the generated `dist/install/` bundle to Gitee, so the source repository remains GitHub-only.

For private intranet hosting, generate a static install bundle and upload the generated `dist/install/` directory as-is:

```bash
npm run install-bundle -- --root https://intranet.example.com/skill-cli
```

That directory contains:

- `scripts/install.ps1`
- `scripts/install.sh`
- `bin/targets/skill-cli-*`
- `manifest.json`
- `README.txt`

After uploading `dist/install/` to `https://intranet.example.com/skill-cli/`, users can run:

```bash
node scripts/check-install-host.js --root https://intranet.example.com/skill-cli --all-binaries
```

```powershell
irm https://intranet.example.com/skill-cli/scripts/install.ps1 | iex
```

```bash
curl -fsSL https://intranet.example.com/skill-cli/scripts/install.sh | bash
```

If binaries are hosted somewhere else, point the installer at the directory containing the `skill-cli-*` files:

```powershell
$env:SKILL_CLI_BASE_URL = "https://intranet.example.com/skill-cli/bin/targets"
irm https://intranet.example.com/skill-cli/scripts/install.ps1 | iex
```

```bash
curl -fsSL https://intranet.example.com/skill-cli/scripts/install.sh | \
  bash -s -- --base-url https://intranet.example.com/skill-cli/bin/targets
```

Useful options:

```powershell
# Install the binary only, without writing ~/.claude components
$env:SKILL_CLI_NO_INSTALL = "1"
irm https://gitee.com/marvin-dev/skill-cli-release/raw/main/scripts/install.ps1 | iex

# For a clean smoke test against a temporary Claude directory
$env:SKILL_CLI_NO_INSTALL = $null
$env:SKILL_CLI_CLAUDE_DIR = "$PWD\.skill-cli-test-claude"
irm https://gitee.com/marvin-dev/skill-cli-release/raw/main/scripts/install.ps1 | iex
```

```bash
# Install the binary only, without writing ~/.claude components
curl -fsSL https://gitee.com/marvin-dev/skill-cli-release/raw/main/scripts/install.sh | bash -s -- --no-install

# For a clean smoke test against a temporary Claude directory
curl -fsSL https://gitee.com/marvin-dev/skill-cli-release/raw/main/scripts/install.sh | \
  bash -s -- --claude-dir "$PWD/.skill-cli-test-claude"
```

## Build from source (Go)

```bash
cd skill-cli
go build -o bin/skill-cli .
# Then run: ./bin/skill-cli --help
# Or via npm: node bin/skill-cli.js --help
```

## Commands

| Command | Description |
|--------|-------------|
| `skill-cli validate &lt;path&gt;` | Validate `SKILL.md`, `skill.contract.yaml`, and `evals/*.yaml` |
| `skill-cli list [--path &lt;dir&gt;]` | List installed skills |
| `skill-cli create &lt;name&gt; [--path &lt;dir&gt;]` | Create a skill scaffold with contract and smoke eval templates |
| `skill-cli test &lt;path&gt;` | Check that a skill has trigger docs, contract, and eval coverage |
| `skill-cli verify &lt;path&gt; [--suite smoke]` | Run a machine-readable verification suite end-to-end |
| `skill-cli generate &lt;name&gt; --desc "..."` | Generate with OMC, then run isolated generator/evaluator contract negotiation and review (default) |
| `skill-cli generate &lt;name&gt; --desc "..." --no-adversarial` | Generate with OMC only, skipping the default adversarial review |
| `skill-cli install [--no-omc]` | Install ECC components into `~/.claude/` (includes OMC by default) |
| `skill-cli install --component omc` | Install only the bundled OMC multi-agent orchestration layer |

## Machine-Readable Skill Layout

Task-oriented skills can now include a deterministic verification harness:

```text
my-skill/
├── SKILL.md
├── skill.contract.yaml
├── evals/
│   └── smoke.yaml
├── fixtures/
└── scripts/
```

- `skill.contract.yaml` defines the executable contract: entrypoint, inputs, outputs, invariants, and datasets.
- `evals/*.yaml` defines runnable verification suites with deterministic checks like file existence, required content, and JSON assertions.
- `skill-cli verify` materializes fixture data into a temp workspace, runs the skill entrypoint, and enforces the declared checks.

`skill-cli verify` executes local code declared by the skill contract, so only run it against trusted skills and repositories.

## Adversarial Skill Generation

`skill-cli generate` runs an independent adversarial evaluator pass after the OMC generation pipeline by default:

```bash
skill-cli generate pdf-to-md --desc "Convert PDF files to Markdown"
```

This mode uses two isolated Claude CLI contexts:

- A generator-side `claude -p` process proposes concrete acceptance criteria for the generated skill.
- An evaluator-side `claude -p` process negotiates the contract, runs `skill-cli validate`, `skill-cli test`, and `skill-cli verify`, and writes `.adversarial/diff-report.json`.

The evaluator fails the run when critical issues are found or the score is below `0.80`. Deterministic gate failures are always converted into critical diff items and cap the score below `0.60`.

Use `--no-adversarial` if you want the older OMC-only generation behavior.

### Generation failure logs

Every `skill-cli generate` run writes a local JSONL session log under:

```text
<skills-dir>/.skill-cli/logs/generate/<skill-name>-<timestamp>.jsonl
```

When generation fails, the CLI prints the exact log path. The log records major stages such as Claude binary resolution, scaffold creation, brainstorm probe, loop start/ticks, deterministic gates, repair attempts, and adversarial review failures, including captured command output and error tails for troubleshooting.

## Publish to npm

Releases are published by GitHub Actions, not by running `npm publish` from a developer machine. The workflow in `.github/workflows/release.yml` runs when a `v*` tag is pushed; it builds the release binaries, creates the GitHub Release, syncs a release-only install bundle to Gitee, prepares `bin/targets/` for the npm package, and publishes with the repository `NPM_TOKEN` secret.

1. Bump `package.json` and `package-lock.json` to the intended version.
2. Run local verification:

   ```bash
   go test ./...
   bash scripts/build-all.sh
   node scripts/check-bin.js
   npm pack --dry-run
   ```

3. Commit the source changes, version files, and regenerated binaries under `bin/targets/`.
4. Create and push the release tag:

   ```bash
   git tag v0.13.10
   git push origin v0.13.10
   ```

The CI release publishes binaries named:

   - `skill-cli-darwin-arm64`, `skill-cli-darwin-x64`
   - `skill-cli-linux-x64`, `skill-cli-linux-arm64`
   - `skill-cli-win32-x64.exe`

Users who `npm install -g ancoder-skill-cli` get a fully bundled package. No extra binary download is required during install.

## Test-Driven Skill Development (100:10:1 Architecture)

skill-cli adopts a test-driven approach to skill development, inspired by [oh-my-claudecode](https://github.com/Yeachan-Heo/oh-my-claudecode)'s multi-agent orchestration patterns. The core principle: **invest the majority of compute in building robust test skills, not the skill itself.**

### Time Allocation: 100:10:1

When creating a skill for a task, the system simultaneously creates a **main skill** and a **test skill**:

| Phase | Time Share | Purpose |
|-------|-----------|---------|
| Test skill development | 90% (100 units) | Build an automated evaluator that compares expected vs actual output, locating specific differences |
| Main skill development | 9% (10 units) | Implement the actual skill, guided by test skill feedback |
| Execution & verification | 1% (1 unit) | Final end-to-end smoke test |

### Architecture

```text
Phase 1: Test Skill Development (90% compute)
  generate structured acceptance criteria
  -> N planners generate test strategies in parallel
  -> critic reviews + eliminates weak strategies
  -> N executors implement test skills in parallel
  -> golden test evaluation (tournament selection)
  -> repeat until precision threshold met
  -> best test skill selected

Phase 2: Main Skill Development (9% compute)
  generate main skill
  -> test skill verifies (independent executor)
  -> structured diff feedback injected into next prompt
  -> repeat until test skill passes
  -> main skill complete

Phase 3: Final Verification (1% compute)
  end-to-end smoke test
```

### Key Design Principles

**1. Separation of Author and Reviewer**

The agent that generates the main skill and the agent that runs the test skill operate in separate contexts. This prevents self-approval bias. The verify phase spawns an independent executor to run the test skill, ensuring honest evaluation (borrowed from OMC's verifier lane pattern).

**2. Structured Diff Feedback**

Test skills output structured diff reports instead of simple pass/fail:

```yaml
diffs:
  - location: "page 3, paragraph 2"
    type: "content_loss"
    severity: "critical"
    expected: "table with 3 columns and 5 rows"
    actual: "table missing entirely"
  - location: "page 5, heading"
    type: "format_drift"
    severity: "warning"
    expected: "## Second-level heading"
    actual: "### Third-level heading"
```

This structured feedback is injected back into the main skill's improvement loop, enabling targeted fixes rather than blind retries.

**3. QA Cycling with Early Exit**

Borrowed from OMC's UltraQA pattern:
- Test skill finds issues -> structured diagnosis -> main skill fixes -> retest -> loop
- Same error appearing 3 times triggers early exit (avoids infinite compute burn)
- Maximum 5 QA cycles per iteration

**4. Tournament Selection for Test Skills**

During the 90% test skill development phase, multiple test strategies are generated in parallel and evaluated against golden tests (known-correct input/output pairs). The strategy with the highest detection precision wins, similar to OMC's self-improve tournament selection.

**5. PRD-Driven Acceptance Criteria**

Test skills define concrete, testable acceptance criteria (not vague "implementation is complete"):

```text
Bad:  "PDF conversion works correctly"
Good: "All tables with merged cells are preserved as HTML <table> blocks
       with correct colspan/rowspan attributes"
```

### Example: PDF-to-Markdown Skill

For a PDF-to-Markdown conversion skill:

- **Test skill** (100 min): Compares original PDF content with generated Markdown, detecting content loss (missing paragraphs, tables, images), format drift (heading levels, list styles), and encoding issues. Outputs structured diffs with page/paragraph-level location info.
- **Main skill** (10 min): Implements PDF parsing and Markdown generation, iteratively improved by test skill feedback.
- **Verification** (1 min): End-to-end smoke test on fixture PDFs.

### `skill_eval` Check Type

The verify system supports a `skill_eval` check type that invokes a test skill as a verification oracle:

```yaml
checks:
  - id: quality-check
    type: skill_eval
    skill: pdf-to-md-test
    config:
      threshold: 0.95
      output_format: structured_diff
```

### Verify Phase: Independent Executor

During the loop's verify phase, a separate Claude executor is spawned to run the test skill. This executor:
- Has no shared context with the main skill's executor
- Produces an objective evaluation report
- Returns structured diff feedback that feeds into the next iteration

This mirrors OMC's principle: "Keep authoring and review as separate passes."

## oh-my-claudecode (OMC) Integration

skill-cli embeds the full [oh-my-claudecode](https://github.com/Yeachan-Heo/oh-my-claudecode) multi-agent orchestration bundle (synced from GitHub release **v4.13.6**) and installs it into `~/.claude/omc/` by default. This gives any skill-cli user a single-command path to OMC's agents, skills, hooks, and runtime scripts without needing to clone the OMC repo or configure the plugin marketplace separately.

### What gets installed

When you run `skill-cli install`, OMC is installed alongside ECC components:

| OMC asset | Install target |
|-----------|---------------|
| 19 agents (analyst, architect, executor, planner, critic, verifier, …) | `~/.claude/omc/agents/` |
| 38 skills (autopilot, ralph, ralplan, deep-interview, team, ultrawork, ultraqa, self-improve, …) | `~/.claude/omc/skills/` |
| Runtime scripts (hook helpers, session lifecycle, skill injector, …) | `~/.claude/omc/scripts/` (executable bit preserved for `.sh`/`.mjs`/`.cjs`/`.js`/`.ts`) |
| `hooks.json` | Merged into `~/.claude/settings.json` with `$CLAUDE_PLUGIN_ROOT` rewritten to the absolute OMC install path |
| Templates | `~/.claude/omc/templates/` |
| `.claude-plugin/` manifest, LICENSE, CHANGELOG, VERSION | `~/.claude/omc/` |

### Flags

```bash
# Default — installs ECC + OMC
skill-cli install

# Skip OMC entirely (opt-out)
skill-cli install --no-omc

# Install only the OMC bundle
skill-cli install --component omc

# Preview without writing files
skill-cli install --dry-run
```

### Browse embedded OMC content

```bash
skill-cli list --type omc       # list embedded OMC agents and skills
skill-cli info autopilot        # show the autopilot skill content
skill-cli doctor                # verify OMC install health and version
```

### Why the hook rewrite matters

OMC hooks are authored for the Claude Code plugin system and reference scripts via `$CLAUDE_PLUGIN_ROOT/scripts/...`. Because skill-cli installs OMC as a plain directory (not as a marketplace plugin), the installer rewrites `$CLAUDE_PLUGIN_ROOT` → `${claudeDir}/omc` at merge time so hooks resolve correctly without the plugin loader.

If you already have OMC installed via the Claude Code plugin marketplace, the skill-cli install places a separate self-contained copy under `~/.claude/omc/` and will not touch the marketplace install. The two copies can coexist; hooks from both sources will simply fire in sequence.

### Upgrading OMC

The embedded OMC version is pinned to the release tagged in `embedded/omc/VERSION`. To bump it, re-run the sync workflow that downloads a fresh GitHub release tarball into `embedded/omc/` and rebuild.

## Meta-Harness (experimental)

`meta-harness/` is a Python sub-project that implements the outer-loop harness
optimizer from [arXiv:2603.28052](https://arxiv.org/abs/2603.28052) (Stanford, 2026).

### Architecture

```
meta-harness search   ←  outer loop (Python, Claude Code proposer)
      │
      └─ skill-cli eval validate / run / ls / diff   ←  evaluator backend (Go)
                │
                └─ harness.py (user-supplied Python)  ←  inner execution layer
```

**Two independent binaries — intentionally decoupled:**
- `skill-cli` knows nothing about `meta-harness`; it only runs harness candidates and emits scores/traces.
- `meta-harness` knows nothing about OMC internals; it calls `skill-cli` via CLI contract only.

### Quick start

```bash
# Build skill-cli
go build -o bin/skill-cli .

# Install meta-harness
cd meta-harness
python3 -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"

# Run smoke test (no API key needed)
cd ..
bash scripts/meta-harness-smoke.sh

# Real search (requires ANTHROPIC_API_KEY + claude CLI)
meta-harness search \
  --suite meta-harness/domains/text_classification/suite.yaml \
  --out search-runs/run-01 \
  --max-iter 5 \
  --k 2 \
  --seed meta-harness/domains/text_classification/seeds/zero_shot.py \
  --seed meta-harness/domains/text_classification/seeds/few_shot.py \
  --skill-cli bin/skill-cli \
  --samples 20
```

### CLI contract (skill-cli eval)

| Command | Description |
|---|---|
| `skill-cli eval validate <dir>` | Cheap structural check (exit 0 = valid) |
| `skill-cli eval run <dir> --suite <f> --out <d>` | Full eval → scores.json + traces/ |
| `skill-cli eval ls --store <d> [--pareto]` | List / filter candidates |
| `skill-cli eval diff <a> <b> --store <d>` | Code + score diff |

### Tuning

The `meta-harness/src/meta_harness/skill.md` file is the most important lever on search quality.
Per Appendix D of the paper: run 3–5 short iterations (`--max-iter 3`) specifically to
debug and refine it before committing to a full run.

## License

MIT
