# Aspect Extension Guide

> Generated by `context init`. This document is business-agnostic — a complete reference for extending knowledge graphs via Aspects.

For the protocol and bundled code runner package, see [`@c4a/extract`](https://www.npmjs.com/package/@c4a/extract).

## What Is an Aspect

An Aspect is a **structured data source** for the knowledge graph. Each aspect extracts information from a specific origin (source code, docs, APIs, config files, etc.) and outputs standardized Sections, anchored to Entity nodes in the knowledge graph.

Unlike the align/compile pipeline (LLM-driven, designed for unstructured documents), aspects are **fully deterministic, zero LLM**, suitable for extraction tasks with stable, well-defined rules.

### Aspect vs Align/Compile

| | Aspect Projection | Align/Compile |
|---|---|---|
| Input | Structured sources (code, DSL, YAML, token tables) | Unstructured sources (Markdown, docs, chat logs) |
| Extraction | `extract.ts` script, pure algorithm | LLM semantic classification + alignment |
| Speed | Seconds | Minutes (depends on LLM call count) |
| Repeatability / traceability | Repeatable output with source-bound evidence | Depends on LLM quality and human review |
| Best for | Source symbols, API references, design tokens, structured FAQ | Product docs, meeting notes, articles |

---

## Directory Structure

```text
.context/aspects/
├── README.md               ← This file
├── code/
│   ├── aspect.yaml         # Built-in code aspect (generated when selected during init)
│   └── README.md           # Aspect maintainer guide; CLI does not read it
├── <your-aspect>/
│   ├── aspect.yaml         # Required: aspect definition
│   ├── extract.ts          # Aspect plugin module (local mode)
│   └── README.md           # Optional: maintainer guide for humans/agents
└── <another-aspect>/
    └── ...

CLI runtime
└── C4A_ASPECT_RUNNER_SDK    # env var containing the CLI-provided SDK module URL
```

`context init` generates `.context/aspects/README.md` from this guide. The runtime SDK is provided by CLI at capture time; teams should not copy protocol IO helpers into every business aspect.

---

## aspect.yaml — Complete Reference

### Local Script Mode (`schema_version: "context.aspect.v2"`)

```yaml
schema_version: "context.aspect.v2"
name: my-aspect                    # Unique identifier; recommended to match directory name

runner:
  script: ./extract.ts             # Relative to the aspect.yaml directory

output:
  bucket: raw/aspect/<name>/<source-slug>/<snapshot-id>/
  files:                           # Final bucket file list
    - source.yaml                  # Generated by CLI
    - manifest.json                # Generated by CLI
    - source-files.jsonl           # source_path/hash_id/bucket_path mapping
    - sections.jsonl               # section-projection.v2 rows

projection:
  protocol: "section-projection.v2"
  anchor_policy: best-effort       # best-effort = warn and skip missing nodes; strict = fail
  allow_empty: false               # reject row_count=0 by default
  large_deprecate_threshold: 0.5   # reject >50% row drop by default

evidence:
  mode: bucket-files               # bucket-files: source_ref resolves against bucket files
```

### Bundled Runner Mode (`schema_version: "context.aspect.v2"` — code aspect only)

```yaml
schema_version: "context.aspect.v2"
name: code

runner:
  package: "@c4a/extract"          # npm package name
  bin: c4a-extract-code            # binary name

plugins:                           # Optional: language plugins
  - package: "@c4a/extract-ts"
    export: TypeScriptPlugin

output:
  bucket: raw/aspect/code/<source-slug>/<snapshot-id>/
  files:
    - source.yaml
    - manifest.json
    - digests.jsonl
    - source-files.jsonl
    - packages.jsonl
    - symbols.jsonl
    - edges.jsonl
    - _meta.yaml

evidence:
  mode: none                       # none: no evidence files; uses built-in code projection
```

> Custom aspects and the built-in `code` aspect both use `schema_version: "context.aspect.v2"`. Local script mode is expressed by `runner.script`; bundled runner mode is expressed by `runner.package` + `runner.bin`. Generic Section projection is selected by `projection.protocol: "section-projection.v2"`.

### Field Reference

| Field | Required | Description |
|---|---|---|
| `name` | Yes | Unique aspect identifier; used as the argument to `capture --aspect <name>` and `compile --aspect <name>` |
| `runner.script` | Yes for local plugin mode | Local plugin module path, relative to aspect.yaml directory; CLI loads it through the aspect host |
| `runner.package` | Yes for bundled runner mode | npm package name |
| `runner.bin` | Yes for bundled runner mode | Binary name from the npm package |
| `output.bucket` | Yes | Raw snapshot storage path template; `<source-slug>` and `<snapshot-id>` substituted by CLI |
| `output.files` | Yes | Final bucket file list. CLI generates `source.yaml`, `manifest.json`, `source-files.jsonl`, and `sections.jsonl` from rows emitted by the aspect plugin |
| `projection.protocol` | Yes for Section projection | Fixed value `"section-projection.v2"`; absence is only valid for built-in code projection |
| `projection.anchor_policy` | Yes | `strict` = fail on unknown node; `best-effort` = warn, record `skipped_anchor_missing[]`, and skip the row |
| `projection.allow_empty` | Optional | Defaults to `false`; reject empty `sections.jsonl` to prevent accidental mass deprecations. This does not bypass `large_deprecate_threshold` |
| `projection.large_deprecate_threshold` | Optional | Defaults to `0.5`; reject a `sections.jsonl` row-count drop beyond this ratio unless `--allow-large-deprecate` is explicit |
| `evidence.mode` | Yes | `bucket-files` = resolve source_ref from `source-files.jsonl` and read copied evidence files from the bucket; `none` = no evidence |

---

## Aspect Plugin Contract

### Invocation

CLI starts the CLI-provided aspect host and passes the aspect module as a plugin. The aspect file is not a self-executing script; it exports a lifecycle object:

```typescript
export default defineAspect({
  async capture(ctx) {
    // read source files through ctx.source
    // emit rows through ctx.emit.section
  },
});
```

### Plugin Context

The plugin does not receive `workspace_root`, `repo_root`, `bucket_dir`, raw bucket paths, or knowledge file paths. Those are host-internal details. The only workspace access surface is the host API:

| Context API | Description |
|---|---|
| `ctx.aspect` | Logical aspect metadata: `name`, `source_slug`, `source_id`, `snapshot_id` |
| `ctx.source.glob({ root, extensions, recursive })` | List source files by normalized POSIX-relative path under the configured repo root |
| `ctx.source.immediateFiles(root, extensions)` | List immediate files under one source directory |
| `ctx.source.readText(fileOrPath)` | Read a source file through the CLI host boundary |
| `ctx.source.exists(fileOrPath)` | Check whether a source file exists through the CLI host boundary |
| `ctx.code.packages` / `ctx.code.symbols` | Active code aspect index for resolving package and symbol node slugs; may be empty before code compile |
| `ctx.code.findPackage(...)` / `ctx.code.findSymbol(...)` | Convenience lookup over the active code index |
| `ctx.emit.node(row)` | Emit one deterministic documentation node. CLI later materializes `nodes.jsonl` and upserts the node before Section projection |
| `ctx.emit.section(row)` | Emit one projected Section row. CLI later materializes `sections.jsonl`, `source-files.jsonl`, hashes, and `source_ref` |
| `ctx.emit.warning(...)` | Emit a non-fatal diagnostic for the capture result |

### Plugin Responsibilities

Business aspect plugins own only source selection and business mapping:

1. Import the CLI SDK URL from `process.env.C4A_ASPECT_RUNNER_SDK`
2. Export `defineAspect({ capture(ctx) { ... } })`
3. Read source content only through `ctx.source`
4. Emit deterministic documentation nodes through `ctx.emit.node({ node_slug, title, type, tags, parent_slug })` when source files should become standalone knowledge files
5. Emit rows through `ctx.emit.section({ node_slug, kind, summary, content, source/source_path, artifact })`
6. Return normally for success; throw for failure

Plugins must not read or write `.context`, raw buckets, knowledge files, `source-files.jsonl`, or `sections.jsonl` directly. Local execution cannot fully sandbox arbitrary Node APIs, but the protocol boundary deliberately avoids passing workspace file-system paths to the plugin so the same aspect can later run behind a remote or isolated host.

The SDK does not export product-specific constants such as package slug maps. Keep those mappings in the business aspect, or derive them from `ctx.code.packages`.

### CLI Host Responsibilities

| Host capability | Why it belongs in CLI |
|---|---|
| Source file I/O | Keeps workspace access behind one boundary and allows future remote/sandbox hosts |
| source path normalization and boundary checks | Prevents `../`, absolute paths, symlink escapes, and platform-specific drift |
| hash and `source_ref` generation | Keeps `file:<path>#<artifact>@<hash12>` canonical |
| `nodes.jsonl`, `sections.jsonl`, and `source-files.jsonl` generation | Prevents each business plugin from reimplementing JSONL and bucket rules |
| raw bucket publish | Ensures validation passes before atomic rename to `raw/aspect/...` |
| `code_index` package/symbol lookup | Lets custom aspects attach to code aspect nodes without parsing `_sources.yaml` |
| common AST helpers | Keeps TSX/JS/MDX/SCSS parsing deterministic and reusable |

### Example: Lifecycle-Based extract.ts

```typescript
const sdkUrl = process.env.C4A_ASPECT_RUNNER_SDK;
if (!sdkUrl) throw new Error("C4A_ASPECT_RUNNER_SDK is required; run through context capture --aspect");

const { artifactFromRelPath, defineAspect, parseMdxDocument } = await import(sdkUrl);

export default defineAspect({
  async capture(ctx) {
    for (const file of await ctx.source.glob({ root: "docs", extensions: [".md", ".mdx"] })) {
      const doc = parseMdxDocument(await ctx.source.readText(file), file.path);
      ctx.emit.section({
        node_slug: "target-node",
        kind: "description",
        summary: doc.title,
        content: doc.bodyMarkdown,
        source: file,
        artifact: artifactFromRelPath(file.path),
      });
    }
  },
});
```

Use `doc.bodyMarkdown` for Section content. It removes a leading document title when it matches `doc.title`, then shifts remaining headings under the rendered Section container. The renderer applies the final heading offset again from the actual node depth, so nested child-node Sections stay structurally valid. `doc.markdown` remains available when a plugin needs the full normalized document.

---

## Section Projection Protocol (`section-projection.v2`)

### nodes.jsonl Row Schema (Optional)

```jsonc
{
  "node_slug": "pkg-slug/colors",
  "title": "Colors",
  "type": "entity",
  "tags": ["module"],
  "summary": "Color design language for this package.",
  "parent_slug": "pkg-slug",
  "code_package": "pkg-slug",
  "language": "English"
}
```

Use `nodes.jsonl` only for deterministic documentation nodes whose slug and parent are derived mechanically from source paths or fixed project rules. `language` may be set when source-bound documentation uses a different language than the workspace generation language. Compile upserts these nodes before projecting Sections. Missing anchors that are referenced only by `sections.jsonl` are still handled by `anchor_policy`; they are not created implicitly.

### sections.jsonl Row Schema

```jsonc
{
  "node_slug": "pkg-slug/symbol/target",     // Target Entity slug
  "kind": "example",                          // Existing SectionKind enum value
  "summary": "basic.tsx",                     // Optional: existing Section.summary
  "content": "# basic.tsx\n\n...",            // Full section text; enters the full-text index
  "source_ref": "file:rel/path/file.tsx#basic@ab12cd34ef56"  // Traceability reference
}
```

`kind` must be one of the existing C4A SectionKind values: `description`, `spec`, `warning`, `principle`, `decision`, `incident`, `example`, `changelog`, `comparison`, or `faq`. Project-specific labels such as `design-system-color` or `component-description` belong in `summary` / `content` or the `source_ref` fragment, not in `Section.kind`.

`source_ref` uses an alias-free existing code-style locator: `file:<rel-path>[#artifact]@<hash12>`. The fragment is required when one file produces more than one Section. During compile, C4A adds the aspect source to the target node's `sources[]` and persists a canonical reference like `src-N#file:<rel-path>[#artifact]@<hash12>`.

This does not add a new persisted `source_ref` format. Persisted Sections still use the existing code-style `src-N#file:...@hash` form. The plugin emits `source_path + artifact`; CLI generates the alias-free `file:...@hash` form and compile assigns `src-N#` because that alias depends on the target node's `sources[]` order.

Canonical grammar:

```text
file:<escaped-posix-rel-path>[#<escaped-fragment>]@<hash12>
```

The verifier must percent-decode the source_ref path first, then compare it by byte-for-byte string equality: `decode(path) === source-files.jsonl.source_path`. Decode failure, a decoded path that is not a normalized POSIX relative path, absolute paths, `..`, NUL, or no matching `source-files.jsonl` row must fail fast. `/` remains the path separator; reserved characters inside path segments and fragments, including whitespace, `#`, `@`, and `%`, must use URL-style percent encoding. Windows backslashes, unescaped whitespace, unescaped `#` / `@`, and hashes other than 12 lowercase hex characters are protocol errors.

Verify, query, source_ref resolution, and code projection drift checks must not classify `src-N#file:...@hash` as a code source_ref by syntax alone. They must first read the target node's `sources[src-N]`: `aspect:code:*` uses the existing code verifier, while `aspect:<custom>:*` uses the `section-projection.v2` bucket-files verifier. Code ownership must not be inferred from `parseCodeSourceRef(...) !== null` alone.

### Anchor Levels

`node_slug` can target existing code/manual nodes or deterministic documentation nodes declared in `nodes.jsonl`:

| Level | slug Example | Suitable Content |
|---|---|---|
| package root | `my-pkg-slug` | FAQ and short package-level notes |
| deterministic doc | `my-pkg-slug/colors` | Design language pages, token tables, guides |
| module | `my-pkg-slug/module-name` | Code aspect module notes |
| symbol | `my-pkg-slug/symbol/myfunction` | Function/class/component examples and descriptions |

### Projection Identity

No separate projection key field is added to the persisted Section schema. Compile derives identity from existing durable fields:

```text
projection_key = source_id + "\0" + node_slug + "\0" + kind + "\0" + alias_free_locator
alias_free_locator = file:<rel-path>[#artifact]
```

`alias_free_locator` comes from the CLI-generated alias-free `source_ref` without the trailing `@hash12`; it does not include `src-N#`. `src-N` is a persisted alias derived from node `sources[]` and can change when source ordering changes, so it must not enter the projection key. Projection keys must be unique within the same `sections.jsonl`. A duplicate key is a plugin conflict, not a merge or last-wins case; compile must fail fast and report the duplicate key with row numbers.

During compile, each projection key is compared against existing knowledge Sections:

| in jsonl? | in knowledge? | projected Section payload | Action |
|---|---|---|---|
| Yes | No | — | add (create) |
| Yes | Yes | same | noop (skip) |
| Yes | Yes | different | update |
| No | Yes | — | deprecate (reason: source-removed) |

Projection is applied as a batch. CLI reads workspace nodes once, computes the diff, mutates sources and Sections in memory, sorts touched nodes, and then atomically rewrites each touched knowledge root file once. It must not call row-level mdrive write commands for every projected row. Protocol or validation failures stop before any knowledge rewrite; unchanged rows do not rewrite knowledge files.

To avoid mass deprecations caused by plugin or host bugs, compile applies mechanical guards before diffing: `row_count=0` is rejected by default unless `projection.allow_empty: true`; a `sections.jsonl` row-count drop beyond `projection.large_deprecate_threshold` is still rejected unless the user explicitly passes `--allow-large-deprecate`. `projection.allow_empty` only says that an empty capture result is valid; it does not authorize deleting a large set of existing Sections. When the large-deprecate guard fires, the error envelope's `next_action.command` must provide the same-aspect command: `context compile --aspect <name> --allow-large-deprecate`.

### kind Convention

`kind` is not a free-form string. It controls renderer grouping, mount validation, verification, and query behavior. Put project-specific taxonomy in `summary`, content headings, or the `source_ref` fragment.

---

## config.yaml Variable Mechanism

### Syntax

```yaml
# .context/config.yaml
workspace:
  name: my-project
  language: English
  repo_root: ${REPO_ROOT:-../}
```

`${VAR:-default}` — uses `default` when `VAR` is not set or is an empty string. Relative values are resolved to absolute paths relative to the workspace root, i.e. the parent directory of `.context/` in embedded layout.

### Variable Source Priority

1. Shell environment variables (`process.env`) — highest priority, for temporary overrides
2. `.context/.env` file — persistent defaults
3. `${VAR:-default}` fallback — last resort

`.context/.env` is loaded into a scoped env map only; CLI must not mutate global `process.env`. Expansion uses an effective env map (`.context/.env` as the base, shell env overriding duplicate keys), and the same effective env is passed to the aspect host subprocess.

### `.env` File

```bash
# .context/.env
# Generated by context init. Leave empty to use config.yaml's fallback.
REPO_ROOT=

# Override with an absolute source repository path when needed.
# REPO_ROOT=/abs/path/to/source/repo
```

### Usage in Plugins

Plugins do not receive `repo_root`. They use POSIX-relative source paths through `ctx.source`; CLI resolves those paths against the configured repo root:

```typescript
const files = await ctx.source.glob({ root: "docs", extensions: [".mdx"] });
const text = await ctx.source.readText(files[0]);
```

---

## Command Reference

### capture

```bash
# Capture a single aspect:
context capture --aspect my-aspect

# Capture multiple aspects (sequential):
context capture --aspect my-aspect another-aspect

# Code capture keeps its existing command and options:
context capture --code <target-or-module-options>
# Requires `.context/aspects/code/aspect.yaml`; if it is missing, CLI asks to install the code aspect.

# --force: re-run plugin even if snapshot unchanged
context capture --aspect my-aspect --force

# --quiet: suppress progress output
context capture --aspect my-aspect --quiet
```

### compile (Project to Knowledge Graph)

```bash
# Compile a single aspect:
context compile --aspect my-aspect

# Global compile orchestrator: project all aspects, then inspect/advance existing LLM compile work:
context compile --all

# The only entry for code projection:
context compile --aspect code

# Explicitly allow this run after confirming a large deprecate is expected:
context compile --aspect my-aspect --allow-large-deprecate

# Compile specific aspects:
context compile --aspect my-aspect

# No argument to --aspect → list all available aspects:
context compile --aspect
# → Available aspects: code, my-aspect, another-aspect
```

`compile --aspect <name...>` is pure TypeScript, zero LLM: read `sections.jsonl` → diff by projection key → batch-apply add/update/deprecate/noop in memory → atomically rewrite touched knowledge roots + append changelog.

`context compile --all` is the global compile orchestrator. It first runs the aspect projection subphase in a fixed order: `code` first, then custom aspects by lexicographic `aspect.name`; then it inspects the current workspace for an existing LLM compile workflow / pending semantic compile work. If there is no remaining work, it returns a noop summary and exits 0. If the remaining state can be mechanically closed or advanced, CLI does so. If Agent-authored draft, judge input, or semantic answers are required, CLI stops, returns the single canonical `next_action.command` plus `input_schema`, and exits non-zero; it does not make semantic decisions for the Agent.

> `context compile` (without flags) is the existing LLM-driven compile entry/help. `context compile --all` orchestrates both aspect projection and the LLM compile workflow; it is not an aspect-only shortcut.
> Agent command/router may accept inputs such as `compile -all` or `compile all`, but must normalize them to the CLI command `context compile --all`.

**Code aspect note**: `code` has no `projection` section in its aspect.yaml. CLI detects this and uses the built-in code projection (maps `symbols.jsonl` / `packages.jsonl` / `edges.jsonl` to Sections). The command form, projection identity model, and zero-LLM guarantee are identical to `section-projection.v2`.

### Inspect Aspect State

```bash
# List all sources (including aspects):
context source list

# View a specific source:
context source get aspect:<name>:<slug>

# Workspace status (includes knowledge stats):
context status

# Validate integrity:
context verify
```

### Query

```bash
# Aspect-produced Sections are queryable just like any other Section:
context query "button example basic"
context query "color tokens"
```

---

## Relationship with Align / Compile / Query

```
┌──────────────────────────────────────────────────────────┐
│                  Knowledge Graph Consumption              │
│  context query  ←──  Full-text index  ──→  All Sections  │
└──────────────────────────────────────────────────────────┘
                           ▲
        ┌──────────────────┴──────────────────┐
        │                                      │
┌───────┴────────┐                    ┌───────┴────────┐
│ Aspect Project │                    │ Align/Compile  │
│ (deterministic)│                    │ (LLM-driven)   │
│                │                    │                │
│ extract.ts     │                    │ capture raw    │
│   ↓            │                    │   ↓            │
│ sections.jsonl │                    │ align          │
│   ↓            │                    │   ↓            │
│ compile --asp  │                    │ compile        │
│   ↓            │                    │   ↓            │
│ Section[]      │                    │ Section[]      │
└───────┬────────┘                    └───────┬────────┘
        │                                      │
        └──────────────────┬──────────────────┘
                           ▼
              ┌───────────────────────┐
              │ knowledge/entity/     │
              │   <slug>.md           │
              │ changelog.md          │
              └───────────────────────┘
```

- **capture**: Acquire raw data. For aspects, this means running `extract.ts` and writing to `raw/`. For align/compile, this means capturing URLs, documents, or Markdown.
- **compile**: Raw data → knowledge Sections. Aspects use deterministic diff; align outputs go through an LLM review pipeline.
- **query**: Sections from both sources are indexed and retrieved uniformly — consumers do not distinguish provenance.

---

## Updating the Code Aspect Template

When the user chooses to install the code aspect, `context init` generates `aspects/code/aspect.yaml` from the template bundled with the CLI. When you upgrade the CLI, the template may have been updated.

```bash
# View the current CLI-bundled template (read-only):
context aspect template code

# Write the latest template to aspects/code/ (overwrites aspect.yaml, preserves README.md):
context aspect template code --write

# Diff current file against the latest template:
context aspect template code --diff
```

> `--write` will not overwrite `README.md` (user-edited maintainer guide). To reset README.md, delete it first, then `--write`.

---

## FAQ

### Can aspects and align/compile coexist?

Yes. The same Entity can have Sections from both aspect projection and align/compile. Aspect compile only touches Sections whose source namespace belongs to the selected aspect.

### Can an aspect create new Entities?

Yes, but only when the plugin explicitly emits deterministic documentation nodes through `ctx.emit.node`, which become `nodes.jsonl`. This is for mechanically named pages such as `pkg/colors` or `pkg/quick-start`.

`sections.jsonl` alone never creates missing nodes. If a Section points at a node that is neither pre-existing nor declared in `nodes.jsonl`, `anchor_policy: strict` fails and `best-effort` records `skipped_anchor_missing[]`.

### Can sections.jsonl be incrementally updated?

No. Every `context capture --aspect` produces a **full** `sections.jsonl`. The compile step automatically diffs by projection key: unchanged Sections are untouched, new rows are added, changed rows are updated, and removed rows are deprecated.

### How do I debug extract.ts?

```bash
# Run through the CLI host so ctx.source and ctx.emit are available:
context capture --aspect my-aspect --force --verbose

# Inspect the materialized snapshot:
cat .context/raw/aspect/my-aspect/my-aspect/latest/nodes.jsonl | jq .
cat .context/raw/aspect/my-aspect/my-aspect/latest/sections.jsonl | jq .
```

### Are aspect-produced Sections searchable?

Yes. Aspect-created Node title/summary and all Section `content` enter the index. Retrieval does not distinguish between aspect, align, or mdrive provenance.

### Can multiple aspects run in parallel?

`context capture --aspect a b c` and `context compile --aspect a b c` execute sequentially. Different aspects operate on different namespaces and could theoretically run in parallel — a future CLI version may add a `--parallel` flag.
