{"_id":"@bouncei/tokenlens","name":"@bouncei/tokenlens","dist-tags":{"latest":"0.0.1"},"versions":{"0.0.1":{"name":"@bouncei/tokenlens","version":"0.0.1","publishConfig":{"access":"public"},"description":"See exactly what's eating your Claude Code context window — and stop it.","type":"module","bin":{"tokenlens":"dist/index.js"},"scripts":{"build":"tsc && chmod +x dist/index.js","dev":"tsx src/index.ts","start":"node dist/index.js","typecheck":"tsc --noEmit","test":"vitest run","prepare":"tsc && chmod +x dist/index.js","prepublishOnly":"pnpm typecheck && pnpm test && pnpm build"},"engines":{"node":">=20"},"license":"MIT","author":{"name":"Joshua Inyang","email":"joshuainyang255@gmail.com"},"homepage":"https://github.com/bouncei/tokenlens#readme","repository":{"type":"git","url":"git+https://github.com/bouncei/tokenlens.git"},"bugs":{"url":"https://github.com/bouncei/tokenlens/issues"},"keywords":["claude","claude-code","tokens","context-window","mcp","anthropic","cli","observability","cost-tracking"],"dependencies":{"@anthropic-ai/tokenizer":"^0.0.4","chokidar":"^4.0.3","kleur":"^4.1.5"},"devDependencies":{"@types/node":"^22.10.0","tsx":"^4.19.0","typescript":"^5.7.0","vitest":"^2.1.0"},"_id":"@bouncei/tokenlens@0.0.1","gitHead":"87e93c5fcd8adedf7049f7050fa0273fdd202329","_nodeVersion":"20.19.6","_npmVersion":"10.8.2","dist":{"integrity":"sha512-uPWbcxN9A1jbHJqKrefNDuSJBXE504DD2bL7VqE//fddu2hlTRs7sAyp44A/gCzuZBp0omXaolHkVRGrJlkKnA==","shasum":"1561f7a7c91b53db0edfd5f228862731694e0570","tarball":"https://registry.npmjs.org/@bouncei/tokenlens/-/tokenlens-0.0.1.tgz","fileCount":45,"unpackedSize":136537,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEUCIHrqG147TLAYEffSb1k4FiVnHhqgqrK2DuAPjx6WHRUfAiEAiWIX9qJuAhueXE4IW6W+ZcdNetsuvON1AR7WCdTyibI="}]},"_npmUser":{"name":"bouncei","email":"joshuainyang255@gmail.com"},"directories":{},"maintainers":[{"name":"bouncei","email":"joshuainyang255@gmail.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/tokenlens_0.0.1_1779717296545_0.32952982992315083"},"_hasShrinkwrap":false}},"time":{"created":"2026-05-25T13:54:56.447Z","0.0.1":"2026-05-25T13:54:56.694Z","modified":"2026-05-25T13:54:56.937Z"},"maintainers":[{"name":"bouncei","email":"joshuainyang255@gmail.com"}],"description":"See exactly what's eating your Claude Code context window — and stop it.","homepage":"https://github.com/bouncei/tokenlens#readme","keywords":["claude","claude-code","tokens","context-window","mcp","anthropic","cli","observability","cost-tracking"],"repository":{"type":"git","url":"git+https://github.com/bouncei/tokenlens.git"},"author":{"name":"Joshua Inyang","email":"joshuainyang255@gmail.com"},"bugs":{"url":"https://github.com/bouncei/tokenlens/issues"},"license":"MIT","readme":"# tokenlens\n\n[![ci](https://github.com/bouncei/tokenlens/actions/workflows/ci.yml/badge.svg)](https://github.com/bouncei/tokenlens/actions/workflows/ci.yml)\n[![license](https://img.shields.io/badge/license-MIT-blue.svg)](./LICENSE)\n[![Node.js](https://img.shields.io/badge/node-%E2%89%A520-green)](https://nodejs.org)\n\n**See exactly what's eating your Claude Code context window — and stop it.**\n\n```bash\nnpx -y github:bouncei/tokenlens status\n```\n\nThat's the install. No clone, no build, no config.\n\n---\n\n## Why this exists\n\nClaude Code's built-in `/context` is [documented](https://github.com/anthropics/claude-code/issues/29971) to undercount MCP overhead by ~40k tokens per session. Anthropic was asked to fix it in March 2026 and closed the issue as \"not planned.\" That's the wedge: the vendor's interest is to keep token consumption opaque to maximize per-user revenue; yours is to see exactly where your $200/mo Claude Max budget goes.\n\n`tokenlens` is aligned with your wallet, not theirs.\n\n## What you actually see\n\nRun against my live session as I'm writing this:\n\n```\n$ tokenlens status\ntokenlens status\n  session  ~/.claude/projects/-Users-josh-tokenlens/abc.jsonl\n  turns    278 assistant\n\nTokens consumed by Anthropic\n  input (uncached)           38.9k\n  cache writes               2.07M   ← bloat lives here\n  cache reads               41.21M   ← cheap\n  output                    298.4k\n  cache hit rate             95.2%\n\nOutput by attributionSkill\n  <unattributed>          ████████████████   208.7k   (213 turns)\n  idea-scout              ██████░░░░░░░░░░    92.6k   (65 turns)\n\nInjected context by source\n  cache_invalidation_or_growth  ████████████████ ! 520.3k   across 16 turns\n  deferred_tools_added          █░░░░░░░░░░░░░░░ ~  25.0k   3 ev, 0 B text\n  skill_listing                 ░░░░░░░░░░░░░░░░     1.5k   1 ev, 13.3 KB text\n  hook_additional_context       ░░░░░░░░░░░░░░░░      791   1 ev, 5.5 KB text\n\nMCP servers (loaded vs. invoked)\n  (~50.5k estimated on tools never called)\n  7dfd9fcf…               ████████████ ~  10.3k   ● never called\n  plugin_playwright_playw…███████░░░░░ ~   5.8k   ● never called\n  playwright              ███████░░░░░ ~   5.8k   ● never called\n  Claude_in_Chrome        ██████░░░░░░ ~   5.5k   ● never called\n  filesystem              ████░░░░░░░░ ~   3.5k   ● never called\n  memory                  ███░░░░░░░░░ ~   2.3k   ● never called\n  ...\n  202 dead tools (use --show-dead for full list)\n```\n\nTwo findings worth the install on their own:\n\n1. **~50,500 tokens** of context overhead this session was spent on **MCP tools that were never invoked**. Every server I had loaded — filesystem, memory, playwright, all 10 of them — got definition-injected and never called. That's `/context` reporting \"you're at 60%\" while 25% of the load is dead weight you could disable with one line in `.mcp.json`.\n\n2. **520k tokens** were written to cache as part of TTL-expiry invalidations across 16 separate moments — roughly $4 of cache-write cost on Opus, paid silently across the session and **invisible to `/context`**. `tokenlens` names it as `cache_invalidation_or_growth` so you can see when it's happening.\n\n## Commands\n\n### `tokenlens status`\n\nOne-shot breakdown of the active session. Reports Anthropic's exact per-turn usage from `message.usage`, attribution by `attributionSkill`, and a share-by-weight breakdown of which injected sources contributed to cache writes. A separate `cache_invalidation_or_growth` bucket captures cache writes that can't reasonably be tied to any visible attachment.\n\nFlags:\n- `--cwd <path>` — resolve the active session for a different working directory\n- `--session <file>` — read a specific `.jsonl` directly\n\n### `tokenlens watch`\n\nSame view as `status`, refreshed live as the session log grows. Useful to keep open in a side terminal while you work.\n\n```bash\ntokenlens watch\n```\n\nPress ctrl-c to stop.\n\n### `tokenlens doctor`\n\nDetects (and optionally fixes) the two pathologies most commonly responsible for hidden context growth per Issue #29971:\n\n1. **Stale plugin versions** — `~/.claude/plugins/cache/<source>/<plugin>/` directories still holding previous versions of installed plugins.\n2. **Duplicate skill symlinks** — `~/.claude/skills/` symlinks pointing to the same canonical target, doubling injection of the same skill.\n\nDefault is dry-run; pass `--fix` to actually remove. Every deletion is logged with full path, and the tool refuses to touch anything outside `~/.claude/`.\n\n```bash\ntokenlens doctor          # report only\ntokenlens doctor --fix    # actually delete\n```\n\n### `tokenlens init` and `tokenlens budget` (v2 preview)\n\nConfigure a weekly token budget and check your current session against it:\n\n```bash\ntokenlens init --tier max5      # writes ~/.claude/tokenlens.json with sane defaults\ntokenlens budget                # shows: ALLOW / WARN / ASK / DENY vs your budget\n```\n\nThese are the building blocks for v2's `PreToolUse` hook — see \"v2 preview\" below.\n\n## v2 preview — proactive budget enforcement (testable today)\n\nThe next surface is a Claude Code plugin that runs on every tool call and:\n\n- **Silently allows** if you're under 50% of your weekly cap\n- **Injects a warning** into Claude's context at 50–80% (Claude can decide to wrap up early)\n- **Asks for confirmation** at 80–95% (you click through)\n- **Denies** at 95%+ (or earlier if `hardCap: true` in your config)\n\nIt's already wired up. Test it locally:\n\n```bash\ngit clone https://github.com/bouncei/tokenlens.git\ncd tokenlens\npnpm install                                # builds dist/\ntokenlens init --tier max5                  # writes ~/.claude/tokenlens.json\nclaude --plugin-dir ./tokenlens-cc          # loads the plugin\n```\n\nNow every tool call hits the budget evaluator. Watch the decisions in `~/.claude/tokenlens.log`. The PreToolUse hook contract is fail-allow — a misbehaving hook will never block your workflow.\n\nMarketplace submission to `claude-plugins-community` is pending.\n\n## How it works\n\n`tokenlens` reads `~/.claude/projects/<encoded-cwd>/<session-uuid>.jsonl`. The session log already contains Anthropic's ground-truth per-turn token counts, so for past turns `tokenlens` reports exact numbers — not estimates.\n\nEach line is a JSON object representing one event in the session. The relevant fields:\n\n- `type: \"assistant\"` rows contain `message.usage`: Anthropic's exact per-turn token count (input, cache_creation, cache_read, output, including the ephemeral_5m vs ephemeral_1h cache split).\n- `attributionSkill` on each assistant row identifies which skill Anthropic attributed the turn to.\n- `type: \"attachment\"` rows include the literal content of injected skills, hook context, todo reminders, and MCP-server tool deltas.\n\nWe aggregate these, share-out each turn's cache_creation across the attachments that preceded it (weighted by tokenized content size, or a heuristic for text-less injections), and cap per-event attribution at 1.5× weight so cache invalidations don't get falsely pinned on whatever attachment happened to be most recent.\n\nFull schema notes in [`docs/SCHEMA.md`](./docs/SCHEMA.md).\n\n## Install (other paths)\n\nThe fastest install is `npx`, but if you want a stable binary on `$PATH`:\n\n```bash\n# Globally\npnpm add -g github:bouncei/tokenlens\n# or\nnpm install -g github:bouncei/tokenlens\n\n# From source\ngit clone https://github.com/bouncei/tokenlens.git\ncd tokenlens\npnpm install      # also builds dist/ via the prepare script\n./dist/index.js status\n```\n\nOnce published to npm:\n\n```bash\nnpx tokenlens status\n```\n\nRequires Node.js 20 or newer.\n\n## Roadmap\n\n- **v1 (shipped, v0.1.0):** CLI `status`, `watch`, `doctor`, `init`, `budget`. Free, OSS.\n- **v2 (in progress, testable now):** [`tokenlens-cc`](./tokenlens-cc/) plugin wiring tokenlens into Claude Code's `PreToolUse` hook — proactive budget warnings, hard caps, and session-context injection. Pending community marketplace review. Free tier stays free; paid tier ($9–19/mo) adds cross-machine sync, weekly digest, team views (not yet built).\n- **v3 (next 12 months):** Same value prop, cross-IDE. Cursor, Cline, Gemini CLI.\n\nSee [`DESIGN.md`](./DESIGN.md) for the architecture overview.\n\n## Caveats\n\n- Reverse-engineered against Claude Code v2.1.138 (May 2026). The session jsonl schema may shift on subsequent versions; tokenlens preserves unknown fields and won't crash on them.\n- The local tokenizer (`@anthropic-ai/tokenizer`) tracks the Claude 2 era. Estimates for content we tokenize locally are ±5% vs. server-side counts. **Past-turn numbers from `message.usage` are exact.**\n- The `cache_invalidation_or_growth` bucket lumps together genuine cache TTL expirations and conversation growth — tokenlens can't yet distinguish them without more session signal. Working on it.\n\n## Contributing\n\nIssues and PRs welcome. The project is small and the surface is well-bounded — start with [`docs/SCHEMA.md`](./docs/SCHEMA.md) to understand the session jsonl format, then [`CONTRIBUTING.md`](./CONTRIBUTING.md) for how to add a new attachment category or analyzer.\n\nIf your `~/.claude/` has a structure tokenlens doesn't recognize, please open an issue with a sample line (anonymized) — every new attachment type we map deepens the wedge.\n\n## License\n\nMIT — see [`LICENSE`](./LICENSE).\n","readmeFilename":"README.md","_rev":"1-d14297f192eacc9eead1036ad91b65b1"}