# OSV Local Cache

This directory holds a per-ecosystem cache of advisories from
[osv.dev](https://osv.dev). It is consumed by the `check_dependency`
and `scan_dependencies` MCP tools so dependency lookups can surface
known vulnerabilities (CVE / GHSA / OSV-ID) in addition to the
curated malicious-packages and typosquat databases.

## Layout

```
vulnerabilities/osv/
├── README.md                  (this file)
├── npm/
│   ├── index.json             package -> advisory index
│   ├── GHSA-xxxx-yyyy-zzzz.json
│   ├── CVE-2024-12345.json
│   └── MAL-2024-1234.json
├── pypi/
├── go/
├── maven/
├── nuget/
├── rubygems/
└── crates/
```

- Each advisory file is the raw OSV record as published by
  osv.dev's bulk export.
- `index.json` is generated by `scripts/ingest-osv.py` and maps
  lower-cased package names → `{id, file, summary, aliases}` so the
  MCP tool can resolve a lookup in O(1) without rescanning every
  file.

## Refresh cadence

Run `scripts/ingest-osv.py` to refresh the cache.

```bash
# Refresh every ecosystem (downloads ~25-200 MB per archive,
# extracts a stride-sampled subset). The default of 500 strikes a
# balance between coverage and repo size; pass --per-ecosystem 0 to
# cache the full archive.
python3 scripts/ingest-osv.py --per-ecosystem 500 --severity-priority

# Cache the full archive for one ecosystem:
python3 scripts/ingest-osv.py --ecosystem npm --per-ecosystem 0
```

The `--severity-priority` flag sorts each archive by qualitative
severity (CRITICAL/HIGH first) before stride-sampling so a constrained
`--per-ecosystem` budget still surfaces the highest-impact advisories.

The script downloads each ecosystem's `all.zip` from
`https://osv-vulnerabilities.storage.googleapis.com/<ECO>/all.zip`,
stride-samples the records (deterministic across runs), and writes
the subset plus a fresh `index.json`.

**Recommended cadence:**

- **Weekly**, as part of the `skills-check update` flow (operators
  pulling the release tarball get the snapshot committed at release
  time).
- **Ad-hoc**, when investigating a specific advisory: re-run with
  `--per-ecosystem 0` to fetch every record for the affected
  ecosystem.

## Why a cache?

The MCP tools must work offline (Devin, Cursor, Copilot, Claude Code
sessions frequently run in network-restricted environments) and
must respond in milliseconds. Querying osv.dev's HTTP API on every
`check_dependency` call would slow tools to a crawl and break in
firewalled CI runners. A local cache trades some staleness for
predictable latency and works without outbound network access.

## Privacy / supply-chain note

The cache contains only public OSV records — no secrets, no
private vulnerabilities. Records are written verbatim from
osv.dev's bulk archive; we do not modify the content. If you find a
record that does not match the upstream copy, that's a bug in
`scripts/ingest-osv.py`.

## Coverage today

This repo ships a deliberately small stride-sampled subset (30
advisories per ecosystem) so the cache is reviewable and the repo
stays a manageable size. Operators who need broader coverage should
re-run the ingest script with a larger `--per-ecosystem` value or
0 (unlimited).
