---
summary: Add HtmlExtractor Tier 3 utility — wraps defuddle + linkedom for extracting main article content and metadata from raw HTML into Markdown or cleaned HTML
breaking: false
---

# 0.6.11 — 2026-04-23

Adds `HtmlExtractor`, a new Tier 3 parsing utility for turning raw HTML into clean article content plus best-effort metadata. Built for MCP servers that wrap scholarly or article APIs and need to hand page content to an LLM without hand-rolling extraction.

## Added

- **`HtmlExtractor` (`htmlExtractor` singleton)** in `src/utils/parsing/htmlExtractor.ts` — wraps [`defuddle`](https://github.com/kepano/defuddle) (modern Readability successor, powers Obsidian Web Clipper) together with [`linkedom`](https://github.com/WebReflection/linkedom) for DOM parsing. Exported from `@cyanheads/mcp-ts-core/utils`. One method:
  - `extract(html, options?, context?)` — returns `{ title?, author?, description?, content, domain?, favicon?, image?, language?, metaTags?, parseTime?, published?, schemaOrgData?, site?, wordCount? }`. Only `content` is guaranteed; all other fields are best-effort based on what the source page exposes.
- **Options** on `ExtractArticleOptions`: `format` (`'markdown' | 'html'`, defaults to `'markdown'`), `url`, `contentSelector`, `removeImages`, `debug`, `language`, `useAsync` (off by default — keeps extraction local and deterministic; opt in to allow Defuddle's third-party API fallbacks for SPAs like Twitter).
- **Peer dependencies** (both optional): `defuddle ^0.18.1`, `linkedom ^0.18.12`. Install with `bun add defuddle linkedom`. Cloudflare Workers note: linkedom works in Workers but adds ~150KB minified plus entity tables — factor into your Worker size budget. JSDOM is also supported via defuddle's node entry but is not the default.
- **Tests** at `tests/unit/utils/parsing/htmlExtractor.test.ts` covering clean articles, metadata extraction, boilerplate removal (nav/sidebar/footer), markdown vs HTML output, `contentSelector` override, `removeImages`, empty input (→ `ValidationError`), SPA shells, and malformed HTML.

## Changed

- **`skills/field-test`** rewritten to v2.0 — pivots from "use the MCP tools already connected in your client" to "start the HTTP server locally and drive it with curl + JSON-RPC." Adds a reusable bash helper (`mcp_start`/`mcp_init`/`mcp_call`/`mcp_stop`) that persists PID/URL/session state across tool invocations, replaces the per-definition category matrix with a universal battery (happy path, parity, input error) plus trigger-gated situational categories so it scales to large servers, and tightens the report format (summary paragraph → grouped findings → numbered cherry-pick options).
- **`biome.json`** `$schema` URL bumped to `2.4.13` to match the `@biomejs/biome` version already in devDependencies.

Resolves [#46](https://github.com/cyanheads/mcp-ts-core/issues/46).
