`: Convert HTML content directly instead of fetching from a URL.\n\n**Examples:**\n\n```bash\n# Convert a Wikipedia page with the Japanese locale\nnpx @aiquants/html-to-markdown https://ja.wikipedia.org/wiki/Node.js --locale ja-JP\n\n# Convert an English page (locale defaults to en-US)\nnpx @aiquants/html-to-markdown https://en.wikipedia.org/wiki/Node.js\n\n# Save to a specific file\nnpx @aiquants/html-to-markdown https://en.wikipedia.org/wiki/Node.js --output ./my-output.md\n\n# Use short option for output\nnpx @aiquants/html-to-markdown https://ja.wikipedia.org/wiki/Node.js --locale ja-JP -o ./nodejs-ja.md\n\n# Convert HTML content directly\nnpx @aiquants/html-to-markdown --html-content 'Sample Title
This is a sample paragraph.
' --output ./sample.md\n\n# Convert HTML content with locale setting\nnpx @aiquants/html-to-markdown --html-content 'Sample Title
This is a localized sample paragraph.
' --locale en-US -o ./sample-en.md\n```\n\n### As a Library\n\n```typescript\nimport { htmlToMarkdown } from '@aiquants/html-to-markdown';\nimport fs from 'fs';\n\nasync function main() {\n // Example 1: Convert from URL\n const url = 'https://en.wikipedia.org/wiki/Node.js';\n const options = {\n locale: 'en-US', // 'en-US' (default) or 'ja-JP'\n };\n\n try {\n const { markdown } = await htmlToMarkdown(url, options);\n fs.writeFileSync('output.md', markdown);\n console.log('Markdown file has been saved as output.md');\n } catch (error) {\n console.error('Error converting HTML to Markdown:', error);\n }\n\n // Example 2: Convert HTML content directly\n const htmlContent = `\n \n \n Sample Title
\n This is a sample paragraph.
\n \n \n \n `;\n\n try {\n const { markdown } = await htmlToMarkdown('', {\n locale: 'en-US',\n htmlContent: htmlContent\n });\n fs.writeFileSync('output-from-html.md', markdown);\n console.log('Markdown file has been saved as output-from-html.md');\n } catch (error) {\n console.error('Error converting HTML to Markdown:', error);\n }\n}\n\nmain();\n```\n\n### As an MCP Server\n\nThis package can be used as a Model Context Protocol (MCP) server, allowing AI assistants and MCP-compatible applications to convert web pages to Markdown.\n\n#### MCP Server Configuration\n\n**For VS Code with MCP extension, add to your `mcp.json`:**\n\n```json\n{\n \"html-to-md\": {\n \"command\": \"npx\",\n \"args\": [\n \"--package=@aiquants/html-to-markdown\",\n \"aiq-html2md-mcp\"\n ],\n \"type\": \"stdio\"\n },\n \"html-to-md-streamable\": {\n \"url\": \"http://127.0.0.1:4001/mcp\",\n \"type\": \"http\",\n \"_comment\": \"Note: You need to start the streamable MCP server separately using 'npx --package=@aiquants/html-to-markdown aiq-html2md-mcp-stream'\"\n }\n}\n```\n\n**For Claude Desktop or other MCP clients, add to your configuration file:**\n\n```json\n{\n \"mcpServers\": {\n \"html-to-markdown\": {\n \"command\": \"npx\",\n \"args\": [\"--package=@aiquants/html-to-markdown\", \"aiq-html2md-mcp\"],\n \"description\": \"Convert HTML content from URLs to Markdown format\"\n }\n }\n}\n```\n\n**Note:** The streamable MCP server runs as an HTTP server on port 4001 and needs to be started separately:\n\n```bash\n# Start the streamable MCP server (requires global installation)\nnpm install -g @aiquants/html-to-markdown\naiq-html2md-mcp-stream\n```\n\nAlternatively, you can use npx to run without global installation:\n\n```bash\n# Using npx with the streamable server binary\nnpx --package=@aiquants/html-to-markdown aiq-html2md-mcp-stream\n```\n\n#### MCP Tools Available\n\n- **html_to_markdown**: Convert HTML content from a URL to Markdown format\n - `url` (required*): The URL of the web page to convert\n - `html_content` (required*): HTML content as a string to convert (alternative to URL)\n - `locale` (optional): Browser locale (`en-US` or `ja-JP`, defaults to `en-US`)\n\n *Either `url` or `html_content` is required\n\n- **save_content_to_file**: Save text content to a file with specified path or directory\n - `content` (required): Text content to save to file (will be saved as Markdown format)\n - `save_path` (optional): Complete file path including filename to save content\n - `save_directory` (optional): Directory path to save content with auto-generated filename\n - `filename` (optional): Base filename to use when save_directory is specified (extension .md will be added automatically)\n\n- **url_to_markdown_file**: Convert web pages or HTML strings directly to Markdown files (combines conversion and file saving)\n - `url` (required*): The URL of the web page to convert and save\n - `html_content` (required*): HTML content as a string to convert and save (alternative to URL)\n - `locale` (optional): Browser locale (`en-US` or `ja-JP`, defaults to `en-US`)\n - `save_path` (optional): Complete file path including filename to save the converted content\n - `save_directory` (optional): Directory path to save the converted content with auto-generated filename\n - `filename` (optional): Base filename to use when save_directory is specified (extension .md will be added automatically)\n\n *Either `url` or `html_content` is required, and either `save_path` or `save_directory` is required\n\n**Streamable MCP Tools (for real-time progress updates):**\n\n- **html_to_markdown_streamable**: Same as `html_to_markdown` but with real-time progress updates\n- **save_content_to_file**: Same file saving functionality as in standard MCP\n- **url_to_markdown_file_streamable**: Same as `url_to_markdown_file` but with real-time progress updates and streaming support\n\n**File Saving Options:**\n\n- **`save_path`**: Specify the complete file path including filename and extension where you want to save the content.\n - Example: `/path/to/output/my-page.md`\n - The directory will be created automatically if it doesn't exist.\n - You cannot use both `save_path` and `save_directory` at the same time.\n\n- **`save_directory`**: Specify only the directory where you want to save the file. The filename will be auto-generated based on the URL.\n - Example: `/path/to/output/` (filename will be auto-generated like `page.md`)\n - For URLs like `https://example.com/articles/my-article`, the filename becomes `my-article.md`\n - The directory will be created automatically if it doesn't exist.\n - You cannot use both `save_path` and `save_directory` at the same time.\n\n**File Saving Behavior:**\n\n- All content is saved as Markdown format (`.md` files)\n- The directory will be created automatically if it doesn't exist\n- With `save_path`: File is saved to the exact specified path\n- With `save_directory`: File is saved with auto-generated filename based on the URL or custom filename if provided\n\n#### MCP Usage Examples\n\n##### Example 1: Basic HTML to Markdown conversion\n\n```json\n{\n \"method\": \"tools/call\",\n \"params\": {\n \"name\": \"html_to_markdown\",\n \"arguments\": {\n \"url\": \"https://example.com\",\n \"locale\": \"en-US\"\n }\n }\n}\n```\n\n##### Example 2: Save content to a specific file\n\n```json\n{\n \"method\": \"tools/call\",\n \"params\": {\n \"name\": \"save_content_to_file\",\n \"arguments\": {\n \"content\": \"# My Content\\n\\nThis is my markdown content.\",\n \"save_path\": \"/path/to/my-file.md\"\n }\n }\n}\n```\n\n##### Example 3: Convert URL directly to Markdown file (One-step operation)\n\n```json\n{\n \"method\": \"tools/call\",\n \"params\": {\n \"name\": \"url_to_markdown_file\",\n \"arguments\": {\n \"url\": \"https://example.com/article\",\n \"save_directory\": \"/path/to/output/\",\n \"filename\": \"my-article\",\n \"locale\": \"en-US\"\n }\n }\n}\n```\n\n##### Example 4: Streamable conversion with real-time progress\n\n```json\n{\n \"method\": \"tools/call\",\n \"params\": {\n \"name\": \"url_to_markdown_file_streamable\",\n \"arguments\": {\n \"url\": \"https://example.com/large-page\",\n \"save_path\": \"/path/to/large-page.md\",\n \"locale\": \"ja-JP\"\n }\n }\n}\n```\n\n### Using MCP Server Programmatically\n\n```typescript\nimport { createMcpServer, createStreamableMcpServer } from '@aiquants/html-to-markdown';\n\n// Create standard MCP server\nconst mcpServer = createMcpServer();\nawait mcpServer.start(8000); // Port 8000\n\n// Create streamable MCP server\nconst streamableMcpServer = createStreamableMcpServer();\nawait streamableMcpServer.start(8000); // Port 8000\n```\n\n## API\n\n### `htmlToMarkdown(urlOrHtml, options?)`\n\nConverts the HTML content of a given URL or HTML string to Markdown.\n\n- `urlOrHtml` (`string`, **required**): The URL of the web page to convert, or when using `htmlContent` option, this can be any string (commonly used as identifier).\n- `options` (`object`, optional): Options for the conversion process.\n\n#### `options` object\n\n- `locale` (`string`, optional): Specifies the locale to use for the browser context and console messages.\n - `'en-US'` (default)\n - `'ja-JP'`\n- `htmlContent` (`string`, optional): HTML content as a string instead of fetching from URL. When provided, the function will convert this HTML content directly instead of fetching content from the `urlOrHtml` parameter.\n\n## How It Works\n\nThis tool follows these steps for conversion:\n\n1. **Fetch HTML (Playwright)**:\n - Launches a browser with Playwright using the specified locale.\n - Navigates to the page and waits until the `networkidle` event, ensuring dynamic content is fully loaded before fetching the HTML.\n\n2. **Pre-process HTML (Rehype)**:\n - `rehype-parse`: Parses the HTML into a HAST (HTML Abstract Syntax Tree).\n - `rehype-raw`: Preserves elements like `script` and `style`.\n - `rehypeSanitizeHtml` (custom plugin): Removes empty comment nodes and unnecessary whitespace.\n - `rehypeAbsoluteLinks` (custom plugin): Converts relative paths in `href` and `src` attributes to absolute paths.\n - `rehypeWikipediaFootnotes` (custom plugin): Transforms Wikipedia footnotes into standard Markdown format.\n - `rehypeSlug` & `rehypeAutolinkHeadings`: Adds IDs to headings and automatically generates anchor links.\n\n3. **Convert to Markdown (Remark)**:\n - `rehype-remark`: Converts the HAST to an MDAST (Markdown Abstract Syntax Tree), with custom handling for links (``) and table cells (``, ` | `).\n - `remark-gfm`: Adds support for GitHub Flavored Markdown (GFM), including tables and strikethrough.\n - `remark-stringify`: Serializes the MDAST into a Markdown string.\n\n## Dependencies and Licenses\n\nThis project is built upon the following open-source software. We are grateful to the developers of these libraries.\n\n| Package | License |\n| --- | --- |\n| @modelcontextprotocol/sdk | MIT |\n| cors | MIT |\n| express | MIT |\n| github-slugger | ISC |\n| happy-dom | MIT |\n| hast | MIT |\n| hast-util-to-html | MIT |\n| playwright | Apache-2.0 |\n| rehype-parse | MIT |\n| rehype-raw | MIT |\n| rehype-remark | MIT |\n| rehype-slug | MIT |\n| remark-gfm | MIT |\n| remark-stringify | MIT |\n| unified | MIT |\n| unist-util-visit | MIT |\n| yargs-parser | ISC |\n\n*This list is generated based on the `dependencies` in `package.json`. For the most accurate and up-to-date license information, please refer to the individual packages.*\n\n## Author\n\n- **GitHub**: [fehde-k](https://github.com/fehde-k)\n- **X (formerly Twitter)**: [@fehdek](https://x.com/fehdek)\n\n## License\n\nMIT\n","readmeFilename":""} |