{"_id":"get-llms-txt","name":"get-llms-txt","dist-tags":{"latest":"1.0.1"},"versions":{"1.0.1":{"name":"get-llms-txt","version":"1.0.1","description":"Generate LLM-friendly llms.txt files from markdown and MDX content files","license":"MIT","homepage":"https://github.com/romankurnovskii/get-llms-txt#readme","repository":{"type":"git","url":"git+https://github.com/romankurnovskii/get-llms-txt.git"},"bugs":{"url":"https://github.com/romankurnovskii/get-llms-txt/issues"},"keywords":["nextjs","llms","get-llms-txt","ai","llm-context","documentation","markdown","mdx","static-site-generator","gatsby","astro","docusaurus","vitepress"],"author":{"name":"Roman Kurnovskii","email":"r.kurnovskii@gmail.com","url":"https://romankurnovskii.com"},"type":"module","sideEffects":false,"exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js"},"./package.json":"./package.json"},"bin":{"get-llms-txt":"dist/bin.js"},"scripts":{"start":"tsx src/bin.ts","build":"tsc","type-check":"tsc --noEmit","lint":"eslint \"{src,tests,example/src}/**/*.{ts,tsx}\"","lint:fix":"eslint --fix \"{src,tests,example/src}/**/*.{ts,tsx}\"","format":"prettier --write \"{src,tests,example/src}/**/*.{js,ts,jsx,tsx}\"","format:check":"prettier --check \"{src,tests,example/src}/**/*.{js,ts,jsx,tsx}\"","test":"jest --config jestconfig.json","test:watch":"jest --config jestconfig.json --watch","test:coverage":"jest --config jestconfig.json --coverage","prepare":"husky","prepublishOnly":"npm run type-check && npm run build && npm test && npm run format:check && npm run lint"},"lint-staged":{"*.{ts,tsx,js,jsx}":["eslint --fix","prettier --write"]},"dependencies":{"commander":"^14.0.2","glob":"^11.1.0"},"devDependencies":{"@types/glob":"^8.1.0","@types/jest":"^30.0.0","@types/node":"^25.0.3","@typescript-eslint/eslint-plugin":"^8.50.1","@typescript-eslint/parser":"^8.50.1","eslint":"^9.39.2","eslint-config-prettier":"^10.1.8","eslint-plugin-prettier":"^5.5.4","husky":"^9.1.7","jest":"^30.2.0","lint-staged":"^16.2.7","prettier":"^3.7.4","ts-jest":"^29.4.6","tsx":"^4.21.0","typescript":"^5.9.3"},"engines":{"node":">=22.0.0"},"gitHead":"18cd4b827455a2fa0dad538a148d7a86866a9158","_id":"get-llms-txt@1.0.1","_nodeVersion":"25.2.1","_npmVersion":"11.6.2","dist":{"integrity":"sha512-fHUK842hISjPvI7091hcNtawBEAIfA+MURFpqYx3nwTWuGrYRIugO7ZCQEPHHC18ud1IRgxQs4U0YWXviai99A==","shasum":"836e2efe40e0682e3d927a456ce001ba26a59624","tarball":"https://registry.npmjs.org/get-llms-txt/-/get-llms-txt-1.0.1.tgz","fileCount":11,"unpackedSize":40822,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEUCIH+KLmZmCcQr5VDUTuYicQ5yJeCFh3i9Jn6sTk9FS48LAiEA5SYvfOG/mYjfFt5sU9a1yczo0AYd/6SbPSvdxi0ivlA="}]},"_npmUser":{"name":"romankurnovskii","email":"r.kurnovskii@gmail.com"},"directories":{},"maintainers":[{"name":"romankurnovskii","email":"r.kurnovskii@gmail.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/get-llms-txt_1.0.1_1766860225159_0.044224936936370085"},"_hasShrinkwrap":false}},"time":{"created":"2025-12-27T18:30:25.158Z","1.0.1":"2025-12-27T18:30:25.308Z","modified":"2025-12-27T18:30:25.597Z"},"maintainers":[{"name":"romankurnovskii","email":"r.kurnovskii@gmail.com"}],"description":"Generate LLM-friendly llms.txt files from markdown and MDX content files","homepage":"https://github.com/romankurnovskii/get-llms-txt#readme","keywords":["nextjs","llms","get-llms-txt","ai","llm-context","documentation","markdown","mdx","static-site-generator","gatsby","astro","docusaurus","vitepress"],"repository":{"type":"git","url":"git+https://github.com/romankurnovskii/get-llms-txt.git"},"author":{"name":"Roman Kurnovskii","email":"r.kurnovskii@gmail.com","url":"https://romankurnovskii.com"},"bugs":{"url":"https://github.com/romankurnovskii/get-llms-txt/issues"},"license":"MIT","readme":"

get-llms-txt
Generate LLM-friendly llms.txt files from markdown and MDX content

\n\n

\n Generate llms.txt files and markdown versions of all content files for LLM consumption, following the llms.txt specification. Optimized for Next.js projects using MDX/MD files, but works with any project that uses markdown or MDX content.\n

\n\n[![NPM version][npm-image]][npm-url]\n![npm-typescript]\n[![License][github-license]][github-license-url]\n\n## šŸŒŸ Features\n\n- šŸ” **Markdown/MDX scanning** - Recursively finds all `.mdx` and `.md` files in your content directory\n- šŸ“„ **Generate `llms.txt`** - High-level, markdown-like index of your project with categorized file lists\n- šŸ“š **Markdown conversion** - Converts MDX files to plain markdown, removing JSX components and metadata\n- šŸŽ›ļø **Configurable** - Customize content directory, output directory, base URL, and project metadata\n- šŸ§° **CLI + API** - Use as a CLI tool or call programmatically in your own scripts\n- āš” **Fast and efficient** - Only processes content files, ignores `node_modules`, `.next`, and `out` directories\n\n## šŸ“– Usage\n\n### CLI Usage\n\nInstall the package:\n\n```sh\nnpm install --save-dev get-llms-txt\n```\n\nor\n\n```sh\nyarn add -D get-llms-txt\n```\n\nor\n\n```sh\npnpm add -D get-llms-txt\n```\n\nRun the CLI:\n\n```sh\nnpx get-llms-txt\n```\n\nBy default, this will:\n\n- Scan `./content` directory for `.mdx` and `.md` files\n- Generate `llms.txt` in `./out` (if exists) or `./public`\n- Create plain markdown versions in `/md/` directory\n\n**Basic usage for MD/MDX files:**\n\nThe tool processes any `.md` or `.mdx` files in your content directory:\n\n- Extracts metadata (title, description, tags)\n- Converts MDX to plain markdown (removes JSX components)\n- Generates categorized `llms.txt` index file\n- Creates individual `.md` files for each content file\n\n### CLI Options\n\n```sh\nnpx get-llms-txt [options]\n```\n\n**Options:**\n\n- `-c, --content-dir ` - Content directory to scan (default: `./content`)\n- `-o, --output-dir ` - Output directory for `llms.txt` and md files (default: `./out` if exists, otherwise `./public`)\n- `-u, --base-url ` - Base URL for links in `llms.txt` (default: empty)\n- `-n, --project-name ` - Project name for `llms.txt` (default: \"Personal Website & Blog\")\n- `-d, --project-description ` - Project description (default: auto-generated)\n- `-h, --help` - Show help message\n\n### Examples\n\n```sh\n# Use default settings\nnpx get-llms-txt\n\n# Specify custom directories\nnpx get-llms-txt --content-dir ./content --output-dir ./out\n\n# With base URL for production\nnpx get-llms-txt --base-url https://example.com --output-dir ./out\n\n# Custom project name and description\nnpx get-llms-txt --project-name \"My Blog\" --project-description \"Technical blog about software development\"\n```\n\n### Programmatic Usage\n\n```typescript\nimport {generateLlmsFiles} from 'get-llms-txt';\n\nawait generateLlmsFiles({\n contentDir: './content',\n outputDir: './out',\n baseUrl: 'https://example.com',\n projectName: 'My Next.js Project',\n projectDescription: 'A collection of technical content',\n});\n```\n\n## šŸ› ļø Installation\n\n```sh\nnpm install --save-dev get-llms-txt\n```\n\n## API\n\n### `generateLlmsFiles(options: GenerateOptions): Promise`\n\nMain function to generate llms.txt files.\n\n**Parameters:**\n\n- `options.contentDir` (string) - Content directory to scan\n- `options.outputDir` (string) - Output directory for llms.txt and md files\n- `options.baseUrl?` (string) - Base URL for links in llms.txt\n- `options.projectName?` (string) - Project name for llms.txt\n- `options.projectDescription?` (string) - Project description\n\n### `processMDXFile(filePath: string): ProcessedContent`\n\nProcess an MDX file and extract metadata and content.\n\n### `processMDFile(filePath: string): ProcessedContent`\n\nProcess a Markdown file and extract metadata and content.\n\n### `extractTitle(filePath: string, metadata: FileMetadata, content: string): string`\n\nExtract title from file metadata, first H1, or filename.\n\n### `extractDescription(metadata: FileMetadata, content: string): string | undefined`\n\nExtract description from metadata or first paragraph.\n\n## What It Does\n\n1. **Scans Content Directory**: Recursively finds all `.mdx` and `.md` files in the content directory\n2. **Extracts Metadata**: Extracts title, description, and other metadata from MDX files\n3. **Converts to Markdown**:\n - Removes MDX metadata exports\n - Strips JSX components\n - Converts to plain markdown\n4. **Generates Individual .md Files**: Creates markdown versions in `/md/` directory\n5. **Generates llms.txt**: Creates `llms.txt` file in the output directory with:\n - Project name and description\n - Content structure overview\n - Categorized file lists with links\n\n## Output Structure\n\n```\n/\n ā”œā”€ā”€ llms.txt\n ā””ā”€ā”€ md/\n ā”œā”€ā”€ blog/\n ā”‚ ā”œā”€ā”€ post1.md\n ā”‚ ā””ā”€ā”€ ...\n ā”œā”€ā”€ apps/\n ā”‚ ā”œā”€ā”€ app1.md\n ā”‚ ā””ā”€ā”€ ...\n ā””ā”€ā”€ research/\n ā””ā”€ā”€ ...\n```\n\n## File Processing\n\n### MDX Files\n\nThe script supports **two common metadata formats** used in Next.js projects:\n\n1. **YAML Frontmatter** (most common):\n\n ```mdx\n ---\n title: My Post\n description: A great post\n tags: [blog, tech]\n ---\n\n # My Post\n\n Content here...\n ```\n\n2. **Export const metadata** (Next.js MDX format):\n\n ```mdx\n export const metadata = {\n title: 'My Post',\n description: 'A great post',\n tags: ['blog', 'tech'],\n };\n\n # My Post\n\n Content here...\n ```\n\nThe script automatically detects and handles both formats. It also:\n\n- Removes React/JSX components\n- Strips import statements\n- Converts to plain markdown\n- Preserves markdown structure\n\n### MD Files\n\n- Extracts YAML frontmatter (if present)\n- Preserves markdown content as-is\n\n## Integration with Build Process\n\nFor Next.js static export (or any static site generator), add to your build script:\n\n```json\n{\n \"scripts\": {\n \"build\": \"next build && npx get-llms-txt --output-dir ./out\",\n \"deploy:prod\": \"next build && npx get-llms-txt --output-dir ./out --base-url https://example.com && firebase deploy --only hosting\"\n }\n}\n```\n\n## Compatibility\n\nāœ… **Optimized for Next.js projects** using MDX/MD files, but also works with:\n\n- āœ… Any static site generator (Gatsby, Astro, etc.)\n- āœ… Documentation sites (Docusaurus, VitePress, etc.)\n- āœ… Any project with markdown/MDX content files\n\n**Supported formats:**\n\n- āœ… YAML frontmatter (most common)\n- āœ… `export const metadata` (Next.js MDX format)\n- āœ… Mixed formats\n- āœ… Files with no metadata (falls back to filename/H1 extraction)\n- āœ… Any directory structure (recursively scans)\n- āœ… Locale suffixes (`.en`, `.ru`, etc.) are handled automatically\n\n**Requirements:**\n\n- Node.js 22+\n- MDX or MD files in a content directory (configurable)\n- TypeScript support (for programmatic usage)\n\n## URL Generation\n\n- Locale suffixes (`.en`, `.ru`) are removed from filenames\n- All files are converted to `.md` extension\n- Directory structure is preserved\n- URLs are relative to the base URL (if provided)\n- Index files are handled specially (uses directory name)\n\n## License\n\nMIT\n\n[package-name]: get-llms-txt\n[npm-url]: https://www.npmjs.com/package/get-llms-txt\n[npm-image]: https://img.shields.io/npm/v/get-llms-txt\n[github-license]: https://img.shields.io/github/license/romankurnovskii/get-llms-txt\n[github-license-url]: https://github.com/romankurnovskii/get-llms-txt/blob/main/LICENSE\n[npm-typescript]: https://img.shields.io/npm/types/get-llms-txt\n[build-status]: https://github.com/romankurnovskii/get-llms-txt/workflows/CI/badge.svg\n[build-status-url]: https://github.com/romankurnovskii/get-llms-txt\n[install-size]: https://packagephobia.com/badge?p=get-llms-txt\n[install-size-url]: https://packagephobia.com/result?p=get-llms-txt\n","readmeFilename":"README.md","_rev":"1-8ff313378babb1d2c802fc76751da4f9"}