{"_id":"@ananotherdeveloper/drain3js","_rev":"2-61d22a90b1ef2f2d19abe49a4f018f10","name":"@ananotherdeveloper/drain3js","dist-tags":{"latest":"0.9.11-rev1"},"versions":{"0.9.11":{"name":"@ananotherdeveloper/drain3js","version":"0.9.11","keywords":["drain3","drain","log","parser","template","miner","log-parsing","log-template","log-clustering","streaming"],"author":{"name":"ananotherdeveloper"},"license":"MIT","_id":"@ananotherdeveloper/drain3js@0.9.11","maintainers":[{"name":"ananotherdeveloper","email":"work.rastogi.deepanshu@gmail.com"}],"homepage":"https://github.com/ananotherdeveloper/drain3js#readme","bugs":{"url":"https://github.com/ananotherdeveloper/drain3js/issues"},"dist":{"shasum":"21e54dc47736dbfde72bbf435a4a51e0cb644f31","tarball":"https://registry.npmjs.org/@ananotherdeveloper/drain3js/-/drain3js-0.9.11.tgz","fileCount":47,"integrity":"sha512-gHxXf/DM+A53clxflLbEaoOJuzuMsfAyiz/BMUqz0Vaoje1GOdNguexO2kOV7MiK5P9Yd+DO6I0gA5f5Tu6tOA==","signatures":[{"sig":"MEQCIDqvIv0bGfQEGYyFvBEif7Ee7m8hv/8IkdVJNptah7asAiBB5sZjZ91/CnTzf0wDgzNf8Cu6o57ZW4oZZ5Y55SaGig==","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":122150},"main":"./dist/index.js","type":"module","_from":"file:ananotherdeveloper-drain3js-0.9.11.tgz","types":"./dist/index.d.ts","engines":{"node":">=18"},"exports":{".":{"import":{"types":"./dist/index.d.ts","default":"./dist/index.js"}}},"scripts":{"lint":"biome check .","test":"vitest run","build":"tsc","format":"biome format --write .","prepack":"npm run lint && npm run typecheck && npm run build && npm test","lint:fix":"biome check --write .","typecheck":"tsc --noEmit","test:watch":"vitest"},"_npmUser":{"name":"ananotherdeveloper","email":"work.rastogi.deepanshu@gmail.com"},"_resolved":"/Users/deepanshurastogi/personal-projects/public-repos/drain3js/ananotherdeveloper-drain3js-0.9.11.tgz","_integrity":"sha512-gHxXf/DM+A53clxflLbEaoOJuzuMsfAyiz/BMUqz0Vaoje1GOdNguexO2kOV7MiK5P9Yd+DO6I0gA5f5Tu6tOA==","repository":{"url":"git+https://github.com/ananotherdeveloper/drain3js.git","type":"git"},"_npmVersion":"11.6.1","description":"TypeScript port of Drain3 - persistent & streaming log template miner","directories":{},"_nodeVersion":"24.11.0","dependencies":{"lru-cache":"^11.0.0"},"_hasShrinkwrap":false,"devDependencies":{"vitest":"^2.0.0","typescript":"^5.4.0","@types/node":"^20.0.0","@biomejs/biome":"^2.4.10"},"_npmOperationalInternal":{"tmp":"tmp/drain3js_0.9.11_1775718125759_0.9533733439341117","host":"s3://npm-registry-packages-npm-production"}},"0.9.11-rev1":{"name":"@ananotherdeveloper/drain3js","version":"0.9.11-rev1","description":"TypeScript port of Drain3 - persistent & streaming log template miner","type":"module","main":"./dist/index.js","types":"./dist/index.d.ts","exports":{".":{"import":{"types":"./dist/index.d.ts","default":"./dist/index.js"}}},"scripts":{"build":"tsc","test":"vitest run","test:watch":"vitest","typecheck":"tsc --noEmit","lint":"biome check .","lint:fix":"biome check --write .","format":"biome format --write .","prepack":"npm run lint && npm run typecheck && npm run build && npm test"},"dependencies":{"lru-cache":"^11.0.0"},"devDependencies":{"@biomejs/biome":"^2.4.10","@types/node":"^20.0.0","typescript":"^5.4.0","vitest":"^2.0.0"},"keywords":["drain3","drain","log","parser","template","miner","log-parsing","log-template","log-clustering","streaming"],"author":{"name":"ananotherdeveloper"},"license":"MIT","repository":{"type":"git","url":"git+https://github.com/ananotherdeveloper/drain3js.git"},"homepage":"https://github.com/ananotherdeveloper/drain3js#readme","bugs":{"url":"https://github.com/ananotherdeveloper/drain3js/issues"},"engines":{"node":">=18"},"_id":"@ananotherdeveloper/drain3js@0.9.11-rev1","_integrity":"sha512-HXB9cbzoDE4oyKg/7NxLDMPlcDBc56+Mry4gu0sN8d6Ur5xYSO6yO17HxbdZgXsIwsfGlNZgi2OsxKXMlZg2iw==","_resolved":"/Users/deepanshurastogi/personal-projects/public-repos/drain3js/ananotherdeveloper-drain3js-0.9.11-rev1.tgz","_from":"file:ananotherdeveloper-drain3js-0.9.11-rev1.tgz","_nodeVersion":"24.11.0","_npmVersion":"11.6.1","dist":{"integrity":"sha512-HXB9cbzoDE4oyKg/7NxLDMPlcDBc56+Mry4gu0sN8d6Ur5xYSO6yO17HxbdZgXsIwsfGlNZgi2OsxKXMlZg2iw==","shasum":"e8cd6779bf7958f8b5634dc6a5c44ca4132896a7","tarball":"https://registry.npmjs.org/@ananotherdeveloper/drain3js/-/drain3js-0.9.11-rev1.tgz","fileCount":47,"unpackedSize":122319,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEYCIQC0pH2XqtuoXoqGir1lvnKZWdeOdrw2H9BOWEtPCsArMwIhAOPB/qsmClWw3v+2YmitJKkUgt3GWou3DszA3yOsk5ce"}]},"_npmUser":{"name":"ananotherdeveloper","email":"work.rastogi.deepanshu@gmail.com"},"directories":{},"maintainers":[{"name":"ananotherdeveloper","email":"work.rastogi.deepanshu@gmail.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/drain3js_0.9.11-rev1_1775718871220_0.6091011657740899"},"_hasShrinkwrap":false}},"time":{"created":"2026-04-09T07:02:05.654Z","modified":"2026-04-09T07:14:31.484Z","0.9.11":"2026-04-09T07:02:05.910Z","0.9.11-rev1":"2026-04-09T07:14:31.352Z"},"bugs":{"url":"https://github.com/ananotherdeveloper/drain3js/issues"},"author":{"name":"ananotherdeveloper"},"license":"MIT","homepage":"https://github.com/ananotherdeveloper/drain3js#readme","keywords":["drain3","drain","log","parser","template","miner","log-parsing","log-template","log-clustering","streaming"],"repository":{"type":"git","url":"git+https://github.com/ananotherdeveloper/drain3js.git"},"description":"TypeScript port of Drain3 - persistent & streaming log template miner","maintainers":[{"name":"ananotherdeveloper","email":"work.rastogi.deepanshu@gmail.com"}],"readme":"# drain3js\n\nTypeScript port of [Drain3](https://github.com/logpai/Drain3) - a persistent and streaming log template miner that uses a fixed-depth parse tree.\n\n[![npm version](https://img.shields.io/npm/v/%40ananotherdeveloper%2Fdrain3js.svg)](https://www.npmjs.com/package/@ananotherdeveloper/drain3js)\n[![CI](https://github.com/ananotherdeveloper/drain3js/actions/workflows/ci.yml/badge.svg)](https://github.com/ananotherdeveloper/drain3js/actions/workflows/ci.yml)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n\n## What is Drain3?\n\nDrain3 extracts log templates from raw log messages in a streaming fashion. Given logs like:\n\n```\nConnected to 10.0.0.1\nConnected to 10.0.0.2\nDisk error on /dev/sda1\nDisk error on /dev/sdb2\n```\n\nIt mines templates:\n\n```\nConnected to <IP>\nDisk error on <*>\n```\n\nBased on the paper: *\"Drain: An Online Log Parsing Approach with Fixed Depth Tree\"* by Pinjia He, Jieming Zhu, Zibin Zheng, and Michael R. Lyu (ICWS 2017).\n\n## Installation\n\n```bash\nnpm install @ananotherdeveloper/drain3js\n```\n\n## Quick Start\n\n```typescript\nimport { TemplateMiner, TemplateMinerConfig } from '@ananotherdeveloper/drain3js';\n\nconst config = new TemplateMinerConfig();\nconst miner = new TemplateMiner(null, config);\n\nconst logs = [\n  'Connected to 10.0.0.1',\n  'Connected to 10.0.0.2',\n  'Disk error on /dev/sda1',\n  'Disk error on /dev/sdb2',\n];\n\nfor (const log of logs) {\n  const result = miner.addLogMessage(log);\n  console.log(`Template: ${result.templateMined} (cluster #${result.clusterId})`);\n}\n```\n\n## Features\n\n- **Streaming**: Process logs one at a time - no need to batch\n- **Persistent state**: Save and restore miner state via pluggable persistence handlers\n- **Masking**: Regex-based masking to normalize IPs, numbers, hex values, etc. before mining\n- **Two algorithms**: Standard `Drain` (token-count tree) and `JaccardDrain` (Jaccard similarity)\n- **Parameter extraction**: Extract variable parameters from logs using mined templates\n- **Memory efficient**: Optional LRU-based cluster eviction with configurable max clusters\n- **Inference mode**: Match logs against existing templates without creating new clusters\n\n## Configuration\n\nUse `TemplateMinerConfig` to customize behavior:\n\n```typescript\nimport { TemplateMinerConfig } from '@ananotherdeveloper/drain3js';\n\nconst config = new TemplateMinerConfig();\n\n// Load from a JSON config object\nconfig.load({\n  drain: {\n    engine: 'Drain',         // 'Drain' or 'JaccardDrain'\n    simTh: 0.4,              // Similarity threshold (default: 0.4)\n    depth: 4,                // Parse tree depth (default: 4, minimum: 3)\n    maxChildren: 100,        // Max children per node (default: 100)\n    maxClusters: null,       // Max clusters, null = unlimited (default: null)\n    extraDelimiters: ['_'],  // Additional token delimiters (default: [])\n    parametrizeNumericTokens: true,\n  },\n  masking: {\n    instructions: [\n      { regexPattern: '((?:\\\\d+\\\\.){3}\\\\d+)', maskWith: 'IP' },\n      { regexPattern: '0x[a-fA-F0-9]+', maskWith: 'HEX' },\n      { regexPattern: '\\\\d+', maskWith: 'NUM' },\n    ],\n    maskPrefix: '<',\n    maskSuffix: '>',\n  },\n  snapshot: {\n    snapshotIntervalMinutes: 5,\n    compressState: true,\n  },\n});\n\nconst miner = new TemplateMiner(null, config);\n```\n\nYou can also load configuration from a JSON file:\n\n```typescript\nconfig.load('/path/to/config.json');\n```\n\n## Persistence\n\nSave and restore miner state using persistence handlers:\n\n```typescript\nimport { TemplateMiner, TemplateMinerConfig, FilePersistence } from '@ananotherdeveloper/drain3js';\n\nconst persistence = new FilePersistence('./drain3_state.bin');\nconst config = new TemplateMinerConfig();\nconst miner = new TemplateMiner(persistence, config);\n\n// State is automatically saved based on snapshotIntervalMinutes\n// and whenever a new cluster is created\n```\n\nBuilt-in persistence handlers:\n\n| Handler | Description |\n|---------|-------------|\n| `FilePersistence` | Saves state to a local file |\n| `MemoryBufferPersistence` | Stores state in memory (useful for testing) |\n\nImplement the `PersistenceHandler` interface for custom storage (Redis, S3, etc.):\n\n```typescript\nimport { PersistenceHandler } from '@ananotherdeveloper/drain3js';\n\nclass RedisPersistence implements PersistenceHandler {\n  saveState(state: Buffer): void { /* ... */ }\n  loadState(): Buffer | null { /* ... */ }\n}\n```\n\n## Masking\n\nMask sensitive or variable data before template mining:\n\n```typescript\nconfig.load({\n  masking: {\n    instructions: [\n      { regexPattern: '((?:\\\\d+\\\\.){3}\\\\d+)', maskWith: 'IP' },\n      { regexPattern: '0x[a-fA-F0-9]+', maskWith: 'HEX' },\n      { regexPattern: '(?<=user=)\\\\w+', maskWith: 'USER' },\n    ],\n  },\n});\n```\n\nInput: `Connected to 10.0.0.1 by user=admin`\nAfter masking: `Connected to <IP> by user=<USER>`\n\n## Parameter Extraction\n\nExtract variable parameters from logs using mined templates:\n\n```typescript\nconst result = miner.addLogMessage('Connected to 10.0.0.1');\nconst params = miner.extractParameters(result.templateMined, 'Connected to 10.0.0.1');\n// [{ value: '10.0.0.1', maskName: 'IP' }]\n```\n\n## Inference Mode\n\nMatch logs against existing templates without creating new clusters:\n\n```typescript\n// Training phase\nminer.addLogMessage('User alice logged in');\nminer.addLogMessage('User bob logged in');\n\n// Inference phase\nconst match = miner.match('User charlie logged in');\nif (match) {\n  console.log(`Matched template: ${match.getTemplate()}`);\n}\n```\n\nThe `match()` method supports three search strategies:\n- `'never'` (default) - Only search the tree path\n- `'fallback'` - Tree search first, then full search if no match\n- `'always'` - Always search all clusters for the token count\n\n## API Reference\n\n### `TemplateMiner`\n\n| Method | Description |\n|--------|-------------|\n| `addLogMessage(logMessage)` | Process a log message and return mining result |\n| `match(logMessage, strategy?)` | Match against existing templates |\n| `extractParameters(template, logMessage)` | Extract parameters with mask names |\n| `getParameterList(template, logMessage)` | Extract parameter values only |\n| `saveState(reason)` | Manually save state |\n| `loadState()` | Manually load state |\n\n### `MinerResult`\n\n| Field | Type | Description |\n|-------|------|-------------|\n| `changeType` | `string` | `'cluster_created'`, `'cluster_template_changed'`, or `'none'` |\n| `clusterId` | `number` | ID of the matched/created cluster |\n| `clusterSize` | `number` | Number of logs in the cluster |\n| `templateMined` | `string` | The current template string |\n| `clusterCount` | `number` | Total number of clusters |\n\n### Low-level API\n\nUse `Drain` or `JaccardDrain` directly for advanced use cases:\n\n```typescript\nimport { Drain } from '@ananotherdeveloper/drain3js';\n\nconst drain = new Drain(4, 0.4, 100);\nconst [cluster, changeType] = drain.addLogMessage('Connected to 10.0.0.1');\nconsole.log(cluster.getTemplate());\n```\n\n## Attribution\n\nThis project is a TypeScript port of [Drain3](https://github.com/logpai/Drain3) by IBM Research.\n\nThe Drain algorithm is based on the paper:\n> Pinjia He, Jieming Zhu, Zibin Zheng, and Michael R. Lyu. \"Drain: An Online Log Parsing Approach with Fixed Depth Tree,\" *Proceedings of the 2017 IEEE International Conference on Web Services (ICWS)*, 2017.\n\n## License\n\n[MIT](LICENSE)\n","readmeFilename":"README.md"}