{"_id":"@archivator/content-divinator","_rev":"1-e230da6ae4df214dacf333310944471a","name":"@archivator/content-divinator","dist-tags":{"latest":"0.2.0"},"versions":{"0.2.0":{"name":"@archivator/content-divinator","version":"0.2.0","description":"Attempt at guessing stuff, summarize content, based on raw text. A naïve Natural Language Processing toolkit.","keywords":["text","processing","stop","words","normalization"],"homepage":"https://github.com/renoirb/archivator/blob/v3.x-dev/packages/content-divinator","repository":{"type":"git","url":"git+https://github.com/renoirb/archivator.git","directory":"packages/content-divinator"},"license":"MIT","author":{"name":"Renoir Boulanger","email":"contribs@renoirboulanger.com"},"type":"module","exports":{"import":"./dist/index.mjs"},"main":"dist/index.mjs","module":"dist/index.mjs","types":"dist/index.d.ts","scripts":{"api":"use-run-all api-extractor api-documenter fix","api-documenter":"api-documenter markdown --input temp --output docs","api-extractor":"api-extractor run -c config/api-extractor.json --verbose --local","build":"use-run-all clean build:* api","build:node-esm":"use-cross-env BROWSERSLIST='maintained node versions' use-bili -c bili.config.ts --format esm --file-name index.mjs","clean":"use-cross-env use-rimraf .rpt2_cache dist *.*.log temp","fix":"use-cross-env conventions-code-formatter prettier '**/*.{ts,json,md}' --write","lint":"use-cross-env use-eslint --fix 'src/**/*.ts'","prepublishOnly":"rushx build","sort-package-json":"use-cross-env conventions-code-formatter sort-package-json","test":"use-cross-env use-jest --detectOpenHandles","test:snapshots":"use-cross-env use-jest --detectOpenHandles -u"},"dependencies":{"@rushstack/node-core-library":"^3.25.0"},"devDependencies":{"@babel/core":"^7.10.0","@babel/plugin-transform-runtime":"^7.10.0","@babel/preset-env":"^7.10.0","@babel/preset-typescript":"^7.10.0","@babel/runtime-corejs3":"^7.10.0","@microsoft/api-documenter":"^7.8.17","@microsoft/api-extractor":"^7.8.15","@microsoft/node-library-build":"^6.4.0","@renoirb/conventions-code-formatter":"^1.3.0","@renoirb/conventions-use-bili":"^1.3.0","@renoirb/conventions-use-eslint":"^1.2.1","@renoirb/conventions-use-jest":"1.1.1","@renoirb/conventions-use-prettier":"^1.3.0","@renoirb/conventions-use-typescript-3":"^1.2.1","@renoirb/tools-bundling-helpers":"^1.2.1","@types/jest":"^26.0.0","@types/node":"^14.14.11","core-js":"^3.8.0","jest":"^25.5.0","ts-jest":"^25.5.0","tslib":"^2.0.0","typescript":"^3.9.0"},"engines":{"node":">=10"},"publishConfig":{"access":"public"},"bugs":{"url":"https://github.com/renoirb/archivator/issues"},"_id":"@archivator/content-divinator@0.2.0","_nodeVersion":"14.15.1","_npmVersion":"6.14.9","dist":{"integrity":"sha512-PIN9AenlW6RaJMsnuhk7MkyiPz2Q/N+ub8GPr5fqM8Iyx6JNAOLKkWzsx808bQoLB83EjX6j4eCtmnZg2EIBjA==","shasum":"879888f2b3070ceb7283468de19c708a4c179484","tarball":"https://registry.npmjs.org/@archivator/content-divinator/-/content-divinator-0.2.0.tgz","fileCount":55,"unpackedSize":283419,"npm-signature":"-----BEGIN PGP SIGNATURE-----\r\nVersion: OpenPGP.js v3.0.13\r\nComment: https://openpgpjs.org\r\n\r\nwsFcBAEBCAAQBQJf2pl1CRA9TVsSAnZWagAADzoP/R3rEJwsfyPYM10h0Y/w\nbW2B4cA8ZKPUMXoPNAGOzRRlACd0WYKrlLZxWCr3BIKccDU78z9aVyiNazdE\nNQCGoey9722R5JlXhuPeSftu++y4OMH9SCWLVtsRYZx0J+Ow4Liwm1ESytxa\nnyiRCqmMbfgKN8MJDD2AHhCfVc6Yy4M83pAeSfSwIiGT+spVb40ghS0hzkvF\n7rfkFnwg9JuvU04LG8y6NyE2Q2QQh67MJEP8cDfFxiNQkInyQtCLNx5uGXX0\n0aBPVh0m0ScF8Xr5Eirbk4lhahzbxJ868SPMDzr/rOe4u/mjYUD3gmwfR7J3\nwbr/OnjzLGuQckkHwsmoUPxVMSeUuVZZ2iDWBziDlPJ5PGrbfRAxytS2seVO\n1xBwjxwVsoF27w3ESLFZqL592WGy4Bn645PkJAhmaMx9cTiNv/k16qhGhvY8\nxBn1/+SH2WouvfDznQw8HPFkaYd4OsVfsFD0rAPFCnNIkJ7+rOuTbsp+AHdA\nkQlCM3nd0JZ0tkjC5xfWEoGcc1vxoCTrhrDY3wxOcqcLJGimfDOdBrhCSJAW\nrjv4iEl8OtcPCm8Pbe78UQULfhUGNx0aIfcxB5q2urv7dneWlq2gP92UGSKM\ncqC3GdiaQsPpWR4AD7ZMX36vhmIdsLPGRIdm1v/vd4Bw3Yt1ncEbpA11NBa0\noejq\r\n=NXg1\r\n-----END PGP SIGNATURE-----\r\n","signatures":[{"keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA","sig":"MEYCIQDa+C9bFL0eQYN28aBPlkJrJh8P/uT0VRaDNG75zkdy2AIhAOkW7YP6ZiA7ZnRt48FYvepn5s2cQqidW9LBTfVk2bvL"}]},"_npmUser":{"name":"renoirb","email":"hello@renoirboulanger.com"},"directories":{},"maintainers":[{"name":"renoirb","email":"hello@renoirboulanger.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages","tmp":"tmp/content-divinator_0.2.0_1608161652852_0.16890828441536465"},"_hasShrinkwrap":false}},"time":{"created":"2020-12-16T23:34:12.635Z","0.2.0":"2020-12-16T23:34:12.982Z","modified":"2022-04-04T15:18:34.745Z"},"maintainers":[{"name":"renoirb","email":"hello@renoirboulanger.com"}],"description":"Attempt at guessing stuff, summarize content, based on raw text. A naïve Natural Language Processing toolkit.","homepage":"https://github.com/renoirb/archivator/blob/v3.x-dev/packages/content-divinator","keywords":["text","processing","stop","words","normalization"],"repository":{"type":"git","url":"git+https://github.com/renoirb/archivator.git","directory":"packages/content-divinator"},"author":{"name":"Renoir Boulanger","email":"contribs@renoirboulanger.com"},"bugs":{"url":"https://github.com/renoirb/archivator/issues"},"license":"MIT","readme":"# [@archivator/content-divinator][repo-url]\n\n> Attempt at guessing stuff, summarize content, based on raw text. A naïve\n> Natural Language Processing toolkit.\n\nThis is, by no means, an actual attempt at Machine Learning. It’s simply a few\nhelpers to help automate maintenance of metadata from imported content.\n\n[repo-url]:\n  https://github.com/renoirb/archivator/blob/v3.x-dev/packages/content-divinator\n  'Content Divinator'\n[npmjs-package-badge]:\n  https://img.shields.io/npm/v/%40archivator%2Fcontent-divinator?style=flat-square&logo=appveyor&label=npm&logo=npm\n[npmjs-package]: https://www.npmjs.com/package/%40archivator%2Fcontent-divinator\n[bundlesize-badge]:\n  https://img.shields.io/bundlephobia/min/%40archivator%2Fcontent-divinator?style=flat-square\n  'Bundle Size'\n[dependabot-badge]:\n  https://img.shields.io/librariesio/release/npm/%40archivator%2Fcontent-divinator?style=flat-square&logo=appveyor&logo=dependabot\n\n| Version                                      | Size                                 | Dependencies                                                           |\n| -------------------------------------------- | ------------------------------------ | ---------------------------------------------------------------------- |\n| [![npm][npmjs-package-badge]][npmjs-package] | ![npm bundle size][bundlesize-badge] | ![Libraries.io dependency status for latest release][dependabot-badge] |\n\n## Usage\n\n_See also_:\n\n- [API Extractor code-review signature](../../common/reviews/api/content-divinator.api.md)\n- [API Documentor generated docs](./docs/index.md)\n\n### ContentDivinator\n\n```js\nimport { ContentDivinator, utils } from '@archivator/content-divinator'\n\n/**\n * Input text to parse.\n * Let’s say we want to know the most used words\n * so we can guess what it is about\n */\nconst textContent = `\n  How much wood would a woodchuck chuck\n  if a woodchuck could chuck wood?\n  He would chuck, he would, as much as he could,\n  and chuck as much wood as a woodchuck would\n  if a woodchuck could chuck wood.\n`\n\n/**\n * What we want is to count word usage.\n *\n * Let’s say, we’d want to get the following knowledge\n * from the above text.\n *\n * We do not need words such as: a,if,He, they don’t bring any value.\n * They’re called \"Stop words\".\n * They're different for each language.\n */\nconst desiredTextHashMap = {\n  chuck: 5,\n  woodchuck: 4,\n  wood: 4,\n}\n\n/**\n * Stop Words are words that are mostly noise when\n * we try to analyze what it is about.\n *\n * For this purpose, a \"word\" are letters and numbers in sequence.\n *\n * Notice in stopWords, we're not adding 'that', it is present 3 times in the text above.\n */\nconst stopWords = ['a', 'as', 'could', 'he', 'how', 'if', 'much', 'would']\nconst divinator = new ContentDivinator(stopWords)\n/** @type {Map<string, number>} */\nconst textMap = divinator.words(sourceDocumentText)\nconsole.log(textMap)\n// > Map { 'chuck' => 5, 'woodchuck' => 4, 'wood' => 4 }\n// Convert ECMAScript 2015+ Map to an Object.create(null) type of Hash-Map.\n/** @type {Object.<string, number>} */\nconst hashMap = utils.convertMapToRecordHashMap(textMap)\nconsole.log(hashMap)\n// > { chuck: 5, woodchuck: 4, wood: 4 }\n```\n\n## Bookmarks\n\n### Stop Words libraries\n\n- [Brasilian](https://www.npmjs.com/package/brazilian-stop-words)\n- [Turkish](https://github.com/ahmetax/trstop)\n- _English_\n  - [Textlint to find filler words, buzzwords](https://github.com/sapegin/textlint-rule-stop-words)\n\n### Refactor into coroutine?\n\nDuring March 5th refactor work session, [I’ve attempted in making use of\ncoroutine][coroutine-gist]. But I’ve dropped the idea when my [favourite\nJavaScript author said _he hasn’t seen strongly typed usage of\ncoroutine_][renoir-axel-tweets].\n\n[renoir-axel-tweets]: https://twitter.com/renoirb/status/1236386606266953731\n[coroutine-gist]:\n  https://gist.github.com/renoirb/e7d344cb88524800c247c6842e4eb550\n\n- https://github.com/DefinitelyTyped/DefinitelyTyped/blob/master/types/co/index.d.ts\n- https://gist.github.com/OrionNebula/bd2d4339497a2c05e599d7d24038d290\n- https://github.com/danoctavian/node-coroutine-utils\n- https://github.com/wowts/coroutine\n- http://calculist.org/blog/2011/12/14/why-coroutines-wont-work-on-the-web/\n- https://www.bennadel.com/blog/3264-thoughts-on-defining-coroutines-as-class-methods-in-node-js-and-typescript.htm\n","readmeFilename":"README.md"}