{"_id":"@alien-worlds/aw-workers","_rev":"5-ba77c5412eb9a06922168ed28d646be5","name":"@alien-worlds/aw-workers","dist-tags":{"latest":"0.0.5"},"versions":{"0.0.4":{"name":"@alien-worlds/aw-workers","version":"0.0.4","license":"ISC","_id":"@alien-worlds/aw-workers@0.0.4","maintainers":[{"name":"dallas.johnson","email":"dallasj001@gmail.com"},{"name":"atirmazi","email":"ahmad.t@dacoco.io"}],"homepage":"https://github.com/Alien-Worlds/aw-workers#readme","bugs":{"url":"https://github.com/Alien-Worlds/aw-workers/issues"},"dist":{"shasum":"30bbc7e16a42f10dede57102dd83d3661ef30045","tarball":"https://registry.npmjs.org/@alien-worlds/aw-workers/-/aw-workers-0.0.4.tgz","fileCount":35,"integrity":"sha512-m7cS+uQtcKRyljS7xPccvEcZcLKx6E5O0PnwYCZ5VJPPqhdCrfJonn0ExBC/q4hsLTPjI+CK2edhhUHnCh0tlQ==","signatures":[{"sig":"MEUCIQCRqIkx7jfCc2SozbE2g8vB918GS5kV9xZC5QlkT1UTfAIgBUfd8LS9VjgPIeW4h+W2uQzn8idv48Yqys+oiPa3D1Y=","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}],"unpackedSize":52267},"main":"build/index.js","types":"build/index.d.ts","gitHead":"ffa3ed05353244a4e5bde700aeda57eaf654895f","scripts":{"lint":"eslint . --ext .ts","build":"yarn clean && tsc -b","clean":"rm -rf ./build","format":"prettier --write \"src/\"","lint-fix":"eslint . --ext .ts --fix","test:unit":"jest --config=jest.config.unit.js","prepublish":"yarn clean && tsc --project tsconfig.build.json","format-check":"prettier --check \"src/\""},"_npmUser":{"name":"atirmazi","email":"ahmad.t@dacoco.io"},"repository":{"url":"git+https://github.com/Alien-Worlds/aw-workers.git","type":"git"},"_npmVersion":"9.5.0","description":"Welcome to the Workers package - a part of the Alien Worlds project, an open-source project. Workers is a package built on the node.js `workers_threads` module, enabling scripts to run in parallel.","directories":{},"_nodeVersion":"19.7.0","dependencies":{"async":"^3.2.4","ts-node":"^10.9.1"},"_hasShrinkwrap":false,"packageManager":"yarn@3.2.3","devDependencies":{"jest":"^27.4.5","eslint":"^8.23.1","ts-jest":"^27.1.3","prettier":"^2.7.1","typescript":"^4.8.2","@types/jest":"^27.0.3","@types/node":"^18.7.14","eslint-config-prettier":"^8.5.0","eslint-plugin-prettier":"^4.2.1","@typescript-eslint/parser":"^5.37.0","@typescript-eslint/eslint-plugin":"^5.37.0"},"_npmOperationalInternal":{"tmp":"tmp/aw-workers_0.0.4_1691507404330_0.8787716616437553","host":"s3://npm-registry-packages"}},"0.0.5":{"name":"@alien-worlds/aw-workers","version":"0.0.5","license":"ISC","_id":"@alien-worlds/aw-workers@0.0.5","maintainers":[{"name":"rkamysz","email":"radoslaw.kamysz@gmail.com"},{"name":"lbobka","email":"volodymyr@dacoco.io"},{"name":"dallas.johnson","email":"dallasj001@gmail.com"},{"name":"atirmazi","email":"ahmad.t@dacoco.io"}],"homepage":"https://github.com/Alien-Worlds/aw-workers#readme","bugs":{"url":"https://github.com/Alien-Worlds/aw-workers/issues"},"dist":{"shasum":"4d89e17d666e4a925c3f32728cf6dec16e74ff59","tarball":"https://registry.npmjs.org/@alien-worlds/aw-workers/-/aw-workers-0.0.5.tgz","fileCount":35,"integrity":"sha512-7th9biXDopccNKep5CJbEeSDtxQGDUBOROUZIj/GSIhY/XzKCGPOGFuLgn+N/ZoHjoQizdsSvphdXehvZrrmEw==","signatures":[{"sig":"MEQCIBc0PQc9LP+uvzpuZgffF8I6YGV/8Nsd8CwRXJ6AM7WZAiB7BBxkvNLZ7VXx9o9jMCLGQMsAti9+nYuP7mNsalOvtw==","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}],"unpackedSize":52589},"main":"build/index.js","types":"build/index.d.ts","gitHead":"3ed66f562b8a11708e2adf597f4b7ab2f6b9a423","scripts":{"lint":"eslint . --ext .ts","build":"yarn clean && tsc -b","clean":"rm -rf ./build","format":"prettier --write \"src/\"","lint-fix":"eslint . --ext .ts --fix","test:unit":"jest --config=jest.config.unit.js","prepublish":"yarn clean && tsc --project tsconfig.build.json","format-check":"prettier --check \"src/\""},"_npmUser":{"name":"rkamysz","email":"radoslaw.kamysz@gmail.com"},"repository":{"url":"git+https://github.com/Alien-Worlds/aw-workers.git","type":"git"},"_npmVersion":"8.3.1","description":"Welcome to the Workers package - a part of the Alien Worlds project, an open-source project. Workers is a package built on the node.js `workers_threads` module, enabling scripts to run in parallel.","directories":{},"_nodeVersion":"17.4.0","dependencies":{"async":"^3.2.4","ts-node":"^10.9.1"},"_hasShrinkwrap":false,"packageManager":"yarn@3.2.3","devDependencies":{"jest":"^27.4.5","eslint":"^8.23.1","ts-jest":"^27.1.3","prettier":"^2.7.1","typescript":"^4.8.2","@types/jest":"^27.0.3","@types/node":"^18.7.14","eslint-config-prettier":"^8.5.0","eslint-plugin-prettier":"^4.2.1","@typescript-eslint/parser":"^5.37.0","@typescript-eslint/eslint-plugin":"^5.37.0"},"_npmOperationalInternal":{"tmp":"tmp/aw-workers_0.0.5_1693467302860_0.22605273837239137","host":"s3://npm-registry-packages"}}},"time":{"created":"2023-08-08T15:10:04.197Z","modified":"2026-04-20T11:37:47.826Z","0.0.4":"2023-08-08T15:10:04.517Z","0.0.5":"2023-08-31T07:35:03.088Z"},"bugs":{"url":"https://github.com/Alien-Worlds/aw-workers/issues"},"license":"ISC","homepage":"https://github.com/Alien-Worlds/aw-workers#readme","repository":{"url":"git+https://github.com/Alien-Worlds/aw-workers.git","type":"git"},"description":"Welcome to the Workers package - a part of the Alien Worlds project, an open-source project. Workers is a package built on the node.js `workers_threads` module, enabling scripts to run in parallel.","maintainers":[{"email":"dallasj001@gmail.com","name":"dallas.johnson"},{"email":"volodymyr@dacoco.io","name":"lbobka"},{"email":"muhammad.g@dacoco.io","name":"salman.gurmani"}],"readme":"# Workers\n\nWelcome to the Workers package - a part of the Alien Worlds project, an open-source project. Workers is a package built on the node.js `workers_threads` module, enabling scripts to run in parallel.\n\n## Dependencies\n\n- [async](https://github.com/caolan/async)\n\n## Table of Contents\n\n- [Description](#description)\n- [Components](#components)\n  - [Worker](#worker)\n  - [WorkerLoader](#workerloader)\n  - [WorkerPool](#workerpool)\n    - [Properties](#properties)\n    - [Methods](#methods)\n- [Get Started](#get-started)\n- [Create Worker](#create-worker)\n- [Create WorkerLoader](#create-workerloader)\n- [Use Worker Dependencies](#use-worker-dependencies)\n- [Use Worker Pool](#use-worker-pool)\n- [Contributing](#contributing)\n- [License](#license)\n\n\n\n## Description\n\nThe Workers package enables efficient use of multi-threading via the `worker_threads` module of Node.js. Each worker's code is stored in separate TypeScript files which are then loaded using the worker loader. A worker pool manages all the workers and orchestrates the execution of ordered tasks in the pool.\n\nWorkers run in-memory, hence enabling saving data as native types and making them available to other workers in the pool. The communication between the worker and the pool is done via messages. This package has defined the necessary messages for the proper operation of the entire process.\n\nWorkers can be loaded dynamically in `worker-loader-script`, but that's not the only script that can be externally specified. You can also specify the worker loader path and loader dependencies. Hence, workers can be used for various purposes without needing to modify the base components.\n\n## Components\n\n### Worker\n\n`Worker` is the base class for all processes loaded by the pools. It consists of basic methods, the only one that needs to be overridden is `run()`. A `Worker` has access to `sharedData` which is a shared data container.\n\nA worker operates similarly to a Promise with `progress`, `resolve` and `reject` methods used to communicate with the pool and denote the current work state of a given worker. These methods can pass the appropriate data, facilitating work progress information, and must be called on success or error. Failure to invoke these methods after the completion of work, will result in the worker remaining in the used pool, preventing further task assignment.\n\n### WorkerLoader\n\n`WorkerLoader` is a container that creates `Worker` instances and transfers needed dependencies. Hence, dependencies are not generated with every worker creation - they all use the same. `DefaultWorkerLoader` includes the necessary implementations of the `load` and `setup` methods. However, if your workers require additional settings for specialized operations, you can create your own worker loader and pass its path in the worker pool options.\n\n### WorkerPool\n\n`WorkerPool` is used for creating, managing and deleting workers on demand. In the worker pool options, you can specify the maximum number of threads and the number of threads that cannot be used (to safeguard against the exhaustion of all threads without knowing their count).\n\nOn creation of a worker pool, you can also transfer predefined data in the `sharedData` storage area. Additionally, you can specify the worker loader in which the `Worker` instance and loader dependencies are created. If not specified, a default will be assigned, requiring the path to the worker file each time `getWorker()` is called.\n\n#### Properties:\n\n- `workerMaxCount`: The maximum number of workers in the pool.\n- `workerLoaderPath`: The path to the worker loader script.\n- `workerLoaderDependenciesPath`: The path to the worker loader dependencies.\n- `availableWorkers`: A list of available worker proxies.\n- `activeWorkersByPid`: A map of active workers by their process IDs.\n- `sharedData`: Shared data passed to each worker.\n- `workerReleaseHandler`: The handler function for releasing a worker.\n\n#### Methods:\n\n- `static async create(options: WorkerPoolOptions)`: A static asynchronous function that creates a new instance of the WorkerPool class.\n- `async setup(options: WorkerPoolOptions)`: Sets up the worker pool by creating and initializing the worker proxies.\n- `get workerCount()`: Returns the number of workers in the pool.\n- `private async createWorker()`: An asynchronous function that creates a worker instance.\n- `async getWorker(pointer?: string)`: Retrieves an available worker from the pool, or returns `null` if no worker is available.\n- `async releaseWorker(id: number, data?: unknown)`: Releases a worker back to the pool.\n- `removeWorkers()`: Removes all workers from the pool.\n- `hasAvailableWorker()`: Checks if there is an available worker in the pool. Returns `true` if an available worker exists, `false` otherwise.\n- `hasActiveWorkers()`: Checks if there are active workers in the pool. Returns `true` if active workers exist, `false` otherwise.\n- `countAvailableWorkers()`: Returns the number of available workers in the pool.\n- `countActiveWorkers()`: Returns the number of active workers in the pool.\n- `onWorkerRelease(handler: WorkerReleaseHandler)`: Registers a handler for the worker release event.\n\nPlease refer to the method documentation for more details on how to use each method and the parameters they accept.\n\n\n## Get Started\n\nTo install the `@alien-worlds/aw-workers` package, use the following command:\n\n```bash\nyarn add @alien-worlds/aw-workers\n```\n\n## Create Worker\n\nWhen creating a worker class, the first basic thing you need to remember is to use **default** export or have only one export in the worker file. You also need to know that you can pass data to the worker, but only in the form of native objects. If you pass class instances, they will not contain methods because this data is sent as a message to the thread. Also remember that at the end of the job you have to call resolve rub reject to fire the worker. Just like Promise\n\n\n```typescript\nexport default class YourWorker extends Worker<YourSharedData> {\n  // constructor is only required when you use custom worker loader and you pass extra data to the worker\n  constructor(private dependencies: YourDependencies) {\n    super();\n    // ...\n  }\n  // In short, this is the method called when you call \"run\" on worker from the pool.\n  // Remember that you can only pass native type argument\n  public async run(data: unknown): Promise<void> {\n    try {\n      //...\n      this.resolve({ ... });\n    } catch (error) {\n      this.reject(error);\n    }\n  }\n}\n```\n\n## Create WorkerLoader\n\nIn normal/simple cases, you don't need to set custom worker loader or loader dependencies. We do this only if the worker needs to use 3rd party components and which should not be instantiated every time the worker is started.\n\n```typescript\n// An example of a worker loader that creates loads a specific worker from a given path.\n\nexport default class CustomWorkerLoader extends DefaultWorkerLoader<CustomSharedData, CustomWorkerLoaderDependencies> {\n  protected workers: WorkerContainer;\n\n  public async setup(sharedData: CustomSharedData): Promise<void> {\n    const { workersPath } = sharedData;\n\n    await super.setup(sharedData, workersPath);\n    \n    // some additional setup work\n  }\n\n  // this WorkerLoader will load \n  public async load(pointer: string): Promise<Worker> {\n    const {\n      dependencies: { ...some, workersPath },\n    } = this;\n    const { sharedData } = this;\n    const workerClasses = await import(workersPath);\n    const worker: Worker = new workerClasses[pointer](\n      {\n        ...some,\n        ...\n      },\n      sharedData\n    ) as Worker;\n    return worker;\n  }\n}\n```\n\n## Use Worker Dependencies\n\n**It is not required to create a WorkerLoaderDependencies.** This mechanism was introduced specifically for the benefit of one of the components of the Alien Worlds open source project. All the operations we perform here can also be performed in your custom worker loader itself. It all depends on the concept of your project.\nHowever, when creating WorkerLoaderDependencies you must remember that the class should have the initialize method. In it, you create all the necessary components that will be publicly available to the environment, i.e. worker loader and later passed to the worker.\n\n\n```typescript\nexport class YourWorkerLoaderDependencies extends WorkerLoaderDependencies {\n  public componentA: ComponentA;\n  public componentB: ComponentB;\n  public componentC: ComponentC;\n  ...\n  public async initialize(...args: unknown[]): Promise<void> {\n    const [requiredForA, requiredForB, requiredForC, ...] = args;\n    this.compoentA = await createComponentA(requiredForA);\n    this.compoentB = await createComponentB(requiredForB);\n    this.compoentC = await createComponentC(requiredForC);\n    ...\n  }\n}\n\n```\n\n## Use Worker Pool\n\nThe use of WorkerPool is limited to calling a worker at the right moment according to the logic of your application and reacting to actions coming from the worker. Remember to release a worker from service and return him to the pool after each work done or not done\n\n\n```typescript\n// configure worker pool\nconst workerPool = await WorkerPool.create({\n  threadsCount: 4, // or use inviolableThreadsCount\n  sharedData: { ... },\n  workerLoaderPath: '/path/to/your/worker-loader', // if not passed default will be used\n  workerLoaderDependenciesPath: '/path/to/your/worker-loader/dependencies' // optional\n});\nworkerPool.onWorkerRelease(() => {\n  // Called when a worker is returned to the pool\n});\n// if you won't provide your custom workerLoaderPath you will have to pass path to the worker script\n// eg. workerPool.getWorker('path/to/the/worker');\nconst worker = await workerPool.getWorker();\n\nif (worker) {\n  worker.onMessage(message => {\n    if (message.isTaskResolved()) {\n      // remember to release the worker\n      workerPool.releaseWorker(message.workerId);\n      // ...\n    } else if (message.isTaskRejected()) {\n      // remember to release the worker\n      workerPool.releaseWorker(message.workerId);\n      // ...\n    } else if (message.isTaskProgress()) {\n      // handle work in progress...\n    }\n  });\n  // handle error\n  worker.onError((id, error) => {\n    workerPool.releaseWorker(id);\n  });\n  // run worker\n  worker.run({ ... });\n}\n```\n\n## Contributing\n\nWe welcome contributions from the community. Before contributing, please read through the existing issues on this repository to prevent duplicate submissions. New feature requests and bug reports can be submitted as an issue. If you would like to contribute code, please open a pull request.\n\n## License\n\nThis project is licensed under the terms of the MIT license. For more information, refer to the [LICENSE](./LICENSE) file.\n\n","readmeFilename":"README.md"}