{"_id":"@altiplano/taskserver","name":"@altiplano/taskserver","dist-tags":{"latest":"0.0.1"},"versions":{"0.0.1":{"name":"@altiplano/taskserver","version":"0.0.1","description":"Altiplano tasks server","license":"MIT","repository":{"type":"git","url":"https://github.com/synw/altiplano.git"},"keywords":["llm","ai"],"scripts":{"build":"tsc -p .","start":"run-s build servejs","serve":"tsc -w & nodemon -q -w src dist/bin/runserver.js","runserver":"node dist/bin/runserver.js"},"dependencies":{"@altiplano/usellama":"^0.0.3","@altiplano/inferserver":"^0.0.6","@koa/cors":"^4.0.0","koa":"^2.14.1","koa-body":"^6.0.1","koa-router":"^12.0.0","koa-static":"^5.0.0","llama-node":"0.1.6","ws":"^8.13.0","yaml":"^2.3.1"},"devDependencies":{"@altiplano/types":"^0.0.1","@types/koa":"^2.13.6","@types/koa-router":"^7.4.4","@types/koa-static":"^4.0.2","@types/node":"^18.15.11","autoprefixer":"^10.4.4","nodemon":"^2.0.22","npm-run-all":"^4.1.5","path":"^0.12.7","ts-node":"^10.9.1","ts-node-esm":"^0.0.6","tslib":"^2.5.0","typescript":"^4.9.5"},"main":"dist/index.js","type":"module","bin":{"taskserver":"dist/bin/runserver.js"},"preferGlobal":true,"publishConfig":{"access":"public","registry":"https://registry.npmjs.org/"},"licenseText":"MIT License\n\nCopyright (c) 2022 synw\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n","_id":"@altiplano/taskserver@0.0.1","dist":{"shasum":"a6154eac0a0906bc70cc13bc717ef22b7bc858b4","integrity":"sha512-0uP2g9GpFrOiH62pWQZdAI7fTXf5MwLn3rbs2RdfY0HYutCLi97E32s2/vEeSz2fDytzOnbvdP49Ymgmz73m5Q==","tarball":"https://registry.npmjs.org/@altiplano/taskserver/-/taskserver-0.0.1.tgz","fileCount":33,"unpackedSize":26147,"signatures":[{"keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA","sig":"MEYCIQCnncYe5XqpHhrBgTZ082/K8c13VtOt0GOCgZ+JlnrmvQIhALmoj6h0j3exndX7KhiGRWySY5FSIrCHE4Ho2jyyOjDT"}]},"_npmUser":{"name":"synw","email":"synwx@pm.me"},"directories":{},"maintainers":[{"name":"synw","email":"synwx@pm.me"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages","tmp":"tmp/taskserver_0.0.1_1687538593980_0.6342819569499891"},"_hasShrinkwrap":false}},"time":{"created":"2023-06-23T16:43:13.867Z","0.0.1":"2023-06-23T16:43:14.124Z","modified":"2023-06-23T16:43:14.589Z"},"maintainers":[{"name":"synw","email":"synwx@pm.me"}],"description":"Altiplano tasks server","keywords":["llm","ai"],"repository":{"type":"git","url":"https://github.com/synw/altiplano.git"},"license":"MIT","readme":"# Altiplano tasks server\n\nA language model tasks api server. Serve tasks from simple yaml files\n\n## Install:\n\n```bash\nyarn global add @altiplano/taskserver\n# or \nnpm install -g @altiplano/taskserver\n```\n\n## Usage\n\n### Create a task\n\n```bash\nmkdir tasks\ncd tasks\ntouch mytask.yaml\n```\n\nThe task content:\n\n```yaml\nname: fix_json\nmodel: open-llama-7B-open-instruct.ggmlv3.q5_1\nmodelConf:\n  - nCtx: 512\ninferParams:\n  - topK: 40,\n  - topP: 0.1,\n  - temp: 0,\n  - repeatPenalty: 1\ntemplate: |-\n  ### Instruction: Fix this invalid json:\n\n  {prompt}\n  ### Response: (Answer only with json)\n```\n\nA task is a model with it's config, some inference parameters and a template. Doc:\n[models conf params](https://github.com/synw/altiplano/tree/main/packages/usellama#loading-a-model), \n[inference params](https://github.com/synw/altiplano/tree/main/packages/usellama#inference-parameters)\n\n### Run the tasks server\n\n### Command\n\nUse the following command:\n\n```bash\ntaskserver path/to/models/dir path/to/tasks/folder model_to_run-optional\n```\n\nExample:\n\n```bash\ntaskserver /an/absolute/path/to/models/dir task open-llama-7B-open-instruct.ggmlv3.q5_1.bin\n```\n\nThis will load the model at startup if a model name is provided\n\n### As library\n\nRun your own server using the library:\n\n```ts\n#!/usr/bin/env node\n\nimport { argv, exit } from \"process\";\nimport { useTaskServer } from \"../dist/server.js\";\n\n\nfunction _runserver(modelsDirPath, tasksDir, loadModel) {\n  const { app } = useTaskServer(\n    tasksDir,\n    [],\n    {\n      modelsDirPath: modelsDirPath,\n      loadModel: loadModel,\n    },\n    true,\n  );\n  app.listen(5143, () => {\n    console.log(\"Tasks server running on port 5143\");\n  });\n}\n```\n\nParameters of `useTaskServer`:\n\n- `tasksDir`: *string* **required**: the directory where the tasks are\n- `routes`: *Array<(r: Router) => void>*: optional extra routes. Check the [inferserver doc](https://github.com/synw/altiplano/tree/main/packages/inferserver#router-options)\n- `params`: *InferServerParams*: the server params: one is required: `modelsDirPath`, the path to the models directory\n- `verbose`: *boolean*: verbosity (default *false*)\n\n### Execute a task\n\nHit the `/task/execute` endpoint with a prompt and a task name:\n\n```bash\ncurl -X POST -H \"Content-Type: application/json\" -d '{\"prompt\": \"{\\\"a\\\":1,}\", \\\n   \"task\": \"code/json/fix\"}' http://localhost:5143/task/execute\n```\n\n","readmeFilename":"README.md"}