{"_id":"@dagster-io/dagster-pipes","name":"@dagster-io/dagster-pipes","dist-tags":{"latest":"0.1.0"},"versions":{"0.1.0":{"name":"@dagster-io/dagster-pipes","version":"0.1.0","main":"./dist/index.js","scripts":{"test":"jest","build":"tsc","lint":"eslint --fix","prettier-fix":"prettier \"./src/**/*.t{s,sx}\" -w"},"author":{"name":"dagster"},"license":"Apache-2.0","description":"A typescript implementation of the pipes protocol, for integration with https://github.com/dagster-io/dagster","devDependencies":{"@eslint/js":"^9.20.0","@types/jest":"^29.5.14","@types/node":"^22.13.1","eslint":"^9.20.1","jest":"^29.7.0","ts-jest":"^29.2.5","typescript":"^5.7.3","typescript-eslint":"^8.24.0","yargs":"^17.7.2"},"_id":"@dagster-io/dagster-pipes@0.1.0","gitHead":"167899a14f264c0c0bbed63e93e8ad0209c4dea6","types":"./dist/index.d.ts","_nodeVersion":"20.14.0","_npmVersion":"10.7.0","dist":{"integrity":"sha512-zIJ+AQIzAq/YSRjunHVF9Chom0q8kzs7KHkC41Yq4Lqg27fWTpfaqoXDMCQWkLqUYDIUPV4gfbftAS3w+WJK2Q==","shasum":"12689819c64029bfc91111ee651d3f877c66a346","tarball":"https://registry.npmjs.org/@dagster-io/dagster-pipes/-/dagster-pipes-0.1.0.tgz","fileCount":21,"unpackedSize":114906,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEUCIQCcICfIv7lICUPLd3cXUtkg0kznmGztVqZVc/bnD0AAiAIgYgH7eupv3Bgf3jOSezkdr4U92gMWkFUULInTVDQoEMc="}]},"_npmUser":{"name":"colton.padden","email":"colton.padden@fastmail.com"},"directories":{},"maintainers":[{"name":"dish_","email":"dish@dagsterlabs.com"},{"name":"salazarm93","email":"salazarm93@gmail.com"},{"name":"colton.padden","email":"colton.padden@fastmail.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/dagster-pipes_0.1.0_1740081301261_0.33341404800154484"},"_hasShrinkwrap":false}},"time":{"created":"2025-02-20T19:55:01.189Z","0.1.0":"2025-02-20T19:55:01.462Z","modified":"2025-02-20T19:55:01.829Z"},"maintainers":[{"name":"dish_","email":"dish@dagsterlabs.com"},{"name":"salazarm93","email":"salazarm93@gmail.com"},{"name":"colton.padden","email":"colton.padden@fastmail.com"}],"description":"A typescript implementation of the pipes protocol, for integration with https://github.com/dagster-io/dagster","author":{"name":"dagster"},"license":"Apache-2.0","readme":"# dagster-pipes-typescript\n\nA pipes implementation for the typescript programming language.\n\nAllows integration between any typescript process, and the dagster orchestrator.\n\n## Usage\n\n### Prerequisites\n\n- [install node and npm](https://nodejs.org/en/download)\n- install the typescript compiler (`npm install -g typescript`)\n\n### Installation\n\n`@dagster-io/dagster-pipes` is available as an npm package:\n\n```sh\nnpm install @dagster-io/dagster-pipes\n```\n\n### Example\n\n> In addition to the examples below - you can find an example dagster deployment with a typescript pipes asset in the `example-project/` directory.\n\nAfter installing the `@dagster-io/dagster-pipes` npm package, your typescript process can import it to integrate with dagster.\n\n```typescript\nimport * as dagster_pipes from '@dagster-io/dagster-pipes';\n\nusing context = dagster_pipes.openDagsterPipes()\n\ncontext.logger.info(\"this is a log message from the external process\")\ncontext.logger.warning(\"This is an example warning from the external process\")\n\ncontext.reportAssetMaterialization(\n    {\n        \"row_count\": 100\n    }\n)\ncontext.reportAssetCheck(\n    \"example_typescript_check\", true\n)\n```\n\nAnd, in the dagster orchestration code, add the following:\n\n```python\nimport subprocess\nfrom pathlib import Path\nimport dagster as dg\n\n@dg.asset(\n    check_specs=[\n        dg.AssetCheckSpec(\n            name=\"example_typescript_check\",\n            asset=\"example_typescript_asset\"\n        )\n    ]\n)\ndef example_typescript_asset(\n    context: dg.AssetExecutionContext, \n    pipes_subprocess_client: dg.PipesSubprocessClient\n) -> dg.MaterializeResult:\n    external_script_path = dg.file_relative_path(__file__, \"../external_typescript_code/\")\n    \n    subprocess.run([\"npm install\"],\n                    cwd=external_script_path).check_returncode()\n    subprocess.run([\"tsc\"],\n                   cwd=external_script_path).check_returncode()\n\n    return pipes_subprocess_client.run(\n        command=[\"node\", Path(external_script_path)],\n        context=context,\n    ).get_materialize_result()\n\n\ndefs = dg.Definitions(\n    assets=[example_typescript_asset],\n    resources={\n        \"pipes_subprocess_client\": dg.PipesSubprocessClient()\n    },\n)\n\n```\n\n## Development\n\n### Compiling the development version\n\n```\nnpm install && npm run build\n```\n\n### Testing\n\n```\nnpm test\nuv run --all-extras pytest\n```\n\n### Linting\n\n```\nnpm run lint\n```\n","readmeFilename":"README.md"}