{"_id":"@alleninstitute/shift","_rev":"2-2d3f4e1454e30d4b3fef11e7c56759c2","name":"@alleninstitute/shift","dist-tags":{"latest":"0.1.0"},"versions":{"0.1.0":{"name":"@alleninstitute/shift","version":"0.1.0","keywords":["data","transform","join","compose","etl","dataset","datasets","adapter","adapters"],"license":"AISL","_id":"@alleninstitute/shift@0.1.0","maintainers":[{"name":"lanesawyer","email":"npm@lanesawyer.dev"},{"name":"fcollman","email":"forrestc@alleninstitute.org"},{"name":"allen_skylermoosman","email":"skyler.moosman@alleninstitute.org"},{"name":"jarbuck","email":"joel.arbuckle@alleninstitute.org"}],"contributors":[{"name":"Joel Arbuckle","email":"joel.arbuckle@alleninstitute.org"},{"name":"Lane Sawyer","email":"lane.sawyer@alleninstitute.org"},{"name":"Skyler Moosman","email":"skyler.moosman@alleninstitute.org"},{"name":"Noah Shepard","email":"noah.shepard@alleninstitute.org"}],"dist":{"shasum":"7257a2ed7cbed20afc447be44a75c9c5d8eb26e9","tarball":"https://registry.npmjs.org/@alleninstitute/shift/-/shift-0.1.0.tgz","fileCount":5,"integrity":"sha512-WL7DvW3rhDQLNXrtG8lrsitzUzHMb/5zsdHYsO+ACy2r4brPV1+sTETjsPxod4uP3Nf5a8VhATqAbpxVBRkRBw==","signatures":[{"sig":"MEUCIQCbGvlSqO10+GK+j01UVGFpYIg3KqTLeACRP4gd090noAIgYXKhEjGb2yNEl2AD7jK+gX14Z1o4rwJmdG4HKduBtT0=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":71395},"main":"./dist/index.js","type":"module","types":"./dist/index.d.ts","volta":{"node":"24.14.0","pnpm":"10.33.0"},"source":"./src/index.ts","engines":{"node":">=22"},"exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js"}},"gitHead":"c9606373dc52c1922b5ae33800fed8a942b8ccb5","scripts":{"fmt":"oxfmt","lint":"oxlint src","test":"vitest run","build":"vite build","test:ci":"vitest run","changelog":"git-cliff -o CHANGELOG.md && oxfmt CHANGELOG.md","fmt:check":"oxfmt --check","typecheck":"tsc --noEmit","test:watch":"vitest --watch","build:watch":"vite build --watch","test:coverage":"vitest run --coverage"},"_npmUser":{"name":"jarbuck","email":"joel.arbuckle@alleninstitute.org"},"_npmVersion":"11.9.0","description":"A library for defining, transforming, combining, and loading datasets from anywhere","directories":{},"_nodeVersion":"24.14.0","dependencies":{"neverthrow":"8.2.0"},"publishConfig":{"registry":"https://registry.npmjs.org"},"_hasShrinkwrap":false,"packageManager":"pnpm@10.33.0","devDependencies":{"vite":"8.0.10","oxfmt":"0.46.0","oxlint":"1.63.0","vitest":"4.1.5","git-cliff":"2.12.0","typescript":"6.0.3","@types/node":"24.12.2","vite-plugin-dts":"4.5.4"},"peerDependencies":{"zod":"^4.0.0"},"_npmOperationalInternal":{"tmp":"tmp/shift_0.1.0_1778705029781_0.7034294982325646","host":"s3://npm-registry-packages-npm-production"}}},"time":{"created":"2026-05-13T20:43:49.682Z","modified":"2026-07-08T17:35:43.065Z","0.1.0":"2026-05-13T20:43:49.934Z"},"license":"AISL","keywords":["data","transform","join","compose","etl","dataset","datasets","adapter","adapters"],"description":"A library for defining, transforming, combining, and loading datasets from anywhere","contributors":[{"name":"Joel Arbuckle","email":"joel.arbuckle@alleninstitute.org"},{"name":"Lane Sawyer","email":"lane.sawyer@alleninstitute.org"},{"name":"Skyler Moosman","email":"skyler.moosman@alleninstitute.org"},{"name":"Noah Shepard","email":"noah.shepard@alleninstitute.org"}],"maintainers":[{"email":"npm@lanesawyer.dev","name":"lanesawyer"},{"email":"forrestc@alleninstitute.org","name":"fcollman"},{"email":"skyler.moosman@alleninstitute.org","name":"allen_skylermoosman"},{"email":"joel.arbuckle@alleninstitute.org","name":"jarbuck"},{"email":"nshepar@gmail.com","name":"nope_froyo"}],"readme":"# Allen Institute / shift\r\n\r\n`@alleninstitute/shift` is a TypeScript library for defining, transforming, combining, and loading datasets from anywhere. It provides a composable, adapter-driven model for describing data pipelines in terms of typed datasets, then executing those pipelines in a consistent, error-safe way.\r\n\r\n## Level Of Support\r\n\r\n**No Support Guaranteed:** While we welcome feedback and questions, the `shift` library is currently provided as-is with no guarantee of direct support, updates, or bug fixes.\r\n\r\n## Core Concepts\r\n\r\n### Datasets\r\n\r\nA **Dataset** is a typed description of a data shape, independent of where or how that data is fetched. There are three kinds:\r\n\r\n| Type                 | Description                                                                                                                       |\r\n| -------------------- | --------------------------------------------------------------------------------------------------------------------------------- |\r\n| `SourceDataset`      | A dataset backed by an adapter (e.g. a GraphQL endpoint, REST API, or database).                                                  |\r\n| `TransformedDataset` | Wraps another dataset and maps its output shape into a new one.                                                                   |\r\n| `ComposedDataset`    | Joins two datasets — loads the left side first, then uses those results to drive the query on the right side, and merges the two. |\r\n\r\n### Adapters\r\n\r\nA **DatasetAdapter** is an interface that adapters implement to connect datasets to real data sources. An adapter is responsible for two things:\r\n\r\n1. Providing a way to generate a `SourceDataset`. This is not an explicit requirement of the interface itself, as this process can differ widely from case to case, but `SourceDataset`s can only exist if adapters provide a way to produce them. For example, this could take the form of a function that takes a query-like description of data to retrieve, which is then stored inside a `SourceDataset`-implementing object that can be loaded later.\r\n2. `loadDataset(dataset, params)` — Executes the query for a given source dataset with the provided `LoadParameters`, returning an `AsyncLoadResult<Loadset<Def>>`. (`Def` being the type defined for the input dataset's data shape.)\r\n3. `isEmpty(loadset)` — Determines whether a loaded result set is empty.\r\n\r\nAdapters are kept separate from the shift library itself and live in their own packages. Upcoming first-party adapter packages may include `@alleninstitute/shift-graphql` and `@alleninstitute/shift-rest`.\r\n\r\n### Lifecycle Functions\r\n\r\nThe library exposes three primary lifecycle functions for building and executing dataset pipelines:\r\n\r\n- **`transform(dataset, fn, isEmpty?)`** — Wraps a dataset with a mapping function. The function receives the loaded output of the base dataset and returns a new shaped object. An optional `isEmpty` callback can override the default empty-check for the resulting loadset.\r\n- **`compose(left, right, preparer, composer, isEmpty?)`** — Composes two datasets. The `preparer` function receives the caller's `LoadParameters` and returns left/right load configurations. After loading `left`, the right-side config can derive its parameters from the left result. The `composer` function then merges both loadsets into the final output shape.\r\n- **`load(dataset, params?)`** — Executes the full dataset tree, recursively loading source datasets, applying transforms, and resolving compositions. Accepts optional `LoadParameters` (sort, filter, pagination, etc.). Returns an `AsyncLoadResult<Loadset<Def>>`.\r\n\r\nImportantly, `transform()` and `compose()` do not _execute_ a load; instead, they are the two primary _definition_ functions, along with whatever definition functionality provided by the adapter(s) being used. Their job is to define the expected behavior and output of the post-processing of any data loaded from the `SourceDataset`, as part of the `load()` call.\r\n\r\n### Error Handling\r\n\r\nShift internally uses [neverthrow](https://github.com/supermacro/neverthrow) for explicit, type-safe error handling. Loading a dataset returns a `ResultAsync`, empowering individual adapters to use field-level error propagation if desired. If Shift's built-in `validation` mechanisms are used for validating received data, these will automatically wrap all fields in `Result`, with errors propagated through the pipeline as `DataProcessingError` values rather than thrown exceptions. `Result` and `ResultAsync` from `neverthrow` are wrapped inside `LoadResult` and `AsyncLoadResult`, which automatically associate the `DataProcessingError` as the error type of the result.\r\n\r\n`DataProcessingError` carries three fields:\r\n\r\n- `internalMessage` — A developer-facing description of what went wrong.\r\n- `userMessage` _(optional)_ — A message safe to show in a UI.\r\n- `details` _(optional)_ — Arbitrary context for debugging.\r\n\r\n#### Result Helpers\r\n\r\n- `pass(value)` / `passAsync(value)` — Wraps a value in an `Ok` `LoadResult`/`LoadResultAsync`.\r\n- `fail(error)` / `failAsync(error)` — Wraps an error in an `Err` `LoadResult`/`LoadResultAsync`.\r\n- `asFailure(error)` — Converts an unknown thrown value into a `DataProcessingError`.\r\n- `coalesce(result, fallback)` — Returns the existing result, or an `Ok` wrapping `fallback` if the result is nullish.\r\n\r\n#### Deep Result Utilities\r\n\r\nWhen working with nested `Result` structures (common after granular validation), these helpers recursively traverse and unwrap values:\r\n\r\n- `deepUnwrap(result)` — Recursively unwraps nested Results. Returns the first `Err` encountered (fail-fast).\r\n- `deepUnwrapSoft(result)` — Like `deepUnwrap`, but replaces nested `Err` values with `undefined` instead of short-circuiting.\r\n- `deepReduce(result, initialValue, reducer)` — Traverses a Result value tree depth-first, calling a reducer at each node. Fail-fast on `Err`.\r\n- `deepReduceSoft(result, initialValue, reducer)` — Like `deepReduce`, but silently skips `Err` branches instead of short-circuiting.\r\n\r\n### Load Parameters\r\n\r\n`LoadParameters` is the adapter-agnostic interface for holding any parameters required for querying data. Each adapter is expected to internally convert these fields into its own format (e.g. GraphQL query variables) before performing the query.\r\n\r\n```typescript\r\ntype LoadParameters<SortInput, FilterInput, OtherVars> = {\r\n    // Array of typed sort directives, based on an adapter-specific sort shape\r\n    sort?: TypedSortState<SortInput>;\r\n\r\n    // Adapter-specific filter shape\r\n    filter?: FilterInput;\r\n\r\n    // Filters that take precedence over any other filters specified, thus \"scoping\" the query\r\n    scopeFilter?: FilterInput;\r\n\r\n    // Cursor-based, offset-based, or none\r\n    pagination?: Pagination;\r\n\r\n    // Additional adapter-specific variables\r\n    vars?: OtherVars;\r\n};\r\n```\r\n\r\n### Pagination\r\n\r\nThree pagination strategies are supported:\r\n\r\n| Class              | Style                               | Fields                            |\r\n| ------------------ | ----------------------------------- | --------------------------------- |\r\n| `CursorPagination` | Cursor-based (e.g. GraphQL Relay)   | `direction`, `cursor`, `pageSize` |\r\n| `OffsetPagination` | Offset/limit                        | `offset`, `pageSize`              |\r\n| `NoPagination`     | Single page/no specified pagination | _(none)_                          |\r\n\r\nWhen composing datasets, the right-side dataset supports automatic multi-page loading (up to 20 pages). Pagination can be customized for the right-side dataset loads by passing a `Pagination` object in the `preparer`'s `right.pagination` config variable. A custom `pageReducer` callback can also be supplied to control how pages are merged.\r\n\r\n### Loadsets\r\n\r\nA **Loadset** is the output of a loaded dataset — the data plus metadata about the response:\r\n\r\n```typescript\r\ntype Loadset<Def> = {\r\n    data: Def;\r\n    metadata: LoadsetMetadata;\r\n};\r\n\r\ntype LoadsetMetadata = {\r\n    totalCount: number;\r\n    pageInfo?: {\r\n        hasNextPage: boolean;\r\n        hasPreviousPage: boolean;\r\n        startCursor: string | null;\r\n        endCursor: string | null;\r\n        pageStartOffset: number | undefined;\r\n    };\r\n};\r\n```\r\n\r\n#### Utility functions\r\n\r\n- `emptyLoadset()` — Creates a `Loadset` with empty data and zeroed metadata.\r\n- `emptyLoadsetMetadata()` — Creates default empty `LoadsetMetadata`.\r\n- `isLoadsetEmpty(dataset, loadset)` — Checks whether a loadset is empty, using the dataset's `isEmpty` callback if defined, otherwise falling back to `metadata.totalCount === 0`.\r\n\r\n### Validation\r\n\r\nThe `validation` module provides utilities for performing **granular, per-field Zod validation** of raw API responses. Rather than failing an entire response when one field is invalid, shift can represent each field's parse result individually as a `LoadResult`, allowing callers to handle partial data gracefully.\r\n\r\n#### Workflow\r\n\r\n1. **Build a validation tree** from a Zod schema:\r\n\r\n    ```typescript\r\n    const tree = buildValidationTree(myZodSchema);\r\n    ```\r\n\r\n    This recursively processes the schema into a `ZodValidationTree` of scalar, object, and array nodes, preserving optional/nullable status at each level.\r\n\r\n2. **Parse raw data** against the tree:\r\n    - `parseObject(val, tree)` — Validates each field of an object individually, returning a record of per-field `LoadResult` values. This is typically what would be called on the full tree of data, unless the data is coming back as an array.\r\n    - `parseArray(val, tree)` — Validates an array, returning a `LoadResult<Array>`.\r\n    - `parseScalarWithSchema(val, schema)` — Validates a single scalar value.\r\n\r\n#### Advanced features\r\n\r\n- **Fragment alternatives** — Supports type-conditional inline fragments, such as those in GraphQL (`... on TypeName`). The `__typename` field in incoming data selects the correct validation branch.\r\n- **Loose mode** — When `loose: true`, unexpected keys in the input are passed through un-validated rather than discarded.\r\n\r\n### Utilities\r\n\r\n#### JSON Parsing\r\n\r\n- `safeParseJSON(val)` — Non-throwing `JSON.parse` that returns a `LoadResult<unknown>`.\r\n- `asyncSafeParseJSON(val)` — Async variant returning an `AsyncLoadResult<unknown>`.\r\n\r\n#### Type Guards\r\n\r\n- `isNullish(val)` / `isNotFound(val)` — Checks for `null` or `undefined`.\r\n- `isObject(val)` — Checks for non-nullish objects.\r\n- `isRecord(val)` — Checks for plain key-value records.\r\n- `isStringKeyedRecord(val)` — Checks for records with only string keys.\r\n- `isPromise(val)` — Checks for promise-like objects.\r\n\r\n---\r\n\r\n## Getting Started\r\n\r\n### Installation\r\n\r\n```bash\r\npnpm add @alleninstitute/shift\r\n```\r\n\r\n### Basic Usage\r\n\r\n**Note:** In this example, the data returned from the `usersDataset` has been simplified by removing the `LoadResult` layer from the picture (this can be done at the will of the Adapter by not wrapping the output `Def` in `DefinitionResults` when generating a Source Dataset). See examples below to see how `LoadResult` would be interacted with in a typical scenario.\r\n\r\n```typescript\r\nimport { transform, load } from '@alleninstitute/shift';\r\n\r\n// Assume `usersDataset` is a SourceDataset obtained from an adapter package\r\n// (e.g. a GraphQL adapter that creates typed datasets from document nodes)\r\n\r\nconst activeNamesDataset = transform(usersDataset, (data) => ({\r\n    names: data.users.map((u) => u.name),\r\n}));\r\n\r\nconst result = await load(activeNamesDataset, { filter: { active: true } });\r\n\r\nresult.match(\r\n    (loadset) => console.log(loadset.data.names),\r\n    (err) => console.error(err.userMessage ?? err.internalMessage)\r\n);\r\n```\r\n\r\n### Composing Datasets\r\n\r\n**Note:** In this example, the data returned from the `usersDataset` and `postsDataset` has been simplified by removing the `LoadResult` layer from the picture (this can be done at the will of the Adapter by not wrapping the output `Def` in `DefinitionResults` when generating a Source Dataset). See examples below to see how `LoadResult` would be interacted with in a typical scenario.\r\n\r\n```typescript\r\nimport { compose, load } from '@alleninstitute/shift';\r\n\r\n// Compose: load users first, then fetch their posts and attach them inline\r\nconst usersWithPostsDataset = compose(\r\n    usersDataset,\r\n    postsDataset,\r\n    // preparer: splits incoming params into left/right load configs\r\n    (params) => ({\r\n        left: { params },\r\n        right: {\r\n            getParams: (usersLoadset) => ({\r\n                filter: { authorId: { in: usersLoadset.data.users.map((u) => u.id) } },\r\n            }),\r\n        },\r\n    }),\r\n    // composer: attach each user's posts directly onto the output user object\r\n    (usersLoadset, postsLoadset) => ({\r\n        users: usersLoadset.data.users.map((user) => ({\r\n            ...user,\r\n            posts: postsLoadset.data.items.filter((p) => p.authorId === user.id),\r\n        })),\r\n    })\r\n);\r\n\r\nconst result = await load(usersWithPostsDataset, { filter: { active: true } });\r\n```\r\n\r\n### Chaining Transforms\r\n\r\n`transform` returns a `Dataset`, so transforms are freely chainable. Each step only sees the output shape of the previous one.\r\n\r\n```typescript\r\nimport { transform, load } from '@alleninstitute/shift';\r\n\r\n// First transform: strip fields not needed downstream\r\nconst projectSummariesDataset = transform(projectsDataset, (data) => ({\r\n    summaries: data.projects.map((projects) => projects.map((p) => ({ id: p.id, name: p.name, status: p.status }))),\r\n}));\r\n\r\n// Second transform: partition by status\r\nconst partitionedProjectsDataset = transform(projectSummariesDataset, (data) => ({\r\n    active: data.summaries.map((summaries) => summaries.filter((p) => p.status.isOk() && p.status.value === 'active')),\r\n    archived: data.summaries.map((summaries) =>\r\n        summaries.filter((p) => p.status.isOk() && p.status.value === 'archived')\r\n    ),\r\n}));\r\n\r\nconst result = await load(partitionedProjectsDataset, { filter: { teamId: 'team-42' } });\r\nresult.match(\r\n    (loadset) => console.log(loadset.data.active.length, 'active projects'),\r\n    (err) => console.error(err.internalMessage)\r\n);\r\n```\r\n\r\n### Composing and Then Transforming\r\n\r\nA `ComposedDataset` is itself a `Dataset`, so it can be wrapped in a `transform` to reshape the merged output.\r\n\r\n```typescript\r\nimport { compose, transform, load } from '@alleninstitute/shift';\r\n\r\n// Compose: load teams, then fetch each team's members and attach them inline\r\nconst teamsWithMembersDataset = compose(\r\n    teamsDataset,\r\n    membersDataset,\r\n    (params) => ({\r\n        left: { params },\r\n        right: {\r\n            getParams: (teamsLoadset) => ({\r\n                filter: {\r\n                    teamId: {\r\n                        in: teamsLoadset.data.teams\r\n                            .unwrapOr([])\r\n                            .map((t) => t.id.unwrapOr(null))\r\n                            .filter(Boolean),\r\n                    },\r\n                },\r\n            }),\r\n        },\r\n    }),\r\n    (teamsLoadset, membersLoadset) => {\r\n        const members = membersLoadset.data.members.unwrapOr([]);\r\n        return {\r\n            teams: teamsLoadset.data.teams.map((teams) =>\r\n                teams.map((team) => ({\r\n                    ...team,\r\n                    members: membersLoadset.data.members.filter(\r\n                        (m) => team.id.isOk() && m.teamId.isOk() && m.teamId.value === team.id.value\r\n                    ),\r\n                }))\r\n            ),\r\n        };\r\n    }\r\n);\r\n\r\n// Transform the composed result to produce a flat leaderboard sorted by member count\r\nconst leaderboardDataset = transform(teamsWithMembersDataset, (data) => ({\r\n    leaderboard: data.teams.map((teams) =>\r\n        teams\r\n            .sort((a, b) => b.members.unwrapOr([]).length - a.members.unwrapOr([]).length)\r\n            .map((team, index) => ({\r\n                rank: index + 1,\r\n                teamName: team.name,\r\n                memberCount: team.members.map((mems) => mems.length),\r\n            }))\r\n    ),\r\n}));\r\n\r\nconst result = await load(leaderboardDataset, { filter: { active: true } });\r\n```\r\n\r\n### Paginating the Right Side of a Compose\r\n\r\nWhen the right-side dataset results spans multiple pages, the loader automatically paginates to include all available results, up to a maximum of 20 pages loaded. By default, pages are merged by concatenating arrays at matching keys. A custom `pageReducer` can be supplied for full control over how pages are accumulated, and a custom `Pagination` object can be used to specify other settings, such as page size. (Note: currently, `'after'` is the only direction value ever used by the right-side pagination logic; specifying `'before'` will have no effect.)\r\n\r\n```typescript\r\nimport { compose, load, CursorPagination } from '@alleninstitute/shift';\r\n\r\nconst specimensWithAllImagesDataset = compose(\r\n    specimensDataset,\r\n    imagesDataset,\r\n    (params) => ({\r\n        left: { params },\r\n        right: {\r\n            getParams: (specimensLoadset) => ({\r\n                filter: {\r\n                    // Fetch all images for all specimens in the left result in one query\r\n                    specimenId: {\r\n                        in: specimensLoadset.data.specimens.map((specimens) =>\r\n                            specimens.map((s) => s.id.unwrapOr(null)).filter(Boolean)\r\n                        ),\r\n                    },\r\n                },\r\n                // Request pages of size 50 — the loader will keep fetching until hasNextPage\r\n                // is false or 20 pages are fetched\r\n                pagination: new CursorPagination(undefined, undefined, 50),\r\n            }),\r\n            // Custom reducer: concatenate image arrays and carry forward the latest metadata\r\n            pageReducer: (accumulated, page) => {\r\n                if (page.data.images.isErr()) {\r\n                    return accumulated;\r\n                }\r\n                return {\r\n                    data: { images: [...accumulated.data.images, ...page.data.images.value] },\r\n                    metadata: {\r\n                        totalCount: accumulated.metadata.totalCount + page.metadata.totalCount,\r\n                        pageInfo: page.metadata.pageInfo,\r\n                    },\r\n                };\r\n            },\r\n        },\r\n    }),\r\n    (specimensLoadset, imagesLoadset) => ({\r\n        // Attach images to their matching specimen by specimenId\r\n        specimens: specimensLoadset.data.specimens.map((specimen) => ({\r\n            ...specimen,\r\n            images: imagesLoadset.data.images.filter(\r\n                (img) => specimen.id.isOk() && img.specimenId.isOk() && image.specimenId.value === specimen.id.value\r\n            ),\r\n        })),\r\n    })\r\n);\r\n```\r\n\r\n### Conditionally Skipping the Right Side\r\n\r\nIf `getParams` returns `null`, the right load is skipped entirely and the composer receives an empty loadset for the right side. This is useful when the right query only makes sense if the left result contains data.\r\n\r\n```typescript\r\nimport { compose, load } from '@alleninstitute/shift';\r\n\r\nconst specimenWithAnnotationsDataset = compose(\r\n    specimensDataset,\r\n    annotationsDataset,\r\n    (params) => ({\r\n        left: { params },\r\n        right: {\r\n            getParams: (specimensLoadset) => {\r\n                const specimens = specimensLoadset.data.specimens.unwrapOr([]);\r\n                // No specimen found — skip fetching annotations entirely\r\n                if (specimens.length === 0) {\r\n                    return null;\r\n                }\r\n                return { filter: { specimenId: { in: specimens.map(\r\n                    (s) => s.id.unwrapOr(null)).filter(Boolean)\r\n                }}};\r\n            },\r\n        },\r\n    }),\r\n    (specimensLoadset, annotationsLoadset) => {\r\n\r\n        return {\r\n            specimens: specimensLoadset.data.specimens.map(specimens => specimens.map(s => ({\r\n                ...s,\r\n                annotations: (annotationsLoadset?.data.annotations.unwrapOr([]) ?? []).filter(((result) => result.map())\r\n            }))) {\r\n                ...specimenLoadset.data.specimen,\r\n                annotations: annotationsLoadset.data.annotations ?? [],\r\n            }\r\n        }\r\n    }\r\n);\r\n\r\nconst result = await load(specimenWithAnnotationsDataset, { vars: { id: 'spec-001' } });\r\n```\r\n\r\n---\r\n\r\n## Development\r\n\r\n### Prerequisites\r\n\r\n- [Node.js](https://nodejs.org/) 24.x (managed via [Volta](https://volta.sh/))\r\n- [pnpm](https://pnpm.io/) 10.x (managed via [Volta](https://docs.volta.sh/advanced/pnpm))\r\n\r\n### Setup\r\n\r\n```bash\r\npnpm install\r\n```\r\n\r\n### Scripts\r\n\r\n| Command              | Description                              |\r\n| -------------------- | ---------------------------------------- |\r\n| `pnpm lint`          | Run linter (OXLint)                      |\r\n| `pnpm build`         | Build the library to `dist/`             |\r\n| `pnpm build:watch`   | Build in watch mode                      |\r\n| `pnpm test`          | Run tests                                |\r\n| `pnpm test:watch`    | Run tests in watch mode                  |\r\n| `pnpm test:ci`       | Run tests once (CI mode)                 |\r\n| `pnpm test:coverage` | Run tests with coverage report           |\r\n| `pnpm typecheck`     | Type-check without emitting output       |\r\n| `pnpm fmt`           | Format all files with OxFmt              |\r\n| `pnpm fmt:check`     | Check formatting without writing         |\r\n| `pnpm changelog`     | Generate `CHANGELOG.md` from git history |\r\n\r\n### Project Structure\r\n\r\n```\r\nsrc/\r\n  lib/\r\n    datasets/\r\n      input/        # transform() and compose() — dataset construction\r\n      loading/      # load() — recursive dataset execution, source delegation\r\n      output/       # Loadset types, deep Result utilities (deepUnwrap, etc.)\r\n    utils/\r\n      parsing/      # Safe JSON parsing utilities\r\n      typing/       # Type guards and type-level utilities\r\n    validation/     # Granular per-field Zod validation (buildValidationTree, parseObject, etc.)\r\n  presets/          # Planned first-party adapter presets (e.g. GraphQL)\r\n```\r\n","readmeFilename":"README.md"}