{"_id":"@wgpu-fusion/core","_rev":"2-11607bd3e651b722e396f98c46ea1b77","name":"@wgpu-fusion/core","dist-tags":{"latest":"0.1.0"},"versions":{"0.1.0":{"name":"@wgpu-fusion/core","version":"0.1.0","keywords":["webgpu","transformer","inference","kernel-fusion","llm","browser-ai","compute-shader","wgsl","gpu"],"author":{"name":"Ahmet Baris Gunaydin","email":"abgunaydin94@gmail.com"},"license":"MIT","_id":"@wgpu-fusion/core@0.1.0","maintainers":[{"name":"abgunaydin","email":"abgunaydin94@gmail.com"}],"homepage":"https://github.com/abgnydn/webgpu-fusion-sdk#readme","bugs":{"url":"https://github.com/abgnydn/webgpu-fusion-sdk/issues"},"dist":{"shasum":"5ce41f1160f00f7a5e4f7faf36029bfbd4e5eb34","tarball":"https://registry.npmjs.org/@wgpu-fusion/core/-/core-0.1.0.tgz","fileCount":50,"integrity":"sha512-YVVSXdjpJY+ybEJIagClfz/Vxe1MIrLgFjj07NluHpwG7vy/KoJYX1cl/7c9nbr9LK9wt+NiiQ53Iri0iALsYg==","signatures":[{"sig":"MEYCIQDRz6EhnEFC4C8WYQGI4Z94uQSNWxwMdUxKp7HOJYlctwIhAK0E7Hqj9w/9r7jKTuZsXxkfLMrW477L7e2flJEfa0sC","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":228280},"main":"dist/index.js","type":"module","types":"dist/index.d.ts","exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js"}},"gitHead":"ecd8631677bdf21e06e56a2bea1e50f74e83ca25","scripts":{"dev":"tsc --watch","test":"vitest run","build":"tsc","test:watch":"vitest","prepublishOnly":"npm run test && npm run build"},"_npmUser":{"name":"abgunaydin","email":"abgunaydin94@gmail.com"},"repository":{"url":"git+https://github.com/abgnydn/webgpu-fusion-sdk.git","type":"git"},"_npmVersion":"11.7.0","description":"Fused WebGPU compute kernels for transformer inference — 4,081x avg on Apple Silicon, 826x on phones (487 devices tested)","directories":{},"_nodeVersion":"24.5.0","_hasShrinkwrap":false,"devDependencies":{"vitest":"^4.1.2","typescript":"^5.7.0","@webgpu/types":"^0.1.52"},"_npmOperationalInternal":{"tmp":"tmp/core_0.1.0_1775494816294_0.6463897329642865","host":"s3://npm-registry-packages-npm-production"},"deprecated":"Renamed to @webgpu-fusion/core. Install: npm install @webgpu-fusion/core"}},"time":{"created":"2026-04-06T17:00:16.188Z","modified":"2026-05-04T13:21:39.786Z","0.1.0":"2026-04-06T17:00:16.459Z"},"bugs":{"url":"https://github.com/abgnydn/webgpu-fusion-sdk/issues"},"author":{"name":"Ahmet Baris Gunaydin","email":"abgunaydin94@gmail.com"},"license":"MIT","homepage":"https://github.com/abgnydn/webgpu-fusion-sdk#readme","keywords":["webgpu","transformer","inference","kernel-fusion","llm","browser-ai","compute-shader","wgsl","gpu"],"repository":{"url":"git+https://github.com/abgnydn/webgpu-fusion-sdk.git","type":"git"},"description":"Fused WebGPU compute kernels for transformer inference — 4,081x avg on Apple Silicon, 826x on phones (487 devices tested)","maintainers":[{"name":"abgunaydin","email":"abgunaydin94@gmail.com"}],"readme":"# @webgpu-fusion/core\n\nFused WebGPU compute kernels for transformer inference. **4,081x average** on Apple Silicon, **826x on phones**. 487 real-world devices tested.\n\nOne import. One dispatch. All tokens, all layers, all operations fused into a single GPU kernel.\n\n## Install\n\n```bash\nnpm install @webgpu-fusion/core\n```\n\n## Quick Start\n\n```js\nimport { FusedTransformer } from '@webgpu-fusion/core'\n\n// Create a fused transformer (parallel mode, 64 GPU threads)\nconst model = await FusedTransformer.create({\n  dModel: 128,\n  nHeads: 2,\n  nLayers: 4,\n  maxSeqLen: 64,\n})\n\n// Benchmark with random weights\nconst stats = await model.benchmark({ runs: 10 })\nconsole.log(`${stats.tok_per_sec.toFixed(0)} tok/s | ${stats.mean_ms.toFixed(1)} ms | 1 dispatch`)\n\n// Or load real weights and run inference\nconst weights = new Float32Array(/* your model weights */)\nmodel.loadWeightsWithDefaults(weights)\n\nconst embeddings = new Float32Array(64 * 128) // [seqLen * dModel]\nconst result = await model.forward(embeddings)\nconsole.log(result.output) // Float32Array of hidden states\nconsole.log(`${result.tok_per_sec.toFixed(0)} tok/s`)\n\n// Clean up\nmodel.destroy()\n```\n\n## Why This Is Fast\n\nCurrent browser inference engines dispatch **separate GPU kernels** for each operation:\n\n```\nToken 1: dispatch LN → dispatch Attn → dispatch LN → dispatch FFN  (4 round-trips)\nToken 2: dispatch LN → dispatch Attn → dispatch LN → dispatch FFN  (4 round-trips)\n... × 64 tokens × 4 layers = 1,024 GPU round-trips\n```\n\nThis library fuses everything into **1 dispatch**:\n\n```\nSingle dispatch → all tokens × all layers × all ops in one kernel\n```\n\nThe parallel variant uses 64 GPU threads with shared memory to also parallelize the matrix multiplications within the fused kernel.\n\n## Benchmark Results\n\n### Paper (Apple M2 Pro)\n\n| Config | Unfused | Parallel Fused | Speedup | vs PyTorch MPS |\n|--------|---------|---------------|---------|---------------|\n| D=32, L=1 | 265 ms | **4.0 ms** | **66x** | **161x** |\n| D=64, L=4 | 3,841 ms | **25.4 ms** | **151x** | **42x** |\n| D=128, L=4 | 14,568 ms | **59.9 ms** | **243x** | **18x** |\n| D=256, L=1 | 14,246 ms | **31.1 ms** | **458x** | **44x** |\n\n### Real-World (487 devices, 8 GPU vendors)\n\n| GPU Vendor | Avg Speedup | Peak Speedup |\n|---|---|---|\n| **Apple Silicon** (M1/M2/M3) | **4,081x** | 79,021x |\n| **Qualcomm Adreno** (Android phones) | **826x** | 13,541x |\n| **NVIDIA** (Blackwell, Lovelace, Ampere) | **70x** | 159x |\n| **ARM Mali** | **55x** | — |\n\nMobile: 15,000 tok/s avg, 213,000 tok/s peak. Higher speedups on mobile reflect worse dispatch overhead — kernel fusion benefits them most.\n\nRun it on your device: [gpubench.dev/transformer](https://gpubench.dev/transformer)\n\n## API\n\n### `FusedTransformer.create(options)`\n\n| Option | Type | Default | Description |\n|--------|------|---------|-------------|\n| `dModel` | number | required | Hidden dimension |\n| `nHeads` | number | required | Attention heads |\n| `nLayers` | number | required | Transformer layers |\n| `ffnMultiplier` | number | 4 | FFN expansion (dModel * ffnMultiplier) |\n| `maxSeqLen` | number | 256 | Maximum sequence length |\n| `mode` | `'parallel'` \\| `'single-thread'` | `'parallel'` | Kernel mode |\n| `precision` | `'f32'` \\| `'f16'` | `'f32'` | Compute precision |\n| `workgroupSize` | number | 64 | Threads per workgroup (parallel mode) |\n\n### `model.forward(embeddings, seqLen?)`\n\nReturns `InferenceResult` with `output` (Float32Array), `elapsed_ms`, `tokens`, `tok_per_sec`.\n\n### `model.benchmark({ warmup?, runs?, seqLen? })`\n\nReturns `BenchmarkStats` with `mean_ms`, `std_ms`, `median_ms`, `tok_per_sec`, etc.\n\n### `model.loadWeights(weights)` / `model.loadWeightsWithDefaults(weights)`\n\nLoad Float32Array of model weights. `WithDefaults` sets LayerNorm gamma=1, beta=0 automatically.\n\n### `getGPU()`\n\nReturns `{ device: GPUDevice, info: GPUInfo }` with GPU capabilities.\n\n## Requirements\n\n- Chrome 123+, Firefox 139+, Safari 18+ (WebGPU support)\n- For f16: browser must support `shader-f16` feature\n\n## Research\n\nBased on two preprints:\n\n- [Single-Kernel Fusion for Sequential Fitness Evaluation](https://doi.org/10.5281/zenodo.19343570) — 159-720x (paper), 4,081x avg real-world\n- [Single-Kernel Fusion for Autoregressive Transformer Decoding](https://doi.org/10.5281/zenodo.19344277) — 66-458x (paper), 826x avg on phones\n\n## License\n\nMIT\n","readmeFilename":"README.md"}