{"_id":"@backloghq/opslog-s3","_rev":"6-ef4cc2b38acc98d27b9f69302239c11b","name":"@backloghq/opslog-s3","dist-tags":{"latest":"0.4.1"},"versions":{"0.1.0":{"name":"@backloghq/opslog-s3","version":"0.1.0","keywords":["opslog","s3","storage-backend","event-sourcing","multi-writer","aws"],"author":{"name":"mbocevski"},"license":"MIT","_id":"@backloghq/opslog-s3@0.1.0","maintainers":[{"name":"fenrirbaest","email":"marko.bocevski@gmail.com"}],"dist":{"shasum":"7afd3628cfddb9b0e80fe3a49c1e6f72e37909b8","tarball":"https://registry.npmjs.org/@backloghq/opslog-s3/-/opslog-s3-0.1.0.tgz","fileCount":8,"integrity":"sha512-uYTYKecU0UCuwBxMrhQZDC89KP1avO2thcZhc6wiXGiL/MXDn1wFb+tjUSw4mR57D+4XFMoj13ZksipkXBmipA==","signatures":[{"sig":"MEQCIFPlcbZnuZ5wghk1KlMgENBVgSaIe5Z0iWyRCe70s6JBAiATI+Prr6yuDRgRxD30WWJHQc9pwvPCPzObSJe3ei7jCg==","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":17771},"main":"dist/index.js","type":"module","types":"dist/index.d.ts","engines":{"node":">=20"},"gitHead":"91214ab0b9b6d6ff6726cdf472a8984504096aca","scripts":{"dev":"tsc --watch","lint":"eslint src/ tests/","test":"vitest run","build":"tsc","test:watch":"vitest","test:coverage":"vitest run --coverage","test:integration":"vitest run tests/s3-real.test.ts"},"_npmUser":{"name":"fenrirbaest","email":"marko.bocevski@gmail.com"},"_npmVersion":"11.12.1","description":"S3 storage backend for @backloghq/opslog. Enables multi-writer concurrency across machines via Amazon S3.","directories":{},"_nodeVersion":"25.9.0","dependencies":{"@aws-sdk/client-s3":"^3.800.0"},"_hasShrinkwrap":false,"devDependencies":{"eslint":"^10.0.0","vitest":"^4.1.2","@eslint/js":"^10.0.0","typescript":"~6.0.2","@types/node":"^25.0.0","@backloghq/opslog":"^0.3.0","typescript-eslint":"^8.58.0","@vitest/coverage-v8":"^4.1.2","@aws-sdk/credential-providers":"^3.1027.0"},"peerDependencies":{"@backloghq/opslog":">=0.3.0"},"_npmOperationalInternal":{"tmp":"tmp/opslog-s3_0.1.0_1775757323963_0.0979789242833995","host":"s3://npm-registry-packages-npm-production"}},"0.1.1":{"name":"@backloghq/opslog-s3","version":"0.1.1","keywords":["opslog","s3","storage-backend","event-sourcing","multi-writer","aws"],"author":{"name":"mbocevski"},"license":"MIT","_id":"@backloghq/opslog-s3@0.1.1","maintainers":[{"name":"fenrirbaest","email":"marko.bocevski@gmail.com"}],"dist":{"shasum":"2a48401c9d3228f4aef71d80ed6a35439df8248d","tarball":"https://registry.npmjs.org/@backloghq/opslog-s3/-/opslog-s3-0.1.1.tgz","fileCount":8,"integrity":"sha512-yw2NSvVeL+pfUifCE2D/0PG5T+yujaib+bYq2jbb0c1HCdax4o6RwIIe3aPIRaZZ1bv2mGW++uFiU51awqncEQ==","signatures":[{"sig":"MEQCIF9DQBVT2RWGO5OWng0cGrOgphHoLeGkjj+5rIeRjYnDAiB28X9kZsyJ4Jr6hv4g4wpH6dadovSojNARVj2xchfWfQ==","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":23482},"main":"dist/index.js","type":"module","types":"dist/index.d.ts","engines":{"node":">=20"},"gitHead":"545251c9be2a47189cba920d1c49d2bf3d1332df","scripts":{"dev":"tsc --watch","lint":"eslint src/ tests/","test":"vitest run","build":"tsc","test:watch":"vitest","test:coverage":"vitest run --coverage","test:integration":"vitest run tests/s3-real.test.ts"},"_npmUser":{"name":"fenrirbaest","email":"marko.bocevski@gmail.com"},"_npmVersion":"11.12.1","description":"S3 storage backend for @backloghq/opslog. Enables multi-writer concurrency across machines via Amazon S3.","directories":{},"_nodeVersion":"25.9.0","dependencies":{"@aws-sdk/client-s3":"^3.800.0"},"_hasShrinkwrap":false,"devDependencies":{"eslint":"^10.0.0","vitest":"^4.1.2","@eslint/js":"^10.0.0","typescript":"~6.0.2","@types/node":"^25.0.0","@backloghq/opslog":"^0.3.0","typescript-eslint":"^8.58.0","@vitest/coverage-v8":"^4.1.2","@aws-sdk/credential-providers":"^3.1027.0"},"peerDependencies":{"@backloghq/opslog":">=0.3.0"},"_npmOperationalInternal":{"tmp":"tmp/opslog-s3_0.1.1_1775758964193_0.8063731631956914","host":"s3://npm-registry-packages-npm-production"}},"0.2.0":{"name":"@backloghq/opslog-s3","version":"0.2.0","keywords":["opslog","s3","storage-backend","event-sourcing","multi-writer","aws"],"author":{"name":"mbocevski"},"license":"MIT","_id":"@backloghq/opslog-s3@0.2.0","maintainers":[{"name":"fenrirbaest","email":"marko.bocevski@gmail.com"}],"dist":{"shasum":"62892214fc68f8e7f3eda47bbe3bea1779791b62","tarball":"https://registry.npmjs.org/@backloghq/opslog-s3/-/opslog-s3-0.2.0.tgz","fileCount":9,"integrity":"sha512-Y7rUVnPzQtcAynrdMhGKu1gkARdUbgIVhgMDYhqfHj/Qf7w/JJ1i+uUp7cvTFXxss7sw8ZfB3EMmomrW4PeqTA==","signatures":[{"sig":"MEUCIQDjNSN3n7DbVOof+gcVXtMTcwDTyuS4sRRoEpcMf9WCXQIgddOauc8ICDv7Wyc1NQsm1tbFcdcnYlx2lSlKilQeGsM=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":26124},"main":"dist/index.js","type":"module","types":"dist/index.d.ts","engines":{"node":">=22"},"gitHead":"6e083c4d519f2ce53e1c99529d9fef67be8003d8","scripts":{"dev":"tsc --watch","lint":"eslint src/ tests/","test":"vitest run","build":"tsc","test:watch":"vitest","test:coverage":"vitest run --coverage","test:integration":"vitest run tests/s3-real.test.ts"},"_npmUser":{"name":"fenrirbaest","email":"marko.bocevski@gmail.com"},"_npmVersion":"11.12.1","description":"S3 storage backend for @backloghq/opslog. Enables multi-writer concurrency across machines via Amazon S3.","directories":{},"_nodeVersion":"25.9.0","dependencies":{"@aws-sdk/client-s3":"^3.800.0"},"_hasShrinkwrap":false,"devDependencies":{"eslint":"^10.0.0","vitest":"^4.1.2","@eslint/js":"^10.0.0","typescript":"~6.0.2","@types/node":"^25.0.0","@backloghq/opslog":"^0.6.0","typescript-eslint":"^8.58.0","@vitest/coverage-v8":"^4.1.2","@aws-sdk/credential-providers":"^3.1027.0"},"peerDependencies":{"@backloghq/opslog":">=0.6.0"},"_npmOperationalInternal":{"tmp":"tmp/opslog-s3_0.2.0_1775836447124_0.13530181948311726","host":"s3://npm-registry-packages-npm-production"}},"0.3.0":{"name":"@backloghq/opslog-s3","version":"0.3.0","keywords":["opslog","s3","storage-backend","event-sourcing","multi-writer","aws"],"author":{"name":"mbocevski"},"license":"MIT","_id":"@backloghq/opslog-s3@0.3.0","maintainers":[{"name":"fenrirbaest","email":"marko.bocevski@gmail.com"}],"dist":{"shasum":"382b803a827e5dd5890e2de997149e3d69ebce2b","tarball":"https://registry.npmjs.org/@backloghq/opslog-s3/-/opslog-s3-0.3.0.tgz","fileCount":9,"integrity":"sha512-wPjjhtwLyINewVkSs+wasAWvkbXEbn5tUnPP5dD8TBRNmWiP2P10fOOfRf+axVK1PdFVlXgs0/GXnRPZEhfS4w==","signatures":[{"sig":"MEUCIQCCPByn5Pj02OO8tKrD6ZSMMSDoM4zjl5N5iJGlsrDWNAIgC7lur/GbepuDUe8MOSe+ps3w+8PzsTXC0i0zj4rxZSE=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":27081},"main":"dist/index.js","type":"module","types":"dist/index.d.ts","engines":{"node":">=22"},"gitHead":"192f7a39a85925a7cb1cde27163fc9cbdcde6379","scripts":{"dev":"tsc --watch","lint":"eslint src/ tests/","test":"vitest run","build":"tsc","test:watch":"vitest","test:coverage":"vitest run --coverage","test:integration":"vitest run tests/s3-real.test.ts"},"_npmUser":{"name":"fenrirbaest","email":"marko.bocevski@gmail.com"},"_npmVersion":"11.12.1","description":"S3 storage backend for @backloghq/opslog. Enables multi-writer concurrency across machines via Amazon S3.","directories":{},"_nodeVersion":"25.9.0","dependencies":{"@aws-sdk/client-s3":"^3.800.0"},"_hasShrinkwrap":false,"devDependencies":{"eslint":"^10.0.0","vitest":"^4.1.2","@eslint/js":"^10.0.0","typescript":"~6.0.2","@types/node":"^25.0.0","@backloghq/opslog":"^0.7.0","typescript-eslint":"^8.58.0","@vitest/coverage-v8":"^4.1.2","@aws-sdk/credential-providers":"^3.1027.0"},"peerDependencies":{"@backloghq/opslog":">=0.7.0"},"_npmOperationalInternal":{"tmp":"tmp/opslog-s3_0.3.0_1775908669812_0.22383972377999406","host":"s3://npm-registry-packages-npm-production"}},"0.4.0":{"name":"@backloghq/opslog-s3","version":"0.4.0","keywords":["opslog","s3","storage-backend","event-sourcing","multi-writer","aws"],"author":{"name":"mbocevski"},"license":"MIT","_id":"@backloghq/opslog-s3@0.4.0","maintainers":[{"name":"fenrirbaest","email":"marko.bocevski@gmail.com"}],"dist":{"shasum":"52ae8c939d6cb6aea5b22664009626e193ebd1b4","tarball":"https://registry.npmjs.org/@backloghq/opslog-s3/-/opslog-s3-0.4.0.tgz","fileCount":9,"integrity":"sha512-JDBkWV5b2OJOXrSyzR8lv8cW6fRpRGAEaQdH0ghU+CoT6bR0xkqYYbxwgMFKVILwQNTW29wRzE0yCjAX/F+p6g==","signatures":[{"sig":"MEQCIBGlMie6lELIjqkP4k2LnZT6cTII1nq05ejvnhRp4DIPAiA8sNFFb6sUHFVczsHqOOrbs1aXkJTvwwz8ouwLofb0Sw==","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":28171},"main":"dist/index.js","type":"module","types":"dist/index.d.ts","engines":{"node":">=22"},"gitHead":"56b7216ef75005db16b94dbafcbe34266ed93812","scripts":{"dev":"tsc --watch","lint":"eslint src/ tests/","test":"vitest run","build":"tsc","test:watch":"vitest","test:coverage":"vitest run --coverage","test:integration":"vitest run tests/s3-integration.test.ts"},"_npmUser":{"name":"fenrirbaest","email":"marko.bocevski@gmail.com"},"_npmVersion":"11.12.1","description":"S3 storage backend for @backloghq/opslog. Enables multi-writer concurrency across machines via Amazon S3.","directories":{},"_nodeVersion":"25.9.0","dependencies":{"@aws-sdk/client-s3":"^3.800.0"},"_hasShrinkwrap":false,"devDependencies":{"eslint":"^10.0.0","vitest":"^4.1.2","@eslint/js":"^10.0.0","typescript":"~6.0.2","@types/node":"^25.0.0","@backloghq/opslog":"^0.8.0","typescript-eslint":"^8.58.0","@vitest/coverage-v8":"^4.1.2","@aws-sdk/credential-providers":"^3.1027.0"},"peerDependencies":{"@backloghq/opslog":">=0.8.0"},"_npmOperationalInternal":{"tmp":"tmp/opslog-s3_0.4.0_1775922236972_0.4521535902539071","host":"s3://npm-registry-packages-npm-production"}},"0.4.1":{"name":"@backloghq/opslog-s3","version":"0.4.1","description":"S3 storage backend for @backloghq/opslog. Enables multi-writer concurrency across machines via Amazon S3.","main":"dist/index.js","types":"dist/index.d.ts","scripts":{"build":"tsc","dev":"tsc --watch","lint":"eslint src/ tests/","test":"vitest run","test:watch":"vitest","test:coverage":"vitest run --coverage","test:integration":"vitest run tests/s3-integration.test.ts"},"keywords":["opslog","s3","storage-backend","event-sourcing","multi-writer","aws"],"author":{"name":"mbocevski"},"license":"MIT","type":"module","engines":{"node":">=22"},"dependencies":{"@aws-sdk/client-s3":"^3.800.0"},"peerDependencies":{"@backloghq/opslog":">=0.8.0"},"devDependencies":{"@aws-sdk/credential-providers":"^3.1027.0","@backloghq/opslog":"^0.8.0","@eslint/js":"^10.0.0","@types/node":"^25.0.0","@vitest/coverage-v8":"^4.1.2","eslint":"^10.0.0","typescript":"~6.0.2","typescript-eslint":"^8.58.0","vitest":"^4.1.2"},"gitHead":"d805d5708866b89ed81507d6d7e89a5b77e00eed","_id":"@backloghq/opslog-s3@0.4.1","_nodeVersion":"25.9.0","_npmVersion":"11.13.0","dist":{"integrity":"sha512-+ej9vaD/0AHPVr5uJwfUk9LVthiFaOObSRZS4HQvoAaWJ7zV2R0hL3cY1v0bqPez8P6bBhx0Gl1wUHYNg2nMuw==","shasum":"165e6cdbb0855d14f453310efa5655036e24c588","tarball":"https://registry.npmjs.org/@backloghq/opslog-s3/-/opslog-s3-0.4.1.tgz","fileCount":9,"unpackedSize":28193,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEUCICDeLUD9cWNcXT+WRSs6Wd+znNRMQqQV72/iELsmj7oRAiEAibvoahrIoC+L0wqc9r/2PXJkI9m3oJMoQztonjjj4nc="}]},"_npmUser":{"name":"fenrirbaest","email":"marko.bocevski@gmail.com"},"directories":{},"maintainers":[{"name":"fenrirbaest","email":"marko.bocevski@gmail.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/opslog-s3_0.4.1_1777977142702_0.3831640633873583"},"_hasShrinkwrap":false}},"time":{"created":"2026-04-09T17:55:23.844Z","modified":"2026-05-05T10:32:23.060Z","0.1.0":"2026-04-09T17:55:24.103Z","0.1.1":"2026-04-09T18:22:44.389Z","0.2.0":"2026-04-10T15:54:07.285Z","0.3.0":"2026-04-11T11:57:49.949Z","0.4.0":"2026-04-11T15:43:57.137Z","0.4.1":"2026-05-05T10:32:22.936Z"},"author":{"name":"mbocevski"},"license":"MIT","keywords":["opslog","s3","storage-backend","event-sourcing","multi-writer","aws"],"description":"S3 storage backend for @backloghq/opslog. Enables multi-writer concurrency across machines via Amazon S3.","maintainers":[{"name":"fenrirbaest","email":"marko.bocevski@gmail.com"}],"readme":"# opslog-s3\n\nAmazon S3 storage backend for [@backloghq/opslog](https://github.com/backloghq/opslog). Store your event-sourced data in S3 and enable multi-writer concurrency across machines.\n\n## Install\n\n```bash\nnpm install @backloghq/opslog @backloghq/opslog-s3\n```\n\n## Usage\n\n```typescript\nimport { Store } from \"@backloghq/opslog\";\nimport { S3Backend } from \"@backloghq/opslog-s3\";\n\nconst store = new Store<{ title: string; status: string }>();\nawait store.open(\"my-store\", {\n  backend: new S3Backend({\n    bucket: \"my-bucket\",\n    prefix: \"opslog/my-store\",\n    region: \"us-east-1\",\n  }),\n  agentId: \"agent-A\",\n});\n\nawait store.set(\"task-1\", { title: \"Build API\", status: \"active\" });\nconst task = store.get(\"task-1\");\nawait store.close();\n```\n\nAnother agent on a different machine can write to the same store:\n\n```typescript\nconst store = new Store<{ title: string; status: string }>();\nawait store.open(\"my-store\", {\n  backend: new S3Backend({\n    bucket: \"my-bucket\",\n    prefix: \"opslog/my-store\",\n    region: \"us-east-1\",\n  }),\n  agentId: \"agent-B\",\n});\n\n// Sees agent-A's writes\nstore.get(\"task-1\"); // { title: \"Build API\", status: \"active\" }\n\n// Writes to its own WAL — no contention with agent-A\nawait store.set(\"task-2\", { title: \"Write tests\", status: \"active\" });\nawait store.close();\n```\n\n## Options\n\n```typescript\nnew S3Backend({\n  bucket: \"my-bucket\",         // S3 bucket name (required)\n  prefix: \"path/to/store\",    // Key prefix (optional, no trailing slash)\n  region: \"us-east-1\",        // AWS region (required if no client provided)\n  client: myS3Client,         // Pre-configured S3Client (optional)\n  lockTtlMs: 60000,           // Lock TTL in ms (default: 60000)\n});\n```\n\n### Pre-configured client\n\nPass your own `S3Client` if you need custom configuration (credentials, endpoint, middleware):\n\n```typescript\nimport { S3Client } from \"@aws-sdk/client-s3\";\n\nconst client = new S3Client({\n  region: \"us-east-1\",\n  endpoint: \"http://localhost:9000\", // MinIO, LocalStack, etc.\n  forcePathStyle: true,\n});\n\nconst backend = new S3Backend({ bucket: \"my-bucket\", client });\n```\n\n## How It Works\n\n### Storage layout\n\n```\ns3://bucket/prefix/\n  manifest.json                       # Store metadata\n  snapshots/snap-<ts>.jsonl           # Immutable state captures (JSONL)\n  ops/agent-<id>-<ts>.jsonl           # Per-agent WAL files\n  archive/archive-<period>.json       # Archived records\n```\n\n### WAL operations (per-batch objects)\n\nS3 has no native append. Instead of downloading, concatenating, and re-uploading a single growing file, the S3Backend stores each write as a separate S3 object:\n\n```\nops/agent-A-1744200000/\n  batch-1744200001-0000.jsonl   (ops from first write)\n  batch-1744200002-0000.jsonl   (ops from second write)\n  batch-1744200003-0000.jsonl   (ops from third write)\n```\n\n- **Append**: single PutObject per write (no download needed)\n- **Read**: ListObjectsV2 + parallel GetObject per batch, with incremental caching (only new batches are downloaded on subsequent reads)\n- **Truncate**: delete last batch object (or modify if it contains multiple ops)\n\nStores created with v0.1.0 (single `.jsonl` files) are transparently supported — the backend detects the format automatically.\n\n### Locking\n\nUses S3 conditional writes (`IfNoneMatch: *`) for lock acquisition — fails atomically if the lock already exists. Lock objects carry a TTL; stale locks from crashed agents are automatically recovered.\n\n### Blob storage + byte-range reads\n\n```typescript\nawait backend.writeBlob(\"data/records.jsonl\", buffer);\nconst full = await backend.readBlob(\"data/records.jsonl\");\nconst range = await backend.readBlobRange(\"data/records.jsonl\", 1024, 256); // HTTP Range header\n```\n\n`readBlobRange` uses S3 `GetObject` with `Range: bytes=N-M` header — reads only the requested bytes. Enables O(1) point lookups in JSONL record stores without downloading the full object.\n\n### Conflict resolution\n\nWhen multiple agents write to the same key, opslog's Lamport clock ordering resolves conflicts via last-writer-wins. See [opslog multi-writer docs](https://github.com/backloghq/opslog#multi-writer-mode) for details.\n\n## IAM Permissions\n\nThe S3Backend needs these S3 permissions on the bucket/prefix:\n\n```json\n{\n  \"Effect\": \"Allow\",\n  \"Action\": [\n    \"s3:GetObject\",\n    \"s3:PutObject\",\n    \"s3:DeleteObject\",\n    \"s3:ListBucket\"\n  ],\n  \"Resource\": [\n    \"arn:aws:s3:::my-bucket\",\n    \"arn:aws:s3:::my-bucket/opslog/*\"\n  ]\n}\n```\n\n## Development\n\n```bash\nnpm run build          # Compile TypeScript\nnpm run lint           # ESLint\nnpm test               # Run tests (uses in-memory mock S3, no AWS needed)\nnpm run test:coverage  # Tests with coverage\n```\n\n## License\n\nMIT\n","readmeFilename":"README.md"}