{"_id":"6sky","_rev":"1-f486d3b19a31c7faab248116730964b5","name":"6sky","dist-tags":{"latest":"0.9.13"},"versions":{"0.9.11":{"name":"6sky","version":"0.9.11","keywords":["bluesky","bsky","bski","atproto","firehose","car","dag/cbor","dag","cbor"],"author":{"name":"Oleg Mihailik"},"license":"MIT","_id":"6sky@0.9.11","maintainers":[{"name":"omihailik","email":"mihailik@gmail.com"}],"homepage":"https://github.com/colds-ky/bski#readme","bugs":{"url":"https://github.com/colds-ky/bski/issues"},"dist":{"shasum":"8e2272ef5aacc83b23358c6b08d6f849f46ddf74","tarball":"https://registry.npmjs.org/6sky/-/6sky-0.9.11.tgz","fileCount":512,"integrity":"sha512-e7x8VHZtTtQahmpla8yINxsNr7yj4w+6ObtdSbXno6lqwUwdU4SgVK9b8+fnsoYnvhg/lGcWlkzFMfjpZRlFUQ==","signatures":[{"sig":"MEYCIQCALFseU3p6aJ8gsqc/eOBPF43HYq3TfYssugrajQoSBgIhANvzqGbGApRJe4OErzgdmLxlTn6N1CFqGp0sMHLdNLvD","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":1816813},"main":"index.cjs","type":"module","exports":{"import":"./index.js","require":"./index.cjs"},"gitHead":"34693055799badeb43417ae65b7ea0b3fec22cdc","scripts":{"test":"node tests/test.cjs","build":"esbuild --bundle index.js --tsconfig=tsconfig.json --analyze --format=iife --sourcemap --global-name=bski --outfile=index.cjs","build-rollup":"rollup index.js --format=iife --sourcemap --name=bski --file=index.cjs"},"_npmUser":{"name":"omihailik","email":"mihailik@gmail.com"},"repository":{"url":"git+https://github.com/colds-ky/bski.git","type":"git"},"_npmVersion":"11.0.0","description":"BlueSky firehose/WebSocket and CAR in plain JavaScript","directories":{},"_nodeVersion":"20.11.1","dependencies":{"@atcute/car":"^2.0.1"},"_hasShrinkwrap":false,"devDependencies":{"rollup":"^4.32.1","esbuild":"^0.24.2"},"_npmOperationalInternal":{"tmp":"tmp/6sky_0.9.11_1738490338958_0.35044654894549554","host":"s3://npm-registry-packages-npm-production"}},"0.9.13":{"name":"6sky","version":"0.9.13","author":{"name":"Oleg Mihailik"},"license":"MIT","description":"BlueSky firehose/WebSocket and CAR in plain JavaScript","repository":{"type":"git","url":"git+https://github.com/colds-ky/bski.git"},"main":"index.cjs","exports":{"import":"./index.js","require":"./index.cjs"},"type":"module","scripts":{"build":"esbuild --bundle index.js --tsconfig=tsconfig.json --analyze --format=iife --sourcemap --global-name=bski --outfile=index.cjs","build-rollup":"rollup index.js --format=iife --sourcemap --name=bski --file=index.cjs","test":"node tests/test.cjs"},"keywords":["bluesky","bsky","bski","atproto","firehose","car","dag/cbor","dag","cbor"],"devDependencies":{"esbuild":"^0.24.2","rollup":"^4.32.1"},"dependencies":{"@atcute/car":"^2.0.1"},"_id":"6sky@0.9.13","gitHead":"5d59bda8b0acb8100f7a7b24d473e1c19a122c62","bugs":{"url":"https://github.com/colds-ky/bski/issues"},"homepage":"https://github.com/colds-ky/bski#readme","_nodeVersion":"20.11.1","_npmVersion":"11.0.0","dist":{"integrity":"sha512-OQ285H+kO7Xx40xBC4Yfucp/GVZh1Ji1d+DRGGO/VWZWIxMqH1OBLrfrG87xr0ujDHM99UFzG5eYg1UGZNFGIw==","shasum":"0c22802466272eddc7de5034789cfe6760811b9c","tarball":"https://registry.npmjs.org/6sky/-/6sky-0.9.13.tgz","fileCount":501,"unpackedSize":1812546,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEUCIQC4q8/Pek1JPdeXdZCeTCaEHNmlwMpTHdGpX+YLj4gdpwIgFadm9b37q4Csadr6hlCmLWZQsfIJdHQu+EAxB8v3MAM="}]},"_npmUser":{"name":"omihailik","email":"mihailik@gmail.com"},"directories":{},"maintainers":[{"name":"omihailik","email":"mihailik@gmail.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/6sky_0.9.13_1738505623082_0.07326113942315615"},"_hasShrinkwrap":false}},"time":{"created":"2025-02-02T09:58:58.957Z","modified":"2025-02-02T14:13:43.562Z","0.9.11":"2025-02-02T09:58:59.195Z","0.9.13":"2025-02-02T14:13:43.385Z"},"bugs":{"url":"https://github.com/colds-ky/bski/issues"},"author":{"name":"Oleg Mihailik"},"license":"MIT","homepage":"https://github.com/colds-ky/bski#readme","keywords":["bluesky","bsky","bski","atproto","firehose","car","dag/cbor","dag","cbor"],"repository":{"type":"git","url":"git+https://github.com/colds-ky/bski.git"},"description":"BlueSky firehose/WebSocket and CAR in plain JavaScript","maintainers":[{"name":"omihailik","email":"mihailik@gmail.com"}],"readme":"# Bski raw CAR/CBOR format parsing\n\nParsing raw binary content of the realtime **firehose** from WebSocket, and **CAR** account repository snapshot data.\n\nSelf-contained, zero dependencies.\n\n## Installation\n\n```\nnpm install bski\n```\n\n## Reading firehose\n\n```JavaScript\nimport { firehose } from 'bski'; // import from npm\n\nconst chunk = [];\nfor await(const msg of firehose.each()) {\n  chunk.push(msg);\n  if(chunk.length === 1000) break;\n}\n```\n\n[ ![Example using bski to fetch 1000 messages from the firehose](example-firehose.png) ](https://tty.wtf/%23+BSki+firehose//%60%60%60JavaScript/import+%7B+firehose+%7D+from+'bski';//const+chunk+=+%5B%5D;/for+await(const+msg+of+firehose.each())+%7B/++chunk.push(msg);/++if(chunk.length+===+1000)+break;/%7D//chunk/%60%60%60//)\n\n## Reading CAR\n\n```JavaScript\nimport { readCAR } from 'bski'; // import from npm\n\nconst did = 'did:plc:z72i7hdynmk6r22z27h6tvur';\nconst car = await fetch('https://puffball.us-east.host.bsky.network/xrpc/com.atproto.sync.getRepo?did=' + did)\n  .then(x => x.arrayBuffer());\n\nconst records = readCAR(did, car);\n```\n\n[ ![Example using bski to fetch CAR snapshot file and parse it](example-car.png) ](https://tty.wtf/%23+BSki+reading+CAR+snapshot//See+@bsky.app+likes,+tweets,+follows+etc.//%60%60%60JavaScript/import+%7B+readCAR+%7D+from+'bski';//const+did+=+'did:plc:z72i7hdynmk6r22z27h6tvur';/const+car+=+await+fetch('https:%2F%2Fpuffball.us-east.host.bsky.network%2Fxrpc%2Fcom.atproto.sync.getRepo%3Fdid='+%2B+did)/++.then(x+=%3E+x.arrayBuffer());//const+records+=+readCAR(did,+car);//%60%60%60//)\n\n## API details\n\n### firehose\n\n```TypeScript\nfirehose(address = 'wss://bsky.network/xrpc/com.atproto.sync.subscribeRepos'):\n  AsyncIterable<FirehoseRecord[]>\n```\n\nConnects to a firehose via WebSocket (defaults to the central server https://bsky.network/\nor potentially a local server i.e. PDS if the address parameter is provided).\n\nYields records in batches `FirehoseRecord[]`.\n\nBatching lets you consume records at your own speed. If you're iterating\nand immediatelly processing &mdash; they will come in batches of one.\nIf your code stalls in process, they will queue up and next iteration will come\nwith whole pile at once.\n\nExiting the iterator loop disconnects from the WebSockets and discards any unprocessed records.\n\n```TypeScript\nfirehose.each(address?): AsyncIterable<FirehoseRecord>\n```\n\nSame as the firehose() above, but always reporting records one by one.\n\nThe queueing still happens behind the scene, but if your code stalls it will still\nreceive each record separately. That comes with a small performance penalty.\n\n### readCAR\n\n```TypeScript\nreadCAR(messageBuf: ArrayBuffer | Uint8Array, did: string): FirehoseRepositoryRecord[]\n```\n\nParses binary CAR/DAG/CBOR format that is the archive/database format for BlueSky account history.\n\nThe parser is pretty fast: 50Mb repository takes 1-2 seconds. However, for a web app that delay could be jarring. Enter sequenceReadCAR:\n\n```TypeScript\nsequenceReadCAR(messageBuf: ArrayBuffer | Uint8Array, did: string):\n  Iterable<FirehoseRepositoryRecord | undefined>\n```\n\nParsing that binary, yielding the parsed records in implementation-defined batches.\n\nThis lets your code parse CAR even on the main\nthread incrementally, without freezing the app.\n\n## Additional metadata on the records\n\nApart from capturing the built-in BlueSky fields,\nboth **firehose** and **readCAR** collect a couple extras:\n\n* **repo** the DID of the account making the record (post/like etc.)\n* **uri** standard `at://<did>/<type>/<hash>` way of referring to events in ATProto\n* **cid** another standard identifier, a bit longer and less useful\n* **action** mostly just `create` but can also be `delete` or `update` (think updating user profile)\n* **time** BlueSky-observed time (different from self-reported time that can be spoofed by a poster)\n* **receiveTimestamp** Unix-style time the message is received from WebSocket\n* **parseTime** useful for tracking performance, averages to 0.3 millisecond\n\n## History and references\n\nThe firehose functionality existed in\n[colds.ky](https://colds.ky) codebase for a while,\nusing some of the packages referenced by the official\n[@atproto/api](https://www.npmjs.com/package/@atproto/api):\n\n* [@ipld/car](https://github.com/ipld/js-car) - Apache 2.0 and MIT\n* [cbor-x](https://github.dev/kriszyp/cbor-x) - MIT\n* [multiformats](https://github.com/multiformats/js-multiformats) - Apache 2.0 and MIT\n* [cborg](github.com/rvagg/cborg) - Apache 2.0\n* [@ipld/dag-cbor](https://github.com/ipld/js-dag-cbor) - Apache 2.0 and MIT\n* [varint](https://github.com/chrisdickinson/varint) - MIT\n\nBut those are complex and broader-purpose libraries.\nLater [@mary.my.id](https://bsky.app/profile/mary.my.id) created leaner,\nmore focused set of libraries to transcode some of the same formats, [@atcute/*](https://github.com/mary-ext/atcute) - MIT license.\n\nAnd now this library here is taking in only few necessary bits,\nfocusing on singular use case: parsing realtime firehose,\nand account repository CAR.\n\n# License and links\n\n[MIT](LICENSE) Oleg Mihailik\n\n* [GitHub](https://github.com/colds-ky/bski)\n* [npm](https://www.npmjs.com/package/bski)\n* [@oyin.bo](https://bsky.app/profile/oyin.bo)","readmeFilename":"README.md"}