{"_id":"@kaliber/firebase-queue","_rev":"49-974b65c2bb6bd73665a48d03d5f23e47","name":"@kaliber/firebase-queue","dist-tags":{"latest":"1.3.0"},"versions":{"1.0.0":{"name":"@kaliber/firebase-queue","version":"1.0.0","keywords":["job","task","queue","worker","firebase","realtime","pipeline"],"author":{"name":"EECOLOR","email":"eecolor@gmail.com"},"license":"MIT","_id":"@kaliber/firebase-queue@1.0.0","maintainers":[{"name":"daveykropf","email":"davey@kaliber.net"},{"name":"eecolor","email":"eecolor@gmail.com"},{"name":"klaasman","email":"klaaslandsman@gmail.com"}],"homepage":"https://github.com/kaliberjs/firebase-queue","bugs":{"url":"https://github.com/kaliberjs/firebase-queue/issues"},"dist":{"shasum":"1b5ff8dd1b3abb95c880f4eb22e7909d915ccb0c","tarball":"https://registry.npmjs.org/@kaliber/firebase-queue/-/firebase-queue-1.0.0.tgz","fileCount":6,"integrity":"sha512-RDAwqJv/ZeXbOFsUOFnzPusAIqEkVxxhdGksxEE5BGJZ3UkNGnKxHBcrOpogoY/VwVOx9ePWDlYkKW7MXOnXqQ==","signatures":[{"sig":"MEUCIGQiNRP3LWbHPqwmZY6cGTMwbSsRAELCYhRkcE51OLEWAiEAyCCzvAENAUyurWp9A3kAH997DnpEIcVgLKJ/aC94dYI=","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}],"unpackedSize":23266,"npm-signature":"-----BEGIN PGP SIGNATURE-----\r\nVersion: OpenPGP.js v3.0.4\r\nComment: https://openpgpjs.org\r\n\r\nwsFcBAEBCAAQBQJbXiYFCRA9TVsSAnZWagAAYSkP/2VjvhutrjVQnUftli8f\nZ9sI/vhkCRTqv3oiGZjzsz2tVAAX8CKtrYdYbSexrWS4ibOmuabNB4BLd1Iu\nQnDYCvOsOCsLBIBUqKx8nFakYiwCA1hvhs7kLybq6X7Ytiat3ZiKs5y3O+3w\nE1tRuLig9k2upyhSPDvHVbfXmNOB/AwUHVYJMEL1653scSB+p+ntR9CMe9zF\nwif/ktapM17/lmd15rwCRDPfA1DcEwiQ+QyFrICRW5yKvFP5YgFMOU55qWzk\ni38uP0QVj27kQegQjgGoBktmb8GNpeyqARi/TYy/Mpwy7Fq50CPeY0QaUJs5\nuY4EdHIbq67juRigXXB5wqws7ppHhd/nz7bapEmYGlJinVqWPfNOqCCDHxKx\n1mBUKS2LZSPqA7DQCqhCFC1tGSY77LrrUA3ZBha4eUS7o846IK5/p31ezQin\nSoVXuq8b15Yfi+OOB6Q9rhQRo8sUHIQ+nCyLcAe8XZKkL5mMtXW7nZBEjT6a\nSvQLhWu97Np7CNd4OvU6S9oZAVADvRQHCMxkE4TP6DtuQVqdcYRl75KDQ5Rl\nKUr+6ZNZU4JWae6Oro3NbelkREf2NBu3skExlTwzHa2uRPLkGfaMRTZcCIfB\n0eUgO5o8LpwWeq51Vc33AYi0tmncSVs6dGvJcA1F9wTLYHe0DouU/0HNDD7i\nsdY0\r\n=ye43\r\n-----END PGP SIGNATURE-----\r\n"},"main":"src/queue.js","files":["src/**","LICENSE","README.md","package.json"],"gitHead":"1f503546d98fe6ef13a5b396cc0e6f685bf94bb0","scripts":{"test":"npm-run-all --race --silent --parallel server tests","tests":"nyc node tests/","commit":"git-cz","server":"firebase-server"},"_npmUser":{"name":"eecolor","email":"eecolor@gmail.com"},"repository":{"url":"git+https://github.com/kaliberjs/firebase-queue.git","type":"git"},"_npmVersion":"5.6.0","description":"A fault-tolerant, multi-worker, multi-stage job pipeline built on Firebase","directories":{},"_nodeVersion":"8.11.3","publishConfig":{"access":"public"},"_hasShrinkwrap":false,"devDependencies":{"nyc":"^12.0.2","husky":"^1.0.0-rc.13","firebase":"^5.2.0","coveralls":"^3.0.2","commitizen":"^2.10.1","npm-run-all":"^4.1.3","@commitlint/cli":"^7.0.0","firebase-server":"^0.12.0","semantic-release":"^15.8.1","cz-conventional-changelog":"^2.1.0","@commitlint/config-conventional":"^7.0.1"},"_npmOperationalInternal":{"tmp":"tmp/firebase-queue_1.0.0_1532896773711_0.44093456726213653","host":"s3://npm-registry-packages"}},"1.1.0":{"name":"@kaliber/firebase-queue","version":"1.1.0","keywords":["job","task","queue","worker","firebase","realtime","pipeline"],"author":{"name":"EECOLOR","email":"eecolor@gmail.com"},"license":"MIT","_id":"@kaliber/firebase-queue@1.1.0","maintainers":[{"name":"daveykropf","email":"davey@kaliber.net"},{"name":"eecolor","email":"eecolor@gmail.com"},{"name":"klaasman","email":"klaaslandsman@gmail.com"}],"homepage":"https://github.com/kaliberjs/firebase-queue","bugs":{"url":"https://github.com/kaliberjs/firebase-queue/issues"},"dist":{"shasum":"0a6099b00149f81b3c8775407f67f882892750dd","tarball":"https://registry.npmjs.org/@kaliber/firebase-queue/-/firebase-queue-1.1.0.tgz","fileCount":6,"integrity":"sha512-7deK0xQlG8Rtb/eXLGVoayVEzv5Lzbzz/VAmbJ5VoyIYHZ5m5BklkFr7/0hOq1gDMIvOqW3qXAAsWOPIAW6UyQ==","signatures":[{"sig":"MEYCIQD+r42o6vQKQLNxkHWNd3m6WSC/hUJsmi4t9YNri5skMgIhAO/Q9U2QjaBXEzoM+L47b1U0QoWRTkHv3vuh6vmZpSvk","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}],"unpackedSize":23427,"npm-signature":"-----BEGIN PGP SIGNATURE-----\r\nVersion: OpenPGP.js v3.0.4\r\nComment: https://openpgpjs.org\r\n\r\nwsFcBAEBCAAQBQJccw/+CRA9TVsSAnZWagAAGbwQAJbZ3l6oGlmuOBdj37Wu\njCC/2gGLElEL1VJUiQ05K/ZDlMM4pYSPVMYnadzj6pkIUKNpv7vaivBQYwrd\nqSSKWiV0T6EPppgcY2Ou1yX58V+jrXB+SYjRKVd8iXrU/+/GwIZ/SftipP/p\n/8FA0Sw/Z07SN0N83CLR4mjhSSjBx4U9gWBNy7EM7K0TTBxwHILS9ZR/cpbN\nrgiLfXQEOalX2pHz6uIPmAHz93aYmOBn3M1A1HLHwseguiPUk26RS4LRuJni\nmxvMWLUrqWd4x7hqQC4fUrgMXrrQMJgQQ4FEFZerPJvznCvCsc40f4NVFZTX\nV+C1z+YCMkuz9N0Ke0lz4U9kk8g0TR982OomsRhxkUo62Xssrj6iq98zDBMI\nCds/EIbO4ROlsHy0X9bAYnylVfvhHWFv/vJgbISK5ZBaw+F4TJdg6sKl6IqZ\nfe0Aa48DteC8qxJZUyJVmemRJsURS0LSawaVy2tZyBkRPrxabi/rqTWpDgZB\n9Cy/eLvQhJP4UUSYiCQl3LXXNP/2VumChvhte1b+PINYmWOSmzk8cmwnGMwB\n8FPvoES1Tnd4Wvr22cTpbAMNRr6/Zj866687mn88A8ya2y/wUQKln4rS4x97\nW2XIgsAfAIviEC8X40op60dvAbugdlopwMd+Ab8qyTbFR21geRTFhCAv/j+o\nqV4s\r\n=HlAN\r\n-----END PGP SIGNATURE-----\r\n"},"main":"src/queue.js","gitHead":"233ddc230e5bb5d786a1e1859897b992dfe06085","scripts":{"test":"npm-run-all --race --silent --parallel server tests","tests":"nyc node tests/","commit":"git-cz","server":"firebase-server"},"_npmUser":{"name":"eecolor","email":"eecolor@gmail.com"},"repository":{"url":"git+https://github.com/kaliberjs/firebase-queue.git","type":"git"},"_npmVersion":"6.4.1","description":"A fault-tolerant, multi-worker, multi-stage job pipeline built on Firebase","directories":{},"_nodeVersion":"10.15.1","publishConfig":{"access":"public"},"_hasShrinkwrap":false,"devDependencies":{"nyc":"^13.3.0","husky":"^1.3.1","firebase":"^5.8.4","coveralls":"^3.0.3","commitizen":"^3.0.7","npm-run-all":"^4.1.5","@commitlint/cli":"^7.5.2","firebase-server":"^1.0.1","semantic-release":"^15.13.3","cz-conventional-changelog":"^2.1.0","@commitlint/config-conventional":"^7.5.0"},"_npmOperationalInternal":{"tmp":"tmp/firebase-queue_1.1.0_1551044605559_0.919505161895307","host":"s3://npm-registry-packages"}},"1.2.0":{"name":"@kaliber/firebase-queue","version":"1.2.0","keywords":["job","task","queue","worker","firebase","realtime","pipeline"],"author":{"name":"EECOLOR","email":"eecolor@gmail.com"},"license":"MIT","_id":"@kaliber/firebase-queue@1.2.0","maintainers":[{"name":"daveykropf","email":"davey@kaliber.net"},{"name":"eecolor","email":"eecolor@gmail.com"},{"name":"klaasman","email":"klaaslandsman@gmail.com"}],"homepage":"https://github.com/kaliberjs/firebase-queue","bugs":{"url":"https://github.com/kaliberjs/firebase-queue/issues"},"dist":{"shasum":"d00df6890051a0567251cdaced99357416691d25","tarball":"https://registry.npmjs.org/@kaliber/firebase-queue/-/firebase-queue-1.2.0.tgz","fileCount":6,"integrity":"sha512-yoxo8RQLcyFyudjDInj0FjY95JWS2QGtKYLAG2C7M/NShWsNraws3yYX/CpnZbClH3XWglp73maWgECcsaOpGA==","signatures":[{"sig":"MEYCIQCF5AYRLKZzWbeSmFYf9XMd7B1s+KkgGijixqyrztIblQIhAL6F1rQmarotZQn7rkfaDQLz1R+WMsWLitY/zWsb3miP","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}],"unpackedSize":23427,"npm-signature":"-----BEGIN PGP SIGNATURE-----\r\nVersion: OpenPGP.js v3.0.4\r\nComment: https://openpgpjs.org\r\n\r\nwsFcBAEBCAAQBQJcuONGCRA9TVsSAnZWagAAdbYP/ju3koSUocJ5EKu/YwCh\nwgXy3SSEuYhpy3PvIv9Q5aSHjEvc/XWG4/ajsIe78YYPeo1Q/TjI/LHmKuU0\nMms5G0Wt4bWL2nxZnfPKMxUcLa+XqGukY/jRfAbwB/NE5ttjZVtmlfRaWoD2\nQv0VcIMLnKhS+sMzLK2aWaajP3odZDOF3L3EN8tNguvn3RpQDvutYuf0YQ5G\nEwfiv4poToF236B5kYPr9I9VEU+C0KNBKlfLmotpZ4G+ohXWpbaO4bYSo4nc\nZZ/L3Z70XtPBBUK6u4kdGanAWmK0KJFMnrOlY2u5tMD66sg8qPhUc/jCwS4t\nSz4wO1tWDh2x0biKR3gUWtHRfmdJxtMPg4Tp+rrBdAgIiOVQHJJ1rhnNbSUi\n6RP0nuq8avgJ+6WyHPOKWbDZGY+kxh79rcaIFZmgVOxHIcTJd807/siJuCto\n58dp/+SG+3IcfnTRPSN9ocG6IkFPssbLQ0g9r8m5IOuV3YlqkcvFl8f7fUQ4\nPgvTkqnYY7h76PMFC7NTvDkFQ78splwMqKMBxvZwfFvG0NTNqLn0stK3GJ91\nVvFMvGt8gTqFu2HVpcnnNDM9X7RcAR/wIrhC21PGAXJ/ChIHOgnJWeAIGqkz\nAcgRaxCrXqwNIka+YWeCkp4sfqFcX/wXGX0Q/9jUuHWzv2kkgH/OhuNIH9vw\n9n/U\r\n=UKI3\r\n-----END PGP SIGNATURE-----\r\n"},"main":"src/queue.js","gitHead":"f9fe2d807579238f5f9f0be41b34dfd036fc98a5","scripts":{"test":"npm-run-all --race --silent --parallel server tests","tests":"nyc node tests/","commit":"git-cz","server":"firebase-server"},"_npmUser":{"name":"eecolor","email":"eecolor@gmail.com"},"repository":{"url":"git+https://github.com/kaliberjs/firebase-queue.git","type":"git"},"_npmVersion":"6.5.0","description":"A fault-tolerant, multi-worker, multi-stage job pipeline built on Firebase","directories":{},"_nodeVersion":"10.15.3","publishConfig":{"access":"public"},"_hasShrinkwrap":false,"devDependencies":{"nyc":"^14.0.0","husky":"^1.3.1","firebase":"^5.9.4","coveralls":"^3.0.3","commitizen":"^3.1.0","npm-run-all":"^4.1.5","@commitlint/cli":"^7.5.2","firebase-server":"^1.0.1","semantic-release":"^15.13.3","cz-conventional-changelog":"^2.1.0","@commitlint/config-conventional":"^7.5.0"},"_npmOperationalInternal":{"tmp":"tmp/firebase-queue_1.2.0_1555620677635_0.2168067337696622","host":"s3://npm-registry-packages"}},"1.3.0":{"name":"@kaliber/firebase-queue","version":"1.3.0","keywords":["job","task","queue","worker","firebase","realtime","pipeline"],"author":{"name":"EECOLOR","email":"eecolor@gmail.com"},"license":"MIT","_id":"@kaliber/firebase-queue@1.3.0","maintainers":[{"name":"jerryduijm","email":"jduijm@kaliber.net"},{"name":"hjsielcken","email":"hsielcken@kaliber.net"},{"name":"rmostertkaliber","email":"rmostert@kaliber.net"},{"name":"kaliber-owner","email":"development@kaliber.net"},{"name":"daveykropf","email":"davey@kaliber.net"},{"name":"eecolor","email":"eecolor@gmail.com"},{"name":"pkuepers","email":"pkuepers@kaliber.net"},{"name":"larsvankleef","email":"spamlabtap@gmail.com"}],"homepage":"https://github.com/kaliberjs/firebase-queue","dist":{"shasum":"4a649b7ee8ed19fa95f73a0372f2ad9be394efce","tarball":"https://registry.npmjs.org/@kaliber/firebase-queue/-/firebase-queue-1.3.0.tgz","fileCount":8,"integrity":"sha512-k4KM3S4GL4txOfZdu/hdpZp0KpVqLpoGgNZpjIK9Cfsn9/4G/PmHh2LTdHTeQzplkG4pLhp0/OWzZLYJ6X9kNw==","signatures":[{"sig":"MEYCIQCui6CBXiqDK4lYsOQ7VsORUAVyVkjUW0zYfoS/R4SVsgIhANkUKQp5mZdx3zv5N/WNBjTP30aeEaDEIt9wTudtPrw1","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}],"unpackedSize":22819,"npm-signature":"-----BEGIN PGP SIGNATURE-----\r\nVersion: OpenPGP.js v4.10.10\r\nComment: https://openpgpjs.org\r\n\r\nwsFzBAEBCAAGBQJjd3P4ACEJED1NWxICdlZqFiEECWMYAoorWMhJKdjhPU1b\r\nEgJ2VmqptQ/+LEEVs6ofB/30/oNyWkKJEoCjrjmItahjgbhDueadKdRXm+QF\r\nHjDlc6l0sSE73uzuYHFxrFtEOixKrm4kFPH7EuYiYTRsIi0FwCj8Yu5z4fVz\r\naH5tJWAuwjmvvWFOQjdxlA45p3/c2b0zrX9UO4bnNHqTNCVZmtR2aEb5qxtT\r\nxCwORuapxXjnXfVvh5yrWOql0R6KDz5Qa0MwUmTr3zda8SuM9oUx3Adj1t4g\r\nLDA8d4oVxBWutulCGdUvB7973IWPXdp8mVeStlhv4zxknfIm7L9g2KDAhgBh\r\nmPphxIlJYwIXhFlurS5RF1UJ3PbRiyCIPxbtTJZYmFHVsGsE7+H/opt3YaR6\r\n3E2aVSnlFHjP4N13itSBChiyVUqnJolnbfifLwqCdJL60S2tShhAbi//op+j\r\n6DsA4BGEa0XfIHaZCU9Ez3A4G6tAbZZWaqPYOFAYXikKqtiGJY++Ew8bk4oe\r\nMeBtx6aeal7KiU3qykX1bzZIWdl8IrB9UoDuSFohPfJUBqMj3UYoGaxsIbfj\r\nFYtkebmRq/H1ZHJc8Fd03pxKyS4+A35IuT/YGlm+r1TvN6eG4HlAtplbhagv\r\nMxU8p+GpJXrpXwngypyaGNE7tx/MrhUfNQZ9VFX2c9pr/nOMJkcjgS5spMJj\r\n3c4ZS12SawUC20EYrHN+P5wz1K6XvYMoCXw=\r\n=zHLn\r\n-----END PGP SIGNATURE-----\r\n"},"main":"src/queue.js","scripts":{"test":"npm-run-all --race --silent --parallel server tests","tests":"nyc node tests/","server":"firebase-server"},"_npmUser":{"name":"eecolor","email":"eecolor@gmail.com"},"repository":{"url":"https://github.com/kaliberjs/firebase-queue.git","type":"git"},"description":"A fault-tolerant, multi-worker, multi-stage job pipeline built on Firebase","directories":{},"licenseText":"The MIT License (MIT)\n\nCopyright (c) 2016 Firebase\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n","publishConfig":{"access":"public"},"_hasShrinkwrap":false,"devDependencies":{"nyc":"^15.1.0","firebase":"8.1.1","npm-run-all":"^4.1.5","firebase-server":"^1.1.0"},"_npmOperationalInternal":{"tmp":"tmp/firebase-queue_1.3.0_1668772856476_0.07446004407413698","host":"s3://npm-registry-packages"}}},"time":{"created":"2018-07-29T20:39:33.517Z","modified":"2026-09-09T12:43:12.951Z","1.0.0":"2018-07-29T20:39:33.890Z","1.1.0":"2019-02-24T21:43:25.700Z","1.2.0":"2019-04-18T20:51:17.808Z","1.3.0":"2022-11-18T12:00:56.776Z"},"author":{"name":"EECOLOR","email":"eecolor@gmail.com"},"license":"MIT","homepage":"https://github.com/kaliberjs/firebase-queue","keywords":["job","task","queue","worker","firebase","realtime","pipeline"],"repository":{"url":"https://github.com/kaliberjs/firebase-queue.git","type":"git"},"description":"A fault-tolerant, multi-worker, multi-stage job pipeline built on Firebase","maintainers":[{"email":"davey@kaliber.net","name":"daveykropf"},{"email":"ewestra@kaliber.net","name":"eecolor"},{"email":"jduijm@kaliber.net","name":"jerryduijm"},{"email":"development@kaliber.net","name":"kaliber-owner"},{"email":"nhoenderop@kaliber.net","name":"nhoenderop"},{"email":"jbrunemann@kaliber.net","name":"janbru"},{"email":"asmit@kaliber.net","name":"albertkaliber"},{"email":"mvanbrakel@kaliber.net","name":"mvanbrakel"},{"email":"Rvanschaik@kaliber.net","name":"r0h-oh"},{"email":"claan@kaliber.net","name":"1cln"},{"email":"mreimus@kaliber.net","name":"maart-22"},{"email":"ldavidsondescelles@kaliber.net","name":"yoreljenkins"},{"email":"mdevalk@kaliber.net","name":"mdevalk1983"},{"email":"guido.evertzen@gmail.com","name":"guido.evertzen"},{"email":"svroom@kaliber.net","name":"svroom-kaliber"},{"email":"tomharms93@proton.me","name":"tomharms"},{"email":"rinus.vh@hotmail.com","name":"rinus.vh"},{"email":"pgoddijn@gmail.com","name":"pgoddijn"},{"email":"jvankralingen@kaliber.net","name":"jvankralingen"}],"readme":"_This is a heavily modified fork from the original [Firebase Queue library](https://github.com/firebase/firebase-queue). Everything, including the documentation, has been given a complete overhaul._\n\n----\n[![npm version](https://img.shields.io/npm/v/@kaliber/firebase-queue.svg)](https://www.npmjs.com/package/@kaliber/firebase-queue)\n\n\n# Firebase Queue\n\nA fault-tolerant, multi-worker, multi-stage job pipeline built on the [Firebase Realtime\nDatabase](https://firebase.google.com/docs/database/).\n\n```bash\nyarn add @kaliber/firebase-queue\n```\n\n## Table of Contents\n\n * [Usage](#usage)\n * [Documentation](#documentation)\n * [Contributing](#contributing)\n * [Thanks](#thanks)\n * [Differences](#differences)\n * [Motivation](#motivation)\n\n## Usage\n\nBasic usage example\n\n```js\nconst firebase = require('firebase-admin')\nconst Queue = require('@kaliber/firebase-queue')\n\nconst app = firebase.initializeApp(..., 'my-queue')\nconst tasksRef = app.database().ref('tasks')\n\n// the queue starts processing as soon as you create an instance\nconst queue = new Queue({ tasksRef, processTask, reportError })\n\n// capture shutdown signal to perform a gracefull shutdown\nprocess.on('SIGINT', async () => {\n  await queue.shutdown()\n  process.exit(0)\n})\n\nasync function processTask(task) {\n  try {\n    // do the work and optionally return a new task\n  } catch (e) {\n    reportError(e)\n    throw e // this marks the task as failed\n  }\n}\n\nfunction reportError(e) {\n  console.error(e)\n  // also report the error to your error tracker (Rollbar, Sentry, RayGun, ...)\n}\n```\n\n## Documentation\n\n* [Guide](docs/guide.md)\n\n\n## Contributing\n\nIf you'd like to contribute to Firebase Queue, please first read through our [contribution\nguidelines](.github/CONTRIBUTING.md).\n\n\n## Thanks\n\nI want to thank the original authors of Firebase Queue. Their work has helped me tremendously in\ncreating the version that is today. While I do not agree with some of the design decisions, I\nunderstand how the code evolved. Any negative comment from me about their code is not aimed at the\noriginal developers themselves, I am well aware how different priorities can lead to other choices.\n\n\n## Differences\n\nThis library is a stripped down version of the original one. Here I will try to motivate why I made\ncertain choices.\n\n### No more timeout\n\nThe original library had a way to specify a timeout. I eventually figured out this was a completely\nseparate system and moved it out of the worker. When I was done moving it I realized that providing\na mechanism for timeouts should not be in this library. The biggest issue with timeouts is that you\nsomehow have to cancel the process or worker that is handling the task and how this works is very\ndependent on the specific processing.\n\nThere are very valid reasons for having a concept of a time, most notably the case where the node\nprocess gets killed before the task is resolved. It however is very application specific how you\nwant to deal with these kinds of situations.\n\nI can imagine creating a separate library (or maybe just another component) for these types of\nsituations. We would need some real use cases to figure out how to implement this.\n\nFinally, a last reason to remove timeouts is because that one setting accounted for quite a large\npart of the code and introduced numerous tricky scenario's.\n\nAn easy workaround for timeouts is the following:\n\n```js\nfunction wait(x) { return new Promise(resolve, setTimeout(_ => { resolve('timeout') }, x)) }\n\nasync function processTask(task) {\n  const result = await Promise.race([doTheWork(task), wait(3000)])\n  if (result === 'timeout') ...\n  else ...\n}\n```\n\n### No more retries\n\nThe original library allowed you to retry tasks when they failed. Retrying is very much dependent on\nthe type of error. If a code / syntax error is happening there is no point in retrying. If it's a\nflaky internet connection there is.\n\nAdding retries to the process function itself is trivial, so I moved the burden to the users of the\nlibrary. To achieve similar behavior you can catch any error in the process function and return a\nnew task with an incremented `_numRetries` prop if the existing `_numRetries` prop is still below\nthe threshold.\n\n```js\nasync function processTask({ _numRetries = 0, ...task }) {\n  try {\n    await doTheWork(task)\n  } catch (e) {\n    if (_numRetries > 2) throw e\n    else return { ...task, _numRetries: _numRetries + 1 }\n  }\n}\n```\n\n### No more specs from Firebase\n\nThe original library retrieved it's specs from Firebase. In the last few years that I used the\nlibrary I never found the need to change the spec at runtime. The mechanism made the library harder\nto use with custom specs and required addition security rules. On top of that it introduced more\ncomplexity in the library itself.\n\nSimilar behavior can be achieved outside of the library. Simply listen for spec changes in Firebase\nand shutdown / recreate previously created queues.\n\n```js\nlet queue = null\nspecsRef.on('value', async snapshot => {\n  if (queue) {\n    const q = queue\n    queue = null\n    await q.shutdown()\n  }\n  if (!queue) {\n    queue = new Queue({ spec: snapshot.val() || undefined })\n  }\n})\n```\n\n### No more sanitize\n\nThe process function now by default receives the sanitized tasks. It however now also receives the\nsnapshot that contained that data, allowing the user access to all queue specific properties.\n\nIn practice we never needed sanitize, but we often required access to the key of the snapshot. This\nkey was only available when sanitize was set to `false`.\n\nThe new design has a cleaner interface with a few advantages:\n\n- It allows access to the `key`\n- It allows access to the `ref`\n- It allows access to all queue specific properties\n\n```js\nfunction processTask(sanitizedTask, { snapshot }) {\n  ...\n}\n```\n\n### No more suppress stack\n\nWe never used the option to suppress the stack trace because in our minds it is valuable\ninformation. I can see why you would however want to do it. A simple workaround would be to catch\nthe error and throw only it's message while logging the full error to another location.\n\n```js\nasync function processTask(task) {\n  try {\n    await doTheWork(task)\n  } catch (e) {\n    logError(e)\n    throw e.message\n  }\n}\n```\n\n### No more callbacks\n\nPromises won and especially with the async/await syntax it is a lot more useful. The `processTask`\nfunction can now just return a promise and based on it's result the task will be resolved or\nrejected. Updating the progress of a task now also returns a promise.\n\n```js\nasync function processTask(task, { setProgress }) {\n  const result = await doTheWorkPart1(task)\n  await setProgress(50)\n  await doTheWorkPart2(result)\n}\n```\n\n### No more dependencies\n\nThe original library had some dependencies on other libraries. I totally understand some of the\nchoices for these libraries. I however tend to want to keep my dependencies to a total minimum.\n\n- `rsvp`    - We do not need this anymore in the current landscape.\n- `winston` - This might be a bad choice, but the library now requires a `reportError` function when\n              the communication with firebase fails. This allows us to report errors to services\n              like Rollbar and Raygun. I personally never liked logging for development purposes and\n              prefer step debuggers or simple `console.log` statements. On top of that, the library\n              is very much simplified and a lot easier to grasp.\n- `uuid`    - Firebase has a perfect utility to generate unique enough id's. We'll just use that.\n- `lodash`  - While the lodash library is a great one, I found that we do not need it's power\n              anymore.\n\nAs for testing, I replaced `istanbul` with `nyc` because it is so much easier to use. I removed the\ntesting frameworks because they only added unneeded complexity. I also threw out Gulp, I don't see\nany reason to use it.\n\n### No more dynamic worker count\n\nThe original library had functions that allowed users of the queue to change the amount of workers\nat runtime. In practice we never needed to do that. Providing a similar behavior is quite easy to\nachieve by either creating extra queue instances or recreating the queue with `numWorkers` set to\nanother value.\n\nIt really depends on your use case which is most appropriate in your situation.\n\nIn the situation that you need to continue processing at all costs, just add / remove queues to\nchange the amount of active workers. Note that a queue is a very lightweight object, so creating\nmore than one is no problem.\n\n```js\nconst queues = [createQueue()]\n\nfunction createQueue() {\n  return new Queue({ ... })\n}\n\nfunction scaleUp() {\n  queues.push(createQueue())\n}\nasync function scaleDown() {\n  if (queues.length > 1) {\n    const queue = queues.pop()\n    await queue.shutdown()\n  }\n}\n```\n\nAnother solution is to just shutdown the current queue and restart it with a different amount of\nworkers.\n\n```js\nlet numWorkers = 1\nlet queue = createQueue(numWorkers)\n\nfunction createQueue(numWorkers) {\n  return new Queue({ ..., options: { numWorkers } })\n}\n\nasync function scaleUp() {\n  await queue.shutdown()\n  numWorkers += 1\n  queue = createQueue(numWorkers)\n}\nasync function scaleDown() {\n  await queue.shutdown()\n  numWorkers = Math.max(1, numWorkers - 1)\n  queue = createQueue(numWorkers)\n}\n```\n\n## Motivation\n\n### Future\n\nThe original Firebase Queue is no longer actively being maintained. The original repository points\nto Firebase functions as a way to achieve the same thing. The problem with functions is that they do\nnot have the desired execution guarantees. This requires you to combine them with a library like\nFirebase Queue if you want specific guarantees.\n\nWe use Firebase Queue extensively for request/reponse systems and in a lot of cases the Firebase\nREST API is the API of our application. Firebase Queue neatly helps us with handling requests. This\ndesign helps us greatly:\n\n- No more writing REST API's\n- Better security - the business logic is no longer accessible from the internet\n- Separation of concerns, we can simply add another listener to the same location in Firebase\n\n### Tests, design and bugs\n\nWhen I worked on Firebase Queue I noticed that the tests were not ideal and very heavily tied to the\nimplementation. While trying to untangle that I encountered non ideal design in the code and noticed\nthat some important features were not actually being tested.\n\nDuring this extensive period of refactoring I encountered several bugs for specific edge cases and\nalso a few places where errors would vanish without a trace.\n\nWhile slowly moving towards a better design and better and more complete set of tests I realized\nthat I should start from scratch with testing. Before I could do that however I needed to reduce the\namount of moving parts and only keep the essentials. I have now added (I think) 100% coverage of all\nfunctionality in files which are now a lot smaller than the original 2000+ lines.\n\nThe production code itself should now be very readable and more easily understood. The line count\nhas dropped dramatically and this is not only caused by removing functionality, but also by slowly\ncarving out a more ideal version. This is in no way critique to the original authors, it is not\nsomething that can be done in a time-constraint environment (work).\n\n### Fun\n\nI really like the concept and have used the original library a lot. On top of that, I like\nuntangling code and carving it into something (I think is) more beautiful.\n\nI also enjoy the idea that this might be useful for other people.\n","readmeFilename":"README.md"}