{"_id":"@arc-lang/arc-jobs","name":"@arc-lang/arc-jobs","dist-tags":{"latest":"0.1.0"},"versions":{"0.1.0":{"name":"@arc-lang/arc-jobs","version":"0.1.0","description":"Production job queues for Arc — SQLite, Redis, and SQS adapters with scheduling, deduplication, and progress tracking","main":"src/index.js","types":"src/types/index.d.ts","bin":{"arc-jobs":"bin/arc-jobs.js"},"exports":{".":"./src/index.js","./adapters/memory":"./src/adapters/memory.js","./adapters/sqlite":"./src/adapters/sqlite.js","./adapters/redis":"./src/adapters/redis.js","./dashboard":"./src/dashboard/handler.js"},"scripts":{"test":"bun test tests/","test:watch":"bun test --watch tests/","test:coverage":"bun test --coverage tests/"},"keywords":["arc","jobs","queue","celery","background-jobs","job-queue","sqlite","redis","bun","task-queue","cron","scheduler"],"repository":{"type":"git","url":"git+https://github.com/arc-language/arc-jobs.git"},"homepage":"https://arc-language.dev/docs/jobs","bugs":{"url":"https://github.com/arc-language/arc-jobs/issues"},"license":"MIT","engines":{"bun":">=1.0.0","node":">=18.0.0"},"peerDependencies":{"bun":">=1.0.0"},"peerDependenciesMeta":{"bun":{"optional":true}},"optionalDependencies":{"ioredis":">=5.0.0","better-sqlite3":">=9.0.0"},"publishConfig":{"access":"public"},"gitHead":"a8424885005de6158b786fc94d774f592f3832e5","_id":"@arc-lang/arc-jobs@0.1.0","_nodeVersion":"24.15.0","_npmVersion":"11.12.1","dist":{"integrity":"sha512-Gcuob5FZ4tmkmK7DyOqXswntqSh1RnplzW6lu5VbBTqk/kwJn3LxjsaKh+c94WP2YX2/iKFC9Olb0Qdf7c3xxg==","shasum":"76c8e4a6499720608235215acd59aea34f8d6eba","tarball":"https://registry.npmjs.org/@arc-lang/arc-jobs/-/arc-jobs-0.1.0.tgz","fileCount":18,"unpackedSize":83251,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEYCIQCh6WeatXotGKrB3Zl374wFiNCY1RGABlxgHz6HCffIjwIhALQf6IvMctPUvkVMRvM52HlbOdJznid6lEN0rrjsvu/g"}]},"_npmUser":{"name":"kobecuppens","email":"kobecuppens@hotmail.com"},"directories":{},"maintainers":[{"name":"kobecuppens","email":"kobecuppens@hotmail.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/arc-jobs_0.1.0_1780407179113_0.7236024818552638"},"_hasShrinkwrap":false}},"time":{"created":"2026-06-02T13:32:58.929Z","0.1.0":"2026-06-02T13:32:59.264Z","modified":"2026-06-02T13:32:59.551Z"},"maintainers":[{"name":"kobecuppens","email":"kobecuppens@hotmail.com"}],"description":"Production job queues for Arc — SQLite, Redis, and SQS adapters with scheduling, deduplication, and progress tracking","homepage":"https://arc-language.dev/docs/jobs","keywords":["arc","jobs","queue","celery","background-jobs","job-queue","sqlite","redis","bun","task-queue","cron","scheduler"],"repository":{"type":"git","url":"git+https://github.com/arc-language/arc-jobs.git"},"bugs":{"url":"https://github.com/arc-language/arc-jobs/issues"},"license":"MIT","readme":"# arc-jobs\n\nProduction background jobs for [Arc](https://arc-lang.com) — SQLite, Redis, and SQS adapters with scheduling, deduplication, and progress tracking.\n\n**Better than Django Celery because:**\n- Zero external broker for development — SQLite queue runs in-process\n- Compile-time type-safe — wrong job signatures fail at `arc build`, not in production\n- No separate worker process — jobs run inside your server\n- `@unique` prevents duplicate jobs (celery-once built-in)\n- `@schedule` replaces Celery Beat without a second process\n\n```arc\n@queue payments\n@priority high\n@unique strategy=skip\njob ProcessPayment(orderId: Int, amount: Float)\n  const order = db.orders.find(orderId)\n  stripe.charge(order.userId, amount)\n\n@schedule \"0 9 * * *\"\njob DailyDigest()\n  const users = db.users.findMany({ active: true })\n  for user in users\n    email.send({ to: user.email, subject: \"Your digest\" })\n\n@progress\njob ImportCSV(fileId: Int)\n  const rows = db.uploads.find(fileId)\n  for i, row in rows\n    job.progress(i / rows.length * 100)\n    processRow(row)\n```\n\n## Install\n\n```bash\nbun add arc-jobs\n```\n\nThen add queue configuration to `arc.config.json`:\n\n```json\n{\n  \"queues\": {\n    \"default\":   { \"backend\": \"sqlite\" },\n    \"payments\":  { \"backend\": \"redis\", \"url\": \"${REDIS_URL}\" },\n    \"reports\":   { \"backend\": \"sqlite\", \"timeout\": 300000 }\n  }\n}\n```\n\nOr run the interactive setup:\n\n```bash\narc-jobs init\n```\n\n## Quick Start\n\n### 1. Define jobs in your `.arc` server files\n\n```arc\n// server/jobs.arc\n\n// Basic job — uses the \"default\" queue\njob SendWelcomeEmail(userId: Int)\n  const user = db.users.find(userId)\n  email.send({ to: user.email, subject: \"Welcome!\" })\n\n// High-priority job on a dedicated Redis queue\n@queue payments\n@priority high\n@retries 5\njob ProcessPayment(orderId: Int, amount: Float)\n  // ... payment logic\n\n// Prevent duplicate sends while job is running or pending\n@queue notifications\n@unique timeout=3600000 strategy=skip\njob SendInvoice(invoiceId: Int)\n  email.send({ to: \"...\" })\n\n// Schedule without a separate process\n@schedule \"0 9 * * 1\"\njob WeeklyReport()\n  // ... runs every Monday at 9am\n```\n\n### 2. Call jobs from routes\n\n```arc\n@route post \"/orders\" -> Response\n  const order = db.orders.create(parseBody(request))\n  ProcessPayment(order.id, order.total)   // fire-and-forget\n  SendWelcomeEmail.delay(86400000, order.userId)  // delayed 24h\n  json(order, 201)\n```\n\n### 3. Build and run\n\n```bash\narc build-server .\nbun dist/server.js\n```\n\nThat's it. No broker to set up. Jobs run inside your server process.\n\n---\n\n## Job Annotations Reference\n\n### `@queue <name>`\n\nRoute the job to a named queue from `arc.config.json`. Omitting `@queue` uses the `\"default\"` queue.\n\n```arc\n@queue payments\njob ProcessPayment(orderId: Int)\n  // uses the \"payments\" queue (Redis, in this example)\n```\n\n### `@schedule \"<cron>\"`\n\nRun the job on a cron schedule. No separate Celery Beat process — the scheduler runs inside your server.\n\n```arc\n@schedule \"0 9 * * *\"     // daily at 9am UTC\n@schedule \"*/15 * * * *\"  // every 15 minutes\n@schedule \"0 0 1 * *\"     // first of every month\njob CleanupOldSessions()\n  db.sessions.deleteMany({ expiresAt: { lt: now() } })\n```\n\nCron format: `min hour dom month dow` (5-field standard cron).\n\n### `@priority high|normal|low`\n\nControls dequeue order within a queue. High-priority jobs are processed before normal, normal before low.\n\n```arc\n@priority high\njob ProcessPayment(orderId: Int)\n  // dequeued before normal jobs\n```\n\n### `@retries <n>`\n\nOverride the default retry count (default: 3). Failed jobs retry with exponential backoff.\n\n```arc\n@retries 5\n@backoff 2000     // base backoff in ms (doubles each retry)\njob SyncInventory(productId: Int)\n  // retries up to 5 times: 2s, 4s, 8s, 16s, 32s\n```\n\n### `@timeout <ms>`\n\nOverride the default job timeout (default: 30,000ms). Jobs that exceed this are treated as failures.\n\n```arc\n@timeout 300000   // 5-minute timeout\njob GenerateReport(reportId: Int)\n  // heavy computation\n```\n\n### `@concurrency <n>`\n\nLimit how many instances of this job run simultaneously across all workers.\n\n```arc\n@concurrency 2\njob ResizeImages(assetId: Int)\n  // max 2 running at a time\n```\n\n### `@unique`\n\n**celery-once equivalent.** Prevents duplicate jobs when one is already pending or running. Lock is keyed by job name + serialized args. Stale locks (crashed workers) auto-expire.\n\n```arc\n@unique                          // default: skip duplicates silently\n@unique strategy=skip            // same as above\n@unique strategy=reject          // throw JobAlreadyRunning error\n@unique strategy=replace         // cancel existing, enqueue new\n@unique timeout=3600000          // lock TTL in ms (default: 1 hour)\n@unique timeout=300000 strategy=skip\njob SendInvoice(invoiceId: Int)\n  email.send({ to: \"...\" })\n```\n\nCalling `SendInvoice(42)` twice while the first is pending → second is silently discarded (with `strategy=skip`).\n\n### `@progress`\n\nEnables `job.progress(pct)` inside the job body. Progress streams to `/_arc/jobs/:id/progress` via SSE, which can be consumed by `@live` pages.\n\n```arc\n@progress\njob ImportCSV(fileId: Int)\n  const rows = db.uploads.find(fileId)\n  for i, row in rows\n    job.progress((i + 1) / rows.length * 100, { processed: i + 1 })\n    processRow(row)\n```\n\n### `@then <JobName>`\n\nAuto-enqueue another job on success. Validated at compile time — `arc build` fails if `JobName` doesn't exist.\n\n```arc\n@then ProcessOrder\njob ValidateOrder(orderId: Int)\n  // ... validate; if this succeeds ProcessOrder(orderId) is auto-called\n\njob ProcessOrder(orderId: Int)\n  // ...\n```\n\n---\n\n## Calling Jobs\n\n```arc\n// Fire and forget (returns Promise<string> job id)\nSendInvoice(invoiceId)\n\n// Delayed execution\nSendReminderEmail.delay(86400000, userId)   // delay in ms\n\n// Run at a specific time\nDailyDigest.at(new Date(\"2026-06-01T09:00:00Z\"))\n\n// Custom idempotency key\nProcessPayment.unique(\"order-42-payment\", orderId, amount)\n\n// Check job status from a route\n@route get \"/jobs/:id\" -> Response\n  const status = await Queue.status(id)\n  json(status)\n```\n\n---\n\n## Queue Adapters\n\n### SQLite (default — zero ops)\n\nBest for: single-server apps, development, apps with < ~10k jobs/min.\n\n```json\n{\n  \"queues\": {\n    \"default\": { \"backend\": \"sqlite\" }\n  }\n}\n```\n\n- **~15,000 jobs/sec** in WAL mode\n- Zero infrastructure — uses your existing `app.db`\n- Persistent across restarts\n- Jobs survive server crashes\n\n### Redis (high-throughput)\n\nBest for: high-volume apps, multi-worker deployments, horizontal scaling.\n\n```json\n{\n  \"queues\": {\n    \"default\": {\n      \"backend\": \"redis\",\n      \"url\": \"${REDIS_URL}\"\n    }\n  }\n}\n```\n\n- **100,000+ ops/sec**\n- Priority queues via sorted sets\n- `@unique` locks via atomic `SET NX PX` (exact celery-once behavior)\n- Requires `Bun.Redis` (built-in) or `ioredis` (`bun add ioredis` for Node.js)\n\n### Mixed (recommended for production)\n\nUse SQLite for non-urgent work, Redis for time-critical paths:\n\n```json\n{\n  \"queues\": {\n    \"default\":       { \"backend\": \"sqlite\" },\n    \"payments\":      { \"backend\": \"redis\", \"url\": \"${REDIS_URL}\" },\n    \"notifications\": { \"backend\": \"redis\", \"url\": \"${REDIS_URL}\" }\n  }\n}\n```\n\n---\n\n## Admin Dashboard\n\nA built-in dashboard is served at `/_arc/jobs` when your server is running:\n\n- **Overview**: pending / running / completed / failed counts per queue (live, 2s refresh)\n- **Active jobs**: elapsed time, progress bar for `@progress` jobs, cancel button\n- **Schedules**: cron expression + next fire time\n- **Active locks**: `@unique` locks with remaining TTL, force-unlock button\n- **Dead letter queue**: failed jobs with Replay button\n\n---\n\n## CLI\n\n```bash\n# Interactive queue setup\narc-jobs init\n\n# Show queue depths and job counts\narc-jobs stats\narc-jobs stats --db path/to/app.db\n\n# Replay dead letter queue\narc-jobs replay\narc-jobs replay --job SendInvoice         # specific job type\narc-jobs replay --db path/to/app.db\n\n# Real-time terminal monitor (refreshes every 2s)\narc-jobs monitor\narc-jobs monitor --db path/to/app.db\n```\n\n---\n\n## Testing\n\nIn `NODE_ENV=test`, all queues use a synchronous in-memory adapter that does **not** auto-process jobs. Call `Queue.flush()` explicitly to run them.\n\n```javascript\n// tests/jobs.test.js\nimport { Queue, SendInvoice, ProcessPayment } from './dist/server.test.js'\n\ntest('SendInvoice is enqueued on order creation', async () => {\n  Queue.reset()\n\n  const res = await fetch('http://localhost:3001/orders', {\n    method: 'POST',\n    body: JSON.stringify({ amount: 99 }),\n  })\n  assert.strictEqual(res.status, 201)\n  Queue.assertEnqueued('SendInvoice', [42])\n})\n\ntest('SendInvoice executes correctly', async () => {\n  Queue.reset()\n  await SendInvoice(42)\n  await Queue.flush()\n  assert.strictEqual(Queue.completed('SendInvoice').length, 1)\n  assert.strictEqual(Queue.dead().length, 0)\n})\n\n// Test @unique deduplication\ntest('duplicate SendInvoice calls are deduplicated', async () => {\n  Queue.reset()\n  await SendInvoice(42)\n  await SendInvoice(42)  // duplicate — skipped\n  assert.strictEqual(Queue.pending('SendInvoice').length, 1)\n})\n\n// Test @then chain\ntest('@then chain auto-enqueues ProcessOrder', async () => {\n  Queue.reset()\n  await ValidateOrder(99)\n  await Queue.flush()\n  assert.strictEqual(Queue.completed('ValidateOrder').length, 1)\n  await Queue.flush()  // process the auto-enqueued job\n  assert.strictEqual(Queue.completed('ProcessOrder').length, 1)\n})\n```\n\nBuild a test-mode server with `NODE_ENV=test arc build-server .`.\n\n---\n\n## Performance vs Django Celery\n\n| | arc-jobs (SQLite) | arc-jobs (Redis) | Django Celery |\n|---|---|---|---|\n| **Enqueue latency** | ~0.1ms (local write) | ~0.5ms (local Redis) | ~1-5ms (network hop) |\n| **Throughput** | ~15k jobs/sec | ~100k jobs/sec | Millions/min (distributed) |\n| **Setup** | Zero — uses app.db | `bun add ioredis` | Redis + worker process + Celery Beat |\n| **Worker process** | In-process | In-process | Separate `celery worker` |\n| **Scheduler** | In-process setInterval | In-process setInterval | Separate `celery beat` |\n| **Type safety** | Compile-time | Compile-time | Runtime (stringly-typed) |\n| **`@unique`** | Built-in | Built-in | Requires celery-once |\n| **Test mode** | Synchronous flush | Synchronous flush | Needs mock broker |\n\nArc jobs on Bun outperform Python Celery for I/O-bound work (webhooks, email, API calls) — the common 80% case. Celery wins for CPU-bound tasks (ML, image processing) that benefit from multi-process parallelism.\n\n---\n\n## Cloudflare Workers\n\nOn the Cloudflare target, arc-jobs maps to native CF primitives:\n\n| Feature | CF Primitive |\n|---|---|\n| Queue | CF Queues binding |\n| `@schedule` | CF Cron Triggers (in `wrangler.toml`) |\n| `@unique` | Durable Objects |\n| `@progress` | Durable Objects state |\n\nNo configuration needed — `arc build --target cloudflare` handles it automatically.\n\n---\n\n## GitHub Actions CI\n\n```yaml\n# .github/workflows/test.yml\nname: Test\non: [push, pull_request]\njobs:\n  test:\n    runs-on: ubuntu-latest\n    steps:\n      - uses: actions/checkout@v4\n      - uses: oven-sh/setup-bun@v2\n      - run: bun install\n      - run: bun test packages/arc-jobs/tests/\n```\n\n---\n\n## License\n\nMIT\n","readmeFilename":"README.md","_rev":"1-ed91937883d760859d08e55a72e253bb"}