# Jobs — scheduled connector polls that fan out to pipelines or chat

## What a Job is

A Job is a new Forge primitive (alongside Task and Pipeline). It periodically
calls a **connector tool** and, for every new item in the result, **dispatches**
to either a Pipeline run or a Chat session.

Typical use cases:
- Every 30 min: `teams.list_messages_in_current_chat` → for each unread → POST
  to a chat session that asks the assistant to summarize and reply.
- Every hour: `mantis.search_bugs query="version:25.4 status:open"` → for each
  new bug → trigger the `bug-triage` pipeline bound to project `my-app` with
  `bug_id`, `summary` as params.

Job vs the other two:

| Concept | Triggered by | Does what |
|---|---|---|
| Task (existing) | manual / Telegram / API | one CLI-agent invocation |
| Pipeline (existing) | manual / schedule / GitHub issue / **Job** | DAG of CLI-agent tasks |
| **Job** (new) | scheduler tick (every `schedule_interval_minutes`) | calls one connector tool, dedups, dispatches per item |

The CLI keeps `forge task` / `forge pipeline` unchanged. A new `forge jobs`
subcommand handles Job CRUD + manual trigger.

## Storage (all sqlite)

The Job definition lives in sqlite — no yaml files — so the extension can
create / edit / delete jobs without writing to disk.

```sql
CREATE TABLE IF NOT EXISTS jobs (
  id                          TEXT PRIMARY KEY,         -- short uuid
  name                        TEXT NOT NULL,
  enabled                     INTEGER NOT NULL DEFAULT 1,
  schedule_interval_minutes   INTEGER NOT NULL DEFAULT 30,

  -- Source: which connector tool to call each tick
  source_connector            TEXT NOT NULL,            -- plugin_id, e.g. 'mantis'
  source_tool                 TEXT NOT NULL,            -- tool name, e.g. 'search_bugs'
  source_input                TEXT NOT NULL DEFAULT '{}', -- JSON: passed verbatim as tool input

  -- How to find items in the tool result + dedup key
  items_path                  TEXT,                     -- dotted path; empty = whole result if array
  dedup_field                 TEXT NOT NULL,            -- dotted path on each item, e.g. 'id' or 'meta.uuid'

  -- Dispatch (one of two modes)
  dispatch_type               TEXT NOT NULL,            -- 'pipeline' | 'chat'
  dispatch_params             TEXT NOT NULL DEFAULT '{}', -- JSON; shape depends on type — see below

  -- Bookkeeping
  last_run_at                 TEXT,
  next_run_at                 TEXT,
  created_at                  TEXT NOT NULL DEFAULT (datetime('now')),
  updated_at                  TEXT NOT NULL DEFAULT (datetime('now'))
);

CREATE TABLE IF NOT EXISTS job_runs (
  id              TEXT PRIMARY KEY,
  job_id          TEXT NOT NULL REFERENCES jobs(id) ON DELETE CASCADE,
  started_at      TEXT NOT NULL DEFAULT (datetime('now')),
  finished_at     TEXT,
  status          TEXT NOT NULL DEFAULT 'running',  -- running | ok | error
  items_seen      INTEGER NOT NULL DEFAULT 0,
  items_new       INTEGER NOT NULL DEFAULT 0,
  items_dispatched INTEGER NOT NULL DEFAULT 0,
  error           TEXT,
  trigger         TEXT NOT NULL DEFAULT 'schedule'  -- schedule | manual
);

CREATE TABLE IF NOT EXISTS job_seen (
  job_id          TEXT NOT NULL REFERENCES jobs(id) ON DELETE CASCADE,
  dedup_key       TEXT NOT NULL,
  first_seen_at   TEXT NOT NULL DEFAULT (datetime('now')),
  PRIMARY KEY (job_id, dedup_key)
);

CREATE TABLE IF NOT EXISTS job_dispatches (
  id                  TEXT PRIMARY KEY,
  job_run_id          TEXT NOT NULL REFERENCES job_runs(id) ON DELETE CASCADE,
  item_key            TEXT NOT NULL,
  item_preview        TEXT,                       -- short rendering for the UI
  dispatch_type       TEXT NOT NULL,              -- mirrored from job
  dispatch_target_id  TEXT,                       -- pipeline_id OR chat session_id
  status              TEXT NOT NULL DEFAULT 'dispatched',  -- dispatched | error
  error               TEXT,
  created_at          TEXT NOT NULL DEFAULT (datetime('now'))
);

CREATE INDEX IF NOT EXISTS idx_job_runs_job ON job_runs(job_id, started_at DESC);
CREATE INDEX IF NOT EXISTS idx_job_dispatches_run ON job_dispatches(job_run_id, created_at DESC);
```

## dispatch_params shape

### `dispatch_type = 'pipeline'`

```json
{
  "workflow_name": "bug-triage",
  "project_path": "/Users/me/code/my-app",
  "project_name": "my-app",
  "input_template": {
    "bug_id": "{{item.id}}",
    "summary": "{{item.summary}}",
    "priority": "{{item.priority}}"
  }
}
```

Per item: render `input_template` (string substitution `{{item.<path>}}`), then
call existing `triggerPipeline(project_path, project_name, workflow_name,
rendered_input, dedup_key=item_key)`.

### `dispatch_type = 'chat'`

```json
{
  "agent_profile": "my-litellm",            // optional; falls back to settings.chatAgent
  "session_title_template": "Bug {{item.id}}: {{item.summary}}",
  "message_template": "Process Mantis bug {{item.id}} (priority {{item.priority}}).\nSummary: {{item.summary}}\n\nFind the relevant branch and prepare a triage note.",
  "reuse_session": false                    // if true, all items append to one session; else fresh session per item
}
```

Per item: POST `/api/sessions` (or reuse) on chat-standalone, then POST a
message rendered from `message_template`. The chat agent gets the item and
decides what tools to call (could be another connector call, code edit via
shell, etc.).

## Template syntax

Values inside `{{ ... }}` use dotted paths into the item:
- `{{item.id}}` → item.id
- `{{item.assignee.name}}` → item.assignee.name
- `{{item}}` → JSON.stringify(item) (for dumping into a chat message)

Missing paths render to empty string. Non-string values render via
`String(...)` (numbers/booleans) or `JSON.stringify(...)` (objects).

Only `{{item.*}}` is supported in v1. `{{now}}` / `{{settings.*}}` are not
needed yet.

## Scheduler

Runs inside the same Next.js worker as `pipeline-scheduler` (started in
`lib/init.ts`). Every 60s:

1. Load all `enabled = 1` jobs where `next_run_at IS NULL OR next_run_at <= now`.
2. For each: insert a `job_runs` row, dispatch the tick asynchronously, advance
   `next_run_at = now + interval_minutes`, set `last_run_at = now`.
3. Per-tick work happens in a background promise (HTTP / SSE / spawn don't block
   the scheduler loop).

A tick is **idempotent** — if the previous tick is still running for the same
job (run row marked `status = 'running'` with no `finished_at`), the next tick
skips it and logs.

## A tick in detail

1. Resolve the connector + tool definition via `lib/plugins/registry` (same
   path the chat dispatcher uses).
2. Call `dispatchTool({id: '<job-run-id>', name: '<connector>.<tool>', input:
   source_input})` from `lib/chat/tool-dispatcher`. This handles `http` /
   `shell` / `browser` uniformly. (Browser-protocol jobs require the extension
   bridge to be connected at tick time — if not, surface as run error.)
3. Parse the tool's `content` string as JSON. Apply `items_path` to find the
   array (e.g. `bugs` → `result.bugs`). If parsed value is already an array,
   use it directly.
4. For each item:
   - Compute `key = pickPath(item, dedup_field)`. Stringify.
   - `INSERT OR IGNORE INTO job_seen (job_id, dedup_key)` — if a new row was
     written, it's a new item.
   - For new items only: dispatch via the configured type, insert into
     `job_dispatches` with the resulting target id.
5. Update `job_runs` row with counts + finished_at + status.

## API surface

```
GET    /api/jobs                       list (enabled+disabled)
POST   /api/jobs                       create
GET    /api/jobs/:id                   detail (with last 20 runs)
PATCH  /api/jobs/:id                   partial update
DELETE /api/jobs/:id                   cascade delete
POST   /api/jobs/:id/run               manual fire-now (returns run id)
GET    /api/jobs/:id/runs?limit=20     run history
GET    /api/jobs/:id/runs/:run_id      run detail (with dispatches)
POST   /api/jobs/:id/reset_dedup       wipe job_seen rows (force re-process all)
```

All gated by the existing Forge middleware (X-Forge-Token).

## CLI

```
forge jobs                              list
forge jobs new <interactive prompts>    create
forge jobs run <id>                     fire now
forge jobs runs <id>                    history
forge jobs rm <id>
forge jobs disable <id> / enable <id>
forge jobs reset <id>                   wipe dedup state
```

## Extension UI (Tasks → Jobs)

`TasksTab.tsx` renames to `JobsTab.tsx`. Tab label "Tasks" → "Jobs".

Sections:
1. **Active jobs** — for each: name, schedule, last run time + items new/seen,
   next run countdown, enable toggle, "Run now" button.
2. **+ New job** — modal:
   - name, interval
   - source: connector dropdown (from `/api/connectors`), tool dropdown
     (populated from selected connector's tools), input JSON
   - items_path + dedup_field (with examples per connector)
   - dispatch type radio: pipeline | chat
     - pipeline: workflow dropdown (`/api/pipelines/workflows`), project
       dropdown (`/api/projects`), input_template builder
     - chat: agent profile dropdown (`/api/agents`), session_title template,
       message_template (textarea), reuse_session toggle
3. **Recent runs** (collapsed by default) — last 20 across all jobs.

Per architecture rule: extension is display + DOM connectors only. All
scheduling, dedup, dispatch happens server-side. The extension talks to Forge
HTTP only — no local state for jobs.

## Out of scope (v1)

- Cron expressions (interval-only)
- Item filtering before dispatch (just write the connector tool to return
  filtered)
- Conditional dispatch (always-fire on new item; for skip-logic, use the
  pipeline / chat to decide internally)
- Webhook triggers (still poll-based)
- Backfill controls (initial tick auto-marks everything as seen without
  dispatching, so a brand-new job doesn't fire on history — controlled by
  `mark_existing_as_seen` flag at create time, default true)
