Skip to main content
⚠️ AppKit is in an early development phase and is subject to change without notice. DO NOT use in production environments.

# Plugins

Plugins are modular extensions that add capabilities to your AppKit application. They follow a defined lifecycle and have access to shared services like caching, telemetry, and streaming.

For complete API documentation, see the `Plugin` class reference.

## Built-in plugins​

### Server plugin​

Provides HTTP server capabilities with development and production modes.

Key features:

- Express server for REST APIs
- Vite dev server with hot module reload
- Static file serving for production
- Remote tunneling to deployed backends

The Server plugin uses the deferred initialization phase to access routes from other plugins.

#### What it does​

- Starts an Express server (default `host=0.0.0.0`, `port=8000`)
- Mounts plugin routes under `/api/<pluginName>/...`
- Adds `/health` endpoint (returns `{ status: "ok" }`)
- Serves frontend:
  - Development (`NODE_ENV=development`): runs a Vite dev server in middleware mode
  - Production: auto-detects static frontend directory (checks `dist`, `client/dist`, `build`, `public`, `out`)

#### Minimal server example​

The smallest valid AppKit server:

    // server/index.ts
    import { createApp, server } from "@databricks/appkit";

    await createApp({
      plugins: [server()],
    });

#### Manual server start example​

When you need to extend Express with custom routes:

    import { createApp, server } from "@databricks/appkit";

    const appkit = await createApp({
      plugins: [server({ autoStart: false })],
    });

    appkit.server.extend((app) => {
      app.get("/custom", (_req, res) => res.json({ ok: true }));
    });

    await appkit.server.start();

#### Configuration options​

    import { createApp, server } from "@databricks/appkit";

    await createApp({
      plugins: [
        server({
          port: 8000,          // default: Number(process.env.DATABRICKS_APP_PORT) || 8000
          host: "0.0.0.0",     // default: process.env.FLASK_RUN_HOST || "0.0.0.0"
          autoStart: true,     // default: true
          staticPath: "dist",  // optional: force a specific static directory
        }),
      ],
    });

### Analytics plugin​

Enables SQL query execution against Databricks SQL Warehouses.

Key features:

- File-based SQL queries with automatic type generation
- Parameterized queries with type-safe SQL helpers
- JSON and Arrow format support
- Built-in caching and retry logic
- Server-Sent Events (SSE) streaming

#### Basic usage​

    import { analytics, createApp, server } from "@databricks/appkit";

    await createApp({
      plugins: [server(), analytics({})],
    });

#### Where queries live​

- Put `.sql` files in `config/queries/`
- Query key is the filename without `.sql` (e.g. `spend_summary.sql` → `"spend_summary"`)

#### SQL parameters​

Use `:paramName` placeholders and optionally annotate parameter types using SQL comments:

    -- @param startDate DATE
    -- @param endDate DATE
    -- @param limit NUMERIC
    SELECT ...
    WHERE usage_date BETWEEN :startDate AND :endDate
    LIMIT :limit

Supported `-- @param` types (case-insensitive):

- `STRING`, `NUMERIC`, `BOOLEAN`, `DATE`, `TIMESTAMP`, `BINARY`

#### Server-injected parameters​

`:workspaceId` is injected by the server and must not be annotated:

    WHERE workspace_id = :workspaceId

#### HTTP endpoints​

The analytics plugin exposes these endpoints (mounted under `/api/analytics`):

- `POST /api/analytics/query/:query_key`
- `POST /api/analytics/users/me/query/:query_key`
- `GET /api/analytics/arrow-result/:jobId`
- `GET /api/analytics/users/me/arrow-result/:jobId`

#### Format options​

- `format: "JSON"` (default) returns JSON rows
- `format: "ARROW"` returns an Arrow "statement_id" payload over SSE, then the client fetches binary Arrow from `/api/analytics/arrow-result/:jobId`

### Execution context and `asUser(req)`​

AppKit manages Databricks authentication via two contexts:

- ServiceContext (singleton): Initialized at app startup with service principal credentials
- ExecutionContext: Determined at runtime - either service principal or user context

#### Headers for user context​

- `x-forwarded-user`: required in production; identifies the user
- `x-forwarded-access-token`: required for user token passthrough

#### Using `asUser(req)` for user-scoped operations​

The `asUser(req)` pattern allows plugins to execute operations using the requesting user's credentials:

    // In a custom plugin route handler
    router.post("/users/me/data", async (req, res) => {
      // Execute as the user (uses their Databricks permissions)
      const result = await this.asUser(req).query("SELECT ...");
      res.json(result);
    });

    // Service principal execution (default)
    router.post("/system/data", async (req, res) => {
      const result = await this.query("SELECT ...");
      res.json(result);
    });

#### Context helper functions​

Exported from `@databricks/appkit`:

- `getCurrentUserId()`: Returns user ID in user context, service user ID otherwise
- `getWorkspaceClient()`: Returns the appropriate WorkspaceClient for current context
- `getWarehouseId()`: `Promise<string>` (from `DATABRICKS_WAREHOUSE_ID` or auto-selected in dev)
- `getWorkspaceId()`: `Promise<string>` (from `DATABRICKS_WORKSPACE_ID` or fetched)
- `isInUserContext()`: Returns `true` if currently executing in user context

#### Development mode behavior​

In local development (`NODE_ENV=development`), if `asUser(req)` is called without a user token, it logs a warning and falls back to the service principal.

## Using plugins​

Configure plugins when creating your AppKit instance:

    import { createApp, server, analytics } from "@databricks/appkit";

    const AppKit = await createApp({
      plugins: [
        server({ port: 8000 }),
        analytics(),
      ],
    });

For complete configuration options, see `createApp`.

## Plugin management​

AppKit includes a CLI for managing plugins. All commands are available under `npx @databricks/appkit plugin`.

### Create a plugin​

Scaffold a new plugin interactively:

    npx @databricks/appkit plugin create

The wizard walks you through:

- Placement: In your repository (e.g. `plugins/my-plugin`) or as a standalone package
- Metadata: Name, display name, description
- Resources: Which Databricks resources the plugin needs (SQL Warehouse, Secret, etc.) and whether each is required or optional
- Optional fields: Author, version, license

The command generates a complete plugin scaffold with `manifest.json`, TypeScript class, and barrel exports — ready to register in your app.

### Sync plugin manifests​

Scan your project for plugins and generate `appkit.plugins.json`:

    npx @databricks/appkit plugin sync --write

This discovers plugin manifests from installed packages and local imports, then writes a consolidated manifest used by deployment tooling. Plugins referenced in your `createApp({ plugins: [...] })` call are automatically marked as required.

Use the `--silent` flag in build hooks to suppress output:

    {
      "scripts": {
        "sync": "appkit plugin sync --write --silent",
        "predev": "npm run sync",
        "prebuild": "npm run sync"
      }
    }

### Validate manifests​

Check plugin manifests against the JSON schema:

    # Validate manifest.json in the current directory
    npx @databricks/appkit plugin validate

    # Validate specific files or directories
    npx @databricks/appkit plugin validate plugins/my-plugin appkit.plugins.json

The validator auto-detects whether a file is a plugin manifest or a template manifest (from `$schema`) and reports errors with humanized paths and expected values.

### List plugins​

View registered plugins from `appkit.plugins.json` or scan a directory:

    # From appkit.plugins.json (default)
    npx @databricks/appkit plugin list

    # Scan a directory for plugin folders
    npx @databricks/appkit plugin list --dir plugins/

    # JSON output for scripting
    npx @databricks/appkit plugin list --json

### Add a resource to a plugin​

Interactively add a new resource requirement to an existing plugin manifest:

    npx @databricks/appkit plugin add-resource

    # Or specify the plugin directory
    npx @databricks/appkit plugin add-resource --path plugins/my-plugin

## Creating custom plugins​

If you need custom API routes or background logic, implement an AppKit plugin. The fastest way is to use the CLI:

    npx @databricks/appkit plugin create

For a deeper understanding of the plugin structure, read on.

### Basic plugin example​

Extend the `Plugin` class and export with `toPlugin()`:

    import { Plugin, toPlugin } from "@databricks/appkit";
    import type express from "express";

    class MyPlugin extends Plugin {
      name = "myPlugin";

      // Define resource requirements in the static manifest
      static manifest = {
        name: "myPlugin",
        displayName: "My Plugin",
        description: "A custom plugin",
        resources: {
          required: [
            {
              type: "secret",
              alias: "apiKey",
              resourceKey: "apiKey",
              description: "API key for external service",
              permission: "READ",
              fields: {
                scope: { env: "MY_SECRET_SCOPE", description: "Secret scope" },
                key: { env: "MY_API_KEY", description: "Secret key name" }
              }
            }
          ],
          optional: []
        }
      };

      async setup() {
        // Initialize your plugin
      }

      myCustomMethod() {
        // Some implementation
      }

      async shutdown() {
        // Clean up resources
      }

      exports() {
        // an object with the methods from this plugin to expose
        return {
          myCustomMethod: this.myCustomMethod
        }
      }
    }

    export const myPlugin = toPlugin<typeof MyPlugin, Record<string, never>, "myPlugin">(
      MyPlugin,
      "myPlugin",
    );

### Config-dependent resources​

The manifest defines resources as either `required` (always needed) or `optional` (may be needed). For resources that become required based on plugin configuration, implement a static `getResourceRequirements(config)` method:

    interface MyPluginConfig extends BasePluginConfig {
      enableCaching?: boolean;
    }

    class MyPlugin extends Plugin<MyPluginConfig> {
      name = "myPlugin";

      static manifest = {
        name: "myPlugin",
        displayName: "My Plugin",
        description: "A plugin with optional caching",
        resources: {
          required: [
            { type: "sql_warehouse", alias: "warehouse", resourceKey: "sqlWarehouse", description: "Query execution", permission: "CAN_USE", fields: { id: { env: "DATABRICKS_WAREHOUSE_ID" } } }
          ],
          optional: [
            // Listed as optional in manifest for static analysis
            { type: "database", alias: "cache", resourceKey: "cache", description: "Query result caching (if enabled)", permission: "CAN_CONNECT_AND_CREATE", fields: { instance_name: { env: "DATABRICKS_CACHE_INSTANCE" }, database_name: { env: "DATABRICKS_CACHE_DB" } } }
          ]
        }
      };

      // Runtime: Convert optional resources to required based on config
      static getResourceRequirements(config: MyPluginConfig) {
        const resources = [];
        if (config.enableCaching) {
          // When caching is enabled, Database becomes required
          resources.push({
            type: "database",
            alias: "cache",
            resourceKey: "cache",
            description: "Query result caching",
            permission: "CAN_CONNECT_AND_CREATE",
            fields: {
              instance_name: { env: "DATABRICKS_CACHE_INSTANCE" },
              database_name: { env: "DATABRICKS_CACHE_DB" },
            },
            required: true  // Mark as required at runtime
          });
        }
        return resources;
      }
    }

This pattern allows:

- Static tools (CLI, docs) to show all possible resources
- Runtime validation to enforce resources based on actual configuration

### Key extension points​

- Route injection: Implement `injectRoutes()` to add custom endpoints using `IAppRouter`
- Lifecycle hooks: Override `setup()`, and `shutdown()` methods
- Shared services:
  - Cache management: Access the cache service via `this.cache`. See `CacheConfig` for configuration.
  - Telemetry: Instrument your plugin with traces and metrics via `this.telemetry`. See `ITelemetry`.
- Execution interceptors: Use `execute()` and `executeStream()` with `StreamExecutionSettings`

Consuming your plugin programmatically

Optionally, you may want to provide a way to consume your plugin programmatically using the AppKit object. To do that, your plugin needs to implement the `exports` method, returning an object with the methods you want to expose. From the previous example, the plugin could be consumed as follows:

    const AppKit = await createApp({
      plugins: [
        server({ port: 8000 }),
        analytics(),
        myPlugin(),
      ],
    });

    AppKit.myPlugin.myCustomMethod();

See the `Plugin` API reference for complete documentation.

## Caching​

AppKit provides both global and plugin-level caching capabilities.

### Global cache configuration​

    await createApp({
      plugins: [server(), analytics({})],
      cache: {
        enabled: true,
        ttl: 3600,              // seconds
        strictPersistence: false,
      },
    });

Storage auto-selects Lakebase V1 (Provisioned) persistent cache when healthy, otherwise falls back to in-memory. Support for Lakebase Autoscaling coming soon.

### Plugin-level caching​

Inside a Plugin subclass:

    const value = await this.cache.getOrExecute(
      ["myPlugin", "data", userId],
      async () => expensiveWork(),
      userKey,
      { ttl: 300 },
    );

## Plugin phases​

Plugins initialize in three phases:

- Core: Reserved for framework-level plugins. Initializes first.
- Normal: Default phase for application plugins. Initializes after core.
- Deferred: Initializes last with access to other plugin instances via `config.plugins`. Use when your plugin depends on other plugins (e.g., Server Plugin).