GrowthBook A/B Testing

Feature flagging and experiment management using GrowthBook for gradual rollouts, A/B tests, and targeted feature releases.

Internal Infrastructure GrowthBook integration is used for Claude Code's own product development telemetry and experiment tracking. It is not a user-facing feature — this documentation is for contributors and integrators.

Overview

GrowthBook provides feature flagging and A/B testing within Claude Code. It enables controlled rollouts of new features, experiment tracking, and targeted feature releases based on user segments. The SDK is initialized at startup and evaluates flags locally with caching for minimal latency.

Architecture

Feature Flags — Boolean, string, number, and JSON flag types evaluated locally
Experiments — Multi-variant A/B tests with statistical tracking and automatic analysis
Forced Variations — Override flag values for testing and debugging
Sticky Bucketing — Consistent user experience across sessions based on stable identifiers
Local Caching — Feature definitions cached after initial fetch for offline resilience

Usage in Code

Feature flags are accessed through a cached evaluation function. The cache may return slightly stale values within a session, which is acceptable for product experimentation:

// Check if a feature is enabled (cached evaluation)
const isEnabled = getFeatureValue_CACHED_MAY_BE_STALE(
  'tengu_plum_vx3',  // Feature key
  false                  // Default value
);

// Use the feature value
if (isEnabled) {
  // New behavior path
} else {
  // Control behavior path
}

Configuration

GrowthBook is configured via environment variables. The SDK connects to the GrowthBook API at startup to fetch the latest feature definitions, then caches them locally for offline operation:

Variable	Purpose
`GROWTHBOOK_API_HOST`	GrowthBook API endpoint (self-hosted or cloud)
`GROWTHBOOK_CLIENT_KEY`	Client-side SDK key for feature flag access
`GROWTHBOOK_FORCE_VARIATION`	Force a specific variation for testing

How It Works

On startup, Claude Code initializes the GrowthBook SDK with the configured API host and client key
Feature definitions are fetched from the GrowthBook server and cached locally
Throughout the session, code checks feature flags using the cached evaluation function
Flag values are re-evaluated periodically or on explicit refresh
Experiment results are tracked and sent back to GrowthBook for statistical analysis
Forced variations can override normal bucket assignment for testing

Best Practices

Always provide a sensible default value (usually false for features not yet released)
Use descriptive, unique feature keys (e.g., tengu_plum_vx3 follows the internal naming convention)
Cache-aware design: do not rely on flag values being real-time; they are evaluated at session start
Clean up stale flags after experiments conclude