# {{projectName}}

An AI chat microservice powered by [Runtime Web](https://runtimeweb.com/docs) with AWS Bedrock, DynamoDB session memory, and OpenSearch vector search (RAG).

## Features

- **AI Chat API**: POST `/chat` endpoint for conversational AI
- **AWS Bedrock Integration**: Claude model family support for AI responses
- **Session Memory**: DynamoDB-based conversation history persistence
- **Vector Search (RAG)**: OpenSearch KNN for retrieving relevant context
- **Serverless Architecture**: AWS Lambda with automatic scaling

## Getting Started

### Prerequisites

- Node.js >= 18.0.0
- AWS CLI configured with credentials
- Yarn or npm
- AWS account with Bedrock model access enabled

### Installation

```bash
npm install
# or
yarn install
```

### Environment Setup

Copy the example environment file and configure your values:

```bash
cp .env.example .env
```

Edit `.env` with your configuration (see [Environment Variables](#environment-variables) below).

### Development

Start the local development server:

```bash
npm run dev
# or
yarn dev
```

The application will be available at `http://localhost:3000`.

## Chat API Endpoint

### POST /chat

Send a message to the AI assistant and receive a response.

#### Request

```json
{
  "sessionId": "user-123-session-456",
  "message": "What is the capital of France?"
}
```

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `sessionId` | string | Yes | Unique identifier for the conversation session |
| `message` | string | Yes | User's message to send to the AI |

#### Response (200 OK)

```json
{
  "sessionId": "user-123-session-456",
  "response": "The capital of France is Paris.",
  "timestamp": "2024-01-15T10:30:00.000Z"
}
```

| Field | Type | Description |
|-------|------|-------------|
| `sessionId` | string | Echo of the session identifier |
| `response` | string | AI-generated response |
| `timestamp` | string | ISO 8601 timestamp of the response |

#### Error Responses

| Status | Error | Description |
|--------|-------|-------------|
| 400 | `sessionId is required` | Missing sessionId in request body |
| 400 | `message is required` | Missing message in request body |
| 400 | `Invalid JSON in request body` | Malformed JSON payload |
| 500 | `Internal server error` | Service error (check CloudWatch logs) |

#### Example Usage

```bash
curl -X POST https://your-api-endpoint/chat \
  -H "Content-Type: application/json" \
  -d '{
    "sessionId": "my-session-123",
    "message": "Hello, how can you help me today?"
  }'
```


## AWS Bedrock Configuration

### Model Selection

Configure the Bedrock model via the `BEDROCK_MODEL_ID` environment variable:

```bash
# In .env or runtime.config.ts
BEDROCK_MODEL_ID=anthropic.claude-3-sonnet-20240229-v1:0
```

### Available Claude Models

| Model ID | Description | Use Case |
|----------|-------------|----------|
| `anthropic.claude-3-sonnet-20240229-v1:0` | Balanced performance | General chat (default) |
| `anthropic.claude-3-haiku-20240307-v1:0` | Fast, cost-effective | High-volume, simple queries |
| `anthropic.claude-3-opus-20240229-v1:0` | Most capable | Complex reasoning tasks |

### Enabling Bedrock Access

1. Open the [AWS Bedrock Console](https://console.aws.amazon.com/bedrock/)
2. Navigate to **Model access** in the left sidebar
3. Click **Manage model access**
4. Enable access for the Claude models you want to use
5. Wait for access to be granted (usually instant)

### IAM Permissions

The Lambda execution role requires the following permission:

```json
{
  "Effect": "Allow",
  "Action": "bedrock:InvokeModel",
  "Resource": "*"
}
```

This is automatically configured by the CDK AiStack.

## DynamoDB Session Memory

### Schema

The session memory table stores conversation history with the following schema:

| Attribute | Type | Description |
|-----------|------|-------------|
| `sessionId` | String (PK) | Unique session identifier (partition key) |
| `messages` | List | Array of conversation messages |
| `createdAt` | String | ISO timestamp of session creation |
| `updatedAt` | String | ISO timestamp of last update |
| `ttl` | Number | Optional TTL for automatic expiration (Unix timestamp) |

### Message Structure

Each message in the `messages` array has the following structure:

```json
{
  "role": "user",
  "content": "What is the capital of France?",
  "timestamp": "2024-01-15T10:30:00.000Z"
}
```

| Field | Type | Values | Description |
|-------|------|--------|-------------|
| `role` | string | `user`, `assistant` | Message sender role |
| `content` | string | — | Message text content |
| `timestamp` | string | ISO 8601 | When the message was created |

### Table Configuration

The table is created by the CDK AiStack with:

- **Billing Mode**: Pay-per-request (on-demand)
- **TTL**: Optional automatic expiration via `ttl` attribute
- **Removal Policy**: `DESTROY` for dev/staging, `RETAIN` for production

### Environment Variable

```bash
MEMORY_TABLE_NAME=ai-chat-sessions
```

The table name is output by CDK after deployment.

## OpenSearch Vector Search

### Overview

OpenSearch provides vector similarity search (KNN) for the RAG (Retrieval-Augmented Generation) pattern. When a user sends a message, the service searches for semantically similar documents to provide relevant context to the AI.

### Index Mapping

Create an index with the following mapping for vector search:

```json
PUT /knowledge-base
{
  "mappings": {
    "properties": {
      "content": { "type": "text" },
      "embedding": {
        "type": "knn_vector",
        "dimension": 1536,
        "method": {
          "name": "hnsw",
          "space_type": "cosinesimil",
          "engine": "nmslib"
        }
      },
      "metadata": { "type": "object" }
    }
  },
  "settings": {
    "index": {
      "knn": true
    }
  }
}
```

### Populating the Knowledge Base

#### 1. Generate Embeddings

Use AWS Bedrock Titan Embeddings to convert your documents to vectors:

```typescript
import { BedrockRuntimeClient, InvokeModelCommand } from '@aws-sdk/client-bedrock-runtime';

const client = new BedrockRuntimeClient({});

async function getEmbedding(text: string): Promise<number[]> {
  const response = await client.send(new InvokeModelCommand({
    modelId: 'amazon.titan-embed-text-v1',
    contentType: 'application/json',
    accept: 'application/json',
    body: JSON.stringify({ inputText: text }),
  }));
  
  const result = JSON.parse(new TextDecoder().decode(response.body));
  return result.embedding;
}
```

#### 2. Index Documents

Index your documents with their embeddings:

```typescript
import { Client } from '@opensearch-project/opensearch';

const client = new Client({ node: process.env.OPENSEARCH_ENDPOINT });

async function indexDocument(id: string, content: string, metadata?: object) {
  const embedding = await getEmbedding(content);
  
  await client.index({
    index: 'knowledge-base',
    id,
    body: {
      content,
      embedding,
      metadata,
    },
  });
}

// Example: Index a document
await indexDocument('doc-1', 'Paris is the capital of France.', { source: 'geography' });
```

#### 3. Bulk Indexing

For large datasets, use bulk indexing:

```typescript
async function bulkIndex(documents: Array<{ id: string; content: string; metadata?: object }>) {
  const operations = [];
  
  for (const doc of documents) {
    const embedding = await getEmbedding(doc.content);
    operations.push({ index: { _index: 'knowledge-base', _id: doc.id } });
    operations.push({ content: doc.content, embedding, metadata: doc.metadata });
  }
  
  await client.bulk({ body: operations });
}
```

### Environment Variables

```bash
# OpenSearch domain endpoint (output by CDK)
OPENSEARCH_ENDPOINT=https://search-your-domain.us-east-1.es.amazonaws.com

# Index name (optional, defaults to 'knowledge-base')
OPENSEARCH_INDEX=knowledge-base
```

### Graceful Degradation

If OpenSearch is unavailable or not configured, the chat service continues to work without RAG context. A warning is logged, and the AI responds based on conversation history alone.


## Environment Variables

| Variable | Required | Default | Description |
|----------|----------|---------|-------------|
| `AWS_REGION` | Yes | `us-east-1` | AWS region for all services |
| `BEDROCK_MODEL_ID` | Yes | `anthropic.claude-3-sonnet-20240229-v1:0` | Bedrock model ID |
| `MEMORY_TABLE_NAME` | Yes | `ai-chat-sessions` | DynamoDB table name |
| `OPENSEARCH_ENDPOINT` | No* | — | OpenSearch domain endpoint |
| `OPENSEARCH_INDEX` | No | `knowledge-base` | OpenSearch index name |

*OpenSearch is optional. Without it, RAG functionality is disabled but chat still works.

## Deployment

### Prerequisites

1. AWS CLI configured with appropriate credentials
2. CDK bootstrapped in your AWS account:
   ```bash
   npx cdk bootstrap aws://ACCOUNT-ID/REGION
   ```
3. Bedrock model access enabled (see [Bedrock Configuration](#aws-bedrock-configuration))

### Deploy to Development

```bash
npm run deploy:dev
# or
yarn deploy:dev
```

This deploys with:
- DynamoDB table with `DESTROY` removal policy
- OpenSearch domain (t3.small.search instance)
- Lambda functions with Bedrock/DynamoDB/OpenSearch permissions

### Deploy to Staging

```bash
npm run deploy:staging
# or
yarn deploy:staging
```

Same as dev but with staging-specific configuration from `runtime.config.ts`.

### Deploy to Production

```bash
npm run deploy:prod
# or
yarn deploy:prod
```

Production deployment includes:
- DynamoDB table with `RETAIN` removal policy (data preserved on stack deletion)
- OpenSearch domain with `RETAIN` removal policy
- Consider using larger OpenSearch instance types for production workloads

### Post-Deployment Steps

After deployment, CDK outputs the resource endpoints:

```
Outputs:
{{projectName}}-AiStack-dev.SessionTableName = ai-chat-sessions-dev
{{projectName}}-AiStack-dev.SearchDomainEndpoint = https://search-xxx.us-east-1.es.amazonaws.com
```

Update your `.env` or `runtime.config.ts` with these values:

```bash
MEMORY_TABLE_NAME=ai-chat-sessions-dev
OPENSEARCH_ENDPOINT=https://search-xxx.us-east-1.es.amazonaws.com
```

### Deployment Stages

| Stage | Use Case | Data Retention | Instance Sizes |
|-------|----------|----------------|----------------|
| `dev` | Development/testing | Destroyed on stack delete | Minimal |
| `staging` | Pre-production testing | Destroyed on stack delete | Minimal |
| `prod` | Production | Retained on stack delete | Scale as needed |

## Project Structure

```
.
├── src/
│   ├── index.tsx                # Application entry point
│   ├── App.tsx                  # Root React component
│   ├── routes.tsx               # Route configuration
│   ├── handlers/                # Lambda function handlers
│   │   └── chat.handlers.ts     # Chat API handler
│   ├── services/                # Service implementations
│   │   ├── bedrock.service.ts   # AWS Bedrock integration
│   │   ├── memory.service.ts    # DynamoDB session memory
│   │   └── search.service.ts    # OpenSearch vector search
│   ├── ties/                    # Dependency injection
│   │   ├── ai.ties.ts           # AI services registration
│   │   └── index.ts             # Ties barrel export
│   └── types/                   # TypeScript type definitions
│       └── chat.types.ts        # Chat-related types
├── public/
│   └── index.html               # HTML template
├── cdk/                         # CDK infrastructure (generated)
│   ├── bin/
│   │   └── app.ts               # CDK app entry point
│   └── lib/
│       ├── ai-stack.ts          # AI resources (DynamoDB, OpenSearch)
│       ├── infra-stack.ts       # Base infrastructure
│       └── web-stack.ts         # Lambda and API Gateway
├── .env.example                 # Environment variable template
├── runtime.config.ts            # Runtime framework configuration
├── tsconfig.json                # TypeScript configuration
└── package.json                 # Project dependencies
```

## Configuration

Edit `runtime.config.ts` to customize:

- **Project name**: Used for AWS resource naming
- **Deployment stages**: dev, staging, prod configurations
- **AWS regions and profiles**: Per-stage AWS settings
- **Environment variables**: Per-stage service configuration
- **Feature flags**: Enable/disable AI features

### Feature Flags

```typescript
features: {
  enableAiChat: true,        // Enable chat endpoint
  enableSessionMemory: true, // Enable DynamoDB persistence
  enableVectorSearch: true,  // Enable OpenSearch RAG
  enableWarmup: true,        // Reduce Lambda cold starts
}
```

## Troubleshooting

### "Access denied to Bedrock model"

1. Verify Bedrock model access is enabled in the AWS Console
2. Check Lambda execution role has `bedrock:InvokeModel` permission
3. Ensure the model ID is correct and available in your region

### "OpenSearch query failed"

1. Verify `OPENSEARCH_ENDPOINT` is correctly set
2. Check Lambda execution role has OpenSearch permissions
3. Ensure the index exists and has the correct mapping
4. The service degrades gracefully—chat works without search

### "Session not found"

This is normal for new sessions. The service creates sessions automatically on first message.

### Cold Start Latency

Enable Lambda warmup in `runtime.config.ts`:

```typescript
features: {
  enableWarmup: true,
}
```

## Learn More

- [Runtime Web Documentation](https://runtimeweb.com/docs)
- [AWS Bedrock User Guide](https://docs.aws.amazon.com/bedrock/)
- [Amazon DynamoDB Developer Guide](https://docs.aws.amazon.com/dynamodb/)
- [Amazon OpenSearch Service](https://docs.aws.amazon.com/opensearch-service/)
- [AWS CDK Documentation](https://docs.aws.amazon.com/cdk/)

## License

Elastic License 2.0
