{"_id":"@brightly/pg-hybrid-search","name":"@brightly/pg-hybrid-search","dist-tags":{"beta":"0.5.0-beta","latest":"0.5.0-beta"},"versions":{"0.5.0-beta":{"name":"@brightly/pg-hybrid-search","version":"0.5.0-beta","description":"Hybrid search toolkit for Postgres (pgvector + BM25 + rerank)","type":"module","main":"dist/src/index.js","types":"dist/src/index.d.ts","bin":{"pg-hybrid":"dist/src/cli.js","pg-hybrid-search":"dist/src/cli.js"},"scripts":{"build":"rm -rf dist && tsc && mkdir -p dist/src && cp -r src/sql dist/src/sql","example":"npm run build && node dist/example/basic.js","test":"jest","test:watch":"jest --watch","test:coverage":"jest --coverage","prepublishOnly":"npm run build"},"keywords":["postgres","postgresql","pgvector","vector-search","hybrid-search","embedding","full-text-search","bm25","rerank"],"author":{"name":"Brightly Virya"},"license":"MIT","dependencies":{"pg":"^8.12.0","dotenv":"^16.4.5","commander":"^12.1.0"},"devDependencies":{"@types/node":"^22.5.0","@types/pg":"^8.11.6","@types/jest":"^29.5.12","jest":"^29.7.0","ts-jest":"^29.1.2","typescript":"^5.5.4"},"engines":{"node":">=18.0.0"},"repository":{"type":"git","url":"git+https://github.com/Brightlyviryaa/pg-hybrid-search.git"},"bugs":{"url":"https://github.com/Brightlyviryaa/pg-hybrid-search/issues"},"homepage":"https://github.com/Brightlyviryaa/pg-hybrid-search#readme","_id":"@brightly/pg-hybrid-search@0.5.0-beta","gitHead":"1ec915b353fce49b6c95a2cefce1100630b64822","_nodeVersion":"20.15.1","_npmVersion":"10.7.0","dist":{"integrity":"sha512-N55U3a034Ucmy627x9r6ULc/J6o2M66PVedpDeZutlsgK0Qt2W1oUzOBEZI+DPincYP1jdCnaH2WFckZJstw0A==","shasum":"80a7d935188e38dbc01eb582267dbbe3a9f97ad6","tarball":"https://registry.npmjs.org/@brightly/pg-hybrid-search/-/pg-hybrid-search-0.5.0-beta.tgz","fileCount":33,"unpackedSize":60901,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEUCIQCPE+xlKfnEnsM4Lbo98ME68KxwRxmliZyLqF7747MHvQIgAgjLd1s47F+VgANBIPZSO4Ep1vTMGQL9ccbma/pFUK4="}]},"_npmUser":{"name":"brightly","email":"virya.brightly@gmail.com"},"directories":{},"maintainers":[{"name":"brightly","email":"virya.brightly@gmail.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/pg-hybrid-search_0.5.0-beta_1756714596520_0.715371670375049"},"_hasShrinkwrap":false}},"time":{"created":"2025-09-01T08:16:36.391Z","0.5.0-beta":"2025-09-01T08:16:36.744Z","modified":"2025-09-01T08:16:37.064Z"},"maintainers":[{"name":"brightly","email":"virya.brightly@gmail.com"}],"description":"Hybrid search toolkit for Postgres (pgvector + BM25 + rerank)","homepage":"https://github.com/Brightlyviryaa/pg-hybrid-search#readme","keywords":["postgres","postgresql","pgvector","vector-search","hybrid-search","embedding","full-text-search","bm25","rerank"],"repository":{"type":"git","url":"git+https://github.com/Brightlyviryaa/pg-hybrid-search.git"},"author":{"name":"Brightly Virya"},"bugs":{"url":"https://github.com/Brightlyviryaa/pg-hybrid-search/issues"},"license":"MIT","readme":"<div align=\"center\">\n  <img src=\"src/image/PG-HYBRID-SEARCH-LOGO.png\" alt=\"PG Hybrid Search Logo\" width=\"200\" height=\"200\">\n\n# pg-hybrid-search\n\nVersion: v0.5.0 (Beta) — Open for feedback\n\n**🚀 Advanced Hybrid Search Toolkit for PostgreSQL**\n\n*Seamlessly combine vector similarity, BM25 full-text search, and AI-powered reranking*\n\n[![npm version](https://badge.fury.io/js/%40brightly%2Fpg-hybrid-search.svg)](https://badge.fury.io/js/%40brightly%2Fpg-hybrid-search)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![TypeScript](https://img.shields.io/badge/%3C%2F%3E-TypeScript-%230074c1.svg)](http://www.typescriptlang.org/)\n[![Node.js Version](https://img.shields.io/badge/node-%3E%3D18.0.0-brightgreen)](https://nodejs.org/)\n\n[![GitHub stars](https://img.shields.io/github/stars/Brightlyviryaa/pg-hybrid-search?style=social)](https://github.com/Brightlyviryaa/pg-hybrid-search/stargazers)\n[![GitHub forks](https://img.shields.io/github/forks/Brightlyviryaa/pg-hybrid-search?style=social)](https://github.com/Brightlyviryaa/pg-hybrid-search/network/members)\n[![GitHub issues](https://img.shields.io/github/issues/Brightlyviryaa/pg-hybrid-search)](https://github.com/Brightlyviryaa/pg-hybrid-search/issues)\n[![GitHub pull requests](https://img.shields.io/github/issues-pr/Brightlyviryaa/pg-hybrid-search)](https://github.com/Brightlyviryaa/pg-hybrid-search/pulls)\n\n---\n\n</div>\n\n## 📋 Table of Contents\n\n- [🌟 Overview](#-overview)\n- [✨ Key Features](#-key-features)\n- [🚀 Quick Start](#-quick-start)\n- [📦 Installation](#-installation)\n- [⚙️ Configuration](#️-configuration)\n- [📚 API Reference](#-api-reference)\n- [📖 Complete API Documentation](API_DOCUMENTATION.md) ⭐\n- [🖥 CLI Tools](#-cli-tools)\n- [🆕 What's New (v0.5.0 Beta)](#-whats-new-v050-beta)\n- [💡 Usage Examples](#-usage-examples)\n- [🏗 Database Schema](#-database-schema)\n- [⚡ Performance Optimization](#-performance-optimization)\n- [🔧 Development](#-development)\n- [🤝 Contributing](#-contributing)\n- [📄 License](#-license)\n\n## 🌟 Overview\n\n**pg-hybrid-search** is a powerful, production-ready library that brings advanced search capabilities to PostgreSQL applications. Combining the precision of vector similarity search with the versatility of full-text search and the intelligence of AI-powered reranking.\n\n### Why Choose pg-hybrid-search?\n\n- 🎯 **Best of Both Worlds**: Vector similarity + BM25 full-text search\n- 🤖 **AI-Enhanced**: Optional reranking with Voyage AI for superior relevance\n- 🚀 **Performance Focused**: Optimized queries and connection pooling\n- 🛡️ **Type Safe**: Full TypeScript support with comprehensive types\n- 🔧 **Developer Friendly**: Simple CLI tools and intuitive API\n- 📈 **Production Ready**: Battle-tested in real-world applications\n\n## ✨ Key Features\n\n<table>\n<tr>\n<td width=\"50%\">\n\n### 🔍 **Advanced Search Capabilities**\n- **Vector Search**: Cosine similarity using OpenAI embeddings\n- **Full-text Search**: PostgreSQL's powerful BM25 algorithm\n- **Hybrid Search**: Intelligent combination with custom weights\n- **AI Reranking**: Voyage Rerank v2 integration\n- **Multi-Index Support**: Isolated document collections for different use cases\n\n</td>\n<td width=\"50%\">\n\n### 🛠️ **Developer Experience**\n- **TypeScript First**: Complete type safety & IntelliSense\n- **CLI Tools**: Easy schema management\n- **Flexible API**: Simple yet powerful functions\n- **Well Documented**: Comprehensive guides & examples\n\n</td>\n</tr>\n</table>\n\n## 🚀 Quick Start\n\n### RAG Search with AI SDKs (Anthropic/OpenAI)\n\nPerfect for building AI assistants with retrieval-augmented generation:\n\n#### With Anthropic Claude SDK\n\n```typescript\nimport { createClient } from '@brightly/pg-hybrid-search';\nimport Anthropic from '@anthropic-ai/sdk';\n\nconst client = createClient();\nconst knowledge = client.index(\"knowledge-base\");\nconst anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });\n\n// 1. Setup search tool for Claude\nconst searchTool = {\n  name: \"search_knowledge\",\n  description: \"Search the knowledge base for relevant information\",\n  input_schema: {\n    type: \"object\",\n    properties: {\n      query: { type: \"string\", description: \"Search query\" },\n      limit: { type: \"number\", description: \"Number of results\", default: 5 }\n    },\n    required: [\"query\"]\n  }\n};\n\n// 2. RAG-powered chat function\nasync function ragChat(userMessage: string) {\n  const message = await anthropic.messages.create({\n    model: \"claude-3-5-sonnet-20241022\",\n    max_tokens: 1000,\n    tools: [searchTool],\n    messages: [{\n      role: \"user\",\n      content: userMessage\n    }]\n  });\n\n  // Handle tool calls\n  if (message.content[0].type === 'tool_use') {\n    const toolCall = message.content[0];\n    \n    // Execute search\n    const searchResults = await knowledge.search({\n      query: toolCall.input.query,\n      limit: toolCall.input.limit || 5,\n      reranking: true\n    });\n\n    // Continue conversation with search results\n    const followUp = await anthropic.messages.create({\n      model: \"claude-3-5-sonnet-20241022\",\n      max_tokens: 1000,\n      messages: [\n        { role: \"user\", content: userMessage },\n        { role: \"assistant\", content: message.content },\n        {\n          role: \"user\",\n          content: [{\n            type: \"tool_result\",\n            tool_use_id: toolCall.id,\n            content: JSON.stringify(searchResults.map(r => r.raw_content))\n          }]\n        }\n      ]\n    });\n\n    return followUp.content[0].text;\n  }\n\n  return message.content[0].text;\n}\n\n// Usage\nconst response = await ragChat(\"What are the latest developments in AI?\");\nconsole.log(response);\n```\n\n#### With OpenAI SDK\n\n```typescript\nimport { createClient } from '@brightly/pg-hybrid-search';\nimport OpenAI from 'openai';\n\nconst client = createClient();\nconst docs = client.index(\"documentation\");\nconst openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });\n\n// 1. Define search function for OpenAI\nconst searchFunction = {\n  name: \"search_docs\",\n  description: \"Search documentation for relevant information\",\n  parameters: {\n    type: \"object\",\n    properties: {\n      query: { type: \"string\", description: \"Search query\" },\n      limit: { type: \"number\", description: \"Number of results\", default: 3 }\n    },\n    required: [\"query\"]\n  }\n};\n\n// 2. RAG chat with function calling\nasync function ragChatGPT(userMessage: string) {\n  const completion = await openai.chat.completions.create({\n    model: \"gpt-4\",\n    messages: [{ role: \"user\", content: userMessage }],\n    functions: [searchFunction],\n    function_call: \"auto\"\n  });\n\n  const message = completion.choices[0].message;\n\n  if (message.function_call) {\n    // Execute search\n    const args = JSON.parse(message.function_call.arguments);\n    const searchResults = await docs.search({\n      query: args.query,\n      limit: args.limit,\n      reranking: true\n    });\n\n    // Continue with search results\n    const followUp = await openai.chat.completions.create({\n      model: \"gpt-4\",\n      messages: [\n        { role: \"user\", content: userMessage },\n        message,\n        {\n          role: \"function\",\n          name: \"search_docs\",\n          content: JSON.stringify(searchResults.map(r => r.raw_content))\n        }\n      ]\n    });\n\n    return followUp.choices[0].message.content;\n  }\n\n  return message.content;\n}\n\n// Usage\nconst answer = await ragChatGPT(\"How do I implement vector search?\");\nconsole.log(answer);\n```\n\n### Basic Library Usage\n\n#### Modern Client API (Recommended)\n\n```typescript\nimport { createClient } from '@brightly/pg-hybrid-search';\n\nconst client = createClient();\n\n// 1. Insert documents with automatic embedding generation\nawait client.index(\"documents\").add(\"Machine learning revolutionizes data analysis\");\nawait client.index(\"documents\").add(\"PostgreSQL provides excellent full-text search\");\n\n// 2. AI-powered semantic search with reranking\nconst results = await client.index(\"documents\").search({\n  query: \"AI data analysis\", \n  limit: 5,\n  reranking: true\n});\n\n// 3. Results include relevance scores and original content\nresults.forEach(result => {\n  console.log(`Score: ${result.hybrid_score || result.rerank_score}`);\n  console.log(`Content: ${result.raw_content}`);\n});\n```\n\n#### Simplified Functional API (Also Available)\n\n```typescript\nimport { add, search } from '@brightly/pg-hybrid-search';\n\nawait add(\"Machine learning revolutionizes data analysis\");\nconst results = await search({ query: \"AI data analysis\", limit: 5 });\n```\n\n## 📦 Installation\n\n```bash\n# Install the package\nnpm install @brightly/pg-hybrid-search\n\n# Initialize database schema\nnpx @brightly/pg-hybrid-search init\n```\n\n### Prerequisites\n\n| Requirement | Version | Purpose |\n|-------------|---------|---------|\n| **Node.js** | ≥18.0.0 | Runtime environment |\n| **PostgreSQL** | ≥15.0.0 | Database with pgvector support |\n| **pgvector** | Latest | Vector similarity operations |\n| **OpenAI API** | - | Embedding generation |\n| **Voyage AI API** | - | Reranking (optional) |\n\n## ⚙️ Configuration\n\n### Environment Variables\n\nCreate a `.env` file in your project root:\n\n```env\n# Database Connection (Required)\nDATABASE_URL=postgresql://username:password@localhost:5432/your_database\n\n# OpenAI Configuration (Required)\nOPENAI_API_KEY=sk-your-openai-api-key-here\nEMBED_MODEL=text-embedding-3-small  # Optional: default model\n\n# Voyage AI Configuration (Optional - for reranking)\nVOYAGE_API_KEY=pa-your-voyage-api-key-here\nRERANK_MODEL=rerank-2.5-lite  # Optional: default rerank model\n```\n\n### Database Setup\n\n```bash\n# One-time schema initialization\nnpx @brightly/pg-hybrid-search init\n\n# Verify installation\npsql -d your_database -c \"SELECT COUNT(*) FROM vector_table;\"\n```\n\n## 📚 API Reference\n\n> 📖 **[View Complete API Documentation →](API_DOCUMENTATION.md)**  \n> *Comprehensive guide with sequence diagrams, payload examples, and detailed usage patterns*\n\n### Modern Client API (Recommended)\n\n#### Creating a Client\n\n```typescript\nimport { createClient } from '@brightly/pg-hybrid-search';\n\nconst client = createClient();\n```\n\n#### Index Operations\n\n```typescript\n// Get an index reference\nconst index = client.index(\"your-index-name\");\n\n// Add documents\n// Optional: set language per document (improves BM25 in multilingual apps)\nconst documentId = await index.add(\"Your document content\", \"indonesian\");\n\n// Remove documents  \nawait index.remove(documentId);\n\n// Destroy an index (delete all rows with this index name)\nconst deleted = await index.destroy();\nconsole.log(`Index cleared: ${deleted} rows removed`);\n```\n\n#### Search Operations\n\n```typescript\n// Hybrid search (default)\nconst results = await index.search({\n  query: \"your search query\",\n  limit: 10,                // Optional: number of results (default: 10)\n  reranking: true,          // Optional: enable AI reranking (default: false)\n  weights: {                // Optional: custom hybrid weights\n    vectorW: 0.7,\n    textW: 0.3\n  },\n  topNForRerank: 50        // Optional: candidates for reranking (default: 50)\n});\n\n// Pure vector search\nconst vectorResults = await index.search({\n  query: \"your search query\",\n  limit: 10,\n  vectorOnly: true         // Enable vector-only mode\n});\n```\n\n#### Search Options Interface\n\n```typescript\ninterface ClientSearchOptions {\n  query: string;           // Search query text\n  limit?: number;          // Number of results to return\n  reranking?: boolean;     // Enable AI-powered reranking\n  vectorOnly?: boolean;    // Use vector search only (no BM25)\n  weights?: SearchWeights; // Custom scoring weights\n  topNForRerank?: number;  // Candidates to consider for reranking\n}\n\ninterface SearchWeights {\n  vectorW: number;         // Vector search weight (0-1)\n  textW: number;           // Text search weight (0-1)\n}\n```\n\n---\n\n<!-- Simplified/legacy functional API removed in v0.5.0-beta. Use Modern Client API. -->\n\n### Type Definitions\n\n```typescript\ninterface SearchResult {\n  id: string;\n  raw_content: string;\n  cosine_sim?: number;        // Vector similarity score\n  ts_score?: number;          // BM25 text search score  \n  hybrid_score?: number;      // Combined normalized score\n  rerank_score?: number;      // AI reranking score\n  created_at?: string;\n  updated_at?: string;\n}\n\ninterface HybridWeights {\n  vectorW: number;            // Vector search weight (0-1)\n  textW: number;              // Text search weight (0-1)\n}\n\n```\n\n## 🖥 CLI Tools\n\nThe CLI provides essential database management commands:\n\n```bash\n# Initialize database schema (safe to run multiple times)\nnpx @brightly/pg-hybrid-search init\n\n# Reset schema (⚠️ destructive - requires confirmation)\nnpx @brightly/pg-hybrid-search reset -y\n\n# Show help\nnpx @brightly/pg-hybrid-search help\n```\n\n### CLI Command Reference\n\n| Command | Description | Usage |\n|---------|-------------|-------|\n| `init` | Creates tables, indexes, and triggers | `pg-hybrid init` |\n| `reset -y` | ⚠️ Drops all tables and indexes | `pg-hybrid reset -y` |\n| `help` | Shows command documentation | `pg-hybrid help` |\n\n## 💡 Usage Examples\n\n### Modern Client API Examples\n\n#### Basic Document Management\n\n```typescript\nimport { createClient } from '@brightly/pg-hybrid-search';\n\nconst client = createClient();\nconst movies = client.index(\"movies\");\n\n// Insert documents\nconst docIds = await Promise.all([\n  movies.add(\"Star Wars: A space opera epic with Jedi knights\"),\n  movies.add(\"Blade Runner: Cyberpunk dystopian future with replicants\"),\n  movies.add(\"The Matrix: Virtual reality and artificial intelligence thriller\")\n]);\n\n// AI-powered semantic search with reranking\nconst results = await movies.search({\n  query: \"space opera with jedi\",\n  limit: 5,\n  reranking: true\n});\n\nconsole.log(`Found ${results.length} relevant movies`);\n\n// Clean up\nawait Promise.all(docIds.map(id => movies.remove(id)));\n```\n\n#### Advanced Search Scenarios\n\n```typescript\n// Semantic-focused search with custom weights\nconst semanticResults = await movies.search({\n  query: \"futuristic AI rebellion\",\n  limit: 10,\n  weights: { vectorW: 0.9, textW: 0.1 }\n});\n\n// Keyword-focused search\nconst keywordResults = await movies.search({\n  query: \"cyberpunk dystopian\",\n  limit: 10,\n  weights: { vectorW: 0.2, textW: 0.8 }\n});\n\n// High-precision search with reranking\nconst precisionResults = await movies.search({\n  query: \"epic space battles with lightsabers\",\n  limit: 3,\n  reranking: true,\n  topNForRerank: 20\n});\n```\n\n#### Pure Vector Search\n\n```typescript\n// Vector similarity only\nconst vectorResults = await movies.search({\n  query: \"heroic journey in space\",\n  limit: 5,\n  vectorOnly: true\n});\n```\n\n### Simplified Functional API Examples\n\n#### Basic Document Management\n\n```typescript\nimport { add, search, remove } from '@brightly/pg-hybrid-search';\n\n// Insert documents\nconst docIds = await Promise.all([\n  add(\"PostgreSQL is a powerful relational database\"),\n  add(\"Vector databases enable semantic search capabilities\"),\n  add(\"Full-text search provides keyword-based retrieval\")\n]);\n\n// Search with hybrid approach (default)\nconst results = await search({\n  query: \"database search capabilities\",\n  limit: 3\n});\nconsole.log(`Found ${results.length} relevant documents`);\n\n// Clean up\nawait Promise.all(docIds.map(id => remove(id)));\n```\n\n### Advanced Search Strategies\n\n```typescript\n// Semantic-focused search (higher vector weight)\nconst semanticResults = await search({\n  query: \"AI innovation\",\n  limit: 10,\n  weights: { vectorW: 0.9, textW: 0.1 }\n});\n\n// Keyword-focused search (higher text weight)\nconst keywordResults = await search({\n  query: \"machine learning\",\n  limit: 10,\n  weights: { vectorW: 0.2, textW: 0.8 }\n});\n\n// Vector-only search\nconst vectorResults = await search({\n  query: \"artificial intelligence\",\n  vectorOnly: true\n});\n```\n\n### Enterprise Pipeline with Reranking\n\n```typescript\nimport { createClient } from '@brightly/pg-hybrid-search';\n\nasync function enterpriseSearch(query: string) {\n  const client = createClient();\n  const knowledge = client.index('knowledge');\n  // High-precision search with AI reranking\n  const results = await knowledge.search({\n    query,\n    limit: 15,\n    reranking: true,\n    topNForRerank: 100\n  });\n  return results.map((result, index) => ({\n    rank: index + 1,\n    id: result.id,\n    content: result.raw_content,\n    relevanceScore: result.rerank_score ?? result.hybrid_score,\n    confidence: result.rerank_score ? 'high' : 'medium'\n  }));\n}\n\nconst enterpriseResults = await enterpriseSearch('sustainable technology solutions');\n```\n\n### Batch Operations\n\n```typescript\nimport { createClient } from '@brightly/pg-hybrid-search';\n\n// Efficient bulk insertion using Modern Client API\nconst client = createClient();\nconst docs = client.index('bulk');\n\nconst documents = [\n  'Document 1 content...',\n  'Document 2 content...',\n  'Document 3 content...'\n];\n\nconst ids = await Promise.all(documents.map(text => docs.add(text, 'english')));\nconsole.log(`Inserted ${ids.length} documents`);\n```\n\n## 🏗 Database Schema\n\nThe library automatically creates and manages the following schema (v0.5.0 Beta):\n\n```sql\n-- Main table for storing documents and embeddings with multi-index support\nCREATE TABLE vector_table (\n  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),\n  index_name TEXT NOT NULL DEFAULT 'default',\n  raw_content TEXT NOT NULL,\n  lang TEXT NOT NULL DEFAULT 'simple',\n  embedding VECTOR(1536) NOT NULL,\n  content_tsv TSVECTOR,\n  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),\n  updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()\n);\n\n-- Optimized indexes for performance\nCREATE INDEX idx_vector_table_embedding \n  ON vector_table USING ivfflat (embedding vector_cosine_ops) \n  WITH (lists = 100);\n\nCREATE INDEX idx_vector_table_tsv \n  ON vector_table USING GIN (content_tsv);\n\nCREATE INDEX idx_vector_table_index_name\n  ON vector_table (index_name);\n\n-- Keep timestamps fresh\nCREATE OR REPLACE FUNCTION set_updated_at_pg_hybrid() RETURNS TRIGGER AS $$\nBEGIN\n  NEW.updated_at = NOW();\n  RETURN NEW;\nEND;\n$$ LANGUAGE plpgsql;\n\nDROP TRIGGER IF EXISTS trg_set_updated_at_pg_hybrid ON vector_table;\nCREATE TRIGGER trg_set_updated_at_pg_hybrid BEFORE UPDATE ON vector_table\n  FOR EACH ROW EXECUTE FUNCTION set_updated_at_pg_hybrid();\n\n-- Maintain multilingual TSV per-row\nCREATE OR REPLACE FUNCTION update_content_tsv_pg_hybrid() RETURNS TRIGGER AS $$\nBEGIN\n  NEW.content_tsv := to_tsvector(pg_hybrid_safe_regconfig(NEW.lang), coalesce(NEW.raw_content, ''));\n  RETURN NEW;\nEND;\n$$ LANGUAGE plpgsql;\n\nDROP TRIGGER IF EXISTS trg_update_content_tsv_pg_hybrid ON vector_table;\nCREATE TRIGGER trg_update_content_tsv_pg_hybrid\n  BEFORE INSERT OR UPDATE ON vector_table\n  FOR EACH ROW EXECUTE FUNCTION update_content_tsv_pg_hybrid();\n```\n\n### Schema Features\n\n- **UUID Primary Keys**: Globally unique identifiers\n- **Multi-Index Support**: Isolated document collections with `index_name` field\n- **Vector Storage**: 1536-dimensional embeddings (OpenAI standard)\n- **Multilingual TSVector**: Per-row language via `lang` with GIN index\n- **Timestamps**: Automatic creation and update tracking\n- **Optimized Indexes**: IVFFlat for vectors, GIN for text search, B-tree for index names\n\n### Upgrade Guide (pre-0.5.0 → 0.5.0)\n\nIf you already have data, run a safe migration:\n\n```sql\nALTER TABLE vector_table ADD COLUMN IF NOT EXISTS lang TEXT NOT NULL DEFAULT 'simple';\n\n-- Recreate content_tsv as a regular column if it was generated\nDROP INDEX IF EXISTS idx_vector_table_tsv;\nALTER TABLE vector_table DROP COLUMN IF EXISTS content_tsv;\nALTER TABLE vector_table ADD COLUMN content_tsv TSVECTOR;\n\n-- Recreate trigger\n-- (Use the function definitions shown in the schema above)\n```\n\nRe-run CLI init (idempotent) to ensure functions/triggers exist:\n\n```bash\nnpx @brightly/pg-hybrid-search init\n```\n\n### Query Notes\n\n- Hybrid search computes text score with: `websearch_to_tsquery(pg_hybrid_safe_regconfig(lang), $query)`\n- Provide `lang` at insert time for better BM25 (e.g., `'english'`, `'indonesian'`)\n\n## ⚡ Performance Optimization\n\n### Vector Index Tuning\n\n```sql\n-- Adjust lists parameter based on your dataset size\n-- Rule of thumb: lists = sqrt(total_rows)\n\n-- For datasets < 10K documents\nCREATE INDEX CONCURRENTLY idx_vector_small \n  ON vector_table USING ivfflat (embedding vector_cosine_ops) \n  WITH (lists = 50);\n\n-- For datasets > 100K documents  \nCREATE INDEX CONCURRENTLY idx_vector_large\n  ON vector_table USING ivfflat (embedding vector_cosine_ops) \n  WITH (lists = 500);\n```\n\n### Connection Optimization\n\n```typescript\n// The library uses connection pooling by default\n// You can access the pool for advanced configuration\n\nimport { pool } from '@brightly/pg-hybrid-search';\n\n// Monitor pool status\nsetInterval(() => {\n  console.log(`Active connections: ${pool.totalCount}`);\n  console.log(`Idle connections: ${pool.idleCount}`);\n}, 30000);\n```\n\n### Search Performance Tips\n\n1. **Batch Similar Queries**: Group related searches to amortize embedding costs\n2. **Tune Hybrid Weights**: Adjust based on your content and query patterns  \n3. **Optimize Rerank Usage**: Use `topNForRerank` between 50-200 for best balance\n4. **Monitor Query Performance**: Use `EXPLAIN ANALYZE` for slow queries\n\n### Debugging\n\n- Set `PG_HYBRID_DEBUG=1` or `PG_HYBRID_DEBUG_RERANK=1` to print light rerank diagnostics\n  - Shows selected rerank model and usage payload when available\n  - Useful for tuning `topNForRerank` and verifying model selection\n\n### Example App (Seeding + Isolation)\n\n```bash\n# Run the bundled example that seeds three indexes (A/B/C)\nnpm run example\n\n# Shows:\n# - Per-index seeding with optional language\n# - Isolation checks (searching A won’t return B/C)\n# - Rerank demo with topNForRerank=50\n# - Hybrid weights customization\n```\n\n## 🔧 Development\n\n### Local Development Setup\n\n```bash\n# Clone the repository\ngit clone https://github.com/Brightlyviryaa/pg-hybrid-search.git\ncd pg-hybrid-search\n\n# Install dependencies\nnpm install\n\n# Build the project\nnpm run build\n\n# Set up test environment\ncp .env.example .env\n# Edit .env with your database credentials\n\n# Initialize test database\nnpm run build && node dist/src/cli.js init\n```\n\n### Project Structure\n\n```\npg-hybrid-search/\n├── src/\n│   ├── cli.ts              # Command-line interface\n│   ├── db.ts               # Database connection management\n│   ├── embedding.ts        # OpenAI embedding integration  \n│   ├── search.ts           # Core search functionality\n│   ├── rerank.ts           # Voyage AI reranking\n│   ├── index.ts            # Main exports\n│   ├── sql/\n│   │   ├── init.sql        # Schema initialization\n│   │   └── reset.sql       # Schema cleanup\n│   └── image/\n│       └── PG-HYBRID-SEARCH-LOGO.png\n├── dist/                   # Compiled JavaScript\n├── package.json\n├── tsconfig.json\n└── README.md\n```\n\n### Building and Testing\n\n```bash\n# Development build\nnpm run build\n\n# Test CLI functionality\nnpm run build && node dist/src/cli.js init\n\n# Test basic operations (requires test database)\nnode -e \"\nconst { upsertDocument, searchHybrid } = require('./dist/index.js');\nupsertDocument('test document').then(id => \n  searchHybrid('test', 1).then(console.log)\n);\n\"\n```\n\n## 🤝 Contributing\n\nWe welcome contributions from the community! Here's how you can help:\n\n### Ways to Contribute\n\n- 🐛 **Bug Reports**: Found an issue? [Create an issue](https://github.com/Brightlyviryaa/pg-hybrid-search/issues/new?template=bug_report.md)\n- ✨ **Feature Requests**: Have an idea? [Submit a feature request](https://github.com/Brightlyviryaa/pg-hybrid-search/issues/new?template=feature_request.md)\n- 📝 **Documentation**: Improve docs, add examples, fix typos\n- 🔧 **Code**: Submit pull requests for bug fixes or new features\n\n### Development Workflow\n\n1. **Fork** the repository\n2. **Create** a feature branch: `git checkout -b feature/amazing-feature`\n3. **Commit** your changes: `git commit -m 'Add amazing feature'`\n4. **Push** to your branch: `git push origin feature/amazing-feature`  \n5. **Submit** a Pull Request\n\n### Code Standards\n\n- Follow existing TypeScript patterns\n- Add tests for new functionality\n- Update documentation for API changes\n- Ensure backward compatibility when possible\n\n## 🆘 Support & Community\n\nNeed help or want to connect with other users?\n\n- 📖 **Documentation**: You're reading it! Check the [API Reference](#-api-reference)\n- 🐛 **Issues**: [GitHub Issues](https://github.com/Brightlyviryaa/pg-hybrid-search/issues) for bugs and feature requests\n- 💬 **Discussions**: [GitHub Discussions](https://github.com/Brightlyviryaa/pg-hybrid-search/discussions) for general questions\n- 📧 **Email**: Open an issue for direct support needs\n\n### Troubleshooting\n\nCommon issues and solutions:\n\n## 🆕 What's New (v0.5.0 Beta)\n\n- Multilingual BM25 via per-row `lang` column\n  - `content_tsv` dihitung oleh trigger dengan `to_tsvector(pg_hybrid_safe_regconfig(lang), raw_content)`\n  - Query hybrid memakai `websearch_to_tsquery(pg_hybrid_safe_regconfig(lang), query)`\n- Exposed hybrid weights in search: `{ weights: { vectorW, textW } }`\n- Configurable rerank candidate pool: `{ topNForRerank }` (contoh: 50–200)\n- Example seeding multi-index + isolasi index (A/B/C): `npm run example`\n- CLI bin diperbaiki untuk `npx @brightly/pg-hybrid-search <cmd>`\n\n| Issue | Solution |\n|-------|----------|\n| `pgvector extension not found` | Install pgvector: `CREATE EXTENSION vector;` |\n| `OpenAI API rate limits` | Implement request batching and retry logic |\n| `Slow vector searches` | Tune IVFFlat index parameters |\n| `Connection pool exhausted` | Check for connection leaks, increase pool size |\n\n## 📊 Benchmarks\n\nPerformance characteristics on a standard setup (PostgreSQL 15, 4 CPU cores, 8GB RAM):\n\n| Operation | Documents | Time | Notes |\n|-----------|-----------|------|-------|\n| Document insertion | 1,000 | ~2.5s | Including embedding generation |\n| Vector search | 100K docs | ~50ms | With IVFFlat index |\n| Hybrid search | 100K docs | ~75ms | Combined vector + text |\n| Rerank (50 candidates) | - | ~200ms | Voyage API latency |\n\n*Benchmarks may vary based on document size, query complexity, and infrastructure.*\n\n## 🗺️ Roadmap\n\nPlanned features and improvements:\n\n- [ ] **Multi-language Support**: Enhanced tokenization for non-English content\n- [ ] **Custom Embedding Models**: Support for Hugging Face and other providers  \n- [ ] **Advanced Filtering**: Metadata-based search filtering\n- [ ] **Batch Reranking**: Optimize multiple query reranking\n- [ ] **Performance Monitoring**: Built-in metrics and observability\n- [ ] **Migration Tools**: Version upgrade utilities\n\n## 📄 License\n\nThis project is licensed under the **MIT License** - see the [LICENSE](LICENSE) file for details.\n\n```\nMIT License\n\nCopyright (c) 2024 Brightly Virya\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n```\n\n---\n\n<div align=\"center\">\n\n### 🌟 If this project helps you, please give it a star! \n\n[![GitHub stars](https://img.shields.io/github/stars/Brightlyviryaa/pg-hybrid-search?style=social)](https://github.com/Brightlyviryaa/pg-hybrid-search/stargazers)\n\n**Built with ❤️ for the Indonesian AI & Web3 community**\n\n*Empowering developers to build intelligent search experiences*\n\n</div>\n","readmeFilename":"README.md","_rev":"1-4eee56673732688996333df755a94145"}