
Various services that are integrated with Harbor. The link in the service name will lead you to a dedicated page in Harbor's wiki with details on getting started with the service.

# Frontends

This section covers services that can provide you with an interface for interacting with the language models.

- [AnythingLLM](https://github.com/av/harbor/wiki/2.1.9-Frontend:-AnythingLLM) <span style="opacity: 0.5;">`Frontend`, `Partial Support`</span><br/>
The all-in-one Desktop & Docker AI application with built-in RAG, AI agents, and more.

- [BionicGPT](https://github.com/av/harbor/wiki/2.1.8-Frontend:-BionicGPT) <span style="opacity: 0.5;">`Frontend`</span><br/>
on-premise LLM web UI with support for OpenAI-compatible backends

- [Chat Nio](https://github.com/av/harbor/wiki/2.1.10-Frontend:-Chat-Nio) <span style="opacity: 0.5;">`Frontend`</span><br/>
Comprehensive LLM web interface with built-in marketplace

- [ComfyUI](https://github.com/av/harbor/wiki/2.1.2-Frontend:-ComfyUI) <span style="opacity: 0.5;">`Frontend`, `Workflows`</span><br/>
The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface.

- [Hollama](https://github.com/av/harbor/wiki/2.1.6-Frontend:-hollama) <span style="opacity: 0.5;">`Frontend`</span><br/>
A minimal web-UI for talking to Ollama servers.

- [HuggingFace ChatUI](https://github.com/av/harbor/wiki/2.1.4-Frontend:-ChatUI) <span style="opacity: 0.5;">`Frontend`</span><br/>
A chat interface using open source models, eg OpenAssistant or Llama. It is a SvelteKit app and it powers the HuggingChat app on hf.co/chat.

- [KoboldCpp](https://github.com/av/harbor/wiki/2.2.16-Backend:-KoboldCpp) <span style="opacity: 0.5;">`Satellite`, `Frontend`, `Backend`</span><br/>
KoboldCpp is an easy-to-use AI text-generation software for GGML and GGUF models.

- [LibreChat](https://github.com/av/harbor/wiki/2.1.3-Frontend:-LibreChat) <span style="opacity: 0.5;">`Frontend`</span><br/>
Open-source ChatGPT UI alternative supporting multiple AI providers (Anthropic, AWS, OpenAI, Azure, Groq, Mistral, Google) with features like model switching, message search, and multi-user support. Includes integration with DALL-E-3 and various APIs.

- [Lobe Chat](https://github.com/av/harbor/wiki/2.1.5-Frontend:-Lobe-Chat) <span style="opacity: 0.5;">`Frontend`</span><br/>
An open-source, modern-design AI chat framework. Supports Multi AI Providers( OpenAI / Claude 3 / Gemini / Ollama / Azure / DeepSeek), Knowledge Base (file upload / knowledge management / RAG ), Multi-Modals (Vision/TTS) and plugin system.

- [Mikupad](https://github.com/av/harbor/wiki/2.1.11-Frontend:-Mikupad) <span style="opacity: 0.5;">`Frontend`</span><br/>
LLM Frontend in a single HMTL file

- [mistral.rs](https://github.com/av/harbor/wiki/2.2.6-Backend:-mistral.rs) <span style="opacity: 0.5;">`Frontend`</span><br/>
Blazingly fast LLM inference.

- [ol1](https://github.com/av/harbor/wiki/2.3.19-Satellite:-ol1) <span style="opacity: 0.5;">`Frontend`</span><br/>
A simple Gradio app implementing an o1-like chain of reasoning with Ollama.

- [Omnichain](https://github.com/av/harbor/wiki/2.3.16-Satellite:-omnichain) <span style="opacity: 0.5;">`Frontend`</span><br/>
Visual programming for AI language models

- [Open WebUI](https://github.com/av/harbor/wiki/2.1.1-Frontend:-Open-WebUI) <span style="opacity: 0.5;">`Frontend`</span><br/>
widely adopted and feature rich web interface for interacting with LLMs. Supports OpenAI-compatible and Ollama backends, multi-users, multi-model chats, custom prompts, TTS, Web RAG, RAG, and much much more.

- [oterm](https://github.com/av/harbor/wiki/2.1.12-Frontend-oterm) <span style="opacity: 0.5;">`Frontend`, `CLI`</span><br/>
The text-based terminal client for Ollama.

- [Parllama](https://github.com/av/harbor/wiki/2.1.7-Frontend:-parllama) <span style="opacity: 0.5;">`Frontend`</span><br/>
TUI for Ollama

# Backends

This section covers services that provide the LLM inference capabilities.

- [AirLLM](https://github.com/av/harbor/wiki/2.2.11-Backend:-AirLLM) <span style="opacity: 0.5;">`Backend`</span><br/>
70B inference with single 4GB GPU (very slow, though)

- [Aphrodite](https://github.com/av/harbor/wiki/2.2.5-Backend:-Aphrodite-Engine) <span style="opacity: 0.5;">`Backend`</span><br/>
Large-scale LLM inference engine

- [faster-whisper-server](https://github.com/av/harbor/wiki/2.2.14-Backend:-Speaches) <span style="opacity: 0.5;">`Backend`, `Audio`, `Partial Support`</span><br/>
Legacy version of Speaches, use that instead.

- [KoboldCpp](https://github.com/av/harbor/wiki/2.2.16-Backend:-KoboldCpp) <span style="opacity: 0.5;">`Satellite`, `Frontend`, `Backend`</span><br/>
KoboldCpp is an easy-to-use AI text-generation software for GGML and GGUF models.

- [KTransformers](https://github.com/av/harbor/wiki/2.2.13-Backend:-KTransformers) <span style="opacity: 0.5;">`Backend`</span><br/>
A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations

- [llama.cpp](https://github.com/av/harbor/wiki/2.2.2-Backend:-llama.cpp) <span style="opacity: 0.5;">`Backend`</span><br/>
LLM inference in C/C++

- [lmdeploy](https://github.com/av/harbor/wiki/2.2.10-Backend:-lmdeploy) <span style="opacity: 0.5;">`Backend`, `Partial Support`</span><br/>


- [Nexa SDK](https://github.com/av/harbor/wiki/2.2.15-Backend:-Nexa-SDK) <span style="opacity: 0.5;">`Backend`, `Partial Support`</span><br/>
Nexa SDK is a comprehensive toolkit for supporting ONNX and GGML models.

- [Ollama](https://github.com/av/harbor/wiki/2.2.1-Backend:-Ollama) <span style="opacity: 0.5;">`Backend`</span><br/>
Get up and running with Llama 3.2, Mistral, Gemma 3, and other large language models.

- [openedai-speech](https://github.com/av/harbor/wiki/2.2.7-Backend:-openedai-speech) <span style="opacity: 0.5;">`Backend`, `Audio`</span><br/>
An OpenAI API compatible text to speech server using Coqui AI's xtts_v2 and/or piper tts as the backend.

- [Parler](https://github.com/av/harbor/wiki/2.2.8-Backend:-Parler) <span style="opacity: 0.5;">`Backend`, `Audio`</span><br/>
Inference and training library for high-quality TTS models.

- [SGLang](https://github.com/av/harbor/wiki/2.2.12-Backend:-SGLang) <span style="opacity: 0.5;">`Backend`</span><br/>
SGLang is a fast serving framework for large language models and vision language models.

- [Speaches](https://github.com/av/harbor/wiki/2.2.14-Backend:-Speaches) <span style="opacity: 0.5;">`Backend`, `Audio`</span><br/>
an OpenAI API-compatible speech server (formerly `faster-whisper-server`), both TTS and STT

- [TabbyAPI](https://github.com/av/harbor/wiki/2.2.4-Backend:-TabbyAPI) <span style="opacity: 0.5;">`Backend`</span><br/>
An OAI compatible exllamav2 API that's both lightweight and fast

- [Text Generation Inference](https://github.com/av/harbor/wiki/2.2.9-Backend:-text-generation-inference) <span style="opacity: 0.5;">`Backend`</span><br/>
Inference engine from HuggingFace.

- [vLLM](https://github.com/av/harbor/wiki/2.2.3-Backend:-vLLM) <span style="opacity: 0.5;">`Backend`</span><br/>
A high-throughput and memory-efficient inference and serving engine for LLMs

# Satellites

Additional services that can be integrated with various Frontends and Backends to enable more features.

- [aichat](https://github.com/av/harbor/wiki/2.3.14-Satellite:-aichat) <span style="opacity: 0.5;">`Satellite`, `CLI`</span><br/>
All-in-one LLM CLI tool featuring Shell Assistant, Chat-REPL, RAG, AI tools & agents.

- [Aider](https://github.com/av/harbor/wiki/2.3.13-Satellite:-aider) <span style="opacity: 0.5;">`Satellite`, `CLI`</span><br/>
Aider is AI pair programming in your terminal.

- [autogpt](https://github.com/av/harbor/wiki/2.3.15-Satellite:-AutoGPT) <span style="opacity: 0.5;">`Satellite`, `Partial Support`</span><br/>
Create, deploy, and manage continuous AI agents that automate complex workflows.

- [Bolt.new](https://github.com/av/harbor/wiki/2.3.24-Satellite:-Bolt.new) <span style="opacity: 0.5;">`Satellite`</span><br/>
Prompt, run, edit, and deploy full-stack web applications.

- [cloudflared](https://github.com/av/harbor/wiki/2.3.8-Satellite:-cloudflared) <span style="opacity: 0.5;">`Satellite`, `API`, `CLI`</span><br/>
A helper service allowing to expose Harbor services over the internet.

- [cmdh](https://github.com/av/harbor/wiki/2.3.9-Satellite:-cmdh) <span style="opacity: 0.5;">`Satellite`, `CLI`</span><br/>
Create Linux commands from natural language, in the shell.

- [Dify](https://github.com/av/harbor/wiki/2.3.3-Satellite:-Dify) <span style="opacity: 0.5;">`Satellite`, `Workflows`</span><br/>
An open-source LLM app development platform.

- [Fabric](https://github.com/av/harbor/wiki/2.3.10-Satellite:-fabric) <span style="opacity: 0.5;">`Satellite`, `CLI`</span><br/>
LLM-driven processing of the text data in the terminal.

- [Flowise](https://github.com/av/harbor/wiki/2.3.31-Satellite:-Flowise) <span style="opacity: 0.5;">`Satellite`, `Workflows`</span><br/>
Drag & drop UI to build your customized LLM flow.

- [gptme](https://github.com/av/harbor/wiki/2.3.36-Satellite-gptme) <span style="opacity: 0.5;">`Satellite`, `CLI`</span><br/>
A simple CLI tool to interact with LLMs.

- [Harbor Bench](https://github.com/av/harbor/wiki/5.1.-Harbor-Bench) <span style="opacity: 0.5;">`Satellite`, `CLI`, `Built-in`, `Eval`</span><br/>
Harbor's own tool to evaluate LLMs and inference backends against custom tasks.

- [Harbor Boost](https://github.com/av/harbor/wiki/5.2.-Harbor-Boost) <span style="opacity: 0.5;">`Satellite`, `API`, `Built-in`</span><br/>
Connects to downstream LLM API and serves a wrapper with custom workflow. For example, it can be used to add a CoT (Chain of Thought) to an existing LLM API, and much more. Scriptable with Python.

- [JupyterLab](https://github.com/av/harbor/wiki/2.3.18-Satellite:-JupyterLab) <span style="opacity: 0.5;">`Satellite`</span><br/>
Helper service to author/run Jupyter notebooks in Python with access to Harbor services.

- [K6](https://github.com/av/harbor/wiki/2.3.27-Satellite:-K6) <span style="opacity: 0.5;">`Satellite`, `CLI`</span><br/>
A modern load testing tool, using Go and JavaScript - https://k6.io

- [KoboldCpp](https://github.com/av/harbor/wiki/2.2.16-Backend:-KoboldCpp) <span style="opacity: 0.5;">`Satellite`, `Frontend`, `Backend`</span><br/>
KoboldCpp is an easy-to-use AI text-generation software for GGML and GGUF models.

- [LangFlow](https://github.com/av/harbor/wiki/2.3.32-Satellite:-LangFlow) <span style="opacity: 0.5;">`Satellite`, `Workflows`</span><br/>
A low-code app builder for RAG and multi-agent AI applications.

- [LangFuse](https://github.com/av/harbor/wiki/2.3.6-Satellite:-langfuse) <span style="opacity: 0.5;">`Satellite`, `API`</span><br/>
Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets.

- [Latent Scope](https://github.com/av/harbor/wiki/2.3.38-Satellite-Latent-Scope) <span style="opacity: 0.5;">`Satellite`</span><br/>
A new kind of workflow + tool for visualizing and exploring datasets through the lens of latent spaces.

- [LibreTranslate](https://github.com/av/harbor/wiki/2.3.41-Satellite-LibreTranslate) <span style="opacity: 0.5;">`Satellite`</span><br/>
A free and open-source machine translation.

- [LiteLLM](https://github.com/av/harbor/wiki/2.3.5-Satellite:-LiteLLM) <span style="opacity: 0.5;">`Satellite`, `API`</span><br/>
LLM proxy that can aggregate multiple inference APIs together into a single endpoint.

- [LitLytics](https://github.com/av/harbor/wiki/2.3.21-Satellite:-LitLytics) <span style="opacity: 0.5;">`Satellite`, `Partial Support`, `Workflows`</span><br/>
Simple analytics platform that leverages LLMs to automate data analysis.

- [llama-swap](https://github.com/av/harbor/wiki/2.3.40-Satellite-llamaswap) <span style="opacity: 0.5;">`Satellite`, `API`</span><br/>
Runs multiple llama.cpp servers on demand for seamless switching between them.

- [lm-evaluation-harness](https://github.com/av/harbor/wiki/2.3.17-Satellite:-lm-evaluation-harness) <span style="opacity: 0.5;">`Satellite`, `CLI`, `Eval`</span><br/>
A de-facto standard framework for the few-shot evaluation of language models.

- [mcpo](https://github.com/av/harbor/wiki/2.3.43-Satellite-mcpo) <span style="opacity: 0.5;">`Satellite`, `Tools`</span><br/>
Turn MCP servers into OpenAPI REST APIs - use them anywhere.

- [MetaMCP](https://github.com/av/harbor/wiki/2.3.42-Satellite-MetaMCP) <span style="opacity: 0.5;">`Satellite`, `Tools`</span><br/>
Allows to manage MCPs via a WebUI, exposes multiple MCPs as a single server.

- [Morphic](https://github.com/av/harbor/wiki/2.3.34-Satellite-Morphic) <span style="opacity: 0.5;">`Satellite`</span><br/>
An AI-powered search engine with a generative UI, similar to Perplexity and Perplexica.

- [n8n](https://github.com/av/harbor/wiki/2.3.23-Satellite:-n8n) <span style="opacity: 0.5;">`Satellite`, `Workflows`</span><br/>
Fair-code workflow automation platform with native AI capabilities.

- [OmniParser](https://github.com/av/harbor/wiki/2.3.30-Satellite:-OmniParser) <span style="opacity: 0.5;">`Satellite`</span><br/>
A simple screen parsing tool towards pure vision based GUI agent.

- [Open Interpreter](https://github.com/av/harbor/wiki/2.3.7-Satellite:-Open-Interpreter) <span style="opacity: 0.5;">`Satellite`, `CLI`</span><br/>
A natural language interface for computers.

- [Open WebUI Pipelines](https://github.com/av/harbor/wiki/2.3.25-Satellite:-Open-WebUI-Pipelines) <span style="opacity: 0.5;">`Satellite`, `API`, `Workflows`</span><br/>
UI-Agnostic OpenAI API Plugin Framework.

- [OpenHands](https://github.com/av/harbor/wiki/2.3.20-Satellite:-OpenHands) <span style="opacity: 0.5;">`Satellite`, `Partial Support`</span><br/>
A platform for software development agents powered by AI.

- [OptiLLM](https://github.com/av/harbor/wiki/2.3.33-Satellite:-OptiLLM) <span style="opacity: 0.5;">`Satellite`, `API`</span><br/>
Optimising LLM proxy that implements many advanced workflows to boost the performance of the LLMs.

- [Perplexica](https://github.com/av/harbor/wiki/2.3.2-Satellite:-Perplexica) <span style="opacity: 0.5;">`Satellite`</span><br/>
An AI-powered search engine. It is an Open source alternative to Perplexity AI.

- [Plandex](https://github.com/av/harbor/wiki/2.3.4-Satellite:-Plandex) <span style="opacity: 0.5;">`Satellite`, `CLI`</span><br/>
AI driven development in your terminal.

- [Promptfoo](https://github.com/av/harbor/wiki/2.3.28-Satellite:-Promptfoo) <span style="opacity: 0.5;">`Satellite`, `CLI`</span><br/>
Test your prompts, agents, and RAGs. A developer-friendly local tool for testing LLM applications.

- [Qdrant](https://github.com/av/harbor/wiki/2.3.26-Satellite:-Qdrant) <span style="opacity: 0.5;">`Satellite`</span><br/>
Qdrant - High-performance, massive-scale Vector Database and Vector Search Engine.

- [RAGLite](https://github.com/av/harbor/wiki/2.3.39-Satellite-RAGLite) <span style="opacity: 0.5;">`Satellite`</span><br/>
Python toolkit for Retrieval-Augmented Generation (RAG)

- [Repopack](https://github.com/av/harbor/wiki/2.3.22-Satellite:-Repopack) <span style="opacity: 0.5;">`Satellite`, `CLI`</span><br/>
A powerful tool that packs your entire repository into a single, AI-friendly file.

- [SearXNG](https://github.com/av/harbor/wiki/2.3.1-Satellite:-SearXNG) <span style="opacity: 0.5;">`Satellite`</span><br/>
A privacy-respecting, hackable metasearch engine. Highly configurable and can be used for Web RAG use-cases.

- [SQL Chat](https://github.com/av/harbor/wiki/2.3.35-Satellite-SQL-Chat) <span style="opacity: 0.5;">`Satellite`</span><br/>
Chat-based SQL client, which uses natural language to communicate with the database.

- [SuperGateway](https://github.com/av/harbor/wiki/2.3.44-Satellite-supergateway) <span style="opacity: 0.5;">`Satellite`, `CLI`, `Tools`</span><br/>
A simple and powerful API gateway for LLMs.

- [TextGrad](https://github.com/av/harbor/wiki/2.3.12-Satellite:-TextGrad) <span style="opacity: 0.5;">`Satellite`</span><br/>
Automatic "Differentiation" via Text - using large language models to backpropagate textual gradients.

- [Traefik](https://github.com/av/harbor/wiki/2.3.37-Satellite-traefik) <span style="opacity: 0.5;">`Satellite`, `API`</span><br/>
A modern HTTP reverse proxy and load balancer that makes deploying microservices easy.

- [txtai RAG](https://github.com/av/harbor/wiki/2.3.11-Satellite:-txtai-RAG) <span style="opacity: 0.5;">`Satellite`</span><br/>
RAG WebUI built with txtai.

- [Webtop](https://github.com/av/harbor/wiki/2.3.29-Satellite:-Webtop) <span style="opacity: 0.5;">`Satellite`</span><br/>
Linux in a web browser supporting popular desktop environments.
  