![\"Tardi](\"assets/tardi-logo.svg\"){kind=logo}
\n\n \n
\n Deterministic testing for LLM agents and non-deterministic pipelines.\n
\n\n\n ![\"NPM](\"https://img.shields.io/npm/v/tardi-cli.svg\")
\n ![\"License\"](\"https://img.shields.io/badge/License-ISC-blue.svg\"){kind=icon}
\n ![\"Build](\"https://img.shields.io/github/actions/workflow/status/Nyx-abu/tardi/ci.yml?branch=master\")
\n About •\n Features •\n The Pipeline •\n Installation •\n Quickstart •\n Examples & Output\n
\n\n---\n\nTardi is an open-source testing framework engineered specifically for evaluating agentic workflows, autonomous LLM scripts, and non-deterministic applications. \n\n## About\n\nTesting AI agents presents a unique challenge: traditional testing frameworks cannot evaluate non-deterministic natural language outputs, and raw \"LLM-as-a-judge\" evaluation pipelines are prohibitively expensive and prone to hallucination at scale.\n\n**Tardi solves this by implementing a tiered assertion gauntlet.** Instead of blindly sending every agent execution trace to an evaluation model, Tardi enforces strict deterministic constraints first. If your agent crashes, hangs in an infinite loop, or returns malformed JSON, Tardi fails the test immediately—preventing unnecessary LLM API calls and accelerating your feedback loop.\n\n## Features\n\n- **Tiered Evaluation Engine:** Catch process crashes, timeouts, and schema mismatches deterministically before triggering expensive LLM judges.\n- **Concurrency & Rate-Limiting:** Execute test suites in parallel with intelligent chunking to maximize throughput without exceeding API rate limits.\n- **Provider Agnostic:** Built on the standard Vercel AI SDK, Tardi supports hot-swappable evaluation models from OpenAI, Google, Anthropic, and local endpoints.\n- **Interactive REPL & Natural Language CLI:** Includes a zero-friction CLI environment for rapid test synthesis, execution, and debugging, powered by an onboard NLP intent parser.\n- **Secure Credential Management:** Safely stores provider API keys in your native OS keychain during local development.\n\n## The Tardi Pipeline\n\nWhen you execute an iteration, Tardi evaluates the agent's output through a strict, cost-saving pipeline. Deterministic checks always run first.\n\n```mermaid\ngraph TD\n Start([Run Agent]) --> ProcessCheck{Process Crashed?}\n ProcessCheck -- Yes --> FailCrash[FAIL: CRASH]\n ProcessCheck -- No --> TimeoutCheck{Exceeded Timeout?}\n \n TimeoutCheck -- Yes --> FailTimeout[FAIL: TIMEOUT]\n TimeoutCheck -- No --> RegexCheck{Matches Regex?}\n \n RegexCheck -- No --> FailRegex[FAIL: REGEX_MISMATCH]\n RegexCheck -- Yes --> SchemaCheck{Valid JSON Schema?}\n \n SchemaCheck -- No --> FailSchema[FAIL: SCHEMA_MISMATCH]\n SchemaCheck -- Yes --> LLMJudge{LLM Judge Approves?}\n \n LLMJudge -- No --> FailJudge[FAIL: LLM_JUDGE_FAIL]\n LLMJudge -- Yes --> Pass([PASS])\n \n style FailCrash fill:#fbd4d4,stroke:#f87171,color:#000\n style FailTimeout fill:#fbd4d4,stroke:#f87171,color:#000\n style FailRegex fill:#fbd4d4,stroke:#f87171,color:#000\n style FailSchema fill:#fbd4d4,stroke:#f87171,color:#000\n style FailJudge fill:#fbd4d4,stroke:#f87171,color:#000\n style Pass fill:#dcfce7,stroke:#4ade80,color:#000\n```\n\n## Installation\n\nInstall the CLI globally via npm to use the `tardi` command anywhere:\n\n```bash\nnpm install -g tardi-cli\n```\n\n## Quickstart\n\n### 1. Auto-Synthesize from a GitHub Repository\nTardi can automatically clone an agent repository, detect its entry point, and synthesize a test gauntlet based on golden runs.\n\n```bash\ntardi github https://github.com/Nyx-abu/demo-agent.git\n```\n\n\n ![\"Tardi](\"assets/demo-github.svg\")
\n
\n ![\"Tardi](\"assets/demo-terminal.svg\")
\n