---
name: video-production
description: >
  Full-stack video production assistant. Analyzes video content visually (Gemini),
  generates transcriptions/SRT subtitles, plans and creates motion graphics (Remotion),
  generates B-roll images/videos, produces timeline XMLs for Premiere/DaVinci.
  Downloads YouTube videos with yt-dlp.
  Use for: video analysis, visual analysis, describe video, what's in this video,
  transcription, subtitles, motion graphics, B-roll, shorts, timeline XML, clip cutting,
  silence removal, After Effects, Premiere Pro, DaVinci Resolve, YouTube download.
  Keywords: video edit, ffmpeg, remotion, after effects, premiere, davinci, shorts, subtitles,
  motion graphics, clip, render, transcribe, xml, timeline, b-roll, talking head, analyze,
  yt-dlp, youtube, download, gemini, vision
allowed-tools:
  - Read
  - Write
  - Edit
  - Bash
  - Glob
  - Grep
  - Agent
  - AskUserQuestion
  - TaskCreate
  - TaskUpdate
  - WebFetch
---

# Video Production — Strategy Map

## ⚠️ DEFAULT RULE: Video Analysis = Visual Analysis (NOT Transcription)

**The agent has built-in vision for images. For videos, always use Gemini via Kolbo MCP.**

| Media type | Action |
|------------|--------|
| **Image** (jpg, png, etc.) | Agent reads it directly — no upload needed |
| **Video** — "analyze", "describe", "what's in this?", "what prompts?", file path with no instruction | `upload_media` → `chat_send_message` + Gemini |
| **Transcription** — "transcribe", "subtitles", "SRT", "what's being said", "captions" | `transcribe_audio` only |
| Both visual + transcript | Run both |

**Never use ffmpeg to extract frames for analysis. Never use local Ollama/vision models. Commit to the right action — do not ask the user. Wait for `chat_send_message` to return before proceeding — it polls until done (up to 2 min). Do NOT fall back to ffmpeg or any other approach if it takes time.**

---

## Kolbo MCP Tools (Active When `kolbo auth login` Is Done)

These are available as MCP tools — use them directly without any Python/API key setup:

| Tool | Use |
|------|-----|
| `upload_media` | Upload local file to Kolbo CDN → get stable public URL |
| `chat_send_message` | Send message + `media_urls` array to Gemini for visual analysis |
| `transcribe_audio` | Transcribe audio/video to text + SRT (ElevenLabs Scribe) |
| `generate_image` | Generate B-roll images |
| `generate_video` | Generate B-roll videos |
| `generate_video_from_image` | Animate a still into video |
| `generate_music` | Generate background music |
| `generate_speech` | TTS for voiceover |
| `generate_sound` | Sound effects |
| `list_models` | Browse available models by type |
| `check_credits` | Check remaining Kolbo credit balance |

### Visual Analysis Workflow — MANDATORY for all video analysis

**Step 1 is NOT optional. You cannot skip `upload_media` or construct the URL yourself.**

```
Step 1: upload_media({ source: "/absolute/path/to/video.mp4" })
  → Returns: { url, thumbnail_url, ... }
  → Save the "url" field — this is the CDN URL you will pass to Gemini
  → NEVER use thumbnail_url (it's a JPG preview, not the video)

Step 2: chat_send_message({
  message: "Describe this video in detail. What is shown?",
  media_urls: ["<url from step 1>"]   ← must be an array, must be the "url" field
})
→ returns: { content: "..." }
```

**❌ Common mistakes that break video analysis:**
- Skipping `upload_media` and passing a local file path to `chat_send_message` — local paths don't work
- Using the transcription `.txt` URL as the `media_urls` value — Gemini needs the actual video CDN URL
- Using `thumbnail_url` instead of `url` from the `upload_media` response
- Calling `transcribe_audio` first then passing its output URL as the video — transcription gives text, not video

**Omit `model`** — Smart Select detects video/audio and auto-routes to Gemini.
**Sessions do NOT remember media between messages.** On retry: reuse the same CDN `url` from step 1 (no re-upload needed) but always pass `media_urls` again.

**Batch analysis (many videos)**: Pass `model: "gemini-3.1-flash-lite-preview"` explicitly for cheaper bulk runs.

For YouTube videos — download first with yt-dlp (see below), then follow steps 1–2 above.

---

## Pipeline

```
Input: local video / YouTube URL / uploaded file

→ [DEFAULT] Visual Analysis: upload_media → chat_send_message (Gemini)
→ [EXPLICIT REQUEST] Transcription: transcribe_audio → SRT / text
→ [EDITING] FFmpeg: cut, silence removal, 9:16 conversion
→ [MOTION GRAPHICS] Remotion: compositions, captions, B-roll
→ Output: Premiere XML / DaVinci EDL / MP4s / SRT
```

## APIs & Capabilities

| Service | Use |
|---------|-----|
| Kolbo MCP (`upload_media` + `chat_send_message`) | **Primary** — visual video/image analysis via Gemini |
| Kolbo MCP (`transcribe_audio`) | **Primary** — transcription, word-level SRT, multilingual |
| yt-dlp | Download YouTube/social media videos |
| FFmpeg | Local video editing, cutting, silence removal, format conversion |
| Remotion Lambda | Cloud render motion graphics |
| fal.ai (MCP) | Image & video B-roll generation |
| ElevenLabs | TTS, voice cloning, SFX (via Kolbo MCP `generate_speech`) |
| Suno | Background music (via Kolbo MCP `generate_music`) |

> Kolbo MCP tools need no API keys — auth is handled by `kolbo auth login`.
> FFmpeg/yt-dlp need to be installed locally on the machine.

## YouTube / Social Media Download (yt-dlp)

Download video from YouTube, TikTok, Instagram, Twitter, etc.:

```bash
# Best quality MP4
yt-dlp -f "bestvideo[height<=1080][ext=mp4]+bestaudio/best" \
  --merge-output-format mp4 \
  -o "%(id)s.%(ext)s" <url>

# With subtitles
yt-dlp -f "bestvideo[height<=1080][ext=mp4]+bestaudio/best" \
  --write-auto-sub --sub-lang en --convert-subs srt \
  --merge-output-format mp4 \
  -o "%(id)s.%(ext)s" <url>

# Audio only (for transcription)
yt-dlp -f "bestaudio" --extract-audio --audio-format mp3 -o "%(id)s.%(ext)s" <url>
```

After download → upload to Kolbo CDN with `upload_media` → analyze visually with `chat_send_message`.

---

## Key Rules

- **FFmpeg on Windows**: always copy inputs to `tempfile.mkdtemp()` first (handles spaces in paths)
- **9:16 shorts**: blurred background + centered content — never crop the original
- **Hebrew / RTL subtitles**: ASS format, Heebo Bold font, `\pos()` for RTL rendering
- **Video quality standard**: `-c:v libx264 -crf 18 -c:a aac -b:a 128k`
- **Image generation**: prefer fal MCP server over Python scripts when available
- **Silence removal**: `silencedetect -35dB:d=0.4` → trim+concat → `atempo=1.14`

## Transcription

Use ElevenLabs Scribe for word-level SRT with speaker diarization:
```python
import requests

def transcribe(audio_path, api_key, language="he"):
    with open(audio_path, "rb") as f:
        response = requests.post(
            "https://api.elevenlabs.io/v1/speech-to-text",
            headers={"xi-api-key": api_key},
            files={"file": f},
            data={"model_id": "scribe_v1", "language_code": language,
                  "timestamps_granularity": "word", "diarize": True}
        )
    return response.json()
```

## 9:16 Shorts — Blurred Background

```python
filter_complex = (
    "[0:v]split[bg][fg];"
    "[bg]scale=1080:1920:force_original_aspect_ratio=increase,"
    "crop=1080:1920,gblur=sigma=40[blurred];"
    "[fg]scale=1080:1920:force_original_aspect_ratio=decrease,"
    "pad=1080:1920:(ow-iw)/2:(oh-ih)/2:color=black@0[front];"
    "[blurred][front]overlay=0:0"
)
```

## Silence Removal

```python
import subprocess, json

def detect_silence(video_path, noise_db=-35, duration=0.4):
    result = subprocess.run([
        "ffmpeg", "-i", video_path,
        "-af", f"silencedetect=noise={noise_db}dB:d={duration}",
        "-f", "null", "-"
    ], capture_output=True, text=True)
    # Parse silence_start/silence_end from stderr
    ...
```

## RTL (Hebrew/Arabic) Subtitles

For comprehensive RTL subtitle handling, load the `subtitle-production` skill — it contains full patterns for:
- Simple SRT burn-in with Heebo font + `Encoding=177`
- ASS per-word positioning for karaoke (with PIL `~0.74` scale factor)
- Remotion RTL captions with CSS `direction: rtl` and all the flip rules
- RTL progress bar with FFmpeg `geq` filter

**CRITICAL**: Any inline ASS tag (`\c`, `\K`, `\1c`, etc.) between RTL words breaks Unicode bidi in libass — words render LTR. Use separate Dialogue lines per word instead.

For Remotion RTL layout rules (padding flips, transform-origin, gradient direction), load the `typography-video` skill.

## Remotion Motion Graphics

For motion graphics rendering, use the `remotion-best-practices` skill for detailed Remotion patterns.

For cloud rendering via Remotion Lambda:
```bash
npx remotion lambda render <serve-url> <composition-id> --out output.mp4
```

## Premiere Pro XML Timeline

```python
def generate_premiere_xml(clips, output_path, fps=30):
    # Generate FCP7 XML compatible with Premiere Pro
    ...
```

## Output Structure

Organize outputs per project:
```
<project>/
├── raw/          # original footage
├── transcripts/  # SRT, word-level JSON
├── clips/        # cut segments
├── shorts/       # 9:16 vertical versions
├── b-roll/       # generated B-roll images/videos
├── motion/       # Remotion compositions
└── export/       # final deliverables + XML timelines
```

## Check Before Writing New Scripts

Before writing a new script, ask the user if they already have one for the task — they may have existing tools for clipping, silence removal, or subtitle burning.
