大模型评测 — prd-w38

步骤:prd · 裁判:google/gemini-3.1-pro-preview · 2026-09-16T05:57:03.777Z
怎么读这份报告 —— 两个透镜: 战绩(胜–负–平)是成对偏好排名 —— 每个模型在若干场捉对中的胜负平(单一裁判、正反消偏)。它可靠地表达方向,但分辨率粗:对局少时,0 意味「每场都输」,而非「输出差」。绝对分(1–5)是对每份输出单独打的分 —— 看它才能判断质量差距的大小(强的亚军仍紧贴冠军,比如 4.2 对 4.5,即使 0 胜)。两者都不是百分制;提高 trials 和输入数量可让二者更精确。

PRD

🏆 kimi-k3 · 4 场对局中胜出 3 场 · 战绩 3–0–1 (胜–负–平) · 绝对分 5.0/5

Output B is the superior PRD because it maintains strict discipline against scope creep, provides highly testable edge-case criteria, and uses a more robust referencing structure for development.

综合排名

#模型战绩 胜–负–平绝对分 1–5平均成本平均延迟成功率
1 kimi-k3
3–0–1 共 4 场 · 88%		 5.0 $0.0615 51.8s 100%
2 deepseek-v4-pro
0–1–3 共 4 场 · 38%		 5.0 $0.0086 62.2s 100%
3 sonnet-5
0–2–2 共 4 场 · 25%		 5.0 $0.0236 24.1s 100%

维度剖面 — 1–5 绝对分 · 雷达(圆心 1,边缘 5)

sonnet-5deepseek-v4-prokimi-k3
完整性可测试性无幻觉结构2345sonnet-5 · 完整性: 5.0/5sonnet-5 · 可测试性: 5.0/5sonnet-5 · 无幻觉: 4.8/5sonnet-5 · 结构: 5.0/5deepseek-v4-pro · 完整性: 5.0/5deepseek-v4-pro · 可测试性: 5.0/5deepseek-v4-pro · 无幻觉: 5.0/5deepseek-v4-pro · 结构: 5.0/5kimi-k3 · 完整性: 5.0/5kimi-k3 · 可测试性: 5.0/5kimi-k3 · 无幻觉: 5.0/5kimi-k3 · 结构: 5.0/5

维度偏好 — 对局胜率,相对

模型完整性可测试性无幻觉结构
kimi-k3501007588
deepseek-v4-pro500380
sonnet-550503863
对决理由 · 总体 + 11 处维度分歧(共 6 场对比)

这些是单场、相对 A/B 的裁判理由(单一输入、仅取首个成功输出),不是该维度的综合质量评价;维度层只列出与总体判定不一致的项(标 ⚠,即真正的取舍点),一致的已隐藏。绝对质量请看上方「绝对分 1–5」表。

输入AB维度胜者理由
todo-app sonnet-5 deepseek-v4-pro 总体 平局 Output A is the stronger PRD because it strictly follows the brief without adding unrequested feature sections, and its Given/When/Then acceptance criteria provide superior testability and structure.
todo-app sonnet-5 deepseek-v4-pro 可测试性 ⚠ sonnet-5 Output A uses numbered Acceptance Criteria with strict Given/When/Then formatting, making it much easier to translate directly into automated tests.
todo-app sonnet-5 deepseek-v4-pro 结构 ⚠ sonnet-5 Output A provides a more rigorous structure with stable, numbered AC IDs (AC1, AC2) and clear BDD syntax that developers and QA can easily reference.
todo-app sonnet-5 kimi-k3 总体 kimi-k3 Output B is the superior PRD because it maintains strict discipline against scope creep, provides highly testable edge-case criteria, and uses a more robust referencing structure for development.
todo-app sonnet-5 kimi-k3 完整性 ⚠ 平局 Both PRDs fully cover all the core requirements outlined in the implied brief without missing any functional areas.
todo-app deepseek-v4-pro kimi-k3 总体 kimi-k3 Output B is the superior PRD because its numbered acceptance criteria make it highly testable, and it maintains strict discipline regarding the scope of the brief.
todo-app deepseek-v4-pro kimi-k3 完整性 ⚠ 平局 Both PRDs fully cover all the core requirements outlined in the brief without missing any functionality.
chat-app sonnet-5 deepseek-v4-pro 总体 平局 Output A is the superior PRD because it maintains strict product focus without dictating technical architecture, and its numbered acceptance criteria make it highly actionable for developers and testers.
chat-app sonnet-5 deepseek-v4-pro 可测试性 ⚠ sonnet-5 Output A uses stable, numbered IDs for every single acceptance criterion (e.g., AC-1.1), making it much easier for QA to track and verify.
chat-app sonnet-5 deepseek-v4-pro 结构 ⚠ sonnet-5 Output A provides a highly disciplined structure with clear, hierarchical numbering for both requirements and acceptance criteria.
chat-app sonnet-5 kimi-k3 总体 kimi-k3 Output B delivers a more robust and developer-ready PRD by addressing edge cases like full history pagination and explicitly documenting its product decisions to resolve brief ambiguities.
chat-app sonnet-5 kimi-k3 完整性 ⚠ 平局 Output B includes pagination to ensure the entire message history is accessible, whereas Output A only loads the last 50 messages with no way to view older ones.
chat-app sonnet-5 kimi-k3 无幻觉 ⚠ 平局 Both outputs introduce minor, standard technical constraints not explicitly in the brief (Output A adds max message length; Output B adds password length and pagination) to make the app buildable.
chat-app sonnet-5 kimi-k3 结构 ⚠ 平局 Output B includes a highly valuable 'Key Product Decisions' section that explicitly resolves ambiguities for developers, alongside standard metadata and clear IDs.
chat-app deepseek-v4-pro kimi-k3 总体 平局 Although Output B has slightly better structure and testability, Output A wins overall because it strictly follows the prompt's rule to avoid scope creep and unrequested features like pagination.
chat-app deepseek-v4-pro kimi-k3 可测试性 ⚠ kimi-k3 Output B uses the Given/When/Then format for its acceptance criteria, making them highly concrete and directly translatable into automated tests.
chat-app deepseek-v4-pro kimi-k3 结构 ⚠ kimi-k3 Output B uses stable, hierarchical IDs for every single acceptance criterion (e.g., AC-1.1), making it easier for developers to reference.