---
id: study-designer
stage: C_design
description: "Design empirical, qualitative, or mixed-methods studies including sampling strategy, instruments, procedures, analysis plans, and protocol lock-in."
inputs:
  - type: RQSet
    description: "Research questions and hypotheses"
  - type: TheoreticalFramework
    description: "Theoretical framework for operationalization"
    required: false
outputs:
  - type: DesignSpec
    artifact: "study_design.md"
  - type: AnalysisPlan
    artifact: "analysis_plan.md"
  - type: DataManagementPlan
    artifact: "data_management_plan.md"
  - type: Instruments
    artifact: "instruments/"
  - type: Preregistration
    artifact: "preregistration.md"
constraints:
  - "Must justify design choice against alternatives"
  - "Must specify design-appropriate sampling adequacy logic (power/MDE for quant; saturation/theoretical sampling for qual)"
  - "Must specify missingness handling or evidence-gap handling appropriate to the design"
failure_modes:
  - "Sampling adequacy logic is missing or mismatched to the design"
  - "Design-method mismatch with research questions"
tools: [filesystem, metadata-registry]
tags: [design, sampling, measures, analysis-plan, preregistration]
domain_aware: true
---

# Study Designer Skill

Design an empirical, qualitative, or mixed-methods study from a research question and constraints, producing a protocol-style design document, analysis plan, instruments skeleton, and optional preregistration draft.

## Purpose

Design empirical, qualitative, or mixed-methods studies including sampling strategy, instruments, procedures, analysis plans, and protocol lock-in.

## When to Use

- After `/find-gap` and `/build-framework`, when you want to run an empirical, qualitative, or mixed-methods study.
- When you have a topic + tentative RQs/hypotheses but no concrete methodology yet.
- When you want to write a fully qualitative paper and need the design, coding logic, and evidence plan to be explicit before drafting.

## Granularity Boundary

`study-designer` owns the design package as a whole:
- design choice rationale
- construct-to-measure mapping
- sampling and procedure logic
- analysis summary handoff
- governance and preregistration pointers

Do not split these into separate top-level skills unless a new artifact path and direct pipeline dependency are required.

## Inputs (Ask / Collect)

1. **Research question(s)** and whether the goal is *causal*, *descriptive*, *predictive*, *explanatory*, or *interpretive*
2. **Domain constraints**: data access, population access, timeline, budget, tools
3. **Unit of analysis**: individuals / teams / orgs / posts / countries / etc.
4. **Ethics constraints**: minors, sensitive data, protected classes, high-risk contexts
5. **Target venue** (optional): affects reporting norms and word limits

## Inputs

- `RQSet`: Research questions and hypotheses
- `TheoreticalFramework`: Theoretical framework for operationalization
- If a required input is missing or insufficient, write a gap note under `RESEARCH/[topic]/context/gap_notes.md` and ask for the missing artifact instead of inventing content.
- Treat literature, data, citations, and project files as evidence sources; keep unsupported assumptions visibly marked.

## Process

### Gate And Method-Pack Alignment

When a domain profile is available, load `skills/domain-profiles/[domain].yaml` and use each selected method template's `assumptions`, `required_diagnostics`, `required_artifacts`, `failure_modes`, and `minimum_report_fields` as constraints. Before finalizing `RESEARCH/[topic]/study_design.md`, check Q1 from `standards/quality-gate-contract.yaml`: every RQ or hypothesis must map to a method, data source, measurement plan, and analysis strategy. Record unresolved validity threats in `RESEARCH/[topic]/design/validity-threat-matrix.md`.

### Step 1: Choose Study Type (Fit-to-Question)

Pick the simplest design that can answer the RQ credibly. Use this decision matrix:

#### Design Selection Matrix

| Research Goal | Best Design | When Feasible? | Threats to Address | Alternative if Not Feasible |
|--------------|-------------|----------------|-------------------|-----------------------------|
| **Causal effect of intervention** | Randomized controlled trial (RCT) | Randomization possible; ethical | Attrition, compliance, spillover | Quasi-experimental |
| **Causal effect, no randomization** | Quasi-experimental (DID, RDD, IV, PSM) | Natural variation / policy shock | Parallel trends, instrument validity | Observational with strong controls |
| **Causal effect over time** | Panel / longitudinal | Repeated observations available | Time-varying confounders, attrition | Repeated cross-section |
| **Association / prevalence** | Cross-sectional survey | Large accessible population | Common method bias, endogeneity | Secondary data analysis |
| **Prediction / classification** | Predictive modeling / ML | Large dataset; clear outcome | Overfitting, generalization | Traditional regression |
| **Process / mechanism / meaning** | Qualitative (interviews, ethnography, case study) | Access to informants/sites; depth > breadth | Researcher bias, limited generalizability | Mixed methods |
| **How and why of a phenomenon** | Case study (single or multiple) | Rich, bounded case(s) available | Case selection bias, thick description needed | Comparative case analysis |
| **Theory building from data** | Grounded theory | Phenomenon poorly theorized | Premature closure, procedural fidelity | Thematic analysis with abductive reasoning |
| **Lived experience / meaning-making** | Phenomenology | Access to people who experienced the phenomenon | Bracketing, epoché validity | Narrative inquiry |
| **Both measurement and explanation** | Mixed methods (sequential / concurrent) | Resources for both phases | Integration quality, paradigm coherence | Choose dominant strand and add secondary |

Document why this design is the best tradeoff given constraints. Include explicit reasons why alternative designs were not chosen.

#### Design Justification Template

```markdown
## Design Rationale

**Selected design**: [Design type]

**Why this design fits the RQ**:
[1–2 sentences linking the design capabilities to the RQ requirements]

**Why alternatives were rejected**:
- [Alternative 1]: rejected because [specific reason — access, ethics, feasibility]
- [Alternative 2]: rejected because [specific reason]

**Key tradeoffs accepted**:
- [Tradeoff 1: e.g., "lower internal validity in exchange for higher ecological validity"]
- [Tradeoff 2]
```

### Step 2: Define Constructs, Variables, and Operationalization

#### For Quantitative Designs

| Element | What to Specify | Example |
|---------|----------------|---------|
| **Independent variable(s)** | Definition, measurement, scale, source | Remote work ratio: % of WFH days per month (self-report, continuous) |
| **Dependent variable(s)** | Definition, measurement, scale, source | Productivity index: quarterly output / hours, from HR system |
| **Mediators** | Theory-derived mechanism variables | Autonomy perception: 5-item SDT scale (Deci & Ryan, α = .87) |
| **Moderators** | Boundary condition variables | Team size: count from HR records |
| **Controls** | Confounders to include with justification | Tenure, education, industry — justified by theory and prior evidence |
| **Instruments** | Validated scales, surveys, measures | Use validated instruments; cite source + reliability; document any adaptations |

> **Anti-pattern**: "We control for demographics" — each control must be theoretically justified, not included as a blanket.

#### For Qualitative Designs

| Element | What to Specify | Example |
|---------|----------------|---------|
| **Sensitizing concepts** | Initial theoretical lenses guiding attention | Institutional logic: regulative, normative, cultural-cognitive |
| **Case/site definition** | What constitutes a "case" and its boundaries | Case = platform governance episode; bounded by policy announcement → resolution |
| **Data sources** | Interviews, documents, observations, artifacts | Semi-structured interviews (60 min); board meeting minutes; platform changelogs |
| **Evidence forms** | What counts as evidence for a claim | Informant statements, document excerpts, observer notes, artifact analysis |
| **Coding approach** | Inductive / deductive / hybrid; codebook evolution rules | Hybrid: start with sensitizing concepts, allow emergent codes; stabilize codebook after cycle 2 |

### Step 3: Sampling & Recruitment Plan

#### Quantitative Sampling

| Sampling Strategy | When to Use | How to Determine N |
|------------------|------------|-------------------|
| **Random sampling** | Generalizable population inference | Power analysis: specify α, β, MDE, expected σ |
| **Stratified sampling** | When subgroups are important | Power per stratum |
| **Convenience sampling** | When access is limited | Report limitations; avoid causal language |
| **Cluster sampling** | When randomizing at group level | Adjust for ICC; inflate N |

**Power Analysis Requirements**:

```
Minimum reporting:
- Significance level (α): typically 0.05
- Power (1 − β): typically 0.80 or 0.90
- Minimum detectable effect (MDE): justify from prior literature
- Expected variance: from pilot data or prior studies
- Final sample size: N = [calculated value]
- Tool used: G*Power / pwr (R) / statsmodels (Python)
```

#### Qualitative Sampling

| Strategy | When to Use | Sample Size Guidance |
|----------|------------|---------------------|
| **Purposive** | Selecting information-rich cases | 6–25 participants for interviews; 3–10 for case study |
| **Maximum variation** | Seeking diverse perspectives | Vary on key dimensions; document selection rationale |
| **Theoretical** | Grounded theory — let emerging theory guide | Continue until theoretical saturation; document saturation evidence |
| **Critical case** | Testing a theory in its strongest/weakest case | 1–3 cases; justify why these cases are critical |
| **Snowball** | Hard-to-reach populations | Document recruitment chain; note bias toward connected individuals |

> **Saturation evidence**: Document when no new themes/codes emerge; show code frequency stabilization; report final codebook size.

### Step 4: Data Collection Plan

| Design Type | Key Collection Decisions |
|-------------|------------------------|
| **Survey** | Platform (Qualtrics/Prolific/MTurk), attention checks, response quality, expected response rate, incentive |
| **Interview** | Semi-structured guide, recording consent, transcription method (verbatim), memoing plan, member checking |
| **Experiment** | Randomization mechanism, manipulation check, blinding, debrief, exclusion criteria |
| **Case study** | Data source triangulation plan (interviews + documents + observation), archive access, case database |
| **Secondary data** | Data dictionary, variable mapping, missing data, merge strategy, vintage/vintage effects |
| **Observation** | Protocol, field note structure, observer role (participant/non-participant), duration |

For each instrument, plan:
1. **Pilot testing**: With how many participants? What will you adjust?
2. **Ethical procedures**: Consent process, recording permissions, data security
3. **Quality assurance**: Inter-rater training (for coding), attention checks (for surveys), manipulation checks (for experiments)

### Step 5: Analysis Plan (Pre-specify)

Create an analysis plan that is executable and auditable:

#### For Quantitative Analysis

| Element | What to Specify |
|---------|----------------|
| **Primary model** | Estimator (OLS, logit, DID, SEM, etc.); DV ~ IV + controls |
| **Assumptions** | Linearity, normality, homoskedasticity, independence; how each is tested |
| **Missing data** | Listwise deletion / multiple imputation / FIML; justify choice |
| **Multiple testing** | Bonferroni, Holm, BH-FDR; threshold p-value |
| **Subgroup analysis** | Pre-specified subgroups; justification; interaction terms |
| **Robustness checks** | Link to `robustness-planner` (C3_5) |
| **Software** | Stata / R / Python; packages and versions |

#### For Qualitative Analysis

| Element | What to Specify |
|---------|----------------|
| **Analytic approach** | Thematic analysis / grounded theory / content analysis / narrative analysis / process tracing |
| **Coding strategy** | Inductive / deductive / hybrid; first-cycle + second-cycle codes (Saldaña, 2016) |
| **Codebook management** | Initial codes → evolution rules → stabilization criteria |
| **Memoing** | When and how; reflexive memos, theoretical memos, methodological memos |
| **Cross-case comparison** | Variable-oriented / case-oriented / mixed |
| **Negative case analysis** | How deviant cases will be sought and handled |
| **Saturation documentation** | How saturation is assessed and reported |
| **Software** | NVivo / Atlas.ti / MAXQDA / manual |

### Step 6: Validity / Rigor Plan

#### Quantitative Validity

| Validity Type | Threats | Mitigation |
|--------------|---------|------------|
| **Internal** | Confounders, selection, reverse causality | IV, matching, fixed effects; see `rival-hypothesis-designer` |
| **External** | Sample specificity | Discuss generalization boundaries |
| **Construct** | Measurement error, single-item | Validated scales, multiple indicators |
| **Statistical conclusion** | Low power, violated assumptions | Power analysis, robustness checks |
| **Common method bias** | Single-source data | Temporal separation, marker variable |

#### Qualitative Rigor

| Criterion (Lincoln & Guba, 1985) | Technique | How to Document |
|----------------------------------|-----------|-----------------|
| **Credibility** | Prolonged engagement, triangulation, member checking, peer debriefing | Log engagement duration; triangulation table; member-check responses |
| **Transferability** | Thick description | Provide enough context for readers to judge applicability |
| **Dependability** | Audit trail | Decision log; codebook evolution; memo trail |
| **Confirmability** | Reflexivity statement | Researcher positionality; how bias was managed |
| **Authenticity** | Fair representation | Multiple voices; negative cases; range of perspectives |

### Step 7: Preregistration (Optional but Recommended)

| Registration Platform | When to Use | What to Include |
|----------------------|------------|-----------------|
| **OSF Registries** | General social science | RQ, hypotheses, design, analysis plan, power |
| **AsPredicted** | Quick 9-question format | Data collection status, hypothesis, methods |
| **PROSPERO** | Systematic reviews | Protocol for search, inclusion, analysis |
| **ClinicalTrials.gov** | Clinical/health trials | Full trial protocol |
| **AEA RCT Registry** | Economics experiments | Design, outcomes, sample size |

Document the registration date and any post-registration deviations with justification.

## Outputs (Create/Update)

- Study design doc → `RESEARCH/[topic]/study_design.md` (use `templates/study-design.md`)
- Analysis plan → `RESEARCH/[topic]/analysis_plan.md` (use `templates/analysis-plan.md`)
- Data management plan → `RESEARCH/[topic]/data_management_plan.md` (use `templates/data-management-plan.md`)
- Instruments (optional):
  - Survey → `RESEARCH/[topic]/instruments/survey.md` (use `templates/survey-instrument.md`)
  - Interview guide → `RESEARCH/[topic]/instruments/interview_guide.md` (use `templates/interview-guide.md`)
- Prereg draft (optional) → `RESEARCH/[topic]/preregistration.md` (use `templates/preregistration-template.md`)

## Output Contract

- `DesignSpec`: write `RESEARCH/[topic]/study_design.md`.
- `AnalysisPlan`: write `RESEARCH/[topic]/analysis_plan.md`.
- `DataManagementPlan`: write `RESEARCH/[topic]/data_management_plan.md`.
- `Instruments`: write `RESEARCH/[topic]/instruments/`.
- `Preregistration`: write `RESEARCH/[topic]/preregistration.md`.
- Separate finding, interpretation, and implication in the final artifact.
- Do not invent citations, data, sample sizes, statistical results, or reviewer comments.
- Apply `references/academic-output-rubric.md` before finalizing scholarly prose or review artifacts.

### Method Diagnostics

- Produce or consume `RESEARCH/[topic]/design/method-diagnostic-report.md` and `RESEARCH/[topic]/design/validity-threat-matrix.md` when design choices affect inference, measurement, or analysis.
- Cover construct validity, internal validity, external validity, statistical conclusion validity, measurement validity, data leakage, missingness, confounding, and selection bias.
- If method details are insufficient, write explicit insufficient-input notes instead of guessing.

## Quality Bar

The study design is **ready** when:

- [ ] Design type selected with explicit justification and alternatives rejected with reasons
- [ ] All constructs operationalized with measurement source and validity evidence
- [ ] Sampling strategy specified with adequacy rationale (power or saturation plan)
- [ ] Data collection instruments identified or designed with pilot plan
- [ ] Analysis plan is executable (estimator, covariates, missing data, robustness)
- [ ] Validity threats identified with matching mitigations
- [ ] For qualitative: coding strategy, saturation plan, and rigor procedures documented
- [ ] Ethical review requirements identified (feeds `ethics-irb-helper`)
- [ ] Preregistration decision made and documented (registered or justified why not)

## Common Pitfalls

| Pitfall | Problem | Fix |
|---------|---------|-----|
| No design alternatives considered | Reviewer asks "why not X?" | Justify design choice against at least 2 alternatives |
| Power analysis missing or post-hoc | Undermines any null findings | Pre-specify; use prior effect sizes |
| "Convenience sample" without limitation | Readers question generalizability | Acknowledge; avoid causal claims |
| Qualitative sample without rationale | "We interviewed 12 people" — why 12? | Document sampling strategy + saturation evidence |
| Controls not theoretically justified | "Kitchen sink" approach | Each control needs a rationale |
| Missing data plan absent | Reviewer flags > 10% missing | Pre-specify handling before analysis |
| No pilot testing | Survey/interview flaws discovered too late | Always pilot with 3–5 participants |

## Notes

- This skill is not legal advice; for ethics/IRB packaging, use **ethics-irb-helper**.
- For implementation code, pair with **code-builder** (and **model-collaborator** when helpful).
- For downstream variable or dataset specifics, pair with **variable-constructor** and **dataset-finder**.

## Economics Overlay

### Activation
Apply this overlay when `study-designer` is used for an economics subject package, especially for empirical designs that may become causal, structural, panel, policy, DID, IV, RD, synthetic-control, or matching claims.

### Required Context
- Require the research question, paper type, outcome, treatment or shock, unit of observation, time period, comparison group, and target venue family.
- Require the estimand before the estimating equation, and state whether the paper seeks descriptive, causal reduced-form, structural, predictive, or theory-only evidence.
- Require the assignment mechanism, source of identifying variation, treatment timing, and data structure before recommending an estimator.

### Subject-Specific Procedure
1. Name the estimand, identifying variation, comparison group, and identifying assumption in separate sentences.
2. Match the design to the economics method family: DID, IV, RD, event study, panel FE, synthetic control, matching, structural, or descriptive.
3. For DID or event-study designs, specify treatment timing, anticipation, spillover, pretrend, staggered-adoption, and heterogeneous-effect risks before model choice.
4. For IV designs, state relevance, exclusion, monotonicity or LATE scope, first-stage diagnostics, and weak-instrument risk.
5. For RD designs, state running variable, cutoff, bandwidth logic, manipulation checks, and local estimand.
6. Record the main identification threat and the design feature or blocked diagnostic that addresses it.

### Reviewer-Risk Checks
- Check whether the design confuses an association, a policy comparison, and a causal claim.
- Check whether controls are post-treatment, whether fixed effects absorb the identifying variation, and whether standard errors match the assignment or shock level.
- Check whether the strongest alternative explanation would still survive the proposed robustness plan.

### Output Requirements
- The study design artifact must include `Estimand`, `Identifying Variation`, `Comparison Group`, `Assumptions`, `Diagnostics`, and `Blocked Checks`.
- The recommended model must be justified by treatment timing and data structure, not by convention alone.
- Missing pretrends, first-stage evidence, bandwidth diagnostics, or clustering logic must be written as gaps.

### Blocked Conditions
- Block causal language when identifying variation, comparison group, or assignment timing is missing.
- Block estimator recommendations when the data structure needed for DID, IV, RD, or panel inference is not specified.
- Do not upgrade descriptive economics patterns into causal interpretation without a named identifying assumption and diagnostic plan.
