# Instruction-Tuning

## Papers

### 2021

- (2021-04) **Cross-task generalization via natural language crowdsourcing instructions.** [paper](https://arxiv.org/abs/2104.08773)
- (2021-04) **Adapting language models for zero-shot learning by meta-tuning on dataset and prompt collections** [paper](https://aclanthology.org/2021.findings-emnlp.244/)
- (2021-04) **Crossfit: A few-shot learning challenge for cross-task general- ization in NLP** [paper](https://arxiv.org/abs/2104.08835)

- (2021-09) **Finetuned language models are zero-shot learners** [paper](https://openreview.net/forum?id=gEZrGCozdqR) 

  > FLAN

- (2021-10) **Multitask prompted training enables zero-shot task generalization**  [paper](https://openreview.net/forum?id=9Vrb9D0WI4)

- (2021-10) **MetaICL: Learning to learn in context**  [paper](https://arxiv.org/abs/2110.15943)

### 2022

- (2022-03) **Training language models to follow instructions with human feedback.**  [paper](https://arxiv.org/abs/2203.02155)

- (2022-04) **Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks** [paper](https://arxiv.org/abs/2204.07705)

- (20220-10) **Scaling Instruction-Finetuned Language Models**  [paper](https://arxiv.org/pdf/2210.11416.pdf)

  > Flan-T5/PaLM

### 2023 

- (2023-04) **WizardLM: Empowering Large Language Models to Follow Complex Instructions** [paper](https://arxiv.org/abs/2304.12244)

## Useful Resources

- [Instruction-Tuning-Papers](https://github.com/SinclairCoder/Instruction-Tuning-Papers) - A trend starts from `Natrural-Instruction` (ACL 2022), `FLAN` (ICLR 2022) and `T0` (ICLR 2022).
