# Case Study

This directory includes some case analysis. We compare the both method(grep + Claude Context semantic search) and the traditional grep only method.

These cases are selected from the Princeton NLP's [SWE-bench_Verified](https://openai.com/index/introducing-swe-bench-verified/) dataset. The results and the logs are generated by the [run_evaluation.py](../run_evaluation.py) script. For more details, please refer to the [evaluation README.md](../README.md) file.

- 📁 [django_14170](./django_14170/): Query optimization in YearLookup breaks filtering by "__iso_year"
- 📁 [pydata_xarray_6938](./pydata_xarray_6938/): `.swap_dims()` can modify original object

Each case study includes:
- **Original Issue**: The GitHub issue description and requirements
- **Problem Analysis**: Technical breakdown of the bug and expected solution
- **Method Comparison**: Detailed comparison of both approaches
- **Conversation Logs**: The interaction records showing how the LLM agent call the ols and generate the final answer.
- **Results**: Performance metrics and outcome analysis

## Key Results
Compared with traditional grep only, the both method(grep + Claude Context semantic search) is more efficient and accurate.

## Why Grep Fails

1. **Information Overload** - Generates hundreds of irrelevant matches
2. **No Semantic Understanding** - Only literal text matching
3. **Context Loss** - Can't understand code relationships
4. **Inefficient Navigation** - Produces many irrelevant results

## How Grep + Semantic Search Wins

1. **Intelligent Filtering** - Automatically ranks by relevance
2. **Conceptual Understanding** - Grasps code meaning and relationships  
3. **Efficient Navigation** - Direct targeting of relevant sections
