Incidental cues in a prompt can change which algorithm a model chooses by up to 100 percentage points, among solutions that pass the same tests. Standard benchmarks, which check only outcomes, cannot detect this. We study how context steers generated code, who decides what counts as correct when agents repair programs, and how to build LLM pipelines for code and SQL that can be checked.
Research Goal
Measuring and controlling what LLMs generate