Language models can generate plausible code that fails its tests. This thesis investigates whether internal attention patterns predict failure before generation finishes, allowing computation to shift toward more promising candidates. The student will build on the Prober repository to train and evaluate lightweight prediction models.