Stanford CS329A Self-Improving AI Agents | Part 7 | Self-Improvement and Deep Research Agents

Improving AI performance for complex reasoning and coding tasks relies on combining large-scale output generation with intelligent filtering and dynamic search tools that trigger only when the model identifies gaps in its own knowledge.

• Large-scale exploration — Systems improve problem-solving by generating many candidate solutions, as the likelihood of success grows in a predictable, linear fashion with the number of samples [18:07].

• Clustering methods — Since testing every output against external platforms is too slow, agents group similar code snippets and prioritize those that appear functionally distinct to maximize diversity in submissions [07:24].

• Scoring models — Moving from simple rule-based filters to learned reward models allows the system to predict with higher accuracy which candidates will pass hidden evaluation tests [26:03].

• Conditional searching — Reasoning models monitor their own internal process for uncertainty, initiating external web searches only when they encounter unknown facts or hit a logical wall [49:55].

• Result synthesis — Instead of inserting raw search data directly into prompts, models analyze and extract only the relevant findings, which prevents information noise from disrupting the logical reasoning flow [52:47].

• How does grouping similar solutions help when complete evaluation of all samples is impossible?

• Why does summarizing external search results improve model performance compared to providing raw documents?