Jeff Dean: The 1% Rule for Building in AI

The evolution of AI engineering is shifting from scaling model size toward creating automated loops where systems improve themselves and perform complex, long-running tasks through "context engineering." Success now depends on identifying high-value domains and optimizing for data movement efficiency rather than general compute power.

• Automated loops — Machine learning systems will increasingly improve their own capabilities by decomposing tasks and running fast, repetitive experimentation cycles [02:00].

• Energy bottleneck — Moving data inside processors costs thousands of times more energy than the actual calculation, making optimized chips essential for efficient inference [12:16].

• Context engineering — Builders should focus on orchestration, retrieval, and memory tools to guide models, rather than just training larger parameter sets [16:46].

• Agent reliability — To prevent long-running processes from failing, use multiple agent paths and evaluation layers to search for the most accurate solution [23:46].

• Domain focus — New ventures gain an edge by tackling niche scientific or engineering problems where custom data sets outweigh general model knowledge [26:34].

• First principles — Re-evaluating core industry assumptions—such as designing systems that function despite high hardware error rates—can unlock radical architectural advancements [38:04].

• Simulation speed — Creating models that approximate complex simulators can accelerate scientific discovery by executing tasks millions of times faster than traditional methods [45:14].

• How does an automated experimentation loop enhance machine learning system performance?

• Why is batching used in training to mitigate data movement costs?