AutoGrad Changed Everything (Not Transformers) [Dr. Jeff Beck]

Current AI development is stuck on scaling predictive language models, but real intelligence requires systems that act like the human brain: modular, object-oriented, and grounded in physical reality. By shifting from giant, monolithic networks to collections of small, causal models, AI can better handle novel situations and maintain an accurate understanding of the world.

• AutoGrad's impact — This technology, not the transformer itself, is the true reason for the recent boom, as it turned complex math into a manageable engineering problem [20:21].

• Bayesian brains — Humans operate optimally by ignoring irrelevant data and combining sensory inputs based on their reliability rather than processing every piece of available information [03:19].

• Language limitations — Using text as the foundation for intelligence is flawed because self-reported human explanations of behavior are often unreliable and disconnected from actual thought processes [35:24].

• Modular architecture — Future systems should consist of thousands of small, independent models that represent individual objects, allowing them to be swapped, combined, and updated without retraining the entire system [42:24].

• Handling uncertainty — When robots encounter unknown items, they should track "surprisal" to identify gaps in their knowledge and fetch new models on the fly instead of hallucinating answers [55:45].

• How does tracking "surprisal" help an AI agent interact with new environments?

• Why is language considered an unreliable foundation for grounding intelligence?