Why AI Tokens are so Expensive - Computerphile
Computerphile · 25:22 · 2 weeks ago
Agentic AI is costly because these systems are fundamentally inefficient at managing long-running tasks. They operate by re-processing the entire history of a conversation—including every instruction, previous thought, and file content—every time they generate a single new word, causing token usage to grow exponentially as a task becomes more complex.
- Auto-regressive processing — AI models function by generating text one piece at a time; to create the next word, the system must re-process the entire conversation history from the start .
- Exponential compounding — As an agent operates, the context grows with every step, forcing the model to re-ingest thousands of tokens repeatedly just to generate a single new output .
- Agentic workflow — Coding assistants require multiple cycles to complete a task, including:
- Internal "thinking" or planning phases
- Automated tool usage for reading external files
- Incremental code patching
- Final verification summaries
- Resource intensity — A task involving multiple files can consume over 50,000 tokens rapidly, far exceeding the usage of a standard chatbot .
- Billing shift — Services are moving from flat fees to token-based pricing, exposing high operational costs that were previously hidden by company subsidies .