Videos
Humans + AI: 60 Years of Exponential Progress
Insights
- Exponential growth in compute, doubling every two years for decades, has made AI progress inevitable and transformative.
- When compute and data scale together, machine learning accelerates faster than any human-designed alternative.
- AI delivers the most value in complex, data-rich environments where knowledge can be compressed and interfaces automated.
- Agents act as connectors that turn powerful foundation models into robust, deployable systems through validation, orchestration, and governance.
In this video, Dr. James Fergusson, Dept. of Applied Mathematics and Theoretical Physics, University of Cambridge, traces 60 years of exponential progress in computing and explains why the combination of compute and data inevitably leads to rapid advances in artificial intelligence. He explores how scaling algorithms outperform handcrafted approaches, how agents turn foundation models into usable systems, and why building the “roads and bridges” around AI is now the critical challenge.
Humans + AI: 60 Years of Exponential Progress
Dr. James Fergusson:
I think this is the most impressive thing humans have done, technologically. We have come up with a technology and we make it twice as good.
Every two years and we've done it for 60 years. And here's this beautiful straight, now I've got transistors, it's not quite the right thing, but perfectly straight thing, even when we couldn't make computers faster, it was here, but we just made them more parallel. And the final thing ended up, still perfectly straight. So this is twice as good every two years, but the thing about exponentials that people don't think is if you keep going with them, it gets about 30 times better every decade. So if you look at what are we going to do 10 years in the future?
We're going to have 30 times as much compute. And there are things that were very difficult suddenly become easy. If you think about 20 years, it's a factor of 1,000. And if you think of the 60 years we've done it, computers have got a billion times better in 60 years. And there's nothing we've ever done before that's improved by a factor of a billion.
The Challenge: Turning Data into Value
Dr. James Fergusson:
I think this is the universal challenge that we're all facing, is that every industry has this exponential growth in data and has these huge data lakes forming that no one really knows how to get value out of yet. And part of today is hopefully working at how we can extract value from that data and allow you to run your businesses better and deliver more value to your clients.
Exponential Compute + Data → Inevitable AI
Dr. James Fergusson:
If you have exponential compute and you have exponential data, then machine learning is inevitable.
That's all machine learning is. It is a way of automatically using compute to understand data. And what we've seen is if you have these two things growing exponentially, you get super exponential growth in machine learning. So this is a log plot from the 50s through to, well, not now, but a few years ago. And you can see the size of the models and the power of the models is growing exponentially on a log plot. So this is the super exponential kind of growth, which is why we see the sort of explosion of what AI can do and the way we sort of wake up every morning and think, oh, what do I have to try and learn now? What new things come out overnight?
Scaling Algorithms Always Win
Dr. James Fergusson:
There's this bitter lesson of machine learning, which is that any algorithm that scales with compute will always win because it's powered by exponential growth. So even if now you can do something much better with some advanced mathematical fitting formula or some complicated thing, if it doesn't scale with compute, eventually it'll lose. Because machine learning, grows with compute. Compute grows exponentially. And you can't innovate at a rate of 30x every decade.
AI for Business Value
Dr. James Fergusson:
This is when you think about where would I want to use these kind of tools in my business, you think, well, I've got to have complexity and data. And then, you what do I want to do? Do I want to encode knowledge? Do I want to get all of my legacy systems that have data and I want to use AI to sort of look at all of them and compress that knowledge down into something? That then I can ask questions. And the other one is flexible interface. Do I want to use this as a way of talking to lots of different systems or of talking to my computer and getting my computer to do things automatically?
How Agents Are Built
Dr. James Fergusson:
This is the idea that you start off with these brilliant foundation models, have ChatGPT, Gemini, Anthropic, Stable Diffusion, Segment Anything, and our own Polymathic. And then you build these agents on top of them. So I think of agents as like special hats that you put onto a foundation model to say, you're not general anymore, now you're specific, right? You write code or you talk to this database or whatever you do. And you always sort of think, I always think of these as like the engines. We think of the Industrial Revolution, we've built a steam engine. Brilliant, but not useful until you connect it to things. And this is the connector, right? This is what we turn our engines into cars. So this is how we build our steering wheels, our tires, our brakes, everything else. Safety systems.
And that's what it looks like. So you say, I'm going to build agents, I'm going to have ones that talk to databases or pull data in. I'm going to have one that use all my tools, particular software packages I can use. I search webs, I build reports. You have ones that write the code. And then you have ones that keep an eye on what's going on, right? So this, if you're in a company, here's the manager. And then here are the teams working on different things. And the most important one is the validation team, right? So we've seen large language models are really, really good at criticising because they're trained on people and people are much better at criticising than they are at making things. And so if you put agents and say your job is to point out everything that's wrong with what everything else does, that's how you can build something that's really robust. Because you can just have it checking at every step is what you did actually right.
The Next Frontier: Simulation, Mathematical AI & Agents
Dr. James Fergusson:
We're interested in areas where we think it'll change the way we do science. Not so much, this is a better way of modelling something we already do.
But how do we actually change the way we do science or sort of have that step change, that fundamental change that we are seeing in other areas, right? So we think about how do we do simulations better? Lots of areas of science depend on simulations or essentially modelling. This obviously works, of industry does lots of modelling, but this is also, you could think, if you wanted to build a digital twin of your customers to do some kind of planning on your network, or digital twin of your network, how do I want to think about doing this change would affect my network, what kind of things could be better, how can I optimize it better? That sort of falls into this space, how to use AI to automate, this, and this is the foundation model kind of stuff. With the mathematical side, which is how do we understand how AI works and how do we build AI that works better? So you can think of this like we didn't build good heat engines until we had thermodynamics. We want to build the thermodynamics of how AI works. And then the agents part is how do we automate all of the things we do to make sure we could do science much faster or in a fully automated way.
Building the Roads and Bridges for Agentic AI
Dr. James Fergusson:
The key challenge I think at the moment is really how do we build cars, but also the roads and bridges, right? How do we get these brilliant models that we're building and make them actually deployable in a robust, safe way? How do we build the agents that allow them to do that, but how do we build all the connectors that allow those models to interact with everything else we want?
