Videos
Scaling Agentic AI: Insights from Columbia’s Vishal Misra
Insights
- Small language models tailored to specific tasks will replace one-size-fits-all approaches, driving efficiency and accuracy.
- Reasoning models that simulate internal monologues enable agentic AI to deliver more reliable and context-aware outputs.
- Enterprises must proceed carefully: read-based use cases of agentic AI will scale faster than write-based ones due to risks like hallucinations and data loss.
How will agentic AI evolve for enterprise use?
Recorded at the Infosys Topaz Columbia University Enterprise AI Center, this interview features Professor Vishal Misra, Vice Dean of Computing and AI at Columbia University. He explores where agentic AI is headed, highlighting three critical themes:
- Why small language models are emerging as a more practical, efficient alternative to all-encompassing large models
- How reasoning models enhance reliability by allowing AI to analyze situations before responding
- Where enterprises must draw guardrails—embracing read-focused use cases first while exercising caution with write-based automation
Drawing on his perspective as a researcher and technology leader, Professor Misra underscores both the promise and the perils of agentic AI. This interview offers business and technology leaders a roadmap to scale AI responsibly while unlocking new enterprise-grade value.
What are the top three AI trends you see for 2026 and beyond?
Vishal Misra:
Agentic AI is one of the big themes that is emerging now for the next couple of years at least. Within agentic AI, I think we will see a lot more usage of small language models. The trend has been so far large language models. So a language model that knows both about cricket and quantum gravity and mRNA. You don't really need all of that if you want to perform specific tasks. So I think small language models are going to emerge. The other thing that's going to happen is, again, this is a very recent trend coming out of research labs, is that reasoning models have become really powerful. Earlier, what used to happen with the large language models, I mean earlier, it's a very new field, is your pre-trained model contained all sort of the expertise. You would ask a prompt, and it will immediately give a response. Now what people have realized that letting these models reason makes them perform a lot better. So it's like an internal monologue that goes on through the models trying to analyze the situation and coming up with a better answer. I think reasoning models are going to take a big role in this whole agentic AI world that we live.
What concerns you most about implementing AI at scale?
Vishal Misra:
We have all been aware of this issue of these models hallucinating. And these hallucinations have improved quite a lot, but we are not there yet. When you talk about enterprise level efficiency, there was a time when engineering used to mean five minutes reliability, sending the man to the moon, things like that. We are not there yet with prompt engineering. So there's a lot of guardrails that need to be put in of what these models can do and especially in an agentic framework. So what I think is going to happen is that there are two kinds of use cases for agentic AI. One I would say is the read use case where these agents will go and fetch data, read data from various data sources, whether they are SAS API or database, and reason over them and provide insights, which today takes a long time for people to get, but if these agents can help. So that's a great use case. I think that will be implemented faster than the write use case. So the write use case is where these agents will go and write stuff to databases. There you have to be very careful. Giving them write access and you know, if something goes wrong, they hallucinate and they wipe out an entire database, then you're in big trouble. So I think write use cases will take a longer time to be implemented. Read use cases will take fast, will be faster. And that's what will build confidence in the enterprise that these things actually can do enterprise grade performance.
