Videos
IBM on Scaling Enterprise AI, Agentic Systems & Governance
Insights
- The shift from POCs to production is exposing weak foundations, making architecture and governance critical to scaling AI.
- Fragmentation across teams, tools, and environments is slowing enterprise AI, requiring standardized guardrails for agent-based systems.
- Smaller, enterprise-trained models are becoming essential to balance cost, performance, and accuracy at scale.
At MWC 2026, Eoin Coughlan of IBM explores how enterprise AI is moving from pilots to real-world operationalization. The conversation highlights how fragmented tools, inconsistent standards, and weak foundations are slowing progress as organizations attempt to scale. He discusses the growing importance of governance layers that can monitor agent-based systems in real time, ensuring accuracy, trust, and performance. The discussion also addresses the shift toward smaller, domain-specific models that offer greater efficiency and control compared to large, general-purpose systems. Eoin highlights how enterprises must strengthen their foundations, align architectures, and adopt disciplined operating models to unlock AI value at scale.
Eoin Coughlan:
I think now we've come to the point where we're going, moving from POC to pilots into operationalization. And that's where some of the challenges come in, where people understand that actually the foundation and framework that they need needs to be stronger in order to move to the next step. Otherwise you don't get the acceleration because you don't gain confidence in the overall structure.
Enterprise AI challenges
Eoin Coughlan:
The first one I think is the fragmentation, different departments. They work with their own tools on their own environments, and they create their own use cases in their own way. Some adhere to certain security policies, certain APIs that they build. But when you put that together as an overall kind of interactive agent to agent discussion, it completely changed because now you need to have far more guidelines and guardrails about how you can build these particular agents so that you can give them the autonomy that they need, while still maintaining the trust and governance across that.
Autonomous AI needs control
Eoin Coughlan:
So what you now have is multiple agents communicating using tools with autonomy. So you need to have a governance layer which is looking at that technology in real-time and judging the outputs that it’s created. So you're looking at the input data to the quality there. You're looking at the outputs that are created, making sure that those models don't need to be retrained, retuned, in order to keep the accuracy level. That’s what we're seeing now is that we're moving on to evaluation where we're looking at the accuracy of the model, but also the performance of the models.
Smaller models enable scale
Eoin Coughlan:
Small language models, large language models frontier, it will be easy to say, let's go for a frontier model because we are strained on so much data and it knows so many things generally, but the challenge comes with cost and performance, right? So if you want to distribute your AI, then what happens is you want to have smaller models which are built on your enterprise data, gives you more accuracy, but also keeps the cost of the control.
AI is already delivering value
In our organization, our CEO made it an imperative that every department needed to look at how they could improve their productivity using tool sets that we have. It’s important for us to use our own technology in order to see the challenges that our customers have using the technology and the value that we get. So we did that across our business. We now have over $4.5 billion worth of productivity in the last two to three years. So we've seen the benefits of it. So we absolutely believe this is, you know, a fantastic thing.
