Videos
Enterprise AI at Scale: Insights from Mark Merhom, New York Life
Insights
- Scaling agentic AI requires robust MLOps and data readiness to support enterprise-wide adoption.
- Smaller, domain-specific AI models are more sustainable, effective, and easier to implement than massive foundational models.
- Agentic AI unlocks immediate value in automating repetitive tasks, generating insights, and enhancing everyday communication.
How will agentic AI redefine enterprise insurance at scale?
Recorded at the Infosys Topaz Columbia University Enterprise AI Center, this interview features Mark Merhom, Strategic Capabilities - AI & Data, New York Life. He explores how agentic AI is moving beyond pilots into scalable, secure enterprise adoption—emphasizing critical themes such as:
- Why MLOps and data quality are the foundation for scaling AI across 12,000 agents
- How smaller, domain-specific models offer a sustainable path for adoption
- Where agentic AI adds value through task automation, insight generation, and enterprise-ready assistants
Drawing on his experience in enterprise data and analytics, Mark highlights how businesses can balance innovation with confidentiality, security, and sustainability—providing insurance and technology leaders with a clear view of AI’s transformative role in shaping the future of the industry.
What are some of the concerns you have with implementing AI, and agentic AI systems, at scale?
Mark Merhom:
I think you said it right there, to add scale, that's the tough thing to do. We have been successful at building some models on the lower scale, but bringing it to market, bringing it maybe for our entire agent population, which is 12,000 agents, and having that many concurrent users using a tool of some sort, that's where a lot of the MLOps and data teams work has to come in to make sure that it's scalable, it's ready, and it's efficiently being used.
Our research shows that the era AI experimentation is over, and real business benefits are close at hand. From your perspective, what critical levers, like data readiness, must organizations pull to be truly prepared?
Mark Merhom:
Something we say a lot in the data world is junk in, junk out. And I think it's the same thing with AI. If we're not ready with the data that supports the use cases with the AI agents, then it's going to just hallucinate. It's going to give lots of bad results. And the worst part would be if people trust those results because they're getting it from an AI agent. So, I think it's very important to have the data ready, have the accuracy ensured, make sure that you're feeding it good quality inputs so that you can build on those use cases for the company.
Given the energy demands of large language models, do you see smaller, domain-specific models as a more sustainable fit for business?
Mark Merhom:
I think for sure you don't need a foundational, large scale model operating for every need in a company. The smaller, scalable agents, I think those can be more widely adopted because there's more specific use cases within an organization and you might just need it for that specific reason. So yeah, building those is a lot easier, less complex as well. So for sure, we don't need to build a large model at every use case, but these smaller ones will make things a lot easier to implement and are more usable. And I think the companies that lean into that strategy will be much more effective implementing AI and embedding it into their systems across than ones that don't.
What do you think will be the top three use cases for agentic AI in your industry?
Mark Merhom:
More broadly speaking, even outside of insurance, but where I see AI being used the most is one, automating tedious tasks. I think we can all agree that nobody wants to do any kind of repetitive work and we can leverage lots of what's out there today. We don't even have to wait for tomorrow for that type of automation. Two, insights generation, I think will be very important, being in analytics for a long time. It's great to even get, just ask an AI agent, what insights can you generate from this data and have a starting point to explore deeper into it. And it's been great at doing that today. So, getting some leads on insights generation can help you look and dig deeper into certain areas, which is awesome for us in that space today. And just honestly as a personal assistant. We all use email, we all use messaging, and you know, verbiage and communication can sometimes be hard, in the world of AI, it's so much easier to reword, to fix, to transform your messaging, and I think that's a tool that everybody in the company can use right away.
What does Enterprise AI really mean? How is it different from consumer AI?
Mark Merhom:
If you think about it, they're really the same thing. It’s just domain knowledge differences, right? I mean with enterprise AI, especially we have you know in New York Life the key insight there is that there's knowledge bases that are that we expose it to that are proprietary, they're internal to the company and that makes the people interacting with that model able to get specific insights about New York's Life's business that we want to keep internal and confidential. So there's a lot of security features to enterprise AI, but it's a very important thing to do because companies are not going to interact with the consumer side to protect confidentiality and security needs. So that would be the main thing. But other than that, they look a lot alike. It's just you're training on a specific domain that's not available to the public sector.
What do you think will be the top three AI trends in 2026?
Mark Merhom:
It's hard to predict where AI is going to go tomorrow, but I think some things for sure. It's going to be transformative. It already is. In the world today, a lot of the conversation went from AI is a nice to have to it's a must have now. So we're already at that injunction where AI is a must have and it'll be interesting as more adoption happens throughout the industry to see, okay, how reliant we've become on AI. Other than that, I think we're going to continue to move with speed, continue to move fast. We want to just make sure we're trying to keep up as best as we can with the innovation while also making sure that there are certain guardrails to protect us all from going off track in ways that we don't expect. I've heard many stories of using AI agents today where, you know, okay, you're trying to use it in a repeatable, scalable fashion, but what it produces as an answer today might be very different than next week. So I think we'll also see some stability start to fall in place to make sure that it's actually usable and trustworthy to do the job for a long time coming.
What did you take away from today’s session?
Mark Merhom:
One of the main things I've heard that I think I'll take away is we don't need to build AI agents with 99% accuracy. Get them to 50%, get them to 60%, and allow them to train and learn. We are in that phase right now where we're all learning. So if you wait to get to the 99%, you're going to be behind because you're not allowing the agent to actually learn through experience, which is what they're becoming really good at.
