Rafee Tarafdar on AI Fatigue, Token Guzzlers, and Slop

Jeff Kavanaugh:

I'm Jeff Kavanaugh here at the Infosys Connect conference in Los Angeles. Very happy to be here with Rafee Tarafdar, our chief technology officer. Rafee, it's great to see you.

Rafee Tarafdar:

Good to be here, Jeff.

Jeff Kavanaugh:

It's great to see you in person. Dive right in, because obviously AI and AI value is the topic of the day. What is it in your role looking after technology and direction for the company that you see as the, I'll call it the biggest current focus for Infosys and also for our clients in this area?

Rafee Tarafdar:

I think the biggest focus right now for unlocking value is largely to see how do we accelerate with speed.

Jeff Kavanaugh:

So speed.

Rafee Tarafdar:

Accelerate with speed, because I think that's when the value will get delivered. If you're able to actually push things into.

Jeff Kavanaugh:

So no more three year ERP implementations.

Rafee Tarafdar:

No more three year ERP So for this, again, the way I look at it is there are five anti-patterns that I see today, which are preventing a lot of this value unlock. One is most organizations are dealing with AI fatigue because there is a lot of...

Jeff Kavanaugh:

The pilots, the experiments.

Rafee Tarafdar:

And also the AI itself is changing so fast that you will, you know, for three years, it's like running on a treadmill with changes that are constantly happening, and it is not stopping. So unless you have a right way to deal with these new changes, able to identify value for the enterprise, and then be able to

Jeff Kavanaugh:

So you get this fatigue coming into your whole approach.

Rafee Tarafdar:

So that needs to be addressed, and there are ways to do that. The second is to also deal with a lot of these token guzzlers. Because today what's happening is…

Jeff Kavanaugh:

Token guzzlers. We have some new terms that are coming out here.

Rafee Tarafdar:

So what's happening is, see today, most of the agentic implementations are also not very optimized, which means we've already seen that some of them rolled out and then they had to pull it back because they're so…

Jeff Kavanaugh:

They're slow, they consume a lot.

Rafee Tarafdar:

A lot, so it's not very efficient. So we need to have a way to manage these token guzzlers to cost aware architecture so that you manage it well. The third thing is there's a lot of AI slop. Right, so a lot of what is generated may not be of value.Value in terms of either cost or revenue or other.

Jeff Kavanaugh:

Or accuracy.

Rafee Tarafdar:

Or accuracy as well.

Rafee Tarafdar:

So that needs to be dealt with. The fourth is, today most of these AI vendors are productizing a lot of capabilities. So which means a lot of commoditization is happening. Now when this commoditization happens, the way the enterprises can create value is by managing their business context and applying it to these AI models and agents to drive higher accuracy and value. So that's again something that

Jeff Kavanaugh:

So if they productize it, then the enterprise has to take the burden of tailoring it to their situation.

Rafee Tarafdar:

So that needs to happen. And then eventually in all of these, they also have to make sure that this also needs a mindset change, which means you have to come out of your comfort trap. For 20 years, I've done this way and we can't continue doing the same way. So you need to move to this exponential way. So I think if these five anti-patterns can be addressed with AI vitality, with your cost aware architecture, with a clinic based approach, with context and harness, and with an exponential mindset, then a lot of value can be unlocked.

Jeff Kavanaugh:

And you're calling them anti-patterns because they're patterns that aren't healthy.

Rafee Tarafdar:

Correct.

Jeff Kavanaugh:

Okay. Got it. A few years ago when we co-wrote The Live Enterprise, there was a chapter on digital runway and the architecture and the evolving. And one of the key concepts that still rings so true is you've got to have an evolving architecture.

Rafee Tarafdar:

Correct.

Jeff Kavanaugh:

How do you design an architecture to evolve for AI?

Rafee Tarafdar:

So when I said the first anti-pattern is AI fatigue, see one question is, now, how do I abstract anything that can change? And so if I look at the AI world, the abstraction has to be done at several levels. The first is you abstract models out, so that I can choose model based on the purpose, which could be a small mid-size frontier model, which could be any-

Jeff Kavanaugh:

Poly model.

Rafee Tarafdar:

Poly model, that's first. Second, the agent framework themselves are evolving, which means I need the ability to choose any agent framework.

Jeff Kavanaugh:

And is the agent framework by industry or is it some structural thing?

Rafee Tarafdar:

This is more on the tech implementation for agents. So this is like a Google ADK, a Lang graph, or you have the AWS strands. So it's more framework like that. Then you have agent run times, which run these agents. Then you have different AI platforms on each of the cloud. Then you have different business platforms, enterprise platforms, data platforms that have to be connected. So, across this ecosystem, using standards, using the philosophy of abstract, isolate, and standardize is what will enable that you ensure that everything is abstracted so that as and when things change, you have the ability to pick the best. But I should also say that because the field itself is changing so fast, you can't expect 100% abstraction. So some of it you will have to refactor because take for example, two years back we used to talk about prompts. Now we are saying, okay, instead of prompts we need to do skills. Obviously it's a different manifestation, but then these standards are evolving towards skills and that's how everybody is now building it. So as the tech changes, some of these refactoring and changes will also be required.

Jeff Kavanaugh:

Do you think, asymptotically, the pace itself will start to mature and stabilize? Or how long do you see it still being this not volatile but rapid and churning?

Rafee Tarafdar:

No see, I think if I go layer by layer, models are today more or less fairly stabilized. But if I go to agent layer, then there is a lot of evolution that is…

Jeff Kavanaugh:

Okay, so each of the different levels of the stack, layers of the stack, there's different levels of maturity. it's managing those, you say abstracting, but looking at them individually on their own maturity.

Rafee Tarafdar:

Correct.

Jeff Kavanaugh:

Okay.

Rafee Tarafdar:

Because see, I think you have to look at it and put them on a four dimensional map. Is the tech today very early and is still evolving? Second, is it where you're largely doing custom development? Third is where it becomes productized. Fourth is where it becomes commoditized and a utility. Right? So it's four layers of maturity.

Jeff Kavanaugh:

Four layers of maturity.

Rafee Tarafdar:

So when anything is commoditized in utility, then it's a lot more mature. You can plug and play change. Anything which is productized is a good way because you don't have to build it. Anything which is in custom, that is where it's still changing. So now you need to make a decision. Do I wait? Do I do something or not do something? And then there are things in research which you track.

Jeff Kavanaugh:

Well, you mentioned decisions. So imagine you're speaking to your peers, your counterparts out there in large companies who are grappling with this. Are they equipped right now? Are they struggling? What's keeping them up at night?

Rafee Tarafdar:

Yeah, I think, see, the struggle will be at least in two areas. So one, I think, is this whole mindset change. I think that takes a little longer. Second, finding the right AI use cases that can deliver value beyond productivity. That, again, is a bit of a struggle. The third is taking more and more of these things to production and having that velocity in the engineering process to do it. That again I think is a problem today.

Jeff Kavanaugh:

Now, the CTOs are probably more comfortable with the third, but those other two items, it's change management, it's the process, it's almost like it's a COO responsibility.

Rafee Tarafdar:

That's right. I think it's everybody's responsibility.

Jeff Kavanaugh:

It is, it is. But as far as people who have traditionally had that, it's almost like you need this superhuman person to sort it all out because if you have silos, it's not working.

Rafee Tarafdar:

No, one is that Jeff, but I also think... everybody in the organization has to undergo that mental change, that mindset change, because unless everybody changes, this change is not possible. So while the leadership can drive it, but then everybody has to go through that process.

Jeff Kavanaugh:

Where I was headed with it, maybe the superhuman is this whole capability compared to yesterday, tomorrow compared to yesterday. What is it though that an organization needs to do so that more of those folks can have that mindset shift?

Rafee Tarafdar:

I think this goes back to what we had talked about in The Live Enterprise book, which is about microchange. So I think they have to start with a microchange approach. Is start driving change in few teams, few areas, perfect it, drive it, and then keep scaling it across.

Jeff Kavanaugh:

Because it is tailored to their context.

Rafee Tarafdar:

Yeah.

Jeff Kavanaugh:

And if the future is not written yet, then there is no zero to one or zero to 100.

Rafee Tarafdar:

Yeah.

Jeff Kavanaugh:

You're incremental. What is it that people are getting wrong right now that they're keeping them from it?

Rafee Tarafdar:

I wouldn't say they're getting wrong. It is just that the process is taking little longer because of multiple reasons. Probably one could be the risk approach.

Jeff Kavanaugh:

Well, that's what I meant by getting it wrong is maybe they're not moving aggressively enough. Maybe they're not iterating enough.

Rafee Tarafdar:

Yeah, you're right. So I think the way you deal with the AI risks and manage it based on the different areas that are working, that sometimes takes a little longer. The decisions take longer, the approvals take longer, because of which the cycle gets extended. That is one. The second bit also is doing this right way for the right set of use cases. Because initially everybody's focus was on productivity. Everybody said, let's use it to drive productivity, be it at an individual level. Now we're moving it at a team level. But then we need to now move beyond that.

Jeff Kavanaugh:

If you had one thing that you'd recommend your counterparts out there to think about and to do, what is it that they should focus on over the next nine, 12 months?

Rafee Tarafdar:

I would just say democratize innovation and drive distributed innovation across the organization. And then focus on value.

Jeff Kavanaugh:

Value is your North Star.

Rafee Tarafdar:

Measurable, tangible value.

Jeff Kavanaugh:

Value is your North Star. Got it. Listen, I know we have to get back to the rest of the conference. Thank you so much for your time, Rafee. Always a pleasure.

Jeff Kavanaugh:

I'm Jeff Kavanaugh. Until next time, keep learning, keep sharing.