AI Balances Scale, Trust, and Human Judgment at Navy Federal Credit Union

Jeff Mosier:

Hi, I'm Jeff Mosier. I'm the marketing content lead for the Infosys Knowledge Institute. I'm here today with Pam Piligian, the CMO and Senior Vice President at Navy Federal Credit Union. Pam, thanks for joining us today and making time.

Pam Piligian:

Great, thank you for the invitation.

Jeff Mosier:

Can you tell us a little bit about how AI is changing how your marketing organization operates?

Pam Piligian:

I think AI is a big enabler for all of us. And so it's changing us both in terms of content creation and things that we used to think were impossible, now we can scale. Navy Federal's a credit union, the largest credit union, serving 15 million members. So you can imagine, we try to do a lot of personalization at scale, but at the context of 15 million, we can't do 15 million versions of everything. But AI makes a lot of that possible, not to that 15 million level, but to a part that we can really use content to optimize against results.

It's also a big enabler for our analytics where we can get smarter or faster on what work is working and what work needs to either be revised or really put down. So I think in a lot of ways for us, it's really just a big enabler.

Jeff Mosier:

How have you seen it be effective as far as speed versus quality? I know that is a big issue with AI. It was a big issue before AI in fact. How do you see that balancing out?

Pam Piligian:

I'm glad you used the word balance because fast but not good doesn't really matter. We've got to have quality. And I think also for anybody in the banking business or services business overall, the personalization matters to get it right. So accuracy is still the tipping point of that scale. So if we have to be wrong to be faster, then it's not a path we're willing to take or one that makes sense. Particularly when we measure a lot of our metrics in terms of member trust. How do our members trust the information that we're giving them and the advice we're giving them? So therefore, we really think that that honesty and we treat that trust sacred, so we make sure that we're not doing anything in any way, shape or form with AI that would take out a ding in that trust.

Jeff Mosier:

How do you determine what you think is going to work? How do you know ahead of time that you're going to be able to strike that right balance with whatever tool or strategy you're executing?

Pam Piligian:

Well, we don't. I will say we do a lot of testing and learning. And we have a testing agenda that we talk about is always on. And so how do we apply the learnings from a small scale into a broad scale effort and really optimize against what we're doing? We don't look at it as we can't be 100% right for 100% of the time, but if we can be more right, more often, it does help us in terms of just making that connection that's so important.

Jeff Mosier:

Did you have to create that experimentation program or however you evaluate these tools and strategies, did you have to start that from scratch or did you already have an idea of how to do all of this?

Pam Piligian:

Well, it definitely has been a test and learn journey because we have a couple of tech tools that helped us with our website, always do a lot of AB testing. So we were able to scale from some of those foundational learnings to that. Also, we have an email platform delivery system that has some machine learning baked in. So we were able to take some of those learnings and build on it. But for us really, there always, always has to be a human involved in the AI implementation. And that's a critical part of the chain, if really honesty, transparency, and trust are on your scale, on your, if you will, scorecard on what's important.

Jeff Mosier:

Yeah, at Infosys we talk a lot about human in the loop. How difficult is it to know when to put that human in that process and create that human gate versus just being able to let the AI go and do what it needs to do?

Pam Piligian:

It's a great question. For us at Navy Federal Credit Union because we are a service business and we really put that at the top of the pyramid, the human's always in the loop. So for us, it's the question of at what level and how do you best use that human? So we talk about implementing both AI, obviously, as well as NI, natural intelligence. And that's what we really need to bring to the table to really make this truly an enabler for our business. So for us, it's not when to include the human because they're always there and that's one of our guiding principles, it's just to what extent and how.

Jeff Mosier:

What are some of the biggest surprises that you found with your implementation of AI or experimentation, even about things that you tried and maybe discarded because they didn't work or did not appear like they would work?

Pam Piligian:

I'll give you one that's maybe a little bit dated. But we decided that we'd send everybody a birthday greeting. And it's like, who doesn't like to hear happy birthday? And let's just say without revealing who they are, some people don't, and some people didn't like the fact. So I think we just, in some of those cases, we've had a little bit of a stop. And before you make assumptions, test some things because what seemed like a really, this is easy, everybody would love to hear happy birthday, not so much. So I think for us, part of it is just taking that moment to stop and say, what we're doing, how does it link to the strategy, is there an AI application that could help us scale it, how could we maximize our learnings from a test environment earlier on before we scale?

Jeff Mosier:

It sounds like that's referencing what's becoming a classic AI understanding, which is just because you can do it doesn't mean you should.

Pam Piligian:

Correct, correct. Just because you can doesn't mean you should. And it also doesn't mean you should for everybody. Targeting still matters. I think everybody who's really been in the marketing game a long time knows that. And I think AI just reminds us because sometimes we get over our skis because we think about this is such an enabler to go fast quickly and efficiently, and as you said earlier on, it's looping back to the should you.

Jeff Mosier:

Yeah. You mentioned speed, you mentioned efficiency. What are some other benefits or value that you've gotten from AI so far in marketing?

Pam Piligian:

One of the things that we're just doing now and we can do more of is quality analytics. And as simple as, I mean, we do a lot of email, email's a very effective part of our member marketing communications, so we send a lot of it. But then identifying simple things like what are the 10 best factors that the most effective emails share in common? That analysis would've taken a month before. And now being able to get that analysis pretty quickly, we're able to apply those learnings to next. So it's really helped us on the analytics side get smarter, faster about some of our past efforts.

Jeff Mosier:

Are there some use cases that you found that have not worked out for you? Where is AI disappointed for you? You mentioned the emails for birthdays, but some of the uses, are there some areas where you just found AI just is not going to work for us, at least now?

Pam Piligian:

I will tell you, we did some, and are still experimenting with content creation, and we found that overall AI was really helpful in creating a first draft if you will. When we tried to use AI upfront in terms of creation, we didn't get things that were as much on brand, we didn't get things that were really speaking to our member base that we really know really well. So we ended up kind of pivoting and saying the best way for us, and I'm not saying this is true for all brands, is really about using AI to create some of those first drafts. And then the versioning often is still created by a human.

Jeff Mosier:

Do you think that's going to change in the future as the tools and technology get better and more sophisticated?

Pam Piligian:

I think it will get better as it learns. Because it's like everything else with the data equation, garbage in, garbage out. So as we get better about what we put in the query and we get better at being better query engineers or prompt writers, we'll get better output. But that is a process too in terms of learning how that works best.

Jeff Mosier:

Yeah. And eventually it'll just shrink that amount that the people have to do at the end. But you still think that there's always going to be room at the end for that person to make sure before it goes out that it is brand appropriate and on target, right?

Pam Piligian:

Once again, I can't speak for all brands, but I think for us, yes. And that's really just because we're not using AI to replace humans, we're using AI to help humans. So I can't see an equation where we would take that human out of the mix despite how smart it gets if you will. So we'll see as it evolves and we learn more, but that's kind of where we're at now.

Jeff Mosier:

Is AI changing at all how you're measuring marketing outcomes or marketing benefits?

Pam Piligian:

It is, but we also are there trying to make sure that we still are using common sense. I tell people, like for my team, we always talk about metrics that matter. We can measure a lot of things, but sometimes it's not the metrics that matter, it was the emotion, it was the connection, which is still softer side of the equation. So for us, it's really trying to figure out how do we balance those.

And then also, kind of back to your point about what we can do and what we should do, is the other part of that equation, that triangle is win. AI is incredibly fast, but do we sometimes need time in market for things to seed, time for our consumers and our target audience and for us, our members to see things and connect. So that time we can't always speed up the process, particularly for somebody that may be going through a very long journey on buying a new financial product or service or a home for the first time. We can't speed that up so how do we use the learning along the way to make that smoother and easier, but knowing that the time is still going to be the ... it's really in the hands of the member and their decision on how fast we can go. So we're not pushing that to go faster than we should.

Jeff Mosier:

How is AI affecting your marketing strategy or changing what your marketing strategy is?

Pam Piligian:

The strategy is very similar, the tactics in terms of what we implement is what's changed. We've always had a committed focus to testing and learning, and sometimes that's been slower than we like. And AI has really helped us speed up that agenda, and we've been able to learn faster and then apply it to current programs.

For our member base, we celebrate, we call them two tent pole events, with Military Appreciation Month in May and then Veterans Day in November. Sometimes you can't move when those happen, they're calendar events. So before AI, we really were learning from this year what we were going to implement next year. And now we've been able to put some real time testing in the market during those tent pole events and then still optimize in the same cycle. So in ways like that, it really has helped us be more effective in our marketing.

But that said, it hasn't changed the strategy. The strategy for us there is still about connecting with our members, connecting with our prospects and show them that this is a moment that matters. And then talk to them about how we're going to recognize it.

Jeff Mosier:

Do you think it'll ever get to a point where it starts affecting the strategy?

Pam Piligian:

I think it could. I think when you think about all good strategy is built on a solid data foundation. So I think to the point where your data gets better and more robust, it could impact the things that are possible and in the timeframe that it's possible.

Jeff Mosier:

What are some of the biggest challenges you're facing with AI, what's either holding you back or things that kind of keep you up at night?

Pam Piligian:

Well, it's an exciting time because we've got a lot of possibilities. It doesn't work for everything and it doesn't replace everything. So the key is just trying to figure out ... we test some things that don't work and we test some things that do work. And so we still are very much in a do, learn, grow environment. And like I said, in a regulated business and in a service business where that human contact really matters, we've got to make sure that we remember those foundational things when we're looking at what we do.

So trying to apply it in a responsible way to get better at what we do, and I think that journey will be a ever changing one. I think about things like search as simple as when you are optimizing to one major player that had 80% of the share, that I won't say was easy, but you kind of have to figure out some of the algorithms. But when you're optimizing to five or six different players to really get to a monopoly, it does change what you have to do.

Jeff Mosier:

Do you feel like any of the challenges are changing? Are they getting easier? Are they getting harder because of so many more tools and so many more options, so many more ways to use AI?

Pam Piligian:

C, all of the above. I think some things are getting easier because it's faster and it's real time, as I mentioned. I think other things are harder because of that and because of all the possibilities. When I think about AI search, and there's four or five players you really have to take into account, and each one has a different algorithm in terms of what's weighted for content. So that's a simple example on how it's kind of complicated things. And I can think about other times where on the analysis side, it's just helped us get to better answers faster that we could apply quicker. So that's why I say it's C, all of the above.

Jeff Mosier:

How do you navigate all the challenges related to how quickly AI is evolving, by the time you get a tool implemented or new tactics implemented, it's already been replaced by something else or maybe 10 other things in the market, how do you keep up with it?

Pam Piligian:

Well, we're trying, I will say that. We're trying to keep up with it. And we're trying to identify the things that really matter. How could it help us instead of being what could we use AI to do? It's like, what could we use AI to do that are things we're already doing that we could do better and be more efficient at? Versus trying to bring online new capabilities. That can be limiting, but it also can help you make more progress sooner in smaller areas and a handful of territories versus trying to do too much too fast.

Jeff Mosier:

Yeah. Do you feel like some of the caution that you have to proceed with is sapping some innovation, is it making it harder to innovate in some of these areas because of the dangers?

Pam Piligian:

It definitely does, but it also makes it easier to innovate because things that you can test faster, that's the number one enabler, that has been for us. I do think sometimes the whole just, what do we do next? When before you were looking at one or two tools and then you're looking at 10 or 11, that's different. And it takes longer sometimes to get to the right tool and the right technique and the right process. It can be faster and something that also slows you down too.

Jeff Mosier:

How is AI changing how you approach hiring new people, upskilling, re-skilling, training, all of this, how does that change things?

Pam Piligian:

Well, we're not using AI as much for interviewing and hiring. We're doing some specific skills training on how people can use AI in their jobs. And have set up a council that allows people to present their own ideas on what they'd like to do with AI for their jobs. So we've used it that way to help make our employees part of the team that's more forward on these new solutions. But we're not using AI for hiring, others may be, but we're not.

Jeff Mosier:

What about training and getting people prepared to use all these AI tools and kind of approach marketing in a different way with AI capabilities?

Pam Piligian:

We're definitely doing training that way. And that is broader than marketing, that's enterprise-wide because we know that there's lots of applications across the entire credit union. So we're doing a lot of training on AI principles, different components too. Like we'll do data brick training, we'll do opportunities where there's like a challenge and people can apply different AI tools, so some case study work together. But most of those teams are cross-function teams, and that really has been helpful too in terms of getting a product owner, a channel owner, and marketing people together to think about how AI might help them solve a problem.

Jeff Mosier:

I know that change management is also often an issue with new technology or emerging technology, how is it with AI, are people on board with it?

Pam Piligian:

Yes and no. There are some people that are very excited, there's some people that are very apprehensive. And the truth is probably once again somewhere in the middle. It's not going to replace you, but it should make your job easier for you to do. And there's some people that are still wrestling with, what is that? And I think in some ways we all are. How do you create that balance that makes sense?

For us as a service brand, the person in the mix really does have the most significant voice. And the next voice is from our members and from our prospects and the community that we serve. So we're starting there. So you could say that may slow us down, but we think that's the only way that keeps us on track to make sure we're doing the right things.

Jeff Mosier:

What do you think are the long-term implications of AI in marketing, how is it going to transform marketing, big picture?

Pam Piligian:

I think we are all figuring out where AI can make us better, not just faster, but better. And so I think eventually it will help us, particularly for brands that care about personalization and personalization at scale, I think there's some real capabilities there. And I think on efficiency front, there's some nice benefits there. So I think it'll help us in a lot of ways get better at our craft.

Jeff Mosier:

Thanks for the great insights, I really appreciate you taking time to join us today.

Pam Piligian:

Thank you. It's great to talk about AI.