Ahead in the Cloud: The Joy of Software Testing with Deb Costello of Spark New Zealand

Chad: Welcome to Ahead in the Cloud, where business leaders share what they've learned on their cloud journey. I'm Chad Watt, Infosys Knowledge Institute researcher and writer. Today, I am speaking with Deb Costello, the domain chapter lead for release testing and automation at Spark New Zealand, a major telecom and technology company in New Zealand. Welcome to Ahead in the Cloud, Deb.

Deb: Thank you, Chad. I'm so pleased to be here.

Chad: Deb, you've had a career in tech and telecom, and specifically in testing. For those who may not know or may not have the right definition, give me your explanation of testing, in one breath.

Deb: We make the customer experience amazing by ensuring that the product that we build works.

Chad: Perfect. How do you do that?

Deb: With a lot of science, research, and a lot of studies. So what we do is we work really closely with architecture and development. We go, "What are the changes that we are making? What are the weaknesses? What's the risk profile?" Then we behave like customers. We do anti-pattern. And then we just keep trying to find the weakness in what we want to show our customers. And then eventually, the business and testing and development all agree that we've got to a good place, and then we launch to the market.

Chad: Tell us a little bit about Spark New Zealand's reputation and missions beyond connectivity and communications.

Deb: First of all, we are the biggest telco in New Zealand. And we are homegrown, so we came from the post office into what was telecom and now is Spark. And we play a big part in our community. So we truly believe that everyone in New Zealand should have digital equity. So our purpose is to help all of New Zealand win big in a digital world. Sounds really easy, right? But it's really hard. Not everyone has access to laptops and phones. Not everyone has internet in their home, because they can't afford it. So Spark spends a lot of time, effort, not only as a company, but as a group of employees, trying to bring that to light.

Chad: That's really something. And you're right, I mean, it's one thing to connect most people, it's one thing to connect 90% of people, but that last mile, last kilometer, is really a challenge. And it leaves those who are left out at a big disadvantage. That's really something.

Deb: Yeah, because if you think of COVID, there was kids who didn't have phones and laptops to be able to attend virtual schools. So we helped out, we stepped up. We gave homes data, and we gave kids laptops so that they could not be left behind. Really important.

Chad: Wow, that's really something. So at Spark New Zealand, what are you testing right now, and why is that important?

Deb: So right now, we test all the products that go to market for our customers and for our internal users, so for our CSRs, our customer service desk, to how they do their job, how they interact with customers, customer records. If you purchase something on your phone and you want to buy some data, we test that. And why is that important? Well, we love our customers. We're also in a very competitive market. So if we don't get this right, our customers will leave us and go to someone who does. So it's really important that we make sure that we give our customers the best experience they can have.

Chad: In the telecom world we live in today, the products that you send to your customers are not boxed and shrink-wrapped and mailed. What is the nature of these products, mostly?

Deb: Our customers, as soon as they pick up that phone and they log onto the network, they actually need a data package so that they can go and do everything on their phone. They need voice, they need text. And of course, we need to build them. They need to be able to get to ads. So we have to make sure all that works first time, every time. And then when it gets to the point of production that it's seamless for our customers, that they don't know that we've changed something behind the scenes. We often laugh in a telco world ... I shouldn't say laugh, but we go, "We don't want to be in the media if we get it wrong." You yourself would know, when you pick up your phone, if it doesn't work, there is nothing worse. We can get quite angry about it, and people go to the media. So if there is no noise when we've released, we know we've done a good job.

Chad: Right. And I think as we've gotten better and more connected, we all have increased our expectations that things will be instantaneous. Higher expectations, higher quality service and products, kind of play into you.

Deb: There's a really awesome thing about working for a telco that people don't realize. Whenever there is an emergency, an event, the network will go down. And then telcos will go in and they'll fix the network. We have the absolute pleasure then of seeing people reach out to their loved ones and talk to them. Here in New Zealand, recently we had some major floods, which took the backbone of telco out. Working together with the other telcos, we actually put in cell sites that allowed people to phone their loved ones. And we got to see that on their faces. It is an incredible thing that we do. And we know it every day, that what we give our customers are connections to their friends, their loved ones, their families. And that's powerful.

Chad: And there's these artists, rockstar developers are rolling out these new products. So moving beyond, for lack of a better word, manual testing, how do you bring automation into testing to keep up, keep pace, catch up?

Deb: Great question. So recently at Spark, we reviewed the automation that we were doing. And we partnered with Infosys to do that, because you need people that do this around the world every day to be able to go, "How can we think and be different?" So we started off with a few key principles. One was we wanted an automation-first mindset. So what we're doing today, could we automate it all, could we automate part of it? And then, how could we get continuous testing? Because you can automate, but unless you can run it continuously all the time, everywhere, you don't get the benefit of it. And then how could we shift it left? So how could we give the developers more feedback faster on the quality of the code, so that context switching wasn't so much? And then how could we get to the point where actually we could tell the squad, "You can release when you're ready."

Infosys helped us with our tool selection process and we narrowed some tooling down and then we went into an experimentation phase. We did eight weeks of experimentation with two tools. And we put them side by side. We ran the same tests through them. What we worked out really quickly is, one, is we needed to code automated testing, not script it, not record it, but actually write code; be developers when we were writing our tests. So we started having to think about coding standards and pull records.

Chad: Talk to me a little bit more about the difference between scripting your automation and coding your automation.

Deb: For me, and this is just my Deb term, scripting means still you need to press the keys, you still need some manual work to be able to run the test. Testers can't automate to write tests, but they're usually heavier maintenance. They turn out to be quite brittle. And not everyone can run them anytime, anywhere. When we started treating our automation as code, we were able to put it in our pipeline. And we are now able to run our automated tests on any of our environments, and anyone can run the pipeline. And we don't need someone on their laptop pressing keys to do it. So now, our people aren't doing test execution. They're doing test design, test leadership, test thought management. The actual physical running of the test is done by a machine.

Chad: Wow. That's really something. Now, how critical is that when we talk about companies that have shifted things to cloud, companies that have multiple cloud providers and cloud systems interoperating in the background, do you account for that as well?

Deb: We do. So Spark is a really large company in New Zealand. Worldwide, we're not large, but in New Zealand we are large. We are in the business of data centers, as well as being a telco. But we live in a hybrid world, so we still use the big cloud providers. We use our own infrastructure. We still have bare metal. So when we write tests, we have to take into account how they are run and when they are run. By automating, it allows us to run many frequently. So now I can run the same test on bare metal that I can do on hybrid without a lot of overhead. Even more importantly, if you think about, as a telco, all the phones that our customers have, all the operating systems they have, now I can run tests on all of them. And it doesn't take me a month, it takes me hours. So we go to production with confidence.

Chad: Hey, that's terrific. So how do testers keep up, and how do you capture as many errors as possible?

Deb: A tester's job is not to find defects. That's the outcome of what we do. So we do find a lot of defects, but that's because we're interrogating software all the time to find the weakness before our customers find it. So if you turn that question on its head and go, "Actually, we're not here to find errors. We're here to find quality" ... We're here to be able to tell you, Chad, as the product owner, as the business owner, that the software that you want to launch to production, what state it will look like in production. Is it good enough for the customer?

Now, we might have an opinion and go, "It's really buggy," because testers and developers, we're always like, "Oh, it's not quite right. It's not quite right." But you might go, "Actually, you know what? Those three defects that are still not fixed, customers will be okay with that. I'm going to launch." And that's the power that we give our business, is we're not here to decide if something's good enough or not good enough or got lots of errors, undiscovered or not. We are here to say, "This is what it looks like today. This is the quality that will be in production, good or bad."

Chad: Wow, okay. So you can set that standard, as of the moment. And that's actually very interesting you say that, because we live not only in a developer's world, but in an agile, DevOps world, where the state of the art is today going to change tomorrow. So how do you find the best quality in new places that are constantly changing?

Deb: So testers are really good at wearing multiple hats. So we think like our customers because we are our customers. We think like developers because we are learning to code, if we don't already code. And so we bring the two together. So then you put on that, "I'm going to interrogate the software, I'm going to see where the limits are, where the risks are," but on top of that, with my scientific mindset, I'm going to record where I find defects, bugs, unknown behavior. And I'm going to analyze it all the time so that as we create new software, I'm able to go, "Well, the last time that this area of software was changed or software that integrated to this particular software, I know I found defects in this area." And that means I can trigger my testing to be more robust in certain areas. So you have to use that critical thinking mindset that testers bring to be able to do that deep dive and interrogate and also keep up. Keeping up with new technology is really hard, so we spend a lot of time learning and researching.

Chad: Learning and researching. Actually, since you raised that, what is at the cutting-edge right now? What are you learning and researching about?

Deb: Everyone talks about AI: "Oh, AI is really awesome." It is really awesome, but it's not quite there to write code yet, "yet" being the key word. It will be one day. So we are creating this bunch of coders to be able to take us into the future so that when AI arrives and it's good enough, our people will be able to adapt.

Chad: I think you're right. It's very interesting to think about AI. AI is good at producing results. I think it's good at catching errors and keeping you from making your mistakes in the code. It's good at solving for dumb mistakes, but not good at doing something smarter than you. We do now live in a data-saturated world. And telecom companies, that's always been the case, but it's even more so the case now. How should companies think about their data, and what's changed about that in this move to data everywhere, data on demand, and data being passed from system to system?

Deb: Yeah, it's really important. This is something I was thinking about recently. And I had the luxury of hearing our brand executive do a talk on our data, our customer data. And some of the things he said really resonated with me and made me really proud to work for Spark. Our customer data, it's a gift that our customers trust us with their data. So we have to be really honest about what we are doing with that data and what data we keep and what data we don't keep. And of the data that we do keep, what are we using it for, and why are we using it that way? And to have a really strong contract with our customers so that they know that we don't breach their security and their privacy with that data.

I myself, of course, I'm a customer of Spark. Not only do I work with them, but I'm a customer. I don't want them sharing all my data with you, Chad. I mean, I don't want you to know what I have for dinner on a Friday night, because I use my phone to make a takeaway order. So I want that data to be secure. And I love that here at Spark, we treat our customer data as an absolute gift to us, and that we have to be so respectful of it. And we know that, as a telco, we get hit a lot with people trying to access data. So we spend a lot of time protecting our customer data. And it's something that telcos do all the time. And of course, when we fail, worldwide media.

Chad: What does it look like when a company is excellent at testing?

Deb: So when a company is excellent at testing, first of all, you have happy customers, because the products that they're buying from you work. You have happy shareholders, because the customers are buying the products. More importantly, your employees are actually really happy, because you start being seamless. Code is created and deployed. Everybody's working together. Everybody is talking about quality, about the customer experience. And it's no longer a throw-it-over-the-fence mentality. It's, "We're all in it together."

Chad: What does it look like when an enterprise has neglected its testing capabilities and is lousy at testing?

Deb: Well, normally, the customers are leaving because the products that are being delivered are buggy. You've got a bit of a blame culture going on because you have developers and testers, operations pointing at each other, going, "It's all your fault. I didn't do it." If you don't have any testing capability ... And remember, testing's an activity, not a role. So if you've got developers who aren't even doing testing, imagine what that code would look like. It could be quite scary.

Chad: So I'm going to give you the testing magic wand. If I give you this testing magic wand, what's the one thing that you would change, perfectly and completely? Where do you point that wand?

Deb: That's a really hard question. What I would say is my testing magic wand, I would have developers realize, on mass, code is only good when it's working for the customers. So therefore, you guys should test your code more.

Chad: Very good, very good. This interview is part of our collaboration with MIT Tech Review, in partnership with Infosys Cobalt. Visit our content hub on technologyreview.com to learn more about how businesses across the globe are moving from cloud chaos to cloud clarity. I'm Chad Watt, with the Infosys Knowledge Institute. Until next time, keep learning and keep sharing, y'all.