SOFTWARE TEST ENVIRONMENTS AS A SERVICE

Arta :
Hello everyone. Welcome back to QualiFYI. I’m your host, Arta Shita and I’m joined here today by Christine Bentsen, head of product marketing of continuous testing at Broadcom.

Christine:
Hi Arta.

Arta :
Hi. So excited to have you on our show.

Christine:
Thank you.

Arta :
Let’s get right into it. You are a thought leader extraordinaire of continuous testing. You all should check out continuoustesting.com. It’s fire. Tell us what is continuous testing?

Christine:
Continuous testing is the newest way to do testing. Traditionally testing was done as a time-boxed event at the end of the software delivery lifecycle. It was done by a team of dedicated testers, and it happened at the very end. Right?

Arta :Right.

Christine:
A time-boxed event.

Arta :
Right.

Christine:
Now, with the advent of new practices like DevOps and continuous delivery in Agile, that causes a bottleneck because it happens at the end.

Arta :
Right.

Christine:
What continuous testing does is it shifts testing and the idea of quality left into planning and Dev and right into production and using data from the application life cycle from the pipeline to continuously improve quality and to make everyone responsible for quality. This also allows you to find and fix problems as soon as they happen in the application delivery life cycle when they’re easier to identify and faster and cheaper to fix.

Arta :
Fascinating. You mentioned bottlenecks. You mentioned finding the problems right at the right time, but surely there are some common challenges that people face in continuous testing. Can you walk us through some?

Christine:
Yes, absolutely. So of course companies that are trying to do DevOps and continuous delivery, and that are having these bottlenecks, and they see that the bottleneck face testing, of course they’re like, well let’s do continuous testing. That makes a lot of sense.

Arta :
Right, no brainer.

Christine:
But they have trouble adopting continuous testing, and one of the common problems is that the tools that those traditional testing teams have been using, they’re typically built with a UI, they’re a little bit clunky.

Arta :
Right.

Christine:
They were built for different teams, right? Sometimes the licensing is restrictive, what they support is restrictive, and they just don’t work for the way that these Agile teams and Dev testers want to work.

Arta :
Mm-hmm (affirmative).

Christine:
What these folks want to do is they want to do things as code. They want to do it in their IDEs without having to switch to another application. They want it embedded in the CICD tool chain, right. And they want to work with open source.

Arta :
Yeah.

Christine:
All of these things do not work with those legacy testing tools. Then we also have some problems with continuous testing bottlenecks itself.

Arta :
Mm-hmm (affirmative).

Christine:
Right. Every year, Broadcom does a report with Sogeti, the continuous testing report, and we’re about to release the new one, and I’ll give you a couple of previews about what’s going to be in this one.

Arta :
Would love that.

Christine:
So year over year we found that companies are still struggling with continuous testing. They’re still struggling in a lot of the same areas. My summary of the report was everything is hard and we can’t figure it out. Right? But there are a lot of problems and we asked them, what are the chief problems that you have?

Arta :
Right.

Christine:
Things that they told us is they have trouble getting visibility into the pipeline.

Arta :
Okay.

Christine:
They have trouble assessing performance from the end user standpoint, like when you release the application under peak load or from global perspectives, what’s that going to look like to your end user? What’s the impact on your end user? They have trouble with test environments, right? They spend a lot of time trying to get those test environments right. There’s a lot of things that happen actually within… Oh I forgot one, one of the things that they all told us, many people tell us in the report that they’re doing continuous testing, but then when you drill down into it, they say they’re having trouble achieving testing in Sprint, which is continuous testing.

Arta :
Right.

Christine:
There’s a lot of disconnects, and like I said, a lot of companies are really struggling with making continuous testing work. But back to the test environment piece, right?

Arta :
Yeah.

Christine:
So this is a stat that we see, not only in this report, but I see it in lots of different reports, how much time and effort companies are spending trying to get test environments on demand, spin them up and spin them down when they need them.

Arta :
I’m so happy you bring that up because that’s exactly what we’re into. How important then is automated environment provisioning?

Christine:
Drilling down into that report a little bit more, and again this is a sneak preview into what is coming soon.

Arta :
Yeah. I feel special.

Christine:
Yes. The respondents told us that they’re spending 42% of their time, their team’s time building, and maintaining, and decommissioning these test environments.

Arta :
Wow.

Christine:
Right? If you think about how much time is being spent on this, if you took that time and you turned that effort into innovation-

Arta :
yeah.

Christine:
-or doing other things that were adding-

Arta :
Value.

Christine:
To your bottom line.

Arta :
Yeah.

Christine:
Right.

Arta :
Right.

Christine:
What would that do in your organization? How would that change your organization? It would be fantastic, right?

Arta :
Right, absolutely.

Christine:
And so another thing we asked them is what would help you achieve more continuous testing maturity? 40% of them said automated test environments.

Arta :
Well, did you tell them to hit up Quali?

Christine:
We can do that.

Arta :
Okay, great. Perfect. It sounds like there’s a lot that’s going to be happening in the future of continuous testing then if you’re uncovering all of these issues and bottlenecks, can you give us a glimpse at what we can look forward to?

Christine:
I’m sure that you have heard that AI is a big thing.

Arta :
Oh, huge.

Christine:
Yes.

Arta :
Yeah.

Christine:
Everybody is talking about AI.

Arta :
Yeah.

Christine:
And of course we’re going to have to test AI systems.

Arta :
Right.

Christine:
That’s really big. But not only are we going to have to test AI systems, but the technology of AI is really applicable and going to be fantastic for continuous testing. Instead of running all of the tests all the time to ensure quality, you can now have AI help you identify which tests you need to run in which order. It can help synthesize data from across multiple applications that you have in your organization and say, ah, here are some problems that you could address to raise quality. There’s just a lot of different things that AI can help you with to make your test cycles more efficient and more accurate, and so you use resources more wisely, and ultimately you improve quality.

Arta :
Right.

Christine:
But before we do that, just stepping back, the next big thing, I think, to make continuous testing more of a reality and to help more companies achieve that maturity is we need to make continuous testing easier to adopt.

Arta :
Okay. How do you do that?

Christine:
Yeah, so you have all these companies that say, well, continuous testing looks good, but we’re not able to do it. Making continuous testing easy, having an integrated platform, having it work in the UI, and also for shift left teams.

Arta :
Right.

Christine:
You have one application where you can share reports, you can share services, because you’re going to shift left, but probably not all at one time.

Arta :
Mm-hmm (affirmative).

Christine:
So some of your teams are still going to be doing traditional testing, being able to support open source, having all of the applications or all the capabilities that you need at your fingertips, and I know a place that you can get an application like that.

Arta :
Oh wow, I wonder where that could be?

Christine:
That’s what we really want to do is make continuous testing easy for organizations to adopt.

Arta :
And with quality, because you mentioned that word quite a few times.

Christine:
Exactly. It all comes back to your business outcomes.

Arta :
Right.

Christine:
We’re not doing continuous testing just to do more testing faster.

Arta :
Right.

Christine:
We’re doing it to help you achieve your business outcomes, and those business outcomes are being able to respond to marketplace opportunities, being really nimble, and being able to do that with confidence, and the way to do that is through continuous testing.

Arta :
I love that. I feel inspired and I’m not even a tester.

Christine:
I know. I know. There’s a lot coming out, and being able to achieve that quality and speed, it’s going to change the way we do business.

Arta :
Yeah. Oh, I’m so excited and I can’t wait to see the report first of all, and to dig into it. It sounds fascinating.

Christine:
Well thank you. Yes. That’s going to come out in March.

Arta :
Okay.

Christine:
We would love to share it with you and we’re probably going to do a road show, and so-

Arta :
Fantastic.

Christine:
-I hope that you can attend one.

Arta :
Oh that would be awesome. I’ll be in the front row.

Christine:
Thank you.

Arta :
Well, thank you so much for your time today, Christine. It was so much fun talking to you as always, and learning more about continuous testing.

Christine:
Thanks Arta.

Arta :
Visit Quali.com to learn more about Environment as a Service.