Have you watched with horror as your gas bill has gone through the roof this year? Probably. Have you watched with horror as all the bills have gone through the roof this year? Again, probably! Have you equated it to your Test strategy? No, we bet you haven’t. You might be surprised to learn that they have got more in common than you think.  

When discussing Test strategy with clients we often use the analogy of the gas bill to explain three variables that interact to send that bill skyward. Just like with gas bills, there are things you can change and things you cannot. If you’ve found your Test budget has gone the same way as the gas bill recently, here’s how we suggest you could bring it back down.  

Let’s start with the gas analogy. Your bill is dictated by three factors: 

1. Unit cost: how much do you pay per unit for your gas? We can agree that this is currently the biggest variable due to global wholesale prices.  

2. Scope: how much gas are you using? 

3. Efficiency: how well are you using your gas? 

Nicely visualised by this cube, representing the overall cost of your gas:

Where possible we want to slice a section from each side of the cube, reducing its overall size. Let’s look at each side.  

Unit cost: what can you do about the price? 

Applying the analogy to software testing, the ‘unit cost’ is often the section of the cube that our clients have attempted to address. They may have reduced the unit cost of testing by offshoring, on-shoring, or moving suppliers and partners. All of these can reduce the price they pay per unit for testing. Sometimes there is the hidden and expensive cost of time and resource being spent on testing that could be allocated on more valuable tasks elsewhere. As this is the easiest step to take, most large organisations have already done this exercise, but there’s still two sides of the cube to examine.  

Scope: turn down the thermostat! 

The second area is scope. How much are you using? Right now, there is very little competition in the consumer gas market, so most people are now turning their attention to reducing their consumption.  Turn down the thermostat, shut the windows, turn the radiators down, insulate the loft; there are various ways to reduce the amount of gas you use.  

To reduce the scope of your software testing, you can invest in Quality Assurance, which will prioritise prevention rather than detection of issues. You can also shift testing to earlier in the project life cycle which should reduce the cost of defect rectification. We often work with clients to focus on defect-prevention and organise testing around the riskiest areas. Another option is to optimise test design to enable better coverage for a smaller number of test cases, like we did recently on a project when we found implementing a combinatorial test design approach led to a 35% decrease in the number of required tests.  

Efficiency: do you need a new boiler?! 

Once you have optimised the scope, how can you then perform more efficiently?  

Let us go back to our gas bill analogy. When we discuss efficiency in your gas usage, we mean the efficiency of your boiler measured by the amount of heat you produce for the amount of gas used. For example, whether you have an aluminium, or a stainless-steel heat exchange unit will influence how much gas you use. 

When it comes to software testing, you can and should look at reusing assets that already exist. Once we have finished a CQE project, we will have built thousands of test cases which can easily be reused elsewhere. The challenge is often that there is no one to handover to, so the next testing team – unaware of the previous project – assumes they need to create all those assets again. Continuity gaps like this present a lack of efficiency that incur a higher spend. It is also worth looking at automating testing tasks that machines can complete faster than people.  

In our experience, most organisations have acted on tackling unit cost and dabbled with efficiency, but they have not looked at reducing the scope of their usage. Where CQE adds value is consulting with clients on how to reduce their need for testing by reducing consumption up front. A good example here is our latest project with an international energy firm. As CQE has been involved in their HCM roll out from the beginning, we have been able to reduce the testing time by 40%, when compared to non-optimised projects. By focusing on defect prevention, we have driven up quality and our test failure rate in systems integration testing has reduced from 80% to 15%.  

Hopefully, you’ve found this analogy useful. Although the variables differ – generally it’s the unit cost of gas driving the overall cost higher, whereas it’s often the efficiency or scope that drives test spend up – there are many useful parallels we can draw.

If you want to discuss your test spend contact one of our experts here.