Introduction
Have you ever written a test strategy? Do you know why they’re important, or even what should be included in one?
In this blog we will explore the different scenarios and implementations for test strategies that could help you in your own context.
Testing can sometimes be a difficult craft to articulate and explain. A strategy is a high-level overview of everything that we look to cover.
A strategy is a long-term plan of action, the key word being “long-term.”
Gregory, Janet; Crispin, Lisa. Agile Testing Collection, The (Addison-Wesley Signature Series (Cohn)) . Pearson Education.
Many of you may have been part of interviews where this is a hot question too. Therefore, It is vital to understand what the testing process is within your teams. It is also important to have a strategy the team can follow to enhance a product’s quality.
In this blog myself, Laveena Ramchandani and Chris Armstrong, will take you through our experiences and help you with creating a smooth process when it comes to creating a test strategy and explain its importance. At the end of the day we’d rather all have a common understanding of quality and testing approaches.
Test Plan or Test Strategy?
Before starting it’s worth clarifying that both test plans and test strategy are two different things.
I remember a few years back when I interviewed for a role, I was told to work on a testing strategy for the team. I was confused initially as to why there is no testing in the first place! Then I started working on how to bring testing into the team and how we can test the product much more effectively and efficiently. I started from the very basics of what is a testing strategy with a google search. I realised I had been doing bits and bobs of the strategy in the past but calling it something else like ‘Definition of done’ or a ‘test plan’ but this was getting really confusing for me as a strategy is not the same as a test plan. Also, this is a great interview question as to what are the differences between a test plan and a strategy.
Once I researched and recalled from past experience I started writing a test strategy for the product. Things started to be more clear as to what we expect in terms of quality of the product and the testing processes.
Things to include in a test strategy:
This is something testers can have a healthy discussion about within the team. Understand what is important to the team and then draft something from there. However if you feel like following a organised step by step process then the following could be beneficial to be added into your strategies;
- The scope and overview of the product under test
- The test approach, what and how are we going to test (Unit, Integration, Automation testing, System, Regression, Installation/Uninstallation, Usability, Load, Performance, and Security testing)
- Defect management process
- Pairing
- Training/ workshops
- The environments you will use to test
- Release process
- Any testing tools to be used
- Risk analysis
- Approvals or sign-off
Useful characteristics to have
As a tester it’s great to have the following attributes which will also help you drive positively towards a happy strategy:
Be Skeptical- they want proof that the software works as it’s supposed to.
Have an Objective- they make no positive or negative assumptions.
Be Thorough- they are committed to testing every area and possibility that has been determined necessary.
Being systematic and organized they can efficiently create reproducible scenarios to offer as proof of failure.
Some useful Test Strategy models for reference:
There is a wealth of test strategy models and concepts for you to consider when creating your own strategy. Here we will look at just three that we have found useful in the past.
Heuristic Test Strategy Model (Bach/Bolton)
In this model ‘The Testing and Quality Story’ is comprised of sections covering Quality Criteria, Project Environment and Product Elements, all informing Test Techniques to tell the story.
The beauty of this model is that it is full of useful reference points and questions to ask, to better inform your test strategy.
A glance at the Project Environment shows that it covers eight areas:
- Mission
- Information
- Developer Relations
- Test Team
- Equipment & Tools
- Schedule
- Test Items
- Deliverables
You can find the full model here: https://developsense.com/resources/htsm.pdf
In the book Agile Testing, Janet Gregory mentions the following to consider:
- Testing Practices
- Story Testing
- Solution Verification Testing
- User Acceptance Testing
- Exploratory Testing
- Load and Performance Testing
- Test Automation
- Test Results
- Defect Tracking Process
- Test Tools
- Test Environments
The more information that is contained in a document, the less likely it is that someone is going to read it all. Consider what information is really necessary for the stakeholders. Think about how often it is used and what it is used for.
Gregory, Janet; Crispin, Lisa. Agile Testing Collection, The (Addison-Wesley Signature Series (Cohn))
TMMi
The Testing Maturity Model (TMM) was based on the Capability Maturity Model, and first produced by the Illinois Institute of Technology. This model is a more formal approach to various different documents and processes that you could use, from Test Policies to Test Process Optimization.
The TMMi’s definition of a test strategy is ‘An organization-wide or program-wide test strategy that identifies and defines the test levels to be performed is established and deployed.’
This is inclusive of risk assessments, test levels and most importantly, context.
More can be found here: https://tmmi.org/tm6/wp-content/uploads/2018/11/TMMi-Framework-R1-2.pdf
The Benefits of a Test Strategy
There are great benefits in having a guide that can help you test better as a team. Especially when you land into a project where there is NIL testing done or developers are (or worse, not at all) testing the code they write. A test strategy can be a living document too, what I mean by this is your context is always evolving, new tools, new processes are coming into the testing market and your people and product are always changing, your test strategy should change in-line with that to remain relevant and useful. We are also aiming to shift-left in testing as much as possible, so it is a good practice if the document is kept open to change.
Another benefit is, a strategy can be a document that everyone contributes. It is not just the tester’s task to focus on this, software quality is the responsibility of the whole team and therefore so is the strategy.
Furthermore this information gives a team a clear direction about what tasks need to be completed and how they should be tested.
There are many analogies that are used as examples. One that I recall being given in a workshop was using basketball as an example. Say, we were playing the LA Lakers in the play-off, the strategy would encompass our high level mission to become champions, whereas a plan might be stopping LeBron James, how we do that, with whom and so on.
Who is the test strategy for?
As with many processes and documents, the intended audience will inform how it is presented, how granular it is and where it may live. In regulated enforcement, where audits can take place, it is common that a test strategy document will be required evidence for due diligence regarding approaches to quality. A test strategy for this audience is likely to be quite different to one that may exist in a SaaS organisation, where much of a strategy is implicit rather than explicit.
It is a good practice to tailor your strategy according to what is important and useful for your target audience.
To End…
It is a great skill to be able to write a test strategy for a product you are testing, but to be able to write one collaboratively with your team is even greater. It is an achievement because for a team it initiates the thinking process around what is required and what is missing. This would then ease a team with their testing scope and test coverage documented too.
Having a strategy in place also helps other team members and higher management to understand the state of the project at any given point. Furthermore the chances of missing out any test activity can be reduced as there is a guide in place for it.
Once you have a test strategy, keeping it alive, relevant and useful going forward is your greatest mark of success.
Are you up to writing a test strategy after gaining this useful knowledge?
