Research Question

Applied social research often starts with a question based on the problem that needs to be solved. For software, business stakeholders want to solve the problem of releasing unknown quality software. Testers provide information that solves a particular problem. Therefore information objectives guide the testing mission. Identifying the information objectives, what stakeholders actually want or need to know, is the first step. This broadly includes wanting to know if the product is ready for release, or whether it needs more development time (to improve quality) or testing time (more information on quality). With this in mind, the question that is asked of testers is:

“Are there problems that threaten the value of the product to people who matter?” (Bolton, 2020)

This extends to the quality of the project too, because anything that makes development, testing or business analysis harder or slower, affects the quality of the software.

Investigate

Learn about the software under test. Perform background research into initial claims of quality, including what it’s supposed to do and what problem(s) it solves. Read documents and artefacts. Talk with business stakeholders and customers to learn about the problems, expectations, hopes, dreams and desires for the software. If any part of the software is available, investigate that too to learn what it is, what it does and what problem(s) it could solve, either directly or talking with developers. Use heuristic oracles to recognise problems when encountered, but also guide the necessary background research needed for effective testing.

“A good investigator actively searches for information, they don’t wait for it to be handed to them” (Kaner, 2020)

Seek out this background information wherever it may be.

Analyse

Understand how the product, feature or service works or delivers value by breaking it down, forming mental models that feed into hypotheses. Quality is relative to someone’s subjective ideas, thoughts and feelings. Identify them here and use them to model out the software and problem-space with various risk, logistics, coverage maps and test design techniques.

Hypothesise

Supposition new claims and stories of quality that can also act as charters to guide time-boxed testing sessions. Revisit existing hypotheses where evidence falsifies existing ones.

Experiment

Interact and observe with the software using methods and tools to see what happens. The goal is to find evidence that falsifies claims and hypotheses value and to support (and falsify) counter-claims of threatened value. Feed this new information into analyses, hypotheses and evaluations.

Evaluate

Infer and conclude: decide whether hypothesise of quality are accurate based on logical reasoning supported or falsified by empirical evidence observed from experiments. Form an opinions on research question to give final suggestion to stakeholders.

Action

Feedback  and report findings to further guide testing and to communicate quality to stakeholders as sound stories of quality and threatened quality. Include a final recommendation on the research question.