When testing, expectations are a way of assessing the quality of a software product, system or service. If it meets expectations, it probably delivers the desired value, if it doesn’t meet expectations, it probably fails to deliver. Any stakeholder can have expectations about the software but it’s typically thought of it terms of customers or end users. Testers identify important expectations and focus on where they may not be met as this could highlight a problem worth investigating further.
However something can not meet expectations yet still be valuable or good quality. Expectations may be initially low and then exceeded or expectations didn’t exist in the first place and any value is a nice surprise. When testing a software product, system, service or feature, a tester asks themselves or others:
- What are the expectations here?
- How are they not being met?
- Is there still a problem here (or even additional value)?
We recognise a problem with quality when experiences are inconsistent with expectations in regards to some model. Particularly when it results in feelings of disappointment. #softwaretesting #qualityassurance pic.twitter.com/m9riZKyedC— Ben Fellows (@BenGTF46) August 3, 2021
Expectations come from consistencies between related things, in this case what is observed or experienced and a subjective mental model held by a stakeholder. Mental models come from many different sources so inconsistencies with expectations come in many different forms. Any of these model inconsistencies can be used as an oracle to evaluate the quality of software and whether there’s a potential problem that needs reporting.
|Expectations are mental models that may come from one or more sources||Problems may occur when experiences are inconsistent with those mental models|
By applying inconsistency principles with different mental models, testers can:
- Identify potential bugs in the first place either knowing what to look for or where to look for it
- Better evaluate whether the bug is actually a threat to value
- Give weight to bug reports by describing where the unmet expectations came from
For a useful list of inconsistency principles that can be used as oracles, click here.
A large part of software testing is identifying inconsistencies between the software and someone’s expectations of the software. When a problem is encountered, testers give weight to their argument by describing the set of inconsistencies for business stakeholders to easily understand and appreciate before making a decision.