Quality can partly be identified, evaluated and described as “consistency principles” that form the basis of expectations. Consistency principles describe a type of mental model that’s used as a source of comparison for software testers. It’s important for testers to be aware of many different inconsistencies between someone’s mental models and experiences of the software as inconsistency could point to potential lost value or harm or give weight to bug advocacy (Bolton, 2005) (Bolton, 2012).
Consistency principles also are a useful in test strategy and planning, especially when it comes to risk analysis. Finding out what consistencies a tester should focus on at the beginning of a software project will help focus on the important information and finding the bugs that matter the most. Asking stakeholders what they want and expect the most is the best way to obtain this information, however it can also be inferred from other sources.
Like all testing and quality heuristics, consistency principles are a fallible guide to be used intelligently and creatively by a tester, and not a hard rule that applies in all situations. With this in mind, all definitions should be read with “reasonably”. Below is a possible list of consistency principles. There’s no limit to the number of consistency principles that exist or how they can be described or grouped together.
The following list is inspired by AFEW HICCUPPS from (Bolton, 2012):
- Specification Consistency with explicit claims made in project documentation
- Reference Consistency with software programs that share similarities in purpose or function
- Regression Consistency with past versions of the same software program
- Desire Consistency with implicit desires, wants and needs of stakeholders
- Purpose Consistency with the intended purpose of software
- Self Consistency within the software program itself
- Legal Consistency with governmental or judicial statutes, precedents or regulations
- Standards Consistency with standards, recommendations or best practices within the company or external authoritative bodies
- Image Consistency with the image that stakeholders want to project
- Problem Inconsistency with previously reported and resolved problems