Quality has different dimensions, also known as “criteria categories” (Bach, 2019), “attributes” (Kaner, Bach & Pettichord, 2002), “characteristics” (ISO/IEC 25010:2011) or “aspects” (van Daele, 2018). It’s critical for a tester to be aware of different ways a software product, system or service delivers value, so it can be tested for all the ways that it might not do so or may deliver harm. It’s important to be aware of different quality dimensions when testing to paint a broad picture of quality and risk and therefore forms a key part of the quality story.
Quality dimensions can be thought of as lenses. By applying a different lens to the same software feature, user story or element, it can help to illuminate and focus the tester on its quality from a different perspective, a concept used in computer game design (Schell, 2008).
Quality dimensions are recognised as “-ility” nouns from their “ability” to deliver or threaten value to someone in some way. There is no limit to how many quality dimensions there can be or how they are grouped together.
Quality dimensions also are a useful in test strategy and planning, especially when it comes to risk analysis. Whilst a software can deliver quality in different ways, not all ways will be equally important in all contexts. Some will be more important than others, or have greater risk (van Daele, 2018).
Capability is the main dimension of quality, the social or cognitive prosthesis, to which all others are built upon. It’s not possible to have poor performance and usability in completing a task unless it’s first possible to complete a task. Capability is therefore also known as functional quality with all the rest referred to as parafunctional quality.
Like all testing and quality heuristics, quality dimensions are a fallible guide to be used intelligently and creatively by a tester, and not a hard rule that applies in all situations. With this in mind, all definitions include “reasonably”.
- Capability (Functionality) Ability to complete tasks or user functions in reasonable and expected ways
- Performance Ability to complete tasks or user functions in reasonable time and responsiveness
- Usability Ability to complete tasks or user functions reasonably easily, simply and intuitively
- Accessibility Ability to complete tasks or user functions reasonably well as accessibility-needs users
- Concurrency Ability to complete tasks or user functions reasonably simultaneously with similar tasks and with other tasks
- Efficiency Ability to complete tasks or user functions with reasonable system resources and running cost
- Reliability Ability to complete tasks or user functions without unreasonable interruption and with reasonable recovery from interruptions
- Integrity Ability to complete tasks or user functions with reasonable accuracy
- Security Ability to reasonably guard against unauthorised access to complete or alter tasks or to prevent authorised access to do so
- Portability Ability to complete tasks or user functions on different devices and environments
- Localisation Ability to complete tasks or user functions reasonably well in different countries or cultures
- Scalability Ability of all Quality Criteria to reasonably scale with increased usage, size, complexity, market, time etc
- Operability Ability to reasonably deploy, install, configure, update and resolve problems in production with reasonable disruption and time scale
- Testability Ability to reasonably observe and configure the product for testing purposes
- Maintainability Ability to reasonably maintain the product for future development including bug fixing, expansion and repurposing
- Aesthetics Ability to otherwise give a good impression of quality and be reasonably marketable
- Bach, J., 2019. Heuristic Test Strategy Model. Quality Criteria Categories. v.5.7.5 [online] Available at: Link
- Kaner, C., Bach, J. and Pettichord, B., 2002. Lessons Learned in Software Testing: A Context-Driven Approach. John Wiley & Sons: New York. p. 60
- ISO/IEC 25010:2011(en), Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — System and software quality models. 4.2 Product quality model Available at: Link
- Schell, J., 2010. The Art of Game Design: A Book of Lenses. 2nd ed. Taylor and Francis Group.
- van Daele, B., 2018. How To Do Risk Based Software Testing Using TestSphere. Available at: Link