Quality has different dimensions, also known as “criteria categories” (Bach, 2019), “attributes” (Kaner, Bach & Pettichord, 2002), “characteristics” (ISO/IEC 25010:2011) or “aspects” (van Daele, 2018). It’s critical for a tester to be aware of different ways a software product, system or service delivers value, so it can be tested for all the ways that it might not do so or may deliver harm. It’s important to be aware of different quality dimensions when testing to paint a broad picture of quality and risk and therefore forms a key part of the quality story.

Quality dimensions can be thought of as lenses. By applying a different lens to the same software feature, user story or element, it can help to illuminate and focus the tester on its quality from a different perspective, a concept used in computer game design (Schell, 2008).

Quality dimensions are recognised as “-ility” nouns from their “ability” to deliver or threaten value to someone in some way. There is no limit to how many quality dimensions there can be or how they are grouped together.

Quality dimensions also are a useful in test strategy and planning, especially when it comes to risk analysis. Whilst a software can deliver quality in different ways, not all ways will be equally important in all contexts. Some will be more important than others, or have greater risk (van Daele, 2018).

Capability is the main dimension of quality, the social or cognitive prosthesis, to which all others are built upon. It’s not possible to have poor performance and usability in completing a task unless it’s first possible to complete a task. Capability is therefore also known as functional quality with all the rest referred to as parafunctional quality.

Like all testing and quality heuristics, quality dimensions are a fallible guide to be used intelligently and creatively by a tester, and not a hard rule that applies in all situations. With this in mind, all definitions include “reasonably”.

• Capability - Ability to complete tasks in reasonable and expected ways, possibly in multiple different ways (flexibility).
• Performance - Ability to complete tasks in reasonable time and with reasonable responsiveness
• Usability - Ability to complete those tasks reasonably easily, simply and intuitively.
• Accessibility - Ability to complete those tasks reasonably well as accessibility-needs users
• Security - Ability to reasonably guard against completing or altering tasks as unauthorised users or prevent authorised users from doing so
• Reliability - Ability to complete tasks reasonable accurately and when needed

• Concurrency Ability to complete tasks or user functions reasonably simultaneously with similar tasks and with other tasks
• Efficiency Ability to complete tasks or user functions with reasonable system resources and running cost
• Portability Ability to complete tasks or user functions on different devices and environments
• Localisation Ability to complete tasks or user functions reasonably well in different countries or cultures
• Scalability Ability of all Quality Criteria to reasonably scale with increased usage, size, complexity, market, time etc
• Operability Ability to reasonably deploy, install, configure, update and resolve problems in production with reasonable disruption and time scale
• Testability Ability to reasonably observe and configure the product for testing purposes
• Maintainability Ability to reasonably maintain the product for future development including bug fixing, expansion and repurposing
• Aesthetics Ability to otherwise give a good impression of quality and be reasonably marketable

 

Citations

• Bach, J., 2019. Heuristic Test Strategy Model. Quality Criteria Categories. v.5.7.5 [online] Available at: Link
• Kaner, C., Bach, J. and Pettichord, B., 2002. Lessons Learned in Software Testing: A Context-Driven Approach. John Wiley & Sons: New York. p. 60
• ISO/IEC 25010:2011(en), Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — System and software quality models. 4.2 Product quality model Available at: Link
• Schell, J., 2010. The Art of Game Design: A Book of Lenses. 2nd ed. Taylor and Francis Group.
• van Daele, B., 2018. How To Do Risk Based Software Testing Using TestSphere. Available at: Link