Quality has different dimensions, also known as “criteria categories” (Bach, 2019), “attributes” (Kaner, Bach & Pettichord, 2002), “characteristics” (ISO/IEC 25010:2011) or “aspects” (van Daele, 2018). It’s critical for a tester to be aware of different ways a software product, system or service delivers value, so it can be tested for all the ways that it might not do so or may deliver harm. It’s important to be aware of different quality dimensions when testing to paint a broad picture of quality and risk and therefore forms a key part of the quality story.

Quality dimensions can be thought of as lenses. By applying a different lens to the same software feature, user story or element, it can help to illuminate and focus the tester on its quality from a different perspective, a concept used in computer game design (Schell, 2008).

Quality dimensions are recognised as “-ility” nouns from their “ability” to deliver or threaten value to someone in some way. There is no limit to how many quality dimensions there can be or how they are grouped together.

Quality dimensions also are a useful in test strategy and planning, especially when it comes to risk analysis. Whilst a software can deliver quality in different ways, not all ways will be equally important in all contexts. Some will be more important than others, or have greater risk (van Daele, 2018).

Capability is the main dimension of quality, the social or cognitive prosthesis, to which all others are built upon. It’s not possible to have poor performance and usability in completing a task unless it’s first possible to complete a task. Capability is therefore also known as functional quality with all the rest referred to as parafunctional quality.

Like all testing and quality heuristics, quality dimensions are a fallible guide to be used intelligently and creatively by a tester, and not a hard rule that applies in all situations. With this in mind, all definitions include “reasonably”.

• Introduction to Quality Dimensions - Different ways in which value or harm is delivered and recognised by "-ility" nouns
• Capability - Ability to complete tasks in reasonable and expected ways, possibly in multiple different ways (flexibility).
• Performance - Ability to complete tasks in reasonable time and with reasonable responsiveness
• Usability - Ability to complete those tasks reasonably easily, simply and intuitively.
• Accessibility - Ability to complete those tasks reasonably well as accessibility-needs users
• Security - Ability to reasonably guard against completing or altering tasks as unauthorised users or prevent authorised users from doing so
• Reliability - Ability to complete tasks reasonable accurately and when needed
• Security (Project) - Ability for teams to guard against information confidentiality, integrity, availability issues
• Concurrency - Ability to complete tasks reasonably simultaneously with similar or other tasks

• Concurrency Ability to complete tasks or user functions reasonably simultaneously with similar tasks and with other tasks
• Efficiency Ability to complete tasks or user functions with reasonable system resources and running cost
• Portability Ability to complete tasks or user functions on different devices and environments
• Localisation Ability to complete tasks or user functions reasonably well in different countries or cultures
• Scalability Ability of all Quality Criteria to reasonably scale with increased usage, size, complexity, market, time etc
• Operability Ability to reasonably deploy, install, configure, update and resolve problems in production with reasonable disruption and time scale
• Testability Ability to reasonably observe and configure the product for testing purposes
• Maintainability Ability to reasonably maintain the product for future development including bug fixing, expansion and repurposing
• Aesthetics Ability to otherwise give a good impression of quality and be reasonably marketable

 

Project

Quality dimensions don’t just apply to software products, systems and services, but also projects, including teams, companies or the software industry as a whole. These quality dimensions of projects are equally important as anything that threatens the project also can in turn threaten the software product. Testing for project quality to identify and report on these types of risk is an important part of a tester’s job and represents an important area of second-order testing.

• Security (Project) - Ability for teams to guard against information confidentiality, integrity, availability issues

• Productivity Ability to complete work and deliver value to stakeholders to time/cost and quality
• Agility Ability to complete work and solve problems within an agile framework using the agile mindset of lean thinking and adaptability
• Security Ability to guard against information confidentiality breaches, loss or integrity or availability issues
• Diversity Ability to bring different ideas, viewpoints, approaches and risk to problem-solving and solution-building
• Growth Ability to learn, understand, explore and try out new ideas to see how they fail or succeed
• Morale Ability to be happy, motivated, engaged and fulfilled by the project and testing work
• Safety Ability to feel psychologically safe to speak out, try new things and to fail

 

Citations

• Bach, J., 2019. Heuristic Test Strategy Model. Quality Criteria Categories. v.5.7.5 [online] Available at: Link
• ISO/IEC 25010:2011(en), Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — System and software quality models. 4.2 Product quality model Available at: Link
• Kaner, C., Bach, J. and Pettichord, B., 2002. Lessons Learned in Software Testing: A Context-Driven Approach. John Wiley & Sons: New York. p. 60
• Schell, J., 2010. The Art of Game Design: A Book of Lenses. 2nd ed. Taylor and Francis Group.
• van Daele, B., 2018. How To Do Risk Based Software Testing Using TestSphere. Available at: Link