What is Quality?
The Oxford Dictionary defines quality simply as a relative measurement against something else that is in some way comparable:
The standard of something as measured against other things of a similar kind; the degree of excellence of something(Oxford Languages, 2020)
The International Organization for Standardization defines quality as an assessment or measurement (degree) of something (object, the software) against it’s ability to meet a specific need, whether implicit or explicit (requirements):
Degree to which a set of inherent characteristics of an object fulfils requirements(ISO 9000:2015)
Deming suggests another definition of quality as the positive feelings (satisfaction) of paying customers only and that no-one else matters:
Quality is defined from the customer’s point of view as anything that enhances their satisfaction(Deming)
Juran considers both fulfilling requirements for customers (meeting needs or fitness for use) which also results in a positive feeling (satisfaction). They also mention quality as zero deficiencies, which is a tricky proposition since “you can’t please everyone all the time”:
Fitness for use. Those product features which meet the needs of customers and thereby provide product satisfaction. Freedom from deficiencies(Juran)
Like Plato’s “form of the good” or the “Tao” in Eastern philosophy, Pirsig suggests in their “metaphysics of quality” (MOQ) that quality cannot be defined and is unexplainable, but nevertheless is real and is the form to which everything comes from. Quality exists in the moment as an emotive experience only. Defining quality is therefore in retrospect only “after the fact” and is limited by human language:
Quality is a characteristic of thought and statement that is recognised by a non thinking process. Because definitions are a product of rigid, formal thinking, quality cannot be defined. But even though quality can’t be defined, you know what quality is. Quality is the continuing stimulus which our environment puts upon us to create the world in which we live. All of it. Every last bit of it. Now to take that which has caused us to create the world and include it within the world we have created, is clearly impossible. That is why quality cannot be defined. If we do define it we are defining something less than quality itself.(Pirsig, 1974)
However a lot of these definitions have serious problems. They’re simply too narrow and specific to be a good definition.
Weinberg provides a more general, all-encompassing definition of quality and is therefore probably best definition of quality available. They define it as simply any value to any person. This makes it clear that quality is a relationship and only exists relative to a person’s thoughts and ideas. This was later expanded to include “any person who matters” to highlight that decisions on quality always start with who’s value is cared about (Bach and Bolton, 2014) and also expanded to include “some time” to highlight that quality can change (Bolton, 2010):
Quality is value to some person(Weinberg, 1992)
Ashby also gives a good definition of quality that expands on value to include expectations as the assessment or measurement of requirements (wants & needs) and feelings (emotive experience):
Quality consists of Usefulness (the value/worth of something) + Correctness (meeting expectations re wants & needs) + Goodness (the emotive experience of something)(Ashby, 2020)
Below is a model of the definition of quality and how testing interacts with it. Originally inspired by a similar model by (Rogers, 2016):
The more general and all-encompassing the definition of quality, the closer it to describing what quality truly is. The more quality is restricted to a specific, narrow definition, the further away it is from describing what quality truly is and the more it opens up gaps in knowledge and understanding. Software testers apply a good, general, all-encompassing definition of quality to cover all bases and identify many ways in which value could be delivered and threatened and avoid poor, narrow, specific definitions of quality that leave gaps to where important value or serious risk may lie. The best definition of quality available to testers is value to some person(s) who matter at some time which can include constant expectations being met or exceeded and positive emotive experiences.
There’s a lot of relative and subjective aspects to quality and value can be assessed in many ways. This begs the question, what is truth in quality and what is value to some person who matters?
- Prostheses, Tools & Apps - Value in software is primarily its ability to solve a problem, complete a task or achieve a goal making software a social or cognitive prosthesis or tool
- Feelings and Emotions - Focus on people's feelings as the indicator to quality but be careful to avoid being fooled by illusions
- Objective vs Subjective vs Relative - Quality is subjective, relative truth with an objective, relative analysis and is formed from relationships between people and software
- Ashby, D. 2020. How “Good” is your Product? [online] Quality Advocates. Available at: Link
- Bach, J. and Bolton, M. 2014. Quality of testing always matters. Interview with Michael Bolton. Part I. [online] a1qa blog. Available at: Link
- Bolton, M., 2010. Quality Is Value To Some Person At Some Time. [online] Markus Gärtner. Available at: Link
- Deming, W.E. What is quality? [online] International Six Sigma Institute. Available at: Link
- ISO 9000:2015(en). Quality management systems — Fundamentals and vocabulary. 3.6 Terms related to requirement 3.6.2 Quality. Available at: Link
- Juran, J. What is quality? [online] International Six Sigma Institute. Available at: Link
- Oxford Languages., 2020. Quality entry. [online] Google Dictionary. Available at: Link (accessed date)
- Pirsig, R., 1974. Zen and the Art of Motorcycle Maintenance: An Inquiry into Values. 40th an. ed. London: Vintage, pp. 193-194, 236.
- Rogers, R., 2016. A Model Of The Relationship Between Quality, Value, Testing And Risks. [online] richrtesting. Available at: Link
- Weinberg, G., 1992. Software Quality Management: Volume 1: Systems Thinking. New York: Dorset House, p. 7.
Revised and updated: 16/10/2021