What is Quality?

The Oxford Dictionary defines quality simply as a standard or “goodness” of something as measured against something else that is in some way comparable:

The standard of something as measured against other things of a similar kind; the degree of excellence of something

(Oxford Languages, 2020)

The International Organization for Standardization is more specific and defines quality as an assessment or measurement (degree) of something (object, the software) against it’s ability to meet a specific need, whether implicit or explicit (requirements):

Degree to which a set of inherent characteristics of an object fulfils requirements

(ISO 9000:2015)

Deming suggests another definition of quality as the positive feelings (satisfaction) of paying customers only and that no-one else matters:

Quality is defined from the customer’s point of view as anything that enhances their satisfaction

(Deming)

Juran considers both fulfilling requirements for customers (meeting needs or fitness for use) which also results in a positive feeling (satisfaction). They also mention quality as zero deficiencies, which is a tricky proposition since “you can’t please everyone all the time”:

Fitness for use. Those product features which meet the needs of customers and thereby provide product satisfaction. Freedom from deficiencies

(Juran)

Like Plato’s “form of the good” or the “Tao” in Eastern philosophy, Pirsig suggests in their “metaphysics of quality” (MOQ) that quality cannot be defined and is unexplainable, but nevertheless is real and is the form to which everything comes from. Quality exists in the moment as an emotive experience only. Defining quality is therefore in retrospect only “after the fact” and is limited by human language:

Quality is a characteristic of thought and statement that is recognised by a non thinking process. Because definitions are a product of rigid, formal thinking, quality cannot be defined. But even though quality can’t be defined, you know what quality is. Quality is the continuing stimulus which our environment puts upon us to create the world in which we live. All of it. Every last bit of it. Now to take that which has caused us to create the world and include it within the world we have created, is clearly impossible. That is why quality cannot be defined. If we do define it we are defining something less than quality itself.

(Pirsig, 1974)

A lot of these definitions while good have some problems. They’re a bit too narrow or specific to be good definitions and serve more like examples in specific contexts. Counter examples can be given. For example, if there are no requirements can something still be quality? If there’s no customer satisfaction, can it still be considered quality? We may also consider quality to be undefinable, but that doesn’t help the practical purpose of making quality products and systems or delivering quality services.

 

Definition

Weinberg provides a general, all-encompassing definition of quality and is therefore probably the best one available. If someone values something, then that something has quality to them. If they don’t value it, then it doesn’t. This clearly separates what has quality from what doesn’t while leaving the definition open to any ideas of value to any person. This definition also makes it clear that quality is a relationship. The definition was later expanded to include “any person who matters” to highlight that decisions on quality always start with who’s value is cared about (Bach and Bolton, 2014) and “some time” to highlight that quality is not fixed an can change (Bolton, 2010):

Quality is value to some person

(Weinberg, 1992)

Ashby also gives a good compound definition and combined assessment of quality that summarises neatly many of the definitions above:

Quality consists of Usefulness (the value/worth of something) + Correctness (meeting expectations re wants & needs) + Goodness (the emotive experience of something)

(Ashby, 2020)

 

Model

Below is a model of the definition of quality and how testing interacts with it. Originally inspired by a similar model by (Rogers, 2016):

 

Summary The more general and all-encompassing the definition of quality, the better it is. The more quality is restricted to a specific, narrow definition, the more it opens up gaps in knowledge and understanding where risk may lie. Software testers apply a good, general, all-encompassing definition of quality to cover multiple bases and identify many ways in which quality could be delivered and threatened. The best definition of quality available to testers is “value to some person(s) who matter at some time” with considerations to expectations being met and feelings of those who matter.

 

There’s a lot of relative and subjective aspects to quality and value can be assessed in many ways. This begs the question, what is truth in quality and what is value to some person who matters? Next:

• Value - Primarily an ability to solve a problem, complete a task or achieve a goal making software a social or cognitive prosthesis or tool

References

• Ashby, D. 2020. How “Good” is your Product? [online] Quality Advocates. Available at: Link
• Bach, J. and Bolton, M. 2014. Quality of testing always matters. Interview with Michael Bolton. Part I. [online] a1qa blog. Available at: Link
• Bolton, M., 2010. Quality Is Value To Some Person At Some Time. [online] Markus Gärtner. Available at: Link
• Deming, W.E. What is quality? [online] International Six Sigma Institute. Available at: Link
• ISO 9000:2015(en). Quality management systems — Fundamentals and vocabulary. 3.6 Terms related to requirement 3.6.2 Quality. Available at: Link
• Juran, J. What is quality? [online] International Six Sigma Institute. Available at: Link
• Oxford Languages., 2020. Quality entry. [online] Google Dictionary. Available at: Link (accessed date)
• Pirsig, R., 1974. Zen and the Art of Motorcycle Maintenance: An Inquiry into Values. 40th an. ed. London: Vintage, pp. 193-194, 236.
• Rogers, R., 2016. A Model Of The Relationship Between Quality, Value, Testing And Risks. [online] richrtesting. Available at: Link
• Weinberg, G., 1992. Software Quality Management: Volume 1: Systems Thinking. New York: Dorset House, p. 7.

Special Thanks

• Michael Bolton (@michaelbolton) for his corrections and assistance on definitions and value

Updated: 2021-10-16 (added Ashby definition and revised model diagram)