Application in Usability Evaluation

Distinguishing Between Quantitative and Qualitative Assessment

Application in Usability Evaluation

Although the term “assessment” is used casually in everyday conversation, assessment methods are fundamentally divided into two distinct approaches: “quantitative assessment” and “qualitative assessment.” There is no need to be intimidated by these terms—they are far from complicated.

What is Quantitative Assessment?

Quantitative assessment refers to the method of conducting evaluation through numerical data. As the term suggests, it means to “define quantity”—in other words, to evaluate by converting information into numerical form. Concrete examples include test scores, sales revenue, and five-point rating scales on surveys. This is a dependable assessment method when measuring something and serves as a kind of “measuring stick.”

The defining characteristic of quantitative assessment is its objectivity and credibility. The presence of numerical data makes comparison and comprehension straightforward, and as data accumulates over time, trend analysis and statistical validation become possible. This objectivity plays a crucial role in decision-making and the formulation of improvement initiatives.

What is Qualitative Assessment?

In contrast, qualitative assessment refers to an evaluative approach that does not rely on numbers and is relatively subjective in nature. It addresses questions such as “Is it good or poor?” “Is it superior or standard?” or, to put it directly, it evaluates the “quality” of something. Concrete examples include movie reviews, book critiques, and impressions formed during job interviews.

The defining characteristic of qualitative assessment is its ability to evaluate subjective aspects, emotions, and the qualitative dimensions of a subject. It can capture nuances and underlying reasons that numerical data alone cannot convey. Consequently, qualitative assessment is well suited to identifying specific improvement directions and detailed problem areas.

The Complementary Relationship Between Quantitative and Qualitative Assessment

At first glance, it may appear that one assessment method is superior to the other. However, in reality, the appropriate evaluation method depends greatly on the objective and context. Rather, these two approaches are complementary in nature, and their effectiveness is maximized only when they are used in combination.

Practical Application of Both Methods in Usability Evaluation

Interest in usability assessment has increased rapidly in recent years. Usability refers to the quality of whether a product or system is user-friendly, and it has become an important requirement particularly in the development of medical devices, software applications, and web services—systems where users directly interact with the interface.

Application of Quantitative Methods

When measuring usability quantitatively, standardized approaches such as the System Usability Scale (SUS) are widely employed. The SUS comprises ten questions, and by converting user responses into a scale of 0 to 100, one can obtain an objective numerical representation of system usability. This numerical score is effective for comparing different systems and measuring the effectiveness of improvements between versions.

In the medical device sector, international standard IEC 62366-1 (Medical Devices—User Interface Design) mandates the collection of quantitative metrics during user testing, such as success rate, error rate, and task completion time. These metrics serve as objective evidence for assessing whether a system can be used safely and effectively.

Application of Qualitative Methods

Conversely, to capture user perspectives concretely, qualitative assessment methods—such as feedback collection, interviews, and observation—are indispensable. Understanding why users struggle with specific operations or what improvements would enhance usability requires qualitative information. Focus group discussions and in-depth interviews provide valuable insights that numerical data alone cannot reveal.

In medical device regulatory submissions, clinical evaluation and post-market feedback from patients are also treated as forms of qualitative assessment, which regulatory authorities use to comprehensively evaluate the practical utility of a product.

Integrated Application of Both Assessment Methods for Formulating Improvement Strategies

When advancing usability improvements, it is essential to integrate information from both quantitative and qualitative assessment. Quantitative metrics indicate the priority of improvements and the magnitude of improvement effects, while qualitative assessment describes the specific details and implementation approach for improvements.

For example, suppose a decline in the SUS score is identified. Numbers alone do not reveal “what the problem is.” However, if interviews and user observation uncover concrete issues such as “the navigation structure is overly complex” or “access to critical functions is unclear,” the design team can establish a clear improvement strategy. Conversely, when qualitative feedback exists but lacks numerical support, it becomes difficult to convince management of the necessity or priority of improvements.

In this way, quantitative data demonstrates the “necessity and priority” of improvements, while qualitative information illustrates the “specific content and direction” of improvements. Together, they enable the formulation of more effective and feasible initiatives.

Assessment Method	Characteristics	Application Context	Information Obtained
Quantitative Assessment	Numerical basis, objective, comparable	Measuring overall system usability scores, quantifying improvement effects, conducting benchmarking comparisons	Numerical indicators, trends, statistical evidence
Qualitative Assessment	Description-based, subjective, in-depth exploration	Detailed problem identification, understanding user emotions and reasons for confusion, uncovering improvement opportunities	Reasons, context, specific issues, user feedback

Conclusion

Both quantitative and qualitative assessment are essential methods for obtaining important information from different perspectives. Rather than making an either-or judgment about which is superior, it is necessary to understand the characteristics of each and apply them discriminately according to the objective and context.

By regularly asking yourself in daily professional work—”Is quantitative methodology appropriate for this assessment, or is qualitative methodology necessary?”—more effective and credible evaluations can be realized. Furthermore, by integrating the insights from both methodologies, development efforts can focus on creating products and services that deliver genuine value to users.