Why Objective Evidence Matters: Performance Evaluation and Regulatory Requirements in Medical Devices

Introduction

In today’s business landscape, particularly within the medical device industry, the evaluation of product quality and safety must be grounded in objective evidence. Since medical devices directly impact human health and safety, evaluations based on manufacturer bias or subjective interpretation are not permissible. This article explores the critical importance of objective evidence from a medical device regulatory perspective, examining its definition, acquisition methods, and relationship to regulatory requirements in language accessible to newcomers while maintaining professional rigor.

What is Objective Evidence?: Definition and Regulatory Context

Fundamental Definition

Objective evidence refers to verifiable and reproducible evidence that yields the same results regardless of who conducts the evaluation. In contrast, subjective evidence is based on personal feelings and judgments, and results can vary between evaluators.

In the medical device field, this definition becomes significantly more stringent. For instance, the subjective claim that a product is “easy to use” is insufficient. Rather, objective evidence would state: “In a usability study of fifty subjects aged 20-65, forty-five subjects successfully completed the intended operation within five minutes on the touchscreen interface.” Such specific, measurable evidence is required by regulators.

The Role of Objective Evidence in Medical Device Regulation

In international medical device regulatory guidance, objective evidence is particularly emphasized in the following domains.

Evaluation Domain	Examples of Objective Evidence	Regulatory Requirement
Performance Evaluation	Clinical trial data, analytical validation results	IVDR Annex XIII, FDA 510(k)
Usability	Usability test reports, error analysis data	IEC 62366-1, FDA Guidance
Safety	Risk analysis, failure rate data, safety test results	ISO 14971, IEC 60601 Series
Biological Safety	Test reports (based on ISO 10993 Series)	FDA 21 CFR Part 860

Determining True Product Value: The Importance of Observation and Evaluation

The Necessity of Evaluating Real-World Use

To accurately determine a product’s true value, it is essential to directly observe how the product functions in actual use environments and what user experience results. This observation extends beyond functional assessment to encompass ergonomic considerations, interface effectiveness, and identification of potential risk factors.

Consider a medical device manufacturer that claims their design is “intuitive and user-friendly.” However, when healthcare professionals actually use the device in clinical settings, various problems may emerge, including:

• Emergency operations are complex and time-consuming
• Display text is too small to read clearly
• Operating procedures differ from international usability standards, causing confusion
• The device cannot be properly used without extensive manufacturer training

These issues typically arise when manufacturer assumptions take precedence over genuine user perspectives, particularly those of diverse users with varying expertise levels. Medical device regulations address this gap by requiring formal usability testing with representative user populations.

Implementation Considerations

Medical device usability evaluation must incorporate several elements based on IEC 62366-1 (Ergonomic design and evaluation of medical devices for human use), including evaluation by all anticipated user types (physicians, nurses, patients, maintenance technicians, etc.), assessment in standard use environments, and inclusion of use scenarios related to known hazards. Through this comprehensive approach, the gap between theoretical assumptions made during design and actual user behavior can be minimized.

Manufacturer-Independent Evaluation Methods

The Concept of Non-Manufacturer Involvement

Non-manufacturer involved evaluation refers to methods where manufacturers or distributors do not participate in the evaluation process, allowing users to operate the product autonomously and report their experiences. This approach prevents manufacturer intentions and expectations from influencing evaluation results and enables the collection of more authentic, user-centered insights.

Practical Evaluation Approaches

Specific implementation methods for non-manufacturer involved evaluation in the medical device industry commonly include the following:

Clinical Trials: Healthcare professionals use the medical device in actual clinical settings and report on its efficacy, safety, and usability. This is particularly essential for demonstrating the performance of in vitro diagnostic (IVD) and therapeutic medical devices.

Independent Usability Testing: Specialized organizations independent from the manufacturer recruit representative user populations, conduct usability testing in standardized environments, and measure success rates, error rates, and execution times.

User Feedback and Real-World Data: User feedback collected from post-market use environments, adverse event reports, and clinical outcome data serve as crucial sources for continued evaluation of product performance and safety. These data help identify unexpected use patterns and previously unidentified risk factors.

Post-Market Surveillance: This is a critical requirement in medical device regulation, involving continuous monitoring of how the product is actually used in the market and whether any safety concerns arise. EU MDR, IVDR, and FDA requirements all mandate comprehensive post-market surveillance plans.

Evolution of the Regulatory Environment

In recent years, particularly with the staggered implementation of the IVDR (May 2022 through May 2025) and the publication of FDA Real-World Evidence (RWE) guidance (2021), there is growing emphasis not only on clinical trial data but also on data obtained from actual clinical use environments. This development further underscores the importance of evidence collection from independent, user-centric perspectives, distinct from manufacturer influence.

Implementation Requirements for Objective Evidence in Medical Device Regulation

Alignment with International Regulatory Guidance

The primary regulatory requirements for objective evidence in medical device performance evaluation are as follows:

FDA 510(k) (Substantial Equivalence pathway) requires clinical or non-clinical data demonstrating that performance is equivalent to a predicate device.

EU In Vitro Diagnostic Regulation (IVDR) Annex III mandates submission of objective evidence regarding diagnostic accuracy (sensitivity, specificity), analytical validity, and clinical validity.

ISO 13485 (Quality Management System for Medical Devices) requires that all quality requirements from design and development through manufacturing and post-market stages be substantiated by objective evidence.

Documentation of Objective Evidence

When submitting to regulatory authorities, objective evidence must be presented in the following formats:

Verification and validation reports, clinical trial reports (GCP/GLP compliant), analytical performance data, usability engineering reports, and risk management reports. All these documents must clearly demonstrate the measurement methods employed, statistical treatment applied, and how the results contribute to the medical device’s quality and safety.

Conclusion

Quality evaluation and regulatory compliance in the medical device industry require evaluation methodologies grounded in objective evidence. To determine true product value and ensure patient safety and clinical efficacy, perspectives beyond manufacturer theoretical assumptions are necessary—specifically, independent viewpoints from actual users and clinical environments. By integrating multiple evaluation approaches including usability testing, clinical trials, and post-market surveillance, and by continuously collecting and analyzing objective evidence, medical device manufacturers can provide safer and more effective products while gaining regulatory authority confidence. Recognition of the importance of objective evidence and its establishment as organizational culture represents a key factor for maintaining competitiveness in the modern medical device industry.