Evaluation Standards for Medical Device Programs with Behavioral Intervention Components

From Public Comment to Final Notification

On March 4, 2022, the Ministry of Health, Labour and Welfare of Japan initiated a public comment period regarding “Evaluation Standards for Medical Device Programs with Behavioral Intervention Components (Draft).” The comment period closed on April 2, 2022 at 23:59 JST. Following a comprehensive review process by the Pharmaceuticals and Medical Devices Agency (PMDA), this evaluation standard was formally issued as an official PMDA notification in February 2023. This notification has become a critical guidance document in Japan’s regulatory framework for medical device programs and continues to be applied in current regulatory practice.

Definition and Market Background of Behavioral Intervention Medical Device Programs

With the advancement of digitalization and information technology, the development of medical device programs that promote behavioral change through smartphone applications, wearable devices, and other digital tools has expanded rapidly. Unlike traditional medical devices, these programs aim to improve users’ daily habits and generate clinically meaningful outcomes.

Currently developed behavioral intervention medical device programs target a diverse range of therapeutic areas and patient populations. These include chronic diseases such as diabetes mellitus, hypertension, non-alcoholic fatty liver disease (NASH), and alcohol-related disorders; oncology; sleep disorders; neurodegenerative conditions including dementia; and preventive intervention programs aimed at maintaining physical and mental function. In these disease areas, behavioral modification plays a crucial role in disease management and prevention, and when medical device programs can scientifically demonstrate their efficacy, they have the potential to substantially improve the quality of patient care.

Regulatory Background and International Context

Behavioral intervention medical device programs represent a novel product category distinct from traditional physical medical devices, and determining appropriate evaluation methodologies has been a challenge for regulatory authorities worldwide. Internationally, regulatory agencies including the U.S. Food and Drug Administration (FDA), the European Commission, and Canada’s Health Products and Food Branch (HPFB) have progressively issued guidance documents on the evaluation of this product category. Japan’s PMDA has similarly developed its notification with attention to international regulatory convergence.

Comparatively, the European Union’s Medical Device Regulation (MDR) also reflects the need for clear guidance on clinical evaluation of behavioral intervention medical device programs. Japan’s regulatory approach addresses this international need while maintaining consistency with global standards.

Safety, Efficacy, and Usability Considerations in the Evaluation Framework

The PMDA’s notification delineates key evaluation considerations for behavioral intervention medical device programs as follows:

Safety Assessment: The safety profile of medical device programs encompasses not only functional safety but also risks arising from user interface design that may lead to use errors, as well as psychosocial risks. Particularly for programs designed to promote behavioral change, there exists a potential that inappropriately framed intervention messages could paradoxically induce harmful behavioral patterns. Comprehensive risk analysis addressing such scenarios is therefore required.

Efficacy Assessment: Efficacy evaluation of behavioral intervention medical device programs must not rely solely on surrogate markers such as user satisfaction scores or program continuation rates. Instead, clinical demonstration of meaningful outcomes—such as reduction in hemoglobin A1c levels, improvement in blood pressure, or objective indicators of improved lifestyle modifications—is essential.

Usability: Compliance with IEC 62366-1 “Medical devices—Part 1: Application of usability engineering to medical devices” is mandatory. For behavioral intervention programs, evaluation must account for user diversity, including elderly populations and those with limited health literacy.

Regulatory Approach to Clinical Trial (Clinical Investigation) Necessity

Individual Assessment Principle

The PMDA notification establishes the following framework for determining the necessity of clinical evaluation materials:

The necessity of clinical evaluation materials shall be determined on a comprehensive basis considering the specific characteristics of each medical device program, the range of evaluable content through non-clinical methodologies, and the existing scientific evidence base. Therefore, clinical trials are not uniformly mandatory for all behavioral intervention medical device programs. Sponsors are strongly encouraged to utilize PMDA’s consultation system (pre-submission meeting) at early development stages to establish an appropriate clinical evaluation strategy tailored to their specific product.

Principle: Generally Higher Clinical Evidence Standards Apply

Conversely, currently no established approval standards exist specifically for behavioral intervention medical device programs. Given the limited number of approval precedents in this field and the high degree of product heterogeneity, higher levels of scientific evidence are typically expected compared to conventional medical devices.

It is technically and methodologically challenging to adequately establish the efficacy and safety of behavioral intervention programs through non-clinical testing alone. This is because program effectiveness is ultimately validated only through real-world clinical use within actual user populations. Furthermore, numerous variables—including psychosocial factors, user motivation, program adherence, and contextual elements—cannot be replicated in non-clinical environments and substantially influence outcomes.

Consequently, the practical regulatory approach of Japanese authorities reflects the principle that clinical trial data submission is generally considered essential for behavioral intervention medical device programs. However, the definition of “clinical trial” is interpreted flexibly, encompassing not only rigorous pharmaceutical-style investigational approaches but also clinical performance evaluations, real-world evidence (RWE) collection, and other diverse clinical evaluation methodologies.

Recommended Approaches in Submission Practice

When companies consider pursuing regulatory approval for behavioral intervention medical device programs, the following procedural steps are recommended:

Early engagement with PMDA through the formal consultation system is crucial. During these pre-submission meetings, companies should present their product’s specific characteristics, target patient populations, and anticipated efficacy outcomes. PMDA will provide specific guidance regarding whether non-clinical evaluation alone is sufficient, whether clinical data is mandatory, and if mandatory, what sample sizes and observation periods would be considered appropriate.

Concurrently, guidance on documentation requirements at the time of approval submission should also be obtained through individual consultation. Currently, unlike standardized medical device approval pathways (for example, Class II IVDRs), unified submission formats or checklists specific to behavioral intervention programs have not yet been fully standardized. Therefore, proactive information gathering and direct engagement with regulatory authorities on the part of sponsors is critically important.

Future Direction of Regulatory Evolution

As the market for behavioral intervention medical device programs expands rapidly, regulatory guidance frameworks continue to evolve internationally. Japan’s PMDA is similarly advancing its guidance in alignment with these global developments, with anticipated updates and best practice communications forthcoming. Particular attention will likely be devoted to behavioral intervention programs incorporating artificial intelligence and machine learning technologies, as well as adaptive systems wherein the program algorithm evolves over time—for which more detailed evaluation requirements are expected to be articulated.

Furthermore, regulatory harmonization with the European Union’s MDR and IVDR frameworks, combined with ongoing international standardization efforts through organizations such as the International Organization for Standardization (ISO), are anticipated to indirectly influence Japan’s regulatory requirements going forward.