Why We Do Not Seek Probability Estimates for Human Errors and Software Errors

Human Errors and Software Errors: Not “Eventually” but “Inevitably”

In the midst of diligent work, troubles occasionally strike unexpectedly. These may stem from human error—mistakes caused by human action or judgment—or from software errors—defects originating in computer systems.

Why do such troubles occur at all? It may come as a surprise, but these errors should be viewed not as something that “will happen eventually” but as something that “will inevitably happen.” Particularly in the medical device industry, this recognition forms the foundation of ISO 13485 quality management systems and ISO 14971 risk management, making it far more than a matter of business management—it is a critical issue directly connected to patient safety. Let us examine the reasons and countermeasures together.

The Existence of Unpredictable Errors

Human error refers to mistakes that occur as a result of human action or judgment, while software error refers to defects in computer programs. These share a common characteristic: the probability of their occurrence is generally difficult to predict.

For example, when predicting the likelihood of traffic congestion, calculations are made based on data such as the previous day’s temperature, precipitation, and road congestion status. However, it is impossible to reliably quantify and predict the occurrence probability of human errors or software errors in the same manner.

This is because human errors arise from a complex interplay of multiple factors—thought processes, situational judgment, morality, and technical competence—making them difficult to reduce to mathematical formulas. Similarly, software errors stem from program complexity, compatibility issues, and unknown threats such as viruses and cyberattacks, which are likewise difficult to express mathematically. Regarding software errors in medical devices, international standards such as IEC 62304 and ISO/IEC 27001 presume that complete elimination is impossible; rather, detection, mitigation, and management are emphasized as paramount. Consequently, rather than attempting to accurately predict the probability of these errors, risk management approaches evaluate them conservatively—assuming they will inevitably occur. ISO 14971 requires organizations to assume “worst-case scenarios,” implement control measures, and verify their effectiveness.

Management Approaches Focused on Severity

To make effective use of resources allocated to error management—both cost and time—we must focus on the severity of errors and consider this carefully. In simple terms, we evaluate “how significantly an error would impact operations if it were to occur,” and then determine what preventive measures should be implemented. ISO 14971 requires a risk assessment approach that combines severity and detectability, establishing priorities based on risk acceptability criteria.

Let us consider concrete examples. Suppose a large organization undergoes reorganization and establishes a new division with a newly hired junior employee. Which would have greater impact: an error made by this junior employee or a mistake made by a veteran staff member? One might think, “The junior employee has less influence, so it must be safer.” However, neglecting errors because their immediate impact seems minor can lead to significant failures later. In the medical device manufacturing industry, such assumptions directly affect patient safety; therefore, all processes require control measures of a certain minimum level, regardless of an employee’s job history or years of experience. In such cases as well, errors must be assumed to inevitably occur, and layered control measures should be established.

Let us also provide a concrete example regarding software errors. Which would be more impactful: a temporary stoppage of cloud services, or complete failure of the entire company’s core system—particularly one managing medical device manufacturing data or patient records? The answer is obvious. Consequently, preventive measures should be proportionate to the magnitude of potential impact. In the medical device industry, depending on the criticality of such systems, measures such as backup system construction, regular disaster recovery testing, and data integrity controls as specified in industry guidelines such as ICH Q2(R2) and FDA’s “Guidance for Industry: Part 11, Electronic Records; Electronic Signatures” are necessary.

Conclusion

In any case, we must approach both human errors and software errors with the understanding that they “will inevitably occur,” and accordingly develop preventive and corrective strategies. Based on the magnitude of risk associated with each, we assign priorities and work toward stable organizational operations. By practicing systematic risk management based on ISO 13485 and ISO 14971, organizations move beyond reactive responses to implement preventive and continuous improvement. When errors do occur, if we can respond calmly rather than panic, that itself represents a step forward.