The importance of quality assurance in software development has been increasing year by year. Traditionally, testing processes have generally been conducted based on detailed procedural documents called “test scripts.” However, in recent years, with the introduction of the Computer Software Assurance (CSA) guidance issued by the U.S. Food and Drug Administration (FDA) and the GAMP 5 Second Edition published by the International Society for Pharmaceutical Engineering (ISPE), a new testing approach called “unscripted testing” has been recommended. This column explains unscripted testing in a manner that is accessible to beginners while maintaining professional depth.
Overview of Unscripted Testing
Traditionally, creating “detailed procedural documents in advance” (test scripts) has been considered essential for testing. Test personnel execute each step according to the script and record the results. However, “unscripted testing” is a method that does not create detailed test procedure documents in advance, but rather sets general policies and objectives and proceeds with testing while actually operating the system.
This approach has been formally conceptualized in the FDA’s Computer Software Assurance (CSA) guidance (final version: issued September 24, 2025) and ISPE GAMP 5 Second Edition (issued 2022), and does not necessarily require detailed procedures or extensive documentation. It is considered particularly effective for low-risk features or scenarios where process risk is not high, and includes ad hoc testing and exploratory testing.
Regulatory Trends and Industry Standards
Position of FDA CSA Guidance
The FDA’s CSA guidance provides recommendations for computer software assurance for computers and automated data processing systems used as part of medical device production or quality systems. This guidance promotes the transition from traditional Computer System Validation (CSV) to a risk-based approach.
The draft was published in September 2022, and after receiving feedback from the industry, the final version was issued on September 24, 2025. The final version formally added definitions for cloud computing (SaaS, PaaS, IaaS) and clarified that artificial intelligence (AI) systems are also included in the scope. Additionally, as the FDA’s Quality System Regulation (21 CFR Part 820) will be harmonized with ISO 13485:2016 in February 2026, the CSA approach is expected to become even more important in the future.
Relationship with GAMP 5 Second Edition
ISPE’s GAMP 5 Second Edition, issued in July 2022, also emphasizes the concept of unscripted testing. GAMP 5 is a widely adopted guideline for GxP (Good Practice) compliant computerized systems in the pharmaceutical industry, and the second edition introduced a new concept called “critical thinking,” recommending a focus on the system’s fitness for its intended use while avoiding excessive documentation.
GAMP 5 Second Edition and the FDA CSA guidance share the following commonalities:
- Adoption of a risk-based approach
- Emphasis on critical thinking
- Focus on testing with minimal necessary documentation
- Combination of scripted and unscripted testing
- Leveraging supplier validation activities
Relationship with Ad Hoc Testing and Exploratory Testing
Unscripted testing primarily includes the following two methods:
1. Ad Hoc Testing
This is a method of finding defects by freely operating the system based on intuition and experience, without setting specific hypotheses or scenarios. This method has an unstructured nature and relies heavily on the creativity and intuition of the test personnel.
2. Exploratory Testing
This is a method that sets a certain degree of test objectives and charters (concise guidelines that define the scope and goals of exploration) and flexibly adjusts the test content while understanding specifications and behavior through actual operation. Exploratory testing is characterized by simultaneous test design and test execution, and was defined in 1984 by software testing expert Cem Kaner as “a style of software testing that emphasizes the personal freedom and responsibility of the individual tester to continually optimize the quality of their work by treating test-related learning, test design, test execution, and test result interpretation as mutually supportive activities that run in parallel throughout the project.”
Both methods allow the omission of detailed advance procedure planning and require the tester’s thinking ability, experience, skills, and flexibility.
Benefits and Application Scenarios
The greatest benefit of unscripted testing is that “the time spent creating test procedure documents can be redirected to actual testing work.” In the traditional CSV approach, test personnel spent approximately 80% of their time on documentation and only about 20% on actual testing. The approach recommended by CSA and GAMP 5 Second Edition aims to reverse this ratio, allocating 80% of time to actual testing and critical thinking, and 20% to documentation.
This improves the speed of test execution, enabling shorter time to release and more efficient iterations. It also allows flexible handling of unexpected problems, such as discovering defects that could not be anticipated in advance or bugs caused by unexpected usage patterns. Exploratory testing is particularly effective in discovering usability issues and edge cases that are often overlooked in scripted testing.
On the other hand, the FDA CSA guidance recommends this method for “features that do not have high process risk.” Specifically, unscripted testing is considered appropriate for features where software failures are unlikely to affect product quality or patient safety. Conversely, for features with high process risk—that is, features where failures could lead to quality problems or compromise patient safety—detailed scripted testing remains important.
Risk Assessment-Based Testing Strategy
CSA and GAMP 5 Second Edition recommend implementing a risk-based approach through the following four steps:
-
Identify Intended Use: Clearly define what the software’s features, operations, or behaviors are used for.
-
Determine Risk-Based Approach: Assess whether each feature poses process risks (impact on product quality or patient safety).
-
Determine Appropriate Assurance Activities: Based on risk assessment, select appropriate assurance activities such as scripted testing, unscripted testing, automated testing, leveraging supplier validation activities, and continuous monitoring.
-
Establish Appropriate Records: Properly document the assurance activities performed, including the use of digital records (system logs, audit trails, etc.).
Professional Considerations
Since unscripted testing depends on the skills and experience of test personnel, certain education and guidelines are important. Test personnel are required to have a deep understanding of system specifications and behavior and the ability to apply critical thinking to discover unexpected behaviors and potential problems.
Additionally, there is a challenge in ensuring the reproducibility of test results and maintaining evidence trails. In unscripted testing, since detailed procedures are not documented in advance, it is difficult to repeat the same test accurately. To compensate for this, implementing minimal test charters (concise guides describing the purpose and scope of test sessions) and simplified recording of results is recommended.
Furthermore, adopting structured exploratory testing methods such as session-based testing and pair exploratory testing can improve auditability and measurability.
Documentation Approach
The FDA CSA final guidance recommends the following documentation approaches:
- Leveraging digital records: Use system logs, audit trails, and other data generated and maintained by the software as evidence of validation.
- Automated traceability and testing: Reduce the need for manual or paper-based documentation.
- Risk-based documentation: Detailed documentation for features with high process risk, simplified documentation for features without high risk.
Alignment with Agile Development
GAMP 5 Second Edition explicitly demonstrates alignment with agile software development methodologies. Agile development requires a different approach from the traditional linear V-model, as development and testing are conducted continuously in short iterations (sprints). Unscripted testing is well-suited to the flexibility and adaptability of agile development and can quickly respond to changing requirements.
However, risk assessment and appropriate documentation remain important even in agile development. GAMP 5 Second Edition demonstrates methods for ensuring traceability by leveraging artifacts generated from agile development processes (such as emails and audit trails).
Summary
Unscripted testing is an effective new testing method for features without high process risk and for discovering unexpected defects. The latest quality assurance frameworks, including the FDA CSA guidance (2025 final version) and ISPE GAMP 5 Second Edition (2022), actively incorporate this flexible approach.
These guidelines represent a paradigm shift from the traditional documentation-focused Computer System Validation (CSV) to risk-based Computer Software Assurance (CSA), and it is expected that the use of this approach in appropriate scenarios will become even more widespread in the future to improve efficiency and speed up the overall testing process.
However, unscripted testing is not a panacea. For features with high process risk, particularly features that could directly affect product quality or patient safety, detailed scripted testing is still necessary. Therefore, an effective testing strategy requires appropriately combining scripted and unscripted testing and selecting the optimal approach for each feature based on risk assessment.
Medical device manufacturers, pharmaceutical manufacturers, and companies in regulated industries are recommended to prepare to integrate the CSA approach into their Quality Management Systems (QMS) in anticipation of the FDA’s harmonization with ISO 13485:2016 scheduled for February 2026. This will enable them to maintain compliance with regulatory requirements while achieving more efficient validation work and promoting innovation.
Comment