Understanding Data Retention Requirements in Pharmaceutical Regulations

What is Data Retention?

When preserving documents, paper-based media can be stored semi-permanently under appropriate conditions. In contrast, electronic storage media such as hard disk drives, solid-state drives (SSDs), and optical discs face challenges related to deterioration over time. Regardless of quality, the guaranteed lifespan of these media is generally limited to approximately 5 years for consumer-grade SSDs and 3-10 years for other electronic media, depending on usage conditions and storage environment. Even high-quality enterprise-grade storage solutions typically offer warranty periods of 5-10 years at most.

After the warranty period expires, there is a risk that stored electronic records and electronic signatures may be lost, altered, or become unreadable. Even if the media itself has a long warranty period, additional challenges arise such as the discontinuation of drives or software needed to read the data, or format obsolescence due to technological advancement. To ensure data retention throughout the required preservation period, it is necessary to periodically copy records to new media (a process known as migration) at intervals appropriate to the characteristics of the electronic storage media being used. Therefore, ensuring long-term data retention involves considerable ongoing effort and systematic planning.

Regulatory Requirements for Data Retention

The Japanese ER/ES Guidelines (Guidelines on the Use of Electronic Records and Electronic Signatures in Applications for Approval or License of Pharmaceutical Products, etc., issued by the Ministry of Health, Labour and Welfare on April 1, 2005, and updated in March 2024) specify the following requirement for data retention: “Electronic records must be preserved in a state where authenticity and readability are ensured throughout the retention period.”

“Ensuring authenticity” means that the system must be protected by appropriate security measures, with audit trails preserved alongside the records, and that backups of both records and audit trails have been created and are properly maintained. This encompasses several critical elements:

• Clear identification of record creators through unique user IDs
• Automatic generation and preservation of audit trails documenting all data modifications
• Secure backup systems with verified restoration procedures
• Access controls and user authentication mechanisms
• Protection against unauthorized modifications

“Ensuring readability” means that a mechanism exists to retrieve and display the electronic records in human-readable form throughout the retention period, and that relevant metadata, data dictionaries (masters), and documentation are preserved together with the records. This includes:

• Maintaining compatible software and hardware systems
• Preserving format specifications and conversion tools
• Storing master data, code lists, and reference tables
• Documenting system configurations and procedures
• Ensuring compatibility with current viewing technologies

Contemporary Regulatory Context: Data Integrity

In recent years, regulatory authorities worldwide have placed increased emphasis on Data Integrity (DI) as a fundamental requirement that encompasses and extends traditional ER/ES requirements. The ALCOA+ principles (also known as ALCOA CCEA), established by the FDA and expanded by the EMA and MHRA, have become the international standard for ensuring data integrity throughout the pharmaceutical product lifecycle.

The ALCOA+ principles consist of nine fundamental requirements:

ALCOA (Original Five Principles):

• Attributable: Records must clearly identify who performed the action and when
• Legible: Records must be readable and understandable throughout their lifecycle
• Contemporaneous: Records must be created at the time the activity is performed
• Original: The first or source record must be preserved in its original form
• Accurate: Records must be correct, truthful, and reflect actual observations

CCEA (Additional Four Requirements):

• Complete: All data necessary to reconstruct an event must be available
• Consistent: Records must be internally consistent with no contradictions
• Enduring: Records must remain accessible throughout the required retention period
• Available: Records must be readily accessible for review and audit when needed

These principles apply to all data throughout the pharmaceutical lifecycle, from research and development through manufacturing, quality control, clinical trials, and post-marketing surveillance. Modern electronic systems must be validated to demonstrate compliance with these principles, and companies are expected to conduct regular risk assessments to identify and mitigate data integrity vulnerabilities.

Important Distinction: Preservation vs. Backup

It is crucial not to confuse “preservation” with “backup,” as they serve different purposes and require different approaches:

Preservation refers to maintaining records in a state where they can be continuously searched, retrieved, and viewed throughout the required retention period. Preserved records must be readily accessible and maintained in their approved storage location. This is the primary copy used for ongoing business operations and regulatory inspections.

Backup refers to creating duplicate copies of records for disaster recovery and business continuity purposes. Backup copies are typically stored in a separate location and are only accessed when the primary preserved records are unavailable due to system failure or disaster.

Due to this fundamental difference, certain media types are suitable for different purposes:

• Magnetic tape: Appropriate for backup purposes due to cost-effectiveness and high capacity, but unsuitable for preservation because records cannot be readily searched or retrieved without restoring the entire tape
• Online storage systems: Suitable for preservation as they allow immediate search and retrieval
• Cloud-based storage: Increasingly used for both preservation and backup, provided the service meets regulatory requirements for security, data integrity, and accessibility

Modern Storage Technologies and Migration Strategies

The evolution of storage technologies has introduced new considerations for long-term data retention:

Solid-State Drives (SSDs): While SSDs offer superior performance and reliability compared to traditional hard drives, they typically have a warranty period of 3-5 years. Enterprise-grade SSDs may offer longer warranties (5-10 years) based on their Total Bytes Written (TBW) specifications. Organizations must plan for systematic migration before warranty expiration.

Cloud Storage Solutions: Cloud-based storage systems have become prevalent in pharmaceutical companies, offering several advantages:

• Geographic redundancy and disaster recovery capabilities
• Automatic backup and version control
• Scalability to accommodate growing data volumes
• Regular system updates and maintenance by service providers

However, cloud storage introduces new considerations:

• Vendor qualification and ongoing oversight
• Data sovereignty and regulatory compliance across jurisdictions
• Contractual guarantees for long-term data accessibility
• Migration planning in case of vendor changes

Hybrid Approaches: Many organizations now employ hybrid strategies combining on-premises storage for active data with cloud archival for long-term retention, balancing immediate accessibility with cost-effectiveness and regulatory compliance.

Implementation of Effective Data Retention Programs

To maintain compliance with current regulations and ensure long-term data retention, organizations should implement comprehensive programs that include:

Regular Migration Planning: Establish a systematic schedule for data migration based on media characteristics, manufacturer specifications, and risk assessments. Migration should occur well before warranty expiration, typically at 60-70% of the rated lifespan.

Validation and Verification: Each migration must be validated to ensure data integrity is maintained. This includes:

• Verification of data completeness and accuracy after migration
• Confirmation that audit trails and metadata are preserved
• Testing of data readability and accessibility
• Documentation of the migration process

Technology Monitoring: Continuously monitor technological developments and plan for format migrations when:

• Hardware or software becomes obsolete
• Vendors announce end-of-life for critical components
• New regulatory requirements emerge
• Security vulnerabilities are identified

Quality Management Systems: Integrate data retention requirements into the overall Quality Management System with:

• Clear procedures for data preservation and backup
• Defined roles and responsibilities
• Regular audits and inspections
• Continuous improvement processes

Risk-Based Approach to Data Retention

Modern regulatory guidance emphasizes a risk-based approach to data retention. Organizations should:

1. Conduct risk assessments to identify critical data and potential failure modes
2. Prioritize resources based on the importance and vulnerability of different data types
3. Implement controls proportionate to the level of risk
4. Document risk decisions and mitigation strategies
5. Regularly review and update risk assessments as systems and regulations evolve

This risk-based approach allows organizations to focus their efforts on the most critical aspects of data retention while maintaining overall compliance with regulatory requirements.

Conclusion

Ensuring proper data retention in accordance with ER/ES Guidelines and contemporary Data Integrity principles requires careful planning, systematic implementation, and ongoing vigilance. As technology continues to evolve and regulatory expectations increase, organizations must remain proactive in their approach to electronic record preservation, balancing the benefits of new technologies with the fundamental requirements for authenticity, readability, and retention throughout the required preservation period. The integration of ALCOA+ principles into data management systems and processes has become essential for pharmaceutical companies operating in the global regulatory environment.

Note: This article reflects regulatory requirements as of January 2026. Organizations should consult the latest regulatory guidance and qualified professionals for specific compliance requirements.