Computer System Validation - Overview

Computer System Validation - Overview

An initial review of computer system validation (Article 23-002)

April 12, 2023

Purpose of Validation

Introduction

This article is intended to provide an overview of the reasons why validation is a necessary and critical part of software development, implementation, and maintenance in the life sciences. It reviews certain key drivers supporting the significant cost of validation and identifies some of the challenges that software manufacturers and system implementers/users face in achieving validation.

This is a complex and important topic. In addition (and because it is complex), regulatory bodies (FDA, EMEA, etc.) have issued guidances about validation that are decidedly non-prescriptive in detailing how to achieve validation. This is mainly because there are so many variations in computer systems, and the capabilities of systems advance so quickly that a guidance that was proscriptive would be out of date by the time it was approved. Because there are so many grey areas, there will be a number of other articles to follow that explore how best to navigate.

The main purpose of validation is to ensure that the computer systems used in clinical research are reliable and consistent, and that the documentation that proves this are clear and accurate. The goal is to prevent unexpected or unknown changes to the data the system manages.

The reason this is an important area is that clinical research, like all other areas, is wholly dependent on computer systems. The goal of clinical research is to improve people’s lives by making safer and more effective interventions available. Therefore, the review/approve process necessary for regulatory bodies to make those determinations must also ensure the systems that were used to collect the foundational data are reliable.

Data Validity

In the life sciences, we use computer systems to collect, manage, manipulate, and report data. Often this data is relevant to the health of a patient. The data may be used by healthcare providers (HCPs) to make decisions about a patient’s diagnosis and prognosis. In research, data collected as part of a clinical trial are instrumental in determining if an experimental drug or device is safe and effective. For these reasons (and many more), it is important that we have confidence in how a given computer system handles the data it contains.

Some example questions about the system are:

  1. Does it accurately collect the data - meaning, is the value it stores the exact value that was input?
  2. Does the collected value remain static over time - that is, are we sure that the value collected today will be the same value a year from now, assuming no other intentional system or manually introduced changes have occurred?
  3. If a data point is subject to a calculation or other type of conversion, does the system perform that change accurately and consistently?

Essentially, we use validation to gain confidence in a system that the only changes to data are those that are intentional - either by user input or due to programmatic action, and that all changes to data are known and controlled.

We use a capability of the system called an audit trail that requires the system to accurately track all changes to data and allows it to generate a report that documents changes a data point has undergone from the time it was entered until the present. An audity history report is essentially the natural history of that data point.

For each change, the audit trail captures:

  • Who made the change - if the system made the change, it may also be useful to capture the module or microservice(s) involved. If a user made the change, the user name must be captured. This ensures a) the user is allowed to access the system, and b) is allowed (has the appropriate privileges) to make the change.
  • When the change was made - the timestamp when the change was made. It’s important that this is captured as a central system time so that changes can be put into chronological sequence. The central system time (which may be in Greenwich Mean Time (GMT), may be displayed in the local time of a given client computer, but it must derive from the central time.
  • What change was made - the original or previous value of the data field and the new, updated value should be captured.
  • Why the change was made - if the system made the change, the reason may be as simple as, “System.” However, for user-initiated changes, a user entered Change Reason should be required to explain why the change was made.
  • Additional - for user-initiated changes, it may be useful to also capture an optional free-text comment regarding the data change. This is particularly useful when the system is configured to provide a set of standard Change Reasons, which makes reporting more consistent.

System Security

User access

In order to ensure the validity of all data, it is important that only properly credentialed users have access to the system. To that end, the system owner should put in place a robust identification mechanism that prevents unauthorized access. Examples of mechanisms commonly used are: username and password, single sign-on, and two-factor authentication. In conjunction with the mechanism, there should be procedures to support it with methods of oversight to ensure compliance.

Most systems are complex enough that multiple user roles are defined to support users who are tasked with performing different operations in the system. For that reason, a secondary security measure should be in place. Each user should be assigned to a designated role. Each role should be assigned a set of privileges. Each privilege provides the capability to perform certain functions within the system. Without the privilege, a user is unable access and/or perform the operation. So, most users who are given access to the system should only be allowed to perform those functions required and should be restricted from access or performing all other functions.

Another procedure that supports user access security, as well as the validity of data that is entered and modified, is user training. Each user must receive appropriate training prior to be granted access to the system. The training should be limited to the user role assigned and should ensure the user understands how to use the system to complete her assigned tasks.

System access

This refers to more than the system of interest. Obviously, that system must have access to itself. But, where necessary, constraints may be required. In particular, while the system is required to collect data (or “insert” in database parlance) throughout via the user interface or otherwise, there may be fields that the system should be precluded from changing (or “updating”) programmatically after the initial save.

Many systems are connected to others and utilize integrations with external systems to perform complementary activities, provide additional data, access the system’s data, perform analytics, generate reports, and many other functions. In order to create the integration, the systems must be connected. Each connection is a potential source for loss of data control.

Finally, it is important the system is protected against malicious interference by external systems and users. While the intentions of such actors varies, all intrusions represent the potential of unauthorized, un-tracked, and uncontrolled data corruption. It is important that the system is protected from such breaches and is able to withstand attacks and penetration attempts.

Conclusion

Computer system validation is a large topic. We covered the main points regarding its purpose but there many more aspects that are important. If you are contemplating implementing a new system, it is critical to all aspect of this project that you have a clear understanding of the purpose of validation so you can make decisions that are allow your new system to be adequately tested to ensure it meets your intended use reliably and consistently.

This is the first article - quite a number will follow, many expanding upon the topic of validation. However, my knowledge is based mostly on my experience and my understanding of the guidance documents. I welcome your thoughts and perspective on what is a surprisingly subjective topic.