When making decisions about investigating customer complaints using severity of harm, make sure the severity levels are correctly and consistently applied.

In a recent warning letter, the FDA cited a significant deficiency in the Complaints Handling procedure at a manufacturer of a variety of Class II oxygen concentrators.

As summarized in the slide above, the FDA found the "Customer Feedback and Complaint Entry" procedure to be inadequate because the firm failed to request the product back for investigation when the potential severity of harm was higher than anticipated. This is because the procedure assigned the expected severity level to be lower than the threshold for requesting product back for investigation. 

In short, the FDA did not agree with the severity assigned to potential harms such as allergic reaction, chemical burns, hypoxia and hypercapnia. 

Notice the specific example they are using as evidence that the severity level for hypoxia is not correct, and likely higher than the score defined in the "Harms Severity Table Guidelines". In this case, the patient had to be taken to the hospital when the product was not supplying sufficient amount of oxygen.

Although we do not have access to the specific severity scores used by the manufacturer, we can make an educated guess. When medical or surgical intervention is required to preclude permanent impairment, typically a severity score of 3 is assigned on a 1-5 scale. Let us say the assigned severity score was a 2, which generally implies that no medical intervention is needed. This assumption is supported by the manufacturer's initial response to the FDA (find details here CMS#617319), where they provided a justification for the current score by pointing to the device labeling which instructs the user to have a backup supply in case the device malfunctions. The FDA did not accept this justification because the patient had to be given oxygen at the hospital to preclude permanent impairment of a body function or structure or possibly death. 

The FDA finding is against the requirements of Complaint Files, specifically 21 CFR 820.198(a), which require manufacturers to establish and maintain procedures for receiving, reviewing, and evaluating complaints by a formally designated unit. The expectation is that these procedures must be adequate.

In this case, the FDA determined the procedures to be inadequate, because they failed to lead to a decision to request the product back for investigation due to incorrect severity levels assigned to potential harms. 

 From a risk management point of view, this situation illustrates how incorrectly assigned harm severity levels can lead to an ineffective process for monitoring and evaluating risks in the production and post-production phase of the device lifecycle. When malfunctioning products are not investigated, we cannot identify contributing factors or root cause(s) that should be fixed through the CAPA process. 

So how do you prevent this problem?

To prevent this problem, it is first important to understand some of the reasons incorrect or inconsistent harm severity levels may be assigned during the risk analysis. 

One common challenge in defining an appropriate severity level to a potential harm is that we often cannot properly articulate all potential sequence of events that may lead to a hazardous situations where the patient or user is exposed to one or more hazards, which may lead to one or more harms. 

This relationship between a failure event (e.g. a device malfunction), sequence of events, hazardous situation, hazard and harm is illustrated in the following figure:

 Generally a trigger event, such as an oxygen concentrator not supplying enough oxygen to the patient, leads to a sequence of events that creates a hazardous situation for the patient. In this case, the hazardous situation is low oxygen level (i.e. hypoxia). Note that any intervention, including medical or surgical, may lead to another sequence of events and hazardous situation where additional hazards may be present. As a result, a patient or user may experience multiple harms from a single trigger event. 

In practice, it is difficult to correctly and comprehensively identify, analyze and articulate all potential scenarios. Engineers are generally focused on device failures and underlying causes. Medical professionals on the other hand focus on hazardous situations and harms.

Therefore, this analysis is best done in a cross-functional setting that includes both engineers and medical professionals with actual field experience. Even when a cross-functional team with the right kind of expertise is involved, multiple severity levels ranging from low to high may be assigned to the same harm potentially occurring from a range of different scenarios. 

This situation leads to inconsistent application of severity levels for the same harm by different teams during risk analysis. That is why we generally advise our clients to only use the maximum potential severity level in the master table used to make important risk-based decisions throughout the organization. 

So how should we address the FDA's concern in this case?

If the manufacturer had correctly identified a potential scenario where a patient would need to be taken to the hospital in case of the device malfunction, they would have assigned a more appropriate higher severity level to the risk of hypoxia. While it is certainly possible that the severity of some other scenarios would have been lower, as shown in the following figure, the maximum potential severity would have been 3 or higher on a 1-5 scale.  Note that the following figure only provides a few examples for the purpose of illustration only.

This exercise is normally done during hazard analysis to identify different sequence of events, hazardous situations, associated hazards and potential harms. Further, this is not a one-and-done exercise. In the early phase of development, a preliminary hazard analysis is performed. This is continually updated based on post-market surveillance that may identify new hazardous situations, associated hazards and harms. 

A better response to the FDA's finding would be to review the hazard analysis to ensure that all potential scenarios are considered and appropriate harm severity levels are assigned. Ideally, the severity threshold for requesting product back from the field for confirmed malfunctions should be removed. All malfunctioning products should be investigated whenever possible. 

Summary

In summary, consider the following best practices:

1. Initiate a preliminary hazard analysis (PHA) during early product development. You can find a free PHA template here. 
2. Clearly understand sequence of events leading to hazardous situations where the patient or user is exposed to one or more hazards. This relationship should be clearly described in the PHA document. Recognize that multiple sequence of events may be triggered by medical intervention and additional hazards and hazardous situations may be relevant. 
3. Use the best information and medical expertise available at the time of hazard analysis. Continually update this analysis throughout the product lifecycle based on information gathered through your post-market surveillance process.
4. Use only one, maximum potential severity level for each individual harm term in the standard harms table. This will ensure that the risk of harm is assessed and evaluated based on the most severe outcome regardless of the hazardous situation. 
5. It is best not to use the severity level as a criteria to justify not investigating products with a confirmed malfunction. You should always consider the maximum potential severity level regardless of the actual harm experienced by the patient. 

Keeping patient safety in mind will help you drive the right focus and decision making throughout the organization. This is the essential objective of an effective risk management process.