Signal Detection in Post-Market Surveillance Is Broken — Here’s Why
The Problem No One Actually Solves
Everyone agrees on the goal of post-market surveillance: detect safety signals early, before harm scales. That premise is rarely challenged. What is rarely examined is whether the systems we rely on are actually capable of doing that.
They aren’t.
Most organizations operate under the assumption that if they are collecting adverse events, logging complaints, and periodically reviewing trends, they are performing meaningful surveillance. In reality, they are aggregating events and hoping that signal emerges from accumulation. That approach works only when problems are already obvious. It fails precisely in the scenario post-market surveillance is supposed to address—the early, ambiguous phase where intervention actually matters.
The issue is not effort. It is that the system is built on the wrong foundation.
MDR Was Never Designed to Detect Signals
The U.S. Food and Drug Administration Medical Device Reporting framework is often treated as the backbone of surveillance in the United States. That interpretation is convenient, but incorrect.
MDR is a legal construct. It exists to ensure that certain adverse events are reported to the FDA within defined timelines. It is not a surveillance system in any meaningful sense. It does not measure risk, it does not estimate rates, and it does not provide a structured way to evaluate whether device performance is deviating from expectations.
What it produces is a collection of discrete, often incomplete reports of events that have already occurred. Those reports are subject to reporting bias, delayed submission, and highly variable clinical detail. The absence of an event in MDR data does not imply safety. The presence of an event does not imply elevated risk. Yet entire internal processes are built around interpreting these reports as if they were signals.
At best, MDR tells you that something has gone wrong. It does not tell you whether something is going wrong.
Without a Denominator, There Is No Signal
The deeper issue is more fundamental. Most post-market systems lack a denominator.
Counting adverse events in isolation is analytically meaningless. A device that generates fifty adverse events could be catastrophically unsafe or exceptionally safe depending on how often it is used. Without understanding exposure—how many times the device is used, in which patients, under what conditions—there is no way to interpret event counts.
This is where most surveillance efforts quietly break down. Organizations track what is easiest to capture: complaints, service records, MDR submissions. What they often do not track, at least not in a way that is integrated with safety analysis, is utilization.
The result is a system that produces numbers without context. Trends are interpreted based on volume rather than rate. Spikes trigger concern even when usage has increased proportionally. Conversely, stable counts can mask worsening performance if adoption is accelerating.
This is not a subtle statistical issue. It is a structural flaw.
Complaints Feel Rich but Behave Like Noise
If MDR data is sparse, complaint data creates the opposite illusion—depth.
Every organization has a complaint handling system. It is often the most detailed internal repository of post-market information, capturing narratives, device identifiers, user feedback, and investigation outcomes. On the surface, it appears to be an ideal source for signal detection.
In practice, it behaves like a noisy, weakly structured dataset that resists reliable analysis.
Complaint data is shaped by human behavior. Some sites report aggressively, others rarely. Some users provide detailed narratives, others provide minimal information. Categorization is inconsistent, and free-text descriptions carry most of the signal but are difficult to standardize. Duplicate reports and correlated events are common. Investigations introduce additional layers of interpretation, often influenced by internal incentives.
Organizations attempt to impose structure—coding complaints into categories, defining thresholds, running periodic trend reports—but this is essentially a post hoc cleanup of inherently unstable data. It can surface obvious problems. It does not reliably detect subtle ones.
The system gives the appearance of surveillance while functioning primarily as documentation.
Time Is the Hidden Failure Mode
Even when data is available, timing undermines its usefulness.
Most signal detection workflows are periodic. Data is reviewed monthly, quarterly, or in response to predefined triggers. This cadence is a legacy of compliance processes, not a reflection of how signals actually emerge.
In reality, safety signals develop gradually. They appear as small deviations, clusters of atypical events, or shifts in specific subpopulations. These patterns rarely cross predefined thresholds immediately. They require continuous observation to detect early.
When analysis is episodic, detection is delayed by design. By the time a signal is strong enough to be recognized in a quarterly review, it has often been present—quietly—for months. The system is not failing to detect the signal. It is detecting it late, which is functionally equivalent.
Context Is Where Most Systems Collapse
Even when organizations attempt more sophisticated analyses, they encounter a more difficult problem: lack of context.
An adverse event does not exist in isolation. Its significance depends on the patient, the indication, the operator, and the environment in which the device is used. A complication in a high-risk population may be expected. The same complication in a low-risk population may represent a meaningful signal.
Most surveillance systems do not incorporate this level of granularity. Events are logged, categorized, and counted, but rarely contextualized. Clinical variables are either absent or too inconsistent to support reliable analysis. Operational factors—training, workflow, site-specific practices—are almost never integrated.
This reduces signal detection to a superficial exercise. Patterns that depend on interactions between variables remain invisible.
Real-World Data Doesn’t Fix This Automatically
There is a growing assumption that integrating real-world data—electronic health records, claims, registries—will solve the limitations of traditional surveillance. It is an attractive idea, but it is often misunderstood.
Real-world data introduces scale and, importantly, denominators. It provides visibility into how devices are used and what outcomes follow. But it also introduces complexity: confounding, missing data, inconsistent coding, and delayed availability.
More importantly, real-world data is not plug-and-play. It does not become useful simply by being connected to a system. It requires careful definition of exposure, outcomes, and cohorts. It requires adjustment for baseline risk. It requires an understanding of what constitutes a meaningful deviation from expected performance.
Organizations that treat real-world data as an extension of complaint tracking—just more data to aggregate—do not solve the problem. They amplify it.
What Signal Detection Actually Requires
If you step back from the regulatory scaffolding and approach the problem directly, signal detection becomes much clearer.
The task is not to count events. It is to detect when observed outcomes deviate from what should be expected.
That requires, first, a clear understanding of exposure. Without knowing how often and in whom a device is used, there is no baseline against which to compare outcomes. Second, it requires an explicit or implicit model of expected performance. This can come from clinical literature, historical data, or external benchmarks, but it must exist. Third, it requires continuous monitoring, because deviations emerge over time, not at predefined reporting intervals.
None of these elements are conceptually complex. What is difficult is integrating them into systems that were originally designed for documentation and compliance.
Why the System Persists
Given these limitations, it is reasonable to ask why the current model persists.
Part of the answer is historical. The regulatory framework evolved in an era where data was limited, and reporting adverse events was itself a meaningful step forward. That structure remains in place even as the data environment has changed.
Part of the answer is organizational. Post-market surveillance sits at the intersection of regulatory, quality, clinical, and data functions, each with different incentives and priorities. Integration is difficult, and in many cases, not strongly incentivized.
But there is a more subtle factor. A system that detects signals earlier also forces earlier decisions. It creates pressure to investigate, to act, and sometimes to acknowledge uncertainty. Many organizations, consciously or not, optimize for defensibility rather than sensitivity. The current system supports that equilibrium.
Where This Is Heading
That equilibrium is becoming harder to maintain.
Devices are increasingly software-driven, continuously updated, and deployed at scale. The distinction between premarket and post-market performance is blurring. Regulators are signaling a shift toward lifecycle oversight, with greater emphasis on real-world evidence and continuous monitoring.
The existing model—built around MDR, complaints, and periodic review—does not scale to this environment. It was not designed to.
The next phase of post-market surveillance will not be defined by new reporting requirements. It will be defined by the ability to measure device performance continuously, in context, and relative to expectation.
The Bottom Line
Signal detection in post-market surveillance is not failing because of a lack of data or effort. It is failing because it is built on primitives that cannot support the task.
Event counts without denominators. Reports without context. Reviews without continuity.
If the goal is truly early detection of risk, the system has to shift from reporting to measurement. Until that happens, most “signals” will continue to be recognized only after they are no longer subtle—and no longer early.