Pragmatic Free Trial Meta

Pragmatic Free Trail Meta is an open data platform that facilitates research into pragmatic trials. It collects and distributes cleaned trial data, ratings and evaluations using PRECIS-2. This permits a variety of meta-epidemiological analyses that evaluate the effects of treatment across trials of different levels of pragmatism.

Background

Pragmatic studies provide real-world evidence that can be used to make clinical decisions. However, the use of the term "pragmatic" is not consistent and its definition as well as assessment requires clarification. Pragmatic trials are intended to guide clinical practices and policy decisions, not to confirm a physiological hypothesis or clinical hypothesis. A pragmatic study should strive to be as close as it is to actual clinical practices that include recruiting participants, setting, designing, delivery and implementation of interventions, determination and analysis results, as well as primary analysis. This is a significant difference between explanation-based trials, as defined by Schwartz and Lellouch1, which are designed to confirm the hypothesis in a more thorough way.

Trials that are truly pragmatic must avoid attempting to blind participants or clinicians, as this may result in bias in estimates of the effect of treatment. Pragmatic trials should also seek to attract patients from a wide range of health care settings to ensure that the results can be applied to the real world.

Finally studies that are pragmatic should focus on outcomes that are vital for patients, such as quality of life or functional recovery. This is particularly important when it comes to trials that involve the use of invasive procedures or potentially serious adverse events. The CRASH trial29, for example focused on the functional outcome to compare a 2-page case-report with an electronic system for the monitoring of patients in hospitals suffering from chronic heart failure. In addition, the catheter trial28 utilized symptomatic catheter-associated urinary tract infections as its primary outcome.



In addition to these features pragmatic trials should also reduce the procedures for conducting trials and requirements for data collection to cut down on costs and time commitments. Finaly, pragmatic trials should aim to make their results as relevant to real-world clinical practices as possible. This can be accomplished by ensuring their primary analysis is based on the intention to treat method (as described in CONSORT extensions).

Despite these guidelines however, a large number of RCTs with features that defy the concept of pragmatism have been mislabeled as pragmatic and published in journals of all kinds. This can lead to false claims of pragmaticity, and the usage of the term needs to be standardized. The creation of a PRECIS-2 tool that provides an objective, standardized assessment of pragmatic features is a first step.

Methods

In a practical trial it is the intention to inform clinical or policy decisions by showing how an intervention could be incorporated into real-world routine care. This is distinct from explanation trials that test hypotheses regarding the cause-effect relationship in idealised conditions. Therefore, pragmatic trials could have less internal validity than explanatory trials and might be more susceptible to bias in their design, conduct and analysis. Despite their limitations, pragmatic studies can provide valuable data for making decisions within the healthcare context.

The PRECIS-2 tool evaluates an RCT on 9 domains, with scores ranging between 1 and 5 (very pragmatist). In this study, the areas of recruitment, organisation and flexibility in delivery, flexible adherence and follow-up scored high. However, the principal outcome and method of missing data were scored below the practical limit. This suggests that it is possible to design a trial that has good pragmatic features without damaging the quality of its results.

It is hard to determine the level of pragmatism within a specific trial because pragmatism does not have a binary attribute. Certain aspects of a study can be more pragmatic than others. Moreover, protocol or logistic changes during an experiment can alter its pragmatism score. Koppenaal and colleagues found that 36% of the 89 pragmatic studies were placebo-controlled, or conducted prior to the licensing. Most were also single-center. Therefore, they aren't as common and can only be described as pragmatic when their sponsors are accepting of the lack of blinding in these trials.

Another common aspect of pragmatic trials is that the researchers attempt to make their findings more meaningful by analysing subgroups of the sample. This can result in imbalanced analyses and less statistical power. This increases the chance of omitting or misinterpreting differences in the primary outcomes. In the case of the pragmatic trials included in this meta-analysis this was a major issue because the secondary outcomes weren't adjusted for the differences in the baseline covariates.

Additionally practical trials can be a challenge in the gathering and interpretation of safety data. It is because adverse events are usually self-reported and are susceptible to delays, inaccuracies or coding variations. It is therefore crucial to enhance the quality of outcomes for these trials, in particular by using national registry databases instead of relying on participants to report adverse events in the trial's database.

Results

Although the definition of pragmatism may not require that all trials are 100% pragmatic, there are some advantages to incorporating pragmatic components into clinical trials. These include:

Increasing sensitivity to real-world issues, reducing cost and size of the study as well as allowing trial results to be faster implemented into clinical practice (by including patients from routine care). However, pragmatic trials can also have drawbacks. For instance, the right type of heterogeneity could help the trial to apply its results to many different settings and patients. However the wrong kind of heterogeneity could reduce assay sensitiveness and consequently decrease the ability of a trial to detect minor treatment effects.

Many studies have attempted categorize pragmatic trials using various definitions and scoring methods. Schwartz and Lellouch1 created a framework to discern between explanation-based studies that support a physiological hypothesis or clinical hypothesis and pragmatic studies that help inform the selection of appropriate treatments in real world clinical practice. The framework was comprised of nine domains, each scored on a scale ranging from 1-5, with 1 being more informative and 5 suggesting more pragmatic. The domains included recruitment of intervention, setting up, delivery of intervention, flex adherence and primary analysis.

The original PRECIS tool3 was built on the same scale and domains. Koppenaal et al10 created an adaptation of this assessment called the Pragmascope that was simpler to use in systematic reviews. They found that pragmatic reviews scored higher on average in most domains, but scored lower in the primary analysis domain.

This distinction in the analysis domain that is primary could be due to the fact that most pragmatic trials analyse their data in an intention to treat manner, whereas some explanatory trials do not. The overall score for systematic reviews that were pragmatic was lower when the domains of organization, flexible delivery, and follow-up were merged.

It is important to understand that the term "pragmatic trial" does not necessarily mean a poor quality trial, and in fact there is an increasing number of clinical trials (as defined by MEDLINE search, but this is neither sensitive nor specific) that use the term 'pragmatic' in their title or abstract. The use of these terms in abstracts and titles could suggest a greater awareness of the importance of pragmatism, but it isn't clear if this is reflected in the contents of the articles.

Conclusions

In recent times, pragmatic trials are becoming more popular in research as the value of real-world evidence is increasingly recognized. They are randomized clinical trials that compare real-world care alternatives instead of experimental treatments under development. They have populations of patients which are more closely resembling the patients who receive routine care, they employ comparators which exist in routine practice (e.g., existing medications), and they rely on participant self-report of outcomes. This method is able to overcome the limitations of observational research like the biases that are associated with the use of volunteers and the limited availability and coding variations in national registries.

Pragmatic trials have other advantages, like the ability to use existing data sources, and a greater probability of detecting meaningful differences than traditional trials. However, these tests could be prone to limitations that undermine their validity and generalizability. Participation rates in some trials may be lower than anticipated because of the healthy-volunteering effect, financial incentives or competition from other research studies. The necessity to recruit people in a timely fashion also restricts the sample size and impact of many pragmatic trials. Additionally certain pragmatic trials lack controls to ensure that the observed differences are not due to biases in the conduct of trials.

The authors of the Pragmatic Free Trial Meta identified 48 RCTs self-labeled as pragmatist and published until 2022. They assessed pragmatism by using the PRECIS-2 tool, which consists of the eligibility criteria for domains and recruitment criteria, as well as flexibility in adherence to intervention, and follow-up. https://squareblogs.net/santatea8/15-startling-facts-about-pragmatic-slot-experience-you-didnt-know discovered that 14 of the trials scored as highly or pragmatic pragmatic (i.e. scoring 5 or higher) in any one or more of these domains and that the majority of them were single-center.

Trials with a high pragmatism rating tend to have more expansive eligibility criteria than traditional RCTs, which include very specific criteria that aren't likely to be found in clinical practice, and they contain patients from a broad range of hospitals. These characteristics, according to the authors, can make pragmatic trials more useful and relevant to the daily clinical. However, they cannot ensure that a study is free of bias. The pragmatism characteristic is not a fixed characteristic the test that does not possess all the characteristics of an explicative study could still yield valid and useful outcomes.