Pragmatic Free Trial Meta

Pragmatic Free Trial Meta is a free and non-commercial open data platform and infrastructure that facilitates research on pragmatic trials. It collects and distributes cleaned trial data, ratings, and evaluations using PRECIS-2. This allows for a variety of meta-epidemiological studies to compare treatment effect estimates across trials with different levels of pragmatism.

Background

Pragmatic trials provide evidence from the real world that can be used to make clinical decisions. However, the usage of the term "pragmatic" is not uniform and its definition and evaluation requires clarification. Pragmatic trials should be designed to inform clinical practice and policy decisions, rather than to prove an hypothesis that is based on a clinical or physiological basis. A pragmatic study should try to be as similar to the real-world clinical environment as possible, such as its participation of participants, setting up and design as well as the execution of the intervention, determination and analysis of the outcomes, and primary analyses. This is a significant difference between explanatory trials, as described by Schwartz and Lellouch1 which are designed to confirm a hypothesis in a more thorough way.

The trials that are truly pragmatic should not attempt to blind participants or healthcare professionals as this could result in bias in estimates of the effect of treatment. The trials that are pragmatic should also try to enroll patients from a wide range of health care settings to ensure that their findings can be applied to the real world.

Furthermore, trials that are pragmatic must be focused on outcomes that matter to patients, like quality of life and functional recovery. This is particularly relevant when it comes to trials that involve invasive procedures or those with potential dangerous adverse events. The CRASH trial29 compared a 2-page report with an electronic monitoring system for patients in hospitals with chronic cardiac failure. The catheter trial28, however was based on symptomatic catheter-related urinary tract infections as its primary outcome.

In addition to these features, pragmatic trials should minimize the trial procedures and data collection requirements in order to reduce costs. Additionally the aim of pragmatic trials is to make their results as relevant to actual clinical practices as possible. This can be achieved by ensuring their primary analysis is based on the intention-to treat method (as defined in CONSORT extensions).

Many RCTs that don't meet the criteria for pragmatism however, they have characteristics that are contrary to pragmatism, have been published in journals of varying types and incorrectly labeled pragmatic. This could lead to false claims of pragmatism, and the term's use should be made more uniform. The creation of the PRECIS-2 tool, which provides an objective and standard assessment of practical features is a great first step.

Methods

In a pragmatic research study it is the intention to inform clinical or policy decisions by demonstrating how an intervention can be integrated into routine treatment in real-world settings. Explanatory trials test hypotheses about the cause-effect relation within idealized environments. Therefore, pragmatic trials might be less reliable than explanatory trials and might be more susceptible to bias in their design, conduct and analysis. Despite their limitations, pragmatic research can be a valuable source of data for making decisions within the context of healthcare.



The PRECIS-2 tool scores an RCT on 9 domains, with scores ranging between 1 and 5 (very pragmatist). In this study the areas of recruitment, organisation as well as flexibility in delivery flexible adherence, and follow-up were awarded high scores. However, the principal outcome and method of missing data scored below the pragmatic limit. This suggests that it is possible to design a trial that has excellent pragmatic features without compromising the quality of its outcomes.

It is difficult to determine the degree of pragmatism within a specific trial because pragmatism does not have a single attribute. Certain aspects of a study can be more pragmatic than others. Furthermore, logistical or protocol modifications made during a trial can change its score on pragmatism. Additionally, 36% of the 89 pragmatic trials discovered by Koppenaal and colleagues were placebo-controlled, or conducted prior to approval and a majority of them were single-center. Therefore, they aren't very close to usual practice and can only be described as pragmatic in the event that their sponsors are supportive of the lack of blinding in these trials.

Furthermore, a common feature of pragmatic trials is that researchers attempt to make their findings more relevant by analyzing subgroups of the trial. This can result in unbalanced analyses that have lower statistical power. This increases the chance of missing or misdetecting differences in the primary outcomes. This was a problem during the meta-analysis of pragmatic trials because secondary outcomes were not corrected for differences in covariates at baseline.

In addition, pragmatic trials can also have challenges with respect to the collection and interpretation of safety data. This is due to the fact that adverse events are usually self-reported, and therefore are prone to errors, delays or coding variations. It is important to improve the accuracy and quality of the results in these trials.

Results

While the definition of pragmatism may not require that all trials are 100% pragmatic, there are advantages to incorporating pragmatic components into clinical trials. These include:

By including routine patients, the trial results are more easily translated into clinical practice. However, pragmatic trials may also have drawbacks. For instance, the right type of heterogeneity could help the trial to apply its findings to a variety of patients and settings; however the wrong kind of heterogeneity could reduce assay sensitivity and therefore reduce the power of a trial to detect even minor effects of treatment.

Several studies have attempted to classify pragmatic trials using different definitions and scoring methods. Schwartz and Lellouch1 have developed an approach to distinguish between explanation-based trials that support a physiological or clinical hypothesis and pragmatic trials that inform the selection of appropriate treatments in the real-world clinical setting. The framework was comprised of nine domains, each scored on a scale of 1 to 5 with 1 being more informative and 5 suggesting more pragmatic. https://www.openlearning.com/u/fengeroneal-sk2ogy/blog/PragmaticFreeSlotBuffItSNotAsDifficultAsYouThink included recruitment, setting, intervention delivery with flexibility, follow-up and primary analysis.

The initial PRECIS tool3 featured similar domains and scales from 1 to 5. Koppenaal et. al10 devised an adaptation of the assessment, known as the Pragmascope that was simpler to use for systematic reviews. They discovered that pragmatic reviews scored higher on average in most domains, but scored lower in the primary analysis domain.

This difference in the main analysis domain could be due to the fact that most pragmatic trials process their data in the intention to treat way while some explanation trials do not. The overall score was lower for pragmatic systematic reviews when the domains on organisation, flexible delivery and follow-up were merged.

It is crucial to keep in mind that a pragmatic study should not mean a low-quality trial. In fact, there are a growing number of clinical trials that use the word 'pragmatic,' either in their abstracts or titles (as defined by MEDLINE but which is neither precise nor sensitive). The use of these words in abstracts and titles could indicate a greater understanding of the importance of pragmatism, but it isn't clear if this is reflected in the contents of the articles.

Conclusions

In recent years, pragmatic trials are becoming more popular in research as the value of real-world evidence is increasingly recognized. They are randomized clinical trials that evaluate real-world alternatives to care instead of experimental treatments under development. They involve patient populations which are more closely resembling the ones who are treated in routine care, they use comparisons that are commonplace in practice (e.g. existing drugs), and they depend on the self-reporting of participants about outcomes. This method can help overcome the limitations of observational research, for example, the biases that are associated with the use of volunteers and the limited availability and codes that vary in national registers.

Other advantages of pragmatic trials are the ability to utilize existing data sources, and a greater likelihood of detecting meaningful changes than traditional trials. However, pragmatic tests may still have limitations which undermine their effectiveness and generalizability. For instance the participation rates in certain trials might be lower than expected due to the healthy-volunteer influence and financial incentives or competition for participants from other research studies (e.g. industry trials). Many pragmatic trials are also restricted by the necessity to enroll participants on time. Additionally, some pragmatic trials lack controls to ensure that the observed differences aren't due to biases in the conduct of trials.

The authors of the Pragmatic Free Trial Meta identified 48 RCTs self-labeled as pragmatist and published until 2022. The PRECIS-2 tool was employed to evaluate the degree of pragmatism. It includes domains such as eligibility criteria as well as recruitment flexibility as well as adherence to interventions and follow-up. They found that 14 of these trials scored highly or pragmatic practical (i.e. scores of 5 or higher) in one or more of these domains, and that the majority were single-center.

Trials that have a high pragmatism score tend to have higher eligibility criteria than traditional RCTs that have specific criteria that are unlikely to be found in clinical practice, and they contain patients from a broad variety of hospitals. The authors argue that these traits can make pragmatic trials more meaningful and useful for everyday practice, but they do not necessarily guarantee that a trial conducted in a pragmatic manner is free from bias. In addition, the pragmatism that is present in trials is not a definite characteristic A pragmatic trial that does not possess all the characteristics of a explanatory trial can yield reliable and relevant results.