Examining statistical methods to allow comprehensive evaluation and presentation of adverse event data in randomised controlled trials

Proposal
5815

Title of Proposed Research

Lead Researcher

Rachel Phillips

Affiliation

Imperial College London

Funding Source

Potential Conflicts of Interest

Data Sharing Agreement Date

04 March 2019

Lay Summary

BackgroundClinical trials look at new interventions (such as drugs) for the treatment of diseases, infections or disorders. We consider randomised controlled trials (RCTs) the best method to examine the performance of these new treatments. RCTs compare new treatments to existing treatments or dummy treatments called "placebos". We randomly allocate patients that agree to participate in RCTs to one of the treatments and compare their outcomes between treatment groups. We compare the treatments to help us decide whether to use the new treatment in clinical practice. A lot of work has gone into finding the best method to analyse how well these new treatments perform in terms of improving patients' outcomes. But the methods to analyse the harmful side effects are not as well developed. We call the harmful side effects experienced by patients participating in RCTs adverse events. Adverse events can include things such as headaches, vomiting, as well as effects that are more serious. The cause of these harmful effects may not be related to the new treatment. Events that are known or suspected to be related to the new treatment are called adverse drug reactions.Current practiceWe typically report side-effect data as the number of adverse events by treatment group. When comparing treatments, as there are often many adverse events, identifying important side effects involves reviewing a large amount of data. This process relies on either subjective evaluation or statistical approaches to test the data. Using a subjective approach has obvious weaknesses and current statistical methods are not appropriate for side-effect data. Statistical methods can fail to pick up important differences when present and flag more signals than desired when not present. Therefore, current statistical methods may result in more patients taking a treatment where the risk-benefit profile is unbalanced, or erroneously stopping a trial too early. In the last 15 years, statisticians have proposed a handful of alternative statistical methods to identify signals for adverse drug reactions but these methods are not used in clinical trials. We do not currently know why these methods are not being used but we do know they are complex to implement and have a number of methodological limitations. We have developed new statistical methods that are easy to implement in standard statistical software and can be used to monitor all adverse events in clinical trials. We already know that the time an adverse event occurs with respect to the start of treatment is a useful indicator of causality and this is incorporated to improve performance in several of the methods. Some of the methods also incorporate existing knowledge on the adverse event profile to improve performance further. We have tried these methods out on simulated data but we do not know how they work in practice. ObjectivesTo evaluate statistical methods for the analysis of adverse event and continuous safety data, and explore alternative approaches to the presentation of complex adverse event information using real-world clinical trial data.Output If we confirm these methods work well in real world clinical trial data, they would provide a useful tool for future studies. The use of appropriate methods have the potential to improve outcomes for future trial participants by identifying adverse drug reactions earlier than conventional methods and allowing better follow-up and monitoring of these events in later phase studies. In the longer term, improved analysis and presentation of this information will lead to more accurate risk-benefit assessments for patients and prescribers.

Study Data Provided

[{ "PostingID": 1632, "Title": "GSK-BRL-029060/377", "Description": "A Double-blind, Multicentre Placebo-Controlled Study of Paroxetine in Adolescents with Unipolar Major Depression." },{ "PostingID": 1637, "Title": "GSK-BRL-029060/701", "Description": "A multi-center, double blind, placebo-controlled flexible dose study to evaluate the efficacy and safety of Paroxetine in children with major depression." },{ "PostingID": 3146, "Title": "GSK-BRL-029060/676", "Description": "A 16 Week Double-Blind, Placebo-Controlled Study to Investigate the Efficacy and Tolerability of Paroxetine in the Treatment of Children and Adolescents with Social Anxiety Disorder/Social Phobia." },{ "PostingID": 3148, "Title": "GSK-BRL-029060/704", "Description": "A Randomized, Multi-center, 10-week, Double-blind, Placebo-Controlled, Flexible-Dose Study to Evaluate the Efficacy and Safety of Paroxetine in Children and Adolescents with Obsessive Compulsive Disorder" },{ "PostingID": 3793, "Title": "GSK-MEA115588", "Description": "MEA115588 A randomised, double-blind, double-dummy, placebo-controlled, parallel-group, multi-centre study of the efficacy and safety of mepolizumab adjunctive therapy in subjects with severe uncontrolled refractory asthma" },{ "PostingID": 3794, "Title": "GSK-MEA115575", "Description": "MEA115575: A Randomised, Double-Blind, Placebo-Controlled, Parallel-Group, Multicenter Study of Mepolizumab Adjunctive Therapy to Reduce Steroid Use in Subjects with Severe Refractory Asthma" },{ "PostingID": 4878, "Title": "GSK-200862", "Description": "A Randomised, Double-blind, Placebo-controlled, Parallel-group, Multi-centre 24-week Study to Evaluate the Efficacy and Safety of Mepolizumab Adjunctive Therapy in Subjects with Severe Eosinophilic Asthma on Markers of Asthma Control" },{ "PostingID": 14535, "Title": "GSK-MEA117106", "Description": "Study MEA117106: Mepolizumab vs. placebo as add-on treatment for frequently exacerbating COPD patients" },{ "PostingID": 14536, "Title": "GSK-MEA117113", "Description": "Study MEA117113: Mepolizumab vs. Placebo as Add-on Treatment for Frequently Exacerbating COPD Patients Characterized by Eosinophil Level" }]

Statistical Analysis Plan

The statistical methods are based on two approaches: 1) Time-to-event methods utilising a Bayesian approach to incorporate prior information for a selection of example cases, and uninformative prior distributions when implemented as a ‘screening tool' where event rates permit. 2) Regression based approaches as a ‘screening tool' will be examined for binary, count, and continuous data. Monte Carlo simulations are currently being undertaken for the time-to-event approaches to determine the performance of the proposed methods. For example, we are running simulations to examine the power and accuracy across varying trial characteristics e.g. sample size, background event rate.The simplest version of the proposed methods use the Weibull model to incorporate time-to-event and adjusts for treatment group as an ancillary parameter to allow for a separate estimate of the baseline hazard for each treatment group. This allows the shape parameter to differ between treatment groups, which we can use to detect a disproportionality in baseline hazards between groups that would be indicative of a temporal association and raise a flag for a signal of an adverse drug reaction (ADR).The preliminary test results have so far shown good performance in simulations. They have demonstrated good power for greater than 25 events and as low as 10 events dependent on relative timing of the event in the no control group study setting. In the requested clinical study data we will assess how frequently these tests are suitable and by what level of aggregation (e.g. preferred term, higher-level term etc.). For each study and event of interest we will assess whether each of the models correctly or incorrectly flags (or does not flag) an event. We will describe the trial scenarios (e.g. sample size, percentage of the AE in the control group, and ADR rate (percentage increase of the event in treatment group compared to the control group)) in which the models were able to pick up true signals for ADRs and when they were able to discriminate between true ADRs and false signals.We will not be conducting a meta-analysis of the study data but we will look within drug for a selected number of adverse events to incorporate data where previous study data is available. Earlier studies can provide prior information to inform later studies for known ADRs under a Bayesian framework. For example, we will examine the Bayesian Beta-Binomial and Gamma-Poisson models as a means to incorporate prior information about known ADRs and assess their ability at monitoring these events in future studies (Yao et al., 2013; Zhu et al., 2016). The Beta-Binomial model assumes the prior information (from earlier studies) follow a beta distribution and the distribution is updated at each analysis to give a posterior beta distribution. The posterior distribution is then used to calculate the probability that a predefined threshold for the risk difference is crossed and a signal is raised if a predetermined probability threshold is crossed. We will describe the trial scenarios (for both the prior information and ongoing study) in which the Bayesian models appropriately flag a signal for known ADRs.In addition, regression modelling approaches with adjustment for randomisation stratification variables will be explored. For repeated events, we will examine the performance of Poisson regression, negative binomial and Weibull count models, and logistic and linear regression models for binary and continuous outcomes, respectively. This exploration will include adjusting for drug exposure time within the model as well as the simple use of defined study populations including the analysis population e.g. the full intention-to-treat (ITT) population and the safety population defined as those that receive at least one dose (White et al., 2001). For each AE we will report the treatment effect and 95% confidence intervals (CIs). These results will be presented using the recently published guidelines for adverse event reporting to examine the value of displaying information in this way (Lineberry et al., 2016). Novel presentations of comprehensive AE data will also be produced such as volcano plots and dot plots (Lineberry et al. 2016; Amit et al., 2008; Zink et al., 2013).This work will allow us to examine the practically of these methods as well as determining the performance of each across varying trial AE rates and sample size. All analyses will be performed in Stata version 15 or later (StataCorp, 2017).

Publication Citation

Phillips R, Cro S, Wheeler G, Bond S, Morris T P, Creanor S et al. Visualising harms in publications of randomised controlled trials: consensus and recommendations BMJ 2022
DOI:10.1136/bmj-2021-068983