Outcome Measures in Systemic Lupus Erythematosus: Constructing a Meaningful Response Index from Existing Clinical Trial Data

Proposal
911

Title of Proposed Research

Lead Researcher

Lindsy Forbess, MD, MSc

Affiliation

Cedars-Sinai Medical CenterLos Angeles, CAUS

Funding Source

We plan to apply for the NIAMS Clinical Trial Outcome Instrument Development Grant Program (UO1). The funding opportunity announcement (FOA) number is RFA-AR-14-008 and can be found here: http://grants.nih.gov/grants/guide/rfa-files/RFA-AR-14-008.html. This FOA solicits applications for research awards designed to develop new and evaluate existing clinical trial outcome instruments/measures to better assess the benefits (efficacy and effectiveness) and adverse impacts of therapies employed to treat diseases and injuries of interest to the NIAMS. This grant awards up to $150,000 per year for 3 years.

Potential Conflicts of Interest

None

Data Sharing Agreement Date

25 July 2014

Lay Summary

The purpose of this project is to develop a systemic lupus erythematosus (SLE) response index as the standard outcome measure in future trials. Currently, there is no widely accepted method for defining response to therapy and as a result, most SLE trials to date have failed to meet predesigned endpoints, leading to the controversy over whether it is the drug treatments or outcome measures that are unsuccessful in SLE. A similar controversy in rheumatoid arthritis (RA) many years ago was resolved by examining data from placebo-controlled trials with drugs that were only modestly effective. Important clinical variables were selected, criteria for patient improvement determined, and an index was developed that adequately distinguished treated patients from those getting placebo (1). This index (ACR 20/50/70) is used in RA trials and has led to approval of more than 20 drug therapies (2). Recently, a new response index, the SLE responder index (SRI), was developed by examining early phase trial data of belimumab in the treatment of SLE (3). Adjusting this index to later phase trials assisted in belimumab FDA approval, the first drug in almost 40 years to be approved for SLE.

Although the SRI attempts to standardize outcome measures for future SLE trials, it has not yet been widely adopted and has inherent flaws. It comprises criteria from 3 indices, SELENA-SLE Disease Activity Index (SELENA-SLEDAI), Physician Global Assessment, and the British Isles Lupus Assessment Group (BILAG). These instruments were developed from observational cohort data and not meaningful change for individual patients in clinical trials (4,5). The BILAG is available only in English, requires expensive software and extensive training, is time-consuming, and subject to frequent revision. The SELENA-SLEDAI is heavily weighted toward less common clinical manifestations, and does not reflect graded change in SLE activity over time. In addition, the SRI does not include patient-reported outcome instruments, which are important given the known discrepancies between patient and physician perceptions of disease activity (6). The FDA guidance document for developing SLE therapies encourages “the use of patient-reported outcome instruments to measure all relevant and important SLE symptoms” (7).

Now that large scale phase III SLE clinical trial data exist, we plan to move down the same path as in RA and deconstruct the elements that comprise the BILAG and SELENA-SLEDAI, select important clinical and laboratory variables, determine the criteria for patient improvement, and develop an index that distinguishes patients receiving treatment from those getting placebo. This index will be simple to use, based on real individual patient clinical trial data, developed for the purpose of future trial use, and include patient reported outcome measures. It should serve to prevent useful drugs from being discarded due to inadequate trial designs.

Study Data Provided

[{ "PostingID": 1416, "Title": "GSK-HGS1006-C1056", "Description": "A Phase 3, Multi-Center, Randomized, Double-Blind, Placebo-Controlled, 76-Week Study to Evaluate the Efficacy and Safety of Belimumab (HGS1006, LymphoStat-B™), a Fully Human Monoclonal Anti-BLyS Antibody, in Subjects with Systemic Lupus Erythematosus (SLE)

Medicine: belimumab, Condition: Systemic Lupus Erythematosus, Phase: 3, Clinical Study ID: HGS1006-C1056, Sponsor: GSK" },{ "PostingID": 1417, "Title": "GSK-HGS1006-C1057", "Description": "A Phase 3, Multi-Center, Randomized, Double-Blind, Placebo-Controlled, 52-Wk Study to Evaluate the Efficacy and Safety of Belimumab (HGS1006, LymphoStat-B™), a Fully Human Monoclonal Anti-BLyS Antibody, in Subjects With Systemic Lupus Erythematosus (SLE)

Medicine: belimumab, Condition: Systemic Lupus Erythematosus, Phase: 3, Clinical Study ID: HGS1006-C1057, Sponsor: GSK" }]

Statistical Analysis Plan

Table 1. Percentage of patients with individual improvement (_>x% improvement in _>y variables)
x% Improvement
Study _>10%_>20% _>30% _>40% _>50% _>60%
All Placebo Patients
All Drug Treated Patients (10mg/kg)
For each x% percentage of improvement, the number of variables improved by _>x% will be plotted on the x-axis and percent of patients with _>x% improvement will be plotted on the y-axis (see Figure 1 in article by Paulus et al) (1). For example, the number of variables (2,3,4,5,6) improved by _>20% will be plotted on the x-axis and percent of patients with _>20% will be plotted on the y-axis.

By inspection, the criterion definition incorporating _>x% improvement in y or more variables that seems to produce the largest difference between active drug-treated and placebo-treated patients, while retaining an acceptably low percentage of patients with individual improvement (about 5%) will be selected as the SLE response index. For example, among 6 variables (rash, arthritis, fatigue, complement, dsdna and physician global assessment), a definition incorporating _>20% improvement in 4 or more variables (Table 1, where y=4) best distinguished between drug-treated patients and placebo, while retaining a low placebo response rate. Using this definition of requiring 4 or more selected measures to be improved by _>20%, few placebo patients would qualify as improved, whereas significantly more patients receiving treatment would demonstrate improvement. This criterion definition of _>20% improvement in 4 or more variables is the SLE response index.

The new composite index will be developed using a 100-fold bootstrap sample (with replacement) of the training data set (11, 12). Variables and various cut-points selected in at least 50% of the bootstrap samples will be considered in the final modeling index. For each potential model developed, the accuracy of each model to discriminate between placebo and treated groups will be assessed using the c-index, defined as the concordance between predicted probability and actual outcome, and is identical to the area under the curve in a receiver operating characteristic (ROC) curve. The c index estimates the probability of concordance between predicted and observed responses. A value of 0.5 indicates no predictive discrimination and a value of 1.0 indicates perfect separation of patients with different outcomes.

Based on a preliminary analysis of ROC curves provided by GSK of only 8 individual variables [change in physician global assessment (PGA), FACIT-fatigue scale, SF-36 (PCS and MSC scores), double-stranded DNA, complement (C3 and C4), and C-reactive protein (CRP) levels] from the BLISS-52 and 76 studies, only the percent change in C4 levels from baseline to week 52 demonstrated a significant ability to discriminate between placebo and active treated patients. The AUC for percent change in C4 levels was 0.71 with an ideal cutpoint for maximum sensitivity and specificity of a +15% increase. Based on these preliminary results, we feel any new composite index will have a high likelihood of achieving a high level of discrimination between placebo and active treated groups.

The composite index developed from Aim 2 will then be applied to the remaining set of study subjects for validation as part of Aim 3. Each subject in the validation set will be scored according to the newly developed SLE index. Using logistic regression, the SLE index will be used to predict group assignment. The ability of the new SLE Index to distinguish between placebo and treated groups (discrimination) will be quantified by measuring the area under the receiver operating characteristic curve (AUC). Predictive accuracy will be quantified through the c- index (13). We will require the c-statistic to be >0.8 to confirm validity of the SLE Index (14). All analysis will be performed using SAS v9.3 software. The statistical analysis plan will be added after the research is published.

Publication Citation

Failure of a systemic lupus erythematosus response index developed from clinical trial data: lessons examined and learned. Forbess LJ, Bresee C, Wallace DJ, Weisman MH. Lupus. 2017 Jan 1:961203317692433. doi: 10.1177/0961203317692433. [Epub ahead of print] PMID: 28173737

https://journals.sagepub.com/doi/10.1177/0961203317692433