rpact: Confirmatory Adaptive Clinical Trial Design and Analysis

Summary

This R Markdown document provides an example for updating the group sequential boundaries when using an alpha spending function approach based on observed information rates in rpact.

1 Introduction

Group-sequential designs based on \(\alpha\)-spending functions protect the Type I error exactly even if the pre-planned interim schedule is not exactly adhered to. However, this requires re-calculation of the group-sequential boundaries at each interim analysis based on actually observed information fractions. Unless deviations from the planned information fractions are substantial, the re-calculated boundaries are quite similar to the pre-planned boundaries and the re-calculation will affect the actual test decision only on rare occasions.

Importantly, it is not allowed that the timing of future interim analyses is “motivated” by results from earlier interim analyses as this could inflate the Type I error rate. Deviations from the planned infromation fractions should thus only occur due to operational reasons (as it is difficult to hit the exact number of events exactly in a real trial) or due to external evidence.

The general principles for these boundary re-calculation are as follows (see also, Wassmer & Brannath, 2016, p78f):

  • Upates at interim analyses prior to the final analysis:
    • Information fractions are updated according to the actually observed information fraction at the interim analysis relative to the planned maximum information.
    • The planned \(\alpha\)-spending function is then applied to these updated information fractions.
  • Updates at the final analysis in case the observed information at the final analysis is larger (“over-running”) or smaller (“under-running”) than the planned maximum information:
    • Information fractions are updated according to the actually observed information fraction at all interim analyses relative to the observed maximum information. \(\Rightarrow\) Information fraction at final analysis is re-set to 1 but information fractions for earlier interim analyses are also changed.
    • The originally planned \(\alpha\)-spending function cannot be applied to these updated information fractions because this would modify the critical boundaries of earlier interim analyses which is clearly not allowed. Instead, one uses the \(\alpha\) that has actually been spent at earlier interim analyses and spends all remaining \(\alpha\) at the final analysis.

This general principle be implemented via a user-defined \(\alpha\)-spending function and is illustrated for an example trial with a survival endpoint below.

First, load the rpact package

library(rpact)
packageVersion("rpact") # version should be version 2.0.5 or later
## [1] '3.0.3'

2 Original trial design

The original trial design for this example is based on a standard O’Brien & Fleming type \(\alpha\)-spending function with planned efficacy interim analyses after 50% and 75% of information as specified below.

# Initial design
design <- getDesignGroupSequential(sided = 1, alpha = 0.025, beta = 0.2,
    informationRates = c(0.5, 0.75, 1), typeOfDesign = "asOF")
    
# Initial sample size calculation
sampleSizeResult <- getSampleSizeSurvival(
    design = design, 
    lambda2 = log(2)/60,hazardRatio = 0.75,
    dropoutRate1 = 0.025, dropoutRate2 = 0.025, dropoutTime = 12,
    accrualTime = 0,accrualIntensity = 30,
    maxNumberOfSubjects = 1000)
  
# Summarize design
summary(sampleSizeResult)
## Sample size calculation for a survival endpoint
## 
## Sequential analysis with a maximum of 3 looks (group sequential design).
## The sample size was calculated for a two-sample logrank test (one-sided), 
## H0: hazard ratio = 1, H1: hazard ratio = 0.75, control lambda(2) = 0.012, power 80%.
## 
## Stage                                         1      2      3 
## Information rate                            50%    75%   100% 
## Efficacy boundary (z-value scale)         2.963  2.359  2.014 
## Overall power                            0.1680 0.5400 0.8000 
## Number of subjects                       1000.0 1000.0 1000.0 
## Cumulative number of events                 194    291    387 
## Analysis time                              39.1   52.7   69.1 
## Expected study duration                    58.0 
## Cumulative alpha spent                   0.0015 0.0096 0.0250 
## One-sided local significance level       0.0015 0.0092 0.0220 
## Efficacy boundary (t)                     0.653  0.758  0.815 
## Exit probability for efficacy (under H0) 0.0015 0.0081 
## Exit probability for efficacy (under H1) 0.1680 0.3720 
## 
## Legend:
##   (t): approximate treatment effect scale

3 Boundary and power update at the first interim analysis

Assume that the first interim was conducted after 205 rather than the planned 194 events.

The updated design is calculated as per the code below. Note that for the calculation of boundary values on the treatment effect scale, we use the function getPowerSurvival() with the updated design rather than the function getSampleSizeSurvival() as we are only updating the boundary, not the sample size or the maximum number of events.

# Update design using observed information fraction at first interim.
# Information fraction of later interim analyses is not changed.
designUpdate1 <- getDesignGroupSequential(sided = 1, alpha = 0.025, beta = 0.2,
    informationRates = c(205/387,0.75,1), typeOfDesign = "asOF")

# Recalculate the power to get boundary values on the effect scale 
# (Use original maxNumberOfEvents and sample size)
powerUpdate1 <- getPowerSurvival(
    design = designUpdate1, 
    lambda2 = log(2)/60,hazardRatio = 0.75,
    dropoutRate1 = 0.025, dropoutRate2 = 0.025, dropoutTime = 12,
    accrualTime = 0,accrualIntensity = 30,
    maxNumberOfSubjects = 1000, maxNumberOfEvents = 387, directionUpper = FALSE)
powerUpdate1
## Design plan parameters and output for survival data:
## 
## Design parameters:
##   Information rates                            : 0.530, 0.750, 1.000 
##   Critical values                              : 2.867, 2.366, 2.015 
##   Futility bounds (non-binding)                : -Inf, -Inf 
##   Cumulative alpha spending                    : 0.002073, 0.009649, 0.025000 
##   Local one-sided significance levels          : 0.002073, 0.008998, 0.021968 
##   Significance level                           : 0.0250 
##   Type II error rate                           : 0.2000 
##   Test                                         : one-sided 
## 
## User defined parameters:
##   Direction upper                              : FALSE 
##   lambda(2)                                    : 0.0116 
##   Hazard ratio                                 : 0.750 
##   Maximum number of subjects                   : 1000 
##   Maximum number of events                     : 387 
##   Accrual intensity                            : 30 
##   Drop-out rate (1)                            : 0.025 
##   Drop-out rate (2)                            : 0.025 
## 
## Default parameters:
##   Type of computation                          : Schoenfeld 
##   Theta H0                                     : 1 
##   Planned allocation ratio                     : 1 
##   kappa                                        : 1 
##   Piecewise survival times                     : 0.00 
##   Drop-out time                                : 12.00 
## 
## Sample size and output:
##   median(1)                                    : 80.0 
##   median(2)                                    : 60.0 
##   lambda(1)                                    : 0.00866 
##   Accrual time                                 : 33.33 
##   Follow up time                               : 35.81 
##   Analysis times [1]                           : 40.60 
##   Analysis times [2]                           : 52.73 
##   Analysis times [3]                           : 69.14 
##   Expected study duration                      : 57.75 
##   Maximal study duration                       : 69.14 
##   Cumulative number of events [1]              : 205.0 
##   Cumulative number of events [2]              : 290.2 
##   Cumulative number of events [3]              : 387.0 
##   Expected number of events                    : 317.0 
##   Number of subjects [1]                       : 1000.0 
##   Number of subjects [2]                       : 1000.0 
##   Number of subjects [3]                       : 1000.0 
##   Expected number of subjects                  : 1000.0 
##   Reject per stage [1]                         : 0.2097 
##   Reject per stage [2]                         : 0.3294 
##   Reject per stage [3]                         : 0.2610 
##   Overall reject                               : 0.8001 
##   Early stop                                   : 0.5391 
##   Critical values (treatment effect scale) [1] : 0.670 
##   Critical values (treatment effect scale) [2] : 0.758 
##   Critical values (treatment effect scale) [3] : 0.815 
## 
## Legend:
##   (i): values of treatment arm i
##   [k]: values at stage k

The updated information rates and corresponding boundaries as per the output above are summarized as follows:

## Power calculation for a survival endpoint
## 
## Sequential analysis with a maximum of 3 looks (group sequential design).
## The results were calculated for a two-sample logrank test (one-sided), 
## H0: hazard ratio = 1, power directed towards smaller values, 
## H1: hazard ratio = 0.75, control lambda(2) = 0.012.
## 
## Stage                                         1      2      3 
## Information rate                            53%    75%   100% 
## Efficacy boundary (z-value scale)         2.867  2.366  2.015 
## Overall power                            0.2097 0.5391 0.8001 
## Number of subjects                       1000.0 1000.0 1000.0 
## Expected number of events                 317.0 
## Cumulative number of events                 205    291    387 
## Analysis time                              40.6   52.7   69.1 
## Expected study duration                    57.8 
## Cumulative alpha spent                   0.0021 0.0096 0.0250 
## One-sided local significance level       0.0021 0.0090 0.0220 
## Efficacy boundary (t)                     0.670  0.758  0.815 
## Exit probability for efficacy (under H0) 0.0021 0.0076 
## Exit probability for efficacy (under H1) 0.2097 0.3294 
## 
## Legend:
##   (t): approximate treatment effect scale

4 Boundary and power update at the second interim analysis

Assume that the efficacy boundary was not crossed at the first interim analysis and the trial continued to the second interim analysis which was conducted after 285 rather than the planned 291 events. The updated design is calculated in the same way as for the first interim analysis as per the code below. The idea is to use the cumulative alpha spent from the first stage and an updated cumulative alpha that is spent for the second stage. For the second stage, this can be obtained with the original O’Brien & Fleming alpha spending function:

# Update design using observed information fraction at first and second interim.
designUpdate2 <- getDesignGroupSequential(sided = 1, alpha = 0.025, beta = 0.2,
    informationRates = c(205/387,285/387,1), typeOfDesign = "asOF")

# Recalculate power to get boundary values on effect scale 
# (Use original maxNumberOfEvents and sample size)
powerUpdate2 <- getPowerSurvival(
    design = designUpdate2, 
    lambda2 = log(2)/60,hazardRatio = 0.75,
    dropoutRate1 = 0.025, dropoutRate2 = 0.025, dropoutTime = 12,
    accrualTime = 0,accrualIntensity = 30,
    maxNumberOfSubjects = 1000, maxNumberOfEvents = 387, directionUpper = FALSE)
powerUpdate2
## Design plan parameters and output for survival data:
## 
## Design parameters:
##   Information rates                            : 0.530, 0.736, 1.000 
##   Critical values                              : 2.867, 2.393, 2.011 
##   Futility bounds (non-binding)                : -Inf, -Inf 
##   Cumulative alpha spending                    : 0.002073, 0.009005, 0.025000 
##   Local one-sided significance levels          : 0.002073, 0.008356, 0.022154 
##   Significance level                           : 0.0250 
##   Type II error rate                           : 0.2000 
##   Test                                         : one-sided 
## 
## User defined parameters:
##   Direction upper                              : FALSE 
##   lambda(2)                                    : 0.0116 
##   Hazard ratio                                 : 0.750 
##   Maximum number of subjects                   : 1000 
##   Maximum number of events                     : 387 
##   Accrual intensity                            : 30 
##   Drop-out rate (1)                            : 0.025 
##   Drop-out rate (2)                            : 0.025 
## 
## Default parameters:
##   Type of computation                          : Schoenfeld 
##   Theta H0                                     : 1 
##   Planned allocation ratio                     : 1 
##   kappa                                        : 1 
##   Piecewise survival times                     : 0.00 
##   Drop-out time                                : 12.00 
## 
## Sample size and output:
##   median(1)                                    : 80.0 
##   median(2)                                    : 60.0 
##   lambda(1)                                    : 0.00866 
##   Accrual time                                 : 33.33 
##   Follow up time                               : 35.81 
##   Analysis times [1]                           : 40.60 
##   Analysis times [2]                           : 51.93 
##   Analysis times [3]                           : 69.14 
##   Expected study duration                      : 57.82 
##   Maximal study duration                       : 69.14 
##   Cumulative number of events [1]              : 205.0 
##   Cumulative number of events [2]              : 285.0 
##   Cumulative number of events [3]              : 387.0 
##   Expected number of events                    : 317.2 
##   Number of subjects [1]                       : 1000.0 
##   Number of subjects [2]                       : 1000.0 
##   Number of subjects [3]                       : 1000.0 
##   Expected number of subjects                  : 1000.0 
##   Reject per stage [1]                         : 0.2097 
##   Reject per stage [2]                         : 0.3101 
##   Reject per stage [3]                         : 0.2806 
##   Overall reject                               : 0.8004 
##   Early stop                                   : 0.5198 
##   Critical values (treatment effect scale) [1] : 0.670 
##   Critical values (treatment effect scale) [2] : 0.753 
##   Critical values (treatment effect scale) [3] : 0.815 
## 
## Legend:
##   (i): values of treatment arm i
##   [k]: values at stage k

5 Boundary and power update at the final analysis

Assume that the efficacy boundary was also not crossed at the second interim analysis and the trial continued to the final analysis which was conducted after 393 rather than the planned 387 events. The updated design is calculated as per the code below. The idea here to use the cumulative alpha spent from the first and the second stage stage and the final alpha that is spent for the last stage. An updated correlation has to be used and the original O’Brien & Fleming alpha spending function cannot be used anymore. Instead, the alpha spending function needs to be user defined as follows:

# Update boundary with information fractions as per actually observed event numbers
# !! use user-defined alpha-spending and spend alpha according to actual alpha spent
#    according to the second interim analysis 
designUpdate3 <- getDesignGroupSequential(sided = 1, alpha = 0.025, beta = 0.2,
    informationRates = c(205,285,393)/393, 
    typeOfDesign = "asUser",
    userAlphaSpending = designUpdate2$alphaSpent)

# Recalculate power to get boundary values on effect scale 
# (Use planned sample size and **observed** maxNumberOfEvents)
powerUpdate3 <- getPowerSurvival(
    design = designUpdate3, 
    lambda2 = log(2)/60,hazardRatio = 0.75,
    dropoutRate1 = 0.025, dropoutRate2 = 0.025, dropoutTime = 12,
    accrualTime = 0,accrualIntensity = 30,
    maxNumberOfSubjects = 1000, maxNumberOfEvents = 393, directionUpper = FALSE)
powerUpdate3
## Design plan parameters and output for survival data:
## 
## Design parameters:
##   Information rates                            : 0.522, 0.725, 1.000 
##   Critical values                              : 2.867, 2.393, 2.014 
##   Futility bounds (non-binding)                : -Inf, -Inf 
##   Cumulative alpha spending                    : 0.002073, 0.009005, 0.025000 
##   Local one-sided significance levels          : 0.002073, 0.008356, 0.022021 
##   Significance level                           : 0.0250 
##   Type II error rate                           : 0.2000 
##   Test                                         : one-sided 
## 
## User defined parameters:
##   Direction upper                              : FALSE 
##   lambda(2)                                    : 0.0116 
##   Hazard ratio                                 : 0.750 
##   Maximum number of subjects                   : 1000 
##   Maximum number of events                     : 393 
##   Accrual intensity                            : 30 
##   Drop-out rate (1)                            : 0.025 
##   Drop-out rate (2)                            : 0.025 
## 
## Default parameters:
##   Type of computation                          : Schoenfeld 
##   Theta H0                                     : 1 
##   Planned allocation ratio                     : 1 
##   kappa                                        : 1 
##   Piecewise survival times                     : 0.00 
##   Drop-out time                                : 12.00 
## 
## Sample size and output:
##   median(1)                                    : 80.0 
##   median(2)                                    : 60.0 
##   lambda(1)                                    : 0.00866 
##   Accrual time                                 : 33.33 
##   Follow up time                               : 36.95 
##   Analysis times [1]                           : 40.60 
##   Analysis times [2]                           : 51.93 
##   Analysis times [3]                           : 70.28 
##   Expected study duration                      : 58.37 
##   Maximal study duration                       : 70.28 
##   Cumulative number of events [1]              : 205.0 
##   Cumulative number of events [2]              : 285.0 
##   Cumulative number of events [3]              : 393.0 
##   Expected number of events                    : 320.1 
##   Number of subjects [1]                       : 1000.0 
##   Number of subjects [2]                       : 1000.0 
##   Number of subjects [3]                       : 1000.0 
##   Expected number of subjects                  : 1000.0 
##   Reject per stage [1]                         : 0.2097 
##   Reject per stage [2]                         : 0.3101 
##   Reject per stage [3]                         : 0.2862 
##   Overall reject                               : 0.8060 
##   Early stop                                   : 0.5198 
##   Critical values (treatment effect scale) [1] : 0.670 
##   Critical values (treatment effect scale) [2] : 0.753 
##   Critical values (treatment effect scale) [3] : 0.816 
## 
## Legend:
##   (i): values of treatment arm i
##   [k]: values at stage k

6 Overview of all updates

For easier comparison, all discussed boundary updates and power calculations are summarized more conveniently below. Note that each update only affects boundaries for the current or later analyses, i.e., earlier boundaries are never retrospectively modified.

6.1 Original design

## Sample size calculation for a survival endpoint
## 
## Sequential analysis with a maximum of 3 looks (group sequential design).
## The sample size was calculated for a two-sample logrank test (one-sided), 
## H0: hazard ratio = 1, H1: hazard ratio = 0.75, control lambda(2) = 0.012, power 80%.
## 
## Stage                                         1      2      3 
## Information rate                            50%    75%   100% 
## Efficacy boundary (z-value scale)         2.963  2.359  2.014 
## Overall power                            0.1680 0.5400 0.8000 
## Number of subjects                       1000.0 1000.0 1000.0 
## Cumulative number of events                 194    291    387 
## Analysis time                              39.1   52.7   69.1 
## Expected study duration                    58.0 
## Cumulative alpha spent                   0.0015 0.0096 0.0250 
## One-sided local significance level       0.0015 0.0092 0.0220 
## Efficacy boundary (t)                     0.653  0.758  0.815 
## Exit probability for efficacy (under H0) 0.0015 0.0081 
## Exit probability for efficacy (under H1) 0.1680 0.3720 
## 
## Legend:
##   (t): approximate treatment effect scale

6.2 Updated boundaries and power at the first interim analysis

## Power calculation for a survival endpoint
## 
## Sequential analysis with a maximum of 3 looks (group sequential design).
## The results were calculated for a two-sample logrank test (one-sided), 
## H0: hazard ratio = 1, power directed towards smaller values, 
## H1: hazard ratio = 0.75, control lambda(2) = 0.012.
## 
## Stage                                         1      2      3 
## Information rate                            53%    75%   100% 
## Efficacy boundary (z-value scale)         2.867  2.366  2.015 
## Overall power                            0.2097 0.5391 0.8001 
## Number of subjects                       1000.0 1000.0 1000.0 
## Expected number of events                 317.0 
## Cumulative number of events                 205    291    387 
## Analysis time                              40.6   52.7   69.1 
## Expected study duration                    57.8 
## Cumulative alpha spent                   0.0021 0.0096 0.0250 
## One-sided local significance level       0.0021 0.0090 0.0220 
## Efficacy boundary (t)                     0.670  0.758  0.815 
## Exit probability for efficacy (under H0) 0.0021 0.0076 
## Exit probability for efficacy (under H1) 0.2097 0.3294 
## 
## Legend:
##   (t): approximate treatment effect scale

6.3 Updated boundaries and power at the second interim analysis

## Power calculation for a survival endpoint
## 
## Sequential analysis with a maximum of 3 looks (group sequential design).
## The results were calculated for a two-sample logrank test (one-sided), 
## H0: hazard ratio = 1, power directed towards smaller values, 
## H1: hazard ratio = 0.75, control lambda(2) = 0.012.
## 
## Stage                                         1      2      3 
## Information rate                            53%  73.6%   100% 
## Efficacy boundary (z-value scale)         2.867  2.393  2.011 
## Overall power                            0.2097 0.5198 0.8004 
## Number of subjects                       1000.0 1000.0 1000.0 
## Expected number of events                 317.2 
## Cumulative number of events                 205    285    387 
## Analysis time                              40.6   51.9   69.1 
## Expected study duration                    57.8 
## Cumulative alpha spent                   0.0021 0.0090 0.0250 
## One-sided local significance level       0.0021 0.0084 0.0222 
## Efficacy boundary (t)                     0.670  0.753  0.815 
## Exit probability for efficacy (under H0) 0.0021 0.0069 
## Exit probability for efficacy (under H1) 0.2097 0.3101 
## 
## Legend:
##   (t): approximate treatment effect scale

6.4 Updated boundaries and power at the final analysis

## Power calculation for a survival endpoint
## 
## Sequential analysis with a maximum of 3 looks (group sequential design).
## The results were calculated for a two-sample logrank test (one-sided), 
## H0: hazard ratio = 1, power directed towards smaller values, 
## H1: hazard ratio = 0.75, control lambda(2) = 0.012.
## 
## Stage                                         1      2      3 
## Information rate                          52.2%  72.5%   100% 
## Efficacy boundary (z-value scale)         2.867  2.393  2.014 
## Overall power                            0.2097 0.5198 0.8060 
## Number of subjects                       1000.0 1000.0 1000.0 
## Expected number of events                 320.1 
## Cumulative number of events                 206    285    393 
## Analysis time                              40.6   51.9   70.3 
## Expected study duration                    58.4 
## Cumulative alpha spent                   0.0021 0.0090 0.0250 
## One-sided local significance level       0.0021 0.0084 0.0220 
## Efficacy boundary (t)                     0.670  0.753  0.816 
## Exit probability for efficacy (under H0) 0.0021 0.0069 
## Exit probability for efficacy (under H1) 0.2097 0.3101 
## 
## Legend:
##   (t): approximate treatment effect scale

System: rpact 3.0.3, R version 4.0.3 (2020-10-10), platform: x86_64-w64-mingw32

To cite R in publications use:

R Core Team (2020). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/.

To cite package ‘rpact’ in publications use:

Gernot Wassmer and Friedrich Pahlke (2020). rpact: Confirmatory Adaptive Clinical Trial Design and Analysis. R package version 3.0.3. https://www.rpact.org

 

Creative Commons License
This work by Marcel Wolbers, Gernot Wassmer and Friedrich Pahlke is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.