Journal of Clinical Trials
Open Access

Perspectives on Central Statistical Monitoring in Risk Based Approach of Clinical Trials

Author info »

Study Description

Risk-based monitoring approach and role of central statistical monitoring

The role of monitoring activity in clinical trials is to protect patients participating in clinical trials, to confirm that the operation of the trials is complying with protocols and regulatory requirements, and to ensure the accuracy and completeness of reported data [1]. However, the expense of monitoring activities is increasing with complicating clinical trials and it is heavier burden that frequent visits to clinical sites and 100% Source Data Verification (SDV) has been conducted so far [2-5]. To streamline the activities, Risk-Based Monitoring (RBM) approach has been proposed as process control approach to ensure data quality in clinical trials. RBM is conducted so as to establish an appropriate process of clinical site operation and therefore, it is very much important to monitor the stability of a process by data monitoring. Central Statistical Monitoring (CSM) attempts to maintain a state of statistical control by statistically analyzing the data from multiple sites centrally, detecting atypical sites, and making corrective actions [6-12]. Thus, the methods of CSM check the process stability which is established by RBM approach, and it has important role in RBM. In this article, we focus to clearly articulating the current problems on CSM and the matters to consider for further consecutive study activities of CSM.

Current status of central statistical monitoring studies

CSM methods have been studied for detecting process abnormalities in multi-center clinical trials, the famous examples of existing studies are for detecting fraud detection, detecting digit preference, and duplicated patient detection etc. [9-11]. Those methods are studied in the earlier age of CSM studies and can only deal with a specific problem on operational process abnormality detection. There are, however, many types of problems which should be detected on an operational process and there are many types of outcomes which reflect the status of an operational process.

Recently, different kinds of approach to detect process abnormalities of an operational process have been proposed, the methods compare the outcomes between clinical sites and detect the sites whose data trend is different from others as atypical clinical sites with potential abnormal operational process to implement efficient on-site monitoring and inspection. Those methods are reasonable from the process control viewpoint. As shown in Venet et al. [7] as a principle of CSM, reported data are collected based on a common protocol and a case report form for clinical trials. Therefore, it is reasonable to consider worthy of further investigation if an atypical trend in the data is observed since the difference of the data trend in the data may be indicating the difference of the operational process. Earlier examples of that CSM method are based on simply summary statistics [7-11]. However, recently advanced statistical methods which detect atypical sites by comparing outcomes from clinical sites are developed. Desmet et al. [13] has proposed an analysis method for continuous data using linear mixed effects models and Desmet et al. [14] proposed the method analyzing the incidence of events using Beta-Binomial Models. In addition, Hatayama and Yasui [15] have proposed the method using Finite mixture models [16-18] to take account the data contamination from the atypical clinical sites as a robust method against the proportion of atypical clinical sites, since the CSM methods which detect atypical site by comparing outcomes from each site are affected by data contamination from atypical sites.

Problems of central statistical monitoring studies which should be considered

CSM methods have been studied in the decades, and the statistical methods which detect atypical clinical sites with potential operational process abnormalities have been developed recently. Those studies are, however, still imperfect to fit them to practical clinical trials. Most of those methods are based on the assumptions that the number of the clinical sites is large (e.g. 100 or more). CSM, however, has to be conducted in smaller sized clinical trials as well and only smaller number of clinical sites is participating at the beginning of the trials in any clinical trial. Small sample size problems are to be considered also in the CSM study. Current CSM studies are ignoring that site specific difference which can be considered as covariates, however in the practical clinical trials, prognostic factors can be unbalanced between clinical sites since many of clinical trials are conducted with across countries, at least races and location of the site and its medical environment are unbalanced naturally.

In addition, because CSM studies are mainly focusing on statistical part of the monitoring and for ease of explanation, those methods are developed and explained based on single variables, however, in the clinical trials, there are multiple variables which reflect the state of operational process. As a toolbox, it is used that the abnormal detection based on the probability ellipsoid of the bivariate normal distribution on scatter plots and the chi-square statistic of the Mahalanobis distance (the Hotelling’s T2 statistic). Though these multivariate techniques are useful to find abnormal sites, it is difficult to effectively and efficiently take corrective actions by only those techniques since those ones never inform us of what is the cause of abnormal. It is, thus, important to understand that which variable links to which type of operational process, and how to analyze those data in actual, e.g., using fault variable identification methods [19-22]. It seems more appropriate that CSM that analyzes multiple variables simultaneously especially in large sized clinical trials with understanding the relationships of variables and processes which reflected at.

Moreover, there is a crucial problem on current CSM studies. Most of CSM studies evaluate the performances through simulation studies, however, the methods are evaluated at the setting of that CSM is conducted at once in the clinical trials. Performance evaluation in those studies seems problematic since CSM is conducted multiply with the progress of the trials to monitor the stability of an established operational process. If a specific CSM method is conducted in actual clinical trials, it should be selected by performance evaluation which is reflected the real setting of the clinical trial as indicator. As note, Hatayama et al. [15] evaluated their proposed method in the setting of that CSM is conducted multiple times in the clinical trials.

Approach to the problem

There are many issues to be considered in the CSM study as described in the previous section. Authors think that the problem of multiplicity of CSM is important and should be considered promptly since the CSM is conducted multiply in the actual trials to monitor the stability of an established operational process, the method tuned based on the actual usage situation and if the method is not correctly tuned nor proposed it leads to problematic operation of monitoring activities and poor data quality of clinical trials. The problem which will appear when CSM is conducted multiple times in the trials are inflation of the type I error rate and it can be replaced in actual clinical trials as that increasing frequency of meaningless on-site visit which prioritize by CSM. This leads to return to inefficient monitoring activities as it was conducted so far. It does not, however, simply mean that multiplicity of CSM should be adjusted so that target nominal type I error is met. If CSM is tuned so as to maintain nominal type I error rate simply, the method may not detect process abnormalities with high probability, it directly causes the frequent overlooking of process abnormalities in clinical trials. In addition, the multiplicity of comparison comes from not only the frequency of CSM but also the number of clinical sites if we compare the outcomes between each site to detect process abnormalities. Too much conservative detection procedure can be derived if we adjust the multiplicity of comparison stringently, and stringent adjustment of multiple comparisons is not realistic solution to pursue efficient monitoring activities. However, the CSM with the large type I error rate induce frequently unnecessary visits to clinical sites, and it would not be efficient from RBM views.

Authors think that CSM should be investigated its operational characteristics including the type I error rate and the detection performance according to the actual situation of implementation and should be tuned its procedures and settings so that CSM have desirable performance in the real situation of which CSM is to be implemented. And to tune up the methods, it would be useful if it is possible to set up the utility functions which can reflect various types of cost such as not only monitoring cost but the cost which is to be induced by overlooking of process abnormalities. Historically in the statistical quality control of manufacturing process, similar problems with those problems have been studied so far. In the statistical quality control contexts, there are rich studies to challenge the problems of detecting process abnormalities correctly with a desirable performance so that balance cost and detection performance [23,24]. Those rich studies should be reevaluated also for CSM.

Conclusion

In this article, we discussed the current status of CSM studies and the gap which lays out between current CSM studies and practical needs. In addition, we discussed future directions of CSM studies to accelerate practical studies in CSM. To fill the gap of CSM studies and practical needs in clinical trials, though it is basic things, methods are to be developed based on practical assumptions which reflect practical use. In addition, authors think that the basic concept of statistical quality control in manufacturing and CSM in RBM is similar in nature, but their points of similarities are not discussed sufficiently so far. It may be useful to reevaluate the methodologies which are developed in similar different study area, to seek the means to address the technically complicated problems which are to be encountered in developing CSM methods that are actually based on practical assumptions and needs in clinical trials.

References

1. International Conference of Harmonisation, E6 (R2): Guideline for good clinical practice. 2016.
2. Eisenstein E, Collins R, Cracknell BS, Podesta O, Reid ED, Sandercock P, et al. Sensible approaches for reducing clinical trial costs. Clin Trials. 5(1):75
3. Olsen R, Bihlet AR, Kalakou F, Andersen JR. The impact of clinical trial monitoring approaches on data integrity and cost: A review of current literature. Eur J Clin Pharmacol. 2005;72(4):399
4. Eisenstein EL, Lemons PW, Tardiff BE, Schulman KA, Jolly MK, Califf RM. Reducing the costs of phase III cardiovascular clinical trials. Am Heart J. 2005;149:482
5. European medicines agency. Reflection paper on risk-based quality management in clinical trials. 2020.
6. US food and drug administration. Guidance for industry: Oversight of clinical investigations risk-based approach to monitoring. 2013.
7. Venet D, Doffagne E, Burzykowski T, Beckers F, Tellier Y, Genevois-Marlin E, et al. A statistical approach to central monitoring of data quality in clinical trials. Clin Trials. 2012;9:705
8. Kirkwood AA, Cox T, Hackshaw A. Application of methods for central statistical monitoring in clinical trials. Clin Trials. 2013;10:783
9. Oba K. Statistical challenges for central monitoring in clinical trials: A review. Int J Clin Oncol. 2016;21:28
10. Pogue JM, Devereaux PJ, Thorlund K, Yusuf S. Central statistical monitoring: Detecting fraud in clinical trials. Clin Trials. 2013;10:225
11. Buyse M, George SL, Evans S, Geller NL, Ranstam J, Scherrer B, et al. The role of biostatistics in the prevention, detection and treatment of fraud in clinical trials. Stat Med. 1999;18:3435
12. Zink RC, Dmitrienko A, Dmitrienko A. Rethinking the clinically based thresholds of Trans Celerate BioPharma for risk-based monitoring. Ther Innov Regul Sci. 2017;52;560
13. Desmet L, Venet D, Doffagne E, Timmermans C, Burzykowski T, Legrand C, et al. Linear mixed-effects models for central statistical monitoring of multicenter clinical trials. Stat Med. 2014;33:5265
14. Desmet V, Venet D, Doffagne E, Timmermans C, Legrand C, Burzykowski T, et al. Use of the beta-binomial model for central statistical monitoring of multicenter clinical trials. Stat Biopharm Res. 2017;9:1
15. Hatayama T, Yasui S. Bayesian central statistical monitoring using finite mixture models in multicenter clinical trials. Contemp Clin Trials Commun. 2020;19:100566.
16. Gelman A, Carlin JB, Stern HS, Dunson DB, Vehtari A, Rubin DB, et al. Bayesian data analysis. CRC press. 2013.
17. Richardson S, Green PJ. On bayesian analysis of mixtures with an unknown number of components. J Roy Stat Soc. 1997;59:731
18. Mason RL, Tracy ND, Young JC. Decomposition of T2 for multivariate control chart interpretation. J Qual Technol. 1995;27:99
19. Turkoz M, Kim S, Jeong Y, Jeong MK, Elsayed EA, Al-Khalifa KN, et al. Bayesian framework for fault variable identification. J Qual Technol. 2019;51:375
20. Zink RC, Castro-Schilo L, Ding J. Understanding the influence of individual variables contributing to multivariate outliers in assessments of data quality. Pharmaceut Stat. 2018;17:846
21. Zou C, Qiu P. Multivariate statistical process control using LASSO. J Am Stat Assoc. 2009;104:1586
22. Ho C, Case KE. Economic design of control charts: A literature review for 1981
23. Pasha MA, Moghadam MB, Fani SH. A generalized approach to economic design of control charts: A discussion on implementing. Commun Stat Theory Methods. 2017;46:10495-10506.