Article Text

Download PDFPDF

Detailed statistical assessment of the characteristics of the ESMO Magnitude of Clinical Benefit Scale (ESMO-MCBS) threshold rules
  1. Urania Dafni1,2,
  2. Dimitris Karlis3,
  3. Xanthi Pedeli2,
  4. Jan Bogaerts4,
  5. George Pentheroudakis5,
  6. Josep Tabernero6,
  7. Christoph C Zielinski7,
  8. Martine J Piccart8,
  9. Elisabeth G E de Vries9,
  10. Nicola Jane Latino10,
  11. Jean-Yves Douillard10,
  12. Nathan I Cherny11
  1. 1Laboratory of Biostatistics, School of Health Sciences, National and Kapodistrian, University of Athens, Athens, Greece
  2. 2Frontier Science Foundation-Hellas, Athens, Greece
  3. 3Department of Statistics, Athens University of Economics and Business, Athens, Greece
  4. 4Methodology Direction, EORTC Headquarters, Brussels, Belgium
  5. 5Department of Medical Oncology, Ioannina University Hospital, Ioannina, Greece
  6. 6Department of Medical Oncology, Vall d’Hebron University Hospital, Barcelona, Spain
  7. 7Division of Oncology, Medical University Vienna, Vienna, Austria
  8. 8Jules Bordet Institute, Universite Libre de Bruxelles, Bruxelles, Belgium
  9. 9Department of Medical Oncology, University Medical Center Groningen, University of Groningen, Groningen, The Netherlands
  10. 10ESMO Head Office, European Society for Medical Oncology, Lugano, Switzerland
  11. 11Department of Medical Oncology, Cancer Pain and Palliative Medicine Service, Shaare Zedek Medical Center, Jerusalem, Israel
  1. Correspondence to Prof Urania Dafni; udafni{at}nurs.uoa.gr

Abstract

Background The European Society for Medical Oncology (ESMO) has developed the ESMO Magnitude of Clinical Benefit Scale (ESMO-MCBS), a tool to assess the magnitude of clinical benefit from new cancer therapies. Grading is guided by a dual rule comparing the relative benefit (RB) and the absolute benefit (AB) achieved by the therapy to prespecified threshold values. The ESMO-MCBS v1.0 dual rule evaluates the RB of an experimental treatment based on the lower limit of the 95%CI (LL95%CI) for the hazard ratio (HR) along with an AB threshold. This dual rule addresses two goals: inclusiveness: not unfairly penalising experimental treatments from trials designed with adequate power targeting clinically meaningful relative benefit; and discernment: penalising trials designed to detect a small inconsequential benefit.

Methods Based on 50 000 simulations of plausible trial scenarios, the sensitivity and specificity of the LL95%CI rule and the ESMO-MCBS dual rule, the robustness of their characteristics for reasonable power and range of targeted and true HRs, are examined. The per cent acceptance of maximal preliminary grade is compared with other dual rules based on point estimate (PE) thresholds for RB.

Results For particularly small or particularly large studies, the observed benefit needs to be relatively big for the ESMO-MCBS dual rule to be satisfied and the maximal grade awarded. Compared with approaches that evaluate RB using the PE thresholds, simulations demonstrate that the MCBS approach better exhibits the desired behaviour achieving the goals of both inclusiveness and discernment.

Conclusions RB assessment using the LL95%CI for HR rather than a PE threshold has two advantages: it diminishes the probability of excluding big benefit positive studies from achieving due credit and, when combined with the AB assessment, it increases the probability of downgrading a trial with a statistically significant but clinically insignificant observed benefit.

  • ESMO-MCBS
  • Clinical Benefit
  • Threshold Rule
  • Evaluation of Clinical Benefit

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

View Full Text

Statistics from Altmetric.com

Footnotes

  • Contributors NA.

  • Funding This project was funded by ESMO.

  • Competing interests The authors have declared the following: JB: director of EORTC. EORTC conducts many studies sponsored by, or otherwise supported by, a large number of companies. EORTC is an independent research organisation. MJP: board member: Radius. Consultant (honoraria): AstraZeneca, Lilly, MSD, Novartis, Pfizer, Roche-Genentech, Crescendo Biologics, Periphagen, Huya, Debiopharm and PharmaMar. Research grants to institute: AstraZeneca, Lilly, MDS, Novartis, Pfizer, Roche-Genentech, Synthon, Radius and Servier. Speakers bureau/stock ownership: none. GP: consulting and advisory services, and research support: Amgen, Merck, AstraZeneca, Roche, BMS, MSD and Lilly. J-YD: compensated participation to advisory boards, lecture and symposia: Amgen, Merck Serono, Bayer, Roche/Genentech, Sanofi, AstraZeneca, Boehringer-Ingelheim and Sirtex until April 2016. No further compensated participation in industry events from May 2016 onwards. JT: advisory boards for Amgen, Bayer, Boehringer Ingelheim, Celgene, Chugai, Genentech, Lilly, MSD, Merck Serono, Novartis, Pfizer, Roche, Sanofi, Symphogen, Taiho and Takeda. CCZ: honoraria: AstraZeneca, Celgene, Roche, Novartis, Bristol Myers Squibb, MSD, Ariad and Newgen. EGEdV: currently conducting research sponsored by the following companies: Amgen, Roche/Genentech, Chugai Pharma, Synthon, AstraZeneca, Radius Health, CytomX Therapeutics and Nordic Nanovector (all payments to the institution). Consulting or advisory role for the following companies: Synthon, Medication and Merck (all payments to the institution). Not a member of any speakers' bureau.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.