Article Text

Download PDFPDF

EHA evaluation of the ESMO—Magnitude of Clinical Benefit Scale version 1.1 (ESMO-MCBS v1.1) for haematological malignancies
  1. Barbara Kiesewetter1,
  2. Nathan I Cherny2,
  3. Nicolas Boissel3,4,
  4. Francesco Cerisoli5,
  5. Urania Dafni6,7,
  6. Elisabeth G E de Vries8,
  7. Paolo Ghia9,10,
  8. Nicola Gökbuget11,
  9. Verónica González-Calle12,
  10. Brian Huntly13,
  11. Ulrich Jäger14,
  12. Nicola Jane Latino15,
  13. Jean-Yves Douillard15,
  14. Luca Malcovati16,17,
  15. María-Victoria Mateos12,
  16. Gert J Ossenkoppele18,
  17. Kimmo Porkka19,
  18. Markus Raderer1,
  19. Josep-Maria Ribera20,
  20. Lydia Scarfò9,10,
  21. Ruth Wester21,
  22. Panagiota Zygoura7,
  23. Pieter Sonneveld21,22
  1. 1 Department of Medicine I, Clinical Division of Oncology, Medical University of Vienna, Vienna, Austria
  2. 2 Cancer Pain and Palliative Medicine Service, Department of Medical Oncology, Shaare Zedek Medical Center, Jerusalem, Israel
  3. 3 Department of Hematology, Hospital Saint-Louis, Paris, Île-de-France, France
  4. 4 Adolescent and Young Adult Hematology Unit, Diderot University Paris Faculty of Medicine, Paris, Île-de-France, France
  5. 5 European Hematology Association, Den Haag, Zuid-Holland, The Netherlands
  6. 6 Laboratory of Biostatistics, School of Health Sciences, National and Kapodistrian University of Athens, Athens, Greece
  7. 7 Frontier Science Foundation-Hellas, Frontier Science Foundation-Hellas, Athens, Greece
  8. 8 Department of Medical Oncology, University Medical Center Groningen, Groningen, The Netherlands
  9. 9 Strategic Research Program on CLL, Division of Experimental Oncology, IRCCS Ospedale San Raffaele, Milano, Lombardia, Italy
  10. 10 Universita Vita e Salute San Raffaele, Milano, Lombardia, Italy
  11. 11 Department of Hematology/Oncology, Goethe University, Frankfurt am Main, Hessen, Germany
  12. 12 Department of Hematology and Instituto de Investigación Biomédica de Salamanca-IBSAL, University Hospital of Salamanca, Salamanca, Castilla y León, Spain
  13. 13 Cambridge Stem Cell Institute, Department of Haematology, University of Cambridge, Cambridge, Cambridgeshire, UK
  14. 14 Department of Medicine I, Clinical Division of Hematology and Hemostaseology, Medical University of Vienna, Vienna, Austria
  15. 15 Head Office, European Society for Medical Oncology, Lugano, Switzerland
  16. 16 Department of Molecular Medicine, University of Pavia, Pavia, Lombardia, Italy
  17. 17 Department of Hematology Oncology, Fondazione IRCCS Policlinico San Matteo, Pavia, Lombardia, Italy
  18. 18 Department of Hematology, VU University Medical Centre Amsterdam, Amsterdam, Noord-Holland, The Netherlands
  19. 19 Department of Hematology, Helsinki University Hospital Comprehensive Cancer Center, Helsinki, Finland
  20. 20 Clinical Hematology Department, ICO-Hospital Germans Trias i Pujol, Josep Carreras Research Institute, Universitat Autònoma de Barcelona, Barcelona, Catalunya, Spain
  21. 21 Department of Hematology, Erasmus MC Cancer Institute, Rotterdam, Zuid-Holland, The Netherlands
  22. 22 Department of Radiation Oncology, Erasmus MC Cancer Institute, Rotterdam, Zuid-Holland, The Netherlands
  1. Correspondence to Dr Barbara Kiesewetter; barbara.kiesewetter{at}meduniwien.ac.at

Abstract

Objective Value frameworks in oncology have not been validated for the assessment of treatments in haematological malignancies, but to avoid overlaps and duplications it appears reasonable to build up experience on existing value frameworks, such as the European Society for Medical Oncology—Magnitude of Clinical Benefit Scale (ESMO-MCBS).

Methods Here we present the results of the first feasibility testing of the ESMO-MCBS v1.1 for haematological malignancies based on the grading of 80 contemporary studies for acute leukaemia, chronic leukaemia, lymphoma, myeloma and myelodysplastic syndromes. The aims were (1) to evaluate the scorability of data, (2) to evaluate the reasonableness of the generated grades for clinical benefit using the current version and (3) to identify shortcomings in the ESMO-MCBS v1.1 that require amendments to improve the efficacy and validity of the scale in grading new treatments in the management of haematological malignancies.

Results In general, the ESMO-MCBS v1.1 was found to be widely applicable to studies in haematological malignancies, generating scores that were judged as reasonable by European Hematology Association (EHA) experts. A small number of studies could either not be graded or were not appropriately graded. The reasons, related to the differences between haematological and solid tumour malignancies, are identified and described.

Conclusions Based on the findings of this study, ESMO and EHA are committed to develop a version of the ESMO-MCBS that is validated for haematological malignancies. This development process will incorporate all of the usual stringencies for accountability of reasonableness that have characterised the development of the ESMO-MCBS including field testing, statistical modelling, evaluation for reasonableness and openness to appeal and revision. Applying such a scale will support future public policy decision-making regarding the value of new treatments for haematological malignancies and will provide insights that could be helpful in the design of future clinical trials.

  • ESMO-MCBS
  • value frameworks
  • clinical benefit
  • haematologic malignancies
http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, any changes made are indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

View Full Text

Statistics from Altmetric.com

Key questions

What is already known about this subject?

  • The European Society for Medical Oncology—Magnitude of Clinical Benefit Scale (ESMO-MCBS) v1.1 is a validated value scale for solid tumour oncology, but it has not yet been evaluated for the use in haematological malignancies.

What does this study add?

  • Here, we present the results of the first feasibility testing of the ESMO-MCBS v1.1 for haematological malignancies based on grading of 80 contemporary studies for leukaemia, lymphoma, myeloma and myelodysplastic syndromes.

  • The ESMO-MCBS v1.1 was found to be widely applicable to studies in haematological malignancies, generating scores that were judged as reasonable by European Hematology Association (EHA) experts; however, a small number of studies could either not be graded or were not appropriately graded because of shortcomings related to the differences between haematological and solid tumour malignancies.

How might this impact on clinical practice?

  • Based on the findings of this study, ESMO and EHA are committed to develop a version of the score that is robustly validated to grade studies in malignant haematology.

Introduction

In recent years, rapid developments in haematology research resulted in a considerable expansion of treatment options. The development of instruments to measure clinical benefit is essential in the current scenario where increasing numbers of treatments for haematological malignancies (HMs) are becoming available, often targeting a small and defined subpopulation of patients. For this, several value frameworks have been published by different organisations and institutions taking into account or emphasising different aspects contributing to such an evaluation.1 These frameworks vary in terms of their definition of value, target audience and methodology, and each of them has specific limitations, which should be taken into consideration when interpreting their outputs.2 Until now, value frameworks developed in oncology have not been validated in the setting of HMs.

The European Society for Medical Oncology (ESMO) has developed such a value framework called the ESMO—Magnitude of Clinical Benefit Scale (ESMO-MCBS).3 Initially published in 2015, the scale is a validated and reproducible tool in solid tumour oncology with a particular focus on the clinical benefit. The ESMO-MCBS was developed to generate clear, valid and unbiased grading of the magnitude of clinical benefit demonstrated in therapeutic studies that could be used for a number of purposes including public health policy and health technology assessment (HTA), clinical decision-making, medical publication and journalism. The ESMO-MCBS grading highlights those treatments which substantially improve the duration of survival and/or the quality of life (QOL) of patients with cancer and aims to distinguish them from trials demonstrating more limited and sometimes even marginal benefits. The ESMO-MCBS was revised (version 1.1) in 2017, based on feedback and queries from clinicians, patients, researchers and representatives of the pharmaceutical industry, and a dynamic process of internal peer review.4 Version 1.1 incorporates 10 revisions and most importantly allows also for scoring of single-arm studies. The ESMO-MCBS assigns categorical benefit scores to European Medicines Agency (EMA) approved drugs, based on results from ‘positive’ randomised clinical trials: (1) superiority trials that have demonstrated a statistically significant result for the primary endpoint of the study, or secondary in case of overall survival (OS) and (2) non-inferiority trials, reaching a conclusion of non-inferiority. Primary or secondary endpoints included in the scoring system are OS, progression-free survival (PFS), QOL, treatment toxicity or response rates. In developing the ESMO-MCBS scale, ESMO aspired to meet standards for ‘accountability for reasonableness’,5 6 incorporating extensive field testing, statistical modelling7 and peer review of the ‘reasonableness’ of the generated results into the development process. The ESMO-MCBS is currently incorporated in ESMO’s clinical practice guidelines and is being used as part of HTA processes.8 9

The European Hematology Association (EHA) and ESMO have developed a joint initiative to develop a version of the ESMO-MCBS that is validated for HMs. As a first step in this process, we have field tested the current version of the ESMO-MCBS (version 1.1) across a wide spectrum of HMs. The aims of this evaluation were (1) to evaluate the scorability of data derived from contemporary clinical trials in HMs, (2) to evaluate the reasonableness of the generated grades for clinical benefit using the current version and (3) to identify shortcomings in the ESMO-MCBS v1.1 that require amendments to improve the efficacy and validity of the scale in grading new treatments in the management of HMs.

Methods

Study selection

The corresponding disease-oriented EHA scientific working groups identified experts who selected representative treatments currently used in clinical practice with a focus on recently approved drugs and novel strategies, to be evaluated for each of the common haematological malignancies: acute lymphoblastic leukaemia (ALL), acute myeloid leukaemia (AML), chronic lymphocytic leukaemia (CLL), chronic myeloid leukaemia (CML), Hodgkin and non-Hodgkin lymphomas, multiple myeloma (MM) and myelodysplastic syndromes (MDS). The treatments selected underwent a literature search to identify corresponding clinical trials and data.

ESMO-MCBS grading

Identified studies were graded by members of the EHA scientific working groups according to the ESMO-MCBS v1.1 forms4 in accordance with the instructions provided by ESMO. Magnitude of clinical benefit scores range from A to C for treatment strategies with curative intent and 5-1 for treatments with non-curative intent, with scores of A–B and 5-4 relating to a substantial level of clinical benefit. Initial grading by the expert groups were reviewed by the ESMO-MCBS working group for applicability and correctness.

Evaluations

For each disease entity, we evaluated the scorability of the evaluated studies and the reasonableness of the derived scores. Based on these findings, we identified shortcomings in the current version of the ESMO-MCBS that either precluded scoring or which generated grading which was considered not to be a reasonable estimation of benefit when such studies were identified.

Results

The extensive research concluded in 80 studies, 5 of which had either more than two arms or different publications for the same trial presenting results after longer follow-up times (87 studies and/or comparisons in total). In detail, we have scored 7 studies for AML, 5 studies for ALL, 8 studies for CLL, 4 studies for CML, 23 studies for non-Hodgkin and Hodgkin lymphoma, 23 studies for MM and 10 studies for MDS. The ESMO-MCBS v1.1 tool was applied in all the 87 distinct studies and/or subgroups.

Acute myeloid leukaemia

Studies evaluated: Seven studies were evaluated,10–16 three in a curative setting and four in a non-curative setting (table 1).

Table 1

Feasibility testing of the ESMO-MCBS v1.1 for acute myeloid leukaemia (n=7)

Scorability: All studies were published with endpoints and data applicable to the ESMO-MCBS v1.1.

Reasonableness: The separation of studies with curative/non-curative intent corresponds closely to the distinction between intensive versus non-intensive chemotherapy regimens which are the terms usually applied in the treatment of AML. Grading effectively distinguished between high benefit treatment strategies in a curative setting and stratified between higher and lower benefit treatments in a non-curative setting.

Shortcomings: None identified.

Acute lymphoblastic leukaemia

Studies evaluated: Five studies were evaluated,17–23 and these included studies relating to three agents recently approved by EMA for relapsed and refractory ALL (table 2).17–20 22

Table 2

Feasibility testing of the ESMO-MCBS v1.1 for acute lymphoblastic leukaemia (n=5)

Scorability: Four of the five studies were published with endpoints and data applicable to the ESMO-MCBS v1.1. The only not scoreable study was the single-arm study of ponatinib as add-on to standard of care upfront treatment with curative intent.21

Reasonableness: Both the first-in class bispecific antibody blinatumomab (TOWER trial)17 18 and the antibody-drug conjugate inotuzumab ozogamicin (INO-VATE trial)19 20 reached high scores based on positive OS data and favourable QOL data for blinatumomab (ESMO-MCBS v1.1 scores 5 and 4, respectively). The chimeric antigen receptor (CAR) T-cell treatment in children/young adults with relapsed or refractory B-cell ALL was graded with maximal credit of 3 for a single-arm study in a non-curative setting.22 The ponatinib treatment (single-arm PACE trial)23 was assigned grade 2 based on the major molecular response (MMR) in the non-curative setting.

Reasonableness: Grading effectively distinguished between high benefit treatment strategies in a curative stetting and stratified between higher and lower benefit treatments in a non-curative setting.

Shortcomings: One shortcoming was identified:

  1. The ESMO-MCBS v1.1 does not have a form to grade single-arm treatments with curative intent. This shortcoming precluded scoring in one study21 and may also have been relevant to the grading of CAR T-cell salvage therapy which could also be considered as curative.22

Chronic lymphocytic leukaemia

Studies evaluated: Eight studies were evaluated (table 3).24–35

Table 3

Feasibility testing of the ESMO-MCBS v1.1 for chronic lymphocytic leukaemia (n=8)

Scorability: CLL is generally a relatively indolent disease with a very long survival—often decades long—and many patients do not need intervention for many years and when treatment is initiated it commonly generates very long periods of remission. For these reasons, PFS is generally the most relevant and measurable primary endpoint. Since CLL is generally not considered to be a curable disease, all scoring was performed using scales for non-curative disease. One study27 could not be scored because the primary objective of non-inferiority with regard to PFS was not met. Moreover, the published results limited to a subcohort of patients older than 65 years, which are relevant for clinical practice (particularly in view of presented toxicity data) did not show non-inferiority and they were derived from a post hoc exploratory analysis.

Reasonableness: Overall scoring was considered reasonable with the highest grades being achieved by studies demonstrating either mature OS data24–26 or PFS gains with long-term plateauing of PFS,33 or compelling PFS gains.28 29 Grading of the phase III study of ibrutinib versus ofatumumab (RESONATE trial)31 32 was considered to be low; it was credited for PFS advantage including gain in the tail of the curve but was penalised for toxicity associated with the more prolonged drug exposure in continuous treatment (ESMO-MCBS v1.1 score 3). However, the 9% improvement in OS at 12 months was not credited as these results are deemed immature by the ESMO-MCBS criteria. The benefit of novel agents in populations with high unmet need, like relapsed and refractory patients with CLL carrying deletion in chromosome 17 p, was graded reasonably using form 3 for single-arm studies in a non-curative setting.34 35

Shortcomings: One shortcoming was identified:

  1. The EHA scientific working group members felt that compelling immature survival benefit ought to be credited even when the median survival of the control arm has not been reached.

Chronic myeloid leukaemia

Studies evaluated: Four landmark trials addressing the use of tyrosine kinase inhibitors imatinib, nilotinib, dasatinib and bosutinib upfront for chronic phase CML were graded.36–43 Only one of these had mature OS data (table 4).38

Table 4

Feasibility testing of the ESMO-MCBS v1.1 for chronic myeloid leukaemia (n=4)

Scorability: CML is generally considered an incurable disease, but in a small proportion of cases with deep molecular responses the disease may be eradicated. Thus, when mature survival data were available, CML was scored for both curative and non-curative intent.36–38 Contemporary studies in CML treatments are conventionally evaluated using molecular response evaluations.44 45 This differs from the concepts of ‘pathological complete response’ or ‘response rate’ which are terms used in the ESMO-MCBS v1.1. Scoring of these studies was only possible by interpreting deep molecular responses (MMR 4–5) as pathological complete responses (form 1) or major responses (form 2 c).39–43 In one study,36–38 PFS/event-free survival (EFS) gains could not be credited because the PFS of the control arm was very long and had not reached median PFS after 11 years of follow-up.

Reasonableness: In the IRIS study of imatinib versus former standard interferon plus cytarabine, initial scoring at 18 months was credited on the basis of complete cytogenic response for curative intent with a grade of C and improvement in molecular response rate with grade 2.36–38 At 10-year follow-up, the imatinib scores B for curative intent based on survival improvement. While the grades for curative intent were considered reasonable, the EHA working group considered the ESMO-MCBS grade of 2 for non-curable intent to be too low for the benefits observed.

The remaining studies of nilotinib, dasatinib and bosutinib show minor improvements in complete molecular response rates when compared with imatinib (grade 2) in a non-curative setting.39–43 None of these agents had mature data beyond 5 years and consequently they were not graded for curative intent.

Shortcomings: These relatively low scores for imatinib in the non-curative grading appear to indicate two shortcomings in the ESMO-MCBS v1.1:

  1. When PFS (or EFS) is very long, there is no mechanism to credit strong interim gains when the median PFS of the control arm has not yet been reached.

  2. The surrogacy of complete cytogenic response and level 4–5 MMR, defined as 4 to 5-log reduction in BCR–ABL1 transcript levels from a standardised baseline, are much stronger surrogates for survival than pathological complete response and response rate in solid tumours.44 45 Consequently, form 2 c needs to be amended to incorporate evaluation of deep molecular responses.

Indolent non-Hodgkin’s, relapsed/refractory setting of non-diffuse large B-cell lymphoma (non-DLBCL) and Hodgkin’s lymphoma

Studies evaluated: Twelve studies of recently approved drugs for indolent non-Hodgkin’s, relapsed/refractory setting of non-DLBCL and Hodgkin’s lymphoma were evaluated (table 5).46–62

Table 5

Feasibility testing of the ESMO-MCBS v1.1 for indolent non-Hodgkin and relapsed/refractory setting of non-DLBCL and Hodgkin’s lymphoma (n=12)

Scorability: In one of the studies,46 PFS/EFS gains could not be graded because the PFS of the control arm was very long, the median PFS was not reached and only interim gains were reported. The BRIGHT study could not be scored because form 2 c makes no provision for scoring of non-inferiority studies based on response rates.49 50 The remaining 10 studies were published with endpoints and data applicable to the ESMO-MCBS v1.1 and were all evaluable.

Reasonableness: The grading was applicable and was judged by the EHA working group to be reasonable in the evaluated trials, endorsing relatively high benefit grades, that is, ESMO-MCBS v1.1. scores of 4–5 for 7 of the 10 evaluable studies.

Shortcomings: Two shortcomings were observed:

  1. The ESMO-MCBS v1.1 has no mechanism for scoring non-inferiority studies based on response rate.

  2. When PFS (or EFS) is very long, there is no mechanism to credit strong interim gains when the median PFS of the control arm has not yet been reached.

Diffuse large B-cell lymphoma

Studies evaluated: Eleven studies were evaluated63–75; two in the first-line setting with curative intent,63–66 two intensified therapies for first-line and salvage setting, respectively, with both curative and non-curative intent,67 68 two single-arm studies of CAR T-cell salvage therapy70 71 and five in a non-curative setting for relapsed and refractory disease (table 6).69 72–75

Table 6

Feasibility testing of the ESMO-MCBS v1.1 for DLBCL (n=11)

Scorability: All studies incorporated required data for evaluation using the ESMO-MCBS v1.1. Single-arm studies of CAR T-cell therapy for refractory or resistant disease70 71 could not be evaluated for curative intent. The NCIC-CTG LY12 trial could not be graded in the non-curative setting because non inferiority was evaluated on the basis of overall response rate.68

Reasonableness: The grading was applicable and was judged by the EHA working group to be reasonable in the evaluated trials, endorsing high benefit grades for first-line therapies with curative intent.63–67 Lower benefit scores for trials in the relapsed and refractory therapies were considered reasonable.

Shortcomings: One shortcoming was identified:

  1. The ESMO-MCBS v1.1 does not have a form to grade single-arm treatments with curative intent and this shortcoming does not allow for the representation of the full potential benefit of CAR T-cell salvage therapy.70 71

Multiple myeloma

Studies evaluated: Table 7 describes results from eight studies in the first-line setting.76–84 Of these, three were conducted for autologous stem cell transplantation (ASCT) eligible76–78 patients and five are for ASCT ineligible patients.79–84 Table 8 describes the results of a further 15 studies with relapsed or refractory myeloma.85–104

Table 7

Feasibility testing of the ESMO-MCBS v1.1 for first-line multiple myeloma (n=8)

Table 8

Feasibility testing of the ESMO-MCBS v1.1 for relapsed/refractory multiple myeloma (n=15)

Scorability: Most studies incorporated required data for evaluation using the ESMO-MCBS v1.1. The PETHEMA/GEM study comparing VTD (bortezomib, thalidomide and dexamethasone) to TD (thalidomide and dexamethasone) or VBMCP/VBAD/B (vincristine, BCNU, melphalan, cyclophosphamide, prednisone/vincristine, BCNU, doxorubicin, dexamethasone/bortezomib) as induction therapies did not report HRs for the PFS, resulting in precluded scoring with non-curative intent using form 2b.76 The GIMEMA 2005 study could not be scored for non-curative intent because the median PFS of the control arm had not yet been reached.77 The MM5 non- inferiority study78 could not be scored for non-curative intent because non-inferiority was based on response rate.

Reasonableness: First-line treatments for patients who are ASCT eligible are graded both for curative and non-curative intent. The relatively low grades of C for curative intent achieved in two of the ASCT eligible studies76 77 reflect the prevailing consensus that MM is rarely cured. In most studies evaluated, the scale was feasible and the results were consistent with clinical practice.

Shortcomings: Three previously described shortcomings influenced scoring for a small number of these studies.

  1. The ESMO-MCBS v1.1 has no mechanism for scoring non-inferiority studies in a non-curative setting based on response rate.

  2. When PFS (or EFS) is very long, the ESMO-MCBS v1.1 has no mechanism to credit strong interim gains when the median PFS of the control arm has not yet been reached.

  3. The EHA working group members felt that the capitation of PFS at a maximal preliminary grade of 3, with provision for an upgrade based on tail of the curve only if there is a plateau in the study medication PFS with gain of >10% at 12 months, may have undervalued some MM treatments.96 97 The plateau requirement for this adjustment precludes credit for substantial prolonged gains in PFS in this disease entity.

Myelodysplastic syndrome

Studies evaluated: Ten studies were evaluated in this setting.105–114 Of these, two studies were evaluated based on OS or PFS and the remaining eight studies were evaluated based on response rate (table 9).

Table 9

Feasibility testing of the ESMO-MCBS v1.1 for myelodysplastic syndrome (n=10)

Scorability: All studies incorporated required data for evaluation using the ESMO-MCBS v1.1. Clinical benefit measure was, however, partly confounded by the heterogeneity of the available definitions of haematological response and their clinical meaningfulness.

Reasonableness: In the two studies evaluating hypomethylating agents in intermediate-risk/high-risk patients,105 106 the ESMO-MCBS v1.1 graded them with substantial benefit based on either PFS gain or OS gain with improved QOL. In lower risk patients, the remaining eight studies included randomised trials investigating erythropoietin-stimulating agents, lenalidomide in MDS with del(5q) or non-del(5q) and immunosuppressive therapy with antithymocyte globulin plus cyclosporine, compared with best supportive care.107–114 All studies were evaluated based on response rates, but they used a range of different and inconstant criteria, some using International Working Group, or modifications thereof, and other study-specific criteria such as transfusion requirements. All these studies resulted in a final ESMO-MCSB v1.1 score of 2. In one of these studies108 QOL was evaluated and demonstrated to have improved but this was not reflected in grading since there is no QOL bonus for studies in which response rate is the primary outcome.

Shortcomings: The EHA working group identified one shortcoming derived from these evaluations:

  1. In studies evaluating response rate as a primary endpoint, there is no provision of QOL bonus if improved QOL is demonstrated as a secondary outcome.

Discussion

The EHA with currently more than 5000 members is the largest European-based haematology association. In addition to its educational mission, it has a public policy and advocacy role that engages stakeholders, including patient representatives, to improve patient care and to raise awareness for haematology as a distinct medical discipline with specific needs.115 Reflecting these goals, EHA has observed the development of the ESMO-MCBS and its broad utility in solid tumour oncology with great interest, and in the absence of a value tool validated for malignant haematology, we sought to investigate the applicability of the ESMO-MCBS v1.1 as a first step to the development of a version validated for HMs.

There are several major differences in the behaviour of HMs as compared with solid tumour cancers. These differences arise largely from the more variable natural history of HMs which can range from fulminant (acute leukaemia and high-grade lymphomas) to almost benign (low-grade MDS). Furthermore, many of these malignant haematological diseases, even when they are not cured, they are characterised by very long PFS and OS that are rarely seen among incurable solid tumour malignancies. Finally, the endpoints used in the studies of treatments for HMs are sometimes different to those used in solid tumours and in some instances, such as CML, they are even disease-specific. Consequently, at the outset of this project we did not know if ESMO-MCBS v1.1 could be applied to studies in HMs, and if the grading of studies would generate grades considered reasonable by experts in the relevant diseases.

This evaluation of the behaviour of the ESMO-MCBS v1.1 in the grading of 80 studies across the full spectrum of HMs has demonstrated that the ESMO-MCBS v1.1 is widely applicable for the overwhelming majority of analysed studies (90% scoreable studies) and that the generated scores were generally adjudicated by clinical experts to reasonably accord with their evaluation of the magnitude of clinical benefit. In 5 of the 80 studies (6%), the ESMO-MCBS could not be applied at all21 27 46 49 50 78 and in 3 more studies (4%), it could not be applied to one of the evaluable parameters.68 76 77 In the evaluation of imatinib in CML,36–38 it generated scores that were considered to under-represent the true value of the intervention in the opinion of experts in the evaluated diseases.

Based on the analysis of the scorability of studies and the reasonableness of the generated results, this field testing identified six shortcomings in the current version of the ESMO-MCBS that will require redress to improve the applicability and reasonableness of ESMO-MCBS scoring for malignant haematological conditions.

  1. Regarding single-arm studies with curative intent, such as CAR T-cell salvage therapies, the ESMO-MCBS v1.1 does not have a form to grade single-arm treatments with curative intent.

  2. Regarding relatively indolent conditions with a very long PFS (or EFS) or OS such as CLL, CML, indolent lymphoma and MM, there is no mechanism to credit strong interim gains when the median of the control arm has not yet been reached.

  3. The capitation of PFS at a maximal preliminary grade of 3, with provision for an upgrade based on tail of the curve only when there is a plateau in the arm with the study medication, may undervalue treatments with substantial late PFS gain but with no plateauing of the curves.

  4. Regarding the standard molecular surrogate endpoints used for CML, the surrogacy of complete cytogenic response and level 4–5 MMR must be acknowledged and incorporated.

  5. The scale does not make provision for the grading of non-inferiority studies based on response rate criteria.

  6. In studies evaluating response rate as a primary endpoint, there is no provision of QOL bonus if improved QOL is demonstrated as a secondary outcome.

Finally, it must be acknowledged that the results of the scale may not be reasonable for some of the least malignant of the HMs such as low-risk MDS. Most of the studies for MDS were evaluated based on response rates, but there was heterogeneity of the available definitions of haematological response and their clinical meaningfulness. This underlines the need for a stand-alone form regarding studies with such heterogeneity in their response rates.

ESMO and the EHA are committed to the development of a version of the ESMO-MCBS that is validated for HMs. Based on the findings of this study, a revised version of the ESMO-MCBS will be developed to address the identified shortcomings in the current version of the scale regarding the assessment of HMs. This development process will incorporate all the usual stringencies for accountability of reasonableness that have characterised the development of the ESMO-MCBS. This, thus far, included field testing, statistical modelling, evaluation for reasonableness and openness to appeal and revision. Applying such a scale will support future decision-making and will provide insights that could be helpful in the design of future clinical trials.

References

View Abstract

Footnotes

  • BK and NIC contributed equally.

  • Contributors Design of the study: BK, NIC, FC, EGEdV, NJL, J-YD, PS. Collection of trials/data: BK, NB, PG, NG, VG-C, BH, UJ, LM, M-VM, GJO, KP, MR, J-MR, LS, RW, PS. Scoring, analysis and interpretation of data: BK, NIC, NB, UD, EGEdV, PG, NG, VG-C, BH, UJ, LM, M-VM, GJO, KP, MR, J-MR, LS, RW, PZ, PS. Statistical analysis: UD, PZ. Manuscript design and writing: BK, NIC, FC, EGEdV. Manuscript editing and final approval: all authors.

  • Competing interests BK has received honoraria for lectures from Celgene and Novartis; NB has received honoraria for consulting and advisory role from Ariad, Amgen, Novartis and Pfizer; EGEdV declares advisory role for Daiichi Sankyo, Merck, NSABP, Pfizer, Sanofi and Synthon, and institutional financial support for clinical trials or contracted research for Amgen, AstraZeneca, Bayer, Chugai, CytomX Therapeutics, G1 Therapeutics, Genentech, Nordic Nanovector, Radius Health, Roche and Synthon; PG has received honoraria for lectures/advisory boards and research funding from AbbVie/Pharmacyclics, Acerta/AstraZeneca, BeiGene, Janssen, Gilead, Novartis, Sunesis and Verastem; NG has received honoraria for lectures, advisory boards and research support from Amgen, Pfizer and Incyte; VG-C has received honoraria for lectures from Janssen and for participation in advisory boards from Prothena; BH has received honoraria for lectures from Novartis, Pfizer and BMS, and for participation in advisory boards from Novartis, GlaxoSmithKline, Abbvie, Pfizer, BMS and Incyte; UJ has declared honoraria from Abbvie, BMS, Celgene, Amgen, Roche, Novartis, Gilead, Bioverativ, Janssen, MSD, Sandoz, Takeda and Pfizer; M-VM has received honoraria for lectures from Janssen, Celgene, Takeda and Amgen, and for participation in advisory boards from Janssen, Celgene, Takeda, Amgen, AbbVie, GlaxoSmithKline and PharmaMar; MR has received honoraria for lectures from Novarits, Ipsen, Celgene and Eisai; LS has received honoraria for lectures from Janssen, AbbVie, AstraZeneca, and for participation in advisory boards from Janssen; PS has received honoraria for lectures and advisory boards from Celgene, Janssen, Takeda and Amgen, and research support from Celgene, Janssen and Amgen.

  • Patient consent for publication Not required.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data availability statement Data sharing not applicable as no datasets generated and/or analysed for this study.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.