If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Decisions on Further Research for Predictive Biomarkers of High-Dose Alkylating Chemotherapy in Triple-Negative Breast Cancer: A Value of Information Analysis
Department of Psychosocial Research and Epidemiology, Netherlands Cancer Institute-Antoni van Leeuwenhoek Hospital (NKI-AVL), Amsterdam, The Netherlands
Department of Psychosocial Research and Epidemiology, Netherlands Cancer Institute-Antoni van Leeuwenhoek Hospital (NKI-AVL), Amsterdam, The Netherlands
Address correspondence to: W.H. van Harten, Division of Psychosocial Research and Epidemiology, The Netherlands Cancer Institute, Plesmanlaan 121, 1066CX Amsterdam, The Netherlands.
Department of Psychosocial Research and Epidemiology, Netherlands Cancer Institute-Antoni van Leeuwenhoek Hospital (NKI-AVL), Amsterdam, The NetherlandsDepartment of Health Technology and Services Research, University of Twente, Enschede, The Netherlands
To inform decisions about the design and priority of further studies of emerging predictive biomarkers of high-dose alkylating chemotherapy (HDAC) in triple-negative breast cancer (TNBC) using value-of-information analysis.
Methods
A state transition model compared treating women with TNBC with current clinical practice and four biomarker strategies to personalize HDAC: 1) BRCA1-like profile by array comparative genomic hybridization (aCGH) testing; 2) BRCA1-like profile by multiplex ligation-dependent probe amplification (MLPA) testing; 3) strategy 1 followed by X-inactive specific transcript gene (XIST) and tumor suppressor p53 binding protein (53BP1) testing; and 4) strategy 2 followed by XIST and 53BP1 testing, from a Dutch societal perspective and a 20-year time horizon. Input data came from literature and expert opinions. We assessed the expected value of partial perfect information, the expected value of sample information, and the expected net benefit of sampling for potential ancillary studies of an ongoing randomized controlled trial (RCT; NCT01057069).
Results
The expected value of partial perfect information indicated that further research should be prioritized to the parameter group including “biomarkers’ prevalence, positive predictive value (PPV), and treatment response rates (TRRs) in biomarker-negative patients and patients with TNBC” (€639 million), followed by utilities (€48 million), costs (€40 million), and transition probabilities (TPs) (€30 million). By setting up four ancillary studies to the ongoing RCT, data on 1) TP and MLPA prevalence, PPV, and TRR; 2) aCGH and aCGH/MLPA plus XIST and 53BP1 prevalence, PPV, and TRR; 3) utilities; and 4) costs could be simultaneously collected (optimal size = 3000).
Conclusions
Further research on predictive biomarkers for HDAC should focus on gathering data on TPs, prevalence, PPV, TRRs, utilities, and costs from the four ancillary studies to the ongoing RCT.
Descriptive analysis of estrogen receptor (ER)-negative, progesterone receptor (PR)-negative, and HER2-negative invasive breast cancer, the so-called triple-negative phenotype: a population-based study from the California Cancer Registry.
Strategies for subtypes—dealing with the diversity of breast cancer: highlights of the St Gallen International Expert Consensus on the Primary Therapy of Early Breast Cancer 2011.
]. Although second- and third-line treatments exist, these typically increase overall costs but do not contribute sufficiently to improve long-term health outcomes [
]. Therefore, improving first-line treatment seems a promising way forward to decrease both patient morbidity and health care costs in this population.
], and clinical data from a retrospective study conducted alongside a prospective randomized controlled trial (RCT) in our center (the Netherlands Cancer Institute-Antoni van Leeuwenhoek Hospital NKI) [
An aCGH classifier derived from BRCA1-mutated breast cancer and benefit of high-dose platinum-based chemotherapy in HER2-negative breast cancer patients.
], indicate that high-dose alkylating chemotherapy (HDAC) may be an effective treatment option for TNBC tumors without functional BRCA1, also known as BRCA1-like tumors. Furthermore, in an extension of this study, it was found that by further characterizing BRCA1-like tumors with two other biomarkers, X-inactive specific transcript gene (XIST) [
], responses to HDAC treatment increase by 30%, that is, patients with a BRCA1-like profile, high expression of 53BP1 (53BP1+), and low expression of XIST (XIST−) have a 100% response rate compared with the 70% yielded with the BRCA1-like biomarker alone. On the basis of these results, a prospective RCT to test the survival advantage of treating TNBCs with the BRCA1-like biomarker and HDAC was started (Randomized phase II/III study of individualized neo-adjuvant chemotherapy in triple negative breast tumors [TNM trial, NCT01057069]). The trial started in 2010 and is currently ongoing.
As the research on BRCA1-like, XIST, and 53BP1 biomarkers is now progressing from initial clinical studies toward “pivotal” studies to determine their diagnostic, patient, and societal value, early-phase economic evaluation can be applied to improve the efficiency of the research and development process. Early-phase economic evaluations have a decision analytic approach to iteratively evaluate technologies in development so as to increase their return on investment as well as have better patient and societal impact when the technology becomes available [
]. For instance, value-of-information (VOI) methods quantify the potential benefit of additional information in the face of uncertainty. VOI is based on the idea that information is valuable because it reduces the expected costs of uncertainty surrounding a decision. A detailed explanation of the VOI methodology can be found elsewhere [
Because decisions on emerging technologies with scarce clinical studies will inevitably be uncertain, research is expected to be worthwhile but only up to a certain cost of research. VOI methods allow us to estimate an upper bound to the returns of further research expenditures and are particularly helpful in setting research priorities for specific model parameters as well as for specific research designs and sample sizes [
]. The data gathered in and the research infrastructure of the ongoing TNM trial provide an opportunity to reduce uncertainty in a range of parameters that inform the decision problem against additional costs. Therefore, this study aimed to identify for which specific ancillary study designs further research is most valuable, and to inform future decisions on emerging predictive biomarkers for the selection of HDAC for TNBC.
Methods
A Markov model was constructed with three mutually exclusive health states: disease-free survival (DFS), relapse (R) (including local, regional, and distant relapses), and death (D). Our analysis took a Dutch societal perspective and a time horizon of 20 years because the occurrence of relapses and deaths are expected within this time frame [
Quantitative copy number analysis by Multiplex Ligation-dependent Probe Amplification (MLPA) of BRCA1-associated breast cancer regions identifies BRCAness.
]. Effectiveness was assessed in terms of quality-adjusted life-years (QALY) and costs in 2013 euros (€). Future costs and effects were discounted to their present value by a rate of 4% and 1.5% per year, respectively [
Patient Population Studied and Strategies Compared
We modeled five identical cohorts of 40-year-old women with TNBC, four treated with personalized HDAC as dictated by biomarkers and one treated according to current practice, with a mean duration of 1 year (see Fig. 1 and description). Drug regimens were based on a published RCT comparing HDAC and standard chemotherapy efficacy in breast cancer [
BRCA1-like tested by array comparative genomic hybridization (BRCA1-like-aCGH): Women are initially tested for the BRCA1-like profile by aCGH. Those who have a BRCA1-like profile are assigned to the HDAC arm (4-FEC [fluorouracil, epirubicin, and cyclophosphamide], followed by 1-CTC [cyclophosphamide, thiotepa, and carboplatin]), and those missing the profile are assigned to standard chemotherapy (5-FEC).
2.
BRCA1-like tested by multiplex ligation-dependent probe amplification (BRCA1-like-MLPA): MLPA was developed to be more time-efficient, cheaper, and technically less complicated than the aCGH [
Quantitative copy number analysis by Multiplex Ligation-dependent Probe Amplification (MLPA) of BRCA1-associated breast cancer regions identifies BRCAness.
]. We modeled this strategy exactly as the previous one.
3.
BRCA1-like-aCGH followed by XIST and 53BP1 (BRCA1-like-aCGH/XIST-53BP1): Women are initially tested with the BRCA1-like-aCGH classifier, as aforementioned. Patients with a BRCA1-like profile are further tested for XIST and 53BP1 expression, and patients with a non–BRCA1-like profile receive standard chemotherapy. XIST expression is detected with an MLPA assay and 53BP1 by immunochemistry. These markers are interpreted together; patients with a BRCA1-like profile with a low expression of XIST and presence of 53BP1 are considered sensitive for HDAC and thus assigned to HDAC. Patients with any other combination of the markers are considered resistant and are assigned to standard chemotherapy.
4.
BRCA1-like-MLPA followed by XIST and 53BP1 (BRCA1-like-MLPA/XIST-53BP1): This strategy was modeled exactly as the previous one, but by assessing the BRCA1-like status by MLPA.
5.
Current clinical practice: All women are treated with standard chemotherapy.
Fig. 1Decision tree. (Color version of figure available online).
Patients were classified as “respondents” to the assigned chemotherapy when no relapse occurred within the first 5 years and as “nonrespondents” in case such an event occurred within the first 5 years. This time frame was considered a reasonable limit to include all events related to chemotherapy response [
After the intervention, patients enter into the DFS health state of the model, in which they will remain for the first year, accruing the costs and the health-related quality-of-life (HRQOL) weights of the administered chemotherapy. During this year, patients can die from chemotherapy-related toxic events (septicemia and heart failure [
]) or from events not related to breast cancer. Patients can move to the R health state from the first year onward. Patients with a relapse receive treatment and can 1) remain in the R health state and accrue the costs and HRQOL weights of the DFS health state, representing a “cured” relapse, or 2) die from breast cancer or other unrelated cause. We assumed that patients could have only one relapse during the time horizon of the model.
Model Input Parameters
The baseline prevalence of BRCA1-like was derived from three patient series (n = 377) in our hospital [
], including patients enrolled in the TNM trial, and it was considered equal for both MLPA and aCGH tests. The baseline prevalence of BRCA1-like/XIST−/53BP1+ was determined from an existing retrospective study from a prospective RCT in our institute [
An aCGH classifier derived from BRCA1-mutated breast cancer and benefit of high-dose platinum-based chemotherapy in HER2-negative breast cancer patients.
] (n = 60), separately for the MLPA and the aCGH tests. This patient series was also used to derive 1) the positive predictive value (PPV) (proportion of biomarker-positive patients responding to HDAC as determined by the MLPA and aCGH BRCA1-like tests alone, and by their combination with the XIST and the 53BP1 tests); 2) the treatment response rates (TRRs) of biomarker-negative patients as determined by the MLPA and aCGH BRCA1-like tests alone, and by their combination with the XIST and the 53BP1 tests; and 3) the TRRs of patients with TNBC.
The transition probabilities (TPs) of relapse-free survival and breast-cancer–specific survival were estimated from the study by Lester-Coll et al. [
]. Using these data required making the assumption that most relapses in TNBC are metastatic, which is a plausible assumption given that in this subtype 1) metastatic disease is rarely preceded by other recurrences [
A longitudinal prospective study of health-related quality of life in breast cancer patients following high-dose chemotherapy with autologous blood stem cell transplantation.
]. During the first year of the DFS health state, patients were attributed the utility of the chemotherapy received (i.e., standard chemotherapy or HDAC) and during the following 4 years, the HRQOL of DFS. In the first year of the R health state, patients were attributed the utility of R, and in subsequent years, the utility of DFS. We assumed that HRQOL was not affected by BRCA1-like testing itself.
Model costs included costs for biomarker testing, chemotherapy, and breast cancer health states, each of them calculated as a sum of direct medical costs, direct non-medical costs (e.g., patient travel expenses), and productivity losses. Direct and indirect medical costs were derived from literature, the NKI financial department, and Dutch sources on resource use and unit prices [
]. Foreign currencies were converted to 2013 euros (XE currency converter; http://www.xe.com/), and the consumer price index was used to account for inflation [
Quantitative copy number analysis by Multiplex Ligation-dependent Probe Amplification (MLPA) of BRCA1-associated breast cancer regions identifies BRCAness.
Quantitative copy number analysis by Multiplex Ligation-dependent Probe Amplification (MLPA) of BRCA1-associated breast cancer regions identifies BRCAness.
Quantitative copy number analysis by Multiplex Ligation-dependent Probe Amplification (MLPA) of BRCA1-associated breast cancer regions identifies BRCAness.
An aCGH classifier derived from BRCA1-mutated breast cancer and benefit of high-dose platinum-based chemotherapy in HER2-negative breast cancer patients.
An aCGH classifier derived from BRCA1-mutated breast cancer and benefit of high-dose platinum-based chemotherapy in HER2-negative breast cancer patients.
An aCGH classifier derived from BRCA1-mutated breast cancer and benefit of high-dose platinum-based chemotherapy in HER2-negative breast cancer patients.
An aCGH classifier derived from BRCA1-mutated breast cancer and benefit of high-dose platinum-based chemotherapy in HER2-negative breast cancer patients.
An aCGH classifier derived from BRCA1-mutated breast cancer and benefit of high-dose platinum-based chemotherapy in HER2-negative breast cancer patients.
Quantitative copy number analysis by Multiplex Ligation-dependent Probe Amplification (MLPA) of BRCA1-associated breast cancer regions identifies BRCAness.
An aCGH classifier derived from BRCA1-mutated breast cancer and benefit of high-dose platinum-based chemotherapy in HER2-negative breast cancer patients.
An aCGH classifier derived from BRCA1-mutated breast cancer and benefit of high-dose platinum-based chemotherapy in HER2-negative breast cancer patients.
An aCGH classifier derived from BRCA1-mutated breast cancer and benefit of high-dose platinum-based chemotherapy in HER2-negative breast cancer patients.
An aCGH classifier derived from BRCA1-mutated breast cancer and benefit of high-dose platinum-based chemotherapy in HER2-negative breast cancer patients.
An aCGH classifier derived from BRCA1-mutated breast cancer and benefit of high-dose platinum-based chemotherapy in HER2-negative breast cancer patients.
An aCGH classifier derived from BRCA1-mutated breast cancer and benefit of high-dose platinum-based chemotherapy in HER2-negative breast cancer patients.
Quantitative copy number analysis by Multiplex Ligation-dependent Probe Amplification (MLPA) of BRCA1-associated breast cancer regions identifies BRCAness.
An aCGH classifier derived from BRCA1-mutated breast cancer and benefit of high-dose platinum-based chemotherapy in HER2-negative breast cancer patients.
An aCGH classifier derived from BRCA1-mutated breast cancer and benefit of high-dose platinum-based chemotherapy in HER2-negative breast cancer patients.
An aCGH classifier derived from BRCA1-mutated breast cancer and benefit of high-dose platinum-based chemotherapy in HER2-negative breast cancer patients.
Quantitative copy number analysis by Multiplex Ligation-dependent Probe Amplification (MLPA) of BRCA1-associated breast cancer regions identifies BRCAness.
Quantitative copy number analysis by Multiplex Ligation-dependent Probe Amplification (MLPA) of BRCA1-associated breast cancer regions identifies BRCAness.
A longitudinal prospective study of health-related quality of life in breast cancer patients following high-dose chemotherapy with autologous blood stem cell transplantation.
A longitudinal prospective study of health-related quality of life in breast cancer patients following high-dose chemotherapy with autologous blood stem cell transplantation.
Using the assumption of 25% standard deviation of the mean reported value in a logarithmic scale resulted in a negative value, thus we used 10% instead.
Parameters for the distributions: Beta distribution: α/β, Normal distribution: mean/variance, Log-normal distribution: Log mean/log SD
53BP1, tumor suppressor p-53 binding protein; aCGH, array comparative genomic hybridization; Assum, standard deviation is equal to 25% of the mean; HDAC, high-dose alkylating chemotherapy; IHC, immunochemistry; MLPA, multiplex ligation-dependent probe amplification; PBPCT, peripheral blood progenitor cell transplantation; SC, standard chemotherapy; SD, standard deviation; XIST, X-inactive specific transcript gene.
Loss of productivity costs in test are zero.
† Each BRCA1-like MLPA test requires both patient and control samples, each of them costing €9 for the MLPA kit (enzymes and reagents).
‡ The MLPA test requires six control samples and one patient sample in each run. With an optimal sample size of 18 samples, this results in 24 samples.
§ Using the assumption of 25% standard deviation of the mean reported value in a logarithmic scale resulted in a negative value, thus we used 10% instead.
‖ The aCGH test requires labelling of 12 patient samples and one control sample in each run.
¶ We assumed optimal test batching of 12 patient samples in each run.
# Follow-up period in which the patient is controlled until recovery of blood activity.
Includes one trip to the hospital for each FEC cycle and one trip for the hospital for PBPCT (admission and discharge).
†† We assumed patients did not work during chemotherapy (n = 20), during PBPCT procedures (n = 21), or during the post-PBPCT program (n = 20).
‡‡ Source did not report travelling expenses and thus was not added.
§§ Indirect costs were calculated by using resource use of Lidgren et al
Parameter uncertainty was quantified in the decision model by assigning distributions to all parameters that are subject to sampling uncertainty. Following the recommendations by Briggs et al. [
], a beta distribution was assigned to binomial data, such as biomarkers’ prevalence, PPVs, TPs, and TRRs in biomarker-negative patients and patients with TNBC, and a lognormal distribution to rightly skewed data, such as costs. For uncertainty in mean utilities, we followed Brennan et al. [
], who suggested the use of a normal distribution. Because sampling from one utility distribution (HDAC) occasionally produced a parameter value below 0, this was truncated. The parameterization of each distribution can be derived from Table 1. Uncertainty ranges for BRCA1-like–MLPA and BRCA1-like–aCGH prevalence, and for TRR in non–BRCA-1-like patients under both tests came from literature on the tests’ development. This reported a 14% error of the MLPA test versus the aCGH test [
An aCGH classifier derived from BRCA1-mutated breast cancer and benefit of high-dose platinum-based chemotherapy in HER2-negative breast cancer patients.
]. For the utility data, either the standard errors or the 95% confidence intervals of the mean were derived from literature. Because limited information regarding parameter uncertainty is available for costs, we assumed that standard errors of the aggregate costs were equal to 25% of the mean. Nevertheless, if on the logarithmic scale this resulted in negative values, 10% was used. Because literature to characterize uncertainty on specific items of the health-state aggregate costs existed, this was used accordingly in these separate items, with the former assumptions being made for the remaining items of the aggregate value. The joint parameter uncertainty was then propagated through the model using Monte-Carlo simulation with 10,000 random samples from the predefined distributions. Cost-effectiveness acceptability curves (CEACs) were estimated to show the joint decision uncertainty surrounding the expected incremental cost-effectiveness across €0 to €80,000 willingness-to-pay values for one additional QALY.
Value of Further Research and Research Priorities
The expected value of perfect information (EVPI) was calculated for the population expected to benefit from a reduction in uncertainty—patients with TNBC eligible for HDAC, that is, patients younger than 60 years with stage II to IV treatable cancers. The model assumes that the entire affected population will receive the optimal strategy. In the Netherlands, the affected population amounts to 662 patients per annum (of the 6619 women with breast cancer who are younger than 60 years in the Netherlands [
], i.e., treatable metastatic cases). To this figure, an annual discount rate of 4% was applied over a 10-year time horizon of the technology, assumed to be the period during which the information is relevant to inform the decision. The expected value of partial perfect information (EVPPI) requires two-level Monte-Carlo simulation [
], beginning with an outer loop (100) sampling values from the distribution of the parameters of interest and an inner loop (1000) sampling the remaining parameters from their conditional distribution [
]. The parameters of interests were determined on the basis of the type of study design required for further research: 1) RCT to inform the TP; 2) quality-of-life (QOL) survey to provide further information regarding utility weights associated with chemotherapy and breast cancer health states; 3) longitudinal costing study to provide more information on resource use of the tests, the chemotherapy, and the health states; and 4) longitudinal study to provide more information on the biomarkers’ prevalence, PPVs, and the TRRs of biomarker-negative patients and patients with TNBC [
In this study, we prioritize specific further research, designs depending on what type of data are needed and their vulnerability to specific risks of bias, and on the research infrastructure that is available from the TNM trial, which is an ongoing Dutch RCT aiming to provide evidence on the survival advantage (in terms of relapse-free survival and overall survival) of treating BRCA1-like patients with TNBC as detected by MLPA with HDAC versus standard chemotherapy. Thereby, further research was proposed as follows.
Further data on TP, BRCA1-like prevalence, BRCA1-like PPV, and TRRs in biomarker-negative patients and patients with TNBC as identified by MLPA were assumed to come at the expenses of the TNM trial, with the only additional costs of more advanced statistical analysis methods than planned for the original trial (this was defined as study 1). Evidence on BRCA1-like prevalence as determined by aCGH, BRCA1-like/XIST−/53BP1+ prevalence as determined by MLPA and aCGH, and TRRs in biomarker-negative patients and patients with TNBC as identified by aCGH could be derived from undertaking a retrospective study using the TNM trial samples. To determine the prevalence, patient samples would first be tested by aCGH. Subsequently, those resulting BRCA1-like would be tested by 53BP1 and XIST. To determine the PPV and TRR in each case, additional statistical analysis correlating the presence/absence of biomarker with survival data would be performed. The costs for this study would include retesting patient samples and additional statistical analysis (study 2). Evidence on direct medical costs could also be gathered from a retrospective study to the TNM trial. In this study, resource use and unit costs for the relevant parameters would be determined, incurring costs for data collection and statistical analysis (study 3). Evidence on QOL could be derived from an ancillary prospective survey to the TNM trial. Expenses resulting from this trial would be for distributing, collecting, and analyzing the QOL surveys (study 4).
Testing costs for the aCGH, 53BP1, and XIST biomarkers were derived from the financial department of the NKI (€30 for XIST testing, €22 for 53BP1 testing, and €106 for aCGH testing). The costs of performing statistical analysis only, additional data collection and statistical analysis, and a QOL survey were based on the costs of data management and analysis of a mock RCT presented in the literature [
]. From this source we specifically used the average of “academic medical and cancer centers” costs and “oncology group practices” costs. The total costs per patient were estimated at €1325 for study 1, at €1466 for study 2 (including €141 for XIST and 53BP1 testing in 68% BRCA1-like patients and aCGH testing in all patients, and €1325 for the statistical analysis), and at €1325 each for studies 3 and 4. The expected value of sample information (EVSI) was calculated for each of the four studies for a range of sample sizes, starting from 100, using a two-level Monte-Carlo simulation with 5000 inner and 5000 outer loops (the number of loops was increased sequentially to check for convergence, i.e., to check that increasing simulation size [for both inner and outer loops] would not change estimates). The expected net benefit of sampling (ENBS) was subsequently calculated for each study design and n, by subtracting the corresponding costs of research. The n in which the ENBS was maximized was the optimal sample size for each proposed study. Furthermore, we calculated the optimal sample size for the portfolio of studies, by assuming that these are undertaken simultaneously and results of one cannot inform results of others. Under this assumption, the optimal sample size is the combination of sample sizes across studies that maximizes the ENBS [
The BRCA1-like–MLPA/XIST-53BP1, the BRCA1-like–aCGH/XIST-53BP1, and the BRCA1-like–aCGH strategies are expected to be cost-effective at a willingness-to-pay threshold of €80,000/QALY, when compared with current clinical practice, the BRCA1-like–MLPA/XIST-53BP1 and the BRCA1-like–MLPA strategy, respectively. On the contrary, the additional costs of the BRCA1-like–MLPA strategy were not balanced by the gain in health outcomes when compared with the BRCA1-like–aCGH/XIST-53BP1 strategy, resulting in an incremental cost-effectiveness ratio of €94,310/QALY. The CEACs show that at a willingness-to-pay threshold of €80,000/QALY the decision as to which strategy is most cost-effective is uncertain. The base-case results and the CEACs are presented in Figure 2.
Fig. 2Base case results and cost-effectiveness acceptability curves. The strategies are listed in order of increasing costs. In evaluating the incremental cost effectiveness ratios, each strategy’s costs and effects were compared with those of a slightly more expensive strategy.
Results of the EVPI and EVPPI are presented in Figure 3. The EVPI was estimated at €693 million at the prevailing threshold of €80,000/QALY. The EVPPI identified the group of parameters including the biomarkers’ prevalence, the PPVs, and TRRs in biomarker-negative patients and patients with TNBC to be most uncertain (€639 million), followed by utilities (€48 million), cost-related parameters (€40 million), and TPs (€30 million).
Fig. 3EVPI and EVPPI estimates. (Color version of figure available online).
In Figure 4 we present graphically the ENBS and optimal sample size for the four proposed studies separately. These were €600 million and 9000 for study 1, €440 million and 1000 for study 2, €597 million and 200 for study 3, and €446 million and 1000, respectively, for study 4. The optimal sample size for the portfolio of studies was 3000, with an ENBS of €2074 million.
Fig. 4ENBS and optimal sample size for each of the four studies ancillary to the ongoing RCT.
This study found that testing for BRCA1-like alone with the aCGH test and testing for BRCA1-like in combination with the biomarkers XIST and 53BP1 with the aCGH and the MLPA tests may be cost-effective, and that there is substantial value in investing in further research for these diagnostic tests. VOI analysis showed that setting up four ancillary studies to the present TNM trial to collect data on 1) TP and MLPA prevalence, PPV, and TRR; 2) aCGH and aCGH/MLPA plus XIST and 53BP1 prevalence, PPV, and TRR; 3) utilities; and 4) costs would be most efficient in generating information that decreases decision uncertainty around the test and test strategies. The optimal sample size to simultaneously collect data from these four groups of parameters was 3000 patients, with an ENBS of €2074 million.
This article contributes to the literature on real-time applications of EVSI analysis to design and prioritize further research, which is under-represented [
The cost-effectiveness of an RCT to establish whether 5 or 10 years of bisphosphonate treatment is the better duration for women with a prior fracture.
] previously presented an EVSI application in a diagnostic procedure, but most EVSI analyses are applied to treatment interventions. Enhancing the literature on the expected value of further information about diagnostics is relevant for manufacturers because current regulations incentivize research and development of diagnostics relatively poorly [
]. In the meantime, EVSI examples can illustrate how diagnostics’ research and development can be steered more efficiently to increase the returns on investments from a health care and societal perspective. Although many articles indicate the RCT to be the preferred study design to conduct any further research by default, we contribute to the literature in presenting the value of further research for various study designs, depending on what type of data are needed, the risk of bias, and existing research infrastructure.
Apart from the fact that requiring RCTs for all forms of further data collections cannot inherently be justified in a rational way, there are two external motivations to consider the ENBS of non-RCT designs: 1) the evidence requirements for market approval and reimbursement of diagnostics, which are generally less rigidly defined compared with pharmaceuticals, therefore allowing to use other valuable sources of evidence; and 2) lower levels of evidence than the RCTs are increasingly acceptable to decision makers, as recently stated by the Food and Drug Administration [
US Food and Drug Administration. Guidance documents (medical devices and radiation-emitting products) > guidance for industry and Food and Drug Administration staff—factors to consider when making benefit-risk determinations in medical device premarket approvals and de novo classifications. 2012. Available from: http://www.fda.gov/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm267829.htm. [Accessed September 23, 2014].
When calculating the EVSI of study designs other than RCTs, parameter vulnerability to selection bias needs to be assessed. Although this may be of less concern for costs and health-states utility data, selection bias in retrospective and/or observational studies can severely affect effectiveness parameters (such as TRRs and PPVs) and should be prevented or statistically accounted for. The use of retrospective studies alongside RCTs is increasingly promoted because these can generate high-quality evidence while being fast and inexpensive [
]. This is however possible only for diagnostics of already existing chemotherapeutic regimens, for which data on efficacy are already available from RCTs.
Our study was not exempt from limitations. First, by nature of the early-stage analysis, the input data on biomarkers’ prevalence, biomarkers’ PPV, and TRRs in biomarker-negative patients and patients with TNBC were derived from several small retrospective studies. Indeed, EVPPI analysis showed high value in collecting further information on these, and our ENBS analysis suggests how this could be done most efficiently. Second, the TNM trial uses intensified alkylating chemotherapy instead of HDAC. Although this means that the therapy is administered more frequently (2×) and at lower doses (half), it results in equal cumulative doses and equal need for stem-cell transplantation. Therefore, the survival advantage is expected to be similar. Third, the costs of testing were estimated by using optimal test batching, probably an optimistic assumption considering the prevalence of TNBC in the breast cancer population. Nevertheless, it is not expected that this would markedly alter the conclusions of the analysis, because in a previous analysis of our model [
] testing costs were not a key driver of outcomes. Fourth, the research costs used for the ENBS calculations were derived from the published costs of a typical though hypothetical RCT [
]. Although these estimates seem reasonable for a real trial, the use of actual costs may change the results. Fifth, the estimated costs of study 2 ignore the different accuracies of the aCGH and MLPA tests. Although this could translate into additional XIST and 53BP1 testing to derive the prevalence and PPV under the BRCA1-like–aCGH/XIST-53BP1 strategy, we expected these costs to be minimal. Sixth, the EVPI is dependent on estimates of population size, the time horizon, and the discount rate. We based these parameters on the Dutch situation, yet results to other countries require reconsideration of these inputs. Seventh, it is possible that other biomarkers to predict sensitivity to HDAC will be identified in the future. This would add additional comparator(s) to the decision problem, thus increasing EVPI and probably the need for further research. Therefore, this type of analysis needs to be repeated over time (iterative process) to keep up with the latest developments. Furthermore, biases in early-phase evidence are expected, when their design and conduct are not as rigorous as those of a large RCT. In this situation, it is important to characterize the extent of uncertainty because VOI is highly sensitive to this [
]. Although we justified our data sources for both mean values and their variance, and explained data assumptions thoroughly, we did not conduct additional sensitivity analyses on the resulting parameter distributions [
]. Finally, although we accounted for the correlation between the most important cost-effectiveness drivers sensitivity and specificity by using the Dirichlet distribution, we acknowledge that correlations may be present in other input parameters. This could impact the EVPI results and hence the EVSI estimates, with a magnitude depending on the strength of input correlation [
]. We suggest that sophisticated methods that explicitly quantify joint distributions of correlated parameters be considered in further VOI analysis.
Conclusions
This study illustrated the use of full Bayesian VOI analysis in a set of diagnostic tests, for which further research was designed depending on the type of data needed and its vulnerability to specific risks of bias, and on the research infrastructure available from an ongoing RCT.
Acknowledgments
We thank Professor Dr. Sjoerd Rodenhuis, Mr. Philip Schouten, Dr. Petra M. Nederlof, and Dr. Esther H. Lips for sharing their valuable insights regarding BRCA1-like, XIST, and 53BP1 testing in clinical practice.
Source of financial support: This project was funded by the Center of Translational Molecular Medicine (project Breast CARE, grant no. 03O-104).
Descriptive analysis of estrogen receptor (ER)-negative, progesterone receptor (PR)-negative, and HER2-negative invasive breast cancer, the so-called triple-negative phenotype: a population-based study from the California Cancer Registry.
Strategies for subtypes—dealing with the diversity of breast cancer: highlights of the St Gallen International Expert Consensus on the Primary Therapy of Early Breast Cancer 2011.
An aCGH classifier derived from BRCA1-mutated breast cancer and benefit of high-dose platinum-based chemotherapy in HER2-negative breast cancer patients.
Quantitative copy number analysis by Multiplex Ligation-dependent Probe Amplification (MLPA) of BRCA1-associated breast cancer regions identifies BRCAness.
A longitudinal prospective study of health-related quality of life in breast cancer patients following high-dose chemotherapy with autologous blood stem cell transplantation.
The cost-effectiveness of an RCT to establish whether 5 or 10 years of bisphosphonate treatment is the better duration for women with a prior fracture.
US Food and Drug Administration. Guidance documents (medical devices and radiation-emitting products) > guidance for industry and Food and Drug Administration staff—factors to consider when making benefit-risk determinations in medical device premarket approvals and de novo classifications. 2012. Available from: http://www.fda.gov/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm267829.htm. [Accessed September 23, 2014].
Loss of productivity costs in test are zero.
00021-8&id=gr1.jpg){kind=link}
00021-8&id=gr2.jpg){kind=link}
00021-8&id=gr3.jpg){kind=link}
00021-8&id=gr4.jpg){kind=link}