Main

Adjuvant systemic treatment in early breast cancer is aimed at reducing the risk of distance recurrence and breast cancer mortality (Early Breast Cancer Trialists’ Collaborative Group (EBCTCG), 2005). There are several prediction tools available that can aid in the decision to treat patients with systemic therapy, based on clinical and pathological factors (Engelhardt et al, 2013). Prediction tools that can aid in individualising treatment decisions are especially important for older patients, as older patients comprise a heterogeneous group because of differences in comorbidity and functional status (Braithwaite et al, 2010; Barnett et al, 2012). Currently, the Dutch guideline specifically advices to use the prediction tool ‘Adjuvant! Online’ in older patients with early breast cancer (NABON, 2012). Furthermore, the guideline of the International Society for Geriatric Oncology states that although not specifically validated in older patients, Adjuvant! Online can be used older patients with breast cancer to weigh the risks and benefits of adjuvant treatment (Wildiers et al, 2007). Adjuvant! Online is a tool that predicts absolute 10-year overall survival, breast cancer survival and recurrence, and provides clinicians with the expected benefits of endocrine therapy and chemotherapy (Ravdin et al, 2001). It is the most well-known and most widely used clinical prediction model available (Ravdin et al, 2001; Engelhardt et al, 2014).

Recently, we demonstrated that Adjuvant! Online does not accurately predict overall survival and recurrence in older women with early breast cancer (de Glas et al, 2014c). This was most likely explained by the fact that the model was developed in a cohort of women with a maximum age of 69 years (Ravdin et al, 2001). Therefore, alternative models that can accurately predict breast cancer outcome in older patients are required, especially as the number of older patients with breast cancer is rapidly increasing because of the ageing of Western societies (DeSantis et al, 2011, 2013; Biganzoli et al, 2012).

The more recently developed PREDICT tool may serve as an alternative prediction tool for older patients with breast cancer, as the patient cohort in which it was developed included almost 1800 women aged 65 years or over (Wishart et al, 2010). PREDICT calculates expected overall survival at 5 and 10 years, based on several patient and tumour characteristics(Wishart et al, 2010, 2012). In addition, the model provides expected benefits of chemotherapy, endocrine treatment, and trastuzumab. The model was developed in a population-based cohort in the United Kingdom, and validated in two separate cohorts (Wishart et al, 2010, 2011). It includes all variables that are also incorporated in the Adjuvant! Online model except for comorbidity status, but it additionally uses detection-mode (screen-detected or interval cancer), and recently HER2 status and KI67 status were added to the model (Wishart et al, 2012; Eastern Cancer Registry and Information Centre and Cambridge University, 2015).

The aim of this study was to assess the validity of the online PREDICT tool by studying its predictions for 5- and 10-year overall survival, using a population-based cohort of unselected older patients with breast cancer.

Materials and methods

Patients

For this study, we used data from the population-based FOCUS-cohort (Female breast cancer in the elderly; Optimizing Clinical guidelines USing clinico-pathological and molecular data; de Glas et al, 2013; Kiderlen et al, 2013; de Glas et al, 2014c). All consecutive patients with breast cancer aged 65 years and older in the geographically defined region West of the Comprehensive Cancer Center, the Netherlands, between 1997 and 2004 were included in this cohort (N=3672). Information on patient- and tumour characteristics, detection-mode, treatments, adverse events, and recurrences was registered from medical charts. Follow-up on survival status was available until 31 December 2013 through linkage of patients with municipal population registries.

For this study, all patients with unilateral, unicentric, invasive, local adenocarcinoma were included, provided that they received adequate locoregional treatment (either mastectomy or lumpectomy with radiotherapy and axillary lymph node staging) and they did not receive neoadjuvant treatment.

Immunohistochemistry for Ki67and HER-2 status

All tumour material of patients who were included was collected. Stainings for Ki67 and HER-2 status were performed according to the standard protocols (de Kruijf et al, 2010). HER-2-positive breast cancer as 2+ or 3+ by immunohistochemistry, whereas 0–1 were defined as HER-2-negative breast cancer. FISH staining was not available.

Statistical analyses

All patients were manually entered into the online PREDICT tool via the PREDICT website (Eastern Cancer Registry and Information Centre and Cambridge University (2015)). If information on one of the variables was missing, patients were not excluded but the ‘unknown’ category was used. Predicted 5-year and 10-year survival outcomes were registered, taking the treatment that patients actually received into account.

All analyses were performed in IBM SPSS Statistics version 20.0 (Armonk, NY, USA) and R version 3.1.0 (The R Foundation, Vienna, Austria). A P-value of 0.05 was defined as statistically significant.

First, we calculated 5- and 10-year observed overall survival for all subcategories of patients using the Kaplan–Meier method. These observed outcomes were compared with predicted 5- and 10-year survival outcomes of patients using one-sampled T-tests, with predicted outcomes as ‘fixed’ value and observed outcomes as the evaluated variable. These analyses were performed for the whole cohort as well as for all subgroups that were defined by PREDICT. In addition, we calculated observed and predicted outcomes for subgroups based on number of comorbidities according to the ICD-10 classification (World Health Organisation, 1992; 0, 1, 2, 3, and 4 or more) and treatment characteristics (most extensive surgery and systemic treatment).

Next, we assessed the calibration of the model by plotting the observed and predicted 5-year and 10-year overall survival outcomes. For this, the cohort was divided into 10% intervals of the predicted values by PREDICT. Conform methods that were used in previous studies (Olivotto et al, 2005; Mook et al, 2009; de Glas et al, 2014c), intervals that contained less than 100 patients were combined. Calibration was assessed by comparing the calibration plot with the ‘ideal line’ (x=y), using Poisson Regression (van Houwelingen and Putter, 2012).

Finally, discrimination was assessed by composing receiver-operator curves (ROC) and calculating corresponding c-indices. These are calculated by determining the area under the curve (AUC) of the ROC. A c-index can be interpreted as follows: a c-index of 1.0 means that the model has perfect discrimination, whereas a c-index of 0.5 indicates that the model predicts just as well as flipping a coin (Akobeng, 2007).

Results

Overall, 2012 patients from the FOCUS-cohort fulfilled the inclusion criteria and were included. The inclusion flow diagram has been previously published (de Glas et al, 2014c). Patient characteristics are presented in Table 1. Median age was 75.0 years (inter quartile range=69.0–79.0). The majority of patients (1527 out of 2012 patients, 75.9%) had one or more comorbidities.

Table 1 Baseline characteristics and observed and expected overall 5- and 10-year overall survival
Full size table

Observed 5-year overall survival was 74.2% (s.e. 1.0). Predicted 5-year overall survival was 75.9%, which means that the observed and predicted 5-year overall survival differed by 1.7%, 95% confidence interval (CI)=−0.3 to3.7, P=0.089 (Table 1). The difference between predicted and observed 5-year overall survival was largest in the oldest patients (11.3%, 95% CI=3.8–18.8, P=0.003 in patients ⩾85 years). Overall survival was underestimated in patients without comorbidity (predicted: observed overall survival=−3.7%, 95% CI=−7.2 to −0.2, P=0.040), and overestimated in patients with ⩾4 comorbidities (predicted: observed overall survival 11.8%, 95% CI=6.9–16.7, P<0.0001). In addition, 5-year overall survival was underestimated in patients who received breast-conserving surgery (predicted: observed overall survival=−3.2%, 95% CI=−5.6 to −0.8, P=0.008), and overestimated in patients who received mastectomy as most extensive surgery (predicted: observed 5-year overall survival 4.9%, 95% CI=2.4–7.5, P=0.0002). Finally, PREDICT overestimated 5-year overall survival in patients who received endocrine therapy but no chemotherapy (predicted: observed 5-year overall survival 5.3%, 95% CI=2.0–8.6, P=0.002).

Ten-year overall survival was predicted less well than 5-year overall survival, especially in older patients (aged 75 years or over) and patients with multiple comorbidities (5-year predicted and observed overall survival in patients with 4 or more comorbidities differed 11.8%, 95% CI=6.7–16.7, whereas 10-year predicted and observed overall survival in this group differed 20.7%, 95% CI=15.8–25.6).

Predicted and observed 10-year overall survival differed by 4.5%, 95% CI=2.3–6.6, P<0.0001 (Table 1). Again, the largest differences were observed in the oldest patients (predicted: observed 10-year overall survival in patients ⩾85 12.0%, 95% CI=6.9–17.1, P<0.0001), and in patients with many comorbidities (predicted: observed 10-year overall survival in patients with 4 or more comorbidities 20.7%, 95% CI=15.8–25.6, P<0.0001). Furthermore, 10-year overall survival was overestimated in patients who received mastectomy as most extensive surgery (predicted: observed 10-year overall survival 9.1%, 95% CI=6.4–11.8, P<0.0001) and in patients who received both endocrine therapy and chemotherapy (predicted: observed 10-year overall survival 14.6%, 95% CI=0.3–28.9, P=0.045).

Next, calibration was assessed by composing calibration plots for 5-year and 10-year overall survival (Figure 1). Visually, both models resembled the ‘ideal line’ (x=y) reasonably well. Statistical analyses showed that 5-year overall survival did not significantly differ from the ideal line (standardised mortality ratio (SMR)=1.07, 95% CI=0.98–1.16, P=0.133). However, 10-year overall survival was significantly different from the perfect calibration (SMR=1.12, 95% CI=1.05–1.20, P=0.0004).

Figure 1
figure 1

Calibration plot: observed vs predicted overall survival. P-values were calculated using Poisson regression.

Full size image

ROCs and corresponding c-indices are presented in Figure 2A and B. The c-index of the predicted 5-year overall survival was 0.73, 95% CI=0.70–0.75, and 0.74, 95% CI=0.72–0.76, for 10-year overall survival.

Figure 2
figure 2

ROC for 5- and 10-year overall survival. (A) Receiver-operating curve (ROC) for 5-year overall survival. The calculated area under the curve (AUC) was 0.73, 95% CI=0.70–0.75. (B) ROC for 10-year overall survival. The calculated AUC was 0.74, 95% CI=0.72–0.76.

Full size image

Discussion

This study shows that the PREDICT tool can accurately predict 5-year overall survival in older patients with breast cancer. Ten-year overall survival estimates were slightly overestimated and less well calibrated than 5-year overall survival estimates, but still much more accurate than the predictions by Adjuvant! Online.

These findings are mostly in line with two previous validation studies that were performed in the United Kingdom, in which a good performance of the model was observed (Wishart et al, 2010, 2011). However, a recent validation study in younger patients (<40) showed that the model did not accurately predict 5-year overall survival, whereas 10-year overall survival was well calibrated (Maishman et al, 2015), again showing that prediction models cannot be easily extrapolated to other study populations that were not represented in the original study cohort. In contrast, we showed that 10-year overall survival was overestimated by PREDICT by 4.5%, which was much more accurate than 10-year overall survival as predicted by Adjuvant! Online (de Glas et al, 2014c). This superior performance of PREDICT in older patients can most likely be explained by the fact that its predictions were based on a large cohort that did include a large number of older patients, whereas Adjuvant! Online was developed in a data set of patients with a maximum age of 69 years (Ravdin et al, 2001; Wishart et al, 2010). In addition, the unclearly defined comorbidity classification that was used by Adjuvant! Online strongly influenced its predictions (de Glas et al, 2014c). Furthermore, PREDICT has incorporated additional biomarkers (HER2 and Ki67), which may improve its predictions (Wishart et al, 2012).

Still, PREDICT has several limitations that should be addressed before the tool can be implemented in older patients. The main limitation of PREDICT is that only overall survival estimates are reported (Eastern Cancer Registry and Information Centre and Cambridge University (2015)), despite the fact that the original model also predicted breast cancer mortality (Wishart et al, 2010). Breast cancer-specific mortality is an important end point because competing causes of mortality can drive breast cancer treatment decisions in older adults (Wildiers et al, 2013).

Furthermore, PREDICT did not accurately predict overall survival (both at 5 and 10 years) for the oldest patients (⩾85) and in patients with many comorbidities. These findings suggest that these patients were underrepresented in the original patient cohort in which the model was developed. In addition, PREDICT does not incorporate comorbidity status as a predictor, in contrast with Adjuvant! Online. Our data suggest that this is a limitation of the model, as overall survival was underestimated in patients with no comorbidity, whereas it was overestimated in patients with 3 or more comorbidities. This shows that the older breast cancer population is highly heterogeneous because of large differences in comorbidity status. It has been previously shown that comorbidity is a strong predictor for competing mortality (Kiderlen et al, 2014). Therefore, we suggest that future prediction models should incorporate a standardised comorbidity scale in their predictions. In addition, there are large differences between patients in terms of functional status, cognitive status, and physiological reserves.

There are several other prediction models besides PREDICT and Adjuvant! Online that can predict breast cancer outcomes. A recent systematic review identified 20 available prediction models of which Oncotype Dx is probably the most well-known (Engelhardt et al, 2013). Most of these models were based on genetic risk scores, and none of the prediction models besides Adjuvant! Online included comorbidity status (Engelhardt et al, 2013). Furthermore, none of the prediction models has been adequately validated in older patients (Engelhardt et al, 2013), despite the fact that ∼35–45% of patients with breast cancer are older than 65 years at diagnosis in Western countries, and this number is expected to increase in upcoming decades because of ageing of Western societies (DeSantis et al, 2013). Interestingly, the currently running ASTER 70s phase 3 trial (EudraCT 2011–004744-22) investigates the role of genomic grade index to guide indication of adjuvant chemotherapy in patients aged 70 years and older with luminal B breast cancer (Brain, 2014).

New prediction models or improvements of currently available prediction models that can be used in older patients are therefore required, especially as breast cancer mortality increases with age (van de Water et al, 2012). In contrast with younger patients, breast cancer mortality has not improved in older patients in the past decades (Bastiaannet et al, 2011; de Glas et al, 2014b). This may be explained by the fact that current guidelines are mostly based on studies that were performed in younger patients (Wildiers et al, 2007), or in highly selected older populations (van de Water et al, 2014). To improve outcome of older patients with breast cancer, it is essential to individualise treatment based on tumour characteristics, comorbidity status, but also functional status and the patients’ preference. We propose that future models aimed at older patients specifically should use PREDICT as a starting point. Ideally, a new prediction tool specifically designed for older patients with breast cancer should incorporate comorbidity and functional performance. Such a tool should not only predict overall survival, but also toxicity of treatment, quality of life, and functional decline. Both the International Society for Geriatric Oncology and the EORTC state that these so-called patient-related end points should be incorporated in clinical studies for older patients, in order to balance treatment efficacy with adverse events and toxicity in older patients (Wildiers et al, 2013). However, current randomised clinical breast cancer trials rarely incorporate these end points in their design (de Glas et al, 2014a). Large, prospective observational studies will therefore become increasingly important for the older breast cancer population, in order to study these end points and serve as the basis for new prediction tools that can aid in clinical decision making and individualised treatment.

Strengths and limitations

The main strength of this study is the use of a large population-based cohort, with nearly complete 10-year follow-up, that includes all consecutive patients in a geographically defined area in the Netherlands. In addition, we were able to perform central revision of pathological biomarkers in order to avoid intra-observer variation between hospitals. To our knowledge, this is the first study in which the validity of PREDICT was studied in a large cohort of older patients with breast cancer. This study also has its limitations. First, we were not able to perform FISH for HER2-status because of budget reasons, which is a limitation of the study as it may have resulted in overcall of HER2 positivity. Also, there was a large proportion of missing data concerning Her2 and Ki67 status, because of limited amounts of tumour material. Ki67 status remains controversial because of different cutoff points and reliability. We were restricted to the way the model uses Ki67-status (as a continuous variable). Furthermore, there was a large proportion of missing data on detection mode and Tumour Grade. However, the predictions of the model did not strongly differ in patients with missing data, which suggests that data were truly missing completely at random. In addition, despite these missing data, PREDICT still predicted 5-year overall survival reasonably well. Second, the effect of treatment can never be disentangled from the predictions of the model, as validation studies always appear in a retrospective setting, which means that patients already received treatment independent of the predictions of the model. Also, the percentage of 5% of patients receiving chemotherapy is fairly low, which may suggest that the validity of PREDICT may be different in populations where patients receive more chemotherapy. However, it must be noted that differences between observed and predicted survival outcomes did not strongly differ in several subgroups of treatment, which indicates that this has not strongly influenced our analyses. Third, we did not have information on the specific chemotherapy regimen that was used. However, only 5% of patients received chemotherapy, and therefore it is unlikely that this has influenced the results. Finally, it must be noted that comparing observed and predicted outcomes using One-Sampled t-test is strongly influenced by sample size. Therefore, large subgroups may result in significant differences while small subgroups may not. Hence, one should interpret the absolute differences rather than the corresponding P-values.

In conclusion, PREDICT can accurately predict 5-year overall survival in older patients with breast cancer. Ten-year predicted overall survival was, however, overestimated. New prediction tools or improvements of currently available tools that can be used in older patients are needed, taking additional comorbidity into account and focusing on relevant end points for older patients with breast cancer.