Lynch syndrome, or hereditary non-polyposis colorectal cancer (HNPCC), is the most common form of inherited colorectal cancer (CRC).1 2 Individuals with Lynch syndrome are at high risk for developing CRC, as well as tumours in the uterus, ovaries, stomach, small bowel, hepatobiliary system, pancreas, renal pelvis, ureter and bladder.3 Lynch syndrome is associated with underlying mutations in the DNA mismatch repair (MMR) system,4^–6 most commonly in the MLH1/MSH2 genes.7 Mutations in these MMR genes lead to microsatellite instability (MSI) of cancerous tissue8 and loss of expression of the corresponding protein, which can be tested by immunohistochemistry (IHC). Since intensive cancer screening and prophylactic surgery have been shown to reduce the incidence and mortality of CRC and endometrial cancer in individuals with Lynch syndrome,9 10 it is important to identify at risk individuals so that clinicians can provide appropriate recommendations for them and their family members.11

Initial strategies for identification of individuals at risk of Lynch syndrome were based on the fulfilment of the Amsterdam criteria12 13 and the Bethesda guidelines (BG).8 14 However, the Amsterdam I/II criteria have limited sensitivity for the identification of mutation carriers and therefore cannot be used as an exclusive means to identify Lynch syndrome patients.15 16 The BG were therefore developed as a broader screening tool to identify patients whose tumours should be tested for MSI8 and have been subsequently revised on several occasions,2 14 according to empirical data and expert opinion. The optimal method of identifying Lynch syndrome patients is still debated and in flux. Some investigators recommend performing IHC on all CRC specimens,17 while others have shown that fulfilment of the RBG followed by MMR deficiency screening is a more efficient strategy.18 However, some components of the BG remain complex and studies show that they may miss around 10% of mutation carriers.17 19 20 In addition, we have previously shown that medical oncologists refer only a fraction of patients who fulfill the BG for molecular evaluation.21 Finally, the available clinical criteria only provide a bivariate (yes/no) assessment of risk for Lynch syndrome and do not give a quantitative determination of the likelihood of being a mutation carrier. For all of these reasons, it is necessary to provide health care professionals with accurate and user-friendly tools to help identify individuals at risk of Lynch syndrome, as well as to help delineate the best strategy to pursue genetic evaluation.

In light of these issues, several risk prediction models have recently been developed to estimate the likelihood of a mutation in the MMR genes associated with Lynch syndrome (table 1). The PREMM_1,2 model was developed to predict mutations in MLH1/MSH2 in a cohort of individuals at moderate-to-high risk of Lynch syndrome.15 In contrast, the Barnetson prediction model was developed in a young onset, population based CRC cohort for identification of MLH1/MSH2/MSH6 mutation carriers.22 Both models are based on logistic regression analyses that account for personal and family history and are available as web based tools. The MMRpro model23 is a Bayesian tool that mimics an autosomal dominant pattern of inheritance based on parameters of prevalence of MLH1/MSH2/MSH6 mutations and on the predictive values of tumour molecular characteristics. It considers the personal and family history of colorectal and endometrial cancer and data on MSI and is also available online free of charge.

Each of the models was developed in a different population and has not yet been directly compared with the others. The aim of our study was to compare the performance characteristics of the PREMM_1,2 and Barnetson models with clinical criteria and universal molecular screening in a population based cohort of CRC patients. Our major outcomes of interest were the ability of each strategy to identify individuals who should undergo tumour evaluation for MMR deficiency, as well as predict MLH1/MSH2 mutation carriers.

PATIENTS AND METHODS

Patients

Between November 2000 and October 2001, all newly diagnosed CRC patients in 25 hospitals were enrolled in the EPICOLON study, whose goal was to establish the incidence of hereditary and familial CRC forms in Spain.18 24 Exclusion criteria were familial adenomatous polyposis or patient or family refusal to participate in the study. The study was approved by the institutional ethics committee of each participating hospital, and written informed consent was obtained from all patients.

Demographic, clinical, and tumour related characteristics of probands, as well as a detailed family history were obtained using a pre-established questionnaire. Cancer histories in pedigrees were traced backward and laterally at least up to second degree relatives. Age at cancer diagnosis, type and location of the neoplasm and current status were recorded for each affected family member. Data on numbers and current ages of unaffected family members was not systematically collected.

Tumour microsatellite instability analysis and immunostaining

Microsatellite instability testing and immunostaining for MLH1/MSH2 proteins were performed in all patients regardless of age, personal or family history, and tumour characteristics. Microsatellite status was assessed using the five-marker panel proposed by the National Cancer Institute, as described elsewhere.8 Tumours were classified as stable if none of the markers showed instability. Tumours with two or more unstable markers were classified as high level MSI (MSI-H) and tumours with one unstable marker were classified as low level MSI (MSI-L). Paraffin embedded sections were immunostained with antibodies against mismatch repair proteins (anti-MSH2, Oncogene Research Products, Boston, Massachusetts, USA; anti-MLH1, PharMingen, San Diego, California, USA).18 Tumour cells were judged to be negative for protein expression only if they lacked staining in a sample in which normal colonocytes and stroma cells were stained.

MLH1/MSH2 germline mutation analysis

Patients found to have tumours with MMR deficiency (demonstrated by either MSI-H and/or lack of protein expression) underwent MLH1/MSH2 germline genetic testing by both multiple ligation probe amplification (MLPA) analysis and sequencing, as previously described.18 In addition, all individuals with MMR proficient tumours and a PREMM_1,2 score ⩾20% (n = 18) were tested for germline mutations in MLH1/MSH2 and for immunohistochemistry of MSH6 and PMS2 proteins.25

Statistical analysis

Values to each variable included in the equation of the PREMM_1,2 and the Barnetson models were assigned according to the personal and family history of each proband in the cohort. Performance characteristics including sensitivity, specificity, positive predictive value and negative predictive value with 95% confidence intervals (CI) were calculated with respect to the presence of unambiguous MLH1/MSH2 germline mutations for the Amsterdam II criteria, RBG, molecular testing, PREMM_1,2 model and the Barnetson model. Because of the lack of available data on unaffected family members, we were not able to evaluate the performance of the MMRpro model in this cohort.

Sensitivity and specificity for the models depend on the cut-off used for the predicted risk of a mutation. Based on the original model development, we evaluated the following cut-off levels for the PREMM_1,2 model: ⩾5%, ⩾10%, ⩾20%, and ⩾40% and for the Barnetson model: ⩾0.5%, ⩾5%, ⩾20%, and ⩾45%.

Discrimination between patients with and without mutations was quantified by the area under the receiver operating characteristic curve (AUC) with 95% CI. Calibration was assessed by comparing the average predictions from each model to the observed prevalence of mutations. Calculations were performed using the SPSS software package 12.0 (SPSS Inc, Chicago, Illinois, USA).

RESULTS

Clinical characteristics of MLH1/MSH2 mutation carriers

During the study period, 1222 patients with pathologically confirmed colorectal adenocarcinoma were diagnosed and included in the EPICOLON project. Demographic, clinical, and tumour related characteristics of patients included in the study have previously been described.18 Overall, 91 patients (7%) had a tumour demonstrating MMR deficiency, defined as either MSI-H or loss of MLH1/MSH2 protein expression. Germline genetic testing identified eight (0.7%) deleterious mutations in MSH2 (five cases) or MLH1 (three cases) genes, and three variants of unknown significance (two in MSH2 and one in MLH1). Neither MLH1/MSH2 germline mutations nor loss of expression of MSH6/PMS2 proteins were found in the group of individuals with MMR proficient tumours and a PREMM_1,2 score ⩾20%.

Clinical criteria, PREMM_1,2 and Barnetson scores

Distribution of individuals according to fulfilment of clinical criteria and to specified PREMM_1,2 and Barnetson model scores are reported in table 2. Fulfilment of any of the RBG, identification of MMR deficiency, or a PREMM_1,2 score ⩾5% identified all MLH1/MSH2 mutation carriers. A Barnetson score ⩾0.5% missed one MSH2 mutation carrier with a 0.3% predicted probability.

Overall, prevalence of mutation carriers according to risk groups by the PREMM_1,2 and the Barnetson models was lower than the predicted score of identifying a mutation in each pre-specified risk group (table 3). For the PREMM_1,2 model, distribution of mutation carriers was: <5%: 0; 5–9%: 2 (1%); 10–19%: 1 (1%); 20–39%: 3 (13%); ⩾40%: 2 (22%). For the Barnetson model, distribution of mutation carriers was: <0.5%: 1 (0.1%); 0.5–0.9%: 0; 1–4%: 1 (0.7%); 5–14%: 1 (2.5%); 15–19%: 1 (12.5%); 20–24%: 0; 25–34%: 0; 35–44%: 1 (14%); ⩾45%: 3 (21%).

Performance of predictive models

Performance characteristics of the PREMM_1,2 and the Barnetson models for the identification of MLH/MSH2 gene carriers were analysed at the different cut-offs as presented in the original model development (table 4). The AUCs of the PREMM_1,2 and the Barnetson models were 0.93 (95% CI 0.86 to 0.99) and 0.92 (95% CI 0.83 to 1.01), respectively (fig 1). The average prediction of mutation carriers was 5% and 1% by the PREMM_1,2 and Barnetson models, respectively, compared to an observed prevalence of 0.7%.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1 Sensitivity and 1-specificity receiver operating characteristic curves. The two lines illustrate the sensitivity and 1-specificity of the PREMM_1,2 model (—) and the Barnetson model (---). The square represents the sensitivity and 1-specificity value of the Amsterdam criteria and the triangle represents the sensitivity and 1-specificity value for the fulfilment of the revised Bethesda guidelines at single cutoff points, as these criteria are dichotomous.

The PREMM_1,2 model had 100% (95% CI 71% to 100%) sensitivity and 68% (95% CI 65% to 71%) specificity, when a 5% cut-off was used. Positive and negative predictive values were 2% (95% CI 1% to 4%) and 100% (95% CI 99% to 100%), respectively. No mutation carriers had a PREMM_1,2 score <5%. Sensitivity and specificity of increasing cut-offs of the PREMM_1,2 model were as follows: ⩾10%: 75% and 90%; ⩾20%: 62% and 98%; and ⩾40%: 25% and 99%, respectively.

The lowest cut-off analysed for the Barnetson model in the original study was 0.5%, which had 87% (95% CI 51% to 99%) sensitivity, 72% (95% CI 69% to 74%) specificity, 2% (95% CI 1% to 4%) predictive value, and 100% (95% CI 99% to 100%) negative predictive value in our cohort. Sensitivity and specificity of increasing cut-offs of the Barnetson model were: ⩾1%: 87% and 83%; ⩾5%: 75% and 94%; ⩾20%: 50% and 98%; ⩾45%: 37% and 99%.

Sensitivity of the RBG for identification of MLH1/MSH2 mutation carriers was 100% (95% CI 71% to 100%), with 77% (95% CI 74% to 79%) specificity and a PPV of 3% (95% CI 1% to 5%). The Amsterdam II criteria missed four mutation carriers, with 50% (95% CI 22% to 78%) sensitivity, 98% (95% CI 97% to 99%) specificity and 18% PPV (95% CI 7% to 39%).

Performing MMR molecular screening in all tumours identified 91 patients with an MMR deficiency. This strategy had a sensitivity and specificity of 100% (95% CI 71% to 100%) and 94% (95% CI 92% to 95%), respectively, with a PPV of 10% (95% CI 5% to 18%).

Performance of predictive models combined with MMR deficiency

Combination of predictive scores with MMR deficiency increased specificity and PPV for each established cut-off (table 4). Consequently, a PREMM_1,2 score of ⩾5% with MMR deficiency had 97% (95% CI 96% to 98%) specificity and 21% (95% CI 10% to 36%) PPV. For the Barnetson cut off of ⩾0.5% combined with MMR deficiency, specificity increased to 98% (95% CI 97% to 99%) and PPV to 21% (95% CI 10% to 38%). Similarly, fulfilment of the RBG combined with MMR deficiency had 98% (95% CI 97% to 98%) specificity and 22% (95% CI 11% to 38%) PPV. Overall, PPV for each cut-off of the predictive models were substantially higher in combination with the presence of MMR deficiency in the tumour. When combined with MMR deficiency, PPV associated with higher model cutoffs were greater than the combination of fulfilment of the RBG and MMR deficiency (PPV of 29% for PREMM_1,2 ⩾10%, 35% for Barnetson ⩾5% and 22% for the RBG).

Characteristics of CRC patients with MLH1/MSH2 mutations

Detailed personal and family histories of the eight mutation carriers are described in table 5. All mutation carriers but one were female and their median age at diagnosis was 64 years (28–81 years). Among mutation carriers, the predicted probability of identifying a mutation according to the PREMM_1,2 and the Barnetson models ranged from 5–89%, and from 0.3–84%, respectively. All mutation carriers fulfilled at least one of the RBG and showed MMR deficiency in their tumour.

DISCUSSION

We analysed the performance characteristics of the PREMM_1,2 and Barnetson models in a population based cohort of CRC patients for identifying MLH1/MSH2 mutation carriers and compared them with clinical criteria and universal molecular screening for MMR deficiency with IHC and MSI. Because of the lack of available data on unaffected family members, we were not able to evaluate the performance of the MMRpro model in this cohort. The study shows that a ⩾5% cut off of the PREMM_1,2 model identified all MLH1/MSH2 mutation carriers. The lowest threshold (0.5%) of the Barnetson model missed one mutation carrier. Tumour analysis for MMR deficiency improved the performance of the two predictive models at different cut-offs and yielded a similar likelihood of finding a mutation than the combination of fulfilling RBG and MMR deficiency at lower cutoffs and improved upon the RBG at higher thresholds.

Identification of appropriate individuals for genetic testing for Lynch syndrome is challenging due to insufficient specificity of current clinical criteria, the difficulty of implementing MMR deficiency screening for all CRC patients, the low prevalence of mutation carriers, and the high cost of genetic testing if performed in unselected cases. Selecting patients based on clinical criteria, presence of MMR deficiency, or a combination are all strategies that have been recommended by experts before ordering germline genetic testing.17 18 26 According to our analysis, if one were to use the RBG as an initial selection strategy, 287 (23%) of CRC patients in this cohort would have been selected for further MMR deficiency testing. All MLH1/MSH2 mutation carriers detected in this cohort would have been found. If a previous systematic quantitative approach were used such as ⩾5% score of the PREMM_1,2 model, 396 (32%) individuals would be selected to undergo molecular screening and no mutation carriers would have been missed. If the lowest cut-off originally analysed by the authors of the Barnetson model (⩾0.5%) was used, 349 individuals (29%) would have undergone further molecular testing, but one MSH2 mutation carrier would have been missed with a 0.3% predicted probability. Use solely of the Amsterdam II criteria would have missed four mutation carriers (50% sensitivity), although there would have been higher specificity and PPV (98% and 18%, respectively). Universal testing for MMR deficiency would require molecular screening of all 1222 cases to identify eight MLH1/MSH2 mutation carriers.

It is important to emphasise that in the EPICOLON cohort, genetic testing for MLH1/MSH2 was only performed in patients with tumour MMR deficiency, and theoretically, some mutation carriers whose tumours did not exhibit MMR deficiency may have been missed. In an effort to explore this possibility we tested all individuals with a PREMM_1,2 score ⩾20% (n = 18) for germline mutations in MLH1/MSH2 and for immunohistochemistry of MSH6 and PMS2 proteins. Neither mutations nor loss of expression of the corresponding proteins were found. Ideally, the study that would most precisely answer the question of the true performance characteristics of each of the strategies for identification of mutation carriers would require comprehensive analysis of all four mismatch repair genes in a large population based cohort without tumour molecular prescreening for MMR deficiency; such a study has not thus far been undertaken.

Despite the above caveat, the EPICOLON cohort remains a useful population to externally validate the newly developed models. First, EPICOLON is entirely different from the moderate to high risk clinic population in which the PREMM_1,2 model was developed. Although it is also a population based cohort, because we included all ages of CRC, it is not as selected as the young onset series in which the Barnetson model was developed. Our analysis demonstrates that with sensitivities of 100%, the RBG or a PREMM_1,2 cutoff ⩾5% are both excellent options as tools for the initial risk assessment of CRC patients that may be used by a wide variety of clinicians in practice, including primary care physicians, oncologists, gastroenterologists or gynaecologists in deciding who should be referred for further genetic counselling and/or molecular evaluation. The Barnetson model also has excellent sensitivity but may miss a small proportion of mutation carriers if used as the sole method for this purpose. A formal cost effectiveness analysis would be extremely helpful to compare the costs associated with this surely more expensive and labour intensive strategy of universal IHC and/or MSI to identify mutation carriers than a more targeted approach. Of all CRC patients, 68–77% of individuals would not need any further molecular evaluation, and no mutations would have been missed, using the 5% PREMM_1,2 model threshold or the RBG, respectively.

The second question to address is which approach makes the best predictions of the likelihood of being a mutation carrier. The PPVs of the RBG, PREMM_1,2 ⩾5% and Barnetson ⩾0.5% were similar, at 3%, 2%, and 2%, respectively. While the two predictive models would initially select a slightly higher proportion of CRC cases than the RBG, they offer a quantitative risk assessment of finding a mutation and can be used to further refine the estimate of predicted probability, particularly in combination with MSI and IHC results. Performing molecular screening of MMR deficiency with MSI and IHC in all CRC was deemed to be sensitive and specific (100% and 94%), but had a PPV of only 10%. A combined finding of MMR deficiency and fulfilment of the RBG, PREMM_1,2 ⩾5% or Barnetson>0.5% all had similar PPV of 21–22%. At higher cutoffs, however, the models had improved PPVs compared to the RBG.

It is important to consider the limitations of the current validation of the predictive models. In this population based CRC cohort with a low overall mutation rate, both models had higher overall predictions compared to the actual observed rate, with PREMM_1,2 demonstrating poorer calibration. How the models function in other population based cohorts with higher mutation rates27 and in higher risk clinical populations will be important to evaluate in future studies. It is also important to note that in their current forms, both the RBG and the Barnetson models are only applicable to patients with CRC, whereas PREMM_1,2 may also be used for risk assessment and mutation prediction in unaffected individuals and those with extracolonic tumours. Still, how the PREMM_1,2 model functions in these other patient groups needs to be further validated. The total number of mutation carriers in the EPICOLON cohort was small and germline testing for MSH6 was not performed routinely. Although genetic testing was initially only performed in those patients with tumour MMR deficiency, it was subsequently performed in those cases with a PREMM_1,2 score ⩾20% without MMR deficiency.25 Neither mutation carriers nor cases with loss of expression of MSH6 or PMS2 were identified in this group. The low prevalence of mutations may also explain why the two models did not show a good calibration in this study cohort. Finally, comparison of the PREMM_1,2 and the Barnetson model with the MMRpro model23 could not be performed due to lack of data on unaffected family members.

In conclusion, the PREMM_1,2 and the Barnetson models demonstrated similar performance to the RBG for identification of MLH1/MSH2 mutation carriers in a population based cohort of CRC patients and allow quantitative risk assessment of the likelihood of finding an MMR gene mutation. Combination of the predictive models at specific cut-offs with tumour MMR deficiency selects those individuals more likely to carry an MLH1/MSH2 mutation and offers a new clinical strategy to be considered for identification of individuals at risk of Lynch syndrome. Problems in calibration need to be addressed in different settings to warrant transportability of specific cut-offs into clinical decision making.