A systematic review of the clinical validity and clinical utility of DNA testing for hereditary haemochromatosis type 1 in at-risk populations
- 1Southampton Health Technology Assessments Centre, University of Southampton, Southampton, UK
- 2University of Southampton Medical School, Southampton, UK
- 3Institute of Hepatology, University College London, London, UK
- 4Guys and St Thomas’ NHS Foundation Trust, London, UK
- Ms J Bryant, Southampton Health Technology Assessments Centre, Mailpoint 728, Boldrewood, University of Southampton, Southampton, SO16 7PX, UK;
- Received 23 October 2007
- Revised 16 January 2008
- Accepted 18 February 2008
- Published Online First 29 February 2008
Objective: To evaluate the clinical validity and clinical utility of DNA testing in people suspected of having hereditary haemochromatosis and in family members of those diagnosed with the disorder.
Design: A systematic review.
Methods: 15 electronic databases were searched up to April 2007. For assessment of the clinical validity of genotyping for the C282Y mutation in the diagnosis of hereditary haemochromatosis, studies were included if they reported the use of DNA tests in Caucasians of northern European origin with iron overload suggestive of haemochromatosis compared with a control population, and reported or allowed calculation of sensitivity and specificity. For clinical utility, studies were included if participants were Caucasians with iron overload suggestive of haemochromatosis or were relatives of suspected cases, if the study compared a diagnostic strategy incorporating DNA testing with one not incorporating DNA testing, and if the study reported patient-based outcomes or some measure of cost effectiveness.
Results: 11 studies that could be used to evaluate clinical validity of genotyping for the C282Y mutation in the diagnosis of hereditary haemochromatosis were identified. Clinical sensitivity of C282Y homozygosity for hereditary haemochromatosis ranged from 28.4% to 100%; when considering studies that used strict criteria to classify hereditary haemochromatosis clinical sensitivity ranged from 91.3% to 92.4%. No clinical effectiveness studies were found. Two cost effectiveness studies were identified, both of which suggested that gene testing may be cost effective.
Conclusion: DNA testing for hereditary haemochromatosis in at-risk populations has clinical validity and may have clinical utility. The review highlights the limitations of the literature and the methodological difficulties associated with evaluating this genetic test.
Hereditary haemochromatosis (HHC) type 1 (OMIM 235200) results from a genetic disorder of iron metabolism that leads to excessive absorption of iron and a progressive abnormal deposition of iron in vital organs. Initial symptoms are non-specific, such as fatigue, and may often be ignored or misdiagnosed. Patients may present with liver disease, diabetes, arthritis, impotence and heart disease when excess iron has led to organ damage resulting in morbidity in middle life. Untreated haemochromatosis leads to premature death, usually due to liver complications. Treatment by removing excess iron with phlebotomy is effective and, if started before irreversible end organ damage, restores normal life expectancy.1
HHC is one of the most common genetic diseases in people of northern European descent, particularly those of Celtic origin.2 Most cases result from a single mutation (C282Y) in the HFE gene, described in 1996.3 A second mutation (H63D) has also been identified but is not usually associated with symptomatic disease, although some patients are compound heterozygotes (C282Y/H63D). The prevalence of the C282Y homozygous mutation ranges from 0.68% to 1.24% in the general UK population and from 72.8% to 91.3% in people with haemochromatosis.4 However, the phenotypic expression of the mutation, that is the clinical appearance of iron overload, is variable, and not all people homozygous for the C282Y mutation will develop symptoms of haemochromatosis. This low clinical penetrance appears to be due to a complex interplay of genetic status and other factors such as age, sex, environmental influences and the presence of other diseases. The clinical prevalence of the disease is not well established.5
Diagnostic strategies for haemochromatosis include a combination of biochemical tests followed by genetic testing. The initial diagnostic tests used where there is clinical suspicion are transferrin saturation and serum ferritin concentration. Genetic testing can then confirm the hereditary nature of the primary iron overload and venesection can be started.6 Liver biopsy has been used in the past to confirm diagnosis, but owing to its cost and risk, is now limited to determining the degree of hepatic fibrosis and cirrhosis during subsequent management. Because HHC is an autosomal recessive disorder, children and siblings of those affected are at increased risk for inheriting susceptibility to the disease, with probabilities of at least 1 in 20 and at least 1 in 4 respectively. The purpose of gene testing family members is to detect individuals at risk who would benefit from monitoring and treatment, and to exclude the disease and therefore avoid unnecessary interventions in those not at risk. However, it is not clear which are the most appropriate and cost effective diagnostic strategies for using genetic testing.
Owing to the uncertainty about the use of DNA testing for detecting HHC in people suspected of having the disorder on the basis of clinical presentation and disturbed iron parameters, and in family members of those diagnosed with HHC, we were commissioned by the UK National Institute for Health Research (NIHR) Health Technology Assessment Programme to assess the evidence. We conducted a systematic review of DNA testing following methods developed by the Office of Genomics and Disease Prevention (Centers for Disease Control, USA) to evaluate DNA-based genetic testing,7 which is based on original methods by Wald and Cuckle.8 The model recommends that the evaluation of genetic tests should incorporate components of analytical validity, clinical validity, clinical utility and associated ethical and psychosocial implications of DNA tests (the ACCE model). This paper summarises our findings on clinical validity and clinical utility (clinical and cost effectiveness) and discusses some of the methodological issues relating to the assessment of a genetic test in the case of HHC.4
In total, 15 electronic databases (including MEDLINE, the Cochrane Library, HuGENet and Medion) were searched up to April 2007. Additional studies were identified through searching bibliographies of related publications and through contact with experts. Further details are available elsewhere.4
This study evaluated the clinical validity and clinical utility of diagnostic strategies incorporating DNA testing for HHC by seeking the highest level of evidence available. Clinical validity, defined as the ability of the test to detect or predict the phenotype (disorder) of interest, involves establishing the probability that the test will be positive in people with clinical HHC (sensitivity) and the probability that the test will be negative in people without the disease (specificity). In this study, sensitivity refers to the proportion of individuals who have a positive test result for C282Y homozygosity from a group of those who have, or who may be destined to develop, the primary iron overload phenotype. For assessment of validity, studies were included if they reported the use of DNA tests in Caucasians of northern European origin with iron overload suggestive of HHC compared with a control population, and reported or allowed the calculation of sensitivity and specificity.
Clinical utility is defined as the likelihood that the test will lead to improved patient outcome, and incorporates assessment of the risk and benefits of genetic testing (clinical effectiveness), as well as economic evaluation (cost effectiveness). Utility studies were included if they reported the use of DNA tests in Caucasians with iron overload suggestive of HHC (or relatives of suspected cases) compared with any case-identification strategy not involving DNA testing. Clinical effectiveness studies had to report patient based outcomes (such as morbidity or mortality) and cost effectiveness studies had to report any cost effectiveness measure.
Inclusion criteria, decisions about quality criteria, and data extraction were applied independently by two reviewers, with any differences in opinion resolved through discussion. Studies were combined through narrative synthesis with full tabulation of included studies. Quality assessment of clinical validity studies was undertaken using relevant criteria from Quality Assessment of Diagnostic Accuracy Studies (QUADAS)9 combined with questions from criteria developed by Spitzer et al,10 modified to be of relevance for assessing a genetic test. The quality of economic evaluations was assessed using standard checklists.11 12
Quantity and quality of research available
Criteria used to define haemochromatosis varied between studies. Six studies13 14 16 19 21 23 used a definition that is likely to classify HHC incorrectly in terms of raised biochemical parameters (transferrin saturation and serum ferritin), liver biopsy, hepatic iron index or quantitative phlebotomy, whereas the remainder are unclear from the description given. In all studies, it is not clear whether bias has been avoided in the sampling of participants with haemochromatosis, as the methods used have not been described. In five studies, patients with haemochromatosis were reported to be unrelated to one another,13 14 16 19 21 in one study it was unclear whether patients included related individuals,20 and in one study patients with haemochromatosis were reported to include some related individuals.23 Related individuals will share genetic and environmental factors, which could lead to bias.
Control subjects were drawn from different sources in the different studies and may not be appropriate and free from selection bias in terms of representing the relevant population and being phenotypically negative. In four studies the control group was drawn from the general population.13 16 17 22 Three studies recruited the controls from among hospital employees,14 20 or hospital employees, students and their relatives.15 Three studies18 19 21 recruited from particular groups of healthy persons, including blood donors21 and a bone marrow registry.18 The remaining study recruited their control group from several other patient groups.23 Sampling of controls was described as random in four studies,13 16 17 20 with the sampling methods being unclear in five studies,14 15 21–23 and the remaining studies involved volunteers.18 19 Only two studies reported that the individuals in the control group were unrelated to one another,19 21 and in one study the inclusion criteria allowed for related individuals to be included.15
Assessment of clinical validity of DNA testing
Estimates from the included studies of the sensitivity and specificity of the genetic mutation C282Y for the iron overload phenotype in northern European Caucasians are summarised in table 2.
Sensitivity ranges from 28.4% to 100%, which may be accounted for by the variable definitions of primary iron overload used in the included studies. When considering studies that clearly and appropriately define haemochromatosis and are generalisable,13 14 16 19–21 23 sensitivity ranges from 72.2% to 100%. Limiting studies further to those that excluded other causes of iron overload and in which patients were unrelated, the range is 72.2% to 92.4%.13 14 16 21 The range is 91.3% to 92.4% when the small study in southern Germany, which may be on the north–south European divide, is excluded.14
From two studies the estimated sensitivity is low at 28.4%15 and 21.1%.22 This may be explained in one study15 by the fact that genetic testing was performed retrospectively in patients with clinical suspicion of iron overload based only on biochemical measurements, some of which were missing. The other study included patients with excessive alcohol intake and diabetes, where disturbed iron parameters may have been due to cirrhosis and insulin resistance respectively, and not to genetic iron overload.22
Clinical sensitivity was 100% in one small study of 18 patients, which included related subjects.23
Specificity ranges from 98.8% to 100% in the included studies. In two of the five studies that gave an estimate <100%, the single individual homozygous for the C282Y mutation in the control group showed signs of evidence of iron overload.15 No details are given about the homozygous individuals in the control groups of the other three studies, so it is not known if they showed signs of irregular iron loading or whether they would be likely to do so in the future.17 18 23
Quantity and quality of research available
No clinical effectiveness studies meeting the inclusion criteria for the review were identified. Two cost effectiveness studies were identified (table 3).
One study was a cost minimalisation model, which investigated the likely cost of genotyping spouses and whether this would reduce the number of investigations of children.24 The other study was a cost–utility model, which estimated the cost effectiveness of screening for HHC in family members using genotypic tests compared with phenotypic tests and no screening.25 Both studies were of reasonable quality when assessed against standard criteria such as having a clearly defined study question with clearly described and correct comparators, and an appropriate patient group of interest. However, both studies were conducted in North America and it is unclear how these studies relate to the UK NHS. One study used a lifetime horizon and estimated incremental cost effectiveness with appropriate discounting rates and presented sensitivity analyses of the key parameters.25 The other study24 did not consider long-term cost and consequences. It is unclear whether the studies valued the costs and consequences appropriately. The source of costs or how costs are derived are not reported24 or are taken from earlier studies and have not been adjusted for time.25 The conclusions in both the studies appear credible from the results presented.
Assessment of clinical utility of DNA testing (cost effectiveness)
In the study24 that estimated the costs of genotyping spouses of homozygotes in order to reduce the number of investigations of children, costs were estimated for genotyping all children of homozygotes (phenotypic strategy), compared with genotyping spouses and then genotyping children only if the spouse was a heterozygote or homozygote (spousal strategy).
In total, 291 children of homozygotes were investigated using transferrin saturation, serum ferritin and genetic tests. Costs incurred in the phenotypic strategy were CDN $58 200. Of these children, 13 were found to be homozygotes, out of 116 families with the C282Y mutation. In the spousal strategy, 116 spouses were genotyped with subsequent investigation of 22 children, at a total cost of CDN $35 600. Therefore, genotyping of the spouses reduced the number of investigations in children from 291 to 22, with a cost saving of 39%. Thus, genotyping the spouse was found to be the most cost efficient strategy in family testing because it leads to more selective investigation of children for the haemochromatosis gene.
Genotyping siblings and children
In the study25 that developed a decision tree model, three different genetic testing strategies were compared with no testing and testing using iron studies among siblings and children. The first genetic strategy involved gene testing the proband (affected patient) followed by testing the spouse of a homozygous proband and then gene testing of the children if the spouse was heterozygous. The second strategy involved gene testing of the proband followed by testing relatives of a homozygous proband. The third strategy used direct gene testing of relatives before the proband.
Strategies using gene testing were less costly than serum iron studies. Compared with no screening, gene testing of the proband followed by testing a child was the least expensive and most cost effective strategy for one child (incremental cost effectiveness ratio $508 per life year saved). For screening ⩾2 children, gene testing the spouse if the proband was homozygous was the most cost effective strategy (incremental cost effectiveness ratio $3665 per life-year saved). In siblings, all strategies cost less and yielded greater benefits compared with no screening. For all sensitivity analyses varying the prevalence of mutations and cost of the genetic tests, gene testing remained cost effective. The study authors concluded that gene testing for the C282Y mutation is a cost effective method of screening relatives of patients with HHC.
This review, which was guided by an expert advisory panel, systematically considered the evidence on the clinical validity and clinical utility of using DNA testing for HHC in at-risk populations. The review sought to obtain the highest level of evidence, but the evidence meeting the inclusion criteria of the review was limited in quantity and quality; only two cost effectiveness studies were identified and no clinical effectiveness studies were found. Clinical validity studies were observational studies with controls.
Three main reasons contributed to the difficulty of evaluating the validity and utility of DNA testing for HHC in at-risk populations. The first reason was the nature of the literature. There is a large published literature concerning HHC; however, it relates mainly to aspects of population screening and frequencies of the C282Y mutation in different populations with associated levels of iron parameters. In addition, it is not always obvious from titles and abstracts what methods have been used, and some studies that appear to be relevant to the use of DNA testing do not report expected outcomes in an appropriate way. Owing to this wide-ranging literature it was difficult to define a priori inclusion criteria to deliver relevant studies while retaining the focus of the review and allowing manageable synthesis of evidence. Consequently, different inclusion criteria were developed for clinical validity and clinical utility in terms of the study population. For clinical utility, studies considering any Caucasian populations were included, whereas for clinical validity, studies were limited to those in north European populations. This emphasis on north European studies could be criticised for being too restrictive but the decision to do this was taken for pragmatic reasons and to ensure that results were relevant to the UK. For clinical utility of a diagnostic test the best evidence is a randomised controlled trial (RCT), with patients randomly assigned to alternative diagnostic strategies with clinical effectiveness or cost effectiveness reported.26 In the absence of RCTs, it was intended to assess clinical effectiveness of DNA testing by using the highest level of evidence available, which considered suspected cases of HHC (or relatives of cases) and comparative testing strategies, and reported patient-based outcomes. No such studies with appropriate designs and outcomes were found. Some studies that purported to be clinical utility studies were problematic in that they reported gene frequency without any clinical or cost effectiveness measure. In addition, some studies that initially appeared to be comparing diagnostic strategies in fact compared different groups of patients using biochemical and DNA testing algorithms, rather than patients with suspected HHC tested by different diagnostic algorithms with or without DNA tests to assess the utility of DNA testing. Further prospective long-term follow-up studies of DNA testing for HHC would be useful.
The second major difficulty concerned the general problems inherent to the assessment of a genetic test. Traditional diagnostic assessment studies that estimate the sensitivity and specificity of a test require the new test to be compared with a reference standard (gold standard). However, this does not apply in the case of genetic tests, where the gold standard entails gene sequencing to detect mutations.27 Potential alternative gold standards in the case of haemochromatosis are no longer used for diagnostic purposes, such as liver biopsy, which is used mostly for prognosis, and haplotyping, which has been superseded since the discovery of the HFE gene. Thus, traditional diagnostic test assessment studies are inappropriate and not available in this case.
The ideal way to assess clinical validity of DNA testing for haemochromatosis would be to follow a large group of individuals through to expression of phenotypic disease and perform genetic testing. No such population-based cohort studies are available, and such studies could be considered unethical and are therefore unlikely. Therefore, clinical validity has been assessed here by considering studies that identify a group of individuals who have the primary iron-overload phenotype and then determining the proportion who are C282Y homozygous and comparing them with a control group. The controls are individuals that do not have the phenotype of interest according to the disease case definition.28 It is only through the use of such controls that an assessment of specificity can be made. The use of such studies has limitations in that individuals in the control group who test positive may yet go on to develop the phenotypic disease. Although this risk is likely to be small, due to the prevalence and penetrance of HHC, it must be acknowledged that calculation of the specificity of the genetic mutation (C282Y) for HHC from such studies may not be accurate. Specificity of 100% for homozygous C282Y mutation in patients with phenotypic expression has been reported29 but specificity may be <100% as some homozygotes in the control group may never exhibit symptoms.
The third major difficulty associated with evaluating DNA testing for HHC is specific to the condition itself, which is not fully penetrant and for which there is no absolute definition in terms of diagnostic criteria. The disease may sometimes be defined phenotypically in terms of biochemical parameters and at other times genotypically by reference to the mutations that give rise to the disease. It has been suggested that the definition of a genetic disease should require both the clinical features and the presence of the mutation; others think that the presence of either the clinical features or the mutation suffices for a definitive diagnosis. In assessing genetic testing in the case of HHC the definition should be on biochemical or clinical features but not with reference to genotype.28 The definition of cases may vary from signs and symptoms to confirmed iron overload with clinical features, and this can affect the characteristics of the genetic test and the reported clinical validity. Clinical sensitivity from the included studies ranged from 28% to 100%. It can be seen from these results that the reliability of estimating clinical sensitivity of C282Y homozygosity for HHC is particularly susceptible to the definition of the clinical phenotype used in the studies. When the definition is more rigorous and strict criteria for phenotypic expression are used, the clinical sensitivity increases. Clinical validity may also be influenced by other genes involved in iron metabolism; it also varies with the clinical approach, which can be targeted or not according to different symptoms.30
Evidence on the cost effectiveness of DNA testing for HHC is also limited in quantity with one cost-minimisation study and one cost–utility study meeting the inclusion criteria for the review. Although the study types were appropriate and methodological quality was generally reasonable, there are concerns about some of the methods used for deriving costs and whether the costs and consequences have been valued appropriately. However, the conclusions in both studies appear credible from the results presented—that is, gene testing is cost effective for testing relatives of patients with haemochromatosis. These results agree with our economic evaluation, which found that diagnostic strategies incorporating DNA testing are cost saving in terms of cost per case detected, both for people suspected of having HHC and offspring of people diagnosed with HHC.4
In summary, this systematic review was undertaken employing standard a priori methods and recommendations for evaluating a genetic test. Although the methodology was challenging and the evidence limited, it is important to conduct assessment of genetic tests to inform practice. Results suggest that DNA testing for HHC in at-risk populations has validity (with sensitivity >90% and specificity nearly 100%) and may have utility. Genetic testing is therefore likely to be of value in clinical practice. Although it is thought that those expert and interested in the management of the condition already follow a strategy that uses DNA testing in conjunction with testing iron parameters, this practice may be fragmentary and ad hoc, due to the non-specific nature of early presenting symptoms. The development and dissemination of guidelines to physicians in both primary and secondary care, and centralisation of test provision in expert laboratories, in line with the Carter report,31 would assist in making genetic testing available to those people who may benefit from it.
This project was funded by the NHS R&D Health Technology Assessment Programme (project number 05/07/04). The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the Department of Health. We are very grateful to the advisory panel which provided expert advice and comments on the protocol and/or draft of the systematic review. We would also like to thank staff at the Wessex Institute for Health Research and Development.
Funding: Financial Support was received from the NHS R&D Health Technology Assessment Programme
Competing interests: None declared.