Materials and methods

SAMPLES

Samples were selected and anonymised for blinded analysis by Myriad Genetic Laboratories. All samples had been analysed following the routine procedures used for diagnostic testing. DNA was first extracted by Proteinase K digestion from peripheral blood mononuclear cells isolated from each sample and then column purified (QIAGEN Inc, Chatsworth, CA, USA). Aliquots of DNA were amplified by polymerase chain reaction (PCR) using 35 M13 forward and reverse tagged primer pairs to cover coding exons 2-24 of BRCA1(although exon 4, like exon 1, is non-coding and no variants in either were included in the subsequent inter-laboratory comparison). The amplified products were each directly sequenced in forward and reverse directions using fluorescent dye labelled sequencing primers. Chromatographic tracings of each amplicon were analysed by a proprietary computer based review followed by visual inspection and confirmation. Genetic variants were detected by comparison with a consensus wild type sequence constructed forBRCA1. As part of routine analytic processing, all potential genetic variants had been independently confirmed by repeated PCR amplification of the indicated gene region(s) and sequence determination as above.

For the purposes of this study, “mutations” were defined as protein truncating and missense mutations located within exons 2-3 and 5-24 ofBRCA1 as well as intronic sequence alterations occurring no more than 20 bp proximal or 10 bp distal to the ends of these exons. Non-truncating genetic variants were excluded from consideration for the purposes of this study if they had been observed at an allele frequency of greater than 1% of a suitable control population with no evidence for significantly higher frequency in cases than controls, or if published data indicated absence of substantial clinical significance, or if they neither altered the amino acid sequence nor were predicted to affect exon splicing significantly.

The sample set consisted of 65 samples, including 50 that contained a total of 58 mutations of established or potential clinical significance and 15 additional samples in which no mutation had been identified through sequence analysis as above. The positive samples included 20 frameshift mutations (17 deletions, three insertions), 18 nonsense mutations, 15 missense mutations, and five mutations occurring in the non-coding regions adjacent to the beginning or end of the exon (table1). All mutations and genetic variants were named according to a designated convention,6 numbering the nucleotides from the first transcribed base of BRCA1 GenBank entry U14680.

Ten μg of genomic DNA that remained after the completion of routine analysis by Myriad Genetic Laboratories were aliquotted to the participating laboratories per their stated requirements as follows: 4 μg each for SSCP and CSGE and 1 μg each for TDGS and DHPLC. A letter of agreement was provided to each participating laboratory that delineated the principles of the exercise, including the criteria by which sensitivity and specificity would be derived. Differences between laboratories limited to the names or cDNA locations of the mutations were not considered discrepancies for the purpose of this comparison. A Myriad Genetic Laboratories representative (TSF) provided the number and identity of the mutations to a designated representative of the Breast Cancer Information Core (BIC) (LCB) to whom all laboratories subsequently submitted their results. Only when each of the laboratories had completed and submitted final results to the BIC representative were the authors provided with each other's results.

SSCP

All coding regions and exon-intron boundaries ofBRCA1 were amplified from genomic DNA by PCR using either previously described sets of primers7 8 or primer pairs designed in the Ostrander laboratory. PCR was carried out in 12.5 μl volumes with 25 ng genomic DNA as template, 1 × PCR buffer, 1.5 mmol/l magnesium, 0.048 mmol/l each dATP, dTTP, and dGTP, 0.0048 mmol/l dCTP, 0.2 U Taq, (Bioline, USA), and 0.004 mCi [α-P32]) dCTP (Amersham, USA). Initial denaturation was done at 95°C for one minute followed by 35 cycles of 30 seconds at 94°C, 15 seconds at an appropriate annealing temperature, 15 seconds at 72°C, followed by final elongation at 74°C for three minutes. Samples were then diluted 1:3 in formamide buffer (98% formamide, 10 mmol/l EDTA, pH 8, 0.05% bromophenol blue, and 0.05% xylene cyanol), denatured at 99°C for five minutes, immediately placed on ice, and loaded on two types of gels, multiplex 0.5 MDE and non-multiplex 3% glycerol. Selected amplicons were then pooled for electrophoresis; this allowed simultaneous analysis of several fragments chosen according to band size and migration patterns. Electrophoresis was performed at room temperature for 16 to 20 hours at 6 W and eight hours at 8 W for MDE and glycerol gels, respectively. Results were visualised by autoradiography. Amplification and electrophoresis were repeated for confirmation of altered migration patterns. Variant bands were subsequently cut from gels, resuspended in distilled water, resubjected to PCR, and then sequenced with Big Dye Terminator Cycle Sequencing kits (PE Applied Biosystems) in both forward and reverse strand directions.

Following the routine practice of the laboratory, each abnormally migrating fragment was subject to sequence analysis regardless of whether it was also the location of a commonBRCA1 polymorphism, since it has been shown that the abnormal SSCP migration associated with common polymorphisms may mask a coexistent deleterious mutation.9

CSGE

The entire coding region of BRCA1including at least 15-50 bp of each flanking intron was subdivided into 33 segments. To facilitate PCR multiplexing and direct sequencing of selected fragments afterwards, all forward primers were tagged with M13-Forward tails and labelled with fluorescent FAM, HEX, or TET. Reverse primers contained M13-Reverse tails. Oligonucleotides were purchased from Eurogentec, Belgium; their sequences are available from the Devilee lab website (http://www.medfac. leidenuniv.nl/lab-devilee/Lab/csgeolig.htm).

The 33 fragments were amplified in one mono and 16 duplex PCRs as detailed on the website provided above. A 14 μl reaction mixture prepared in each well of a 96 well microtitre plate contained 10 pmol primers, 1 × PCR buffer (50 mmol/l KCl, 10 mmol/l TRIS-HCl, pH 8.4, 2.5 mmol/l MgCl₂, 0.2 mg/ml BSA, 0.2 mmol/l dNTPs), and 0.1 U Goldstar Taq polymerase (EuroGentech, Seraing, Belgium). Subsequently, 1 μl of each DNA sample (50 ng/μl) was added to the reaction mixtures. PCR was performed for 40 cycles consisting of 30 seconds at 94°C, 30 seconds at 58°C, and 30 seconds at 72°C.

After PCR, reaction mixtures corresponding to a given DNA sample were pooled into a 96 well microtitre plate in a HEX:FAM:TET ratio of 3:2:2 for a final volume of 24 μl, in a total of six pools per DNA sample (see above website for details). Seven μl of this mixture were aliquotted into a fresh plate and heat/air dried by exposing to 45°C for one hour. The mixture was dissolved in 2.5 μl of Pink Loading Dye (Amersham Pharmacia, Benelux, Roosendaal, The Netherlands), to which 0.25 μl GeneScan-500 TAMRA size standard and 0.25 μl loading buffer were added (Applied Biosystems). Using an eight channel loading device (Hamilton, Bonaduz, Switzerland), 1.5 μl of this mixture was loaded onto an f-CSGE gel, which had been pre-run for 15 minutes. The samples were subjected to electrophoresis through these gels for 4.5 hours at 1680 V at 30°C. Gels were analysed with GeneScan® and Genotyper® software (Applied Biosystems). Each abnormally migrating fragment was reamplified from the DNA sample using the same primers as above and sequenced in the forward direction using Big Dye Terminator Cycle Sequencing kits (PE Applied Biosystems).

TDGS

All BRCA1 coding exons were amplified from genomic DNA in a 7-plex long distance PCR. Individual exons or parts of exons were amplified in four multiplex groups of nine or 10 fragments each, using the long distance 7-plex PCR products as template, so that the entire BRCA1 coding region was resolved in a total of 37 fragments. Primers for the multiplex short PCR were designed as described.10 11Products of the four multiplex groups were combined, mixed with sample buffer, and loaded directly into the slot of a 2D gel. Electrophoresis was performed in an automated 2D DNA electrophoresis system12 and gels were stained with ethidium bromide. Spot patterns were interpreted by eye for the appearance of four spots rather than one, indicating the presence of a heterozygous mutation or polymorphism. The complete protocol forBRCA1-TDGS has been described previously.11 Each sample was analysed only once, under the same conditions, and fragments that were absent or faint were repeated by one dimensional DGGE (an average of five fragments perBRCA1 gene sample). Fragments that showed a four spot pattern that could be recognised as a previously detected polymorphism on the basis of their characteristic configurations were assigned as such. New variants were subjected to sequence analysis. Sequence analysis was either carried out on a Beckman CEQ2000 sequencer (75% of fragments) or contracted out to DavisSequencing (Davis, CA, USA) (25% of fragments). All 2D patterns are published on the web (http://www.tdgs.saci.org/myriad.html).

DHPLC

For purposes of PCR, BRCA1 was divided into 35 amplicons comprising the coding sequence and adjacent non-coding sequence in the regions of the splice junctions. Primers were designed to minimise overlap between fragments, to improve the robustness of PCR, or to increase the length of fragment screened. The primers used had originally been described for SSCP13 with the exception of the primers for exon 5.8 PCR was performed in a 50 μl volume containing 15 mmol/l Tris-HCl, pH 8.0, 50 mmol/l KCl, 1.5-4.5 mmol/l MgCl₂, 10 mmol/l of dNTPs, 0.25 μmol/l of each primer, and 10 ng of genomic DNA. For all PCR reactions, AmpliTaq Gold (Perkin Elmer, Foster City, USA) was used. The PCR cycling conditions comprised an initial denaturation step at 95°C for seven minutes to activate AmpliTaq Gold. Subsequent denaturing steps were 94°C for 45 seconds and extension steps of 72°C for 30 seconds. In some instances, annealing temperatures were decreased from 63°C by 0.5°C decrements to 56°C in 14 cycles, followed by 21 cycles at 56°C for 20 seconds. In one case, namely exon 23, the annealing temperature was decreased from 67°C to 60°C, while in the case of exon 11EF, it was decreased from 65°C to 58°C. In all other cases, 35 cycles were performed at constant annealing temperatures.

Denaturing high performance liquid chromatography was carried out on an automated HPLC instrument (Transgenomics Inc, San Jose, CA, USA). The DNA separation column was packed with proprietary 2 μ non-porous alkylated poly(styrene-divinylbenzene) particles.14 The mobile phase was 0.1 mol/l triethylammonium acetate buffer, pH 7.0 (TEAA, PE Biosystems, Foster City, CA, USA). Crude PCR products were subjected to an additional three minute, 95°C denaturing step followed by gradual reannealing from 95-65°C over a period of 30 minutes before analysis. Homo- and heteroduplex species were eluted with a linear acetonitrile (Merck, Vienna, Austria) gradient at a flow rate of 0.9 ml/minute. The start and end points of the gradient were adjusted according to the size of the PCR products using an algorithm provided by the WAVE Maker™ system control software (Transgenomics Inc, San Jose, CA, USA). Generally, analysis took eight minutes, including column regeneration and re-equilibration to the starting conditions. The temperature required for successful resolution of heteroduplex molecules was determined by use of the DHPLC melting algorithm available athttp://insertion.stanford.edu/melt.html,15respectively, the WAVE Maker™ software. Appropriate temperature(s) of analysis were determined for each amplicon, with 19 of the 35 amplicons requiring analysis at two temperatures and the rest at one. Known sequence variants, on average four per amplicon, are analysed along with the new samples to establish the proper performance of the DHPLC instrument. The appearance of additional peaks or shoulders was interpreted as indicative of the presence of a mismatch, which was subsequently analysed by sequencing. Nine amplicons known to containBRCA1 polymorphisms with a heterozygosity ⩾5% were sequenced routinely when observed to be heterozygous.8

COST CALCULATION

The costs of mutation analysis were calculated in two ways. The first only took into account the cost of consumable supplies on a per sample basis. The second calculation derived a “universal cost equivalent” that attempts to analyse each method in terms of labour, quantities of supplies (for example, numbers of ABI gels, numbers of oligonucleotide primers, number of PCR reactions) and run times necessary to perform an analysis.

Results

For the set of samples known to containBRCA1 mutations or not (table 1), the reported overall sensitivity of the methods, as summarised in table 2, required not only the initial detection of an abnormality in an amplicon, but also the ability to confirm the mutation by sequence analysis and to report the result correctly to a central source (LCB) who compiled the results. Samples for which PCR amplification could not be attained or for which there was insufficient DNA for sequence confirmation were not counted as “negative” results, but were omitted from the total. Only DHPLC was able to correctly identify each of the 58 mutations in the sample set. Eleven mutations were each missed by at least two laboratories, of which two (both single nucleotide substitutions resulting in premature truncation) were each missed by three of the four laboratories. The results of the individual laboratories are presented below.

SSCP

The laboratory using SSCP correctly identified 33 of 51 mutations (65%) (table 2A), with seven additional mutations occurring in samples that could not be analysed because of insufficient DNA for sequence analysis following SSCP, as discussed below. A false positive result that was reported in one of the 15 negative samples resulted not from a technical error, but instead from a laboratory sample switch that also accounted for one of the false negative results.

After the initial SSCP scan, 58 aberrant bands were detected on MDE gels and an additional five bands were observed on glycerol gels. Initially, two of five variants seen on glycerol gels were not detected on MDE gels owing to the presence of overlapping multiplex bands representing other exons. Reamplification of all possible variants from the initial screen confirmed the presence of 42 abnormal bands out of the 58 mutations (72%) distributed in different exons ofBRCA1. Aberrant electrophoresis of bands was identified for 19 of 20 (95%) frameshift mutations, including 17 of 17 deletions and two of three insertions. The mutation 5385insC (“5382insC”) was missed, although it has been previously detected by SSCP by this laboratory using the same techniques.16-18 This suggests that the efficiency of SSCP in detecting very subtle changes is variable. Abnormal migration was seen for 23 of 38 nucleotide substitutions (61%), including five localised to introns (table 2A). SSCP failed to detect seven of the nine G to T substitutions and four of the 10 C to T substitutions, but abnormal migration occurred with five of six G to A substitutions. Several of the single nucleotide substitutions that did not alter electrophoresis mobility occurred near either end of a PCR amplicon. For example, the missed G to T substitutions that resulted in M1I and E143X each occurred near the ends of exon 2 and exon 7, respectively.

Sequence analysis was performed for all samples for which abnormal migration was identified. In seven instances, the first obtained sequence was not diagnostic and there was insufficient DNA to repeat the procedure; these seven variants were excluded from further calculations of sensitivity. Sequence analysis identified 34 of the remaining 35 mutations for which abnormal migration had been observed, but failed to identify the frameshift mutation 2576delC. Finally, as mentioned above, mislabelled samples resulted in incorrect reporting of two samples, resulting in one false positive and one false negative interpretation in the final report of results.

CSGE

The laboratory using CSGE correctly identified 27 of 45 mutations (60%), with 13 mutations that could not be analysed owing to failure to amplify by PCR as discussed below (table 2B). No mutations were identified in the 15 samples documented not to harbour a sequence alteration.

Abnormal electrophoretic migration by CSGE was present in 34 of 45 (76%) samples for which PCR amplification was successfully performed. Nucleotide substitutions accounted for 10 of the 11 mutations that were missed at this stage of analysis (table 2B). Retrospective evaluation showed subtle differences relative to wild type fragments in three of these 11 false negatives, and two additional peak shifts were sufficiently clear that they represented erroneous interpretation by the observer. The remaining six (all missense changes: G>T, G>T, C>T, T>C, C>A, A>T) did not show migration patterns that were distinguishable from the wild type.

Four additional mutations were missed because of failure of sequence analysis to confirm a mutation following observation of abnormal gel mobility. One of these was a base substitution, a T to G at cDNA nt 855, resulting in the substitution of valine for leucine at amino acid position 246. This variant produced only a very subtle change in the sequence trace at the heterozygote position and was erroneously called negative. The other three were small frameshifting deletions (2072del4, 2080delA, and 2594delC) that are ordinarily considered to be easily detectable by sequencing. Sequence data were analysed using the Staden software, which subtracts the sample sequence trace from a wild type control trace to highlight sequence differences, excluding those parts of the trace that do not meet minimum quality. Sequence analysis was performed only in the forward direction, and since these three mutations were each at the very ends of sequence traces, the quality of data in the region of the mutation was suboptimal. This frameshift was therefore not identified by the Staden program, which went unnoticed by the operator.

Administrative errors led to the remaining three false negative results. In one instance, a mutation detected in one sample was incorrectly assigned to another that was also mutation positive, but both of these mutations were subsequently lost in the final report. An additional mutation, 3875del4, was identified correctly in CSGE and sequence analysis, but reported as negative owing to a clerical error.

TDGS

The laboratory using TDGS correctly identified 51 of 56 mutations (91%); the two remaining mutations could not be identified owing to repeated failure of sequence analysis (table 2C). In addition, three apparently false positive results were reported following sequence analysis as described below.

Each of the five mutations missed by TDGS (L246V, IVS17+1G>T, Y1463X, 3171ins5, and 4510del3insTT) appear to be result of misinterpretation of the 2D gel. In each of these cases, the mutant allele amplified much less efficiently than the wild type, resulting in one intense wild type homoduplex, one very light or absent mutant homoduplex, and two very light heteroduplexes.

The TDGS laboratory reported three mutations that were otherwise not identified by any other techniques, including sequence analysis, and are interpreted as false positive results. Two variants, “196T>C” and “IVS8+1delG”, were seen both in 2D gels and by subsequent sequence analysis of the TDGS fragment. One of these, an apparent “IVS8+1delG” (which has not been previously reported) could not be confirmed in a new DNA sample provided by the source. A “196T>C” variant (not previously reported) was seen again through 1D DGGE and sequencing of a product reamplified from the long range PCR product, although the wild type allele predominated over the mutant allele both in TDGS and sequencing. No indication of this variant was seen by any other laboratory, including the reference sequencing laboratory, despite reanalysis using three different PCR primer pairs and two different PCR cycling conditions. These two presumed false positive results may be the result of mutations introduced during long distance PCR, but are unlikely to have resulted from sample mix ups, since they have not been observed at either the source or any of the other analysing laboratories. A third false positive consisted of an apparent homozygous “S741C” that turned out to be a sequencing reading error owing to overlapping peaks generated by the software used with the automated sequencer. Sequence analysis was performed for another heterozygous mutation in this fragment, and this is interpreted as a sequencing interpretation error rather than a TDGS false positive. Two other samples (in exons 13 and 19) exhibited three spot patterns detected by TDGS that were not subsequently confirmed by sequence analysis, and repeated analysis indicated that PCR byproducts had initially been interpreted erroneously as heteroduplexes.

DHPLC

The analysis of the 65 DNA samples by DHPLC resulted in the detection of all 58 eligible mutations (table 2D). No variants were noted in the 15 samples that did not carry a sequence change.

Only 18 (29%) of the 58 mutations had been observed previously in the laboratory performing DHPLC.8 19 With the exception of three (5%) mutations, all yielded chromatograms with two or more distinct peaks in contrast to the single peak observed for homozygotes. In the remaining three cases (C61G, A1708E, and M1775R), the presence of a mutation was shown by the appearance of a fronting or trailing shoulder, representing distinct peak broadening compared to a homozygous control. All chromatograms of the 58 sequence variants detected in this study together with the corresponding homozygous wild type chromatograms are available from the author upon request (Teresa.Wagner{at}akh-wien.ac.at).

It should be noted that the selection process of the appropriate temperature(s) of analysis for each amplicon was crucial to ensure high sensitivity. Similar to other temperature sensitive methods, the presence of more than one melting domain may require the analysis of a given amplicon at more than one temperature. Only one (C61G) of more than 250 different BRCA1 andBRCA2 mutations analysed to date has a temperature detection window as narrow as 1°C. All other mutations could be easily detected over ⩾2°C. Currently, 19 of the 35BRCA1 amplicons are analysed at two temperatures, while the remaining 16 are screened at one unique temperature

COST ANALYSIS

The cost of consumable supplies used to “prescreen”BRCA1 for the possibility of a mutation were estimated by the performing laboratories to be (in US$) $6.20 for SSCP, $30 for CSGE, $17.60 for TDGS, and $50 for DHPLC, excluding the costs of instrument depreciation, labour, and overheads. Although the cost of reagents was lowest for SSCP, performing SSCP in the fluorescent format would increase the cost of analysis to that of CSGE, which used fluorophors on an ABI 377.

An estimate of the resources required for each method can be derived by consideration as universal cost equivalents, as noted in table 3. For example, the “universal cost equivalent” for running CSGE takes into account a pipetting robot, an automated sequencer, and a thermocycler. The cost of the semiautomatic sequencer used by CSGE is approximately US$110 000 when purchased new. Set up costs include the purchase of oligonucleotides, while ongoing costs are dictated by plastics (microtitre well plates), Taqpolymerase, gel solution, and dye markers. Complete analysis of 16 samples can be finished in 16 hours (or two days), seven hours of which are hands on time. In comparison, the instrument required for DHPLC costs approximately US$75 000. With current instrumentation, it takes 10 hours and 57 minutes, exclusive of PCR, to complete the DHPLC analysis of one patient (table 3). Of that time, only 13 minutes are spent on the actual evaluation of the chromatograms. Two hours and 30 minutes are required to load the DHPLC instrument and to carry out such routine maintenance tasks as the preparation of buffers and the change of the precolumn filter. The actual chromatographic analysis, which is carried out automatically, takes the remaining time of eight hours and 14 minutes, provided that the optional fast clean up protocol is used (Accelerator™, Transgenomic). In contrast to CSGE and DHPLC, SSCP and TDGS can be performed with relatively inexpensive equipment (less than US$10 000 each). It should be emphasised that even for a given method the costs are likely to be highly variable between laboratories using the same methods owing to such variables as institutional discounts for reagents, reaction volumes, the number of samples run at a given time, and employee salaries.

Discussion

Almost all molecular epidemiological data that deal withBRCA1 and BRCA2, and other large genes, have used mutation scanning technologies, such as SSCP, CSGE, and TDGS. This blinded evaluation of four commonly used mutation scanning technologies against known germlineBRCA1 mutation positive and negative samples previously identified by direct sequence analysis showed that commonly used methods vary widely in their sensitivity and specificity, even in expert laboratories. These findings provide a basis for reinterpreting previous estimations of mutation prevalence that were based on these methods. For example, previous evaluation of a population of unselected women with breast cancer attributed 3.3% of cases to mutations inBRCA1 on the basis of SSCP analysis,18 but after correcting for mutations not detected by this method, the actual figure may be closer to 5%. Another series used CSGE to estimate that 16% of women attending a hereditary risk clinic would have mutations inBRCA1,20 but this is likely to be closer to 27%. Neither of these studies analysed the contribution of BRCA2 by these methods, but those studies that have done so have probably underestimated the prevalence of mutations in that gene by a comparable percentage.

The two methods relying solely on DNA conformation, SSCP and CSGE, showed lower mutation detection sensitivity compared to DHPLC and TDGS, both of which separate mutational variants on the basis of melting temperature.5 21 DHPLC was the only method to show comparable sensitivity and specificity to sequence analysis, corroborating previous studies of this method.8 15 22-24 DHPLC also had the highest cost overall, including a relatively high investment per instrument. It is likely that improvements in technology and efficiency, such as the use of arrayed capillary columns and laser induced fluorescence detection, will reduce the time and cost of analysis.24 25

The efficiency and expense of other methods will also presumably benefit from future improvements as well. It appears possible, for example, to substantially reduce the time required for TDGS by using much thinner gels, permitting higher electrophoretic voltages, and the use of fluorescent labelled primers will obviate the need for gel staining (McGrath et al, manuscript submitted).

An advantage of TDGS is that it has the highest throughput of the methods evaluated, mainly because of extensive multiplexing and a low per sample cost. The sensitivity of TDGS was impaired by frequent preferential amplification of the non-mutant allele, resulting in very light heteroduplexes that obscured accurate reading of the gels. This source of error has been previously described,26 although not specifically with regard to this method. Interpretation of the complex spot patterns produced by TDGS was another source of error which also contributed to the reporting of false positive results.

After accounting for sequencing and administrative errors, SSCP and CSGE were comparable as methods to “pre-scan” for mutations (72% and 76% sensitivity, respectively) and both tended to miss the same types of mutations (table 2), particularly single nucleotide substitutions. Despite their relative low sensitivity compared to the other methods, both SSCP and CSGE offer the advantages of being relatively easy to set up and interpret. CSGE is particularly easy to implement, as prolonged optimisation is not necessary. It is also particularly well suited for rapid and cost effective screening for specific mutations provided that conditions are optimised for the specific variants of interest.

It should be noted that the sensitivities of CSGE and SSCP were somewhat lower than previously reported. This may be attributable to the rigorous design of this study, which was intended to simulate the actual challenges faced by a laboratory in the practice of molecular epidemiology. For example, the participants in this comparison were blinded as to which mutations were present as well as which of the samples actually contained mutations, in contrast to studies that have tested the ability of a method to identify a specified mutation known to the investigator.27 In addition, this comparison required each method to find a mutation identified by complete sequence analysis, whereas some other studies have tested the ability of a particular method to identify mutations previously identified by a comparable method.28 The sensitivity of SSCP may be improved by capillary electrophoresis rather than the use of slab gels,23 29 although even with this modification the size of the fragment that can be accurately analysed is still limited to about 300 bp.

Additional specific factors may also have affected the results of this comparison. For example, after completion of the analysis for this study, the CSGE team discovered that the fluorescent labelling of their PCR primers had been inefficient, and the consequent decrease in the signal to noise ratio may have obscured subtle changes in peak shape or position during electrophoresis and contributed to false negative results. This factor may also have contributed to the relatively high failure of detectable PCR products (13 of 58, 22%) in that laboratory compared to the other laboratories that were each able to amplify every sample. It should be noted, though, that problems with the quality of purchased reagents are not unique to the participants of this study, as many, if not most, research laboratories are unfortunately vulnerable to such occasional lapses, which can compromise accuracy unless the quality of each lot is first confirmed. Finally, there are specific mutations that appear to be recalcitrant to detection by certain methods. Several individual mutations were in fact missed by both SSCP and CSGE, particularly single nucleotide substitutions but also the common frameshift mutation 5385insC. This mutation produces only a subtle alteration with SSCP30 and requires the use of different gel conditions for its reliable detection. Awareness of mutation specific weaknesses of particular detection methods is important in the design of large epidemiological studies, especially where frequent mutations such as 5385insC may be systematically missed. Control DNA samples for several mutations inBRCA1 (and BRCA2) should be used to validate detection methods; many such samples are available from the BICBRCA1/BRCA2mutation panel (http://locus.umdnj.edu/nigms/charmut/bicpanel.html).

Another method that has been described for identifying mutations inBRCA1 that was not included in this blinded comparison is the protein truncation test (PTT).31 This method, which uses either DNA or cDNA as the template for RT-PCR, is designed to identify that subset of mutations which result in a truncated protein product. This method therefore misses approximately 20% of the deleterious mutations so far described inBRCA1 that are not protein truncating (www.nhgri.nih.gov/Intramural_research/Lab_transfer/Bic/). In addition to these theoretical limitations, PTT as actually practised is subject to technical sources of error as well.32 When cDNA is used as a template, mutations may be missed owing to “nonsense-mediated decay” of mutated mRNA transcripts.32 In some instances, PTT may be performed using DNA, specifically the large exon 11 ofBRCA1. In addition to missing the described splice site mutations affecting this exon, PTT cannot detect protein truncating mutations occurring either at the extreme 5′ end of the exon, because the synthesised protein products are too small to be detected, or mutations at the extreme 3′ end, because protein products that are within 10% of the size of wild type may not be resolved easily.32 The sensitivity of PTT in detecting clinically significant mutations in BRCA1 is therefore limited.

It should be pointed out that false negative results arose not only from the limitations of the specific methods used for mutation screening, but also during confirmatory sequence analysis. For the CSGE group, the software used was not able to identify three mutations located at the ends of the exons. This indicates not only the potential limitations of such commercially available software, but also the necessity of sequencing in both forward and reverse directions to obtain optimal data for each end of the amplicon. The SSCP group was also unable to characterise the sequences of three mutations that had produced abnormal migration patterns. Sequence analysis by the TDGS group was not a source of false negative results but was responsible for false positive results, and two additional samples could not be sequenced at all. Overall, the results of this study show that robust sequence analysis is crucial to the sensitivity of genetic testing regardless of what method is used to screen for mutations. In order to minimise errors, sequence analysis should be performed in both the forward and reverse directions, use dye primer rather than dye terminator chemistry to minimise sequence specific variability in peak heights, and reanalyse ambiguous or poor quality data.

Despite great care taken by the participating laboratories, non-technical administrative errors also accounted for a total of four false negative results as well as one false positive result. Indeed, while most human geneticists who perform mutation analysis recognise that administrative errors occur to some extent in even the best research laboratories, the frequency of such errors in this setting may not be appreciated. Clerical errors led not only to incorrect reporting of results but also to inaccuracies in the names or locations of several mutations. Although these were not counted as errors for the purposes of this study, such errors could nonetheless lead to false negative results among relatives being tested for the incorrect mutation. In contrast to clinical laboratories where stringent quality control measures are mandated and compliance is carefully regulated, research laboratories are under no such supervision. Research laboratories in the USA that are not specifically certified for clinical diagnostic testing are therefore precluded from returning results for the purposes of patient care. A common practice among research facilities is therefore to confirm positive results in a certified clinical laboratory if such results are to be used in health care. The results of this comparison indicate that it may be advisable to recommend that negative as well as positive test results obtained in a research setting be confirmed by a clinical laboratory, especially in people with a strong family history indicative of a mutation inBRCA1 or BRCA2.

The findings of this study have implications for the interpretation of studies of the genetic epidemiology of hereditary cancer risk that have used the methods evaluated here. Given that even sequence analysis cannot detect the 10-15% of BRCA1 andBRCA2 mutations that consist of large chromosomal rearrangements,33 existing and future molecular epidemiological studies should account for underestimates of mutations owing to the limitations of the techniques employed to identify them.