Article Text

Download PDFPDF

Evidence based medicine in practice: lessons from a Scottish clinical genetics project


OBJECTIVE To establish national clinical guidelines and integrated care pathways for five conditions (tuberous sclerosis (TS), Huntington's disease (HD), myotonic dystrophy (MD), neurofibromatosis type 1 (NF1), and Marfan syndrome (MS)) and audit their use in Scotland.

DESIGN Systematic review of published reports followed by consensus conferences to prepare clinical guidelines and integrated care pathways. Structured review of medical records before and after introduction of integrated care pathways to document changes in practice. Survey of staff views on procedures adopted.

SETTING All four clinical genetics centres in Scotland.

RESULTS Project resulted in reduced variation in practice across centres, improved data recording in medical records, and improved communication with other professional groups. A very poor evidence base for management of patients with the conditions studied was found.

CONCLUSIONS A collaborative structure for undertaking clinical research would improve the evidence base for current practice. National discussion of the boundaries of responsibility of care for the long term management of patients with these disorders is required. The integrated care pathway approach shows promise as a means of facilitating the development of audit within clinical genetics services.

  • clinical guidelines
  • audit
  • evidence based medicine
  • care pathways

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Clinical genetic services are concerned with people and families who are threatened by significant genetic risks. In Scotland, these services are available from four supraregional centres in Edinburgh, Glasgow, Aberdeen, and Dundee. Continuing care for many of these families occurs over many years and involves health professionals from many disciplines, often from more than one clinical genetics centre. Clinical genetics is an outpatient specialty and thus does not benefit from the excellent routine information captured from inpatient records through the national Scottish Morbidity Record system. This lack of access to recorded data from patient consultations has greatly hindered the development of audit within the specialty.1We therefore identified a need to establish multidisciplinary national clinical guidelines to ensure families receive well planned and consistent care and a system for the continuing capture of data from clinical consultations in order to audit clinical genetics services throughout Scotland.

In July 1996, funding was received from the Scottish Office Clinical Resource and Audit Group (CRAG) national projects committee to develop evidence based national clinical guidelines and integrated care pathways (ICPs) for five genetic conditions which account for approximately 30% of all clinical genetics consultations (excluding cancer genetics referrals) in Scotland. These conditions were tuberous sclerosis (TS), myotonic dystrophy (MD), Marfan syndrome (MS), Huntington's disease (HD), and neurofibromatosis type 1 (NF1).


For each of the five study conditions, an initial review of medical records to document the current management of affected subjects was carried out at each centre. Guidelines development groups were formed to establish draft clinical guidelines following the procedures recommended by the Scottish Intercollegiate Guidelines Network (SIGN). SIGN reviewed in detail the documentation from the guideline development process and acknowledged that this was consistent with their published recommendations.2 3

A systematic review of publications from 1980 to 1997 on the five conditions identified 7086 publications. Abstracts or the full text of these papers were read to identify publications which contained data relevant to the management of patients. The 963 publications identified were then reviewed in detail by four teams across Scotland. A summary of the evidence, graded by level of evidence,4 was prepared for each condition for discussion in multidisciplinary meetings with representatives of a wide range of professional groups involved in the care of these patients. These meetings attempted to involve recognised experts within the UK and Scottish representatives of the relevant patient organisation. Draft guidelines were then prepared and were circulated together with the summaries of the published evidence and details of views expressed in the multidisciplinary meeting to all staff. For one condition (Marfan syndrome), the draft guidelines were submitted for comment to an international panel of experts.

National consensus conferences were then held for each of the clinical conditions. Consensus was arrived at by one of the formal consensus development methods, the “consensus development conference.”5 This involved chaired structured discussions of evidence summaries (guidelines development reports) and draft guidelines. Consensus conferences included participation of clinical genetics staff from all four centres, experts from appropriate professional groups, and representatives of patient groups. Decisions were made in this public forum with notes made of varying opinions where these were relevant. Where differing opinions existed opportunities were given to have further discussion, if necessary at a subsequent conference. National guidelines therefore comprised a statement of best clinical practice accepted by all centres. In a few specific (relatively minor) issues it was impossible to achieve this position and the guidelines represented a statement that no-one differed from sufficiently strongly to veto.

Guidelines were then prepared into an integrated care pathway (ICP) format6 that was used in patient medical records. The care pathway both served as a structured record for the efficient capture of key clinical data and as a reminder of the recommendations for patient management contained in the clinical guidelines. Finally, these were discussed in each of the four centres and local protocols and integrated care pathways were agreed upon.

Selected items of clinical information were searched for in the medical records of patients who consulted in the year before and after the ICPs were implemented. These chosen items represented data which were clinically important and, for clinical practices, recorded in sufficient detail in existing medical records to enable this “before and after”comparison to take place. Comparisons were made of the completeness of recording of important clinical data and of selected clinical practices in each of the centres. Results were discussed at subsequent national consensus meetings and care pathways amended accordingly. A survey of medical personnel involved in the development of the first guidelines (for TS) was carried out. A questionnaire invited them to give their views about the various stages of the development in relation to the degree to which these had been time consuming, helpful, and resulted in changes in their own clinical practice.

Copies of the fully referenced guideline development records and ICPs can be found on the Edinburgh clinical genetics department's website at and have been published on the internet by the UK Public Health Genetics Network at A sample page from one of the ICPs is reproduced in fig 1 as an example of the format adopted.

Figure 1

Integrated care pathway for Marfan syndrome.



Table 1 shows the number of publications (from 1980 to 1997) identified through a Medline search on the subject heading of the clinical condition under review. The large number of publications (7086) highlights the intense research activity on these rare inherited disorders. However, only 14% of publications contained data on clinical management and only 3% contained data of sufficient quality and relevance to clinical practice to contribute directly to the development of the clinical guidelines. Only one (0.01% of all publications) randomised controlled trial (RCT) was identified.

Table 1

Publications identified in systematic review of published reports (1980–1997)

Table 2 shows the level of evidence supporting the 122 individual clinical management recommendations in the national clinical guidelines for the five conditions, classified by categories defined by the US Agency for Health Care Policy and Research.4 Of the 7086 articles published since 1980, only one RCT was identified. Furthermore, 66% of the recommendations in the clinical guidelines were based on level C evidence which is considered to indicate “an absence of directly applicable studies of good quality”.

Table 2

Level of evidence base for clinical management recommendations: number by level of evidence as defined by US Agency for Health Care Policy and Research


The views of clinical genetics staff are given in tables 3, 4, and5. A majority of staff found the project “very helpful”, with 71% reporting the national consensus conferences and 50% reporting the preparation of the review of publications to be “very helpful”. Very few found the project “not helpful”. A majority of staff reported that involvement in the project had resulted in a change in their clinical practice with the highest percentage (89%) noting a change in practice because of the use of the ICPs.

Table 3

How time consuming did those involved in the development of tuberous sclerosis guidelines/care pathway find stages of the development?

Table 4

How helpful were the stages of the tuberous sclerosis guidelines/care pathway development?

Table 5

Did the stages of the guideline/care pathway development result in change of practice?


The use of integrated care pathways varied by centre (used in 38-80% of consultations, table 6) with the highest use being in the centre which was the main base of the CRAG funded staff member and which therefore had a higher level of external support for this activity. Overall, 57% of new consultations over the one year period had a completed integrated care pathway as part of the medical record of the patient.

Table 6

Numbers (percentages) of patient consultations over a one year period for which an integrated care pathway was completed (by condition and clinical genetics centre)


Table 7 shows that the completeness of recording of five specific items of clinical data in the medical records increased by an average of 21% following the introduction of the ICPs. The completeness of recording of other data in ICPs was investigated. The median proportion of records with a specific ICP data item recorded correctly was 97% for NF1 (based on a review of 17 data items), 95% for TS (based on a review of 26 data items), 88% for Marfan syndrome (based on a review of 11 data items), and 70% for MD (based on a review of 13 data items). Improved recording of clinical data was a consistent finding across all four centres and for each of the five items of clinical data investigated. In addition, the data were recorded in a standard format across all centres after the introduction of ICPs and thus required much less time to identify and extract from the medical records.

Table 7

Percentage of patients in whom selected important clinical data were recorded fully in their medical record (before and after introduction of integrated care pathway)


Five specific items of clinical data relating to examinations or investigations recommended in the national clinical guidelines were selected for audit. Table 8 shows the improvement in the clinical practices (as defined by adherence to those recommended in national clinical guidelines) for three level B recommendations following the introduction of the ICPs. In addition, variation in practice among the four centres was reduced. Table 9 illustrates similar data for a further two recommendations for which there was level C evidence. This again shows reduced variation in practice following ICP use, but adherence to the national clinical guidelines actually fell. At a subsequent national consensus conference, these recommendations were altered after review of one year's experience in their use. Although there was considerable variation in practice on these issues at the start of the project, unanimous agreement on the most appropriate recommendation was reached.

Table 8

Percentage of patients in whom examinations or investigations recommended (level B evidence) in national guidelines were carried out (before and after introduction of integrated pathways)

Table 9

Percentage of patients in whom examinations or investigations recommended (level C evidence) in national guidelines were carried out (before and after introduction of integrated care pathways)

After the introduction of the ICP for Marfan syndrome, the diagnostic process was more explicit and uniform across the four centres. All centres adopted the Ghent criteria for diagnosis. A review of the medical records of 60 people recorded as being affected with Marfan syndrome before the introduction of the ICP showed that only 24 (40%) had clinical findings which fulfilled Ghent criteria and that this proportion varied by centre.


National clinical guidelines and ICPs were prepared detailing the recommended assessment and management of HD patients. However, the level of evidence supporting these guidelines was poor. The HD ICP was not completed on any patients in three of the Scottish centres. One of the recommendations, adopted by consensus, was to use the Unified Huntington's Disease Rating Scale to monitor patients. In practice, the majority of staff found the rating scale time consuming and were, in the absence of published evidence of effectiveness, not convinced that the use of this scale led to benefit to the patient. In addition, most patients with HD consulted clinical geneticists when they had a specific problem. The use of ICPs designed for a general assessment was found to be inappropriate in these consultations. There was support for the ICP as a statement of recommended follow up, from the Scottish Huntington's Disease Association. The Association's advisors regularly attend HD patients and identified a possible role for the pathway as the basis for a patient held record which could integrate input from multiple health and social care professionals.



An ICP was completed as part of the medical record in 57% of new consultations for four of the five conditions over a one year period. Feedback at consensus conferences showed two major reasons for non-completion. First, in many of these cases, the patient directed the consultation to focus on one specific matter (such as a new pregnancy). The general assessment laid out in the ICP was clearly less appropriate in these circumstances. Second, ICP forms were often not available during consultations in settings outside the regional centres in the early months of the project. The lowest completion rates were for the two conditions (MD, HD) for which clinical recommendations were supported by the poorest evidence base (no level A and only two and three level B recommendations respectively). It is possible that use will increase further once ICPs are revised in response to feedback, so that they are tailored to the needs of individual centres.

The major obstacle to implementation is the time commitment required to initiate and maintain the level of organisation required for such an approach. Clinical genetics departments typically have few staff who may not have the time or be sufficiently familiar with the methods of evidence based medicine to follow this approach. It is important, therefore, that action is taken at a national level to share resources and form close links with other disciplines (such as public health medicine) as required. Concern was raised by staff that the use of ICPs may lead to increased attention given to recording details of the consultation at the expense of establishing good communication with the patient. It will be important to monitor the impact on patient understanding and satisfaction with the consultation.


A core activity in clinical genetics is the calculation and communication of genetic risk through genetic counselling. Since the principles of Mendelian inheritance, the mode of inheritance, and molecular basis of many rare inherited diseases are well established, the evidence base underpinning genetic counselling is secure. However, this is not the case for all other aspects of the clinical management of patients with these conditions. The major problem identified by this project was the very poor evidence base for the clinical management of patients with five important clinical genetics conditions.

Despite extensive publications on the genetic basis of these conditions, only one randomised controlled trial was identified. Although many areas of clinical uncertainty exist, there are very few published, well conducted clinical studies. Feedback from a recent national meeting of medical clinical genetics trainees suggests that clinical research may be accorded lesser prestige and priority than laboratory research.

Most of the guidelines were based largely on the consensus among the four Scottish centres after review of publications, consultation with recognised experts, and consensus meetings to discuss controversial issues (66% of recommendations at level C, table 2). A number of different formal consensus methods (such as nominal group technique, Delphi method, and consensus development conference) have been described and their use in the development of clinical guidelines has been reviewed.7 The consensus conference method adopted in this project resulted in the identification and discussion of key management issues in an open debate. This was successful in drawing together a wide range of knowledge and experience and resulted in increased agreement on best practice and reduced variation in practice between clinical genetics centres in Scotland. A strength of this approach is the opportunity it gives for members to revise their views in the light of discussion which is tightly focused on a review of the published reports, so that an agreed national guideline can be developed.8 9

It is known that consensus methods are vulnerable to the introduction of bias. We attempted to limit these by adopting principles which have been shown to improve the reliability of consensus methods.7 These included forming a heterogeneous group with a membership of more than 12 people; breaking the judgements into a number of discrete units; providing all members with a summary of the published evidence in an accessible format and with grading of evidence; holding the conferences in comfortable surroundings; conducting the discussions through a facilitator who encouraged participation from all members and noted outlying views as well as the agreed consensus.

The major concern with consensus methods is that it is not known whether the judgements reached result in better clinical outcomes.7 Our experience (tables 7, 8, and 9) is that consensus conferences can be successful in reducing variability in clinical practice across regional centres. This is important in clinical genetics where affected members of the same family may be managed at different centres and is likely to result in improved patient satisfaction with services. However, it is not certain to what extent this will result in improved patient outcomes beyond patient satisfaction. Although we monitored practices for only five clinical activities, it is of interest that we found that staff adopted national recommendations only when they were supported by level B evidence. This supports the view that this approach can result in changes in practice when there is at least level B evidence supporting recommendations. It raises doubts about whether this approach will be successful in persuading staff to change their practice in circumstances (such as in many areas of patient management in clinical genetics), in which there is substantial variation in practice and only level C evidence exists.


Clinical genetics services are evolving rapidly. There is a rapid and relatively uncontrolled expansion of cancer genetics activities, with referrals typically rising by about 27% per year.10There is also an (appropriate) expansion of the role of non-medical staff, increasingly carrying out duties which were until recently carried out by consultant staff.11 Given these fundamental changes in staffing and clinical workload, it is important that the quality of patient care in clinical genetics is monitored. Audit of these services against agreed national standards will be an important means of achieving this.

However, there are a number of difficulties in carrying out audit successfully in clinical genetics services. It is an outpatient specialty in which there is little or no access to routine clinical or management information.1 12 Our review of medical records indicated incomplete recording of key clinical data. The most complete source of data is often the summary letters to patients and referring clinicians written after consultations. Therefore audit based on retrospective review of medical records would be severely limited by lack of clinical detail. The small numbers of cases managed by individual clinical genetics centres makes it difficult to quantify experience and learn lessons within a single centre which can result in improvement in the quality of care.

A Medline search using textwords and subject headings for various types of audit and clinical or medical genetics conditions showed only 13 publications from 1966 to September 1999. Furthermore, audit activity in clinical genetics has tended to concentrate on process variables since there have been considerable difficulties in agreeing on suitable and acceptable outcome measures.13 We suggest that the procedures described in this report could form the basis of a national structure for audit of clinical practice. This approach creates a means of setting standards of practice, improving both the quality and quantity of information in patient case notes and capturing key data in a format that can be readily extracted. We believe that a collaborative framework for audit in clinical genetics is essential and that the most successful examples of audit have come from collaboration among centres (such as by the West of Britain Clinical Genetics Group and the Confidential Enquiry into Counselling for Genetic Disorders (CEGEN)12 14 15).


Hereditary conditions often involve multiple pathologies and management can require input from a variety of specialists. ICPs represent a single common record of care1 6 allowing each discipline involved to record their own involvement and provide and be kept up to date with the outcome of appointments with other specialists. In some circumstances, multidisciplinary consensus meetings leading to the production of the ICP might result in an agreement to hold multidisciplinary clinics to coordinate care, where this seems appropriate. However, even where this does not occur, we have found that this process can improve the understanding of the roles of other disciplines and promote teamwork. The settings of standards in liaison with patient groups should ensure that services are sensitive to patient needs and may result, over time, in improved patient satisfaction with services received.


Feedback from interviews with specialist genetic nurses and genetic associates who used the ICPs showed that they found ICPs helpful in supporting them to take on a new role within the clinical genetics departments. The clear specification of agreed best practice contained in the ICPs gives them more confidence to take on an extended role under appropriate supervision. The detailed recording of key clinical data permits close supervision by their consultant colleagues. One possible future approach to the follow up of patients with inherited disorders is that clinical genetic nurse specialists or genetic associates could coordinate the follow up of these patients in community or primary care settings following guidelines expressed in ICPs established by clinical genetics together with the relevant related specialties.6 These ICPs would be kept in the patient's medical records. The specialist nurses or associates would link closely with general practitioners (involving them more actively in the management of these patients) but be supervised by consultant staff in regional clinical genetics departments. This expanded role in both departmental and community settings could release medical staff to concentrate on more specialist areas of clinical genetics, such as dysmorphological assessment of infants and diagnosis of rare or complex syndromes in older children and adults.


Some of the above discussion on evidence base, variations in practice, and audit is relevant to the development of clinical governance within clinical genetics. An important finding in this project was the lack of agreement on definition of the boundaries of responsibility of care in clinical genetics. There is universal agreement that the overall aim of a clinical genetics service is to assist those affected by, or at risk of, genetic disorders to live and reproduce as normally as possible and that genetic counselling is a central activity within the discipline.11 However, there is much less agreement on the extent to which clinical genetics should take prime responsibility for the regular follow up and, where appropriate, regular screening of affected patients to identify complications at an early stage. Practices appear to vary considerably from centre to centre, often dependent on the particular interests of specific clinicians. For example, patients at risk of NF1 in some centres are not followed up but discharged to the care of their general practitioner, whereas in others close follow up in multidisciplinary NF1 clinics takes place.16 There are a number of important problems with this. If responsibility for follow up is not clearly documented then it is less likely to be carried out. Problems may arise when there are staff changes in regional centres. Less priority may be given to carrying out research addressing important clinical management issues (as suggested by the results of our review of publications). Consensus conferences provide a means by which relevant professional groups can meet to discuss responsibilities for and boundaries of care. It is probably preferable that this be carried out at a national level.


This national initiative was successful in creating a formal mechanism through which the centres systematically reviewed medical publications and reached consensus on best practice. This resulted in improved recording of clinical data in medical records (table 7), a reduction in variation in practice (tables 8 and 9) and improved communication between regional departments. The project strengthened links with associated professional disciplines involved in the care of patients with these clinical conditions and formalised links with representatives of patient organisations in the development of best practice guidelines. In the management of patients with TS, for example, this resulted in the majority of patients at each centre receiving appropriate renal and intracranial screening and in an improvement in the recording of clinical details of TS in the medical records.

A survey of staff in all four centres showed that, in general, staff found the development of clinical guidelines a helpful exercise and the ICPs useful instruments in guiding them in their clinical practice. Since the guidelines and ICPs developed in this project cover conditions which account for approximately 30% of consultations (excluding cancer genetics referrals) in clinical genetics departments in Scotland, these findings have potential importance for clinical practice in clinical genetics.

We suggest that it is important that awareness is raised within the clinical genetics profession in the UK of the poor evidence base for current clinical management of patients with rare inherited disorders and the low level of good quality clinical research addressing these issues within the profession. It is important that consideration be given to establishing a national forum to identify key outstanding clinical questions which could be addressed by multicentre research. A collaborative framework could then be developed which could undertake multicentre clinical trials, a “rare diseases trials group”. Some form of central support would be required to coordinate and administer this initiative and provide expert epidemiology and statistics input. An alternative response might be to develop trials based on a Bayesian rather than frequentist approach to analysis, so that results can more readily be extrapolated to clinical practice.17 In parallel with this, we suggest that there is also a need for clinical genetics to define its role in the follow up management of patients with rare inherited disorders.

Although systematic review of published reports is very time consuming, a national framework for conducting such reviews following published SIGN procedures could be established, thus sharing the time commitment among several clinical genetics departments. This could identify areas of clinical genetics practice in which there is sufficient level A and B evidence to support the development of further national clinical guidelines and integrated care pathways based on formal consensus methods. Where insufficient level A and B evidence exists to be able to prepare valid clinical guidelines, our experience suggests that the approach we describe may have limited success in achieving adherence to guidelines but may succeed in reducing variations in practice among centres and in improving communication with other professional groups and patient representatives.

We believe that a collaborative framework for audit in clinical genetics is essential (CEGEN is one example of a successful national initiative). The definition of core datasets of key clinical information and identification of a mechanism for routine data capture would encourage the promotion of national audit and contribute to the establishment of clinical governance within the profession. The use of integrated care pathways shows considerable promise as one means by which audit can be taken forward in clinical genetics. The improved specification of best practice and improved documentation of current practice could create a framework for the development of professional practice and contribute towards postgraduate medical and non-medical training in clinical genetics.


We gratefully acknowledge funding from the Clinical Resources and Audit Group of the Scottish Executive Department of Health. One of the authors (NB) was employed with CRAG funds to coordinate this project across Scotland. The views expressed in this article represent those of the authors and not necessarily of the funding agency. We would like to thank all staff from the Departments of Clinical Genetics in Aberdeen, Dundee, Edinburgh, and Glasgow and those listed below for their active participation on this project. Finally, we thank Alison Stewart for publishing the guidelines and ICPs on the Public Health Genetics Network website ( pathways/care_pathways). The following contributed to the Scottish Clinical Genetics Guidelines Project.Tuberous sclerosis: Dr P Bolton, Nicola Bradshaw, P Brady, Dr C Brewer, Dr J K Brown, Dr H Campbell, Mrs C Falconer, Dr D FitzPatrick, Mrs N Garrett, A Hunt, Prof R MacKie, Dr R F Mueller, G Murray, Dr M O'Regan, Dr M Porteous, Dr F Rodgers, Prof J Stephenson, Mrs D Stirling, Dr W Treasurer, Dr M Watson.Marfan syndrome: Dr P Booth, N Bradshaw, Mr W Church, Mr J S Cockburn, Dr J Dean, Dr J A R Friend, Dr H Gregory, Prof N Haites, Dr M Hall, Dr E Hobson, Mr R R Jeffrey, Dr A Kidd, Dr Z Miedzabroska, Dr S Moore, Dr R Reid, Dr G Russell, Mr T R Scotland, Dr S Simpson, Dr E S Stockdale, Dr S Walton, Prof J Weir.Neurofibromatosis type I: L Arthur, Dr J Berg, N Bradshaw, Dr D Goudie, Dr R Grant, Mrs L McLeish, Dr J McMaster, Dr M F McNicol, Mrs D Ollis, Dr J E Robb.Huntington's disease: Prof D Blackwood, N Bradshaw, R Cetnarskyj, L Farmer, A Hamilton, S Lakeman, A Mayers, K McGarva, M McGill, P McLaughlin, Dr M Porteous, Dr S Simpson, B Smith, Dr G Thomas. Myotonic dystrophy: N Bradshaw, Dr Y Crow, Dr R Davidson, Dr S Hillis, Dr R McWilliam, Dr Patel, V Watters.Consensus team: Dr J Berg, N Bradshaw, Dr D Bonthron, Dr C Brewer, Dr H Campbell, R Cetnarskyj, A Crosbie, Dr Y Crow, Dr R Davidson, Dr J Dean, Dr D FitzPatrick, Dr D Goudie, Dr H Gregory, Prof N Haites, J Harlen, Dr E Hobson, Dr S Holloway, L Irvine, Dr A Kidd, M McJanet, M McGill, L McLeish, Dr Z Miedzabroska, Dr S Moore, D Ollis, P Pearson, Dr M Porteous, L Preston, K Sarson, Dr S Simpson, B Smith, Dr J Tolmie, Dr A Trainer, M Van Mourik, V Watters, Dr M Whiteford, Dr D Wilcox, Dr A Wright, D Young.