Abstract
Many studies are carried out in an effort to find factors that help explain the large unexplained variation in prognosis of breast cancer patients. The principles of good study design and analysis are less well appreciated for prognostic factor studies than for therapeutic trials. The oncology literature is full of results from studies of varying quality, often with conflicting findings. As a consequence, despite the large number of studies, there is still uncertainty about the importance of most prognostic factors. Few recently proposed prognostic factors for breast cancer have become widely accepted. This paper reviews the important methodological issues underlying such research. These issues are illustrated with examples from published studies and recent reviews of papers published in cancer journals. Guidelines are proposed for conducting and evaluating prognostic factor studies which should improve the quality of research in this important area.
Similar content being viewed by others
References
McGuire WL: Breast cancer prognostic factors: evaluation guidelines. J Natl Cancer Inst 83:154-155, 1991
Gasparini G, Pozza F, Harris AL: Evaluating the potential usefulness of new prognostic and predictive indicators in node negative breast cancer patients. J Natl Cancer Inst 85:1206-1219, 1993
Clark GM, Wenger CR, Beardslee S, et al: How to integrate steroid hormone receptor, flow cytometric, and other prognostic information in regard to primary breast cancer. Cancer 71:2157-2162, 1993
Simon R, Altman DG: Statistical aspects of prognostic factor studies in oncology. Br J Cancer 6:979-985, 1994
Gray-Donald K, Kramer MS: Causality inference in observational vs. experimental studies. An empirical comparison. Am J Epidemiol 127:885-892, 1988
Tukey JW: Some thoughts on clinical trials, especially problems of multiplicity. Science 198:679-684, 1977
Fayers PM, Machin D: Sample size: how many patients are necessary? Br J Cancer 72:1-9, 1995
Machin D, Campbell MJ, Fayers PM, Pinol APY: Sample Size Tables for Clinical Studies, 2nd edition. Blackwell, Oxford, 1997
Harrell FE, Lee KL, Matchar DB, Reichert TA: Regression models for prognostic prediction: advantages, problems, and suggested solutions. Cancer Treat Rep 69:1071-1077, 1985
Peduzzi P, Concato J, Feinstein AR, Holford TR: The importance of events per independent variable (EPV) in proportional hazards regression analysis. II. Accuracy and precision of regression estimates. J Clin Epidemiol 48:1503-1510, 1995
Altman DG, De Stavola BL, Love SB, Stepniewska KA: Review of survival analyses published in cancer journals. Br J Cancer 72:511-518, 1995
Axelsson K, Ljung B-ME, Moore DH, et al: Tumor angiogenesis as a prognostic assay for invasive ductal breast carcinoma. J Natl Cancer Inst 87:997-1008, 1995
Press MF, Hung G, Godolphin W, Slamon DJ: Sensitivity of HER-2/neu antibodies in archival tissue samples: potential source of error in immunohistochemical studies of oncogene expression. Cancer Res 54:2771-2777, 1994
Romero H, Schneider J: Different detection rates of HER-2/NEU overexpression in ovarian carcinoma using two different commercially available detection kits. Eur J Cancer 31A:1020-1021, 1995
Molino A, Micciolo R, Turazza M, et al: Prognostic significance of estrogen receptors in 405 primary breast cancers: a comparison of immunohistochemical and biochemical methods. Breast Cancer Res Treat 345:241-249, 1997
Bland JM, Altman DG: Statistical methods for comparing two methods of measurement. Lancet i:307-310, 1986
Cox DR: Regression models and life-tables. J R Stat Soc B 34:187-220, 1972
Fielding LP, Fenoglio-Preiser CM, Freedman LS: The future of prognostic factors in outcome prediction for patients with cancer. Cancer 70:2367-2377, 1992
Chen C-H, George SL: The bootstrap and identification of prognostic factors via Cox's proportional hazards regression model. Stat Med 4:39-46, 1985
Altman DG, Andersen PK: Bootstrap investigation of the stability of a Cox regression model. Stat Med 8:771-783, 1989
Sauerbrei W, Schumacher M: A bootstrap resampling procedure for model building: application to the Cox regression model. Stat Med 11:2093-2109, 1992
Schumacher M, Höllander, N, Sauerbrei W: Resampling and cross-validation techniques: a tool to reduce bias caused by model building? Stat Med 16:2813-2827, 1997
Gamel JW, McCurdy JB, McLean IW: A comparison of prognostic covariates for uveal melanoma. Invest Ophthalmol Vis Sci 33:1919-1922, 1992
Jenks S, Volkers N: Razors and refrigerators and reindeer — oh my. J Natl Cancer Inst 84:1863, 1992
Peters JM, Preston-Martin S, London SJ, et al: Processed meats and risk of childhood leukemia (California, USA). Cancer Causes Control 5:195-202, 1994
Wyatt JC, Altman DG: Prognostic models: clinically useful or quickly forgotten? Br Med J 311:1539-1541, 1995
Altman DG, Royston P: What do we mean by validating a prognostic model? Stat Med, in press
Vach W: Some issues in estimating the effect of prognostic factors from incomplete covariate information. Stat Med 16:57-72, 1997
Sagman U, Maki E, Evans WK, et al: Small-cell carcinoma of the lung: derivation of a prognostic staging system. J Clin Oncol 9:1639-1649, 1991
Thor A, Benz C, Moore D, et al: Stress response protein (srp-27) determination in primary human breast carcinomas: clinical, histologic, and prognostic correlations. J Natl Cancer Inst 83:170-178, 1991
Hart A, Wyatt J: Evaluating black boxes as medical decision-aids: issues arising from a study of neural networks. Med Informatics 15:229-236, 1990
Ohno-Machado L: A comparison of Cox proportional hazards and artificial network models for medical prognosis. Comput Biol Med 27:55-65, 1997
Schwarzer G, Vach W, Schumacher M: On the misuses of artificial neural networks for prognostic factor and diagnostic classification in oncology. University of Freiburg Technical Report No. 46, 1997
Morgan TM, Elashoff RM: Effect of categorizing a continuous covariate on the comparison of survival time. J Am Stat Assoc 81:917-921, 1986
Hilsenbeck SG, Clark GM, McGuire WL: Why do so many prognostic factors fail to pan out? Breast Cancer Res Treat 22:197-206, 1992
Altman DG, Lausen B, Sauerbrei W, Schumacher M: Dangers of using ‘optimal’ cutpoints in the evaluation of prognostic factors. J Natl Cancer Inst 86:829-835, 1994
Altman DG: Suboptimal analysis using ‘optimal’ cutpoints. Br J Cancer 78:556-557, 1998
Jänicke F, Schmitt M, Ulm K, et al: Urokinase-type plasminogen activator antigen and early relapse in breast cancer. Lancet 2:1049, 1989
Jänicke F, Schmitt M, Pache L, et al: Urokinase (uPA) and its inhibitor PAI-1 are strong and independent prognostic factors in node negative breast cancer. Breast Cancer Res Treat 24:195-208, 1993
Knoop A, Andreasen PA, Andersen JA, et al: Prognostic significance of urokinase-type plasminogen activator and plasminogen activator inhibitor-1 in primary breast cancer. Br J Cancer 77:932-940, 1998
Buettner P, Garbe C, Guggenmoos-Holzmann I: Problems in defining cutoff points of continuous prognostic factors: example of tumor thickness in primary cutaneous melanoma. J Clin Epidemiol 50:1201-1210, 1997
Budinha M, Skrk J, Zakotnik B, et al: Prognostic value of total cathepsin B in invasive ductal carcinoma of the breast. Eur J Cancer 31A:661-664, 1995
Durrleman S, Simon R: Flexible regression models with cubic splines. Stat Med 8:551-561, 1989
Hastie T, Sleeper L, Tibshirani R: Flexible covariate effects in the proportional hazards model. Breast Cancer Res Treat 22:241-250, 1992
Royston P, Altman DG: Regression using fractional polynomials of continuous covariates: parsimonious parametric modelling. Appl Stat 43:429-467, 1994
Knorr KL, Hilsenbeck SG, Wenger CR, et al: Making the most of your prognostic factors: presenting a more accurate survival model for breast cancer patients. Breast Cancer Res Treat 22:251-262, 1992
Buyse M: Analysis of clinical trial outcomes: some comments on subgroup analyses. Controlled Clin Trials 10:187S-194S, 1989
Henry JA, McCarthy AL, Angus B, et al: Prognostic significance of the estrogen-regulated protein, cathepsin D, in breast cancer. An immunohistochemical study. Cancer 65:265-271, 1990
Simon R: Confidence limits for reporting results of clinical trials. Ann Intern Med 105:429-435, 1986
Gardner MJ, Altman DG (eds) Statistics with Confidence. British Medical Journal, London, 1989
Simon R: Patient subsets and variation in therapeutic efficacy. Br J Clin Pharmacol 14:473-482, 1982
Aubele N, Auer G, Falkmer U, et al: Improved prognostication in small (pT1) breast cancers by image cytometry. Breast Cancer Res Treat 36:83-91, 1995
Harrell FE, Lee KL, Mark DB: Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 15:361-387, 1996
Haybittle JL, Blamey RW, Elston CW, et al: A prognostic index in primary breast cancer. Br J Cancer 45:361-366, 1982
Todd JH, Dowle C, Williams MR, et al: Confirmation of a prognostic index in primary breast cancer. Br J Cancer 56:489-492, 1987
Brown JM, Benson EA, Jones M: Confirmation of a long-term prognostic index in breast cancer. Breast 2:144-147, 1993
Stern JM, Simes RJ: Publication bias: evidence of delayed publication in a cohort study of clinical research projects. Br Med J 315:640-645, 1997
Ferrandina G, Scambia G, Bardelli F, et al: Relationship between cathepsin-D content and disease-free survival in node-negative breast cancer patients: a meta-analysis. Br J Cancer 76:661-666, 1997
Fox SB, Smith K, Hollyer J, et al: The epidermal growth factor receptor as a prognostic marker: results of 370 patients and review of 3009 patients. Breast Cancer Res Treat 29:41-49, 1994
Rawson NSB, Peto J: An overview of prognostic factors in small cell lung cancer. Br J Cancer 61:597-604, 1990
The International Non-Hodgkin's Lymphoma Prognostic Factors Project: A predictive model for aggressive lymphoma. N Engl J Med 329:987-994, 1993
Rights and permissions
About this article
Cite this article
Altman, D.G., Lyman, G.H. Methodological challenges in the evaluation of prognostic factors in breast cancer. Breast Cancer Res Treat 52, 289–303 (1998). https://doi.org/10.1023/A:1006193704132
Issue Date:
DOI: https://doi.org/10.1023/A:1006193704132