Top

Osteoporosis International

Published in:

01-02-2012 | Opinion Paper

Pitfalls in the external validation of FRAX

Authors: J. A. Kanis, A. Oden, H. Johansson, E. McCloskey

Published in: Osteoporosis International | Issue 2/2012

Abstract

Summary

Recent studies have evaluated the performance of FRAX® in independent cohorts. The interpretation of most is problematic for reasons summarised in this perspective.

Introduction

FRAX is an extensively validated assessment tool for the prediction of fracture in men and women. The aim of this study was to review the methods used since the launch of FRAX to further evaluate this instrument.

Methods

This covers a critical review of studies investigating the calibration of FRAX or assessing its performance characteristics in external cohorts.

Results

Most studies used inappropriate methodologies to compare the performance characteristics of FRAX with other models. These included discordant parameters of risk (comparing incidence with probabilities), comparison with internally derived predictors and inappropriate use and interpretation of receiver operating characteristic curves. These deficits markedly impair interpretation of these studies.

Conclusion

Cohort studies that have evaluated the performance of FRAX need to be interpreted with caution and preferably re-evaluated.

Available only for authorised users

Hippisley-Cox J, Coupland C (2009) Predicting risk of osteoporotic fracture in men and women in England and Wales: prospective derivation and validation of QFractures Scores. Br Med J 339:b4229CrossRef

Chen P, Krege JH, Adachi JD et al (2009) Vertebral fracture status and the World Health Organization risk factors for predicting osteoporotic fracture risk. J Bone Miner 24:495–502CrossRef

Sornay-Rendu E, Munoz F, Delmas PD, Chapurlat RD (2010) The FRAX® tool in French women: how well does it describe the real incidence of fracture in the OFELY cohort. J Bone Miner Res 25:2101–2107PubMedCrossRef

Bolland MJ, Siu AT, Mason BH et al (2010) (2011) Evaluation of the FRAX and Garvan fracture risk calculators in older women. J Bone Miner Res 26:420–427CrossRef

Ensrud KE, Lui LY, Taylor BC et al (2009) A comparison of prediction models for fractures in older women: is more better? Arch Intern Med 169:2087–2094PubMedCrossRef

Tremollieres FA, Pouilles JM, Drewniak N, Laparra J, Ribot CA, Dargent-Molina P (2010) Fracture risk prediction using BMD and clinical risk factors in early postmenopausal women: sensitivity of the WHO FRAX tool. J Bone Miner Res 25:1002–1009PubMedCrossRef

Donaldson MG, Palermo L, Schousboe JT, Ensrud KE, Hochberg MC, Cummings SR (2009) FRAX and risk of vertebral fractures: the fracture intervention trial. J Bone Miner Res 24:1793–1799PubMedCrossRef

Hillier TA, Cauley JA, Rizzo JH (2011) The WHO absolute fracture risk models (FRAX): do clinical risk factors improve fracture prediction in older women without osteoporosis? J Bone Miner Res 26:1774–1782PubMedCrossRef

Kayan K, Johansson H, Oden A et al (2009) Can fall risk be incorporated into fracture risk assessment algorithms: a pilot study of responsiveness to clodronate. Osteoporos Int 20:2055–2061PubMedCrossRef

10.

Pluskiewicz W, Adamczyk P, Franek E et al (2010) Ten-year probability of osteoporotic fracture in 2012 Polish women assessed by FRAX and nomogram by Nguyen et al.—conformity between methods and their clinical utility. Bone 46:1661–1667PubMedCrossRef

11.

Kanis JA, Oden A, Johansson H, McCloskey EV (2008) Expressing fracture risk (letter). Osteoporos Int 19:593–594CrossRef

12.

Kanis JA, Oden A, Johnell O et al (2007) The use of clinical risk factors enhances the performance of BMD in the prediction of hip and osteoporotic fractures in men and women. Osteoporos Int 18:1033–1046PubMedCrossRef

13.

Kanis JA on behalf of the World Health Organization Scientific Group (2008) Assessment of osteoporosis at the primary health-care level. Technical Report. WHO Collaborating Centre, University of Sheffield, UK. Available at http://www.shef.ac.uk/FRAX/

14.

Papaioannou A, Morin S, Cheung AM, Scientific Advisory Council of Osteoporosis Canada (2010) 2010 clinical practice guidelines for the diagnosis and management of osteoporosis in Canada: summary. CMAJ 23(182):864–873

15.

Dawson-Hughes B, National Osteoporosis Foundation Guide Committee (2008) A revised clinician’s guide to the prevention and treatment of osteoporosis. J Clin Endocrinol Metab 93:2463–2465PubMedCrossRef

16.

Grossman JM, Gordon R, Ranganath VK et al (2010) American College of Rheumatology 2010 recommendations for the prevention and treatment of glucocorticoid-induced osteoporosis. Arthritis Care Res (Hoboken) 62:1515–1526CrossRef

17.

National Osteoporosis Foundation (2008) Clinician’s guide to prevention and treatment of osteoporosis. National Osteoporosis Foundation. www.nof.org, Washington, DC

18.

Pencina MJ, D’Agostino RB, Sr RB, D’Agostino RB Jr, Vasan RS (2008) Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond. Statist Med 27:157–172CrossRef

19.

Kanis JA, Johnell O, Oden A et al (2006) The use of multiple sites for the diagnosis of osteoporosis. Osteoporos Int 17:527–534PubMedCrossRef

20.

Blake GM, Patel R, Knapp KM, Fogelman I (2003) Does the combination of two BMD measurements improve fracture discrimination? J Bone Miner Res 18:1955–1963PubMedCrossRef

21.

Leslie WD, Lix LM, Tsang JF, Caetano PA, Program Manitoba Bone Density (2007) Single-site vs multisite bone density measurement for fracture prediction. Arch Intern Med 167:1641–1647PubMedCrossRef

22.

Leslie WD, Lix LM, Johansson H, Oden A, McCloskey E, Kanis JA (2010) Spine-hip discordance and fracture risk assessment: a physician-friendly FRAX enhancement. Osteoporos Int. Oct 20. [Epub ahead of print] PubMed PMID: 20959961

23.

Abrahamsen B, Vestergaard P, Rud B et al (2006) Ten-year absolute risk of osteoporotic fractures according to BMD T score at menopause: The Danish Osteoporosis Prevention Study. J Bone Miner Res 21:796–800PubMedCrossRef

24.

Baron JA, Barrett J, Malenka D et al (1994) Racial differences in fracture risk. Epidemiol 5:42–47CrossRef

25.

Jatrana S, Blakely T (2008) Ethnic inequalities in mortality among the elderly in New Zealand. Aust N Z J Public Health 32:437–443PubMedCrossRef

26.

Levine S, Makin M, Menczel J, Robin G, Naor E, Steinberg R (1970) Incidence of fractures of the proximal end of the femur in Jerusalem: a study of ethnic factors. J Bone Joint Surg Am 52:1193–1202PubMed

27.

Wittich A, Bagur A, Mautalen C et al (2010) Epidemiology of hip fracture in Tucuman, Argentina. Osteoporos Int 21:1803–1807PubMedCrossRef

28.

Elffors I, Allander E, Kanis JA et al (1994) The variable incidence of hip fracture in southern Europe: the MEDOS Study. Osteoporos Int 4:253–263PubMedCrossRef

29.

Jonsson B, Gardsell P, Johnell O, Redlund-Johnell I, Sernbo I (1992) Differences in fracture pattern between an urban and a rural population: a comparative population-based study in southern Sweden. Osteoporos Int 2:269–273PubMedCrossRef

30.

Finsen V, Benum P (1987) Changing incidence of hip fractures in rural and urban areas of central Norway. Clin Orthop Relat Res 218:104–110PubMed

31.

Bulajic-Kopjar M, Wiik J, Nordhagen R (1998) Regional differences in the incidence of femoral neck fractures in Norway. Tidsskr Nor Laegeforen 118:30–33PubMed

32.

Chevalley T, Herrmann FR, Delmi et al (2002) Evaluation of the age-adjusted incidence of hip fractures between urban and rural areas: the difference is not related to the prevalence of institutions for the elderly. Osteoporos Int 13:113–118PubMedCrossRef

33.

Matković V, Kostial K, Simonović I, Buzina R, Brodarec A, Nordin BE (1979) Bone status and fracture rates in two regions of Yugoslavia. Am J Clin Nutr 32:540–549PubMed

34.

Madhok R, Melton LJ 3rd, Atkinson EJ, O’Fallon WM, Lewallen DG (1993) Urban vs rural increase in hip fracture incidence. Age and sex of 901 cases 1980–89 in Olmsted County, U.S.A. Acta Orthop Scand 64:543–548PubMedCrossRef

35.

Kaastad TS, Meyer HE, Falch JA (2008) Incidence of hip fracture in Oslo, Norway: differences within the city. Bone 22:175–178CrossRef

36.

Kanis JA, Oden A, Johnell O, Jonsson B, de Laet C, Dawson A (2001) The burden of osteoporotic fractures: a method for setting intervention thresholds. Osteoporos Int 12:417–427PubMedCrossRef

37.

Melton LJ, Crowson CS, O’Fallon WM (1999) Fracture incidence in Olmsted County, Minnesota: comparison of urban and with rural rates and changes in urban rates over time. Osteoporos Int 9:29–37PubMedCrossRef

38.

Singer BR, McLauchlan CJ, Robinson CM, Christie J (1998) Epidemiology of fracture in 15,000 adults. The influence of age and gender. J Bone Joint Surg 80B:234–238

39.

Lippuner K, Johansson H, Kanis JA, Rizzoli R (2010) FRAX® assessment of osteoporotic fracture probability in Switzerland. Osteoporos Int 21:381–390PubMedCrossRef

40.

Sanders KM, Pasco JA, Ugoni AM et al (1998) The exclusion of high trauma fractures may underestimate the prevalence of bone fragility fractures in the community: the Geelong Osteoporosis Study. J Bone Miner Res 13:1337–1342PubMedCrossRef

41.

Melton LJ (1995) Epidemiology of fractures. In: Riggs BL, Melton LJ (eds) Osteoporosis: etiology, diagnosis and management, 2nd edn. Lippincott-Raven, Philadelphia, pp 225–227

42.

Johnell O, Gullberg B, Kanis JA (1997) The hospital burden of vertebral fracture. A study of national register sources. Osteoporos Int 7:138–144PubMedCrossRef

43.

Nguyen ND, Frost SA, Center JR, Eisman JA, Nguyen TV (2008) Development of prognostic nomograms for individualizing 5-year and 10-year fracture risks. Osteoporos Int 19:1431–1444PubMedCrossRef

44.

Nguyen ND, Frost SA, Center JR, Eisman JA, Nguyen TV (2007) Development of a nomogram for individualizing hip fracture risk in men and women. Osteoporos Int 18:1109–1117PubMedCrossRef

45.

Gillespie LD, Robertson MC, Gillespie WJ et al. (2009) Interventions for preventing falls in older people living in the community. Cochrane Database of Systematic Reviews 2009, Issue 2. Art.No.: D007146.doi:10.1002/14651858.CD007146.pub2

46.

McClung MR, Geusens P, Miller PD et al (2001) Effect of risedronate on the risk of hip fracture in elderly women. Hip Intervention Program Study Group. N Engl J Med 344:333–340PubMedCrossRef

47.

Langsetmo L, Nguyen TV, Nguyen ND et al (2010) Independent external validation of nomograms for predicting risk of low-trauma fracture and hip fracture. CMAJ. doi:10.1503/cmaj.100458

48.

Sk S, Nguyen ND, Center JR et al (2010) Prognosis of fracture: evaluation of predictive accuracy of the FRAX algorithm and Garvan nomogram. Osteoporos Int 21:863–871CrossRef

49.

Van Geel ACM, van den Bergh JPW, Dinant GJ, Geusens PP (2010) Individualizing fracture risk prediction. Maturitas 65:143–148PubMedCrossRef

50.

Geusens P, Van Geel T, Van Den Berg J (2010) Can hip fracture prediction in women be estimated beyond bone mineral density measurement alone. Ther Adv Musculoskelet Dis 2:63–67CrossRef

51.

Van Geel T, Huntjens K, Bours S et al (2010) Comparison of FRAX and Garvan case finding algorithms in patients presenting with a fracture. Bone 47:S188CrossRef

52.

Collins GS, Mallett S, Altman DG (2011) Predicting risk of osteoporotic and hip fracture in the United Kingdom: prospective independent and external validation of QFractureScores. BMJ 342:d3651. doi:10.1136/bmj.d3651

53.

Cummins NM, Poku EK, Towler MR, O'Driscoll OM, Ralston SH (2011) Clinical risk factors for osteoporosis in Ireland and the UK: a comparison of FRAX and QFractureScores. Calcif Tissue Int 89:172–177

54.

Fujiwara S, Hamaya E, Goto W, Masunari N, Furukawa K, Fukunaga M, Nakamura T, Miyauchi A, Chen P (2011) Vertebral fracture status and the World Health Organization risk factors for predicting osteoporotic fracture risk in Japan. Bone 49(3):520–525

Title: Pitfalls in the external validation of FRAX
Authors: J. A. Kanis
A. Oden
H. Johansson
E. McCloskey
Publication date: 01-02-2012
Publisher: Springer-Verlag
Published in: Osteoporosis International / Issue 2/2012
Print ISSN: 0937-941X
Electronic ISSN: 1433-2965
DOI: https://doi.org/10.1007/s00198-011-1846-0

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Pitfalls in the external validation of FRAX

Abstract

Summary

Introduction

Methods

Results

Conclusion

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Summary

Introduction

Methods

Results

Conclusion

Please log in to get access to this content

Other articles of this Issue 2/2012

QCT-based finite element models predict human vertebral strength in vitro significantly better than simulated DEXA

Risk factors for fracture in elderly men: a population-based prospective study

Serum 25(OH)D concentrations in sunny Israel

A study of relationships between bone-related vitamins and minerals, related risk markers, and subsequent mortality in older British people: the National Diet and Nutrition Survey of People Aged 65 Years and Over

Epidemiology and structural basis of racial differences in fragility fractures in Chinese and Caucasians

Treatment satisfaction and persistence among postmenopausal women on osteoporosis medications: 12-month results from POSSIBLE US™