Use of Framingham risk score and new biomarkers to predict cardiovascular mortality in older people: population based observational cohort study
BMJ 2009; 338 doi: https://doi.org/10.1136/bmj.a3083 (Published 09 January 2009) Cite this as: BMJ 2009;338:a3083
- Wouter de Ruijter, general practitioner and clinical researcher1,
- Rudi G J Westendorp, professor2,
- Willem J J Assendelft, professor1,
- Wendy P J den Elzen, clinical researcher1,
- Anton J M de Craen, senior epidemiologist2,
- Saskia le Cessie, statistician3,
- Jacobijn Gussekloo, professor1
- 1Leiden University Medical Center, Department of Public Health and Primary Care (V0-P), PO Box 9600, 2300 RC Leiden, Netherlands
- 2Leiden University Medical Center, Department of Gerontology and Geriatrics (C2-R), PO Box 9600, 2300 RC Leiden, Netherlands
- 3Leiden University Medical Center, Department of Medical Statistics (S5-P), PO Box 9600, 2300 RC Leiden, Netherlands
- Correspondence to: W de Ruijter w.de_ruijter{at}lumc.nl
- Accepted 20 October 2008
Abstract
Objectives To investigate the performance of classic risk factors, and of some new biomarkers, in predicting cardiovascular mortality in very old people from the general population with no history of cardiovascular disease.
Design The Leiden 85-plus Study (1997-2004) is an observational prospective cohort study with 5 years of follow-up.
Setting General population of the city of Leiden, the Netherlands.
Participants Population based sample of participants aged 85 years (215 women and 87 men) with no history of cardiovascular disease; no other exclusion criteria.
Main measurements Cause specific mortality was registered during follow-up. All classic risk factors included in the Framingham risk score (sex, systolic blood pressure, total and high density lipoprotein cholesterol, diabetes mellitus, smoking and electrocardiogram based left ventricular hypertrophy), as well as plasma concentrations of the new biomarkers homocysteine, folic acid, C reactive protein, and interleukin 6, were assessed at baseline.
Results During follow-up, 108 of the 302 participants died; 32% (35/108) of deaths were from cardiovascular causes. Classic risk factors did not predict cardiovascular mortality when used in the Framingham risk score (area under receiver operating characteristic curve 0.53, 95% confidence interval 0.42 to 0.63) or in a newly calibrated model (0.53, 0.43 to 0.64). Of the new biomarkers studied, homocysteine had most predictive power (0.65, 0.55 to 0.75). Entering any additional risk factor or combination of factors into the homocysteine prediction model did not increase its discriminative power.
Conclusions In very old people from the general population with no history of cardiovascular disease, concentrations of homocysteine alone can accurately identify those at high risk of cardiovascular mortality, whereas classic risk factors included in the Framingham risk score do not. These preliminary findings warrant validation in a separate cohort.
Introduction
The prevalence and incidence of cardiovascular disease increases exponentially with age.1 2 3 The absolute numbers of cardiovascular events and deaths that could theoretically be prevented in older people are therefore substantial, although reductions in relative risk might be small (the geriatric paradox).4 Unsurprisingly, some authors have argued strongly for cardiovascular preventive measures in this age group.5 6 7
Selection of older people for secondary prevention is straightforward, since a history of cardiovascular disease itself is the strongest predictor of future cardiovascular morbidity and mortality.8 In primary prevention, patients are identified according to the classic risk factors for cardiovascular disease, including age, sex, systolic blood pressure, total and high density lipoprotein cholesterol, diabetes mellitus, smoking, and electrocardiogram based left ventricular hypertrophy.9 10 11 The Framingham risk score, which includes these classic risk factors, was originally validated for people aged up to 75 years, but has nevertheless been much used in older populations, in the absence of an appropriate alternative.12 13 However, the power of classic risk factors to accurately predict risk of cardiovascular disease seems to diminish with advancing age.11 14 Observational studies in the oldest people (≥85 years) have shown that some of these risk factors become nebulous, or even act in the reverse direction, at this age.15 16 17 18 19 The debate on the low efficacy of primary preventive interventions targeted at classic risk factors in this age group is also ongoing, fuelling the argument that current methods of predicting cardiovascular risk in the oldest people might not be effective.20 21 22 23 24
Several new biomarkers are effective indicators of high risk of cardiovascular disease—namely C reactive protein, folic acid, interleukin 6, homocysteine, fibrinogen, cystatin C, troponin I, various lipoproteins and apolipoproteins, and natriuretic peptides. Their incremental predictive value beyond that of classic risk factors, however, is generally small.25 26 27 28 29 30 31 32 33 34 35 Since the predictive value of classic risk factors wanes with age, we postulated that new biomarkers might be more effective in older populations, not only when added to classic risk factors, but also in isolation. The predictive value of individual new biomarkers in older populations has rarely been studied, and findings were mostly inconclusive, except for raised concentrations of homocysteine and N-terminal pro-brain natriuretic peptide, which are both associated with an increased risk of cardiovascular disease in older people.36 37 38 39 Recently, however, a study in a cohort of men aged 71 years found that the addition of four new biomarkers to a model with classic risk factors significantly improved prediction of death from cardiovascular causes.40 Data about the performance of combinations of new biomarkers in isolation from classic risk factors are still non-existent.
To improve strategies for primary prevention of cardiovascular disease in the oldest people, we examined ways to identify those at high risk by assessing the performance of classic risk factors and some new biomarkers as predictors of cardiovascular mortality over five years in people without cardiovascular disease at age 85 years.
Methods
Study design, setting and population
The Leiden 85-plus Study is an observational, prospective, population based cohort study of inhabitants of the city of Leiden, the Netherlands. Its general aim is to study determinants of successful ageing in the general population of the oldest people. Between September 1997 and September 1999, 705 people in the 1912-14 birth cohort reached the age of 85 years and were eligible to participate in the study. No exclusion criteria were used. Fourteen people died before enrollment; a total of 599 (87%) people gave informed consent to take part and were recruited to the study. All participants were visited at their homes, where they underwent face to face interviews, blood sampling, electrocardiography, and functional tests.
For the present study, we excluded all participants with a history of cardiovascular disease, including myocardial infarction, stroke, peripheral arterial disease (intermittent claudication or surgery for non-cardiac arterial disease), angina pectoris, or heart failure (n=250 of 599, 42%). To this end, we interviewed participants’ primary care physicians about these conditions. Since previously unrecognised (silent) myocardial infarctions are highly prevalent in old age,41 we decided to exclude an additional 22 participants because their baseline electrocardiogram showed evidence of a previous myocardial infarction (defined as the presence of Minnesota codes 1-1 or 1-2, excluding code 1-2-8).42 43 Finally, 25 participants were excluded because of missing data for classic risk factors or new biomarkers, leading to a final sample size of n=302.
Cardiovascular mortality
All participants were followed up for mortality until age 90 years; none was lost to follow-up. Dates of death were obtained from municipality records. Specific data on causes of death were obtained from Statistics Netherlands, where all national death certificates are coded according to the International Classification of Diseases and Related Disorders, 10th revision.44 Causes of death were divided into cardiovascular causes (codes I00-I99) and non-cardiovascular causes (all other codes). Assignment of cause of death was independent of the risk factors that were determined in participants at age 85 years.
Classic risk factors in participants at age 85 years
Systolic blood pressure
Blood pressure was measured on two occasions with a mean interval of two weeks. Systolic blood pressure was recorded at the onset of Korotkoff phase I. The mean of the measured systolic values was used for analyses.
Total and high density lipoprotein cholesterol
Blood samples were obtained for measurement of serum concentrations of total cholesterol and high density lipoprotein, which were analysed on fully automated computerised analysers (Hitachi 747 and 911; Hitachi, Tokyo, Japan).
Smoking
All participants were interviewed about present and past smoking habits. Current and past smokers of cigarettes, cigars, and pipes were judged to have a history of smoking.
Diabetes mellitus
Diabetes mellitus was considered present when the primary care physician stated so, when non-fasting glucose concentrations were greater than 11.0 mmol/l, or when, according to pharmacist records, a participant was taking diabetes medication.
Electrocardiogram based left ventricular hypertrophy
Electrocardiograms were recorded on a Siemens Sicard 440 (Erlangen, Germany) and transmitted to the electrocardiograms core laboratory in the Glasgow Royal Infirmary for automated Minnesota coding.42 All electrocardiograms were reviewed to exclude coding errors due to technical causes. Left ventricular hypertrophy was defined by Minnesota codes 310, 330, or 340.
New biomarkers in participants at age 85 years
We selected four of the new biomarkers that had been measured in the Leiden 85-plus Study: homocysteine and folic acid from the methionine-homocysteine pathway, and C reactive protein and interleukin 6 as markers of inflammation.
Homocysteine
Concentrations of homocysteine were measured in plasma samples with a fluorescence polarisation immunoassay after reduction to the free form with an IMx analyzer (Abbott, Abbott Park, IL, USA; coefficient of variation 2.2-2.5%).
Folic acid
Serum concentrations of folic acid were measured using the dual count solid phase no boil assay (Diagnostic Products Corporation, Los Angeles, CA, USA; coefficient of variation 4.7-7.3%).
C reactive protein
Plasma concentrations of C reactive protein were measured using a fully automated Hitachi 747 analyzer (Hitachi, Tokyo, Japan; detection limit 1 mg/l; coefficient of variation <5%).
Interleukin 6
Serum concentrations of interleukin 6 were measured using a standard enzyme linked immunosorbent assay (Sanquin, Amsterdam, the Netherlands, detection limit 4 pg/ml, coefficient of variation 5-10%).
Data analysis
For every participant, we calculated the Framingham risk score using the modified classic Framingham equation, with 5 year cardiovascular mortality as the endpoint (as opposed to 10 year incidence of coronary heart disease) and including the weighted risk factors age, sex, systolic blood pressure, total and high density lipoprotein cholesterol concentrations, smoking, diabetes mellitus, and electrocardiogram based left ventricular hypertrophy.13 Since all participants were 85 years old, age did not contribute to differences in absolute risk scores. We assigned participants to high risk, intermediate risk, and low risk groups based on tertiles of the calculated Framingham risk scores. We repeated analyses with different cut-off values for risk scores (quartiles, deciles).
We constructed seven new prediction models, each using various combinations of classic risk factors and new biomarkers: (1) all classic risk factors in one model (sex, systolic blood pressure, total cholesterol, high density lipoprotein cholesterol, diabetes mellitus, present or past smoking, and electrocardiogram based left ventricular hypertrophy); (2) homocysteine concentration and sex; (3) folic acid concentration and sex; (4) C reactive protein concentration and sex; (5) interleukin 6 concentration and sex; (6) homocysteine concentration plus all classic risk factors; and (7) all four new biomarkers and sex. We entered each combination of risk factors simultaneously in a Cox proportional hazards model, and, for each model, noted for each participant the linear predictor score (X-β), which represented their individual predicted risk of cardiovascular mortality during the 5 year follow-up period. We assigned participants to a high, intermediate, or low risk group on the basis of tertiles of the calculated X-β values for each model.
We assessed the performance of the different prediction models with three methods.45 First, the tertiles of Framingham risk scores were compared with the observed 5 year cardiovascular mortality using Kaplan-Meier plots and the log rank test. Hazard ratios with corresponding 95% confidence intervals (3rd tertile versus 1st tertile as reference) were calculated using Cox proportional hazard models. Second, using the continuous risk scores from each model, receiver operating characteristic curves with corresponding areas under the curves (neutral value 0.50= risk prediction by pure chance) and 95% confidence intervals were constructed, using cardiovascular mortality versus non-cardiovascular mortality or survival as the outcome. Third, the prediction model based on homocysteine concentration was compared with the Framingham risk score by calculating, for each method, the correctly classified number of participants—those in the high risk tertile who actually died from cardiovascular causes during follow-up. Of the participants who actually died, we calculated the net percentage of participants who were wrongly assigned to the intermediate or low risk group by the Framingham risk score, but correctly assigned to the high risk group by the model based on homocysteine concentration, and we tested this percentage for significance using McNemar’s test.46
Since the aim of the study was to assess the predictive performance of the risk factors and biomarkers, and not to investigate causes of disease, we made no adjustments for potential confounders. To investigate the validity of the results derived from the prediction model based on homocysteine, the first two methods were repeated using cross validated X-β values obtained by the jackknife method.47 We analysed data with SPSS 12.0 for Windows (SPSS, Chicago, IL, USA). The reporting of this observational study followed guidelines from the STROBE statement.48
Results
The table⇓ summarises the baseline characteristics and cardiovascular risk factors of the 302 participants with no history of cardiovascular disease. The majority were women, did not live in institutions, and performed well on the mini-mental state examination. Participants’ self reported quality of life was high. Of the 302 participants, 108 (36%) died during the 5-year follow-up period; 35 (32%) of these deaths were caused by cardiovascular disease.
Baseline characteristics of participants with no history of cardiovascular disease at age 85 years (n=302)
Figure 1⇓ shows 5-year cardiovascular mortality depending on risk tertiles based on the Framingham risk score. We found no differences in cardiovascular mortality between the risk categories (risk ratio for high v low risk category 1.2, 95% confidence interval 0.51 to 2.6). When the Framingham risk factors were entered as separate variables in a Cox proportional hazard model to create a prediction model calibrated for very old people, three new risk categories were obtained, but these categories also did not predict the observed 5-year cardiovascular mortality (fig 1⇓; risk ratio for high v low risk category 1.3, 95% confidence interval 0.57 to 2.7). Accuracy of the model did not improve when the group was divided at different intervals (risk quartiles, risk deciles) or after adjustment for use of antihypertensive medication (data not shown).
Fig 1 Cumulative cardiovascular mortality depending on tertiles of risk obtained from classic risk factors, weighted on the basis of Framingham risk score, or newly calibrated risk score from the Leiden 85-plus Study
Figure 2⇓ shows the performance of the sex adjusted prediction models based on homocysteine, folic acid, C reactive protein, and interleukin 6. Only the model based on homocysteine resulted in significant differences between the risk categories (log rank test, P=0.002); the high risk category had a 3.4-fold (95% confidence interval 1.4 to 8.1) increased risk of cardiovascular mortality compared with the low risk category. This risk did not change after adjustment for creatinin clearance. The high risk category in the model based on folic acid had a 2.2-fold (1.0 to 5.0) increased risk of cardiovascular mortality compared with the low risk category. The model including both homocysteine and folic acid (data not shown in figure 2) showed no improvement of risk prediction beyond the model with only homocysteine; the high risk category had a 1.9-fold (1.2 to 2.9) increased risk of cardiovascular mortality compared with the low risk category.
Fig 2 Cumulative cardiovascular mortality depending on tertiles of risk obtained from prediction models using homocysteine, folic acid, C reactive protein and interleukin 6
When the predictive value of the Framingham risk score and the risk model based on homocysteine alone were compared by receiver operating characteristic curves (figure 3),⇓ the area under the curve for the Framingham risk score was 0.53 (95% confidence interval 0.42 to 0.63) and that for the homocysteine-based model was 0.65 (0.55 to 0.75). For the new model, using classic risk factors and calibrated for very old people, the area under the curve was the same as that for the Framingham risk score at 0.53 (0.43 to 0.64, receiver operating characteristic curve not shown). Combining the Framingham risk score and the model based on homocysteine did not increase discriminative power (area under the curve 0.65, 95% confidence interval 0.54 to 0.75), nor did the power increase for the model based on a combination of all four new biomarkers (area under the curve 0.65, 95% confidence interval 0.55 to 0.75, receiver operating characteristic curve not shown).
Fig 3 Receiver operating characteristic curves showing performance of three prediction models for 5 year cardiovascular mortality based on Framingham risk score, homocysteine concentrations only, and homocysteine concentrations plus Framingham risk score
Of the 35 participants who died from cardiovascular disease during the 5 year follow-up period, the Framingham risk score classified 12 participants in the high risk tertile. According to the model based on homocysteine, 20 of the 35 participants were classified in the high risk tertile. Thus, compared with the Framingham risk score, classification of risk on the basis of homocysteine concentrations alone resulted in a 23% increase in identification of individuals at high risk (P=0.045).46
The performance of the model based on homocysteine did not change after cross validation by the jackknife method using data from the Leiden 85-plus Study (risk ratio for high v low risk category 3.0, 95% confidence interval 1.3 to 7.3; data not shown). Cross validation did not materially change the discriminative power of the homocysteine based model, as derived from the receiver operating characteristic curve (area under the curve 0.62, 95% confidence interval 0.52 to 0.73, receiver operating characteristic curve not shown).
Discussion
Principal findings
In the cohort of the Leiden 85-plus Study, cardiovascular mortality in people at the age of 85 with no history of cardiovascular disease was not accurately predicted by classic risk factors such as those included in the Framingham risk score. By contrast, a single measurement of homocysteine accurately identified those at high risk of cardiovascular mortality. These results suggest that in this age group, risk identification for primary prevention of cardiovascular disease should be based not on classic risk factors anymore, but on plasma concentrations of homocysteine.
For decades the Framingham risk score has been used to predict the 10 year risk of developing coronary heart disease in people with no history of cardiovascular disease.49 Although it has certainly stood the test of time for its original purpose, our study confirms earlier findings of a fall in the predictive abilities of this risk score in older populations.11 14 Therefore, the finding of a more accurate measure of cardiovascular disease risk is very relevant to this population.
The strong association between raised concentrations of homocysteine and cardiovascular morbidity and mortality has been recognised since 1969,50 and has been repeatedly described in various populations.29 31 36 51 52 53 We have shown that homocysteine, unlike the classic risk factors, remains a potent predictor of risk in older people. Its predictive power goes beyond that of classic risk factors, and is equally robust without classic risk factors being included in the model. These features have rarely been demonstrated in new biomarkers for cardiovascular disease.54
By contrast, we were unable to confirm the associations found in younger age groups between cardiovascular disease and concentrations of C reactive protein, interleukin 6, and folic acid.26 30 33 55 The weakness of these markers of inflammation, specifically C reactive protein and interleukin 6, in predicting cardiovascular mortality in the oldest people is in line with findings of the largest prospective study dedicated to C reactive protein and cardiovascular disease in older people to date.56
Strengths and weaknesses
Our study has several strengths. First, the Leiden 85-plus Study is an observational, prospective study of oldest people, in which 87% of the general population participated and follow-up on mortality was complete. These factors add to the external validity of our results. Indeed, given that this age group is the fastest growing segment of the general population in industrialised societies, the findings of the present study might be particularly important. Furthermore our study participants generally had good cognitive function, were mainly not living in institutions, and were satisfied with their quality of life.57 Primary prevention of cardiovascular mortality in these very old people may lengthen independent and contented living, and to this end effective identification of individuals at high risk is essential. Another strength of our study is the consistency of the results across all statistical methods, including a crossvalidation, which supports the validity of our findings.45 46 58
A weakness of our study could be the limited number of new biomarkers that we selected to investigate. Other biomarkers, as well as new imaging techniques such as measurement of coronary artery calcium,59 might have predictive value equal to that of homocysteine in this population. A second limitation of our study is its relatively small size. Despite a high rate of events (35/302, 12%), a type 2 error (failure to reject a null hypothesis that is in fact false) cannot be ruled out. We therefore recommend validation of our findings in a larger cohort. Another potential weakness could have been differential misclassification in causes of death, especially in participants with known classic risk factors.60 However, such inconsistencies would have led to greater differences between risk tertiles based on the Framingham risk score. Laboratory results for the new biomarkers were not reported to clinicians, and therefore any misclassification was independent of these test results (non-differential misclassification).
Clinical implications and future research
We studied the best way to identify patients at high risk (“true prediction”), not the causes that underlie the observed associations. Hence, although homocysteine accurately predicts cardiovascular mortality in very old age, we do not suggest that lowering homocysteine will be beneficial; in fact, so far this approach has been shown to be ineffective.61 62 63 With regard to choosing effective methods for primary prevention of cardiovascular disease, the role of statins needs to be explored, since their beneficial effect in secondary prevention in old age is evident.21 64 Since the underlying atherosclerotic process is unlikely to change with age, statins could also be effective in primary prevention, if the selection of patients at high risk is accurate.
Conclusions
In this study, a model based on homocysteine concentration alone was a better predictor of cardiovascular mortality in very old people with no history of cardiovascular disease than were models based on classic risk factors. These preliminary findings call for validation in a separate cohort and, if confirmed, could eventually lead to a revision of current guidelines and corresponding indicators of quality of care.
What is already known on this subject
The Framingham risk score, based on classic risk factors for cardiovascular disease, is still frequently used to estimate risk in older people
The predictive value of these classic risk factors, such as hypercholesterolemia and systolic hypertension, weakens with age
In the past four decades new biomarkers have been identified that have clear associations with incident cardiovascular disease
What this study adds
In very old age, classic risk factors as included in the Framingham risk score no longer predict 5 year cardiovascular mortality in people with no history of cardiovascular disease
By contrast, a single homocysteine measurement could accurately identify older individuals who are at high risk of cardiovascular mortality
Plasma concentrations of homocysteine, rather than classic risk factors, could potentially be used to select older people for primary preventive interventions
These findings should be validated in a separate cohort
Notes
Cite this as: BMJ 2009;338:a3083
Footnotes
Contributors: WDR analysed and interpreted data, drafted the manuscript, and did statistical analysis. RGJW was responsible for study concept and design, acquisition of data, analysis and interpretation of data, critical revision of the manuscript for important intellectual content, and obtaining funding. WJJA analysed and interpreted data and critically revised the manuscript for important intellectual content. WPJDE acquired, analysed, and interpreted data, drafted the manuscript, and analysed statistics. AJMDC was responsible for study concept and design, acquisition of data, analysis and interpretation of data, and critical revision of the manuscript for important intellectual content. SLC analysed and interpreted data, critically revised the manuscript for important intellectual content, and did statistical analysis. JG was responsible for study concept and design, acquisition of data, analysis and interpretation of data, critical revision of the manuscript for important intellectual content, and obtaining funding. JG is guarantor.
Funding: The Leiden 85-plus Study was partly funded by an unrestricted grant from the Dutch Ministry of Health, Welfare and Sports. The funder played no role in study design, collection, analysis and interpretation of data, writing of the report, nor in the decision to submit the article for publication.
Independence of researchers: All researchers were independent from the funder.
Competing interests: None declared.
Ethical approval: The medical ethical committee of Leiden University Medical Center approved the study.
Provenance and peer review: Not commissioned, externally peer reviewed.
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.