Author + information
- Khurram Nasir, MD, MPH∗ ( )()
- Center for Healthcare Advancement and Outcomes, Baptist Health South Florida, Miami Florida; Miami Cardiovascular Institute, Baptist Health South Florida, Miami Florida; The Johns Hopkins Ciccarone Center for the Prevention of Heart Disease, Baltimore, Maryland; Department of Medicine, Herbert Wertheim College of Medicine, Florida International University, Miami Florida; and the Department of Epidemiology, Robert Stempel College of Public Health, Florida International University, Miami Florida
- ↵∗Reprint requests and correspondence:
Dr. Khurram Nasir, Center for Healthcare Advancement and Outcomes, Baptist Health South Florida, 150 San Remo Avenue, Suite 400, Coral Gables, Florida 33146.
Within the wider context of health care delivery sciences, the focus on medical decision making is a fairly recent occurrence. Dr. Lee Lusted (1), a pioneer in medical decision analytics, published the first major article emphasizing this theme in 1971. A few years later, New England Journal of Medicine (2) followed suit, by dedicating an entire issue to emerging concepts around decision making in medicine. In a classic 1976 editorial, “Is decision analysis useful in clinical medicine?” Ransohoff and Feinstein (3) questioned “Will decision analysis prove to be useful in clinic medicine? Is it worthwhile to learn about this new field?” During the 1970s, I reckon they probably could not predict how rapidly this concept would penetrate medical literature and particularly influence cardiovascular medicine. Over the next 3 decades, hardly any other topic has received as much attention as risk modeling to facilitate decision making for accurate prediction of the presence and severity of coronary artery disease (CAD).
In 1979, Diamond and Forrester (4) recommended principles of decision analysis by presenting a simple, classic model that considers age, sex, and type of chest pain to determine the likelihood of obstructive CAD guiding downstream management. Subsequently, multiple other risk scores were formulated accounting for additional risk factors, such as dyslipidemia, diabetes, and smoking; however, these yielded minimal additive predictive value. Considering the ease of administration and the lack of need for laboratory testing, the Diamond-Forrester model has stood the test of time and remains the most widely used algorithm for CAD risk assessment.
Although revascularization has been the cornerstone for managing obstructive, symptomatic CAD since the 1980s, parallel efforts are gradually shifting toward optimal medical management. There is an emerging consensus that the best candidates for revascularization are those patients with high-risk CAD features, rather than indiscriminately subjecting the entire gamut of obstructive coronary lesions to revascularization procedures. Although the Diamond-Forrester model serves extremely well for predicting those patients at risk for obstructive CAD, the performance of this model in predicting severe CAD is ambiguous, thus creating the need for a prognostic tool that can accurately identify those patients with severe CAD who are likely to derive the greatest benefit from revascularization. This is particularly desirable in the developing world, where appropriate allocation of limited health care resources cannot be overemphasized.
To this point, the efforts by Yang et al. (5) are admirable, because these investigators leveraged a multinational CONFIRM (COronary CT Angiography EvaluatioN For Clinical Outcomes: An InteRnational Multicenter) registry of patients undergoing coronary computed tomography angiography (CTA) and developed a clinical risk prediction model for identifying those patients with high-risk anatomic (HRA) CAD (defined as left main coronary artery diameter stenosis ≥50%, 3-vessel disease [diameter stenosis ≥70%] or 2-vessel disease involving the proximal left anterior descending artery]). Using clinical factors (age, sex, diabetes, hypertension, current smoking, hyperlipidemia, family history of CAD, history of peripheral vascular disease, and chest pain symptoms), the investigators employed robust statistical methods to derive a scoring system to predict HRA CAD in 24,000 patients undergoing CTA for suspected CAD. Similar to earlier models, patients were divided into 3 risk categories: low, intermediate, and high for pre-test probability. Overall, this risk estimation model closely predicted the absolute level of observed risk. The statistical metric often used for calibration, namely the Hosmer-Lemeshow chi-square test, demonstrated an adequate model fit. Notably, the C-statistic was particularly superior for the newly proposed algorithm compared with the Diamond-Forrester model in discriminating those patients with HRA features (5). The results were subsequently confirmed in 7,000 nonoverlapping patients in a validation cohort.
Before advocating widespread implementation of this newly proposed model, one needs to ask whether this adds value to the existing medical decision-making processes. Although the study provides sophisticated and robust statistical analyses, there is limited practical information for clinicians who are keen on understanding the translational implications of this model in every day practice. From a health care delivery perspective in managing patients with suspected CAD, it is crucial to clarify the model’s additive role in influencing choices and to recommend standards of application in regular practice. Here are a few questions that may help facilitate dialogue among varied stakeholders concerning the broader utility of this decision analysis tool: 1) How many patients in this cohort had high-risk CAD features? 2) What proportion of those features were classified as ‘high risk’ after employing the proposed risk model? 3) Once classified as high risk, what is the actual likelihood for the presence of HRA features?
Tables 1 and 5 of the paper by Yang et al. (5) in this issue of iJACC help summarize some of this salient information. First, it is noteworthy that only 4% to 5% patients in the entire derivation and cohort demonstrated features of HRA CAD (see Table 1 in Yang et al. ). Second, even though being classified as “high risk” according to the proposed model yields nearly flawless specificity and a positive likelihood ratio for predicting HRA CAD, <1% of the study cohort belonged to this risk category (see Table 5 in Yang et al. ). Finally, once categorized as “high risk” by using the proposed approach, merely one-fourth of the derivation cohort exhibited features of high-risk CAD. To complicate matters, subsequent application of this approach in the validation cohort further reduces the prevalence of HRA features by 30% in the high-risk group.
To understand the potential translational value of this model better, we can create an experimental situation by applying the proposed risk prediction score to 1,000 hypothetical patients who share baseline characteristics with those seen in the current CONFIRM registry. Based on the estimates provided in Table 5 in Yang et al. (5), by using this risk prediction score, only 7 of 1,000 patients will be categorized as “high risk” in which one can be persuaded to forgo noninvasive testing and proceed directly to invasive coronary angiography. Subsequently, only 1 of 7 of these patients will ultimately demonstrate any features of HRA CAD. Alternatively, as Yang et al. (5) wrote, if the goals are to consider an optimal medical therapy trial in absence of HRA features and avoid downstream interventions, then the notion that further noninvasive testing should be considered in two-thirds of the cohort deemed at intermediate risk is debatable at best because the prevalence of high-risk CAD in this specific group is only 5%. Given the previously stated challenges, the overall applicability of this seemingly promising approach is mitigated and warrants careful scrutiny.
At this juncture, it is worth redirecting attention to pursue a root cause analysis of the discrepancies between strong statistical significance and weak clinical relevance observed in this study. The first striking element is the heterogeneous population mix. One-fourth of the CONFIRM registry participants are asymptomatic, which lowers overall applicability, because cardiologists are primarily confronted with symptomatic patients. For example, a 70-year asymptomatic man who has a family history of coronary heart disease, is a smoker, is diabetic with mild dyslipidemia, and has well-controlled hypertension will be classified as high risk by the proposed risk prediction model. Supported by evidence and guidelines, invasive angiography or revascularization would rarely be considered as a first choice in asymptomatic patients.
The second factor that may explain the limited clinical significance of this risk prediction model is the relatively low value assigned to the presence and severity of classic angina symptoms, which to date remain the central factors influencing management decisions. In the derivation of the risk prediction model, similar weight is given to typical angina and to any of the following conventional risk factors for cardiovascular disease: diabetes, smoking, family history of CHD. A 66-year-old woman with history of smoking, diabetes and known peripheral vascular disease who is presenting with typical anginal pain will be at best considered as having intermediate risk by the current algorithm. This is certainly contrary to conventional wisdom. This reminds me of a question posed by George Diamond (6) a few years ago, “Do symptoms matter?” While explaining the significant discrepancies between HRA CAD lesions and clinical symptoms noted in an earlier study from the same cohort, Diamond (6) eloquently attributed the differences to the following reasons: the inability of a self-administered patient questionnaire to capture the true essence of symptom severity; the inclusion in the CONFIRM registry of features such as jaw pain and arm pain in the definition of typical angina that would be considered atypical using the traditional algorithm; and the absence of patients in the current cohort undergoing coronary CTA and who presented with extreme manifestations of ischemia who would have likely been referred for invasive angiography, thereby resulting in preferential referral bias from patients who are at the lower end on the severity spectrum of ischemia.
These discussions have led me to wonder ‘what if’ a risk prediction model was developed using an appropriately high-risk cohort that excluded asymptomatic patients as well as using additional variables not accounted for in the current study? In a recent study of 551 patients presenting with typical chest pain who proceeded to invasive coronary angiography, Chen et al. (7) found that age, male sex, aortic valve calcification on echocardiography, an abnormal electrocardiogram, diabetes, hyperlipidemia, and dyslipidemia independently predicted CAD severity. Furthermore, when these variables were used to develop a risk prediction scoring system, consistent with the current study, the system performed well ‘statistically’ on calibration metrics and significantly discriminated those patients with and without severe CAD compared with the Diamond-Forrester model. However, the results of Chen et al. (7) compared with the current study by Yang et al. (5) were distinctly different where it mattered the most. First, a significant proportion of patients in the study by Chen et al. (7) was identified as high risk on the scoring metrics (57% vs. 1%). Second, the likelihood that these patients had high-risk CAD features was significantly greater (90% vs. 17% to 24%). These consequential disparities noted could have been avoided if the ‘right’ question had been asked in the ‘right’ population.
In summary, considering the evolving landscape of CAD management, the broad scope of application and the grave consequences of misclassification underscore the continued quest to refine existing risk assessment tools. The current study and similar efforts are the first steps toward the ultimate goal. At the same time, certain key considerations, highlighted earlier, are worth careful scrutiny before embarking on similar subsequent endeavors. Nevertheless, the results from Yang et al. (5) provide the necessary momentum that must carry forward, with cautious optimism and mindful awareness of the subtle intricacies embedded in the “art of science” related to medical decision making.
↵∗ Editorials published in JACC: Cardiovascular Imaging reflect the views of the authors and do not necessarily represent the views of JACC: Cardiovascular Imaging or the American College of Cardiology.
Dr. Nasir has reported that he has no relationships relevant to the contents of this paper to disclose.
- American College of Cardiology Foundation