Author + information
- Received March 12, 2015
- Revision received May 19, 2015
- Accepted June 11, 2015
- Published online March 1, 2016.
- Federico M. Asch, MDa,∗ (, )
- Eugene Yuriditsky, MDa,
- Siddharth K. Prakash, MD, PhDb,
- Mary J. Roman, MDc,
- Jonathan W. Weinsaft, MDc,
- Gaby Weissman, MDa,
- Wm. Guy Weigold, MDa,
- Shaine A. Morris, MD, MPHd,
- William J. Ravekes, MDe,
- Kathryn W. Holmes, MDf,
- Michael Silberbach, MDf,
- Rita K. Milewski, MD, PhDg,
- Barbara L. Kroner, PhD, MPHh,
- Ryan Whitworth, MPHh,
- Kim A. Eagle, MDi,
- Richard B. Devereux, MDc,
- Neil J. Weissman, MDa,
- GenTAC Investigators
- aMedStar Health Research Institute at MedStar Washington Hospital Center, Washington, DC
- bUniversity of Texas, Houston, Houston, Texas
- cWeill Cornell Medical College, New York, New York
- dBaylor College of Medicine, Houston, Texas
- eJohns Hopkins School of Medicine, Baltimore, Maryland
- fOregon Health & Science University, Portland, Oregon
- gUniversity of Pennsylvania School of Medicine, Philadelphia, Pennsylvania
- hRTI International, Rockville, Maryland
- iUniversity of Michigan, Ann Arbor, Michigan
- ↵∗Reprint requests and correspondence:
Dr. Federico M. Asch, Cardiovascular Core Laboratories, MedStar Health Research Institute at MedStar Washington Hospital Center, 100 Irving Street, NW, Suite EB 5123, Washington, DC 20010.
Objectives This study sought to evaluate variability in aortic measurements with multiple imaging modalities in clinical centers by comparing with a standardized measuring protocol implemented in a core laboratory.
Background In patients with aortic disease, imaging of thoracic aorta plays a major role in risk stratifying individuals for life-threatening complications and in determining timing of surgical intervention. However, standardization of the procedures for performance of aortic measurements is lacking.
Methods To characterize the diversity of methods used in clinical practice, we compared aortic measurements performed by echocardiography, computed tomography (CT), and magnetic resonance imaging (MRI) at the 6 GenTAC (National Registry of Genetically Triggered Thoracic Aortic Aneurysms and Cardiovascular Conditions) clinical centers to those performed at the imaging core laboratory in 965 studies. Each center acquired and analyzed their images according to local protocols. The same images were subsequently analyzed blindly by the core laboratory, on the basis of a standardized protocol for all imaging modalities. Paired measurements from clinical centers and core laboratory were compared by mean of differences and intraclass correlation coefficient (ICC).
Results For all segments of the ascending aorta, echocardiography showed a higher ICC (0.84 to 0.93) than CT (0.84) and MRI (0.82 to 0.90), with smaller mean of differences. MRI showed higher ICC for the arch and descending aorta (0.91 and 0.93). In a mixed adjusted model, the different imaging modalities and clinical centers were identified as sources of variability between clinical and core laboratory measurements, whereas age groups or diagnosis at enrollment were not.
Conclusions By comparing core laboratory with measurements from clinical centers, our study identified important sources of variability in aortic measurements. Furthermore, our findings with regard to CT and MRI suggest a need for imaging societies to work toward the development of unifying acquisition protocols and common measuring methods.
Thoracic aortic aneurysms are associated with a number of genetic and familial syndromes and predispose patients to catastrophic events including aortic rupture and dissection (1,2). In these patients, aortic size is the most powerful predictor for such events (3–5). Therefore, imaging of the thoracic aorta plays a major role in risk stratifying individuals and in determining timing of surgical intervention (6). However, there is variability in the methods used to perform aortic measurements across professional guidelines and centers of expertise.
The GenTAC (National Registry of Genetically Triggered Thoracic Aortic Aneurysms and Cardiovascular Conditions) phase I, sponsored by the National Institutes of Health, enrolled individuals with a variety of aortic conditions between 2007 and 2010 (1,7). Transthoracic echocardiography (TTE), computed tomography (CT), and magnetic resonance imaging (MRI) of thoracic aortic segments were interpreted at the clinical centers per local protocols. Because the studies were performed for clinical indications, imaging performance and interpretation protocols were not standardized across centers. To provide a more consistent approach to aortic imaging interpretation, an imaging core laboratory (iCORE) was established for the second phase of GenTAC (GenTAC II). The task of the iCORE (MedStar Health Research Institute, Washington, DC) included collecting TTE, CT, and MRI studies from GenTAC I and blindly performing a de novo analysis of each study by a standardized measurement protocol. The purpose of this study was to evaluate the impact of the standardized protocol on aortic segment measurements across the aforementioned imaging modalities by comparing those performed at clinical centers with those performed by the GenTAC iCORE.
Patients in GenTAC I were enrolled at 6 nationally leading centers for the management of aortic diseases related to genetic conditions (Johns Hopkins, Baltimore, Maryland; Weill Cornell Medical College, New York, New York; University of Pennsylvania, Philadelphia, Pennsylvania; Baylor Medical Center, Houston, Texas; University of Texas, Houston, Texas; Oregon Health & Science University, Portland, Oregon). The rationale and design of GenTAC have been described previously (1).
Imaging analysis at clinical centers and iCORE
Clinical reports of aortic segment measures by TTE, CT, and MRI from GenTAC I were collected from clinical centers, on the basis of the image analysis performed by local protocols (no centralized instructions on how to perform measurements were provided). Subsequently, the same imaging studies were analyzed blindly by the iCORE following a unified protocol designed by consensus between the iCORE and the clinical centers. To further understand the differences in measurements, a retrospective survey provided detail on the methods used at each clinical center.
Analysis at the iCORE was performed on specific commercially available workstations. Aortic measurements were obtained at end-systole whenever electrocardiogram gating was available, with an inner edge to inner edge technique at pre-specified standard locations: aortic valve annulus, sinuses of Valsalva, sinotubular junction, ascending aorta (at the level of the pulmonary artery bifurcation), transverse arch (between the first and second neck vessels), isthmus (1 cm distal to the last neck vessel), descending thoracic aorta (at the level of the pulmonary artery), and suprarenal aorta (below the celiac artery). Measurements by CT and MRI were analyzed with a double-oblique technique (8). The iCORE analyzed only native aortic segments, excluding any grafted measurements.
Statistical analysis was performed by GenTAC’s Data Coordinating Center (RTI international, Rockville, Maryland). To determine agreement between readings, paired measurements from the clinical centers and the iCORE were compared. Bland–Altman analysis, mean of differences, paired t test and intraclass correlation (ICC) were analyzed for each aortic segment. Segments for which <20 cases were available were eliminated from the report. An ICC of ≥0.80 indicated excellent agreement. Mean of differences are reported as clinical centers minus iCORE; therefore, a negative value reflects iCORE measurements to be higher than clinical centers’ and vice versa. Paired t test was used to compare mean differences, with p ≤ 0.05 considered significant. Analysis of covariance was performed using a mixed model for difference between clinical center and iCORE measurements of each segment with fixed effects for imaging modality, diagnosis (in 3 categories: Marfan syndrome, bicuspid aortic valve, other diagnoses), age (in 3 categories: children <12 years, adolescents 12 to 18 years, and adults >18 years), clinical center, and a random effect for the participant identification because some patients had >1 image included. Adjusted least squares means of the differences in measurements were obtained from these mixed models. All analyses were performed using SAS version 9.3 (SAS Institute, Inc., Chicago, Illinois).
GenTAC I enrolled 2,046 individuals with a variety of genetic conditions predisposing to aortic aneurysms, rupture, and dissection. At the time of the analysis, 965 imaging studies (from 930 patients) with paired reports from the clinical center and the iCORE were identified, forming the population for this analysis. Table 1 displays the demographic data.
A retrospective survey of the GenTAC clinical centers showed substantial variability in the methodology used for analyzing CT and MRI images between centers in regard to timing in the cardiac cycle (some in systole, some diastole, majority using nongated images), or leading edge-to-leading edge versus inner edge-to-inner edge techniques. In contrast, methodology of echocardiographic analysis was less variable. Echo acquisition and readings consistently followed American Society of Echocardiography (ASE) recommendations for aortic measurements in pediatric populations (used for both pediatric and adult patients; systolic, inner edge to inner edge ) except for 1 center that followed the recommendations for adults (diastolic, leading to leading edge) (10,11). Importantly for all 3 imaging modalities, the methodology used to measure pediatric and adult patients was consistent within centers.
Agreement Between Clinical Centers and iCORE According to Imaging Modality
For proximal aortic segments, TTE demonstrated excellent agreement (aortic annulus, sinuses of Valsalva and ascending aorta) with ICCs from 0.84 to 0.92. Agreement was good, but lower, for the arch and descending aorta (ICCs 0.70 to 0.71) (Table 2). Although paired comparison showed statistical differences in measures of the aortic valve annulus, sinuses of Valsalva, and transverse arch (but not for other segments), mean differences for segment were small (all 0.01 to 0.08 cm).
In contrast with TTE, CT measurements agreement was good to excellent, with lower ICC for the transverse arch and isthmus (0.75 and 0.73, respectively) than for all other aortic segments (0.84 to 0.86) (Table 3). However, the mean differences between clinical center and iCORE measurements were large for most segments, ranging from 0.01 to 0.47 cm (largest for sinus of Valsalva and isthmus). When analyzing electrocardiogram-gated CT (only 30% in this study), there was an improvement in agreement in the proximal aortic segments (ICC 0.84, mean difference 0.25 cm for the sinus of Valsalva; ICC 0.89 and mean difference 0.11 cm for the ascending aorta), but surprisingly the agreement was lower in the distal segments.
Agreement between clinical and iCORE MRI measurements was excellent for all aortic segments, with ICCs ranging from 0.82 to 0.95 (Table 4). The mean of differences was in the range of 0.002 to 0.12 cm. Unfortunately, the number of cases with available MRI measurements for comparison at the annulus, sinotubular junction, and isthmus was too small to allow a meaningful analysis.
Overall, TTE demonstrated a higher ICC with lower magnitude of mean of differences in measures of the proximal aortic segments (sinuses of Valsalva to the ascending aorta) than CT and MRI. However, the ICC was lower when TTE was used to evaluate the transverse arch, isthmus, and descending aorta. This pattern was also observed with CT; the ICC among proximal segment measures was stronger when compared with CT-based measures of the arch and isthmus. This was in contrast with MRI-based measures demonstrating a high ICC with respect to evaluation of the arch and descending aorta.
Agreement adjusted to image modality, age group, diagnosis, and clinical center
A mixed model with fixed effects for imaging modality, diagnosis, age group, and clinical center found that agreement between clinical centers and iCORE varied consistently based on imaging modality and enrolling centers across segments, but not consistently based on the different patient age groups or enrolling diagnoses.
Compared with CT and MRI, TTE yielded closer agreement between local and core laboratory readings with means of differences between readings generally of ≤0.05 cm for the sinuses of Valsalva and ascending aorta. MRI had a closer agreement than TTE and CT for the transverse arch. The adjusted mean of differences was statistically significant from zero for CT and MRI at the sinuses of Valsalva and ascending aorta and for CT and TTE at the transverse arch. In contrast, they were not significant at the sinotubular junction, isthmus, or descending aorta. The TTE adjusted mean of differences between clinical center and core laboratory readings were significant only for the transverse arch. The difference in agreement for each imaging modality at a per-segment level resulted in significant variability among modalities for the sinuses of Valsalva (p < 0.001), ascending aorta (p = 0.018) and transverse arch (p = 0.001) but not for other aortic segments. The adjusted differences between TTE, CT, and MRI for each of the aortic segments are illustrated in Figure 1.
When analysis was performed to evaluate the agreement of measurements from each clinical center compared with iCORE measurements, there was a significant variability among the centers for the aortic annulus (p = 0.039), sinus of Valsalva (p < 0.001), ascending aorta (p = 0.05), and transverse arch (p = 0.025). This variability in the adjusted model is illustrated in Figure 2.
Our study sought to understand how the different techniques used for aortic measurements in clinical practice affect the variability of aortic measurements reported, and how the implementation of a standardized method by a centralized core laboratory might address this variability. GenTAC provided a unique opportunity to explore this issue, because paired data from clinical and iCORE readings for the same images were available for comparison that included multiple imaging modalities, age groups, and clinical diagnoses. Heterogeneity of analysis protocols among enrolling centers (all prime academic tertiary centers specialized in aortic diseases), particularly with regard to CT and MRI (but not for TTE), was identified by a simple survey of the investigators. Our main findings are as follows. First, TTE has a higher reproducibility between iCORE and clinical centers in measures of the proximal aortic segments, whereas CT and MRI are more reproducible at the arch and descending segments. Second, in a mixed model, there was significant variability among the enrolling centers and the imaging modalities (better consistency of TTE measurements), but not among age groups or diagnoses.
The sources of variability in measurements may be attributed to both the acquisition and measurement techniques. In both aspects, echocardiography has been standardized for many years (9,12,13), in contrast to CT and MRI for which consistency is still lacking. Although ASE guidelines for pediatric (9) and adult populations (10,11) recommend different measuring protocols, they both agree on the acquisition techniques to be used. However, with the goal of improved reproducibility, the inner to inner edge technique has been used in some studies and is often advocated for by experts (14,15). Importantly, recent data suggest that this difference in measurement techniques may not be as great as initially thought and therefore not as clinically significant on its own (16). In our study, all GenTAC centers acquired images following ASE guidelines and 5 of 6 centers measured according to the end systole, inner edge-to-inner edge technique, which was also the chosen technique for the central iCORE. This situation probably accounts for the high reproducibility of echocardiography for aortic segments up to the ascending aorta. It is also conceivable that the differences in measurement timing (systole vs. diastole) and interface identification (leading vs. inner edge) essentially offset each other. That the ICC was lower in comparing measures of the transverse arch to the descending thoracic aorta using TTE is not surprising, because TTE does not visualize these segments consistently with high quality (2). For the same reason, TTE methods for measuring these segments are not described in detail in the recent ASE/European Association of Cardiovascular Imaging guidelines (10).
Our results reveal variability in reliability of CT-based measures at 4 segments (sinuses of Valsalva, ascending aorta, transverse arch, and isthmus). Prior studies demonstrated overestimation of aortic aneurysm size by axial measures (which often pass obliquely through the aorta) when compared with double-oblique (oriented perpendicular to the axis of each aortic segment) (8,17). In a recent study of 50 patients in GenTAC, aortic measures were significantly different at all measured segments between axial and double oblique measures (8). In agreement with that study, our core laboratory CT measurements at the sinuses of Valsalva were smaller than those reported by clinical centers, likely reflecting our adoption of the double-oblique measures.
Although no one would argue that 3-dimensional (3D) methods such as CT and MRI are better for a comprehensive evaluation of the aorta, our results raise some concerns that must be considered for their optimal performance. Specifically, the lack of standardization of imaging protocols for both CT and MRI are remarkable, with regard to both acquisition and image analysis protocols. The recently published multimodality imaging guidelines certainly make an effort to improve these processes, but still fall short of a unified approach (10). Although our data standardized the iCORE measurement techniques, acquisition was left to each center’s preference (only 30% were electrocardiogram gated and there was variability in regard to use of contrast-enhanced and cine images).
Interpretation of imaging data in clinical practice and trials depends on the reliability and accuracy of the measurements performed. Variability of these measurements is related inversely to the standardization of acquisition and measurement techniques. The role of core laboratory interpretations in decreasing such variability has been well-documented in the echocardiography world for structures other than the aorta (18,19) and adopted by the ASE in their guidelines (20,21). Our data reinforce the concept of the need for a core laboratory in multicenter clinical studies by expanding it into aortic dimensions, and particularly highlights the need for standardization of techniques used in CT and MRI.
Although we believe that our study is the first to report on the impact of centralized, standardized reading for aortic dimensions and in describing how the standardization corrects for the heterogeneity of methods across clinical centers and imaging modalities, some limitations must be acknowledged. The number of MRI studies was small, limiting the strength of our findings with regard to this modality. In addition, many patients had aortic grafts and those segments were excluded. A minority of studies were 3D-based (no 3D echo); however, our distribution reflects the clinical practice of 6 academic centers of excellence in genetically related aortic disease. Finally, we could not collect details of acquisition and measuring techniques from the clinical centers on a case-by-case basis, which would have been the ideal scenario to compare techniques. Instead, we surveyed the investigators involved in imaging analysis at each center to describe their local common practices.
Our findings highlight the importance of using a unified protocol for analysis of aortic measurements, and its implementation by a centralized core laboratory. Furthermore, our findings with regard to CT and MRI suggest a need for imaging societies to work toward the development of unifying acquisition protocols and common measuring methods.
COMPETENCY IN PRACTICE-BASED LEARNING: Standardization of the procedures for image acquisition and performance of aortic measurements in multimodality imaging is lacking. This deficit results in variability of reported aortic measurements. Differences in imaging modalities and clinical centers were identified as sources of variability, reflecting the lack of unified protocols used across imaging modalities and clinical centers.
TRANSLATIONAL OUTLOOK: Our findings suggest a need for imaging societies to work toward development of unifying acquisition protocols and common measuring methods. For clinical trials, the use of core laboratories minimizes variability by standardizing procedures.
For a full list of the GenTAC Investigators, please see the online version of this article.
A full list of the GenTAC Investigators are listed in the Online Appendix.
The GenTAC Registry has been supported in total by U.S. Federal Government contracts HHSN268200648199C and HHSN268201000048C from the National Heart, Lung, and Blood Institute and the National Institute of Arthritis and Musculoskeletal and Skin Diseases (Bethesda, Maryland).
The authors have reported that they have no relationships relevant to the contents of this paper to disclose.
- Abbreviations and Acronyms
- computed tomography
- National Registry of Genetically Triggered Thoracic Aortic Aneurysms and Cardiovascular Conditions
- intraclass correlation coefficient
- imaging core laboratory
- magnetic resonance imaging
- transthoracic echocardiography
- Received March 12, 2015.
- Revision received May 19, 2015.
- Accepted June 11, 2015.
- American College of Cardiology Foundation
- Isselbacher E.M.
- Jondeau G.,
- Detaint D.,
- Tubach F.,
- et al.
- Hiratzka L.F.,
- Bakris G.L.,
- Beckman J.A.,
- et al.
- Lopez L.,
- Colan S.D.,
- Frommelt P.C.,
- et al.
- Goldstein S.A.,
- Evangelista A.,
- Abbara S.,
- et al.
- Lang R.M.,
- Bierig M.,
- Devereux R.B.,
- et al.
- Hole T.,
- Otterstad J.E.,
- St John Sutton M.,
- Froland G.,
- Holme I.,
- Skjaerpe T.
- Douglas P.S.,
- DeCara J.M.,
- Devereux R.B.,
- et al.