skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Multi-institutional Validation Study of Commercially Available Deformable Image Registration Software for Thoracic Images

Abstract

Purpose: To assess the accuracy of the commercially available deformable image registration (DIR) software for thoracic images at multiple institutions. Methods and Materials: Thoracic 4-dimensional (4D) CT images of 10 patients with esophageal or lung cancer were used. Datasets for these patients were provided by DIR-lab ( (dir-lab.com)) and included a coordinate list of anatomic landmarks (300 bronchial bifurcations) that had been manually identified. Deformable image registration was performed between the peak-inhale and -exhale images. Deformable image registration error was determined by calculating the difference at each landmark point between the displacement calculated by DIR software and that calculated by the landmark. Results: Eleven institutions participated in this study: 4 used RayStation (RaySearch Laboratories, Stockholm, Sweden), 5 used MIM Software (Cleveland, OH), and 3 used Velocity (Varian Medical Systems, Palo Alto, CA). The ranges of the average absolute registration errors over all cases were as follows: 0.48 to 1.51 mm (right-left), 0.53 to 2.86 mm (anterior-posterior), 0.85 to 4.46 mm (superior-inferior), and 1.26 to 6.20 mm (3-dimensional). For each DIR software package, the average 3-dimensional registration error (range) was as follows: RayStation, 3.28 mm (1.26-3.91 mm); MIM Software, 3.29 mm (2.17-3.61 mm); and Velocity, 5.01 mm (4.02-6.20 mm). These results demonstrate that there was moderate variation among institutions, although themore » DIR software was the same. Conclusions: We evaluated the commercially available DIR software using thoracic 4D-CT images from multiple centers. Our results demonstrated that DIR accuracy differed among institutions because it was dependent on both the DIR software and procedure. Our results could be helpful for establishing prospective clinical trials and for the widespread use of DIR software. In addition, for clinical care, we should try to find the optimal DIR procedure using thoracic 4D-CT data.« less

Authors:
 [1]; ;  [1];  [2];  [3];  [4];  [5];  [6];  [7];  [8];  [9];  [10];  [11];  [1]
  1. Department of Radiation Oncology, Tohoku University Graduate School of Medicine, Sendai (Japan)
  2. Department of Radiation Oncology and Image-Applied Therapy, Kyoto University Graduate School of Medicine, Kyoto (Japan)
  3. Department of Radiation Oncology, Kanagawa Cancer Center, Yokohama (Japan)
  4. Department of Radiotherapy, Tokyo Metropolitan Cancer and Infectious Diseases Center, Komagome Hospital, Tokyo (Japan)
  5. Department of Radiation Oncology, Tokai University School of Medicine, Hachioji (Japan)
  6. Department of Radiological Technology, Tokushima University Hospital, Tokushima (Japan)
  7. Department of Radiation Physics and Technology, Southern Tohoku Proton Therapy Center, Koriyama (Japan)
  8. Department of Radiation Oncology, St Luke's International Hospital, Tokyo (Japan)
  9. Department of Carbon Ion Radiotherapy, Osaka University Graduate School of Medicine, Suita (Japan)
  10. Department of Radiation Oncology, National Cancer Center Hospital, Tokyo (Japan)
  11. Department of Radiation Oncology, Tokyo Bay Advanced Imaging and Radiation Oncology Clinic Makuhari, Chiba (Japan)
Publication Date:
OSTI Identifier:
22645660
Resource Type:
Journal Article
Resource Relation:
Journal Name: International Journal of Radiation Oncology, Biology and Physics; Journal Volume: 96; Journal Issue: 2; Other Information: Copyright (c) 2016 Elsevier Science B.V., Amsterdam, The Netherlands, All rights reserved.; Country of input: International Atomic Energy Agency (IAEA)
Country of Publication:
United States
Language:
English
Subject:
62 RADIOLOGY AND NUCLEAR MEDICINE; CLINICAL TRIALS; COMPUTER CODES; COMPUTERIZED TOMOGRAPHY; ERRORS; IMAGES

Citation Formats

Kadoya, Noriyuki, E-mail: kadoya.n@rad.med.tohoku.ac.jp, Nakajima, Yujiro, Saito, Masahide, Miyabe, Yuki, Kurooka, Masahiko, Kito, Satoshi, Fujita, Yukio, Sasaki, Motoharu, Arai, Kazuhiro, Tani, Kensuke, Yagi, Masashi, Wakita, Akihisa, Tohyama, Naoki, and Jingu, Keiichi. Multi-institutional Validation Study of Commercially Available Deformable Image Registration Software for Thoracic Images. United States: N. p., 2016. Web. doi:10.1016/J.IJROBP.2016.05.012.
Kadoya, Noriyuki, E-mail: kadoya.n@rad.med.tohoku.ac.jp, Nakajima, Yujiro, Saito, Masahide, Miyabe, Yuki, Kurooka, Masahiko, Kito, Satoshi, Fujita, Yukio, Sasaki, Motoharu, Arai, Kazuhiro, Tani, Kensuke, Yagi, Masashi, Wakita, Akihisa, Tohyama, Naoki, & Jingu, Keiichi. Multi-institutional Validation Study of Commercially Available Deformable Image Registration Software for Thoracic Images. United States. doi:10.1016/J.IJROBP.2016.05.012.
Kadoya, Noriyuki, E-mail: kadoya.n@rad.med.tohoku.ac.jp, Nakajima, Yujiro, Saito, Masahide, Miyabe, Yuki, Kurooka, Masahiko, Kito, Satoshi, Fujita, Yukio, Sasaki, Motoharu, Arai, Kazuhiro, Tani, Kensuke, Yagi, Masashi, Wakita, Akihisa, Tohyama, Naoki, and Jingu, Keiichi. Sat . "Multi-institutional Validation Study of Commercially Available Deformable Image Registration Software for Thoracic Images". United States. doi:10.1016/J.IJROBP.2016.05.012.
@article{osti_22645660,
title = {Multi-institutional Validation Study of Commercially Available Deformable Image Registration Software for Thoracic Images},
author = {Kadoya, Noriyuki, E-mail: kadoya.n@rad.med.tohoku.ac.jp and Nakajima, Yujiro and Saito, Masahide and Miyabe, Yuki and Kurooka, Masahiko and Kito, Satoshi and Fujita, Yukio and Sasaki, Motoharu and Arai, Kazuhiro and Tani, Kensuke and Yagi, Masashi and Wakita, Akihisa and Tohyama, Naoki and Jingu, Keiichi},
abstractNote = {Purpose: To assess the accuracy of the commercially available deformable image registration (DIR) software for thoracic images at multiple institutions. Methods and Materials: Thoracic 4-dimensional (4D) CT images of 10 patients with esophageal or lung cancer were used. Datasets for these patients were provided by DIR-lab ( (dir-lab.com)) and included a coordinate list of anatomic landmarks (300 bronchial bifurcations) that had been manually identified. Deformable image registration was performed between the peak-inhale and -exhale images. Deformable image registration error was determined by calculating the difference at each landmark point between the displacement calculated by DIR software and that calculated by the landmark. Results: Eleven institutions participated in this study: 4 used RayStation (RaySearch Laboratories, Stockholm, Sweden), 5 used MIM Software (Cleveland, OH), and 3 used Velocity (Varian Medical Systems, Palo Alto, CA). The ranges of the average absolute registration errors over all cases were as follows: 0.48 to 1.51 mm (right-left), 0.53 to 2.86 mm (anterior-posterior), 0.85 to 4.46 mm (superior-inferior), and 1.26 to 6.20 mm (3-dimensional). For each DIR software package, the average 3-dimensional registration error (range) was as follows: RayStation, 3.28 mm (1.26-3.91 mm); MIM Software, 3.29 mm (2.17-3.61 mm); and Velocity, 5.01 mm (4.02-6.20 mm). These results demonstrate that there was moderate variation among institutions, although the DIR software was the same. Conclusions: We evaluated the commercially available DIR software using thoracic 4D-CT images from multiple centers. Our results demonstrated that DIR accuracy differed among institutions because it was dependent on both the DIR software and procedure. Our results could be helpful for establishing prospective clinical trials and for the widespread use of DIR software. In addition, for clinical care, we should try to find the optimal DIR procedure using thoracic 4D-CT data.},
doi = {10.1016/J.IJROBP.2016.05.012},
journal = {International Journal of Radiation Oncology, Biology and Physics},
number = 2,
volume = 96,
place = {United States},
year = {Sat Oct 01 00:00:00 EDT 2016},
month = {Sat Oct 01 00:00:00 EDT 2016}
}
  • Purpose: To evaluate and compare the deformable image registration algorithms available in the Velocity (Velocity Medical Solutions, Atlanta, GA) and RayStation (RaySearch Americas, Inc., Garden city NY). Methods: Ten consecutive patient cone beam CTs (CBCT) for each fraction were collected. The CBCTs along with the simulation CT were exported to the Velocity and the RayStation software. Each CBCT was registered using deformable image registration to the simulation CT and the resulting deformable vector matrix was generated. Each registration was visually inspected by a physicist and the prescribing physician. The volumes of the critical organs were calculated for each deformable CTmore » and used for comparison. Results: The resulting deformable registrations revealed differences between the two algorithms. These differences were realized when the organs at risk were contoured on each deformed CBCT. Differences in the order of 10% ±30% in volume were observed for bladder, 17 ±21% for rectum and 16±10% for sigmoid. The prostate and PTV volume differences were in the order of 3±5%. The volumetric differences observed had a respective impact on the DVHs of all organs at risk. Differences of 8–10% in the mean dose were observed for all organs above. Conclusion: Deformable registration is a powerful tool that aids in the definition of critical structures and is often used for the evaluation of daily dose delivered to the patient. It should be noted that extended QA should be performed before clinical implementation of the software and the users should be aware of advantages and limitations of the methods.« less
  • Purpose: To investigate the accuracy of various algorithms for deformable image registration (DIR), to propagate regions of interest (ROIs) in computational phantoms based on patient images using different commercial systems. This work is part of an Italian multi-institutional study to test on common datasets the accuracy, reproducibility and safety of DIR applications in Adaptive Radiotherapy. Methods: Eleven institutions with three available commercial solutions provided data to assess the agreement of DIR-propagated ROIs with automatically drown ROIs considered as ground-truth for the comparison. The DIR algorithms were tested on real patient data from three different anatomical districts: head and neck, thoraxmore » and pelvis. For every dataset two specific Deformation Vector Fields (DVFs) provided by ImSimQA software were applied to the reference data set. Three different commercial software were used in this study: RayStation, Velocity and Mirada. The DIR-mapped ROIs were then compared with the reference ROIs using the Jaccard Conformity Index (JCI). Results: More than 600 DIR-mapped ROIs were analyzed. Putting together all JCI data of all institutions for the first DVF, the mean JCI was 0.87 ± 0.7 (1 SD) while for the second DVF JCI was 0.8 ± 0.13 (1 SD). Several considerations on different structures are available from collected data: the standard deviation among different institutions on specific structure raise as the larger is the applied DVF. The higher value is 10% for bladder. Conclusion: Although the complexity of deformation of human body is very difficult to model, this work illustrates some clinical scenarios with well-known DVFs provided by specific software. CI parameter gives the inter-user variability and may put in evidence the need of improving the working protocol in order to reduce the inter-institution JCI variability.« less
  • Purpose: Deformable image registration (DIR) is increasingly being used in various clinical applications. Although there are several DIR packages all making successful attempts at modeling complex anatomical changes using even more complex mathematical approximations, they are all subject to various uncertainties. Many studies have attempted to quantify the spatial uncertainty with DIR. This is the first study to compare the uncertainty for interfraction DIR for 5 different commercially-available algorithms. The aim of this study was to benchmark the performance of the most commonlyused DIR algorithms offered through these 5 software packages: Eclipse, MIM, Pinnacle, RaySearch, and Velocity. Methods: A setmore » of 10 virtual H'N phantoms [Pukala et al. MedPhys. 40(11) 2013] with known deformations were used to determine the spatial errors that might be seen when performing DIR. The “ground-truth” deformation vector field (DVF) was compared to the DVF output of the 5 commercially-available algorithms in order to evaluate spatial errors for six regions of interest (ROIs): brainstem, cord, mandible, left parotid, right parotid, and the external body contour. Results: We found that each software package had varying uncertainties with the various ROIs, but were generally all comparable to one another – with mean spatial errors for each algorithm below 3.5 mm for each ROI (averaged across all phantoms). We also found that no single algorithm was the clear winner over the other 4 algorithms. However, at times, we found huge maximum errors in our results (e.g. phantom #9 maximum errors: right parotid = 22.9 mm, external contour = 30.5mm) with the varying DIR algorithms. Conclusion: Although our evaluation was limited to H'N patients, we show that our methods are a single-assessment analysis tool that could be used by any physicist, within any type of facility, to compare their DIR software before initiating widespread use within their daily radiotherapy practice.« less
  • Purpose: The spatial accuracy of deformable image registration (DIR) is important in the implementation of image guided adaptive radiotherapy techniques for cancer in the pelvic region. Validation of algorithms is best performed on phantoms with fiducial markers undergoing controlled large deformations. Excised porcine bladders, exhibiting similar filling and voiding behavior as human bladders, provide such an environment. The aim of this study was to determine the spatial accuracy of different DIR algorithms on CT images ofex vivo porcine bladders with radiopaque fiducial markers applied to the outer surface, for a range of bladder volumes, using various accuracy metrics. Methods: Fivemore » excised porcine bladders with a grid of 30–40 radiopaque fiducial markers attached to the outer wall were suspended inside a water-filled phantom. The bladder was filled with a controlled amount of water with added contrast medium for a range of filling volumes (100–400 ml in steps of 50 ml) using a luer lock syringe, and CT scans were acquired at each filling volume. DIR was performed for each data set, with the 100 ml bladder as the reference image. Six intensity-based algorithms (optical flow or demons-based) implemented in theMATLAB platform DIRART, a b-spline algorithm implemented in the commercial software package VelocityAI, and a structure-based algorithm (Symmetric Thin Plate Spline Robust Point Matching) were validated, using adequate parameter settings according to values previously published. The resulting deformation vector field from each registration was applied to the contoured bladder structures and to the marker coordinates for spatial error calculation. The quality of the algorithms was assessed by comparing the different error metrics across the different algorithms, and by comparing the effect of deformation magnitude (bladder volume difference) per algorithm, using the Independent Samples Kruskal-Wallis test. Results: The authors found good structure accuracy without dependency on bladder volume difference for all but one algorithm, and with the best result for the structure-based algorithm. Spatial accuracy as assessed from marker errors was disappointing for all algorithms, especially for large volume differences, implying that the deformations described by the registration did not represent anatomically correct deformations. The structure-based algorithm performed the best in terms of marker error for the large volume difference (100–400 ml). In general, for the small volume difference (100–150 ml) the algorithms performed relatively similarly. The structure-based algorithm exhibited the best balance in performance between small and large volume differences, and among the intensity-based algorithms, the algorithm implemented in VelocityAI exhibited the best balance. Conclusions: Validation of multiple DIR algorithms on a novel physiological bladder phantom revealed that the structure accuracy was good for most algorithms, but that the spatial accuracy as assessed from markers was low for all algorithms, especially for large deformations. Hence, many of the available algorithms exhibit sufficient accuracy for contour propagation purposes, but possibly not for accurate dose accumulation.« less
  • Purpose: This study assesses the accuracy of the absorbed dose estimates from CT scans generated by Monte Carlo (MC) simulation using a commercially available radiation dose monitoring software program. Methods: Axial CT studies of an anthropomorphic abdomen phantom with dose bores at a central location and 4 peripheral locations were conducted using a fixed tube current at 120 kV. A 100 mm ion chamber and a 0.6 cc ion chamber calibrated at diagnostic energy levels were used to measure dose in the phantom at each of the 5 dose bore locations. Simulations using the software program's Monte Carlo engine weremore » run using a mathematical model of the anthropomorphic phantom to determine conversion coefficients between the CTDIvol used for the study and the dose at the location of the dose bores. Simulations were conducted using both the software's generic CT beam model and a refined model generated using HVL and bow tie filter profile measurements made on the scanner used for the study. Results: Monte Carlo simulations completed using the generalized beam model differed from the measured conversion factors by an absolute value average of 13.0% and 13.8% for the 100 mm and 0.6 cc ion chamber studies, respectively. The MC simulations using the scanner specific beam model generated conversion coefficients that differed from the CTDIvol to measured dose conversion coefficients by an absolute value average of 7.3% and 7.8% for the 100 mm and 0.6 cc ion chamber cases, respectively. Conclusion: A scanner specific beam model used in MC simulations generates more accurate dose conversion coefficients in an anthropomorphic phantom than those generated with a generalized beam model. Agreement between measured conversion coefficients and simulated values were less than 20% for all positions using the universal beam model.« less