Statistical Modeling Approach to Quantitative Analysis of Interobserver Variability in Breast Contouring

Yang, Jinzhong; Woodward, Wendy A.; Reed, Valerie K.; Strom, Eric A.; Perkins, George H.; Tereffe, Welela; Buchholz, Thomas A.; Zhang, Lifei; Balter, Peter; Court, Laurence E.; Li, X. Allen; Dong, Lei

doi:10.1016/J.IJROBP.2014.01.010

Title: Statistical Modeling Approach to Quantitative Analysis of Interobserver Variability in Breast Contouring

Full Record
Other Related Research

Abstract

Purpose: To develop a new approach for interobserver variability analysis. Methods and Materials: Eight radiation oncologists specializing in breast cancer radiation therapy delineated a patient's left breast “from scratch” and from a template that was generated using deformable image registration. Three of the radiation oncologists had previously received training in Radiation Therapy Oncology Group consensus contouring for breast cancer atlas. The simultaneous truth and performance level estimation algorithm was applied to the 8 contours delineated “from scratch” to produce a group consensus contour. Individual Jaccard scores were fitted to a beta distribution model. We also applied this analysis to 2 or more patients, which were contoured by 9 breast radiation oncologists from 8 institutions. Results: The beta distribution model had a mean of 86.2%, standard deviation (SD) of ±5.9%, a skewness of −0.7, and excess kurtosis of 0.55, exemplifying broad interobserver variability. The 3 RTOG-trained physicians had higher agreement scores than average, indicating that their contours were close to the group consensus contour. One physician had high sensitivity but lower specificity than the others, which implies that this physician tended to contour a structure larger than those of the others. Two other physicians had low sensitivity but specificity similar tomore »« less

Authors:

Yang, Jinzhong ^[1]; Woodward, Wendy A.; Reed, Valerie K.; Strom, Eric A.; Perkins, George H.; Tereffe, Welela; Buchholz, Thomas A. ^[2]; Zhang, Lifei; Balter, Peter; Court, Laurence E. ^[1]; Li, X. Allen ^[3]; Dong, Lei ^[1]

Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas (United States)
Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas (United States)
Department of Radiation Oncology, Medical College of Wisconsin, Milwaukee, Wisconsin (United States)

Publication Date:: Thu May 01 00:00:00 EDT 2014

OSTI Identifier:: 22416566

Resource Type:: Journal Article

Journal Name:: International Journal of Radiation Oncology, Biology and Physics

Additional Journal Information:: Journal Volume: 89; Journal Issue: 1; Other Information: Copyright (c) 2014 Elsevier Science B.V., Amsterdam, The Netherlands, All rights reserved.; Country of input: International Atomic Energy Agency (IAEA); Journal ID: ISSN 0360-3016

Country of Publication:: United States

Language:: English

Subject:: 62 RADIOLOGY AND NUCLEAR MEDICINE; ALGORITHMS; MAMMARY GLANDS; MEDICAL PERSONNEL; NEOPLASMS; PATIENTS; RADIOTHERAPY; SENSITIVITY; SPECIFICITY

Citation Formats


                    Yang, Jinzhong, Woodward, Wendy A., Reed, Valerie K., Strom, Eric A., Perkins, George H., Tereffe, Welela, Buchholz, Thomas A., Zhang, Lifei, Balter, Peter, Court, Laurence E., Li, X. Allen, Dong, Lei, and Scripps Proton Therapy Center, San Diego, California. Statistical Modeling Approach to Quantitative Analysis of Interobserver Variability in Breast Contouring.  United States: N. p., 2014. 
        Web.  doi:10.1016/J.IJROBP.2014.01.010.

Copy to clipboard


                    Yang, Jinzhong, Woodward, Wendy A., Reed, Valerie K., Strom, Eric A., Perkins, George H., Tereffe, Welela, Buchholz, Thomas A., Zhang, Lifei, Balter, Peter, Court, Laurence E., Li, X. Allen, Dong, Lei, & Scripps Proton Therapy Center, San Diego, California. Statistical Modeling Approach to Quantitative Analysis of Interobserver Variability in Breast Contouring.  United States.  https://doi.org/10.1016/J.IJROBP.2014.01.010

Copy to clipboard


                    Yang, Jinzhong, Woodward, Wendy A., Reed, Valerie K., Strom, Eric A., Perkins, George H., Tereffe, Welela, Buchholz, Thomas A., Zhang, Lifei, Balter, Peter, Court, Laurence E., Li, X. Allen, Dong, Lei, and Scripps Proton Therapy Center, San Diego, California. 2014.  
        "Statistical Modeling Approach to Quantitative Analysis of Interobserver Variability in Breast Contouring".  United States.  https://doi.org/10.1016/J.IJROBP.2014.01.010.

Copy to clipboard


                    
@article{osti_22416566,

  title        = {Statistical Modeling Approach to Quantitative Analysis of Interobserver Variability in Breast Contouring},

  author       = {Yang, Jinzhong and Woodward, Wendy A. and Reed, Valerie K. and Strom, Eric A. and Perkins, George H. and Tereffe, Welela and Buchholz, Thomas A. and Zhang, Lifei and Balter, Peter and Court, Laurence E. and Li, X. Allen and Dong, Lei and Scripps Proton Therapy Center, San Diego, California},

  abstractNote = {Purpose: To develop a new approach for interobserver variability analysis. Methods and Materials: Eight radiation oncologists specializing in breast cancer radiation therapy delineated a patient's left breast “from scratch” and from a template that was generated using deformable image registration. Three of the radiation oncologists had previously received training in Radiation Therapy Oncology Group consensus contouring for breast cancer atlas. The simultaneous truth and performance level estimation algorithm was applied to the 8 contours delineated “from scratch” to produce a group consensus contour. Individual Jaccard scores were fitted to a beta distribution model. We also applied this analysis to 2 or more patients, which were contoured by 9 breast radiation oncologists from 8 institutions. Results: The beta distribution model had a mean of 86.2%, standard deviation (SD) of ±5.9%, a skewness of −0.7, and excess kurtosis of 0.55, exemplifying broad interobserver variability. The 3 RTOG-trained physicians had higher agreement scores than average, indicating that their contours were close to the group consensus contour. One physician had high sensitivity but lower specificity than the others, which implies that this physician tended to contour a structure larger than those of the others. Two other physicians had low sensitivity but specificity similar to the others, which implies that they tended to contour a structure smaller than the others. With this information, they could adjust their contouring practice to be more consistent with others if desired. When contouring from the template, the beta distribution model had a mean of 92.3%, SD ± 3.4%, skewness of −0.79, and excess kurtosis of 0.83, which indicated a much better consistency among individual contours. Similar results were obtained for the analysis of 2 additional patients. Conclusions: The proposed statistical approach was able to measure interobserver variability quantitatively and to identify individuals who tended to contour differently from the others. The information could be useful as feedback to improve contouring consistency.},

  doi          = {10.1016/J.IJROBP.2014.01.010},

  url          = {https://www.osti.gov/biblio/22416566},
  journal      = {International Journal of Radiation Oncology, Biology and Physics},

  issn         = {0360-3016},

  number       = 1,

  volume       = 89,

  place        = {United States},

  year         = {Thu May 01 00:00:00 EDT 2014},

  month        = {Thu May 01 00:00:00 EDT 2014}

}

Copy to clipboard

Journal Article:

https://doi.org/10.1016/J.IJROBP.2014.01.010

Other availability

Find in Google Scholar

Search WorldCat to find libraries that may hold this journal

Save / Share:

Export Metadata

Save to My Library

Similar records in OSTI.GOV collections:

Target Volume Delineation for Partial Breast Radiotherapy Planning: Clinical Characteristics Associated with Low Interobserver Concordance

Journal Article Petersen, Ross; Kader, Hosam; Berthelet, Eric; ... - International Journal of Radiation Oncology, Biology and Physics

Purpose: To examine variability in target volume delineation for partial breast radiotherapy planning and evaluate characteristics associated with low interobserver concordance. Methods and Materials: Thirty patients who underwent planning CT for adjuvant breast radiotherapy formed the study cohort. Using a standardized scale to score seroma clarity and consensus contouring guidelines, three radiation oncologists independently graded seroma clarity and delineated seroma volumes for each case. Seroma geometric center coordinates, maximum diameters in three axes, and volumes were recorded. Conformity index (CI), the ratio of overlapping volume and encompassing delineated volume, was calculated for each case. Cases with CI {<=}0.50 were analyzedmore »« less
https://doi.org/10.1016/j.ijrobp.2007.01.070
Variability Among Breast Radiation Oncologists in Delineation of the Postsurgical Lumpectomy Cavity

Journal Article Landis, Daniel; Weixiu, Luo; Jun, Song; ... - International Journal of Radiation Oncology, Biology and Physics

Purpose: Partial breast irradiation (PBI) is becoming more widely used. Accurate determination of the surgical lumpectomy cavity volume is more critical with PBI than with whole breast radiation therapy. We examined the interobserver variability in delineation of the lumpectomy cavity among four academic radiation oncologists who specialize in the treatment of breast cancer. Methods and Materials: Thirty-four lumpectomy cavities in 33 consecutive patients were evaluated. Each physician contoured the cavity and a 1.5-cm margin was added to define the planning target volume (PTV). A cavity visualization score (CVS) was assigned (1-5). To eliminate bias, the physician of record was eliminatedmore »« less
https://doi.org/10.1016/j.ijrobp.2006.11.026
Automatic Segmentation of Whole Breast Using Atlas Approach and Deformable Image Registration

Journal Article Reed, Valerie; Woodward, Wendy; Lifei, Zhang; ... - International Journal of Radiation Oncology, Biology and Physics

Purpose: To compare interobserver variations in delineating the whole breast for treatment planning using two contouring methods. Methods and Materials: Autosegmented contours were generated by a deformable image registration-based breast segmentation method (DEF-SEG) by mapping the whole breast clinical target volume (CTVwb) from a template case to a new patient case. Eight breast radiation oncologists modified the autosegmented contours as necessary to achieve a clinically appropriate CTVwb and then recontoured the same case from scratch for comparison. The times to complete each approach, as well as the interobserver variations, were analyzed. The template case was also mapped to 10 breastmore »« less
https://doi.org/10.1016/j.ijrobp.2008.07.001
Variability of Target and Normal Structure Delineation for Breast Cancer Radiotherapy: An RTOG Multi-Institutional and Multiobserver Study

Journal Article Tai, An; Arthur, Douglas; Buchholz, Thomas; ... - International Journal of Radiation Oncology, Biology and Physics

Purpose: To quantify the multi-institutional and multiobserver variability of target and organ-at-risk (OAR) delineation for breast-cancer radiotherapy (RT) and its dosimetric impact as the first step of a Radiation Therapy Oncology Group effort to establish a breast cancer atlas. Methods and Materials: Nine radiation oncologists specializing in breast RT from eight institutions independently delineated targets (e.g., lumpectomy cavity, boost planning target volume, breast, supraclavicular, axillary and internal mammary nodes, chest wall) and OARs (e.g., heart, lung) on the same CT images of three representative breast cancer patients. Interobserver differences in structure delineation were quantified regarding volume, distance between centers ofmore »« less
https://doi.org/10.1016/j.ijrobp.2008.10.034
TU-H-CAMPUS-JeP2-02: Interobserver Variability of CT, PET-CT and MRI Based Primary Tumor Delineation for Lung Cancer

Journal Article Karki, K; Hugo, G; Saraiya, S; ... - Medical Physics

Purpose: Target delineation in lung cancer radiotherapy has, in general, large variability. MRI has so far not been investigated in detail for lung cancer delineation variability. The purpose of this study is to investigate delineation variability for lung tumors using MRI and compare it to CT alone and PET-CT based delineations. Methods: Seven physicians delineated the primary tumor volumes of nine patients for the following scenarios: (1) CT only; (2) post-contrast T1-weighted MRI registered with diffusion-weighted MRI; and (3) PET-CT fusion images. To compute interobserver variability, the median surface was generated from all observers’ contours and used as the referencemore »« less
https://doi.org/10.1118/1.4957685

Similar Records