skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: SU-E-J-197: Investigation of Microsoft Kinect 2.0 Depth Resolution for Patient Motion Tracking

Abstract

Purpose: Investigate the use of the Kinect 2.0 for patient motion tracking during radiotherapy by studying spatial and depth resolution capabilities. Methods: Using code written in C#, depth map data was abstracted from the Kinect to create an initial depth map template indicative of the initial position of an object to be compared to the depth map of the object over time. To test this process, simple setup was created in which two objects were imaged: a 40 cm × 40 cm board covered in non reflective material and a 15 cm × 26 cm textbook with a slightly reflective, glossy cover. Each object, imaged and measured separately, was placed on a movable platform with object to camera distance measured. The object was then moved a specified amount to ascertain whether the Kinect’s depth camera would visualize the difference in position of the object. Results: Initial investigations have shown the Kinect depth resolution is dependent on the object to camera distance. Measurements indicate that movements as small as 1 mm can be visualized for objects as close as 50 cm away. This depth resolution decreases linearly with object to camera distance. At 4 m, the depth resolution had decreased tomore » observe a minimum movement of 1 cm. Conclusion: The improved resolution and advanced hardware of the Kinect 2.0 allows for increase of depth resolution over the Kinect 1.0. Although obvious that the depth resolution should decrease with increasing distance from an object given the decrease in number of pixels representing said object, the depth resolution at large distances indicates its usefulness in a clinical setting.« less

Authors:
;  [1]
  1. Wayne State University, Detroit, MI (United States)
Publication Date:
OSTI Identifier:
22499301
Resource Type:
Journal Article
Resource Relation:
Journal Name: Medical Physics; Journal Volume: 42; Journal Issue: 6; Other Information: (c) 2015 American Association of Physicists in Medicine; Country of input: International Atomic Energy Agency (IAEA)
Country of Publication:
United States
Language:
English
Subject:
60 APPLIED LIFE SCIENCES; CAMERAS; DEPTH DOSE DISTRIBUTIONS; IMAGES; PATIENTS; RADIOTHERAPY; SPATIAL RESOLUTION

Citation Formats

Silverstein, E, and Snyder, M. SU-E-J-197: Investigation of Microsoft Kinect 2.0 Depth Resolution for Patient Motion Tracking. United States: N. p., 2015. Web. doi:10.1118/1.4924283.
Silverstein, E, & Snyder, M. SU-E-J-197: Investigation of Microsoft Kinect 2.0 Depth Resolution for Patient Motion Tracking. United States. doi:10.1118/1.4924283.
Silverstein, E, and Snyder, M. Mon . "SU-E-J-197: Investigation of Microsoft Kinect 2.0 Depth Resolution for Patient Motion Tracking". United States. doi:10.1118/1.4924283.
@article{osti_22499301,
title = {SU-E-J-197: Investigation of Microsoft Kinect 2.0 Depth Resolution for Patient Motion Tracking},
author = {Silverstein, E and Snyder, M},
abstractNote = {Purpose: Investigate the use of the Kinect 2.0 for patient motion tracking during radiotherapy by studying spatial and depth resolution capabilities. Methods: Using code written in C#, depth map data was abstracted from the Kinect to create an initial depth map template indicative of the initial position of an object to be compared to the depth map of the object over time. To test this process, simple setup was created in which two objects were imaged: a 40 cm × 40 cm board covered in non reflective material and a 15 cm × 26 cm textbook with a slightly reflective, glossy cover. Each object, imaged and measured separately, was placed on a movable platform with object to camera distance measured. The object was then moved a specified amount to ascertain whether the Kinect’s depth camera would visualize the difference in position of the object. Results: Initial investigations have shown the Kinect depth resolution is dependent on the object to camera distance. Measurements indicate that movements as small as 1 mm can be visualized for objects as close as 50 cm away. This depth resolution decreases linearly with object to camera distance. At 4 m, the depth resolution had decreased to observe a minimum movement of 1 cm. Conclusion: The improved resolution and advanced hardware of the Kinect 2.0 allows for increase of depth resolution over the Kinect 1.0. Although obvious that the depth resolution should decrease with increasing distance from an object given the decrease in number of pixels representing said object, the depth resolution at large distances indicates its usefulness in a clinical setting.},
doi = {10.1118/1.4924283},
journal = {Medical Physics},
number = 6,
volume = 42,
place = {United States},
year = {Mon Jun 15 00:00:00 EDT 2015},
month = {Mon Jun 15 00:00:00 EDT 2015}
}
  • Purpose: The purpose of this study is to investigate the feasibility of a low cost, small size positioning assistance simulator system for skull radiography using the Microsoft Kinect sensor. A conventional radiographic simulator system can only measure the three-dimensional coordinates of an x-ray tube using angle sensors, but not measure the movement of the subject. Therefore, in this study, we developed a real-time simulator system using the Microsoft Kinect to measure both the x-ray tube and the subject, and evaluated its accuracy and feasibility by comparing the simulated and the measured x-ray images. Methods: This system can track a headmore » phantom by using Face Tracking, which is one of the functions of the Kinect. The relative relationship between the Kinect and the head phantom was measured and the projection image was calculated by using the ray casting method, and by using three-dimensional CT head data with 220 slices at 512 × 512 pixels. X-ray images were thus obtained by using a computed radiography (CR) system. We could then compare the simulated projection images with the measured x-ray images from 0 degrees to 45 degrees at increments of 15 degrees by calculating the cross correlation coefficient C. Results: The calculation time of the simulated projection images was almost real-time (within 1 second) by using the Graphics Processing Unit(GPU). The cross-correlation coefficients C are: 0.916; 0.909; 0.891; and, 0.886 at 0, 15, 30, and 45 degrees, respectively. As a result, there were strong correlations between the simulated and measured images. Conclusion: This system can be used to perform head positioning more easily and accurately. It is expected that this system will be useful for learning radiographic techniques by students. Moreover, it could also be used for predicting the actual x-ray image prior to x-ray exposure in clinical environments.« less
  • Purpose: Investigate capability and accuracy of Kinect v2 camera for tracking respiratory motion to use as a tool during 4DCT or in combination with motion management during radiotherapy treatments. Methods: Utilizing the depth sensor on the Kinect as well as code written in C#, the respiratory motion of a patient was tracked by recording the depth (distance) values obtained at several points on the patient. Respiratory traces were also obtained using Varian’s RPM system, which traces the movement of a propriety marker placed on the patient’s abdomen, as well as an Anzai belt, which utilizes a pressure sensor to trackmore » respiratory motion. With the Kinect mounted 60 cm above the patient and pointing straight down, 11 breathing cycles were recorded with each system simultaneously. Relative displacement values during this time period were saved to file. While RPM and the Kinect give displacement values in distance units, the Anzai system has arbitrary units. As such, displacement for all three are displayed relative to the maximum value for the time interval from that system. Additional analysis was performed between RPM and Kinect for absolute displacement values. Results: Analysis of the data from all three systems indicates the relative motion obtained from the Kinect is both accurate and in sync with the data from RPM and Anzai. The absolute displacement data from RPM and Kinect show similar displacement values throughout the acquisition except for the depth obtained from the Kinect during maximum exhalation (largest distance from Kinect). Conclusion: By simply utilizing the depth data of specific points on a patient obtained from the Kinect, respiratory motion can be tracked and visualized with accuracy comparable to that of the Varian RPM and Anzai belt.« less
  • Current techniques to acquire patient surface data are often very expensive and lack flexibility. In this study, the use of the Microsoft Kinect to reliably acquire 3D scans of patient surface is investigated. A design is presented to make the system easily applicable to the clinic. Potential applications of the device to radiotherapy are also presented. Scan reproducibility was tested by repeatedly scanning an anthropomorphic phantom. Scan accuracy was tested by comparing Kinect scans to the surface extracted from a CT dataset of a Rando® anthropomorphic phantom, which was considered as the true reference surface. Average signed distances of 0.12more » ± 2.34 mm and 0.13 ± 2.04 mm were obtained between the compared surfaces for reproducibility and accuracy respectively. This is conclusive, since it indicates that the variations observed come largely from noise distributed around an average distance close to 0 mm. Moreover, the range of the noise is small enough for the system to reliably capture a patient's surface. A system was also designed using two Kinects used together to acquire 3D surfaces in a quick and stable way that is applicable to the clinic. Finally, applications of the device to radiotherapy are demonstrated. Its use to detect local positioning errors is presented, where small local variations difficult to see with the naked eye are clearly visible. The system was also used to predict collisions using gantry and patient scans and thus ensure the safety of unconventional trajectories.« less
  • Purpose: To determine whether the Microsoft Kinect Version 2 (Kinect v2), a commercial off-the-shelf (COTS) depth sensors designed for entertainment purposes, were robust to the radiotherapy treatment environment and could be suitable for monitoring of voluntary breath-hold compliance. This could complement current visual monitoring techniques, and be useful for heart sparing left breast radiotherapy. Methods: In-house software to control Kinect v2 sensors, and capture output information, was developed using the free Microsoft software development kit, and the Cinder creative coding C++ library. Each sensor was used with a 12m USB 3.0 active cable. A solid water block was used asmore » the object. The depth accuracy and precision of the sensors was evaluated by comparing Kinect reported distance to the object with a precision laser measurement across a distance range of 0.6m to 2.0 m. The object was positioned on a high-precision programmable motion platform and moved in two programmed motion patterns and Kinect reported distance logged. Robustness to the radiation environment was tested by repeating all measurements with a linear accelerator operating over a range of pulse repetition frequencies (6Hz to 400Hz) and dose rates 50 to 1500 monitor units (MU) per minute. Results: The complex, consistent relationship between true and measured distance was unaffected by the radiation environment, as was the ability to detect motion. Sensor precision was < 1 mm and the accuracy between 1.3 mm and 1.8 mm when a distance correction was applied. Both motion patterns were tracked successfully with a root mean squared error (RMSE) of 1.4 and 1.1 mm respectively. Conclusion: Kinect v2 sensors are capable of tracking pre-programmed motion patterns with an accuracy <2 mm and appear robust to the radiotherapy treatment environment. A clinical trial using Kinect v2 sensor for monitoring voluntary breath hold has ethical approval and is open to recruitment. The authors are supported by a National Institute of Health Research (NIHR) Career Development Fellowship (CDF-2013-06-005). Microsoft Corporation donated three sensors. The views expressed in this publication are those of the author(s) and not necessarily those of the NHS, the National Institute for Health Research or the Department of Health.« less
  • Purpose: To devise and implement a new system to measure and register the patient motion during radiotherapy treatments. Methods: The system can obtain the position of several points in the 3D-space, through their projections in the 2D-images recorded by two cameras. The algorithm needs a series of constants, that are obtained using the images of a calibrated phantom.To test the system, some adhesive labels were placed on the surface of an object. Two cameras recorded the moving object over time. An in-house developed software localized the labels in each image. In the first pair of images, the program used amore » first approximation given by the user. In the subsequent images, it used the last position as an approximate location. The final exact coordinates of the point were obtained in a two-step process using the contrast of the images. From the 2D-positions of the point in each frame, the 3D-trajectories of each of these marks were obtained.The system was tested with linear displacements, oscillations of a mechanical oscillator, circular trajectories of a rotating disk, and with respiratory motion of a volunteer. Results: Trajectories of several points were reproduced with sub-millimeter accuracy in the three directions of the space. The system was able to follow periodic motion with amplitudes lower than 0.5mm; and trajectories of rotating points at speeds up to 200mm/s. The software could also track accurately the respiration motion of a person. Conclusion: A new, inexpensive optical tracking system for patient motion has been demonstrated. The system detects motion with high accuracy. Installation and calibration of the system is simple and quick. Data collection is not expected to involve any discomfort for the patient, nor any delay for the treatment. The system could be also used as a method of warning for patient movements, and for gating. We acknowledge financial support from Fundacion Mutua Madrilena, Madrid, Spain.« less