Publication: From Macro to Micro: Autonomous Multiscale Image Fusion for Robotic Surgery

by Lin Zhang, Menglong Ye, Petros Giataganas, Michael Hughes, Adrian Bradu, Adrian G.H. Podoleanu, and Guang-Zhong Yang

Journal: Robotics and Automation Magazine (2017)

Published: 10th May 2017


This paper is the result of a collaboration with the Applied Optics Group and the Hamlyn Centre for Robotic Surgery at Imperial College, London, to integrate two high-resolution, probe-based imaging systems – endoscopic microscopy and optical coherence tomography (OCT) – with the da Vinci surgical robot, and then using these imaging systems to support automated scanning to generate large-area maps of tissue at high resolution.

The imaging systems and the robot


The two different imaging modalities support the robotic guidance in different ways. The endomicroscope provides an en face view of the tissue (as though you are looking down on it from the top), whereas the OCT gives a cross-sectional image through the tissue. The endomicroscope images are circular, and about a quarter of a millimetre in diameter, while the OCT gives cross-sectional images that are 5 mm in depth (although in practice it can only penetrate 1-2 mm into tissue before the signal becomes too small to be useful). The endomicroscope images were used to guide the position along the surface of the tissue, and the OCT images were used to guide the robot’s distance from the tissue. Between the two, this provided three-dimensional control of the robot, with accuracy measured in the 10’s of microns. This made the robotic control far more accurate than simply relying on its own, internal position estimation.