Perspective shape from shading for wide-FOV near-lighting endoscopes (extended version)

In minimally invasive medical procedures the surgeon uses as guidance the video acquired by near-lighting endoscopic cameras. Near-lighting endoscopes are small size self-illuminated cameras that typically have a wide or very wide field of view (FOV), and hence their images are affected by high radial distortion and small spatial resolution in the periphery of the image (an object of the same size at equal distance gets smaller in the periphery). To help the interpretation of the images by the surgeon, the 3D modeling of the observed cavities is highly interesting and shape-from-shading is a natural choice in the 3D shape reconstruction of bones and organs. However, the shape from shading technique has been traditionally performed using the perspective projection model (perspective shape from shading – PSFS) which is barely appropriate for modeling endoscopes since it does not account for the high distortion and reduced resolution in image periphery. Our aim is thus to adapt the shape from shading technique to the particular case of near-lighting endoscopes. We propose in this paper two improvements to the state of the art methods for PSFS in Near-Lighting Endoscopes. The first contribution is the introduction of the radial distortion model directly in PSFS equations and the second contribution is the compensation of the reduced resolution of the image in its periphery, due to wide FOV. Tests performed in real objects and in a knee bone phantom show that by modeling these two effects our method highly improves the accuracy of the estimation.