Adaptive Depth Map Estimation for Light Field Cameras using Machine Learning

Light field cameras, also known as plenoptic cameras, capture redundant information from the light that emanates from a scene. This redundancy allows a depth estimation of the captured scene and to refocus the image after it has been taken. Even though the light field camera’s concept was first formalized in 1908, it was not until recently that these cameras were produced for commercial use. There are two types of plenoptic cameras, the standard plenoptic camera and the multi-focus plenoptic camera, being Lytro the most popular manufacturer of the standard plenoptic and Raytrix the most popular manufacturer of the multi-focus one. Despite the advantages of the standard plenoptic camera like their simplicity and lower price, our work uses results captured by a multi-focus plenoptic camera due to its quality and higher resolution images.In this thesis we present an automatic algorithm capable of estimating the depth of a captured scene as if it was taken by a pinhole camera. The depth estimation starts with a RANSAC-like algorithm, generating a point cloud. Since this point cloud contains some outliers, in the previous work, an outlier removal filter was applied to achieve a more accurate point set. This point cloud is not immune to occlusions so, in order to solve this problem, we applied a z-buffer, eliminating all the occluded points from the point cloud. Regarding the depth estimation improvements, we present a supervised machine learning method that generates an adaptive depth map with several depths per micro lens, presenting a solid alternative to the methods presented in the previous work. This depth map will serve as an intermediate step to the dense depth map generation.We also present an improved implementation of the dense depth map synthesization algorithm. With the improvements made to this algorithm we are able to estimate a dense depth map regardless of the number of depths per micro lens of the intermediate depth map. On the previous work a plenoptic data simulator was introduced, allowing us to create plenoptic data sets with specific parameters. By knowing the depth ground truth of these data sets we are able to measure the error between our estimation and the ground truth. Being able to do this we can test and improve our algorithm and provide guidelines to future work. Our algorithm was also tested with real plenoptic images provided by Raytrix, but, since Raytrix does not provide the depth ground truth of their data sets, we can not compute the error of our estimations, thus we are only able to make a visual comparison of the results.