3D Video: From Capture to Diffusion

Chapter 7

Multi- and Stereoscopic Matching, Depth and Disparity

7.1. Introduction

Three-dimensional (3D) reconstruction using stereo-correlation relates to the automatic extraction of data about the scene’s 3D structure from 2 to N images acquired simultaneously. In this context, in order to estimate depth within a scene, the 3D points are triangulated using their projections in the images taken from different viewpoints and the characteristics of the capture system. This problem therefore relates to matching¹ homologous pixels (i.e. projections of the same 3D point in images). This research can be based on specific geometric constraints, including the epipolar constraint that creates a first-order indeterminacy, reducing the search space to a segment. The photometry compared between pixels from different images is therefore used to match homologues although anomalies (similar photometries or variations in brightness) may occur, requiring the use of more complex heuristics or information redundancy. Matching pixels, in this context, are known as stereoscopic matching.

We will first introduce the difficulties related to homologue searches as well as primitives and capture geometry. We will then examine the generic algorithms of two existing approaches with the most commonly used constraints and costs. Second, we will concentrate on the occlusion problem by describing two approaches, the first being stereoscopic and the other being multiscopic.

7.2. Difficulties, primitives and stereoscopic ...