RU2479039C1 - Method of enhancing dense and sparse disparity maps, accuracy of reconstructed three-dimensional model and apparatus for realising said method - Google Patents

Abstract

FIELD: physics.

SUBSTANCE: method and apparatus employ computer graphics techniques to project (display, render) a primary three-dimensional model onto virtual viewing positions which coincide with positions of capturing the original images. The decoding texture is calculated, which enables to associate projection pixel coordinates with parameters of geometric beams emitted to corresponding points of the three-dimensional model. Stereoscopic juxtaposition of real images with corresponding displayed projections is carried out. Three-dimensional coordinates of the digital model are improved by translation of the found disparities, having the physical meaning of reverse engineering errors, in adjustments to the three-dimensional model using a decoding image. The process can take place iteratively until convergence, during which more accurate and sparse disparity maps and a three-dimensional model will be obtained.

EFFECT: improved quality of disparity maps and accuracy of the reconstructed three-dimensional model owing to simplification of the task of juxtaposing elements of stereoscopic pairs.

9 cl, 15 dwg

Description

The invention relates to a means of forming three-dimensional models (3D-models) based on two-dimensional images obtained from cameras, and in particular to a method for improving dense and sparse disparity maps, the accuracy of a reconstructed three-dimensional model and a device for implementing the method.

Currently, more and more often there is a need for the formation of various three-dimensional models that can be used for various purposes in different areas. For this, many methods and devices for forming three-dimensional scenes have been developed and exist.

The task of reconstructing the three-dimensional surfaces of objects and scenes from two-dimensional images is relevant at the moment in connection with the needs of three-dimensional data visualization technologies, the entertainment industry, medicine and other areas. The current level of technology has a sufficient number of stereoscopic methods for constructing disparity maps (English "disparity map") and their improvement by filtering methods, since disparity maps in combination with information on the parameters of the cameras that captured the stereo pairs make it possible to build a three-dimensional model of the observed scene and contained in it objects.

Various publications devoted to various problems of computing disparity maps are analyzed in review papers devoted to the classification and comparison of various approaches ([1] D. Scharstein and R. Szeliski, "A taxonomy and evaluation of dense two-frame stereo correspondence algorithms", International Journal of Computer Vision 47 (1/2/3), pp. 7-42, 2002, [2] Myron Z. Brown, Darius Burschka and Gregory D. Hager, "Advances in Computational Stereo", IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 25, NO. 8, August 2003).

The construction of dense disparity maps is associated with the use of rectangular neighborhoods of pixels to assess the degree of correlation of elements of the compared images. To construct sparse disparity maps by comparing feature points on images, descriptors that are scale-independent and rotationally invariant are used ([3] Lowe, David G. (1999). "Object recognition from local scale-invariant features" Proceedings of the International Conference on Computer Vision. 2. pp. 1150-1157, [4] David Gossow, Peter Decker and Dietrich Paulus, "An Evaluation of Open Source SURF Implementations", Lecture Notes in Computer Science, 2011, Volume 6556 (2011, 169-179), which are usually the basis for the subsequent construction of dense disparity maps ([5] Ju Yong Chang, Kyoung Mu Lee, Sang Uk Lee. Stereo matching using iterative reliable disparity map expansion in the color- spatial-disparity space. Pattern Recognition 40 (12): 3705-3713 (2007)).

However, when observing objects from different angles, the proportions of two-dimensional images of objects, their constituent elements and textures are different. For example, a rectangular section of one image may correspond to a trapezoidal section on another, while the shape of the corresponding section depends on the three-dimensional arrangement of a given element of a three-dimensional object; the circle observed from one observation point may look like an ellipse from another (see the example in FIG. 2). This leads to two complications. Firstly, during the construction of a dense disparity map, the calculation of the correlation by the usual method using sliding windows of the same shape for both compared images may not provide an optimal solution, since the neighborhoods do not seem similar for the corresponding pixels. Secondly, the search for correspondence of key points in images from different observation points by descriptors that are resistant to rotations and zooming allows you to find fewer matches due to the presence of projective distortions. The calculation of descriptors that are invariant to projective distortions is not practical in practice, since it will significantly increase the amount of computation. With the growth of the base, that is, the distance between the points of shooting, the problem is aggravated, since 2D images of objects are captured at significantly different angles. The relevance of the invention increases due to the fact that to obtain more accurate three-dimensional models, it is desirable to increase the base between the survey points to increase the accuracy of determining the three-dimensional coordinates of the points of the surfaces of objects using the triangulation method.

Many inventions are devoted to methods for refining disparity maps and three-dimensional models, such as: US 2006056727 A1, WO 2009069958 A2, US 2011080466 A1, KR 20110090488 A, WO 2009125988 A2, RU 2010103906 A, US 2011222756 A1, US 7561732 B1, KR 20070061094 A, WO 2008029345 A1, CN 101557534 A, US 2010309292 A1, US 2005190180 A1, SU 1830447 A1, SU 1793226 A1, KR 20020095752 A, WO 2004049255 A2, US 2004105580 A1, KR 20070061011 A, US 2011026809 A1, US 2005286758 A1, KR 100943 B1, WO 2009008864 A1, US 2009324059 A1, WO 2011104151 A1, WO 2011104151 A1, WO 2011081646 A1, KR 20050058085 A, WO 2009134155 A1, RU 2008144840 A, KR 20100088774 A, US 6046763 A, US 2004223640 A1, US 2003091225 WO 2006046180 A1, US 2010182406 A1, JP 2009043221 A, US 2009067705 A1, KR 100762670 B1, US 2011044531 A1, US 2006029270 A1, US 2011080464 A1, WO 2008156450 A1, US 2010208034 A1, US 2009073170 A1, WO 2009061 305 A1, US 2006 140510 A1, WO 2009124663 A1, KR 20080052363 A, WO 2010007285 A1, US 2010020178 A1, RU 2382406 C1, GB 2478156 A, KR 20100049855 A, US 2010220932 A1, RU 2009110511 A, US 2007286476 A1, WO 2010028718 A2 , RU 2008144840 A, RU 2009110511 A, JP 2006012166 A, WO 2011103865 A2, WO 2011118077 A1, WO 2011123174 A1, US 2011304708 A1, WO 2011157454 A1, US 2011091096 A1, WO 2008075271 A2, US 4695156 A, KR 100257335 B1 2010053416 A1, KR 20100135032 A, WO 2011014229 A1, KR 20110055020 A. However, in these inventions, unlike the proposed method:

(a) synthetic views (virtual view) are produced only as the final result, which serves the purpose of a person viewing a three-dimensional scene from various observation points, and not for purposes of comparison with the original images;

(b) the problem of perspective distortions in the compared images is not addressed;

(c) modules for the generation of synthetic species do not calculate decoding textures;

(d) it is not proposed to calculate the decoding image (texture) and use it to translate disparities between the original and synthetic types into corrections to the original disparity maps.

For example, in patent RU 2382406 C1, as in many similar patents from the above list, the virtual renderer (drawer, generator) is mentioned, but does not have feedback to the disparity calculation unit and does not calculate the decoding image. That is, it is only a means of displaying information and does not take part in the calculations of the disparity card itself or its improvements. In the patent RU 2419880 C2, in the feedback providing an iterative improvement of the disparity map, the view renderer is not mentioned, that is, the virtual view and the decoding texture are not considered as a source of additional information to clarify the disparity map.

In the application US 2010295853, published November 25, 2010, describes a method and system for constructing a three-dimensional model based on a depth map and rendering.

However, in the known method and system, the image received from the camera is not processed by removing moving objects, the disparity map is not compiled, which is then taken into account when constructing a three-dimensional model, and various additional objects are not superimposed.

In the article by Sang-Beom LEE, Sung-Yeol KIM and Yo-Sung HO “Multi-view image generation from depth maps and texture images using adaptive smoothing filter”, The 2008 International conference on Embedded Systems and Intelligent Technology, February 27-29, 2008, Grand Mercure Fortune Hotel, Bangkok, Thailand, found on the Internet at http://imaging.utk.edu/people/sykim/paper/2008_ICESIT_sblee.pdf, describes how to create a three-dimensional scene using the smoothing process by removing voids, building depth and rendering maps.

However, in the known method, the image received from the camera is not processed by removing moving objects, the disparity map is not compiled, which is then taken into account when constructing the three-dimensional model, and various additional objects are not superimposed.

In patent RU 2423018, publ. 06/27/2011, describes a method and system for constructing a three-dimensional model. To build a three-dimensional model, the disparity map of the original image and the depth map are calculated, after which the depth map is smoothed and a three-dimensional model is formed.

However, this patent does not disclose such steps as the removal of moving objects, followed by filling in the empty spaces formed after the removal of objects. Also, this patent does not describe the stage of rendering with the imposition of various additional objects on the disparity map.

The problem to which the claimed invention is directed is to improve the quality of disparity maps and the accuracy of the reconstructed three-dimensional model by simplifying the task of comparing elements of stereo pairs, which is achieved by eliminating differences in the proportions and scales of two-dimensional images of objects, their constituent elements and textures, due to observation of the scene from different angles and distances.

The technical result achieved by the claimed invention is the provision of an improved system, method and machine-readable medium, which provide advanced capabilities for the formation of three-dimensional models based on images received from cameras, with the possibility of overlaying additional objects on these models that are not in the images received from cameras, without loss of realism of the image.

An additional technical result is the reduction of time for processing images from cameras through the use of clusters and the distribution of tasks between these clusters.

The specified technical result is achieved by the fact that the proposed method and device for updating the disparity card (FIG. 1).

In this case, a method for improving the dense and sparse disparity maps, the accuracy of the reconstructed three-dimensional model includes the following operations:

using computer graphics methods, the primary three-dimensional model is projected (displayed, rendered) onto a virtual observation position that coincides with the position of the shooting point of the first source image of the stereo pair, using the second source image of the stereo pair as texture;

during projection, a decoding texture is calculated that allows the coordinates of the projection pixels to be associated with the parameters of the geometric rays released to the corresponding points of the three-dimensional model;

stereo-comparing the actual images with the corresponding displayed projections, while the methods of constructing dense disparity maps and searching for point correspondences (sparse disparity maps) work more productively and are able to find a larger number of matches than when comparing the original images;

translate the disparities found in this way between the original and synthetic image (projection), which have the physical meaning of reverse engineering errors, into corrections to the original dense and / or sparse disparity map using a decoding texture;

recalculate the three-dimensional model based on the updated disparity map.

Additionally, the initial approximation of the three-dimensional model is built by comparing other stereopairs obtained during the shooting of the scene from many different shooting points, the operation described in paragraph 1 of the formula is performed in pairs for several or all of the shooting points of the scene or iterated, up to convergence ( to the absence of significant changes, improvements in the disparity map). In addition, spherical or cylindrical cameras are used to obtain the original images; the original disparity map is filtered by a median filter or another filter that aligns the disparity map to eliminate impulse noise; To calculate (render) the virtual look and decode texture, a graphic processor (GPU) and one of the similar technologies for programming GPUs or implementing shaders (Cg, CUDA, Direct3D, DirectCompute, DirectX, OpenGL, GLSL, etc.) are used.

The method may include the fact that the original image is obtained by shooting the scene not with one camera from various points, but with a set of cameras located at different points in the scene.

The technical result is also ensured by the fact that the device for improving the dense and sparse disparity maps, the accuracy of the reconstructed three-dimensional model includes:

a unit for generating an initial three-dimensional model from an initial disparity map and known parameters of the cameras that performed the shooting;

a virtual view rendering unit that calculates a projection of a virtual view of a three-dimensional model that matches the origin of the survey and the corresponding decoding texture, which allows you to associate the coordinates of the projection pixels with the parameters of the geometric rays released to the corresponding points of the three-dimensional model;

a unit for calculating dense and sparse disparity maps between the original image and the synthetic projection;

a decoding (broadcasting) block of the disparity card as an amendment to the original disparity card using the decoding image.

In this case, using the methods of computer graphics, a primary three-dimensional model based on the input disparity map is projected (displayed, rendered) onto virtual observation positions that coincide with the positions of the shooting points of the source images. An important part of the invention is the calculation of the decoding texture during projection, which allows you to associate the coordinates of the projection pixels with the parameters of geometric rays released to the corresponding points of the three-dimensional model. Stereoscopic comparison of actual images with corresponding displayed projections is carried out. At the same time, methods for constructing dense disparity maps and searching for point correspondences (sparse disparity maps) work more productively and are able to find a larger number of correspondences than when comparing the original images. This allows you to refine the three-dimensional coordinates of the digital model by translating the disparities found in this way, which have the physical meaning of reverse engineering errors, into corrections to the three-dimensional model using the decoding image. The process can occur iteratively until convergence, during which a more accurate dense and sparse disparity map and a three-dimensional model will be obtained.

The method can be described in the form of stages in which:

a) split each image received from each camera into three images;

b) filter the split images obtained in step a) by removing moving objects from each split image and combine the filtered three images into one image;

c) divide each combined image obtained in step b) for each camera into disjoint blocks;

d) compile a disparity map for each divided image obtained in step c) based on cross-correlation;

e) combine fragments of the disparity map to form a newly combined image;

f) select values of the disparity map that are greater than the set value;

g) filter the aforementioned disparity map by removing parts of the image in which the disparity often changes, and fill in the resulting blanks, taking into account the average values of the areas near the borders of the blanks;

h) smooth each filtered image obtained in step g);

i) construct a depth map for each image obtained in step h);

j) form a three-dimensional model by combining the images obtained in step h) for all cameras, taking into account the obtained depth map and rendering the combined images with the superposition of at least one additional object on the resulting combined image.

In a particular embodiment of the method, images are obtained from seven cameras. The cameras have two positions: upper and lower. Each camera forms two images that are offset relative to each other along a vertical axis.

To split the image performed in step a), the cross-correlation method is used. In this case, a split-image filtering is performed by pixel-by-pixel comparison of three images received from each camera and selecting an average pixel value.

In step d), the disjoint blocks obtained in step c) are distributed among the cluster nodes. Moreover, clusters are multithreaded or single-threaded.

The cross-correlation used in step d) is performed by moving the template window of a certain size from the image received from the upper camera and comparing the image in the template window with the image from the lower camera. The disparity determined in step d) is defined as a function of the shift in the image. Moreover, the combination of images in step k) is performed in accordance with the partitioning algorithm performed in step c).

The setpoint used in step e) is the correlation value. At the same time, the parts of the image in which the disparity often changes are the areas of connection of the two fragments of images of the place where the cameras are mentioned, or other objects.

Filling in the blanks performed in step g) is carried out iteratively.

Each camera forms two images, each of which is a panorama of 1105 ° × 180 °. Each split image obtained in step a) has a size of 360 ° × 180 °.

Thus, in contrast to the methods and devices known from the prior art, the claimed invention is a device and a method for improving the disparity map based on a comparison of synthetic and real images similar in geometry and avoiding the inefficiency of the operation of algorithms for matching stereo image elements due to differences in angles shooting. An additional positive effect of the present invention is to reduce the range of searching for disparities in the x and y coordinates, so the required search range for the disparity of pixels between synthetic and real images is proportional to the error in the original disparity map, which, as a rule, is significantly lower than the range of variation in disparity in the process of calculating the original disparity map. This positive effect is especially important if the epipolar lines for comparing the source images are not straight lines (for example, if shooting was carried out with spherical cameras).

Further, the essence of the claimed technical solution is explained in detail with the involvement of graphic materials.

Figure 1. The general schematic diagram of the inventive device.

Figure 2. An illustration of the problem of comparing stereoscopic images associated with perspective distortions of projections obtained with different camera angles.

Figure 3. Explanation of the principle of synthesis of a decoding image and comparison of real and synthetic images (types of scenes).

Figure 4. The list of input and output signals of the proposed device.

Figure 5. The internal structure of the proposed device with a description of the operation of its constituent blocks.

6. Example of a Mountain image taken at point A.

7. Example image taken at point B.

Fig. 8. An example of a synthetic view from point A obtained using image B as a texture, an initial three-dimensional model, and data on camera parameters (internal and external). Evidence of similarity with the actual image obtained from point A.

Fig.9. Visualization of point correspondences obtained for the synthesis of a sparse disparity map in the case of comparing the original images A and B.

Figure 10. Visualization of the example of point correspondence obtained during the comparison of the image "Mountain" A, and the synthetic form of the model obtained from point A (image A *).

11. Point correspondence visualization for “NYS” example obtained using comparison of source images A and B.

Fig. 12. Point correspondence visualization for the “NYS” example obtained by comparing image A and the synthetic look of the model obtained from point A (image A *).

Fig.13. An example of an input NYS disparity card constructed using the SAD method.

Fig.14. Improved the proposed method of a dense disparity card "NYS".

Fig.15. A bar graph of the number of found point correspondences of a sparse disparity map for the examples "Mountain" and "NYS" in the case of comparison of the original species, or species obtained as a result of applying the method proposed in this invention.

Figure 2 shows that the camera observing the three-dimensional object "3D OBJECT" "on the right" forms a projection B, which has significant differences from the projection A formed by the "left" camera. Known from the prior art, methods for comparing such stereopairs using rectangular windows for matching (correlation, alignment) of image elements will not work efficiently, since they assume that the geometric distortions of the elements are insignificant within the match search window.

To overcome this problem, in this method, it is proposed to form a synthetic image A * to compare image B with image A, which is a projection for a virtual observation point C _{A of a} three-dimensional scene model with a texture superimposed on it in the form of image B obtained from the observation point C _B ( see Figure 3). If the three-dimensional scene model used to synthesize the projection A * is an approximation to the true three-dimensional model, then the projection A * will be similar to the original image A and the well-known methods of constructing dense disparity maps (such as SAD, for example), as well constructing sparse disparity maps (searching for corresponding key points by descriptors) will work much more efficiently, since the main differences in the proportions of image elements associated with a different observation angle will be eliminated. The degree of difference in the proportions will decrease with an improvement in the quality of the three-dimensional model, which makes it possible to use the proposed method in iterative mode. In the ideal (limiting, asymptotic) case, if there are no errors in certain positions (shooting points) and the values of the internal parameters of the cameras, three-dimensional image models A and A * should coincide pixel by pixel. The differences in the images A and A * are the disparity map between A and A *, which can be translated into amendments to the disparity map between A and B according to the formula:

where x is the two-dimensional coordinate vector {x, y};

D _AB (x) * - new values of the disparity map between images A and B (vector function having two components - disparity along the x axis and disparity along the axis of the image);

D _{AA *} (x) is the value of the disparity map between A and A * (a similar vector function of disparity along the x and y axes in return values);

U (x) is a vector function (decoding texture), calculated by the virtual-type synthesis module, whose values are the coordinates of the pixels of texture B, the color of which determined the color of the projection pixel A * with the x coordinate. Since the texture mapping process is known from the prior art, the possibility of constructing U (x) can be considered proven, since in order to display the pixel x of the projection A * during the texture mapping process (determining the color of the pixel x), it is necessary to get read access to the corresponding texture element (which is image B), therefore, it is necessary to calculate the texture address, which in essence is (up to a known linear transformation) the value of U (x). In practice, it is most convenient to implement the calculation of U (x) using shaders of the GPU, which is understandable for a specialist in the field of computer graphics.

To increase the stability of the convergence process, formula (1) can be modified by merging the new and previous values of the disparity maps by the formula

,

,

where Φ is the decisive rule by which the merging of old and new disparity values occurs (for example, only insignificant disparity changes can be allowed, not exceeding the threshold or relative degree of change, expressed as a percentage of the disparity value itself, etc.). A special case of the function Ф is the weighted sum

,

,

where the parameter k = 0 ... 1 has a physical meaning of the rate of convergence.

The symmetry of the procedure with respect to images A and B allows you to get a way to refine the disparity map D _BA (x) * by changing the places of variables A and B.

Thus, for the implementation of the above procedure, it is necessary to have the following set of input data (see Figure 4): source images (stereo pair); initial disparity maps obtained by the prior art, which play the role of first approximations; camera settings.

Schematic diagram of a device that implements the specified method is reflected in Figure 5. Block A1 implements, using a method known from the prior art, the formation of a three-dimensional model based on the obtained data on the parameters of the camera (shooting point) B and the disparity map. The output of the module is a three-dimensional model of the scene in the world coordinate system, which is fed to block A2, rendering it in a virtual view A *, which is a view of the three-dimensional model from position A (using camera A parameters for this), superimposing image B as the model texture . Block A2 has two output signals. In addition to image synthesis A *, block A2 calculates the decoding texture U (x) in accordance with the definition given above. The first output of block A2 (image A *) is connected to block A3. And the second output of block A2 (decoding texture) is connected to block A4. Block A3 calculates the sparse and dense disparity maps between images A and A * using one of the methods known to the person skilled in the art (for example, using SAD). Block A4 decoding the disparity card AA * using a decoding image, performing calculations according to formula (1), the output of block A4 is the improved value of the disparity cards for observation point A.

Figure 6-15 shows examples of the technical result of the application of this method. 6 shows an example of the original image A of the scene, code-named "Mountain". 7 shows an example of an original image B obtained from a different shooting point. Fig. 8 shows a synthetic view of a model textured by image B from the observation point A. It can be seen that the proportions of the elements of the objects in the image in Fig. 8 are much more consistent with the observed proportions of the elements in Fig. 6. Figure 9 shows the results of starting the process of searching for point correspondences in the image using key point descriptors invariant to scale and rotation (building a sparse disparity map). Fig.9 is an overlay image A and B, the combined key points of which are connected by lines. Figure 10 is a similar picture, but for images AA *. It can be seen that in Fig. 10 the number of lines is several times larger, and hence the correspondences found, and therefore, the quality of the sparse disparity map is better than in Fig. 9. It is also seen that the lines connecting the points in FIG. 10 are significantly shorter, which allows the use of locality to refine and simplify the search for matches. Similar examples are given for a scene codenamed “NYS” (FIG. 11 and FIG. 12). Figure 13 shows the original dense disparity map for point A of the NYS scene obtained by SAD. On Fig shows the improved proposed in this application method disparity card after three iterations of the device. Visually, a decrease in the number of artifacts, speckles and an increase in the degree of detail of some elements of the disparity map are observed. The quantitative characteristic of the increase as a result of applying the method of the number of found matches for the Mountain and NYS scenes is shown in Fig. 15 and reflects approximately a three-fold increase in the number of found matches (the columns of the diagram for the Moutain scene were increased 10 times for readability).

Claims (9)

1. A method for improving the dense and sparse disparity maps, the accuracy of the reconstructed three-dimensional model, which includes the following operations:
using computer graphics methods, the primary three-dimensional model is projected (displayed, rendered) onto a virtual observation position that coincides with the position of the shooting point of the first source image of the stereo pair, using the second source image of the stereo pair as texture;
during projection, a decoding texture is calculated that allows the coordinates of the projection pixels to be associated with the parameters of the geometric rays released to the corresponding points of the three-dimensional model;
stereo-comparing the actual images with the corresponding displayed projections, while the methods of constructing dense disparity maps and searching for point correspondences (sparse disparity maps) work more productively and are able to find a larger number of matches than when comparing the original images;
translate the disparities found in this way between the original and synthetic images (projections), which have the physical meaning of reverse engineering errors, into corrections to the original dense and / or sparse disparity map using a decoding texture;
recalculate the three-dimensional model based on the updated disparity map. 2. The method according to claim 1, characterized in that the initial approximation of the three-dimensional model is built by comparing other stereo pairs obtained during shooting the scene from many different shooting points. 3. The method according to claim 1, characterized in that the operation set forth in claim 1, is performed in pairs for several or all points of shooting the scene. 4. The method according to claim 1, characterized in that the operations of claim 1 are repeated iteratively, up to convergence (until there are no significant changes, improvements in the disparity map). 5. The method according to claim 1, characterized in that to obtain the source image using spherical or cylindrical cameras. 6. The method according to claim 1, characterized in that the original disparity map is subjected to filtration by a median filter or another filter that aligns the disparity map to eliminate impulse noise. 7. The method according to claim 1, characterized in that a graphic processor (GPU) and one of the similar technologies for programming graphics processors or implementing shaders (Cg, CUDA, Direct3D, DirectCompute, DirectX, are used to calculate (render) the virtual look and decode texture, OpenGL, GLSL, etc.). 8. The method according to claim 1, characterized in that the original image is obtained not by shooting the scene with one camera from various points, but by a set of cameras located at different points in the scene. 9. A device for improving the dense and sparse disparity maps, the accuracy of the reconstructed three-dimensional model, including:
a unit for generating an initial three-dimensional model from an initial disparity map and known parameters of the cameras that performed the shooting;
a virtual view rendering unit that calculates a projection of a virtual view of a three-dimensional model that matches the origin of the survey and the corresponding decoding texture, which allows you to associate the coordinates of the projection pixels with the parameters of the geometric rays released to the corresponding points of the three-dimensional model;
a unit for calculating dense and sparse disparity maps between the original image and the synthetic projection;
a decoding (broadcasting) block of the disparity card as an amendment to the original disparity card using the decoding image.

Images