KR20040006738A - Device for Hierarchical Disparity Estimation and Method Thereof and Apparatus for Stereo Mixed Reality Image Synthesis using it and Method Thereof - Google Patents

Abstract

PURPOSE: A unit for a hierarchical variance estimation and a method thereof, and a device for synthesizing a stereo mixing reality video using the same and a method thereof are provided to reduce a load for a calculation by estimating a variance by raising gradually from a video of a low resolution sequentially. CONSTITUTION: A noise removing unit(511) removes a noise of a stereo video acquired from a camera. A balancing unit(513) solves an imbalance being generated caused by a discordance of a camera in acquiring the stereo video. An edge extractor(515) extracts an edge which is a boundary between two areas having different brightness with respect to a video received from the balancing unit(513). A hierarchical variance estimation unit(100) estimates a variance as to the video received from the pre-processing unit(510). A virtual video processing unit(530) models the same video as obtained from the left and right camera through a stereo rendering process. A video synthesis unit(520) synthesizes a real video and a virtual video using variance information of a real video received from the pre-processing unit(510) and a real video received from the hierarchical variance estimation unit(100), virtual video received from the virtual video processing unit(530), and depth information of the virtual video.

Description

Hierarchical Disparity Estimation and Method Thereof and Apparatus for Stereo Mixed Reality Image Synthesis using it and Method Thereof}

The present invention provides a hierarchical disparity estimating apparatus and a method thereof, and a computer-readable recording medium recording a program for realizing the method, a stereo mixed reality image synthesizing system and a method using the hierarchical disparity estimating apparatus and the method, A computer readable recording medium having recorded thereon a program for realizing the above method, and more particularly, a hierarchical disparity estimating apparatus for use in video media technology and a computer readable program having recorded thereon a program for realizing the method. A recording medium, a hierarchical disparity estimating apparatus and a stereo mixed reality image synthesizing system using the method, and a method and a computer readable recording medium storing a program for realizing the method.

With the development of video technology, today's world is moving away from the limitations of existing two-dimensional information representation and the video media technology that considers the user interface more naturally and realistically for three-dimensional representation, recording, and playback of human visual sense and perception. I became interested in

In addition, the development of computer graphics technology has enriched the content we encounter, and now we have come to mixed reality technology that can directly help human beings in reality.

In general, the mixed reality is that there is a real world environment where a person lives and a virtual environment that is a computer-generated virtual space. There must be an organic relationship between the synthesized real world and the virtual environment. That is, when a user views an image synthesized from the real world environment and the virtual environment, the user must accurately match the real world environment and the virtual world environment in three dimensions so that the user does not feel the synthesized trace.

However, the existing image synthesis for the mixed reality has a problem such as obstruction by the distance difference between objects.

On the other hand, in order to synthesize a mixed reality image, an analysis of the acquired image is required, and a depth estimation process of measuring a distance from the acquired left and right images to an object in the image is essential when a stereo camera is acquired for 3D image acquisition. to be.

The depth estimation may be obtained by Equation 1 using the disparity information d , which is the positional difference between the focal length λ of the camera, the size B of the baseline, and the matching point in each image.

In order to obtain disparity information d , which is a position difference between registration points in each image, Equation 1 uses disparity estimation in the stereo image coding theory.

The conventional disparity estimation method includes a block-based method, an object-based method, a mesh-based method, a disparity-space image (DSI) method, and a pixel-based method. (Pixel-based) method.

However, the conventional disparity estimation method is not suitable for depth estimation for image synthesis due to blocking or excessive computational load.

The present invention has been proposed to solve the above-mentioned problems of the prior art, and provides a hierarchical disparity estimator which reduces the computational load by estimating the disparity while sequentially increasing the resolution from a low resolution image. There is a purpose.

In addition, another object of the present invention is to provide a hierarchical disparity estimation method for reducing the computational load by estimating disparity while sequentially increasing the resolution from a low resolution image.

In addition, the present invention provides a computer-readable recording medium having recorded thereon a program for realizing a hierarchical disparity estimating method for reducing the computational load by estimating disparity while sequentially increasing resolution from a low-resolution image. There is another purpose.

In addition, the present invention provides a stereo mixed reality image synthesizing apparatus using a hierarchical disparity estimator for naturally synthesizing a computer-generated virtual image by extracting a feature in a stereo image and estimating disparity information in the image based on the feature. There is another purpose.

In addition, the present invention provides a stereo mixed reality image synthesis method using a hierarchical disparity estimation method that can be naturally synthesized with a computer-generated virtual image by extracting features in the stereo image and estimating the variation information in the image based on the feature. There is another purpose.

In addition, the present invention is to extract a feature in a stereo image and to estimate the disparity information in the image based on it, to realize a stereo mixed reality image synthesis method using a hierarchical disparity estimation unit that can be naturally synthesized with a computer-generated virtual image Another object is to provide a computer readable recording medium having recorded a program.

1 is a structural diagram illustrating an hierarchical disparity estimator in accordance with an embodiment of the present invention;

FIG. 2A is an exemplary diagram for describing a variation estimation result of the initial variation estimator of FIG. 1; FIG.

FIG. 2B is an exemplary diagram for describing a candidate vector of the block disparity estimator of FIG. 1; FIG.

FIG. 2C is an exemplary view for explaining the fine mutation allocation of the fine mutation allocator of FIG.

3 is a flowchart illustrating a hierarchical disparity estimation method according to the present invention;

FIG. 4 is a detailed flowchart illustrating an example of fine grain allocation of FIG. 3; FIG.

FIG. 5 is a structural diagram for explaining a stereo mixed reality image synthesizing apparatus using a hierarchical disparity estimator according to the present invention; FIG.

6 is a flowchart illustrating a method of synthesizing a stereo mixed reality image using a hierarchical disparity estimator according to the present invention;

7A to 7J are exemplary views for explaining the results of each component of the stereo mixed reality image synthesizing apparatus using the hierarchical disparity estimator according to the present invention.

* Explanation of symbols for the main parts of the drawings

100: hierarchical disparity estimator 101: initial disparity estimator

103: block variation estimator 105: fine mutation allocator

510: preprocessor 511: noise canceller

513: balancer 515: edge extractor

520: Image synthesizer 530: Virtual image processor

According to the present invention for achieving the above object, the variance estimation while sequentially increasing the resolution from a low-resolution image, and in the hierarchical disparity estimation unit for obtaining the disparity information of the image, by half-down sampling the input image Initial variation estimating means for performing initial variation estimation; Block displacement estimating means for performing disparity estimation on a block-by-block basis based on the initial disparity received from the initial disparity estimating means; And fine variation allocating means for allocating fine variation to each pixel by using the edge information of the image and the variation information in units of blocks estimated from the block displacement estimating means.

In addition, the present invention is to estimate the disparity while sequentially increasing the resolution from a low-resolution image, and in the hierarchical disparity estimation method for obtaining the disparity information of the image, the initial disparity estimation is performed by half down sampling the input image The first step to do; A second step of performing variation estimation on a block-by-block basis based on the initial variation; And assigning a fine variation to each pixel by using the edge information of the image and the variation information of the block unit.

The present invention also provides a hierarchical disparity estimation apparatus having a microprocessor to provide a hierarchical disparity estimation method for sequentially estimating disparity from a low resolution image and obtaining disparity information of the image. A first function of performing initial shift estimation by down-sampling an image; A second function of performing variation estimation on a block-by-block basis based on the initial variation; And a computer-readable recording medium having recorded thereon a program for realizing a third function of allocating a fine transition to each pixel by using the edge information of the image and the shift information of the block unit.

The present invention also provides a stereo mixed reality image synthesizing apparatus using the hierarchical disparity estimator of claim 1 to naturally synthesize a computer-generated virtual image and a stereo real image, and performs preprocessing on the stereo real image. Pre-treatment unit for; The hierarchical disparity estimator for performing disparity information estimation hierarchically on an image received from the preprocessing means; A virtual image processor for modeling the virtual image; And image depth using the depth information of the image received from the preprocessor and the virtual image processor, the real image received from the hierarchical disparity estimator, and the depth information of the virtual image received from the virtual image processor. It includes an image synthesizer.

In addition, the present invention is a stereo mixed reality image synthesis method using the hierarchical disparity estimation method of claim 3 to synthesize a computer-generated virtual image and stereo real image, the preprocessing for the stereo real image A first step for performing; A second step of performing disparity information estimation hierarchically on the preprocessed image; A third step of modeling the virtual image; And a fourth step of performing image synthesis using the preprocessed image and the modeled virtual image, the depth information of the real image and the depth information of the virtual image.

The present invention also provides a mixed reality image synthesizing apparatus having a microprocessor to provide a stereo mixed reality image synthesizing method using a hierarchical disparity estimation method in order to naturally synthesize a computer-generated virtual image and a stereo real image. A first function for performing preprocessing on the live-action image; A second function for performing disparity information estimation hierarchically on the preprocessed image; A third function for modeling the virtual image; And a program for realizing a fourth function for performing image synthesis using the preprocessed image and the modeled virtual image, the depth information of the real image and the depth information of the virtual image. Media.

The above objects, features and advantages will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. First of all, in adding reference numerals to the components of each drawing, it should be noted that the same components have the same number as much as possible even if displayed on different drawings. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a structural diagram illustrating an hierarchical disparity estimator in accordance with an embodiment of the present invention.

As shown in the figure, the hierarchical disparity estimator 100 of the present invention includes an initial disparity estimator 101, a block disparity estimator 103, and a fine disparity allocator 105.

In the general block matching, the larger the size of the block, the more information is contained in the block, and thus the accuracy of the variation itself increases. However, since one representative vector is allocated to many pixels, the precision of the overall variation decreases.

In order to overcome this problem, the initial disparity estimator 101 performs a function of performing disparity estimation by down-sampling an input image by 1/2.

In the present invention, the region division and the bidirectional estimation method in block units are applied to the initial variation estimation.

The region division method is a method of performing a matching process by dividing an area around the matched points by first performing matching in order from pixels having the highest probability of matching.

The bidirectional estimation method uses a disparity vector in a left image and a disparity vector from a right image matched with the same absolute value and opposite directions. By using this property, the reliability of the matching process can be estimated. Areas with low reliability may be excluded from the area partitioning process to increase overall reliability.

The transition vector from the left image to the right image ego The point of the right image corresponding to The disparity vector found by the same process In this case, bidirectional matching is satisfied when the following Equation 2 is satisfied.

Pixels that do not satisfy the above equation for a given threshold value ε are classified as hidden regions and are excluded from the region dividing scheme. Pixels satisfying the above equations become region dividing points to limit the search range when estimating the variation of the next pixels. .

FIG. 2A is an exemplary diagram for describing a variation estimation result of the initial variation estimator of FIG. 1.

As shown in the figure, by accurately estimating the variation in the original image based on the initial variations with respect to the input image, the accuracy and performance of the variation can be improved.

The initial variation by the initial variation estimator 101 may be used as a candidate vector for the block variation estimator 103.

The block disparity estimator 103 is responsible for performing disparity estimation in units of blocks in the original image based on the initial disparity received from the initial disparity estimator 101.

In this case, the vector used as the candidate vector is the initial vector to which the block belongs and its neighboring initial vectors.

FIG. 2B is an exemplary diagram for describing a candidate vector of the block disparity estimator of FIG. 1.

As shown in the figure, the search range of the disparity estimation of the block disparity estimator of the present invention is set to ± a around the candidate vector.

However, if the candidate vector exists outside the region to which it belongs according to the divided region at the time of initial variation estimation, it is possible to prevent the wrong variation from being propagated by excluding it from the candidate.

For natural synthesis, precise disparity information in pixel units is required, and in particular, a clear disparity difference at an interface is essential. However, if the variation in the pixel unit is estimated again based on the estimated variation in the block unit, it takes a lot of time, and since the accurate variation is not extracted in the hidden region, the boundary portion may obtain very uncertain variation information.

Accordingly, the fine shift allocator 105 of the hierarchical shift estimator allocates an appropriate shift for each pixel by using the edge information of the image and the shift information in block units estimated from the block shift estimator 103. In charge of the function.

Edge information of the image will be described later.

FIG. 2C is an exemplary view for explaining the fine mutation allocation of the fine mutation allocator of FIG.

As shown in the figure, the fine shift allocator of the present invention assigns a shift of a block to each pixel according to edge information.

3 is a flowchart illustrating a hierarchical disparity estimation method according to the present invention.

As shown in the figure, the variation estimation method of the present invention performs an initial variation estimation by half-sampling (S310), and performs a variation estimation in units of blocks in the original image based on the received initial variation. (S320).

Then, by using the edge information of the image and the disparity information of the block unit estimated by the block disparity estimator 103, an appropriate disparity is allocated to each pixel to estimate the disparity (S330).

FIG. 4 is a detailed flowchart of an exemplary embodiment for explaining the fine mutation allocation of FIG. 3.

The variation allocation of the present invention is performed in units of pixels, and as shown in the figure, starting at the position for each pixel and proceeding in four directions of up, down, left, and right (S401), it is determined whether all four directions have edges (S403). ).

If there are no edges in all four directions, it is determined whether there is a direction to go to another block without meeting the edge (S405), and if so, it is again determined whether the neighboring block is a smooth area (S407).

In the above description, the smoothing region is defined as a region in which a portion without edges is widely distributed.

As a result of the determination, when the neighboring block is a smooth region, the variation of the neighboring block is allocated as the fine variation of the corresponding pixel without being averaged to prevent blurring (S411). As a result, when the neighboring region is not the smooth region, the variation of the neighboring blocks in the directions is averaged and assigned as the fine variation of the pixel (S411).

If there is no direction to cross the other block without meeting the edge (S405), the processing is performed in the same manner as if there is no direction in all four directions.

On the other hand, when all four directions are blocked by the edge (S403), it is determined whether there is the variation information of the block itself (S413), and if there is, the variation information of the block itself is allocated as a fine variation (S415), and the variation within the block is determined. If does not exist, the variation information of the neighboring block and the block to which it belongs is weighted averaged according to the distance and assigned as a fine variation (S417).

Hereinafter, an apparatus and method for synthesizing stereo reality using the hierarchical disparity estimator and the method will be described.

FIG. 5 is a diagram illustrating an embodiment of a stereo mixed reality image synthesizing apparatus using a hierarchical disparity estimator according to the present invention.

As shown in the figure, the stereo mixed reality image synthesizing apparatus of the present invention includes a preprocessor 510, a hierarchical disparity estimator 100, an image synthesizer 520, and a virtual image processor 530. .

In addition, the preprocessor 510 includes a noise canceller 511, a balancer 513, and an edge extractor 515, and mainly processes a live image obtained by a camera.

The noise canceller 511 is responsible for removing noise of a stereo image obtained from a camera.

The noise present in the video signal can be classified into noise independent of the video signal and noise dependent. Since both of them are fundamental causes of deterioration of image quality, it is essential to remove such noise.

The balancer 513 is responsible for eliminating inconsistency (Imbalancing) caused by inconsistency of the camera in the process of acquiring a stereo image.

The inconsistency is caused by differences in the brightness of the acquired image because the parameters of the left and right cameras do not exactly match, or the difference in the brightness of the acquired image due to the distance between the object and the camera due to the position of the light source. It is a problem.

Inconsistencies can cause errors in the estimation of the variance, so they must be resolved during the preprocessing.

The edge extractor 515 is responsible for extracting an edge, which is a boundary between two regions having relatively different contrast, with respect to the image received from the balancer 513.

The hierarchical disparity estimator 100 is responsible for estimating the disparity with respect to the image received from the preprocessor 510.

The hierarchical disparity estimator 100 has been described with reference to FIGS. 1 and 2.

The virtual image processor 530 is responsible for modeling the same image as obtained from the left and right cameras through a stereo rendering process.

Since the virtual image, which is the output of the virtual image processor, extracts depth information during modeling, there is no need to go through a depth estimation process.

The image synthesizing unit 520 may be configured to determine the disparity information of the live image received from the preprocessor 510 and the live image received from the hierarchical disparity estimator 100, and the virtual image received from the virtual image processor 530. It is responsible for synthesizing the real image and the virtual image using the depth information of the virtual image.

The fine variation received from the hierarchical disparity estimator 100 is converted into depth information by Equation 1 above.

The image synthesizing unit 520 compares the depth information of the virtual image and the depth information of the real image for each pixel, and synthesizes the images by arranging images having a large depth at the rear and images having a small depth at the front.

6 is a flowchart illustrating a method of synthesizing a stereo mixed reality image using a hierarchical disparity estimator according to the present invention.

As shown in the figure, in the stereo mixed reality image synthesis method of the present invention, first, a real image is obtained by a stereo camera, and a virtual image is obtained by a computer (S601). In this case, the virtual image already includes depth information of each pixel.

The real image is subjected to preprocessing such as noise removal, balancing, and edge extraction (S603), and hierarchical disparity estimation is performed on the real image to extract depth information of each pixel (S605).

Subsequently, when the depth of the pixel of the real image is greater than the depth of the virtual image pixel, the pixel of the virtual image is disposed (S609), and when the depth of the pixel of the real image is greater than the depth of the virtual image pixel, the actual image Pixels are disposed (S611).

The arrangement of the pixels is performed on all the pixels of the image (S613).

7A to 7K are exemplary views for explaining the results of each component of the stereo mixed reality image synthesizing apparatus using the hierarchical disparity estimator according to the present invention.

FIG. 7A is a 640 × 480 RGB stereo live image obtained by a parallel multi-camera, showing left and right images, respectively. FIG. 7B is “fish” modeled in 3D MAX and stereo-rendered by the virtual image processor 530. And "spot".

FIG. 7C illustrates depth information of the virtual image of FIG. 7B stored in the virtual image processor 530.

In an embodiment of the present invention, the depth information of the left image is obtained by estimating the variation from the left image of FIG. 7A to the right image, and then synthesized into the left image.

FIG. 7D illustrates an image in which noise before passing through the noise canceller 511 is inserted, and FIG. 7 illustrates an image in which noise is removed from the noise canceller 511.

7F illustrates an image output from the edge extractor 515, and FIGS. 7G to 7I illustrate images output from the initial shift estimator 101, the block shift estimator 103, and the fine shift allocator 105, respectively. Indicates.

Lastly, FIG. 7J is an image output from the image synthesizing unit 530 and indicates that a relative sense of depth between objects is expressed.

The method of the present invention as described above may be implemented as a program and stored in a computer-readable recording medium (CD-ROM, RAM, ROM, floppy disk, hard disk, magneto-optical disk, etc.).

The present invention described above is capable of various substitutions, modifications, and changes without departing from the technical spirit of the present invention for those having ordinary skill in the art to which the present invention pertains. It is not limited to the drawing.

The present invention as described above has the effect of reducing the computational load by estimating the depth by estimating the variance while sequentially increasing the resolution from a low resolution image.

In addition, the present invention has an effect that can be naturally synthesized with a computer-generated virtual image by extracting features in the stereo image and estimating the variation information in the image based on the feature.

In addition, the present invention has the effect of reducing the cost and time consumed for image synthesis.

In addition, the present invention is effective to contribute to the active exchange of research in the field of computer graphics and image signal processing.

Claims (16)

In the hierarchical disparity estimator for estimating disparity while sequentially increasing resolution from an image having a low resolution, and obtaining disparity information of the image, Initial disparity estimating means for down-sampling the input image to perform initial disparity estimation; Block displacement estimating means for performing disparity estimation on a block-by-block basis based on the initial disparity received from the initial disparity estimating means; And Fine variation allocating means for allocating fine variation to each pixel by using edge information of the image and variation information in units of blocks estimated from the block displacement estimating means Hierarchical variation estimator comprising a. The method of claim 1, The initial variation estimating means, A hierarchical disparity estimator which performs initial disparity estimation by using a block-based region division and a bidirectional estimation method. The method of claim 2, The initial variation estimated by the initial variation estimating means, And a hierarchical disparity estimating unit used as a candidate vector of the block disparity estimating unit. In the hierarchical disparity estimation method for sequentially estimating the disparity while increasing the resolution from a low-resolution image, and obtaining the disparity information of the image, A first step of performing an initial shift estimation by half-sampling an input image; A second step of performing variation estimation on a block-by-block basis based on the initial variation; And A third step of allocating a fine shift to each pixel by using the edge information of the image and the shift information of the block unit; Hierarchical variation estimation method comprising a. The method of claim 4, wherein The third step, A fourth step of determining whether or not an edge exists in four directions of up, down, left, and right of the pixel; A fifth step of determining whether there is a direction to move to another block when there are no edges as a result of the determination in the fourth step; A sixth step of determining whether the corresponding part is a smoothed area and allocating a fine variation of the corresponding pixel according to the result of the determination in the fifth step if there is a part to be moved to another block; And As a result of the determination of the fourth step, when there are all edges, a seventh step of allocating a fine variation of the corresponding pixel according to whether or not there is variation information of the block itself. Hierarchical variation estimation method comprising a. The method of claim 5, As a result of the determination of the fifth step, when there is no direction to move to another block, an eighth step of performing the seventh step Hierarchical disparity estimation method characterized in that it further comprises. The method according to claim 5 or 6, The sixth step, A ninth step of judging whether a portion to be passed to another block is a smooth area; As a result of the ninth step, in the case of a smooth region, a tenth step of allocating transitions of neighboring blocks as fine transitions of the corresponding pixel; And As a result of the determination in the ninth step, in the case of the non-smooth region, the eleventh step of allocating the variation of the neighboring blocks to be assigned as the fine variation of the corresponding pixel. Hierarchical variation estimation method comprising a. The method according to claim 5 or 6, The seventh step, A ninth step of determining whether there is variation information of the block itself; A tenth step of allocating the variation information of the block itself as the fine variation when the variation information exists as the result of the ninth step; And As a result of the determination of the ninth step, when there is no variance information, an eleventh step of assigning the variance information of the neighboring block and the block to which the block belongs to the fine variance according to the distance Hierarchical variation estimation method comprising a. In the hierarchical disparity estimation apparatus having a microprocessor in order to provide a hierarchical disparity estimation method for sequentially estimating the disparity from the low resolution image and obtaining disparity information of the image, A first function of performing an initial shift estimation by half-sampling an input image; A second function of performing variation estimation on a block-by-block basis based on the initial variation; And A third function of allocating a fine shift to each pixel by using the edge information of the image and the shift information of the block unit A computer-readable recording medium that records a program for realizing the problem. In the stereo mixed reality image synthesizing apparatus using the hierarchical disparity estimator of claim 1 to naturally synthesize a computer-generated virtual image and a stereo live image, A preprocessor for preprocessing the stereo real image; The hierarchical disparity estimator for performing disparity information estimation hierarchically on an image received from the preprocessing means; A virtual image processor for modeling the virtual image; And An image for performing image synthesis using depth information of the image received from the preprocessor and the virtual image processor, the real image received from the hierarchical disparity estimator, and the depth information of the virtual image received from the virtual image processor. Synthesis section Stereo mixed reality image synthesis apparatus using a hierarchical disparity estimator including a. The method of claim 10, The preprocessing unit, Noise removing means for removing noise of the stereo real image; Balancing means for removing inconsistency with respect to the image received from the noise removing means; And Edge extracting means for extracting the edge of the image received from the balancing means Stereo mixed reality image synthesis apparatus using a hierarchical disparity estimator comprising a. The method of claim 10, The virtual image processor, Stereo mixed reality image synthesis apparatus using a hierarchical disparity estimator, characterized in that to extract the depth information of the virtual image. The method of claim 10, The image synthesis unit, Comparing the depth information of the virtual image and the depth information of the real-world image, a stereo image is synthesized using a hierarchical disparity estimator, wherein an image having a large depth is placed at the rear and an image having a small depth is placed at the front. Device. In the stereo mixed reality image synthesis method using the hierarchical disparity estimation method of claim 4 to naturally synthesize a computer-generated virtual image and stereo real image, A first step of performing preprocessing on the stereo real image; A second step of performing disparity information estimation hierarchically on the preprocessed image; A third step of modeling the virtual image; And A fourth step for performing image synthesis by using the preprocessed image and the modeled virtual image, the depth information of the real image and the depth information of the virtual image Stereo mixed reality image synthesis method using a hierarchical disparity estimation method comprising a. The method of claim 14, The fourth step, A fifth step of comparing depth information of the live image with depth information of the virtual image; A sixth step of disposing a pixel of the virtual image in a corresponding pixel when the depth of the actual image is greater than the depth of the virtual image; And A seventh step of disposing the pixel of the live image in the corresponding pixel when the depth of the live image is smaller than the depth of the virtual image as a result of the determination in the fifth step; Stereo mixed reality image synthesis method using a hierarchical disparity estimation method comprising a. In a mixed reality image synthesizing apparatus having a microprocessor to provide a stereo mixed reality image synthesizing method using a hierarchical disparity estimation method in order to naturally synthesize a computer-generated virtual image and a stereo real image, A first function for performing preprocessing on the stereo real image; A second function for performing disparity information estimation hierarchically on the preprocessed image; A third function for modeling the virtual image; And A fourth function for performing image synthesis by using the preprocessed image and the modeled virtual image, the depth information of the real image and the depth information of the virtual image A computer-readable recording medium that records a program for realizing the problem.