KR100926520B1 - Apparatus and method of matching binocular/multi-view stereo using foreground/background separation and image segmentation - Google Patents

Abstract

The present invention relates to a binocular / multi-view stereo matching apparatus and method using object / background separation and image segmentation. The present invention relates to a binocular / multi-view image obtained from an imaging apparatus, divided into an object and a background area, and to each divided area. By dividing binocular / multi-view images and stereo matching based on the segmented segment, object / background separation and image, which can reduce misalignment due to the shielding region, thereby obtaining a more accurate disparity map. An object of the present invention is to provide a binocular / multi-view stereo matching device using segmentation.

To this end, the present invention divides the area of the binocular / multi-view image into an object area and a background area, respectively, and determines an area for generating object and background separation information for the binocular / multi-view image according to the determination result. Way; Disparity estimating means for dividing the binocular / multi-view image for each similar region and individually obtaining an object disparity map and a background disparity map by using the divided binocular / multi-view image and the generated object and background separation information ; And disparity map synthesizing means for synthesizing the object disparity map and the background disparity map to generate a scene disparity map for the binocular / multi-view image.

Disparity Map, Multiview Stereo Matching, Binocular Multiview Image, 3D Image Processing, Shielding Area, Object Area, Background Area, Image Segmentation, Mismatching, Segment Based Stereo Matching

Description

Binocular / Multi-view Stereo Matching Device Using Object / Background Separation and Image Segmentation and Its Method

The present invention relates to a binocular / multi-view stereo matching apparatus and method using object / background separation and image segmentation. More particularly, the present invention relates to a binocular / multi-view image obtained from an imaging apparatus. By dividing the binocular / multi-view image by the divided area and stereo matching based on the segment, the misalignment caused by the shielding area can be reduced, thereby making it possible to obtain more accurate disparity map. A binocular / multi-view stereo matching device using background separation and image segmentation and a method thereof are provided.

The present invention is derived from the research conducted as part of the next generation core technology development project of the Ministry of Information and Communication and the Ministry of Information and Telecommunication Research and Development. [Task Management Number: 2005-S-403-02, Title: Intelligent Integrated Information Broadcasting (Smart TV) )Technology].

Multi-view (stereo) image matching technology for extracting three-dimensional information is emerging as a foundation technology for the field of robot navigation, image-based rendering or the recent three-dimensional display.

The key to multiview image matching is to generate a disparity map for a given multiview (stereo) image. The edge refers to a position difference between two pixels corresponding to two given images. In general, in a stereo (binocular) image, it is assumed that two given images are camera calibrated and image calibrated. And within this hypothesis, the range of variation search for finding the corresponding point is limited to the one-dimensional epipolar line. However, even in such a constraint, there is a problem in that it is not possible to calculate an accurate variation due to the setting used when matching stereo images.

Conventional multi-view (binocular) matching techniques can be largely divided into window-based local methods and global techniques using energy minimization techniques. The local method has the advantage of fast performance, but has the disadvantage that the error of the overall disparity map is large. In addition, the global technique has the advantage of being relatively slow but capable of producing relatively accurate results.

In particular, in the conventional global technique, the cost function in the conventional global technique generally consists of a data term and a smoothing term. The data term is a term indicating how similar the color or brightness is between two pixels through modeling, and the smoothing term is a term indicating how smooth the object to be restored through modeling is.

However, the general cost function in this conventional global technique does not reflect the effect on the shielding of the object. Conventional stereo matching techniques have the problem of producing a result that is not robust to masking (shielding). In addition, although a stereo matching model has recently been proposed in consideration of the shielding area, there is a problem that it is difficult to obtain an accurate multi-view (binocular) matching result because the problem of the shielding area in the stereo matching is not completely solved.

As described above, the conventional multi-view (binocular) image matching technique has a problem that it is not possible to calculate the exact shift due to the setting used when matching the multi-view (binocular) image. In particular, in the conventional stereoscopic image matching technique, the multi-view (binocular) image matching technique using the global technique does not reflect the effect of the masking (shielding) of the target object, and thus is not robust to the masking (shielding). There is a problem that calculates the problem, and to solve this problem is an object of the present invention.

Accordingly, the present invention divides the binocular / multi-view image obtained from the imaging apparatus into an object and a background area, and performs stereo matching based on the segmented by dividing the binocular / multi-view image by each divided area. It is an object of the present invention to provide a binocular / multi-view stereo matching device and method using object / background separation and image segmentation, which can reduce mismatching due to a shielding area and thereby obtain a more accurate disparity map.

The objects of the present invention are not limited to the above-mentioned objects, and other objects and advantages of the present invention which are not mentioned above can be understood by the following description, and will be more clearly understood by the embodiments of the present invention. Also, it will be readily appreciated that the objects and advantages of the present invention may be realized by the means and combinations thereof indicated in the claims.

The present invention has been proposed in order to solve the above problem, and divided the binocular / multi-view image obtained from the imaging device into an object and a background area, and divided the image by dividing the binocular / multi-view image for each divided area ( Segment) is characterized in that the stereo matching.

More specifically, the apparatus of the present invention divides the area of the binocular / multi-view image into an object area and a background area, respectively, and determines the object and background separation information for the binocular / multi-view image according to the determination result. Area determining means for performing; Disparity estimating means for dividing the binocular / multi-view image for each similar region and individually obtaining an object disparity map and a background disparity map by using the divided binocular / multi-view image and the generated object and background separation information ; And disparity map synthesizing means for synthesizing the object disparity map and the background disparity map to generate a scene disparity map for the binocular / multi-view image.

In addition, the apparatus of the present invention, the multi-view image acquisition means for obtaining a binocular / multi-view image from an external imaging device; Camera information extracting means for calculating camera information and base matrix data through camera correction on the obtained binocular / multiview image; And variation map storage means for storing the scene variation map generated by the variation map synthesizing means.

Meanwhile, in the method of the present invention, an area of the binocular / multi-view image is divided into an object area and a background area, respectively, and is determined. Determination step; A variation estimating step of dividing the binocular / multi-view image for each similar region and separately obtaining an object disparity map and a background disparity map using the divided binocular / multi-view image and the generated object and background separation information; And synthesizing the object disparity map and the background disparity map to generate a scene disparity map for the binocular / multi-view image.

The method may further include a camera information extracting step of calculating camera information and base matrix data through camera correction for the binocular / multi-view image.

As described above, the present invention divides the binocular / multi-view image obtained from the imaging apparatus into an object and a background area, and divides the binocular / multi-view image for each divided area into an image based stereo segment. Matching has an effect of reducing mismatching caused by the shielding area.

Furthermore, the present invention has the effect of reducing mismatch by the shielding area and thereby obtaining a more accurate disparity map at the object and background boundary.

In addition, the present invention has an effect of reducing the execution time of stereo matching by separately obtaining the disparity maps for the object and the background area.

The above objects, features, and advantages will become more apparent from the detailed description given hereinafter with reference to the accompanying drawings, and accordingly, those skilled in the art to which the present invention pertains may share the technical idea of the present invention. It will be easy to implement. In addition, in describing the present invention, when it is determined that the detailed description of the known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating an embodiment of a binocular / multi-view stereo matching device using object / background separation and image segmentation according to the present invention.

As shown in FIG. 1, the binocular / multi-view stereo matching device 100 according to the present invention includes a binocular / multi-view image acquisition unit 110, a camera information extractor 120, an area determination unit 130, and variation. The estimation unit 140, the variation map synthesizing unit 150, and the variation map storage unit 160 are included.

The binocular / multi-view image acquisition unit 110 obtains binocular / multi-view images of the same time from a plurality of video cameras (multi-view cameras), respectively. That is, the binocular / multiview image acquisition unit 110 obtains a first viewpoint image from the video camera 1, a second viewpoint image from the video camera 2, and a third viewpoint image from the video camera 3, respectively, Store and synchronize the multi-view images. Here, the binocular / multi-view image includes a background image without an object and a scene image with an object in the background image.

In addition, the camera information extracting unit 120 performs a camera calibration process on the multi-view image acquired by the binocular / multi-view image obtaining unit 110, and displays the camera information such as the focal length and the mutual positional relationship between each viewpoint. Compute the fundamental matrix represented. At this time, the obtained camera information and the base matrix data are stored on a data storage device or a computer memory (not shown), and then used for transforming the disparity map.

Meanwhile, referring to a process of generating a disparity map of an object and a background area, first, in order to generate a disparity map of a separated background area, a disparity map of a background area is calculated in advance using only a background image from which an object is excluded. In order to generate the disparity map of the object region, the stereo matching technique is applied only to the pixels of the object region to reduce the execution time. Thereafter, the disparity map of the background area and the disparity map of the object area are synthesized to finally obtain a scene disparity map.

Hereinafter, a process of separating the object and the background region and generating the disparity map will be described in detail.

The area determiner 130 determines the area of the binocular / multiview image corrected by the camera by the camera information extractor 120 into an object area and a background area, respectively, and determines the area of the binocular / multiview image according to the determination result. Create object and background separation information for. That is, the area determiner 130 separates the background area having a small color or brightness change with time and the object area having a large color or brightness change with time using an object / background separation technique. The region determiner 130 generates object and background separation information about the separated object region and the background region. Detailed components of the area determination unit 130 will be described later in detail with reference to FIG. 2. For example, if the multiview image corrected by the camera by the camera information extractor 120 is three or more views, the area determination unit 130 may select one image pair from one of the remaining images adjacent to the random image among the multiview images and one image pair. Each image pair may be divided into an object region and a background region.

Thereafter, the variation estimator 140 splits the binocular / multi-view image transmitted from the region determiner 130 for each similar region. The disparity estimator 140 estimates the disparity map individually by using the image-divided binocular / multi-view image and the object and the background separation information. That is, the variation estimator 140 estimates a variation map (background variation map) for the background region using the background image, and a variation map (object variation map) for the object region using the scene image and the object and the background separation information. ) Is estimated.

The disparity map synthesis unit 150 generates a disparity map (scene disparity map) for the final scene image by synthesizing the background disparity map and the object disparity map estimated by the disparity estimator 140.

The variation map storage unit 160 stores the scene variation map generated for each pixel in the variation map synthesis unit 150. In this case, the variation map storage unit 160 may store the variation map as a digital image. Thereafter, the scene variation map may be converted into a 3D point cloud or a 3D model using camera information obtained by the camera information extractor 120.

FIG. 2 is a detailed block diagram of an embodiment of the area determiner of FIG. 1 according to the present invention. FIG.

As shown in FIG. 2, the region determiner 130 may include a distribution model estimator 21, a region classifier 22, a mixed distribution model estimator 23, and an object and background region determiner 24. Include. As described above, the area determiner 130 separates a given image into a background area having a small color or brightness change with time and an object area having a large color or brightness change with time using the object / background separation technique.

Referring to FIG. 2, the distribution model estimator 21 uses a background image in which the object is excluded from the binocular / multi-view image to determine a distribution model of color or brightness of each pixel (eg, Gaussian). ) Model). In this case, the background image may be a background image previously obtained or pre-stored by the binocular / backview acquirer 110. Here, the Gaussian model is defined as in Equation 1 below.

The area classifier 22 calculates a color generation probability for each pixel of the binocular / backview image. That is, the area classifier 22 calculates the probability of color or brightness of each pixel of the binocular / multiview image using the Gaussian model estimated as in Equation 1 above. The area classifier 22 classifies the pixel as a background, an object, or an undetermined pixel according to a comparison between the calculated color generation probability and a predefined threshold range.

Subsequently, the mixed distribution model estimator 23 uses a Gaussian Mixture model (Gaussian mixed model) for the background and the object by using color values of pixels belonging to the background or the object classified by the area classifier 22. Estimate The Gaussian mixture model is defined as in Equation 2 below.

는 상기의 [수학식 1]과 같이 정의된 가우시안 모델을 나타낸다.

는 각각의 가우시안 모델의 가중치를 나타내는 변수로서,

을 만족시킨다.here,

Denotes a Gaussian model defined as in Equation 1 above.

Is a variable representing the weight of each Gaussian model.

Satisfies

The object and background region determiner 24 combines a Gaussian mixture model and smoothing constraints for the background and the object estimated by the mixed distribution model estimator 23 to form an energy function as shown in Equation 3 below. define.

는 에너지 함수,

는 객체 및 배경 영역을 나타내는 함수,

는 각각 추정된 배경에 대한 가우시안 믹스쳐 모델,

는 각각 추정된 객체에 대한 가우시안 믹스쳐 모델을 나타낸다.

는 두 인자가 서로 다른 값을 가질 때에는 1, 같은 값을 가질 때에는 0의 값을 가지는 함수로서 평활화 구속조건을 나타낸다. 한편, 상기의 [수학식 3]과 같이 정의된 에너지 함수를 최적화하는 기법으로 그래프 컷(Graph cut) 기법이 적용될 수 있다.here,

Is an energy function,

Is a function representing the object and background area,

Is a Gaussian mix model for each estimated background,

Represents a Gaussian mixture model for each estimated object.

Is a function that has a value of 1 when two arguments have different values, and a value of 0 when they have the same value, indicating a smoothing constraint. Meanwhile, a graph cut technique may be applied as a technique for optimizing an energy function defined as in Equation 3 above.

Subsequently, the object and background region determiner 24 determines the object region or the background region using the defined energy function and generates the object and background separation information according to the determination result. That is, the object and background region determiner 24 repeatedly calculates energy to minimize the energy function of Equation 3 and determines that the object and background region are converged to the minimum value of the object region or the background region. For example, the object and background region determiner 24 arbitrarily assumes an area or a pixel to be determined as an object or a background region. The object and background region determiner 24 compares the minimum value of the energy function obtained for each of the hypothesized regions for each region and determines that the object and the background region are any one of an object or a background region having a lower value.

Next, the disparity estimator 140 separately estimates the disparity map of the object and the background area by using the aforementioned object and background separation information.

3 is a detailed block diagram of an embodiment of the variation estimator of FIG. 1 according to the present invention.

As illustrated in FIG. 3, the disparity estimator 140 includes an image splitter 31, a plane coefficient extractor 32, an energy function calculator 33, and a disparity map generator 34.

First, the image divider 31 divides the binocular / multi-view image, which is divided into an object region or a background region according to the object and background separation information generated by the object and background region determiner 24, into a segment unit that is a similar region unit. Split the image. For example, the image segmentation algorithm may use an average shift algorithm described in detail in "Mean Shift: A Robust Approach Toward Feature Space Analysis, PAMI" published in 2002 by "D. Comaniciu".

Next, the plane coefficient extractor 32 extracts a plane coefficient for each segment of the image segmented by the image divider 31.

Looking at the plane coefficient extraction process in detail, the plane coefficient extractor 32 estimates the initial disparity map using a similarity calculation function such as a sum of squared difference (SSD) within a 3 × 3 window. The plane coefficient extracting unit 32 calculates the plane coefficient by using an iterative calculation method for each segment. In this case, the plane coefficient extractor 32 uses only reliable disparity values obtained through cross checks of the left and right sides of each repetition process, and may perform plane coefficient estimation only for a large segment. This robust planar coefficient extraction method can be used in the paper published in "A global matching framework for stereo computation (ICCV)" published in 2001 by "H. Tao".

In addition, the energy function calculation unit 33 defines an energy function for each segment as shown in Equation 4 below to remove the pixels hidden by the shielding area by using the plane coefficients extracted by the plane coefficient extraction unit 32. do.

는 데이터 항,

는 평활화 항,

는 가리워짐 항을 나타낸다.here,

Is the data term,

Smoothing term,

Indicates a hidden term.

The energy function calculation unit 33 performs matching on a segment basis by using this energy function. In addition, the energy function calculation unit 33 uses the plane coefficient extracted from the plane coefficient extraction unit 32 instead of the existing variation label as the unit of the variation.

Here, looking at the energy function, the first term indicates how similar the colors of the corresponding pixels are as data terms. The second term is a smoothing term, and a Pots model such as Equation 5 below is used.

,

는 서로 인접한 세그먼트의 평면 계수,

는 두 세그먼트 사이의 공통된 경계 길이를 나타낸다.here,

,

Is the plane coefficient of adjacent segments,

Denotes a common boundary length between two segments.

), 두 세그먼트 사이의 공통된 경계 길이(

)에 비례하는 에너지 값을 부과한다.Unlike the general smoothing term, Equation 5 is different when the plane coefficients of adjacent segments are different (

), The common boundary length between two segments (

Impose an energy value proportional to

는 세그먼트

와

사이의 모든 가리워진 화소들의 데이터 항의 합을 나타낸다. 가리워진 화소들은 평면 계수

,

를 사용하여 그에 대응하는 각각의 세그먼트

와

를 워핑함으로써 얻어질 수 있으며, 이는 도 7에 나타나 있다. 즉, 에너지 함수는 데이터 항, 평활화 항, 및 차폐 영역으로 인해 가리워진 화소에 대한 가리워짐 항을 포함한다. 여기서, 에너지 함수에서의 가리워짐 항은 가리워진 화소들의 중복되는 데이터 항을 제거하는 기능을 수행한다. 이는 차폐 영역에 강인한 스테레오 정합 기법을 적용하기 위함이다.The last term is the hidden term,

Is a segment

Wow

It represents the sum of the data terms of all hidden pixels in between. Hidden pixels are plane coefficients

,

Each segment corresponding to it using

Wow

Can be obtained by warping, which is shown in FIG. That is, the energy function includes a data term, a smoothing term, and a hidden term for the pixel that is hidden due to the shielding area. Here, the hidden term in the energy function serves to remove the overlapping data term of the hidden pixels. This is to apply a robust stereo matching technique to the shielding area.

The disparity map generator 34 of the disparity estimator 140 individually generates disparity maps for the object and the background area by using the energy function defined in Equation 5 above. That is, the disparity map generator 34 generates a disparity map for each object region and background region by using an energy function parameter (for example, plane coefficient) in which the energy function is minimized. For example, the variation map generator 34 may minimize an energy function by using a reliability diffusion technique, which is an inference technique using a Markov Random Field.

4 is a flowchart illustrating a binocular / multi-view stereo matching method using object / background separation and image segmentation according to the present invention.

First, the binocular / multi-view image acquisition unit 110 acquires the binocular / multi-view image for the same time from a plurality of video cameras (binocular / multi-view camera) (402).

The binocular / multiview image acquisition unit 110 synchronizes and stores the obtained binocular / multiview image in operation 404.

Subsequently, the camera information extracting unit 120 uses the binocular / multiview image acquired by the binocular / multiview image obtaining unit 110 to perform camera calibration such as focal length and mutual information between each viewpoint through a camera calibration process. A fundamental matrix representing a positional relationship is calculated (406).

Subsequently, the area determining unit 130 classifies the area of the binocular / multi-view image into the object area and the background area, respectively, and generates object and background separation information on the area of the binocular / multi-view image according to the determination result ( 408). That is, the area determiner 130 separates the binocular / multi-view image into a background area having a small color or brightness change with time and an object area having a large color or brightness change with time using an object and background separation technique. The separation result is called object and background separation information.

The disparity estimator 140 splits the binocular / multi-view image divided into the object area and the background area for each similar area, and uses the binocular / multi-view image and the object and the background separation information on the image segmented object area. In operation 410, the object disparity map and the background disparity map for the background region are separately obtained.

Subsequently, the disparity map synthesis unit 150 synthesizes the object disparity map and the background disparity map to generate a scene disparity map for the binocular / multiview image. That is, the variation map synthesis unit 150 generates a variation map for the final scene by synthesizing the variation map for the estimated background region and the variation map for the object region (412).

The final variation calculated for each pixel is stored as a digital image in the form of a disparity map. The disparity map can be converted into a 3D point cloud or a 3D model using camera information.

FIG. 5 is a detailed flowchart illustrating an area determination method using a Gaussian model according to the present invention. FIG.

First, the distribution model estimator 21 acquires a pre-stored background image from which an object is excluded (502). In this case, the background image may be acquired in advance by the binocular / backview image acquirer 110.

The distribution model estimator 21 estimates a Gaussian model for the color or brightness of each pixel of the background image as shown in Equation 1 (504).

The area classifier 22 calculates a probability that such color or brightness occurs for each pixel when an input image including an object is given using a Gaussian model estimated as shown in Equation 1 above. The area classifier 22 classifies each pixel into a background, an object, or a pixel that has not yet been determined according to the calculated probability value (506).

The mixed distribution model estimator 23 estimates a Gaussian Mixture model for the background and the object using the color values of the pixels belonging to the classified background or the object as shown in Equation 2 above. 508).

The object and background region determiner 24 determines the object region or the background region using an energy function defined using the estimated Gaussian mixture model as described above, and generates the object and background separation information according to the determination result. (510).

6 is a detailed flowchart of an embodiment of a method for generating a disparity map using an energy function according to the present invention.

The image splitter 31 divides the image into segments based on the object and the background separated by the object and the background separation information.

The plane coefficient extractor 32 extracts an initial variation and a plane coefficient for each color segment by using the image segmentation result divided in step 602 (604).

The energy function calculator 33 defines an energy function as shown in Equation 4 above using the extracted plane coefficients (606).

The disparity map generator 34 generates disparity maps for the object region and the background region separately using the defined energy function (608).

7 is a diagram illustrating an embodiment of a shielding area according to the present invention.

As shown in FIG. 7, when the two arbitrary image areas 710 and 720 are viewed at different viewpoints, the shielding area 730 is generated due to the area that is covered by the viewpoint of the camera. In addition, in two arbitrary image areas 710 and 720, pixels hidden according to adjacent segments may occur.

The energy function calculation unit 33 according to the present invention calculates the energy function in consideration of the shielding area 730 that appears due to the two arbitrary image areas 710 and 720. That is, the energy function calculation unit 33 subtracts the energy of the shielding area included in the entire data term from the total energy as in [Equation 4] above to consider the shielding area (obscured pixel) 730. Obtain

On the other hand, the method of the present invention as described above can be written in a computer program. And the code and code segments constituting the program can be easily inferred by a computer programmer in the art. In addition, the written program is stored in a computer-readable recording medium (information storage medium), and read and executed by a computer to implement the method of the present invention. The recording medium may include any type of computer readable recording medium.

The present invention described above is capable of various substitutions, modifications, and changes without departing from the technical spirit of the present invention for those skilled in the art to which the present invention pertains. It is not limited by the drawings.

1 is a block diagram of an embodiment of a binocular / multi-view stereo matching device using object / background separation and image segmentation according to the present invention;

2 is a detailed configuration diagram of an embodiment of the region determination unit of FIG. 1 according to the present invention;

3 is a detailed configuration diagram of an embodiment of the variation estimator of FIG. 1 according to the present invention;

4 is a flowchart illustrating an embodiment of a binocular / multi-view stereo matching method using object / background separation and image segmentation according to the present invention;

5 is a detailed flowchart illustrating an area determination method using a Gaussian model according to the present invention;

6 is a detailed flowchart illustrating an embodiment of a method for generating a disparity map using an energy function according to the present invention;

7 is a diagram illustrating an embodiment of a shielding area according to the present invention.

* Explanation of symbols on the main parts of the drawing

110: binocular / multi-view image acquisition unit 120: camera information extraction unit

130: region determination unit 140: mutation estimation unit

150: variation map synthesis unit 160: variation map storage unit

21: distribution model estimation unit 22: region classification unit

23: mixed distribution model estimation unit 24: object and background region determination unit

31: image segmentation unit 32: plane coefficient extraction unit

33: energy function calculation unit 34: disparity map generator

Claims (12)

The binocular / backview is divided into a background area having a small change in color or brightness over time and an object area with a large change in color or brightness over time, and judged according to the determination result. Area determining means for generating object and background separation information; The separated binocular / multi-view image is divided into similar areas in segment units, and the object variation map and the background disparity map are separately generated by using the divided binocular / multi-view image and the generated object and background separation information. Variation estimation means for obtaining; And Disparity map synthesizing means for synthesizing the object disparity map and the background disparity map to generate a scene disparity map for the binocular / multi-view image A binocular / multi-view stereo matching device using object / background separation and image segmentation. The method of claim 1, Multi-view image acquisition means for acquiring binocular / multi-view images from an external imaging device; Camera information extracting means for calculating camera information and base matrix data through camera correction on the obtained binocular / multiview image; And Disparity map storage means for storing a scene disparity map generated by the disparity map synthesizing means A binocular / multi-view stereo matching device using object / background separation and image segmentation further comprising. The method of claim 2, The area determining means, If the multiview image acquired from the imaging apparatus is three or more views, the remaining one image adjacent to an arbitrary image among the acquired multiview images is defined as one image pair, and each image pair is divided into an object region and a background region, respectively. A binocular / multi-view stereo matching device using object / background separation and image segmentation. The method according to any one of claims 1 to 3, The area determining means, Distribution model estimating means for obtaining a background image from which an object is excluded in advance and estimating a distribution model for each pixel of the background image; Calculate the color generation probability for each pixel of the binocular / multi-view image using the estimated distribution model, and compare the calculated color generation probability with a preset threshold range according to the area of the binocular / multi-view image. Area classification means for classifying the data; Mixture distribution model estimation means for estimating a Gaussian mixture distribution model using color values of each pixel of the classified region; And An object for determining an object region or a background region using the estimated Gaussian mixture distribution model using an energy function defined as in Equation 6 below, and generating object and background separation information according to the determination result; Background Area Judgment Means Binocular / multi-view stereo matching device using object / background separation and image segmentation, including <Equation 6>

here,

Is an energy function,

Is a function representing the object and background area,

Is a Gaussian mix model for each estimated background,

Represents a Gaussian mixture model for each estimated object,

Is a function that has a value of 1 if the two arguments have different values, and 0 if they have the same value. The method of claim 4, wherein The variation estimating means, Image segmentation means for dividing the binocular / multi-view image into segment units which are similar region units; Plane coefficient extracting means for extracting plane coefficients for each segmented image segment; Energy function calculation means for defining an energy function in units of segments as shown in Equation (7) to remove the pixels hidden by the shielding area using the extracted plane coefficients; And Disparity map generation means for individually generating disparity maps for an object and a background region as energy parameters for minimizing the energy function using Equation (8) Binocular / multi-view stereo matching device using object / background separation and image segmentation, including <Equation 7>

here,

Is the data term,

Smoothing term,

Represents a hidden (shielded) term, <Equation 8>

here,

,

Is the plane coefficient of adjacent segments,

Denotes a common boundary length between two segments, and Equation 7 is different when the plane coefficients of adjacent segments are different (

), The common boundary length between two segments (

Impose an energy value proportional to delete The binocular / multi-view image is divided into a background area having a small change in color or brightness over time and an object area with a large change in color or brightness over time, and judged according to the determination result. An area determination step of generating object and background separation information about the object; The separated binocular / multi-view image is divided into similar areas in segment units, and the object variation map and the background disparity map are separately generated by using the divided binocular / multi-view image and the generated object and background separation information. Obtaining a variation estimation step; And Disparity map synthesis step of generating a scene disparity map for the binocular / multi-view image by synthesizing the object disparity map and the background disparity map A binocular / multi-view stereo matching method using object / background separation and image segmentation comprising a. The method of claim 7, wherein Camera information extraction step of calculating the camera information and the base matrix data through the camera correction before the region determination step after the binocular / multi-view image is obtained A binocular / multi-view stereo matching method using object / background separation and image segmentation further comprising. The method of claim 8, The area determining step, If a multiview image acquired from an external imaging apparatus is 3 or more views, the remaining one image adjacent to an arbitrary image among the obtained multiview images is defined as one image pair, and each image pair is an object region and a background region, respectively. A binocular / multi-view stereo matching method using object / background separation and image segmentation characterized in that it is divided. The method according to any one of claims 7 to 9, The area determining step, A distribution model estimating step of acquiring a background image including an object in advance and estimating a Gaussian distribution model for each pixel of the background image; Using the estimated Gaussian distribution model, a color generation probability for color or brightness is calculated for each pixel of the binocular / multi-view image, and a background region is compared with the calculated color generation probability with a preset threshold range. Or an area classification step of classifying the object area or the undetermined area; A mixture distribution model estimating step of estimating a Gaussian mixture distribution model using color values of each pixel of the classified region; And An object using the estimated Gaussian mixture distribution model to determine an object region or a background region using an energy function defined as in Equation (9), and generating object and background separation information according to the determination result; Background Area Determination Step Binocular / multi-view stereo matching method using object / background separation and image segmentation, including <Equation 9>

here,

Is an energy function,

Is a function representing the object and background area,

Is a Gaussian mix model for each estimated background,

Represents a Gaussian mixture model for each estimated object,

Is a function that has a value of 1 if the two arguments have different values, and 0 if they have the same value. The method of claim 10, The variation estimating step, An image division step of dividing the binocular / multi-view image into segment units which are similar area units; A plane coefficient extraction step of extracting plane coefficients for each segmented image segment; An energy function calculation step of defining an energy function in units of segments as shown in Equation 10 to remove the pixels hidden by the shielding area using the extracted plane coefficients; And Disparity map generation step of separately generating a disparity map for an object and a background area as an energy parameter that minimizes the energy function by using the energy function for each segment defined below. Binocular / multi-view stereo matching method using object / background separation and image segmentation, including <Equation 10>

here,

Is the data term,

Smoothing term,

Represents a hidden (shielded) term, <Equation 11>

here,

,

Is the plane coefficient of adjacent segments,

Denotes a common boundary length between two segments, and Equation 7 is different when the plane coefficients of adjacent segments are different (

), The common boundary length between two segments (

Impose an energy value proportional to delete

Images