KR100756034B1 - Apparatus and method for compensator of disparity vector - Google Patents

Abstract

Disclosed are a variation vector correcting apparatus and a variation vector correcting method. In the disparity vector correction apparatus for adaptively correcting a first disparity vector of a current pixel estimated from a reference image and a comparative image photographed at the same time, the reliability determiner determines whether the first disparity vector estimated for the current pixel is reliable. If there is no reliability of the first shift vector, it is determined whether there is a reliability of the second shift vector estimated for the next pixel located on the horizontal line of the current pixel, and the correction unit has a reliability of the second shift vector, If the next pixel is similar, the second disparity vector is replaced with the corrected disparity vector of the current pixel, and the multiplexer selects one of the first and second disparity vectors as the final disparity vector of the current pixel according to the determination result of the reliability determination unit. Output Therefore, the intermediate image of the improved image quality can be generated by adaptively correcting the variation vector of the pixel positioned at the edge of the image in consideration of the variation vector of the peripheral pixel.

Intermediate image, disparity vector, correction

Description

Disparity vector correction device and disparity vector correction method {Apparatus and method for compensator of disparity vector}

1 is a view for explaining the characteristics of the edge of the left image and the right image generated in a general stereo image;

2 is a block diagram schematically showing a disparity vector correcting apparatus according to a preferred embodiment of the present invention;

3 is a diagram illustrating two disparity vectors satisfying bidirectional matching;

FIG. 4 is a view for explaining an embodiment in which the corrector of FIG. 2 corrects a shift vector of a current pixel by using a current pixel and a next pixel positioned on a horizontal line of the current pixel;

FIG. 5 illustrates an embodiment in which the corrector of FIG. 2 corrects a disparity vector of a current pixel requiring correction by using a disparity vector of a neighboring pixel having a high reliability among a plurality of neighboring pixels located in the same search region. Drawings, and

FIG. 6 is a flowchart for schematically describing a method for correcting a shift vector according to FIG. 2.

Explanation of symbols on the main parts of the drawings

200: disparity vector correction device 210: reliability determination unit

212: motion error reliability determination unit 214: matching coefficient calculation unit

216: Variation vector reliability determination unit 220: Correction unit

222: similarity calculating unit 224: similarity determining unit

226: disparity vector correction unit 230: multiplexer

The present invention relates to a disparity vector correcting apparatus and a disparity vector correcting method, and more particularly, to a disparity vector correcting apparatus and a disparity vector for adaptively correcting a disparity vector of a pixel located at an edge of an image in consideration of the disparity vector of a peripheral pixel. A vector correction method.

A stereo image composed of two left and right images is obtained through two cameras, and the two obtained images have a lot of spatial redundancy. Therefore, a large amount of data is transmitted when data is transmitted, so a large cost loss occurs, and studies to solve this problem are being actively conducted.

An example is intermediate vector interpolation. The intermediate view image synthesis generates an image for a viewpoint that does not exist by using left and right images, and uses disparity estimation. Disparity estimation finds the similarity between two left and right images and expresses the disparity between the two images as a vector.

1 is a view for explaining the characteristics of the edge of the left image and the right image in a general stereo image.

Referring to FIG. 1, since the right image captured by the first camera includes a sufficient left edge region (①) than the left image captured by the second camera, the shift of the left edge from the left image to the right image is estimated. While is correct, the shift estimation for the left edge from the right image to the left image is performed incorrectly.

On the other hand, since the left image includes a sufficient right edge region (②) compared to the right image, the shift estimation for the right edge from the right image to the left image is accurate, while the estimation of the shift for the right edge from the left image to the right image is accurate. Improperly performed

Therefore, inaccurate shift estimation is easily performed on the left and right edges due to the characteristics of the stereo image, and some of the mid-view images have distortion, thereby degrading the image quality.

SUMMARY OF THE INVENTION The present invention provides a disparity vector correcting apparatus capable of preventing distortion of an image by adaptively correcting a disparity vector incorrectly estimated for left and right edges in consideration of similarity or correlation with surrounding pixels. The present invention provides a method for correcting a disparity vector.

In the disparity vector correcting apparatus for adaptively correcting a first disparity vector of a current pixel estimated from a reference image and a comparative image photographed at the same time according to the present invention for achieving the above object, the estimated with respect to the current pixel A reliability determination unit that determines whether or not there is reliability of the first shift vector, and determines whether or not the reliability of the second shift vector estimated with respect to the next pixel located on a horizontal line of the current pixel when the reliability of the first shift vector is not present; A compensator for replacing the second shift vector with the corrected shift vector of the current pixel when there is reliability of the second shift vector and the current pixel and the next pixel are similar to each other; And a multiplexer for selectively outputting one of the first and second disparity vectors as the final disparity vector of the current pixel according to the determination result of the reliability determination unit.

In more detail, the reliability determining unit compares the first motion error value and the first threshold value of the current pixel calculated for a certain region of the comparison image to determine whether the motion error value is reliable or not. Determination unit; A matching coefficient calculator for comparing the estimated first variation vector with a second threshold and calculating a first bidirectional matching coefficient with respect to the first variation vector of the current pixel; And a variation vector reliability determination unit configured to determine whether or not the reliability of the first variation vector is based on the reliability of the determined first motion error value and the calculated first two-way matching coefficient. If there is no reliability, the motion error reliability determining unit and the matching coefficient calculating unit calculate a reliability and a second bidirectional matching coefficient of the second motion error value for the next pixel, and the disparity vector reliability determining unit calculates the calculated second movement. The reliability of the second variation vector is determined based on the reliability of the error value and the second bidirectional matching coefficient.

The correction unit may include: a similarity calculator configured to calculate a difference value between the current pixel and the next pixel when it is determined that the reliability of the second shift vector is present by the reliability determination unit; A similarity determination unit comparing the calculated difference value with a third threshold value and determining whether the current pixel is similar to the next pixel; And a shift vector corrector configured to correct the second shift vector as a shift vector of the current pixel when it is determined that the current pixel and the next pixel are similar.

Preferably, when it is determined that the current pixel and the next pixel are not similar, the disparity vector corrector has a reliable disparity vector among neighboring pixels of the current pixel, but displaces the neighboring pixel closest to the current pixel. A vector is corrected as the disparity vector of the current pixel.

Meanwhile, the method for correcting a disparity vector according to the present invention for achieving the above object is a method of correcting a disparity vector for adaptively correcting a first disparity vector of a current pixel estimated from a reference image and a comparison image photographed at the same time. (a) determining a reliability of the first shift vector estimated for the current pixel; and if there is no reliability of the first shift vector, a second shift vector estimated for a next pixel located on a horizontal line of the current pixel Determining the reliability of the; (b) replacing the second shift vector with the corrected shift vector of the current pixel if the second shift vector is reliable and the current pixel and the next pixel are similar; And (c) selectively outputting one of the first and second disparity vectors as a final disparity vector of the current pixel according to the determination result of step (a).

Hereinafter, with reference to the drawings will be described the present invention in more detail.

2 is a block diagram schematically showing a disparity vector correcting apparatus according to a preferred embodiment of the present invention.

Referring to FIG. 2, the disparity vector correcting apparatus 200 according to an exemplary embodiment of the present invention is a reliability determining unit 210, a correcting unit 220, and a multiplexer (MUX). 230).

First, the disparity vector correcting apparatus 200 according to the present invention is a part of an image processing apparatus for generating an intermediate image using a plurality of images simultaneously photographed from at least two cameras. It is an adaptive correction device.

In particular, the disparity vector correcting apparatus 200 uses the disparity vectors estimated for each pixel constituting the right image and the motion vector calculated for each pixel to convert the disparity vector, in particular, the disparity vector located at the edge of the image. Useful for calibration.

Hereinafter, images taken simultaneously from two cameras (not shown) are respectively referred to as a left image and a right image applied as a reference image, and an intermediate image is estimated from a left image and a right image applied as a reference image. It is generated using the disparity vector.

In the following description, a pixel to be corrected is referred to as a current pixel R1, and a pixel located on the same horizontal line as the current pixel R1 but located in the same search area W1 is referred to as a next pixel R2.

The reliability judging unit 210 determines whether or not the reliability of the first variation vector pre-estimated with respect to the current pixel currently being processed among the plurality of pixels constituting the right image is determined, and when it is determined that there is no reliability of the first variation vector, The reliability of the estimated second shift vector is determined for the next pixel located on the horizontal line of the current pixel. Here, the reliability determining unit 210 sets a window having a predetermined size for the current pixel, and determines whether there is reliability for the next pixel located in the window.

To this end, the reliability determination unit 210 includes a motion error reliability determination unit 212, a matching coefficient calculation unit 214, and a variation vector reliability determination unit 216.

The motion error reliability judging unit 212 determines the reliability of the first motion error value (Cost Function: CF) of the current pixel using Equation (1).

Referring to Equation 1, CF (x, y) is the first motion error value of the current pixel, CCF (x, y) is the reliability of the first motion error value, TH1 is the first threshold value, and ( x, y) means coordinate data where the current pixel is located. The first motion error value of the current pixel is compared with a predetermined search area of the current pixel and the left image in the right image, and various notifications such as sum of absolute difference (MAD), mean absolute difference (MAD), and mean square error (MSE) It can be calculated by the method, and the present invention applies SAD.

The motion error reliability determiner 212 compares the first motion error value with the set first threshold value TH1 and determines that the first motion error value is less than the first threshold value TH1. On the other hand, if the first motion error value is larger than the first threshold value TH1, it is determined that there is no reliability.

The motion error reliability determination unit 212 determines the motion error reliability of the input pixels in the order of input.

The matching coefficient calculator 214 calculates a first two-way matching coefficient with respect to the first disparity vector by using Equation 2.

Referring to [Equation 2], CBDM (x, y) is the first two-way matching coefficient, V _RL is the disparity vector of the right image for the left image, V _LR is the disparity vector of the left image for the right image, and TH2 means the second threshold. Here, the first disparity vector estimated with respect to the current pixel is composed of V _RL and V _LR .

The matching coefficient calculating unit 214 compares the first estimated shift vector and the set second threshold value TH2, and " | V _RL (x, y) + V _LR (V _RL (x, y), y If || "is smaller than the second threshold value TH2, it is determined that bidirectional matching is performed, and the matching coefficient '1' is calculated. On the other hand, if it is larger than the second threshold value TH2, the matching coefficient calculator 214 determines that the bidirectional matching is not satisfied and calculates a matching coefficient '0'.

The matching coefficient calculator 214 calculates the bidirectional matching coefficients of the disparity vectors estimated for the input pixels in the order of input.

[Equation 2] is the uniqueness of the stereo image (Uniqueness: the characteristic that one point of the stereo image has less than one variation) and the consistency (matching in stereo image) Point represents a characteristic corresponding to a point of the real thing).

3 is a diagram illustrating two disparity vectors satisfying bidirectional matching of a predetermined pixel.

Referring to FIG. 3, I _L is a left image, I _R is a right image, R1 is a current pixel located in the right image, L1 is a comparison pixel located in the left image corresponding to R1, W1 and W2 are search areas, V _RL is a disparity vector, and the right images (I _R) for the left picture (I _L), V _LR denotes a disparity vector of the left picture (I _L) for the right image (I _R).

As shown in FIG. 3, when L1 corresponding to V _RL from R1 and R1 corresponding to V _LR from L1 coincide, the two disparity vectors V _RL and V _LR match bidirectional matching between pixels R1 and R2. Satisfied. This bidirectional match is optimally satisfied when V _RL and V _LR are equal.

Referring back to FIG. 2, the variation vector reliability determination unit 216 determines the reliability of the first variation vector (CDV) using Equation (3).

Referring to [Equation 3], CDV (x, y) is the reliability of the first shift vector (V1), the shift vector reliability determination unit 216 is the first motion error value determined through the equation (1) The reliability (CDV) of the first variation vector is determined based on the confidence degree (CFF) of (CF) and the first bidirectional matching coefficient calculated through Equation (2).

That is, the variation vector reliability determination unit 216 determines that the reliability of the first motion error value (CDV = 1) is both reliable (CDV = 1) when both the reliability (CDV) and the first two-way matching coefficient (CBDM) of the first motion error value are '1'. If either the reliability of the moving error value (CDV) or the first two-way matching coefficient (CBDM) is '0', it is determined that there is no reliability (CDV = 0).

And, if the CDV = 1, disparity vector reliability determination unit 216 is a first disparity vector (V1) for the current pekseol (R1) satisfies the bidirectional matched, because the higher the similarity between two pixels, left picture (I _L) It is determined that there is a pixel matched with the current pixel R1 at, and the intermediate image is generated by using the first estimated shift vector V1 as it is without correction. That is, the variation vector reliability determination unit 216 causes the mux 230 to selectively output the first variation vector V1 as the final variation vector for the current pixel R1.

Wherein one of the two pixels right images (I _R) (i.e., reference image) and the current pixel which is located in, and the other is left picture (I _L) (i.e., the comparison image), the current pixel of the right image (I _R) from The pixel matched to is a pixel having a minimum motion error value SAD. For example, two pixels are R1 and L1 in FIG.

On the other hand, if CDV = 0, the variation vector reliability determination unit 216 determines that no pixel matched with the current pixel R1 exists in the left image I _L , and the same search area and the same horizontal line as the current pixel R1. The reliability of the estimated second shift vector is determined for the next pixel R2 positioned on the image.

Since the motion error reliability determining unit 212 and the matching coefficient calculating unit 214 continuously calculate the motion error reliability and the bidirectional matching coefficient of sequentially input pixels, even if CDV = 0, the variation vector reliability determining unit 216 may determine the time. The reliability of the second shift vector V2 with respect to the next pixel R2 may be determined without delay.

In detail, the variation vector reliability determination unit 216 is the second equation provided by Equation 3 and the reliability of the second motion error value provided from the motion error reliability determination unit 212 and the matching coefficient calculation unit 214. The reliability (CDV) of the second disparity vector is determined using the bidirectional matching coefficient. As a result of the determination, if the reliability of the second variation vector exists, that is, if the CDV of the second variation vector is determined to be '1', the variation vector reliability determination unit 216 sets CDV = 1 to similarity of the correction unit 220. The calculation unit 222 is provided.

The reliability determination unit 210 repeatedly performs the above-described operation until a pixel having CDV = 1 is obtained in an arbitrary search area W1.

4 is a diagram for describing an exemplary embodiment in which the corrector of FIG. 2 corrects a shift vector of a current pixel by using a current pixel and a next pixel considered by the shift vector reliability determination unit.

Referring to FIG. 4, R1 has no reliability of the estimated first shift vector, the current pixel at the position (x, y), and R2 is the search region W1 of the current pixel R1 (represented by a dotted line). The next pixel located on the same horizontal line as the current pixel R1, V1 is the first shift vector, V2 is the second shift vector, C1 is the pixel of the virtual image that matches the current pixel by V1, and C2 is the V2. The pixel that matches the next pixel, d is the horizontal movement distance between the current pixel R1 and the next pixel R2, W2 is the search area at position (x + a), and the virtual image is the left image I _L. The video is not displayed in the left edge of the screen. Here, when the reference image is the left image, the virtual image is located in the right edge region of the right image.

2 to 4, the compensator 220 has a similarity between the current pixel R1 and the next pixel R2 since the reliability of the second shift vector V2 estimated based on the next pixel R2 exists. If it is determined that the result is similar to each other, the second shift vector V2 estimated for the next pixel R2 is replaced with the corrected shift vector of the current pixel R1.

To this end, the corrector 220 includes a similarity calculator 222, a similarity determiner 224, and a disparity vector corrector 226.

The similarity calculator 222 calculates a difference value between the current pixel R1 and the next pixel R2 to correct the disparity vector by using Equation 4.

Here, i and j denote the size of the search area W1, and bx and by denote the maximum size of the search area W1.

Referring to Equation 4, the similarity calculating unit 222 sets the search areas W1 and W2 around the current pixel R1 and the next pixel R2, respectively, and calculates a CV within the search areas W1 and W2. do. This means that when comparing only a single pixel, that is, comparing only the current pixel R1 located at (x, y) and the next pixel R2 located at (x + a, y), the accuracy associated with similarity can be reduced. Because there is.

The similarity determiner 224 determines whether the current pixel R1 and the next pixel R2 are similar by using Equation 5 below.

Here, CR (x, y) is the similarity between the current pixel R1 and the next pixel R2, CV (x, y) is the difference calculated by the similarity calculator 222, and TH3 is the third threshold. Referring to [Equation 5], if the difference value CV calculated by the similarity calculator 222 is smaller than the third threshold value TH3, the similarity determiner 224 may determine the current pixel R1 and the next pixel ( R2) judges to be similar (CR = 1). That is, the similarity determiner 224 determines that the current pixel R1 and the next pixel R2 are located in the same area or the same object.

On the other hand, if the calculated difference value CV is greater than or equal to the third threshold value TH3, the similarity determiner 224 determines that the current pixel R1 and the next pixel R2 are not similar (CR = 0). The current pixel R1 and the next pixel R2 are determined to be located in different areas or objects.

When it is determined that CR = 1 by the similarity determiner 224, the variation vector corrector 226 corrects the second variation vector V2 as the variation vector of the current pixel R1 as shown in [Equation 6].

Referring to Equation 6, disparity (x, y) is the disparity vector Vc of the current pixel R1 to be corrected, and disparity (x + a, y) is the second disparity vector of the next pixel R2 ( As V2), the first shift vector V1 of the current pixel R1 is corrected by the second shift vector V2. That is, the shift vector corrector 226 generates the second shift vector V2 as the corrected shift vector Vc of the current pixel R1.

On the other hand, if the similarity determiner 224 determines that CR = 0, the current pixel R1 is located in the search area W3 as shown in FIG. 5, and is corrected by the shift vector of the next pixel (not shown). It can't be done.

FIG. 5 illustrates an embodiment in which the corrector of FIG. 2 corrects a disparity vector of a current pixel requiring correction by using a disparity vector of a neighboring pixel having a high reliability among a plurality of neighboring pixels located in the same search region. Drawing.

2 and 5, when the similarity determiner 224 determines that CR = 0, the reliability determiner 210 determines the reliability of the disparity vectors estimated for the neighboring pixels of the current pixel R3. To judge. Here, the peripheral pixels are pixels located in the search area W3 in which the current pixel R3 is located.

 The similarity calculator 222 calculates a difference value between each of the at least one peripheral pixel determined to be reliable by the reliability determiner 210 and the current pixel R3 to be corrected. The similarity determiner 224 determines the similarity by comparing the minimum difference value and the third threshold value TH3 among the difference values calculated for each peripheral pixel with reliability. As a result of determination, when the minimum difference value is less than or equal to the third threshold value TH3, the similarity determiner 224 outputs CR = 1.

If the CDV and CR of the peripheral pixels (x + m, y + n) corresponding to the minimum difference value are both '1', the variation vector corrector 226 corrects the current pixel (R3) by using Equation (7). Correct the disparity vector of.

Here, (x, y) is the coordinate data of the current pixel (R3) to be corrected, (x + m, y + n) is the coordinate data of the peripheral pixel (R4) to be used for correction of the current pixel (R3), disparity ( x and y are the disparity vector to be corrected of the current pixel R3, and disparity (x + m and y + n) are the disparity vector V4 of the peripheral pixel R4 to be used to correct the current pixel R3.

Referring to FIG. 5 and [Equation 7], the variation vector corrector 226 is configured to determine the peripheral pixel R4 and the current pixel R3 corresponding to the CDV and the minimum difference value of the variation vector estimated with respect to the peripheral pixels. When the similarity CRs are all '1', the disparity vector V4 'disparity (x + m, y + n)' of the peripheral pixel R4 corresponding to the minimum difference value is the disparity vector of the current pixel (x, y). Correct it if possible. In other words, the disparity vector corrector 226 shifts the neighboring pixel ((x + m, y + n)) at the position closest to the current pixel (x, y) as shown in Equation (7). Replace as a vector Since the disparity vectors of the pixels in the same area (object) are the same, the disparity vector of the neighboring pixels among them can be corrected.

On the other hand, if both of CDV = 1 or the difference CV between each of the peripheral pixels and the current pixel (x, y) are larger than the third threshold value TH3, that is, CR = 0, the variation vector corrector 226 corrects. The disparity vector of the pixels positioned at the edge of the image frame is corrected by correcting the disparity vector of the current pixel (x, y) to be the disparity vector of the next pixel.

On the other hand, when the reliability determination unit 210 described above considers the next pixel R2 of the current pixel R1, if the next pixel R2 is out of the search area W1, the reliability determination unit 210 may perform the determination of FIG. 5. The current pixel R3 is assumed to be located in the search area W3, and the reliability value and the difference between the current pixel R1 and the current pixel R1 are calculated in consideration of the surrounding pixels of the current pixel R3. Correct the disparity vector of.

The mux 230 selectively outputs one of the first and second disparity vectors V1 and V2 as the final disparity vector of the current pixel R1 or R3 according to the determination result of the reliability determination unit 210. In detail, when it is determined that the reliability CDV of the first shift vector is '1', the MUX 230 selectively outputs the first shift vector V1 as the final shift vector of the current pixel R1. . In addition, when the reliability (CDV) of the first shift vector is determined to be '0', the mux 230 determines a shift vector (for example, the highest similarity among the surrounding pixels of the current pixel R1 or R3). The second shift vector V2 or V4 is selectively outputted as the corrected shift vector of the current pixel R1 or R3.

FIG. 6 is a flowchart for schematically describing a method for correcting a shift vector according to FIG. 2.

First, hereinafter, the pixel to be corrected is referred to as a pixel located on the same horizontal line as the current pixel R1 and the next pixel R2 but located in the same search area W1.

2 to 6, the motion error reliability determination unit 212 uses Equation 1, the first motion using the first motion error value of the current pixel R1 and the set first threshold value TH1. The reliability of the error value is determined, and the matching coefficient calculator 214 calculates the first two-way matching coefficient by using Equation 2, the first shift vector V1, and the second threshold value TH2 (S605). ). The motion error reliability determiner 212 and the match coefficient calculator 214 sequentially calculate the reliability of the motion error value calculated for the input pixels and the bidirectional match coefficients of the estimated disparity vectors.

Subsequently, the variation vector reliability determination unit 216 determines the reliability (CDV) of the first variation vector using Equation (3), the reliability of the first motion error value (CCF), and the first bidirectional matching coefficient (CBDM). (S610).

If it is determined in step S610 that the reliability of the first shift vector (CDV) = 0, that is, no reliability exists, that is, the shift vector reliability determination unit 216 moves to the next pixel R2 (S615). It is checked whether the next pixel R2 is located in the search area W1 (S620).

If the next pixel R2 is located in the search area W1 in operation S620, the variation vector reliability determining unit 216 may determine the reliability of the second bidirectional matching coefficient and the second motion error value for the next pixel R2 (CDV). Based on the determination of the reliability (CDV) of the second shift vector (V2) (S625, S630).

If the reliability of the second disparity vector does not exist in operation S630, the variation vector reliability determination unit 216 repeats operations S615 to S630 until a pixel having CDV = 1 is obtained in the search region W1. .

On the other hand, if the reliability of the second disparity vector (V2) in step S630, the similarity calculator 222 is the current pixel (R1) and the next pixel (R2) to correct the disparity vector using [Equation 4] The difference value CV of is calculated (S635).

When the operation S635 is performed, the similarity determination unit 224 determines whether the current pixel R1 and the next pixel R2 are similar by comparing the difference CV calculated in operation S635 with the third threshold value TH3. (S640). That is, if the difference CV calculated in step S635 is less than or equal to the third threshold value TH3, it is determined that the current pixel R1 and the next pixel R2 are similar and located in the same area (object).

Accordingly, the disparity vector corrector 226 corrects the disparity vector of the current pixel R1 as the second disparity vector (S645), and the mux 230 converts the second disparity vector to the final disparity vector of the current pixel R1. Selective output as

When step S645 is performed, the similarity calculator 222 checks whether the current pixel R1 corrected in step S645 is located at the left or right edge of the input image (S650). If it is determined in step S650 that the current pixel R1 is located at the edge, it is determined that the correction of the disparity vector for the pixel located at the edge is completed.

In operation S640, when the difference CV calculated in operation 635 is greater than the third threshold value TH3, the current pixel R1 and the next pixel R2 are not similar to each other, and are located in different regions (objects). I judge it. Accordingly, the similarity calculator 222 calculates a difference value between each of the peripheral pixels of the current pixel R1 and the current pixel R1 to be corrected (S655). The peripheral pixels used in step S655 are located in the search area W3 in which the current pixel R3 is located.

Thereafter, the similarity calculator 222 compares the minimum difference value CV and the third threshold value TH3 among the plurality of difference values calculated in operation S655 (S660). As a result of the comparison, if the minimum difference value is less than or equal to the third threshold value TH3, the variation vector corrector 226 determines the variation vector V4 'disparity (x + m, y) of the peripheral pixel R4 corresponding to the minimum difference value. + n) 'is corrected to become the disparity vector of the current pixel R3. That is, the shift vector corrector 226 replaces the peripheral pixels x + m and y + n at positions closest to the current pixels x and y as the corrected shift vectors of the current pixel.

On the other hand, if the minimum difference value is greater than the third threshold value TH3 in step S660, the shift vector corrector 226 corrects the shift vector of the current pixel (x, y) to be corrected to be the shift vector of the previous pixel. .

On the other hand, if it is determined in step S650 that the current pixel R1 is not located at the edge, the similarity calculator 222 moves to the previous pixel of the current pixel R1 (S670). Here, the previous pixel is a pixel located on the same horizontal line as the current pixel R1 and in the same search area W1.

The correction unit 220 recognizes the previous pixel moved in step S670 as a new current pixel to be corrected at present, recognizes the current pixel R1 corrected in step S645 as the next pixel, and repeats steps S635 to S670. do.

In addition, if it is determined in step S610 that the reliability of the first shift vector V1 exists, the MUX 230 selectively outputs the final shift vector V1 without correcting the first shift vector V1 (S675).

According to the above steps S605 to S675, the disparity vector correction device and the disparity vector correction method according to the present invention selectively correct or preserve the disparity vector estimated for the current pixel to be corrected. In particular, the image quality of the final output intermediate image is improved by adaptively correcting the disparity vector with respect to the current pixel located near the edge.

In the above, a method of generating an intermediate image in a stereo-view display device displaying images captured by two cameras has been described as an example, but images captured by a plurality of cameras continuously positioned at predetermined intervals are displayed. It can also be applied to a multi-view display device.

Meanwhile, the variation vector of each pixel estimated for the purpose of the present invention is to find a feature based disparity estimation scheme that detects and matches feature points of an image, and find the shortest path of a cost function. It is estimated by various techniques such as Dynamic Programming Disparity Estimation Scheme to match, Area Based Disparity Estimation Scheme to measure and match the correlation of the matching window.

In addition, the present invention described above was applied by applying the reference image to the right image, the comparison image as the left image, but the reference image is applied to the left image, the comparison image as the right image, and then corrected the disparity vector as described above. Of course you can.

As described so far, according to the disparity vector correction method according to the present invention, it is possible to correct the disparity vector of the edge pixels incorrectly estimated using the disparity vector of the peripheral pixels. This is because pixels located in the same region of the stereo image have the same or similar disparity vectors. Therefore, a disparity map that is not distorted and is flat may be obtained by correcting the disparity vector that is incorrectly estimated with a disparity vector that is more similar to the actual disparity vector.

In particular, the present invention can improve the quality of the mid-view image by correcting the disparity vector more similarly to the actual disparity vector in consideration of various parameters such as similarity between neighboring pixels, reliability of motion error, and reliability of disparity vector.

Also, since the motion error value and the disparity vector are generated for each pixel during every disparity estimation process, it can be easily applied to the correction of all disparity vectors.

Although the present invention has been described in detail through the representative embodiments, those skilled in the art to which the present invention pertains can make various modifications without departing from the scope of the present invention with respect to the embodiments described above. I will understand. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined by the claims below and equivalents thereof.

Claims (5)

In the disparity vector correction device for adaptively correcting the first disparity vector of the current pixel estimated from the reference image and the comparison image photographed at the same time, The reliability of the first shift vector estimated for the current pixel is determined. If the reliability of the first shift vector is not present, the next pixel is located on the horizontal line of the current pixel and is located in the same search area as the current pixel. A reliability determination unit that determines whether or not the reliability of the second variation vector estimated with respect to A compensator for replacing the second shift vector with the corrected shift vector of the current pixel when there is reliability of the second shift vector and the current pixel and the next pixel are similar to each other; And And a multiplexer for selectively outputting one of the first and second disparity vectors as a final disparity vector of the current pixel according to a determination result of the reliability judging unit. The method of claim 1, The reliability determination unit, A motion error reliability judging unit comparing the first motion error value and the first threshold value of the current pixel calculated for a certain region of the comparison image to determine whether the motion error value is reliable; A matching coefficient calculator for comparing the estimated first variation vector with a second threshold and calculating a first bidirectional matching coefficient with respect to the first variation vector of the current pixel; And And a variation vector reliability determination unit configured to determine whether the first variation vector is reliable based on the reliability of the determined first motion error value and the calculated first bidirectional matching coefficient. If there is no reliability of the first shift vector, the motion error reliability determination unit and the matching coefficient calculation unit calculate the reliability of the second motion error value and the second bidirectional matching coefficient for the next pixel, and the shift vector reliability determination unit And determining the reliability of the second disparity vector based on the calculated reliability of the second motion error value and the second bidirectional matching coefficient. The method of claim 1, The correction unit, A similarity calculator that calculates a difference between the current pixel and the next pixel when it is determined that the reliability of the second shift vector is present by the reliability determination unit; A similarity determination unit comparing the calculated difference value with a third threshold value and determining whether the current pixel is similar to the next pixel; And And a disparity vector corrector configured to correct the second disparity vector as a disparity vector of the current pixel when it is determined that the current pixel and the next pixel are similar to each other. The method of claim 3, wherein The disparity vector corrector If the current pixel and the next pixel are determined to be dissimilar, the disparity vector of the neighboring pixel closest to the current pixel is corrected as a disparity vector of the current pixel having a reliable disparity vector among the neighboring pixels of the current pixel. Disparity vector correction device, characterized in that. In the disparity vector correction method for adaptively correcting the first disparity vector of the current pixel estimated from the reference image and the comparison image photographed at the same time, (a) determining the reliability of the first shift vector estimated with respect to the current pixel, and if the reliability of the first shift vector is not present, is located on the horizontal line of the current pixel and is located in the same search region as the current pixel. Determining the reliability of the second disparity vector estimated for the next pixel; (b) replacing the second shift vector with the corrected shift vector of the current pixel if the second shift vector is reliable and the current pixel and the next pixel are similar; And and (c) selectively outputting one of the first and second disparity vectors as a final disparity vector of the current pixel according to the determination result of step (a).

Images