KR20120069546A - Stereo image proecsiing method, stereo image processing apparatus and display device - Google Patents

Abstract

The stereo image processing apparatus of the present invention includes an image input unit into which a stereo pair image is input, a representative point setting unit that sets a plurality of representative points in a first image of the stereo pair image, and a second image of the stereo pair image. A corresponding point controller for detecting a plurality of corresponding points corresponding to the plurality of representative points, excluding a corresponding point determined as a deviation value, a deviation value calculating unit for determining whether the corresponding point is the deviation value 와, and An approximation curve calculation section for performing curve approximation by remaining correspondence points except for the excluded correspondence points, a height correction value calculation section for calculating a height correction value from a plurality of approximation curves created by the curve approximation, and based on the height correction values. A parallelization execution unit that performs parallelization of the second image, and the first image and the parallelization 2 includes an image output unit for outputting an image.

Description

STEREO IMAGE PROECSIING METHOD, STEREO IMAGE PROCESSING APPARATUS AND DISPLAY DEVICE}

The present invention relates to a stereo image processing method, a stereo image processing apparatus and a display apparatus.

Recently, devices using stereo images, such as 3D TVs, have been widely developed. When performing 3D display from two stereo images, ie, a stereo pair image, the method of extracting depth information from a stereo pair image and using the extracted depth information is performed.

In order to extract the depth information, as shown in Fig. 1 (a), the corresponding point (stereo matching) is searched for between the one image L and the second image R. Depth information is extracted for the first image L by the stereo matching to express the depth. As shown in Fig. 1B, the information is stored as the first image L and the depth information D, and 3D display is performed using the information.

In the case of stereo matching, the corresponding point can be searched effectively when the height (vertical position) of the left and right image objects is aligned. However, when shooting with a stereo camera, there are many cases where the horizontality of the left and right cameras does not completely match. For example, as shown in FIG. 2, when the right camera is inclined compared to the left camera, and the horizontal of the left camera and the horizontal of the right camera do not coincide, the heights of the left and right image objects are not aligned. The correspondence point cannot be retrieved.

In addition, although the photography by a stereo camera is image | photographed by the parallel method or the intersection method, when shooting by the cross method, as shown in FIG. 3, trapezoidal distortion produces | generates according to the characteristic of the imaging method itself. ((Keystone) effect). In this case, the height of the object does not match.

When there is a difference in height in the inputted left-right cameras with respect to the inputted stereo pair image, if the stereo matching is performed as it is and depth information is created for the two-dimensional image, the corresponding point cannot be retrieved correctly.

Therefore, when the correspondence point search is performed on the image photographed by the stereo camera, it is necessary to match the heights of the objects in the left and right images by some preprocessing.

Conventionally, various methods have been used to match the heights of objects in left and right images. For example, what has been typically used is parallelization by epipolar geometry. Especially, the method of calibrating before a imaging | photography beforehand using a well-known pattern is common. However, since it is necessary for the photographer to parallelize a stereo image with a known pattern, it cannot be applied to the stereo image already photographed, for example, when performing 3D display on a display device.

As another method by parallelization by isopolar geometry, there is a method of performing self-calibration (see Japanese Patent Laid-Open No. 2007-200364). In this method, the parallelization is not performed using a known pattern in advance, but the parallelization is performed using a characteristic pattern in an image having a straight line such as a line of a building or a road. However, this method has a problem of requiring a straight edge in the image, and also has a problem in that the size of the pattern increases the error of the photographing matrix and cannot be parallelized accurately.

As another method, on the premise that the scanning line direction of the image coincides with the isopolar line, the row corresponding to the row of the first image is searched from the second image based on the correlation between the left and right images, and the height is increased. There is a matching method (see Japanese Patent Laid-Open No. 2010-128575).

However, in this method, since the search direction is always limited to the scanning line direction, that is, the horizontal direction, and the scanning line direction coincides with the isopolar line, there is a problem that the applied image is limited.

An object of the present invention is to provide a stereo image processing method capable of parallelizing images after photographing for various types of stereo images.

A stereo image processing apparatus according to an embodiment of the present invention includes an image input unit to which a stereo pair image is input, a representative point setting unit that sets a plurality of representative points in the first image of the stereo pair image, and the stereo In the second image of the pair image, a corresponding point control unit for detecting a plurality of corresponding points corresponding to the plurality of representative points and excluding corresponding points determined as deviation values, and a deviation for determining whether the corresponding points are the deviation values or not. A value calculation section, an approximation curve calculation section that performs a curve approximation by remaining correspondence points except for the excluded correspondence points, a height correction value calculation section calculating a height correction value from a plurality of approximation curves created by the curve approximation; A parallelization execution unit that performs parallelization of the second image based on the height correction value, and the first image and parallelization Performing is characterized by having an image output unit for outputting the second image.

Further, in the stereo image processing method according to an embodiment of the present invention, a stereo pair image is input, a plurality of representative points are set in the first image of the stereo pair image, and a second image of the stereo pair image is set. And detecting a plurality of corresponding points corresponding to the plurality of representative points, determining whether the corresponding point is a deviation value, except for the corresponding point determined as the deviation value, and determining the corresponding point after the deviation value determination. Curve approximation, calculating a height correction value from a plurality of approximation curves created by the curve approximation, parallelizing the second image based on the height correction value, and performing the first image and parallelization. The second image is output.

In addition, the display device according to an exemplary embodiment of the present invention includes a video signal input unit to which a video signal including a display panel, a stereo pair image including a first image and a second image is input, and a second of the stereo pair image. A stereo image processing apparatus for performing height correction of an image to generate a corrected stereo pair image, a stereo matching unit for performing stereo matching of the corrected stereo pair image, and generating depth information of the first image, the first image and A parallax image synthesizing unit for creating a parallax image using the depth information, and a timing controller for generating a clock signal for distributing the parallax image to pixels on the display panel, wherein the stereo image processing apparatus includes: the image; An image input unit into which a stereo pair image is input from a signal input unit, and the stereo pair image A representative point setting unit for setting a plurality of representative points in the first image of the plurality of points; and a plurality of corresponding points corresponding to the plurality of representative points in the second image of the stereo pair image, and opposite to the deviation value. A correspondence point control unit excluding the corresponding correspondence point, a deviation value calculator for determining whether the correspondence point is the deviation value 와, an approximation curve calculation unit for performing curve approximation by remaining correspondence points except the excluded correspondence point, and the curve approximation A height correction value calculating unit that calculates a height correction value from the plurality of approximation curves created by the above, a parallelization execution unit that performs parallelization of the second image based on the height correction value, and the first image and parallelization And an image output unit configured to output a corrected stereo pair image composed of the second image that has been subjected to the stereo matching unit. .

According to the present invention, there is provided an apparatus and method capable of performing a parallelization process of a stereo image capable of coping with various kinds of images after shooting.

That is, according to the present invention, in the processing of a stereo image, even in an image having no characteristic pattern such as a straight edge, parallelization processing can be performed, and mismatching in stereo matching can be reduced.

In addition, according to the present invention, even in an image where the scanning line and the isoline line do not coincide, parallelization processing can be performed to reduce mismatching in stereo matching.

1 is a schematic diagram of a method of extracting depth information from a stereo pair image by stereo matching.
2 is a diagram showing an example in which the horizontality of the left and right cameras does not coincide in a stereo pair image.
3 is a diagram showing the shape of trapezoidal distortion in the case of using the intersection method.
4 is a block diagram showing an important configuration of a stereo image processing apparatus according to an embodiment of the present invention.
5 is a diagram for explaining a block matching method.
6 is a diagram illustrating an example of applying a block matching method to a correspondence point search.
7 is a diagram illustrating an example of determination of a deviation value.
8 is a diagram illustrating an example in which curve approximation is performed for a corresponding point.
FIG. 9 is a diagram showing an example of creating an offset map representing a correction value in the height direction from an approximation curve in the stereo image processing apparatus according to the embodiment of the present invention.
10 is a flowchart of a stereo image processing method according to an embodiment of the present invention.
11 is a conceptual diagram illustrating a processing procedure in the stereo image processing method according to an embodiment of the present invention.
12 illustrates a main configuration of a display device according to an exemplary embodiment of the present invention.

Hereinafter, a stereo image processing apparatus according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

4 is a block diagram showing the configuration of a stereo image processing apparatus 100 according to an embodiment of the present invention.

Referring to FIG. 4, the stereo image processing apparatus 100 includes an image input unit 110, a representative point setting unit 120, a corresponding point control unit 130, a calculation unit 140, a height correction execution unit 150, and an image. The output unit 160 is included. Each of the configurations is implemented by logic circuitry such as FPGA or ASIC. Or, it is implemented by a processor such as a CPU or a GPU, a ROM, a RAM, or the like, or a system L in which these are combined.

Stereo pair image data is input to the image input unit 110. The stereo pair image consists of two images, and consists of a first image and a second image respectively picked up by two imaging devices. For example, among the left image and the right image photographed by the stereo camera, the left image is referred to as the first image, and the right image is referred to as the second image. The first picture is a two-dimensional picture, and the second picture can be used for depth information extraction by stereo matching with the first picture.

The representative point setting unit 120 sets the representative point in the first image. The representative point may be, for example, a pixel in which a grid line is set on an image, and an intersection point of the grid lines is located at a multilattice point. The desired number of representative points can be set, and it is preferable to set the number of representative points according to the performance of the device which performs the image processing and the accuracy to be obtained. For example, 400 pixels arranged at equal intervals of 20 columns and 20 columns may be set as the representative point.

The correspondence point control unit 130 detects a correspondence point corresponding to the representative point in the first image in the second image. In detecting the corresponding point, a block matching method may be used.

5 is a diagram for explaining a block matching method.

As shown in Fig. 5, the block matching method uses a pixel value of a pixel around the pixel α of the L image, and selects a corresponding pixel in the R image, that is, a pixel having a high correlation with the pixel of interest. How to search within the search scope. In the block matching method, the pixel value of the peripheral is calculated in each pixel of interest and the pixel of comparison within a search range, and the correlation between the pixel of interest and each pixel of comparison is calculated from the calculated peripheral pixel values. . In the correlation determination, a sum of absolute difference (SAD), a sum of squared intensity difference (SSD), or the like is used. The pixel having the highest correlation with the pixel of interest is the corresponding pixel α 'for the pixel of interest.

In the stereo image processing method according to the embodiment of the present invention, in the first image, the pixel is the pixel of interest at the representative point, and the pixel having the highest correlation among the pixels to be compared is the corresponding point.

For example, in FIG. 6, the corresponding point α 'in the right image R, which is the second image corresponding to the representative point α of the left image L, which is the first image, is within a search range surrounded by a rectangle. Search. As a result of the search, the highest point of correlation is detected as the corresponding point α '.

The search range is around the coordinates in the second image corresponding to the coordinates of the representative point in the first image. Even when the left and right are reversed in the first image and the second image, the corresponding point can be searched by setting the search range in the coordinates reversed from left to right. In addition, the search range can be set to a range of appropriate desired size in accordance with the parallax.

In this way, the corresponding corresponding point is detected for each representative point.

In addition, the correspondence point control part 130 excludes the correspondence point computed as a deviation value by the deviation value calculation part 141 mentioned later.

The calculator 140 includes a deviation value calculator 141, an approximation curve calculator 142, and a height correction value calculator 143.

The deviation value calculation unit 141 determines a corresponding point corresponding to a representative point of a portion having a small feature of the image or a corresponding point exhibiting a tendency different from surrounding corresponding points with respect to the corresponding point detected by the corresponding point control unit 130 as a deviation value. do.

Determination of the deviation value can be performed in two steps. For example, as a 1st step, the corresponding point corresponding to the representative point of the part with few features of an image is made into the deviation value. Since the correspondence point corresponding to the representative point of the part with few features of an image is likely to be selected incorrectly, the correspondence point corresponding to the related part is not used in the post-processing except as a deviation value.

Specifically, in the process of detecting the corresponding point corresponding to the representative point using SAD, the SAD value of the point determined as the representative point, that is, the point with the smallest SAD value is called SAD _1st , and the SAD value is the representative point. When the SAD value of the next small point is called SAD _2nd , the corresponding point is determined as the deviation value when the difference between the value between SAD _2nd and SAD _1st is smaller than the predetermined threshold Th ₁ .

That is, in the case of SAD _2nd -SAD _1st <Th ₁ , it is determined as a deviation value as a portion having a small feature of the image, and in the case of SAD _2nd -SAD _1st > = Th ₁ , the image is sufficiently characterized as a portion. We just leave as correspondence point.

In addition, as a second step, when the detected correspondence point has a tendency different from the tendency shown in the surrounding correspondence points, the adequacy correspondence point is determined as a deviation value. That is, it is a deviation value by comparing the correspondence point used as the determination object of the deviation value with the correspondence point around the correspondence point, and determining whether the correspondence point of the determination object is moving in the same tendency as the surrounding correspondence point, ie, in the same direction. Determine.

FIG. 7 is a diagram showing a specific example in the case where the deviation value is determined by whether the detected corresponding point has a tendency different from the tendency shown in the surrounding corresponding points. In FIG. 7, the correspondence point corresponding to a determination target and the correspondence point corresponding to the representative point of eight surrounding points of the representative point corresponding to the correspondence point are compared. In the correspondence point corresponding to eight representative points of the surroundings, the degree of deviation from the representative point which is the corresponding lattice point, respectively, is represented by a vector, and for the eight vectors thereof, the average vector u _a , v _a is represented. calculation and, if comparing the vector (u, v) of the deviation from its mean value and determines the representative point of the corresponding point on the target, and that the comparison value exceeds a predetermined threshold value (Th _2), and determines the deviation values .

That is, in the case of (uu _a ) ² + (vv _a ) ² > Th ₂ , the corresponding point of the determination target shows a tendency different from the surrounding corresponding points, and is determined as a deviation value, and (u-ua) ² + In the case of (vv _a ) ² <= Th _2, the same tendency as the surrounding corresponding points is shown, and it is left as a corresponding point as it is.

In addition, although only one of the deviation value determinations described as these steps and two steps may be performed, it is preferable to determine the deviation value in two steps in order to calculate an approximation curve with higher precision than in the subsequent processing. . The deviation value may be determined by a method other than the determination of the deviation value shown here.

The approximation curve calculation unit 142 performs a curve approximation on the remaining correspondence point positions except the correspondence points that are determined as deviation values and excluded, and calculates an approximation curve. As the method of curve approximation, for example, the least square method may be used. In addition, the description of a curve approximation here applies to a linear polynomial, ie, includes a linear approximation, and the description of an approximation curve includes an approximation straight line.

8 is a diagram illustrating an example in which curve approximation is performed, and is an approximation curve in which k is calculated. Here, in performing the curve approximation, the calculation is performed excluding the corresponding point determined as the deviation value. In addition, linear approximation is performed in FIG. Since the curve approximation improves the calculation accuracy by performing the curve approximation in the horizontal direction in the case of the display device, it is preferable to perform the curve approximation in the longitudinal direction.

The several approximation curves computed in this way become parallel by parallelization in a 2nd image. In the case of performing the curve approximation in the horizontal direction, it can be leveled by parallelism and can be matched to the scanning direction.

The height correction value calculation part 143 creates the offset map which shows the correction value of a height direction based on the approximation curve computed by the approximation curve calculation part.

9 is a diagram illustrating an example of creating an offset map from an approximation curve indicating a correction value in the height direction in the stereo image processing apparatus according to the embodiment of the present invention.

First, as shown to Fig.9 (a), the height between the approximation curve computed by the approximation curve calculation part 142, and the straight line formed by connecting the representative point corresponding to the corresponding point used for formation of an approximation curve. Detect the difference.

Next, as shown to Fig.9 (b), the difference of detected height is preserve | saved as height direction correction value H, and an offset map is created. In addition, since interpolation is performed between the plurality of approximation curves, that is, the blank lines, an offset map of the height direction correction value is also created between the approximation curves Y'2 and Y'1. The allocation of the height direction correction value H 'between the approximation curves Y'2 and Y'1 gradually decreases the value between the height direction correction value of Y'2 and the height direction correction value of Y'1. By changing.

For example, as shown in Fig. 9B, the height direction correction value is -1 at any offset of Y'1, and the height direction correction value is at any offset of Y'2 located in the vertical direction. In the case of -1.2, the height direction correction value between them is set to be -1.05, -1.1, -1.15 from the side closer to Y'1.

According to the height direction correction value calculated in this way, in the parallelization execution part 150, height correction, ie, parallelization, is performed with respect to the whole 2nd image. In addition, when the parallelization is performed, the image of the upper end and the lower end may be missing by the original image, but at that time, the original image may be used. Thereby, the deviation of the height of a 1st image and a 2nd image can be eliminated.

The corrected stereo pair image is output by the image output unit 160 to a device that performs stereo matching. By stereo matching using the corrected stereo pair picture, the result of good stereo matching can be obtained, and a high quality three-dimensional picture can be obtained.

Next, a stereo image processing method according to an embodiment of the present invention will be described with reference to FIGS. 10 and 11.

10 is a flowchart of a stereo image processing method according to an embodiment of the present invention.

11 is a conceptual diagram showing a processing procedure in the stereo image processing method according to an embodiment of the present invention.

First, stereo pair image data is input (step S110). In Fig. 11, the left reference image is referred to as an L image, and the right representative image is referred to as an R image. In FIG. 10, the stereo pair image of (I) is an input stereo pair image.

Next, a representative point is set for the L image (step S120). In FIG. 11, it shows by (a). The representative point may be a pixel in which a lattice line is set on an image as described above, and the intersection point of the lattice lines is located at a multi-grid point. The representative point can be set to a desired number, and in Fig. 11, 16 representative points of (vertical 4) * (lateral 4) are set.

The corresponding point corresponding to the representative point of the image is detected in the R image (step S130). In FIG. 11, it shows by (b). Since 16 representative points are set, 16 corresponding points are also detected. The above-described block matching method can be used to detect the corresponding point.

For each of the detected corresponding points, it is determined whether or not it is a deviation value (step S140). The determination determines the deviation value as a method of determining a corresponding point corresponding to the representative point of the portion having a small feature in the above-described image as a deviation value and / or a point where the detected corresponding point has a tendency different from the tendency shown in the surrounding corresponding points as the deviation value. How to use. In FIG. 11, the state where the deviation value was determined is shown in (c), and the point shown with the triangle is the correspondence point determined as the deviation value.

The exclusion of the corresponding point determined as the deviation value is performed (step S150). For each corresponding point, step S140 and step S150 may be repeated until the deviation value determination is completed, and after the deviation value determination processing is performed in step S140 for each corresponding point, the steps ( In S150), the exclusion process of the corresponding point determined as the deviation value may be performed. In FIG. 11, the state in which the correspondence point determined as the deviation value was removed in (d) is shown.

The curve approximation is performed with respect to the remaining corresponding points except the corresponding points determined as the deviation values (step S160). In FIG. 11, it shows in (e). As described above, the curve approximation method may include, but is not limited to, the least square method.

In addition, the description of a curve approximation here applies to a linear polynomial, ie, includes a linear approximation, and the description of an approximation curve includes an approximation straight line. In FIG. 11E, a straight line approximation is performed and an approximation curve is calculated.

Using the approximation curve calculated by the curve approximation, an offset map representing the height direction correction value is created (step S170). As described above, the difference in height is detected between the approximation curve and a straight line formed by connecting representative points corresponding to the corresponding points used to form the approximation curve, and for the approximation curves, the height correction of each approximation curve is performed. Based on the value, a sequential height correction value is assigned.

The corresponding image is parallelized using the offset map, and height correction is performed (step S180). In FIG. 10, X (II) is a stereo pair image after correction in which height correction was performed.

The stereo pair image after correction is output (step S190), and the processing ends. The output stereo pair image after correction can be used for stereo matching in post processing. Since the height correction is made, when extracting the depth information in stereo matching, a high-quality 3D image can be displayed without large noise and without distortion.

The image processing apparatus according to one embodiment of the present invention may be used for a display device.

12 is a diagram illustrating a main configuration of a display device according to an embodiment of the present invention.

The display apparatus 1000 according to an exemplary embodiment of the present invention may include a parallelization converter 100, an image signal input unit 200, a stereo matching unit 300, a parallax image synthesis unit 400, and a timing controller 500. And a display panel 600. As a representative example of the display device 1000, a 3D TV or a 3D display may be mentioned as an example.

The video signal input unit 200 receives an external video signal. When a video signal is input from the outside, the video signal input unit 200 inputs the video signal to the parallelization converter as video data.

The parallelization converting unit 100 is the stereo image processing apparatus 100 described above, receives image data, performs height correction on the stereo pair image, and outputs the corrected stereo pair image to the stereo matching unit 300. do.

The stereo matching unit 300 performs stereo matching on the corrected stereo pair image. That is, the corresponding point in each image which captures the same point is calculated | required, and depth information is extracted using the parallax. In other words, the stereo matching unit 300 converts the data into the first image of the corrected stereo pair image and the depth information. The first image and the depth information are sent from the stereo matching unit 300 to the parallax image synthesis unit 400.

The parallax image synthesizing unit 400 creates a parallax image by using the first image and the depth information. That is, the parallax image may be a format according to a desired 3D display method according to the type of the display device 1000. For example, a parallax image corresponding to a parallax barrier using a parallax barrier may be used. It may be a parallax image corresponding to a lenticular lens method. When creating such a parallax video, multi-view rendering is performed.

The timing controller 500 generates a signal for driving the pixels on the display panel 600 with the parallax image created by the parallax image synthesizing unit 400.

The display panel 600 is, for example, an LCD or a PDP, and displays a 3D image (3D image).

In the display apparatus, since the stereo image image | photographed by the various imaging devices is used in various imaging places after photography, the stereo image using a well-known pattern cannot be parallelized. Here, it is necessary to perform the parallelization process in a step before the stereo matching later. According to the display device according to an embodiment of the present invention, it is difficult to cope with conventional post-parallelization processing after imaging, and there is no characteristic pattern such as a straight edge, or a scanning line does not coincide with an isolinear line. It is possible to perform excellent parallelization processing on various stereo pair images, such as non-images.

100: stereo image processing apparatus 110: image input unit
120: representative point setting unit 130: correspondence point control unit
140: calculator 150: height correction execution unit
160: image output unit

Claims (9)

An image input unit to which a stereo pair image is input;
A representative point setting unit that sets a plurality of representative points in the first image of the stereo pair image;
A correspondence point control unit for detecting a plurality of correspondence points corresponding to the plurality of representative points in the second image of the stereo pair image and excluding a corresponding point determined as a deviation value;
A deviation value calculator which determines whether the corresponding point is the deviation value;
An approximation curve calculator for performing curve approximation by remaining correspondence points except for the excluded correspondence points;
A height correction value calculator for calculating a height correction value from a plurality of approximation curves created by the curve approximation;
A parallelization execution unit that performs parallelization of the second image based on the height correction value; And
And an image output section for outputting the first image and the second image subjected to parallelization. The method of claim 1,
The height correction value calculation unit,
The difference between the heights of the straight lines formed by connecting the approximation curves and the corresponding representative points is measured, and the difference of the heights is a height direction correction value, and the plurality of the approximations is based on the height direction correction value. Create an offset map by assigning height direction corrections to the blank rows between the curves,
The parallelization execution unit,
And the second image is parallelized based on the offset map. The method according to claim 1 or 2,
And the corresponding point controller detects the plurality of corresponding points by a block matching method. The method according to any one of claims 1 to 3,
The deviation value calculating section determines the detected corresponding point as the deviation value when the detected corresponding point corresponds to a representative point of a portion having a small feature of an image and / or has a tendency different from a tendency that appears in a corresponding point around. Stereo image processing apparatus characterized in that. Stereo pair image is input,
Set a plurality of representative points in a first picture of the stereo pair picture;
Detect a plurality of corresponding points corresponding to the plurality of representative points in the second image of the stereo pair image;
Determining whether each of the corresponding points is a deviation value, except for the corresponding point determined as the deviation value;
Curve approximation is performed by corresponding points after the deviation value determination except for the excluded corresponding points;
Calculating a height correction value from the plurality of approximation curves created by the curve approximation;
Parallelize the second image based on the height correction value; And
And outputting the first image and the second image subjected to parallelization. The method of claim 5,
Calculation of the height correction value,
The difference between the height and the straight line formed by connecting the approximation curve and the corresponding representative points is measured, and the difference of the height is a height direction correction value, and the plurality of the approximation curves based on the height direction correction value. Creating an offset map by assigning a height direction correction value to a blank line between
The parallelization is based on the offset map. The method according to claim 5 or 6,
The detection of the corresponding point is performed by a block matching method. The method according to any one of claims 5 to 7,
Characterized in that the detected corresponding point is determined as the deviation value when the detected corresponding point corresponds to a representative point of a portion having a small feature of the image and / or has a tendency different from the tendency shown in the surrounding corresponding point. Image processing apparatus. A display panel;
A video signal input unit to which a video signal including a stereo pair image composed of a first image and a second image is input;
A stereo image processing apparatus for performing height correction on the second image of the stereo pair image and generating a corrected stereo pair image;
A stereo matching section for performing stereo matching on the corrected stereo pair image and generating depth information of the first image;
A parallax image synthesizing unit configured to create a parallax image using the first image and the depth information; And
A timing controller configured to generate a clock signal to distribute the parallax image to pixels on the display panel;
The stereo image processing apparatus includes: an image input unit to receive a stereo pair image from the video signal input unit;
A representative point setting unit that sets a plurality of representative points in the first image of the stereo pair image;
A corresponding point controller for detecting a plurality of corresponding points corresponding to the plurality of representative points in the second image of the stereo pair image and excluding corresponding points opposite to the deviation value;
A deviation value calculator which determines whether the corresponding point is the deviation value;
An approximation curve calculator for performing curve approximation by remaining correspondence points except for the excluded correspondence points;
A height correction value calculator for calculating a height correction value from a plurality of approximation curves created by the curve approximation;
A parallelization execution unit that performs parallelization of a second image based on the height correction value; And
And an image output unit for outputting a corrected stereo pair image composed of the first image and the second image that has been parallelized to the stereo matching unit.

Images