JPH09153148A - Correcting condition detector for stereoscopic picture - Google Patents

Abstract

PROBLEM TO BE SOLVED: To detect a correcting condition to be used for matching the epipolar lines of right and left pictures of a twin-lens stereoscopic picture by picture processing without using a test pattern. SOLUTION: A picture transformation part 30 executes picture transformation by transforming the movement, rotation and magnification of either one of right and left pictures taken by right and left cameras. A correspondence point searching part 32 detects correspondence points between the transformed right and left pictures and detects identified points and unidentified points for phase differences based on the correspondence point detection result. An evaluation value calculating part 34 finds out the detection rate of the phase difference defined points in each picture transformation condition and a maximum value detecting part 36 detects a picture transformation condition having the maximum detection rate as a correcting condition for matching epipolar lines.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus for detecting a correction condition of a stereo image necessary for matching epipolar lines in a stereo image, and more particularly, to detect the correction condition by utilizing a detection rate of a phase difference determination point. Regarding the device.

[0002]

2. Description of the Related Art Conventionally, a twin-lens stereo image method has been used as a general-purpose method for obtaining distance information of a three-dimensional image. In the binocular stereo image method, the left and right images obtained by the left and right cameras are scanned to detect corresponding pixels, that is, corresponding points in these images. Distance information can be obtained from the phase difference between corresponding points. In order to reduce false correspondences and improve the detection accuracy of corresponding points, it is important to match the epipolar lines of the left and right images.

As a method of aligning epipolar lines, for example, a test pattern is photographed by left and right cameras and image recognition technology is used (reference: information processing vol.31.
No.5 May 1990 pp.650-659).

[0004]

In the conventional method using the image recognition technique, the epipolar lines of the left and right images are aligned by recognizing a test pattern having a predetermined shape. Therefore, when the epipolar lines of the left and right images are displaced due to some factor while detecting the corresponding points, the displacement can be corrected only when the same pattern as the test pattern exists in these left and right images. That is, if the same pattern as the test pattern does not exist, there is an inconvenience that it cannot be corrected.

Therefore, it has been desired to realize an apparatus capable of detecting the correction condition necessary for matching the epipolar lines of the left and right images even if the same pattern as the test pattern does not exist.

More preferably, even if the same pattern as the test pattern does not exist, it is possible to detect the correction condition necessary for matching the epipolar lines of the left and right images, and to realize an apparatus that can quickly detect the correction condition. Was wanted.

The correction condition detected here is used, for example, as the image conversion condition used when image conversion of one of the left and right images is performed to match the epipolar lines of the left and right images, or the correction condition is adjusted to match the epipolar lines of the left and right images. It may be used as the optical axis adjustment condition used when mechanically adjusting the lens optical axes of the left and right cameras that are capturing images.

[0008]

In order to solve the above-mentioned conventional problems, the invention according to claim 1 is a correction condition detecting device for detecting a correction condition for matching epipolar lines of left and right images, and a condition setting unit. , An image conversion unit, a corresponding point search unit, an evaluation value calculation unit, and a maximum value detection unit.

The condition setting unit sequentially sets a plurality of types of candidate conditions for detecting the correction conditions, and the image conversion unit converts one of the left and right images according to the candidate condition. The image conversion performed by the image conversion unit may include parallel movement, rotation, and magnification conversion of the image, and the candidate conditions are used for the image conversion performed by the image conversion unit. The conversion of the image magnification indicates that the image is enlarged or reduced.

The candidate condition relating to the parallel movement of the image is x
An arbitrary suitable number of conditions Δx ₁ , Δx ₂ , ... In which the parallel movement amount in the axial direction (horizontal direction) is determined stepwise, and the parallel movement amount in the y-axis direction (height direction) is gradually changed. It is possible to use a plurality of conditions Δy ₁ , Δy ₂ , ... Amount of parallel movement in the x-axis direction Δx ₁ , Δx ₂ ,
The one with the maximum evaluation value described later is the parallel movement amount in the x-axis direction suitable for aligning the epipolar lines, and the parallel movement amounts Δy ₁ , Δy ₂ , in the y-axis direction ,.
The one having the maximum evaluation value described later among the ... Is the amount of parallel movement in the y-axis direction suitable for aligning the epipolar lines.

The candidate conditions relating to the rotation of the image include a plurality of conditions Δα ₁ °, Δα ₂ °, etc. of an arbitrary suitable number in which the rotation angle around the x axis is determined stepwise, and the rotation angle around the y axis. A plurality of conditions of an arbitrary suitable number determined in stages Δβ ₁ °, Δβ
_{It is possible to use 2} °, ... Or a plurality of conditions Δγ ₁ °, Δγ ₂ ° ,. Rotation angle around x-axis Δα ₁ °, Δα
That evaluation value to be described later of the ₂ ° ...... is maximum becomes the x-axis rotation angle suitable for matching the epipolar line, y axis rotation angle Δβ ₁ °, Δβ ₂ ° ......
Of these, the one with the largest evaluation value described later is the rotation angle around the y-axis suitable for matching the epipolar line, and the rotation angle around the z-axis Δγ ₁ °, Δγ ₂ °. The maximum rotation angle is the rotation angle around the z-axis suitable for matching the epipolar line.

The candidate conditions relating to the magnification conversion of the image include a plurality of conditions K ₁ of an arbitrary suitable number in which the enlargement ratio is determined stepwise.
K ₂ , ... And a plurality of conditions k ₁ , k ₂ ,.
Of the enlargement ratios K ₁ , K ₂ , ... And the reduction ratios k ₁ , k ₂ , ..., the one having the maximum evaluation value described later is the conversion magnification suitable for matching the epipolar lines.

In order to accurately match the epipolar lines of the left and right images, the parallel movement amount in the x-axis direction, the parallel movement amount in the y-axis direction, the rotation angle around the x-axis, the rotation angle around the y-axis, z It is preferable to detect the candidate condition that maximizes the evaluation value for each type of candidate conditions for all types of candidate conditions such as the rotation angle around the axis and the conversion magnification, but if there is no practical problem, the x-axis Parallel translation amount in the direction, translation amount in the y-axis direction, rotation angle around the x-axis, rotation angle around the y-axis, rotation angle around the z-axis, and only some types of candidate conditions for conversion magnification, The candidate condition having the maximum evaluation value may be detected for each type of candidate condition. still,
The parallel movement of the image in the z-axis direction (depth direction) is equivalent to the magnification conversion of the image.

The condition setting unit may set the candidate condition and the image conversion unit may perform the image conversion only for the image conversion for which the candidate condition having the maximum evaluation value is detected.

The corresponding point search unit sets a plurality of types of search conditions by making the scanning direction, the gradation expression of the pixel to be scanned, or the color of the pixel to be scanned different, and setting the left and right images after image conversion. Are left and right search images.

The corresponding point search unit scans the left and right search images according to the search conditions, compares the pixel features of the left search image and the pixel features of the right search image on the corresponding horizontal scanning lines, and determines the difference between the pixel features. Pixels in the left and right search images whose values are less than or equal to a threshold value are detected as corresponding candidate points, and corresponding candidate points forming a common set in the set of corresponding candidate points detected for each search condition are detected as corresponding points. After that, the corresponding point searching unit detects the identified point and the unknown point of the phase difference in the left and right search images using the corresponding point detection result.

When a plurality of types of search conditions are set by making the scanning directions different, the corresponding candidate points are detected as follows.

That is, among the pixels existing on the corresponding horizontal scanning lines in the left and right search images, the pixel of one image of the left and right search images is the target point, and the pixel of the other image of the left and right search images is the control point. To do. Condition a in which the point of interest scanning direction is the right direction (direction from left to right on the horizontal scanning line) and the control point scanning direction is the right direction, and the point of interest scanning direction is the right direction and the control point scanning direction is the left direction Condition b, which is (direction from right to left on the horizontal scanning line),
Select from four types of conditions: condition c in which the scanning direction of the point of interest is left and the scanning direction of the contrast point is right, and condition d in which the scanning direction of the point of interest is left and the scanning direction of contrast point is left. The corresponding candidate points are detected by setting at least two kinds of conditions as described above as search conditions.

In any of the conditions a to d, the control point on the horizontal scan is sequentially scanned in the control point scanning direction with respect to the ith target point on the horizontal scanning line in the scanning direction of the target point. Go on. When the scanning of the control point is completed without detecting the control point in which the difference between the pixel features is equal to or less than the threshold value for the i-th target point, the control point is scanned next for the (i + 1) th target point. When a control point in which the difference in pixel feature is less than or equal to the threshold is detected for the ith target point, the ith target point is detected.
The n-th point of interest and the control point are detected as corresponding candidate points, and thereafter, the control point is scanned for the (i + 1) th point of interest.

In this way, a plurality of types of search conditions are set, that is, the left and right search images are scanned by changing the scanning direction of the target point or the scanning direction of the target point, and a common set is set in the set of corresponding candidate points obtained for each search condition. The false correspondence can be reduced by detecting the corresponding correspondence candidate points as the candidate points.

When a plurality of types of search conditions are set by changing the gradation expression of the pixel to be scanned, the corresponding candidate points are detected as follows.

That is, the corresponding point search section converts the gradation of the left and right search images to prepare a plurality of types of left and right search images having different gradation expressions. Corresponding candidate points are detected for each of the left and right search images with different gradation expressions, and the corresponding candidate points that are a common set in the set of corresponding candidate points obtained for each gradation expression are detected as corresponding points. For example, with left and right cameras 256
When a black and white image with gradation is obtained as a left and right image and this left and right image is obtained as a left and right search image, gradation conversion is not performed 256
The gradation left and right search images are scanned to detect corresponding candidate points, and the 256 gradation left and right search images are converted into gradations to obtain 32 gradation left and right search images. The corresponding candidate points are detected by scanning. Then, a common set of correspondence candidate points detected from the left and right search images of 256 gradations and 32 gradations is detected as a corresponding point. Alternatively, when a black and white image of 256 gradations is obtained as a left and right image by the left and right cameras and this left and right image is obtained as a left and right search image, the left and right search images of 256 gradations are subjected to gradation conversion and different gradation expressions are obtained. A plurality of types of left and right search images of 16 gradations are generated. In this case, the number of gradation levels is the same 16 gradations, and the left and right search images of 16 gradations with different first and 16th gradation widths are generated here. .
Then, the 16-gradation left and right search images with different gradation representations are scanned to detect corresponding candidate points, and a common set of corresponding candidate points detected for each 16-gradation left and right search images is detected as the corresponding points. To do.

As described above, plural kinds of images having different gradation steps are set as left and right search images to detect corresponding candidate points, and plural kinds of search conditions are set, or the gradation steps are the same. As a set of a plurality of types of search conditions in which a plurality of types of images with different first and last gradation widths are used as the left and right search images to detect corresponding candidate points, and in a set of corresponding candidate points obtained for each search condition By detecting the corresponding candidate points that form a common set as the candidate points, it is possible to reduce the false correspondence.

When a plurality of types of search conditions are set by making the color of the pixel to be scanned different, the corresponding candidate points are detected as follows.

That is, when color images are obtained as left and right images by the left and right cameras, the corresponding point search unit determines two or three colors selected from the red image, the green image and the blue image of the left and right images which are color images. Let the images be left and right search images. Corresponding candidate points are detected for each color image in the left and right search images, and corresponding candidate points that are a common set in the set of corresponding candidate points obtained for each color image are detected as corresponding points.
For example, a red image, a green image, and a blue image of left and right images that are color images are used as left and right search images, and the left and right search images are scanned for each color to detect corresponding candidate points. Then, in the set of corresponding candidate points obtained for the red image, the set of corresponding candidate points obtained for the green image, and the set of corresponding candidate points obtained for the blue image, corresponding candidate points forming a common set are detected as corresponding points.

In this way, when color images are obtained as left and right images by the left and right cameras, a plurality of types of color images selected from the red image, the green image and the blue image of the left and right images are respectively used as the left and right search images. By setting the search conditions and detecting the corresponding points that are common sets in the set of corresponding candidate points obtained for each search condition,
False correspondence can be reduced.

The corresponding point search unit detects the identified point and the unknown point of the phase difference in the left and right search images by using the corresponding point detection results obtained by scanning the left and right search images.

The evaluation value calculation unit obtains the evaluation value of the candidate condition for each kind of candidate condition using the detection rate of the phase difference known point in the left and right search images, and the maximum value detection unit makes the maximum evaluation for each kind of candidate condition. The candidate condition that has obtained the value is detected as the correction condition.

As described above, the parallel movement amount in the x-axis direction, the parallel movement amount in the y-axis direction, the rotation angle around the x-axis, y
With respect to all kinds of candidate conditions of the rotation angle around the axis, the rotation angle around the z axis, and the conversion magnification, the evaluation value of the candidate condition is obtained for each kind of candidate condition, and the evaluation value becomes maximum for each kind of candidate condition. Candidate conditions may be detected, and within a range that does not cause any practical problem, the parallel movement amount in the x-axis direction, the parallel movement amount in the y-axis direction, the rotation angle around the x-axis, the rotation angle around the y-axis. , The rotation angle around the z-axis and the conversion factor only for some types of candidate conditions, the evaluation value of the candidate condition is calculated for each type of the candidate condition, and the candidate condition having the maximum evaluation value is detected. good.

Further, in the correction condition detecting device for detecting the correction condition for matching the epipolar lines of the left and right images, the condition setting unit, the image conversion unit, the gradation conversion unit, the level cutout unit, and the corresponding points. It is characterized by comprising a search unit, an evaluation value calculation unit and a maximum value detection unit.

The condition setting unit sequentially sets a plurality of types of candidate conditions for detecting the correction condition, and the image conversion unit performs image conversion on one of the left and right images of m gradations according to the candidate condition. Image conversion performed by the image conversion unit includes parallel movement of images,
Rotation and magnification conversion can be mentioned, and the candidate condition is used for the image conversion performed by the image conversion unit. Image magnification conversion indicates that the image is enlarged or reduced. The image conversion unit performs image conversion on the left and right images of m gradations obtained by the left and right cameras.

The candidate condition relating to the parallel movement of the image is x
A plurality of arbitrary suitable numbers of conditions Δx ₁ , Δx ₂ , ... In which the translation amount in the axial direction is determined stepwise, and a plurality of arbitrary suitable numbers in which the translation amount in the y-axis direction is determined stepwise. Condition Δy
₁ , Δy ₂ , ... Can be used. The parallel movement amount in the x-axis direction, Δx ₁ , Δx ₂ , ..., which has the maximum evaluation value described later is the parallel movement amount in the x-axis direction suitable for aligning the epipolar line, and the y-axis direction. Of the parallel movement amounts Δy ₁ , Δy ₂ , ..., Which has the maximum evaluation value described later is the parallel movement amount in the y-axis direction suitable for aligning the epipolar lines.

As the candidate conditions relating to the rotation of the image, an arbitrary suitable number of plural conditions Δα ₁ °, Δα ₂ °, etc., in which the rotation angle about the x axis is determined stepwise, and the rotation angle about the y axis are included. A plurality of conditions of an arbitrary suitable number determined in stages Δβ ₁ °, Δβ
_{It is possible to use 2} °, ... Or a plurality of conditions Δγ ₁ °, Δγ ₂ ° ,. Rotation angle around x-axis Δα ₁ °, Δα
That evaluation value to be described later of the ₂ ° ...... is maximum becomes the x-axis rotation angle suitable for matching the epipolar line, y axis rotation angle Δβ ₁ °, Δβ ₂ ° ......
Of these, the one with the largest evaluation value described later is the rotation angle around the y-axis suitable for matching the epipolar line, and the rotation angle around the z-axis Δγ ₁ °, Δγ ₂ °. The maximum rotation angle is the rotation angle around the z-axis suitable for matching the epipolar line.

The candidate conditions relating to the magnification conversion of the image include a plurality of conditions K ₁ of an arbitrary suitable number in which the enlargement ratio is determined stepwise.
K _2, a plurality of conditions of any suitable number of ... and the reduction rate determined in steps k _1, k _2, can be used and ..... Of the enlargement ratios K ₁ , K ₂ , ... And the reduction ratios k ₁ , k ₂ , ..., the one having the maximum evaluation value described later is the conversion magnification suitable for matching the epipolar lines.

In order to accurately match the epipolar lines of the left and right images, the parallel movement amount in the x-axis direction, the parallel movement amount in the y-axis direction, the rotation angle around the x-axis, the rotation angle around the y-axis, z It is preferable to detect the candidate condition having the maximum evaluation value for each type of candidate conditions for all types of candidate conditions of the rotation angle around the axis and the conversion magnification, but if there is no practical problem, the y-axis Direction translation amount, z translation direction translation amount, x-axis rotation angle, y-axis rotation angle, z-axis rotation angle, and conversion factor only for some types of candidate conditions, The candidate condition having the maximum evaluation value may be detected for each type of candidate condition. still,
The parallel movement of the image in the z-axis direction (depth direction) is equivalent to the magnification conversion of the image.

The condition setting unit may set the candidate condition and the image conversion unit may perform the image conversion only for the image conversion for which the candidate condition having the maximum evaluation value is detected.

The gradation conversion unit converts the left and right images after image conversion into left and right images of n gradations (m> n), and the level cutout unit selects one or a plurality of types from the left and right images after gradation conversion. The left and right images having the gradation levels of are cut out as left and right attention level images.

The gradation conversion unit converts the left and right images of m gradations (for example, m = 256) obtained by the left and right cameras into n gradations (for example, n).
= 32) to the left and right images. Then, the level cutout unit pays attention to one or a plurality of kinds of gradation levels in the left and right images of n gradations after gradation conversion, and cuts out the left and right images of the focused gradation level as left and right focused level images.

In this way, the left and right images of m gradations obtained by the left and right cameras are converted into left and right images of n gradations where m> n, and only the left and right images of gradation levels focused on from the left and right images of n gradations. Are cut out as left and right focus level images. As will be described later, the detection speed of the corresponding candidate points can be improved by detecting the corresponding candidate points using the left and right focus level images as the left and right search images.

The corresponding point search unit sets a plurality of types of search conditions by differentiating the scanning direction, the gradation expression of the pixel to be scanned or the color of the pixel to be scanned, and sets the left and right focus level images to the left and right. Let it be a search image.

The corresponding point search unit scans the left and right search images according to the search condition, compares the pixel features of the left search image and the pixel features of the right search image on the corresponding horizontal scanning lines, and determines the difference between the pixel features. Pixels in the left and right search images whose values are less than or equal to a threshold value are detected as corresponding candidate points, and corresponding candidate points forming a common set in the set of corresponding candidate points detected for each search condition are detected as corresponding points. After that, the corresponding point searching unit detects the identified point and the unknown point of the phase difference in the left and right search images using the corresponding point detection result.

When a plurality of types of search conditions are set by making the scanning directions different, the corresponding candidate points are detected as follows.

That is, among the pixels existing on the corresponding horizontal scanning lines in the left and right search images, the pixel of one image of the left and right search images is the target point, and the pixel of the other image of the left and right search images is the control point. To do. Condition a in which the point of interest scanning direction is the right direction (direction from left to right on the horizontal scanning line) and the control point scanning direction is the right direction, and the point of interest scanning direction is the right direction and the control point scanning direction is the left direction Condition b, which is (direction from right to left on the horizontal scanning line),
Select from four types of conditions: condition c in which the scanning direction of the point of interest is left and the scanning direction of the contrast point is right, and condition d in which the scanning direction of the point of interest is left and the scanning direction of contrast point is left. The corresponding candidate points are detected by setting at least two kinds of conditions as described above as search conditions.

In any of the conditions a to d, the control point on the horizontal scanning is sequentially scanned in the control point scanning direction with respect to the i-th target point counted sequentially in the control point scanning direction on the horizontal scanning line. Go on. When the scanning of the control point is completed without detecting the control point in which the difference between the pixel features is equal to or less than the threshold value for the i-th target point, the control point is scanned next for the (i + 1) th target point. When a control point in which the difference in pixel feature is less than or equal to the threshold is detected for the ith target point, the ith target point is detected.
The n-th point of interest and the control point are detected as corresponding candidate points, and thereafter, the control point is scanned for the (i + 1) th point of interest.

In this way, a plurality of types of search conditions are set, that is, the left and right search images are scanned by changing the scanning direction of the point of interest or the scanning direction of the target point, and a common set is set in the set of corresponding candidate points obtained for each search condition. The false correspondence can be reduced by detecting the corresponding correspondence candidate points as the candidate points.

When a plurality of types of search conditions are set by varying the gradation expression of the pixel to be scanned, the corresponding candidate points are detected as follows.

That is, the corresponding point searching section converts the gradations of the left and right search images to prepare a plurality of types of left and right search images having different gradation expressions. Corresponding candidate points are detected for each of the left and right search images with different gradation expressions, and the corresponding candidate points that are a common set in the set of corresponding candidate points obtained for each gradation expression are detected as corresponding points. For example, when a black-and-white image of 32 gradations is obtained as a left and right focused level image and this left and right focused level image of 32 gradations is obtained as a left and right search image, first, a left and right searched image of 32 gradations that is not subjected to gradation conversion is scanned. Correspondence candidate points are detected, and the left and right search images of 32 gradations are subjected to gradation conversion to obtain left and right search images of 16 gradations, and the left and right search images of 16 gradations are scanned to detect correspondence candidate points. And these 3
A common set of corresponding candidate points detected from the left and right search images of 2 gradations and 16 gradations is detected as the corresponding points. Alternatively, a black and white image of 32 gradations is obtained as the left and right focus level images,
When the left and right focus level images of gradation are used as the left and right search images,
The left and right search images of 32 gradations are gradation-converted to generate a plurality of types of left and right search images of 16 gradations with different gradation expressions. In this case, the number of gradation levels is the same 16 gradations, and the left and right search images of 16 gradations with different first and 16th gradation widths are generated here. . Then, the left and right search images of 16 gradations different in gradation expression are scanned to detect corresponding candidate points, and
A common set of corresponding candidate points detected for each of the 6-gradation left and right search images is detected as a corresponding point.

As described above, plural kinds of images having different gradation steps are set as left and right search images, and plural kinds of search conditions for detecting corresponding candidate points are set, or the gradation steps have the same gradation number. As a set of a plurality of types of search conditions in which a plurality of types of images with different first and last gradation widths are used as the left and right search images to detect corresponding candidate points, and in a set of corresponding candidate points obtained for each search condition By detecting the corresponding candidate points that form a common set as the candidate points, it is possible to reduce the false correspondence.

When a plurality of types of search conditions are set by making the color of the pixel to be scanned different, the corresponding candidate points are detected as follows.

That is, when color images are obtained as left and right images by the left and right cameras, the corresponding point searching section determines two or three colors selected from the red image, the green image and the blue image of the left and right images which are color images. Let the images be left and right search images. Corresponding candidate points are detected for each color image in the left and right search images, and corresponding candidate points that are a common set in the set of corresponding candidate points obtained for each color image are detected as corresponding points.
For example, a red image, a green image, and a blue image of left and right images that are color images are used as left and right search images, and the left and right search images are scanned for each color to detect corresponding candidate points. Then, in the set of corresponding candidate points obtained for the red image, the set of corresponding candidate points obtained for the green image, and the set of corresponding candidate points obtained for the blue image, corresponding candidate points forming a common set are detected as corresponding points.

In this way, when color images are obtained as left and right images by the left and right cameras, a plurality of types of color images selected from the red image, the green image and the blue image of the left and right images are used as the left and right search images. By setting the search conditions and detecting the corresponding points that are common sets in the set of corresponding candidate points obtained for each search condition,
False correspondence can be reduced.

The corresponding point searching unit detects the identified point and the unknown point of the phase difference in the left and right search images by using the corresponding point detection results obtained by scanning the left and right search images.

The evaluation value calculation unit obtains the evaluation value of the candidate condition for each type of candidate condition using the detection rate of the phase difference known point in the left and right search images, and the maximum value detection unit determines the maximum value for each type of candidate condition. The candidate condition for which the evaluation value is obtained is detected as the correction condition.

As described above, the parallel movement amount in the x-axis direction, the parallel movement amount in the y-axis direction, the rotation angle around the x-axis, y
With respect to all kinds of candidate conditions of the rotation angle around the axis, the rotation angle around the z axis, and the conversion magnification, the evaluation value of the candidate condition is obtained for each kind of candidate condition, and the evaluation value becomes maximum for each kind of candidate condition. Candidate conditions may be detected, and within a range that does not cause any practical problem, the parallel movement amount in the x-axis direction, the parallel movement amount in the y-axis direction, the rotation angle around the x-axis, the rotation angle around the y-axis. , The rotation angle around the z-axis and the conversion factor only for some types of candidate conditions, the evaluation value of the candidate condition is calculated for each type of the candidate condition, and the candidate condition having the maximum evaluation value is detected. good.

Further, in the correction condition detecting device for detecting the correction condition for matching the epipolar lines of the left and right images, the condition setting part, the image converting part, the corresponding point searching part, the evaluation value calculating part and the maximum value are added. It is characterized by comprising a value detection unit.

The condition setting unit sequentially sets candidate conditions for detecting the height direction correction condition, and the image conversion unit performs image conversion on one of the left and right images according to the candidate condition. The image conversion unit sets the x-axis direction as the horizontal direction, the y-axis direction as the height direction, and the z-axis direction as the depth direction, and performs image conversion for detecting the height direction correction condition in the y-axis direction (height direction). The translation is performed and the candidate condition is used for this image conversion.

The candidate conditions include a plurality of conditions Δy ₁ and Δy of an arbitrary suitable number in which the amount of parallel movement in the y-axis direction is determined stepwise.
₂ , ... can be used. Of the translation amounts Δy ₁ , Δy ₂ , ... In the z-axis direction, the _one having the maximum evaluation value described later is y suitable for matching the epipolar line.
It is the amount of parallel movement in the axial direction, that is, the height direction correction condition.

The corresponding point search unit sets a plurality of types of search conditions by making the scanning direction, the gradation expression of the pixel to be scanned or the color of the pixel to be scanned different, and setting the left and right images after image conversion. Are left and right search images.

Then, the corresponding point search unit scans the left and right search images according to the search conditions, compares the pixel features of the left search image and the pixel features of the right search image on the corresponding horizontal scanning lines, and finds the difference between the pixel features. Pixels in the left and right search images that are equal to or less than the threshold value are detected as corresponding candidate points, and corresponding candidate points forming a common set in the set of corresponding candidate points detected for each search condition are detected as corresponding points. After that, the corresponding point searching unit detects the identified point and the unknown point of the phase difference in the left and right search images using the corresponding point detection result.

When a plurality of types of search conditions are set by making the scanning directions different, the corresponding candidate points are detected as follows.

That is, among the pixels existing on the corresponding horizontal scanning lines in the left and right search images, the pixel of one image of the left and right search images is the target point, and the pixel of the other image of the left and right search images is the control point. To do. Condition a in which the point of interest scanning direction is the right direction (direction from left to right on the horizontal scanning line) and the control point scanning direction is the right direction, and the point of interest scanning direction is the right direction and the control point scanning direction is the left direction Condition b, which is (direction from right to left on the horizontal scanning line),
Select from four types of conditions: condition c in which the scanning direction of the point of interest is left and the scanning direction of the contrast point is right, and condition d in which the scanning direction of the point of interest is left and the scanning direction of contrast point is left. The corresponding candidate points are detected by setting at least two kinds of conditions as described above as search conditions.

In any of the conditions a to d, the control point on the horizontal scanning is sequentially scanned in the control point scanning direction with respect to the i-th target point counted sequentially in the control point scanning direction on the horizontal scanning line. Go on. When the scanning of the control point is completed without detecting the control point in which the difference between the pixel features is equal to or less than the threshold value for the i-th target point, the control point is scanned next for the (i + 1) th target point. When a control point in which the difference in pixel feature is less than or equal to the threshold is detected for the ith target point, the ith target point is detected.
The n-th point of interest and the control point are detected as corresponding candidate points, and thereafter, the control point is scanned for the (i + 1) th point of interest.

In this way, a plurality of types of search conditions are set, in which the left and right search images are scanned by changing the scanning direction of the point of interest or the scanning direction of the target point, and a common set is set in the set of corresponding candidate points obtained for each search condition. The false correspondence can be reduced by detecting the corresponding correspondence candidate points as the candidate points.

When a plurality of types of search conditions are set by changing the gradation expression of the pixel to be scanned, the corresponding candidate points are detected as follows.

That is, the corresponding point searching section converts the gradation of the left and right search images to prepare a plurality of types of left and right search images having different gradation expressions. Corresponding candidate points are detected for each of the left and right search images with different gradation expressions, and the corresponding candidate points that are a common set in the set of corresponding candidate points obtained for each gradation expression are detected as corresponding points. For example, with left and right cameras 256
When a black and white image with gradation is obtained as a left and right image and this left and right image is obtained as a left and right search image, gradation conversion is not performed 256
The gradation left and right search images are scanned to detect corresponding candidate points, and the 256 gradation left and right search images are converted into gradations to obtain 32 gradation left and right search images. The corresponding candidate points are detected by scanning. Then, a common set of correspondence candidate points detected from the left and right search images of 256 gradations and 32 gradations is detected as a corresponding point. Alternatively, when a black and white image of 256 gradations is obtained as a left and right image by the left and right cameras and this left and right image is obtained as a left and right search image, the left and right search images of 256 gradations are subjected to gradation conversion and different gradation expressions are obtained. A plurality of types of left and right search images of 16 gradations are generated. In this case, the number of gradation levels is the same 16 gradations, and the left and right search images of 16 gradations with different first and 16th gradation widths are generated here. .
Then, the 16-gradation left and right search images with different gradation representations are scanned to detect corresponding candidate points, and a common set of corresponding candidate points detected for each 16-gradation left and right search images is detected as the corresponding points. To do.

As described above, a plurality of types of search conditions are set in which a plurality of types of images having different gradation levels are used as left and right search images to detect corresponding candidate points, or the number of gradation levels is the same. As a set of a plurality of types of search conditions in which a plurality of types of images with different first and last gradation widths are used as the left and right search images to detect corresponding candidate points, and in a set of corresponding candidate points obtained for each search condition By detecting the corresponding candidate points that form a common set as the candidate points, it is possible to reduce the false correspondence.

When a plurality of types of search conditions are set by making the color of the pixel to be scanned different, the corresponding candidate points are detected as follows.

That is, when color images are obtained as left and right images by the left and right cameras, the corresponding point searching section determines two or three colors selected from the red image, the green image and the blue image of the left and right images which are color images. Let the images be left and right search images. Corresponding candidate points are detected for each color image in the left and right search images, and corresponding candidate points that are a common set in the set of corresponding candidate points obtained for each color image are detected as corresponding points.
For example, a red image, a green image, and a blue image of left and right images that are color images are used as left and right search images, and the left and right search images are scanned for each color to detect corresponding candidate points. Then, in the set of corresponding candidate points obtained for the red image, the set of corresponding candidate points obtained for the green image, and the set of corresponding candidate points obtained for the blue image, corresponding candidate points forming a common set are detected as corresponding points.

When color images are obtained as left and right images by the left and right cameras as described above, a plurality of types of color images selected from the red image, the green image and the blue image of the left and right images are used as left and right search images. By setting the search conditions and detecting the corresponding points that are common sets in the set of corresponding candidate points obtained for each search condition,
False correspondence can be reduced.

The corresponding point searching unit uses the corresponding point detection results obtained by scanning the left and right search images to find out the unknown and unknown points of the short-distance phase difference in which the amount of shift in the height direction in the left and right search images is large. To detect.

The evaluation value calculation unit obtains the detection rate of the phase difference determination point for the short-distance phase difference in which the shift amount in the height direction in the left and right search images is large, and uses the detection rate to evaluate the height direction candidate condition. Then, the maximum value detection unit detects the height direction candidate condition having the maximum evaluation value as the height direction correction condition.

Further, in the correction condition detecting device for detecting the correction condition for matching the epipolar lines of the left and right images, the condition setting part, the image converting part, the corresponding point searching part, the evaluation value calculating part and the maximum value are set. It is characterized by comprising a detection unit.

The condition setting unit sequentially sets the candidate conditions for detecting the tilt angle correction condition, and the image conversion unit converts one of the left and right images according to the candidate condition. The image conversion unit performs rotation around the z axis as image conversion for detecting the tilt angle correction condition, with the x axis direction as the horizontal direction, the y axis direction as the height direction, and the z axis direction as the depth direction. Use candidate conditions for.

As the candidate conditions, an arbitrary suitable number of plural conditions Δγ ₁ , Δγ ₂ , in which the rotation angle around the z axis is determined stepwise,
... can be used. Rotation angle about z-axis Δγ
_{Among 1} , ₁ , Δγ ₂ , ..., the one with the maximum evaluation value described later is the rotation angle around the z-axis suitable for aligning the epipolar line, that is, the tilt angle correction condition.

The corresponding point search unit sets a plurality of types of search conditions by making the scanning direction, the gradation expression of the pixel to be scanned or the color of the pixel to be scanned different, and setting the left and right images after image conversion. Are left and right search images.

Then, the corresponding point search unit scans the left and right search images according to the search conditions, compares the pixel features of the left search image and the pixel features of the right search image on the corresponding horizontal scanning lines, and finds the difference between the pixel features. Pixels in the left and right search images that are equal to or less than the threshold value are detected as corresponding candidate points, and corresponding candidate points forming a common set in the set of corresponding candidate points detected for each search condition are detected as corresponding points. After that, the corresponding point searching unit detects the identified point and the unknown point of the phase difference in the left and right search images using the corresponding point detection result.

When a plurality of types of search conditions are set by making the scanning directions different, the corresponding candidate points are detected as follows.

That is, among the pixels existing on the corresponding horizontal scanning lines in the left and right search images, the pixel of one image of the left and right search images is the target point, and the pixel of the other image of the left and right search image is the control point. To do. Condition a in which the point of interest scanning direction is the right direction (direction from left to right on the horizontal scanning line) and the control point scanning direction is the right direction, and the point of interest scanning direction is the right direction and the control point scanning direction is the left direction Condition b, which is (direction from right to left on the horizontal scanning line),
Select from four types of conditions: condition c in which the scanning direction of the point of interest is left and the scanning direction of the contrast point is right, and condition d in which the scanning direction of the point of interest is left and the scanning direction of contrast point is left. The corresponding candidate points are detected by setting at least two kinds of conditions as described above as search conditions.

In any of the conditions a to d, the control point on the horizontal scan is sequentially scanned in the control point scanning direction with respect to the i-th target point counted in the scanning direction of the target point on the horizontal scanning line. Go on. When the scanning of the control point is completed without detecting the control point in which the difference between the pixel features is equal to or less than the threshold value for the i-th target point, the control point is scanned next for the (i + 1) th target point. When a control point in which the difference in pixel feature is less than or equal to the threshold is detected for the ith target point, the ith target point is detected.
The n-th point of interest and the control point are detected as corresponding candidate points, and thereafter, the control point is scanned for the (i + 1) th point of interest.

As described above, a plurality of types of search conditions are set such that the left and right search images are scanned by changing the scanning direction of the point of interest or the scanning direction of the target point, and a common set is set in the set of corresponding candidate points obtained for each search condition. The false correspondence can be reduced by detecting the corresponding correspondence candidate points as the candidate points.

When a plurality of types of search conditions are set by changing the gradation expression of the pixel to be scanned, the corresponding candidate points are detected as follows.

That is, the corresponding point searching section converts the gradation of the left and right search images to prepare a plurality of types of left and right search images having different gradation expressions. Corresponding candidate points are detected for each of the left and right search images with different gradation expressions, and the corresponding candidate points that are a common set in the set of corresponding candidate points obtained for each gradation expression are detected as corresponding points. For example, with left and right cameras 256
When a black and white image with gradation is obtained as a left and right image and this left and right image is obtained as a left and right search image, gradation conversion is not performed 256
The gradation left and right search images are scanned to detect corresponding candidate points, and the 256 gradation left and right search images are converted into gradations to obtain 32 gradation left and right search images. The corresponding candidate points are detected by scanning. Then, a common set of correspondence candidate points detected from the left and right search images of 256 gradations and 32 gradations is detected as a corresponding point. Alternatively, when a black and white image of 256 gradations is obtained as a left and right image by the left and right cameras and this left and right image is obtained as a left and right search image, the left and right search images of 256 gradations are subjected to gradation conversion and different gradation expressions are obtained. A plurality of types of left and right search images of 16 gradations are generated. In this case, the number of gradation levels is the same 16 gradations, and the left and right search images of 16 gradations with different first and 16th gradation widths are generated here. .
Then, the 16-gradation left and right search images with different gradation representations are scanned to detect corresponding candidate points, and a common set of corresponding candidate points detected for each 16-gradation left and right search images is detected as the corresponding points. To do.

As described above, plural kinds of images having different gradation steps are set as left and right search images, and plural kinds of search conditions are set to detect corresponding candidate points. As a set of a plurality of types of search conditions in which a plurality of types of images with different first and last gradation widths are used as the left and right search images to detect corresponding candidate points, and in a set of corresponding candidate points obtained for each search condition By detecting the corresponding candidate points that form a common set as the candidate points, it is possible to reduce the false correspondence.

When a plurality of types of search conditions are set by making the color of the pixel to be scanned different, the corresponding candidate points are detected as follows.

That is, when color images are obtained as left and right images by the left and right cameras, the corresponding point searching unit determines two or three types of colors selected from the red image, the green image and the blue image of the left and right images which are color images. Let the images be left and right search images. Corresponding candidate points are detected for each color image in the left and right search images, and corresponding candidate points that are a common set in the set of corresponding candidate points obtained for each color image are detected as corresponding points.
For example, a red image, a green image, and a blue image of left and right images that are color images are used as left and right search images, and the left and right search images are scanned for each color to detect corresponding candidate points. Then, in the set of corresponding candidate points obtained for the red image, the set of corresponding candidate points obtained for the green image, and the set of corresponding candidate points obtained for the blue image, corresponding candidate points forming a common set are detected as corresponding points.

In this way, when color images are obtained as left and right images by the left and right cameras, a plurality of types of color images selected from the red image, the green image and the blue image of the left and right images are used as the left and right search images. By setting the search conditions and detecting the corresponding points that are common sets in the set of corresponding candidate points obtained for each search condition,
False correspondence can be reduced.

The corresponding point search unit uses the corresponding point detection results obtained by scanning the left and right search images to identify the clear and unknown points of the long-distance phase difference in which the shift amount of the tilt angle is large in the left and right search images. To detect.

The evaluation value calculation unit obtains the detection rate of the phase difference determination point for the long-distance phase difference in which the shift amount of the tilt angle is large in the left and right search images, and obtains the evaluation value of the tilt angle candidate condition using the detection rate. The maximum value detection unit detects the tilt angle candidate condition having the maximum evaluation value as the tilt angle correction condition.

[0089]

BEST MODE FOR CARRYING OUT THE INVENTION

<First Embodiment of the Invention of Claim 1> FIG. 1 is a functional block diagram for explaining the first embodiment of the invention of claim 1. In the figure, reference numerals 10 and 12 denote a left camera and a right camera used for the binocular stereo image method. The left image captured by the left camera 10 is stored in the first left memory 14, and the right camera 12 is also stored.
The right image photographed in step 1 is stored in the first right memory 16.

Reference numeral 18 denotes a correction condition detection device. The correction condition detection device 18 of this embodiment detects correction conditions used for epipolar line alignment of the left and right images taken by the left camera 10 and the right camera 12. . In addition to this, the correction condition detection device 18 of the present embodiment performs image conversion of one of the left image and the right image captured by the left camera 10 and the right camera 12 based on the detected correction condition, and the left after image conversion is performed. The image and the right image are stored in the second left memory 20 and the second right memory 22. In the left image and the right image after the image conversion based on the correction condition, the epipolar lines are in the same or almost the same state.

Reference numeral 24 indicates a corresponding point searching device. The corresponding point searching device 24 scans the left image and the right image after the image conversion based on the correction condition to detect corresponding pixels (corresponding points) in the left image and the right image, and further detects the phase difference between the corresponding points. , And stores it in the distance information memory 26. The phase difference between the corresponding points is used as distance information of the three-dimensional image.

Then, the correction condition detecting apparatus 1 of this embodiment
8 includes a condition setting unit 28, an image converting unit 30, a corresponding point searching unit 32, an evaluation value calculating unit 34, a maximum value detecting unit 36, a third left memory 38 and a third right memory 40.

The condition setting unit 28 sequentially sets a plurality of types of candidate conditions for detecting the correction condition, and the image conversion unit 30
Converts one of the left image and the right image obtained by the left camera 10 and the right camera 12 according to the candidate condition.

Here, the condition setting unit 28 stores a plurality of types of candidate conditions for detecting the correction conditions and a plurality of types of correction conditions for matching the epipolar lines. The correction condition to be stored is the correction condition detected by the maximum value detection unit 36.

When the correction condition is detected, the condition setting unit 2
8 outputs the candidate condition to the image conversion unit 30, and the image conversion unit 30 performs image conversion on one of the left image and the right image obtained by the left camera 10 and the right camera 12 according to the candidate condition. The image conversion unit 30 stores the left image and the right image after the image conversion based on the candidate conditions in the third left memory 38 and the third right memory 40. One of the left image and the right image stored in these memories 38 and 40 is an image-converted image, and the other is an image that has not been image-converted.

When the epipolar line is matched,
The condition setting unit 28 outputs the correction condition to the image conversion unit 30, and the image conversion unit 30 outputs the left camera 10 and the right camera 12.
One of the left image and the right image obtained by the above is subjected to image conversion according to this correction condition. The image conversion unit 30 outputs the left image and the right image after the image conversion based on the correction condition to the second left memory 20.
And the second right memory 22. One of the left image and the right image stored in the memories 20 and 22 is an image-converted image, and the other is an image that is not image-converted.

The condition setting section 28 stores candidate conditions relating to parallel movement, rotation and magnification conversion of an image. Therefore, the correction condition detected by the maximum value detection unit 36 is the correction condition relating to the parallel movement, rotation, and magnification conversion of the image, and the image conversion unit 3
Image conversion performed by 0 according to the candidate condition and the correction condition is image parallel movement, rotation, and magnification conversion. Known methods can be used for these parallel movement, rotation, and magnification conversion.

[0098] The candidate condition relating to translation of the image, the x-axis direction plurality example five moving amount Δx ₁ ~Δx ₅ that defines the parallel movement amount in the (horizontal) stepwise, y-axis direction two candidates as candidate condition storing a plurality example five variable Δy ₁ ~Δy ₅ which defines stepwise translation of the (vertical or height direction), thus involving translation of the image Conditions Δx _{1 to} Δx ₅ and Δy _{1 to} Δy ₅
Is stored.

Among the parallel movement amounts Δx _{1 to} Δx _{5 in} the x-axis direction, the one having the maximum evaluation value described later is the correction condition relating to the parallel movement in the x-axis direction, and the parallel movement in the y-axis direction. Among the quantities Δy _{1 to} Δy _5, the one having the maximum evaluation value described later is the correction condition related to the parallel movement in the y-axis direction.

The candidate condition relating to the image rotation is x
A plurality of, for example, five angles Δα ₁ ° to Δα ₅ ° in which the rotation angle around the axis is determined in stages, and a plurality of, for example, five angles Δβ ₁ ° in which the rotation angle around the y axis is determined in stages. Δβ ₅ °
And a plurality of, for example, five angles Δγ ₁ ° to Δγ _{5 in} which the rotation angle around the z-axis is determined stepwise, and therefore, three types of candidate conditions Δα ₁ ° as candidate conditions relating to image rotation.
~ Δα ₅ °, Δβ ₁ ° ~ Δβ ₅ ° and Δγ ₁ ° ~ Δγ ₅
Remember °.

Among the rotation angles Δα ₁ ° to Δα ₅ ° about the x-axis, the one having the largest evaluation value described later is the correction condition relating to the rotation about the x-axis, and the rotation angle Δβ ₁ about the y-axis.
The one with the maximum evaluation value described below among ° to Δβ ₅ ° is the correction condition related to the rotation around the y-axis, and z
Among the rotation angles Δγ ₁ ° to Δγ ₅ about the axis, the one having the maximum evaluation value described later is the correction condition related to the rotation about the z axis.

As the candidate conditions relating to image magnification conversion, a plurality of image enlargement rates are set in stages, for example, five conversion magnifications K _{1 to} K _5, and a plurality of image reduction rates are set in stages. For example, five conversion magnifications k _{1 to} k ₅ are stored, and thus one kind of conversion magnification is stored as a candidate condition relating to the conversion of the image magnification.

Among the conversion magnifications K _{1 to} K ₅ and k _{1 to} k ₅ , the one having the maximum evaluation value described later is the correction condition relating to the magnification conversion. Note that the image magnification conversion is equivalent to parallel movement in the z-axis direction (depth direction).

The corresponding point searching unit 32 sets a plurality of types of search conditions by making the scanning direction, the gradation expression of the pixel to be scanned, or the color of the pixel to be scanned different, and sets the left and right after image conversion. Let the images be left and right search images. Here, a plurality of types of search conditions are set by differentiating the gradation expression of pixels to be scanned, and the left and right images after image conversion are set as left and right search images.

Then, the corresponding point search unit 32 scans the left and right search images according to the search conditions, compares the pixel features of the left search image and the pixel features of the right search image on the corresponding horizontal scanning lines, and compares the pixel feature differences. The pixels of the left and right search images in which is less than or equal to the threshold are detected as corresponding candidate points. Further, the corresponding point searching unit 32 detects, as corresponding points, corresponding candidate points forming a common set in the set of corresponding candidate points detected for each search condition. Here, the pixel feature is the pixel density.

For example, the converted left image and right image stored in the third left memory 38 and the third right memory 40 are 256.
In the case of a grayscale black-and-white image, the 256-gradation left and right images are gradation-converted over the entire surface to obtain a 32-gradation left-right image. Set search conditions. Further, the right and left images of 256 gradations are converted over the entire surface to obtain left and right images of 16 gradations, and a second search condition is set so that the left and right images of 16 gradations are left and right search images. Then, the left and right search images of 32 gradations according to the first search condition are scanned over the entire surface, and the pixels of the left and right search images for which the difference in pixel feature on the corresponding horizontal scanning lines is equal to or less than the threshold value are determined by the first search condition. Detect as correspondence candidate points. Furthermore, the left and right search images of 16 gradations under the second search condition are scanned over the entire surface, and the pixels of the left and right search images for which the difference in pixel feature on the corresponding horizontal scanning lines is less than or equal to the threshold value are determined by the second search condition. Detect as correspondence candidate points. Then, the corresponding candidate points that are a common set in the set of corresponding candidate points detected under the first search condition and the set of corresponding candidate points detected under the second search condition are detected as corresponding points.

As the pixel characteristics, various ones other than the pixel density can be used. For example, if pixels are grouped according to density and a group of pixels in one group is represented as a pattern, the corresponding point search unit 32 determines the width and length of the pattern to which the pixel belongs for each pixel of the left and right search images. It may be detected and the width or length may be used as a pixel feature. In this case, the left search image is scanned when detecting the pixel features of the left search image, and the right search image is scanned when detecting the pixel features of the right search image. Then, the width of the pattern to which the pixel belongs is detected on the horizontal scanning line passing through the pixel for which the pixel feature is to be detected. Further, the length of the pattern to which the pixel belongs is detected on the vertical scanning line passing through the pixel for which the pixel feature is to be detected.

Next, the corresponding point searching unit 32 detects the identified point and the unknown point of the phase difference in the left and right search images by using the corresponding point detection result. Here, the phase difference is estimated and applied to the unclear point (pixels not detected as the corresponding point) based on the phase difference of the corresponding points. An uncertain point that cannot be fitted with a phase difference is detected as a phase difference unknown point. After that, regarding the phase difference applied to the correspondence unknown point,
Test whether the phase difference is correct. Then, the unknown correspondence point of the phase difference determined to be correct by this test is detected as the unknown phase difference point, and the unknown correspondence point of the phase difference determined to be incorrect is detected as the unknown phase difference point.

The phase difference known point and the phase difference unknown point can be detected by various methods other than those described here. For example, without applying the phase difference to the correspondence unknown point, the pixel serving as the corresponding point in the left and right search images may be detected as the phase difference known point and the pixel serving as the correspondence unknown point may be detected as the phase difference unknown point. .

The evaluation value calculation unit 34 obtains the evaluation value of the candidate condition for each kind of candidate condition using the detection rate of the phase difference determination point in the left and right search images. Here, the detection rate of the phase difference determination point in the left and right search images is obtained for each candidate condition, and the detection rate is set as the evaluation value of the candidate condition.

The maximum value detection unit 36 detects, as a correction condition, the candidate condition that has the maximum evaluation value for each type of candidate condition. Here, the maximum value detection unit 36 stores the detected correction condition in the condition setting unit 28.

FIG. 2 is a diagram showing the operation flow of the first embodiment of the invention of claim 1. In the same figure, the flow of the operation in the case of focusing on one type out of a plurality of types of candidate conditions and correction conditions and detecting the correction conditions for the focused species using the candidate conditions is shown.

When the operator inputs an operation start signal to the correction condition detecting device 18 (start), the condition setting section 28 first sets one of the plurality of types of candidate conditions as a target condition candidate condition (S1), Then, the condition setting unit 28 outputs one candidate condition (selected candidate condition) selected from the plurality of candidate conditions of the target species (S2). For example, the candidate condition of the target species is a candidate condition Δ related to the parallel movement in the x-axis direction.
x _{1 to} Δx ₅ and a plurality of candidate conditions Δx ₁ to
One candidate condition Δx ₁ selected from Δx ₅ is output.

Next, the image conversion unit 30 performs image conversion of one of the left image and the right image according to the selection candidate condition input from the condition setting unit 28 (S3).

FIG. 3 is a diagram for explaining the first left and right memories for storing the left and right images before the image conversion (before the image conversion unit finishes the image conversion), and FIG. 4 is after the image conversion (the image conversion unit). FIG. 9 is a diagram for explaining a third left / right memory that stores left and right images after (after the end of image conversion).

As shown in FIGS. 3 (A) and 3 (B), x is stored in pixel units on the storage area 14a of the first left memory 14 and storage area 16a of the first right memory 16, respectively. The y coordinate is set, the left image 42 before image conversion captured by the left camera 10 is stored in the first left memory 14, and the right image 44 before image conversion captured by the right camera 12 is stored in the first right memory 16. Store.

As shown in FIGS. 4 (A) and 4 (B), x is also assigned to each pixel on the storage area 38a of the third left memory 38 and the storage area 40a of the third right memory 40. , Y coordinate is set, and the left image 42 after image conversion is set.
In the third memory 38, and the image converter 30 stores the right image 44 after the image conversion in the third right memory 40.

Here, the left image 42 is not converted into the right image 4
If the image conversion unit 4 performs image conversion, the image conversion unit 30 reads the left image 42 from the left camera 10 from the first left memory 14 and stores it in the third left memory 38 without image conversion. Along with this, the image conversion unit 30 displays the right image 4 from the right camera 12.
4 is read from the first right memory 16 and stored in the third right memory 40 without image conversion. The image conversion is performed by the condition setting unit 28
Perform according to the candidate conditions input from. For example, when the candidate condition Δx ₁ is input, image conversion is performed in which each pixel of the right image 44 is translated by Δx _{1 in the} x-axis direction, and the image-converted right image 44 is stored in the third right memory 40.

When a candidate condition relating to image rotation or magnification conversion is set as the candidate condition, when the left image is converted into an image, as shown in FIG. Image conversion is performed with the pixel in the center of the left image corresponding to the optical axis of the lens as the center of rotation or magnification conversion. Similarly, when performing image conversion on the right image, as shown in FIG. 3B, the pixel at the center of the right image corresponding to the lens optical axis of the right camera 12 is used as the center of rotation or magnification conversion. Do.

Next, the corresponding point search unit 32 sets a plurality of types of search conditions by changing the gradation expression of the pixel to be scanned, and sets the entire left and right images after image conversion as left and right search images, and performs each search. Corresponding candidate points in the left and right search images are detected for each condition (S4).

For example, the image-converted left image 42 and right image 44 stored in the third left memory 38 and the third right memory 40.
Is a black-and-white image with 256 gradations, the left image 42 and the right image 44 with 256 gradations are subjected to gradation conversion to obtain a left image 42 and a right image 44 with 32 gradations. Four
The first search condition for setting the 2 and right images 44 as the left search image and the right search image is set. Further, the left image 42 and the right image 44 of 256 gradations are subjected to gradation conversion to obtain the left image 42 and the right image 44 of 16 gradations, and the left image 42 and the right image 44 of 16 gradations are converted to the left search image and The second search condition for the right search image is set.

The corresponding point searching unit 32 is located on the corresponding horizontal scanning line L (horizontal scanning line L having the same vertical coordinate y; see FIGS. 4A and 4B) in the left and right search images. When the corresponding candidate points are detected under the first search condition, the horizontal scanning line L
The upper left image 42 with 256 gradations is read from the third left memory 38, converted into 32 gradations, and the left image 42 for one line converted into 32 gradations is stored in the left line memory. At the same time, the right image 44 with 256 gradations on the horizontal scanning line L is read from the third right memory 40, converted into 32 gradations, and the right image 44 for one line converted into 32 gradations is stored in the right line memory. .

Then, the corresponding point searching unit 32 scans the left image 42 and the right image 44 for one line according to the first search condition. Here, one pixel of the left image 42 and the right image 44 for one line, which has been subjected to gradation conversion, is used as a target point and the other pixel as a reference point, and in the predetermined main scanning direction, for example, the right direction on the horizontal scanning line L. For the i-th point of interest counting sequentially,
The control points on the horizontal scanning line L are sequentially scanned in a predetermined main scanning direction, for example, to the right. When the scanning of the control point is completed without detecting the control point in which the difference in pixel characteristics, for example, the pixel density, is equal to or less than the threshold value for the i-th point of interest
The control point is scanned for the second point of interest. When a control point in which the difference in pixel feature is less than or equal to the threshold value with respect to the i-th point of interest is detected, the i-th point of interest and the control point are detected as corresponding candidate points. The control point is scanned for the point of interest.

In this way, the left and right search images on the corresponding horizontal scanning lines L are scanned, and when the difference in pixel feature becomes equal to or less than the threshold value, the next point of interest is scanned to detect the corresponding candidate point. When the detection of the corresponding candidate points on the horizontal scanning line L is completed over the entire left and right search images, the detection of the corresponding candidate points according to the first search condition is ended.

When the corresponding candidate points are detected under the second search condition, the left and right search images on the corresponding horizontal scanning lines L are scanned in the same manner as in the case of the first search condition, and the difference in pixel feature is When it is less than or equal to the threshold value, the next point of interest is scanned to detect a corresponding candidate point. When the detection of the corresponding candidate points on the horizontal scanning line L is completed over the entire left and right search images, the detection of the corresponding candidate points according to the second search condition is ended.

After detecting the corresponding candidate points on the horizontal scanning line L over the entire left and right search images for all types of search conditions, the corresponding point searching unit 32 next finds the corresponding candidate points obtained for each search condition. Corresponding candidate points that are a common set in the set are detected as corresponding points (S5).

As described above, the left and right search images on the corresponding horizontal scanning lines L are scanned, and when the difference in pixel feature becomes less than or equal to the threshold value, the next point of interest is scanned to detect the corresponding candidate point. Corresponding candidate points that are a common set in the set of corresponding candidate points obtained for each search condition are a set of corresponding candidate points that are probabilistically probable.

Next, the corresponding point searching unit 32 detects the phase difference known point and the phase difference unknown point using the corresponding point detection result, and evaluates the phase difference known point or the number of detected phase difference unknown points. The value is output to the value calculator 34 (S6).

The phase difference known point and the phase difference unknown point detected here relate to the entire left and right images which are the current left and right search images.

Further, here, the phase difference is estimated and applied to the unclear point (pixels not detected as the corresponding point) based on the phase difference of the corresponding points. An uncertain point that cannot be fitted with a phase difference is detected as a phase difference unknown point. After that, it is verified whether the phase difference applied to the unclear point is correct. Then, the unknown correspondence point of the phase difference determined to be correct by this test is set as the phase difference determination point, and the unknown correspondence point of the phase difference determined to be incorrect is detected as the phase difference unknown point.

Next, the evaluation value calculation section 32 obtains the detection rate of the phase difference known points by using the number of detected phase difference known points, and (S
The detection rate of the phase difference determination point is stored in the maximum value detection unit 36 as the evaluation value of the selection candidate condition output by the condition setting unit 28 in 2) (S7).

The detection rate of the phase difference determination point obtained here relates to the entire right and left images which are the current left and right search images. The detection rate of the phase difference known point in this case is, for example, (the detection rate of the phase difference known point) = (the number of detected phase difference known points in the left search image) / (the left image captured by the left camera) Total number of pixels), (Detection rate of phase difference known points) = (Number of detected phase difference known points in left search image) / (Sum of detected number of phase difference known points and phase difference unknown points in left search image), Alternatively, (the detection rate of the phase difference known point) = (the number of detected phase difference known points in the right search image) / (1 shot by the right camera
(Total number of pixels in one right image), (rate of detection of phase difference known point) = (number of detected phase difference known points in right search image) / (detection of phase difference known point and phase difference unknown point in right search image) It can be expressed as the sum of the numbers).

Next, the condition setting unit 28 determines whether or not all candidate conditions for the target species have been selected (S8).

If the condition setting unit 28 determines that all candidate conditions of the target species have not been selected, the condition setting unit 28 selects the next candidate condition of the target species (S9), and then selects. The candidate condition is output (S2).

For example, among the candidate conditions Δx _{1 to} Δx ₅ of the target species, only Δx ₁ has been selected, and Δx _{2 to} Δx
If the selection for x ₅ has not been completed, one candidate condition Δ selected from the remaining candidate conditions Δx _{2 to} Δx _5.
x ₂ is selected, and the selected candidate condition Δx ₂ is output.

When the condition setting unit 28 determines that all candidate conditions of the target species have been selected, the maximum value detection unit 36 determines the maximum evaluation value from the evaluation values obtained for each candidate condition of the target species. The candidate condition that has been detected and has the maximum evaluation value is detected as the correction condition. Then, the maximum value detection unit 28 stores the correction condition of the target species in the condition setting unit 28 (S10). After that, the correction condition detection device 18 ends the process for detecting the correction condition of the target species (end).

For example, all candidate conditions Δx _{1 to} Δ of the target species
When x ₅ is selected, candidate condition Δx ₁ for the target species
The maximum evaluation value is detected from among the evaluation values obtained for each candidate condition of Δx _{5, and} the candidate condition for which the maximum evaluation value is obtained, for example, Δx
₄ is detected as the correction condition for the target species.

The correction condition detecting device 18 has a translation amount in the x-axis direction, a translation amount in the y-axis direction, a rotation angle around the x-axis, a rotation angle around the y-axis, a rotation angle around the z-axis and conversion. The above-described processing of S1 to S10 is performed for each candidate condition of each type of magnification, and the correction condition is detected for each type of candidate condition.

FIG. 5 is a diagram for explaining the detection of the correction condition. In the figure, the vertical axis shows the evaluation value of the candidate condition, and the horizontal axis shows the candidate condition relating to the parallel movement amount in the x-axis direction.

The evaluation value obtained by individually performing image conversion for each candidate condition Δx _{1 to} Δx _{5 of the} target species is obtained by using the detection rate of the phase difference determination point. Therefore, the left and right images after the image conversion are close to the state where the epipolar lines match, so that the evaluation value is large if the deviation amount of the epipolar lines is small, and the evaluation value is large if the deviation amount of the epipolar lines is large in the left and right images after image conversion. Get smaller. The detection of the correction condition utilizes such a phenomenon, and therefore, by detecting the candidate condition having the maximum evaluation value among the plurality of candidate conditions of the target species as the correction condition, it is suitable for epipolar line matching. The modified condition can be detected.

If the range of change of the candidate conditions is made too large in the plurality of candidate conditions prepared for each kind of candidate condition, it becomes difficult to detect the correction condition suitable for epipolar line alignment. Therefore, in consideration of this point, in a plurality of candidate conditions prepared for each type of candidate condition, a plurality of candidate conditions with different values stepwise with a range of change suitable for epipolar line matching, prepare.

For example, the candidate conditions Δx _{1 to} Δx ₅ relating to the parallel movement in the x-axis direction are made to differ stepwise at intervals of 1/3 pixel, and the candidate conditions Δα _{1 to} Δα ₅ relating to the rotation around the x-axis.
The stepwise varied at 1 ° intervals, varying the candidate condition ΔK ₁ ~ΔK ₅ and Δk ₁ ~Δk ₅ related to magnification conversion in steps of 1% intervals.

Next, the phase difference detection of the corresponding points and the phase difference fitting of the unknown points will be described. The corresponding point searching unit 32
In the set of corresponding candidate points obtained for each search condition, corresponding candidate points forming a common set are detected as corresponding points.
As shown in FIGS. 4A and 4B, corresponding points a and b detected in the left and right search images are pixels on the horizontal scanning line L.

The phase difference is | (x coordinate of corresponding point a in left search image)-(x coordinate of corresponding point in right search image)
When the corresponding point a is located at coordinates (5, 4) and the corresponding point b is located at coordinates (8, 4) as shown in FIG. 4 (A) and FIG. 4 (B), for example. Will detect | 5-8 | = 3 as the phase difference between the corresponding points a and b.

FIG. 6 is a diagram for explaining the phase difference fitting of unclear points. The corresponding point searching unit 32 estimates and applies the phase difference to the unidentified points (pixels not detected as corresponding points) in the left and right search images by using the detected phase difference of the corresponding points.

Various methods can be used for estimating and fitting the phase difference. Here, if there is a correspondence unknown point that is in contact with the corresponding point or near the corresponding point, the position of the corresponding unknown point is calculated. It is estimated that the phase difference is equal to the phase difference of the corresponding points, and the phase difference of the corresponding points is applied to the corresponding unknown point. Specifically, the coordinates of the corresponding points in the left search image or the right search image are (x ₁ , y ₁ ), and the coordinates of the correspondence unknown points in the left search image or the right search image are (x ₂ ,
y ₂ ), the coordinates of the correspondence unknown point (x ₂ , y ₂ )
Satisfies x ₂ = x ₁ and y ₁ −α ≦ y ₂ ≦ y ₁ + α, or x ₁ −β ≦ x ₂ ≦ x ₁ + β and y ₂
= Y ₁ is satisfied, the phase difference at the corresponding point at the coordinates (x ₁ , y ₁ ) is set as the phase difference at the unknown point at the coordinates (x ₂ , y ₂ ). α and β are natural numbers, for example α = β = 2. When there are a plurality of corresponding points around the unknown point, a plurality of phase differences are applied to the unknown point.

For example, in FIG. 6, when α = β = 2, a pixel having coordinates (x ₂ , y ₂ ) satisfying x ₂ = x ₁ and y ₁ −α ≦ y ₂ ≦ y ₁ + α, x ₁ −β ≦ x ₂ ≦ x
Pixels at coordinates (x ₂ , y ₂ ) that satisfy ₁ + β and y ₂ = y ₁ are shown with hatching. When these hatched pixels are unclear points, the phase difference of the corresponding points at the coordinates (x ₁ , y ₁ ) is applied to the unclear points. In the figure, the storage area 38a for one pixel in the third left memory 38 is represented by a rectangle of one square.

Next, the verification of the phase difference applied to the unclear point will be described. Various methods can be used as a method for testing the phase difference, but here, the testing is performed as described below.

When testing the phase difference applied to the uncertain point in the left search image, the phase difference known point in the left search image is used as the test point, and the test is performed from the position corresponding to the test point in the right search image. A pixel in the right search image at a position shifted by the phase difference fitted to the point is used as a reference point. Then, when the difference between the density of the verification point in the left search image and the density of the reference point in the right search image is less than or equal to the threshold value, it is determined that the phase difference fitted to the verification point is correct. Further, when the difference between the density of the verification point in the left search image and the density of the reference point in the right search image exceeds the threshold value, it is determined that the phase difference applied to the verification point is incorrect.

When testing the phase difference applied to the uncertain point in the right search image, the phase difference known point in the right search image is used as the test point, and the test is performed from the position corresponding to the test point in the left search image. A pixel in the left search image at a position shifted by the phase difference fitted to the point is used as a reference point. When the difference between the density of the test point in the right search image and the density of the reference point in the left search image is less than or equal to the threshold value, it is determined that the phase difference fitted to the test point is correct. When the difference between the density of the verification point in the right search image and the density of the reference point in the left search image exceeds the threshold, it is determined that the phase difference of the verification point is incorrect.

Next, with reference to FIG. 7 and FIG. 8, a description will be given of the processing for testing the phase difference fitted to the unidentified point in the left search image. FIG. 7 is a diagram showing the flow of the phase difference verification process performed by the corresponding point search unit. FIG. 8 is a diagram for explaining the positional relationship between the test point and the reference point when the phase difference test is performed on the left search image.

When the corresponding point searching unit 32 finishes fitting the phase difference to the correspondence unknown point in the left search image, it starts verification of the fitted phase difference (start), and first, the phase difference in the left search image. It is determined whether or not all of the correspondence uncertain points fitted with are completely scanned (S1).

(1-A) When it is determined in S1 that all the uncertain points to which the phase difference has been fitted have not been scanned, the corresponding point searching unit 32 applies the phase difference to the left search image. The unclear correspondence point is used as a test point, and the density d of the test point is read from the third left memory 38 (S2).

Next, the corresponding point searching unit 32 uses the pixel in the right search image at the position shifted from the position corresponding to the test point in the right search image by the phase difference L fitted to the test point as a reference point, and makes a reference. The density D at the point is read from the third right memory 40 (S3).

The coordinates (x ₀ ,
y ₀ ), the coordinates (x ₀ −L, y ₀ ) of the reference point in the right search image. For example, in the case where the coordinates (6, 6) of the test point in the left search image and the phase difference L = 2 of the test point as shown in FIG.
As shown in (B), the coordinates (4, 6) of the reference point in the right search image are obtained.

Then, the corresponding point searching unit 32 determines whether or not the difference | d−D | between the read densities d and D is equal to or less than the threshold value THL (S4).

When it is determined in S4 that the density difference | d−D | exceeds the threshold value THL, the corresponding point searching unit 32 determines that the phase difference fitted to the test point is not correct, and the test point is set to the phase difference. It is detected as an unknown point (S5), and after that, the process of S1 is performed.

When it is determined in S4 that the density difference | d−D | is less than or equal to the threshold value THL, the corresponding point searching unit 32 determines that the phase difference fitted to the test point is correct, and the test point is the phase difference. It is detected as a clear point (S6), and thereafter, the process of S1 is performed. In the phase difference test described above, the density difference | d−D
The combination of the test point and the reference point where | is equal to or less than the threshold value THL is a combination of pixels that are probabilistically likely to be corresponding points, and thus it is possible to determine that the stochastic phase difference is correct.

When a plurality of phase differences are applied to the test points, a reference point is obtained for each phase difference and the density difference |
It is determined whether d−D | becomes equal to or less than the threshold value THL. For example, two phase differences 2, with respect to the test point of coordinates (6, 6),
When 1 is applied, the reference point of coordinates (4, 6) and the reference point of coordinates (5, 6) are obtained, and the density difference | d−D | is calculated for each of these two reference points. It is determined whether or not it becomes equal to or less than the threshold value THL.

When the density difference | d−D | exceeds the threshold value THL for all the phase differences, it is determined that the phase difference fitted to the relevant test point is not correct, and the test point is detected as the phase difference unknown point. To do. When there is one or a plurality of phase differences below the threshold THL, it is determined that the phase difference fitted to the test point is correct, and the test point is detected as the phase difference known point.

Even when a plurality of phase differences are applied to the test points, it is determined for each phase difference whether or not the density difference | d−D | becomes less than or equal to the threshold value THL. Therefore, the verification point to which the stochastic phase difference is applied is
It can be detected as a phase difference determination point.

(1-B) When it is determined in S1 that all the uncertain points to which the phase difference has been applied have been scanned, the phase difference test is ended for the uncertain points in the left search image (end).

The threshold value TH used in the phase difference test described above.
L is preferably set to a value larger than the threshold value used for searching the corresponding points in the corresponding point searching unit 32. However, if the threshold value THL is made too large, the probability of probability of the phase difference being determined to be correct is lowered, so to an arbitrary suitable value within the range in which stochastic accuracy that is practically sufficient is obtained,
The threshold value THL is set.

Next, with reference to FIG. 7 and FIG. 9, the processing for testing the phase difference fitted to the uncertain point in the right search image will be described. FIG. 9 is a diagram for explaining the positional relationship between the test point and the reference point when the phase difference test is performed on the right search image.

When the corresponding point searching unit 32 finishes fitting the phase difference to the unidentified point in the right search image, it starts verification of the fitted phase difference (start), and first, the phase difference in the right search image. It is determined whether or not all of the correspondence uncertain points fitted with are completely scanned (S1).

(2-A) If it is determined in S1 that all the uncertain points to which the phase difference has been fitted have not been scanned, the corresponding point searching unit 32 applies the phase difference to the right search image. The correspondence unknown point is used as a test point, and the density d of the test point is read from the third right memory 40 (S2).

Next, the corresponding point searching unit 32 sets the pixel in the left search image at a position deviated from the position corresponding to the test point in the left search image by the phase difference L fitted to the test point as a reference point, and makes a reference. The density D of the point is read from the third left memory 38 (S3).

The coordinates (x ₀ ,
y ₀ ), the coordinates (x ₀ + L, y ₀ ) of the reference point in the left search image. For example, in the case where the pixel at the coordinates (4, 6) in FIG. 9B is the test point in the right search image and the phase difference L = 3 between the test points,
In (A), the pixel at coordinates (7, 6) becomes the reference point in the left search image.

Then, the corresponding point searching unit 32 determines whether or not the difference | d−D | between the read densities d and D is equal to or less than the threshold value THL (S4).

When it is determined in S4 that the density difference | d−D | exceeds the threshold value THL, the corresponding point searching unit 32 determines that the phase difference fitted to the verification point is not correct, and the corresponding verification point is subjected to the phase difference. It is detected as an unknown point (S5), and after that, the process of S1 is performed.

When it is determined in S4 that the density difference | d−D | is less than or equal to the threshold value THL, the corresponding point searching unit 32 determines that the phase difference fitted to the test point is correct, and the test point is the phase difference. It is detected as a clear point (S6), and thereafter, the process of S1 is performed.

(2-B) If it is determined in S1 that the entire surface has been scanned, the phase difference test is completed for the uncertain points in the right search image (end).

When the number of detected phase difference known points in the left search image is used when obtaining the detection rate of the phase difference known point, the phase difference test is performed on the left search image, and after the phase difference test is completed, the left search image is detected. The detection rate of the phase difference known points is obtained using the number of detected phase difference known points.

Further, when the number of detected phase difference known points in the right search image is used when obtaining the detection rate of the phase difference known point, the phase difference test is performed on the right search image, and after the phase difference test is completed, the right search image is detected. The detection rate of the phase difference known point is obtained by using the detected number of the phase difference known point in.

When the number of detected phase difference known points in the left search image or the right search image is not used for obtaining the detection rate of the phase difference known points, the phase difference test of the left search image or the right search image is not performed. Good.

The detection rate of the phase difference known point is obtained by using the number of detected phase difference known points for which the phase difference verification is completed, and the evaluation value of the candidate condition is obtained from the detection rate of the phase difference known point, whereby the epipolar It is possible to improve the detection accuracy of the correction condition necessary for line alignment.

Left camera 1 stored in first left memory 14
The left image from 0 and the right image from the right camera 12 stored in the first right memory 16 are rewritten at a rate of one screen every t seconds.

The correction conditions may be detected one by one from the left and right images of the memories 14 and 16 or may be detected from the left and right images of the memories 14 and 16 every plural screens. .

When the left and right images photographed by the left and right cameras 10 and 12 are still images, the correction condition is detected every plural screens for each type of correction condition to obtain a plurality of correction conditions. It is also possible to obtain an average of a plurality of correction conditions for each type of correction conditions and use the averaged correction conditions to match the epipolar lines of the left and right images captured by the left and right cameras 10 and 12. Even in the case of a still image, the noise situation changes from screen to screen, so that the accuracy of epipolar line alignment can be improved by using an average of correction conditions obtained every few screens.

<Second Embodiment of the Invention of Claim 1> FIG.
FIG. 3 is a functional block diagram provided for explaining a second embodiment of the invention of claim 1. Hereinafter, differences from the first embodiment of the invention of claim 1 will be mainly described, and detailed description of the same points as the first embodiment of the invention of claim 1 will be omitted.

The correction condition detecting device 18 of this embodiment is
Condition setting unit 28, image conversion unit 30, corresponding point search unit 32,
The evaluation value calculation unit 34 and the maximum value detection unit 36, the third left memory 38 and the third right memory 40, and the area cutout unit 4
6 is provided.

The area cutout unit 46 sequentially cuts out the left and right attention area images from a plurality of attention areas vertically dispersed on the left and right images after the image conversion, and the corresponding point search unit 32 outputs the left and right attention area images to the left and right. Corresponding candidate points, corresponding points, known phase differences, and unknown points are detected for each of the left and right search images having the same candidate condition as the search image.

The evaluation value calculation unit 34 obtains the detection rate of the phase difference determination point for each of the left and right search images having the same candidate condition, and calculates the average value of the detection rates obtained for the left and right search images having the same candidate condition as the evaluation value of the candidate condition. And

FIG. 11 is a diagram showing the operation flow of the second embodiment of the invention of claim 1. In the same figure, the flow of the operation in the case of focusing on one type out of a plurality of types of candidate conditions and correction conditions and detecting the correction conditions for the focused species using the candidate conditions is shown.

When the operator inputs an operation start signal to the correction condition detecting device 18 (start), first, the condition setting unit 28 sets one of a plurality of types of candidate conditions as a target condition candidate condition (S1), Then, the condition setting unit 28 outputs one candidate condition (selected candidate condition) selected from the plurality of candidate conditions of the target species (S2).

[0186] For example, the candidate condition Δx ₁ ~Δx ₅ involved the interest species candidate condition parallel movement in the x-axis direction was selected from among the focus species plurality of candidate condition Δx ₁ ~Δx _{5 1}
The candidate conditions Δx ₁ are output.

Next, the image conversion unit 30 performs image conversion on one of the left image and the right image in accordance with the selection candidate condition input from the condition setting unit 28, and outputs the image-converted left image and right image to the third left memory 38 and the third left memory 38. Stored in the third right memory 40 (S
3).

Here, the left image 42 is not converted into the right image 4
Assuming that the image conversion 4 is image conversion, FIG.
As shown in (A), the image conversion unit 30 uses the left camera 10
The left image 42 is read from the first left memory 14 and stored in the third left memory 38 without image conversion. Along with this, as shown in FIGS. 3B and 4B, the image conversion unit 30 reads the right image 44 from the right camera 12 from the first right memory 16 and performs the third right conversion without performing image conversion. It is stored in the memory 40. The image conversion is performed according to the candidate condition input from the condition setting unit 28. For example, when the candidate condition Δx ₁ is input, image conversion is performed in which each pixel of the right image 44 is translated in the x-axis direction by Δx ₁ , and the image-converted right image 44 is obtained.
Is stored in the third right memory 40.

Next, the area cutout unit 46 cuts out the left and right attention area images from a plurality of attention areas dispersed in the vertical direction (y direction) on the left and right images after the image conversion (S4).

FIG. 12 is a diagram for explaining the attention area and the attention area image. In FIG. 12A, the attention area 48 and the attention area image 50 related to the left image 42 stored in the third left memory 38. 12B, a region of interest 52 related to the right image 44 stored in the third right memory 40,
The attention area image 54 is shown. In the figure, attention area images 50, 5
4 is shown with hatching.

As shown in FIGS. 12A and 12B, the left image 42 and the right image 44 after image conversion are stored in the third left memory 38 and the third right memory 40.
A plurality of regions of interest 48 are vertically dispersed on the left image 42 of the third left memory 38, and a plurality of regions of interest 48 are vertically dispersed on the right image 44 of the third right memory 40. The area 52 is set.

In the left and right images 42 and 44, the number of attention areas 48 and 50 and the width in the vertical direction are the same. For example, the number of attention areas 48 and 52 is three and the width in the vertical direction is horizontal. The width is one scan line. The vertical widths of the regions of interest 48 and 52 may be the widths of a plurality of horizontal scanning lines.

Further, the attention area 52 on the right image 44 is set at the same vertical position (y coordinate position) as the plurality of attention areas 48 set on the left image 42. In the illustrated example, the regions of interest 4 are located at the positions y = 3, 6, and 9, respectively.
8 and 50 are set.

Then, the area cut-out section 46 displays the left and right images 42,
From the target areas 48, 52 corresponding to the y coordinates corresponding to 44, for example, the target areas 48, 52 of y = 3, left and right target area images 50, 54 for one line are cut out and stored.

Next, the corresponding point search unit 32 sets a plurality of types of search conditions by changing the gradation expression of the pixel to be scanned, and sets the left and right target area images as left and right search images, and for each search condition. , Corresponding candidate points in the left and right search images are detected (S5).

For example, when the left and right attention area images 50 and 54 stored in the area cutout unit 46 are made into black and white images of 256 gradations, the left and right attention area images 50 and 54 of 256 gradations are subjected to gradation conversion to obtain 32 gradations. The left and right attention area images 50 and 54 are obtained, and the first search condition is set such that the left and right attention area images 50 and 54 of 32 gradations are used as the left and right search images. This 2
The left and right attention area images 50 and 54 of 56 gradations are subjected to gradation conversion to obtain the left and right attention area images 50 and 54 of 16 gradations.
A second search condition is set such that the left and right attention area images 50 and 54 of the gradation are left and right search images.

The corresponding point searching unit 32 has a left and right line memory for storing an image of one line on the horizontal scanning line L corresponding to each other in the left and right search images, and when detecting a corresponding candidate point under the first search condition. Reads the left focus area image 50 of 256 gradations on the horizontal scanning line L from the area cutout unit 46, and reads 3
The left attention area image 50 for one line, which has been converted into 2 gradations and converted into 32 gradations, is stored in the left line memory. Along with this, the right focus area image 54 of 256 gradations on the horizontal scanning line L is read from the area cutout unit 46, converted into 32 gradations, and converted into 32 gradations. To store.

Then, the corresponding point searching unit 32 scans the left target area image 50 and the right target area image 54 for one line under the first search condition. Here, one pixel of the left target area image 50 and the right target area image 54 for one line, which has been subjected to gradation conversion, is used as a target point and the other pixel as a reference point, and a predetermined main scanning direction is set on the horizontal scanning line L. Counting sequentially to i
The control point on the horizontal scanning line L is sequentially scanned in the predetermined main scanning direction with respect to the second target point. When the scanning of the control point is completed without detecting the control point in which the difference between the pixel features is equal to or less than the threshold value for the i-th target point, the control point is scanned next for the (i + 1) th target point. When a control point in which the difference in pixel feature is less than or equal to the threshold value with respect to the i-th point of interest is detected, the i-th point of interest and the control point are detected as corresponding candidate points. The control point is scanned for the point of interest.

In this way, the left and right search images on the corresponding horizontal scanning lines L are scanned here, the left and right target area images 52 and 54, and when the difference between the pixel features becomes less than or equal to the threshold value, the next target point is scanned. Detect corresponding candidate points. When the detection of the corresponding candidate points on the horizontal scanning line L is completed over the entire left and right search images, the detection of the corresponding candidate points according to the first search condition is ended.

When detecting the corresponding candidate points under the second search condition, the left and right search images on the corresponding horizontal scanning lines L are scanned in the same manner as in the case of the first search condition, and the difference in pixel feature is found. When it is less than or equal to the threshold value, the next point of interest is scanned to detect a corresponding candidate point. When the detection of the corresponding candidate points on the horizontal scanning line L is completed over the entire left and right search images, the detection of the corresponding candidate points according to the second search condition is ended.

After detecting the corresponding candidate points on the horizontal scanning line L over the entire left and right search images for all types of search conditions, the corresponding point searching unit 32 next finds the corresponding candidate points obtained for each search condition. Corresponding candidate points that are a common set in the set are detected as corresponding points (S6).

For example, the corresponding candidate points that are a common set of the set of corresponding candidate points detected under the first search condition and the set of corresponding candidate points detected under the second search condition are detected as corresponding points.

Next, the corresponding point searching unit 32 detects the phase difference known point and the phase difference unknown point by using the corresponding point detection result, and evaluates the phase difference known point or the number of detected phase difference unknown points. The value is output to the value calculation unit 34 (S7).

The phase difference known point and the phase difference unknown point detected here are related to one region of interest 48, 52 from which the current left and right search images are read.

Further, here, the phase difference is estimated and applied to the unclear point (pixels not detected as the corresponding point) based on the phase difference of the corresponding points. An uncertain point that cannot be fitted with a phase difference is detected as a phase difference unknown point. After that, it is verified whether the phase difference applied to the unclear point is correct. Then, the unknown correspondence point of the phase difference determined to be correct by this test is set as the phase difference determination point, and the unknown correspondence point of the phase difference determined to be incorrect is detected as the phase difference unknown point.

Next, the evaluation value calculation unit 32 obtains the detection rate of the phase difference determination point using the number of detected phase difference determination points, and stores the detection rate of the phase difference determination point in the maximum value detection unit 36 ( S
8).

The detection rate of the phase difference determination point obtained here is one of the target regions 48 and 5 from which the current left and right search images are read.
It is related to 2. The detection rate of the phase difference known point in this case is, for example, (the detection rate of the phase difference known point) = (the number of detected phase difference known points in the left search image) / (the left image captured by the left camera) Total number of pixels), (Detection rate of phase difference known points) = (Number of detected phase difference known points in left search image) / (Sum of detected number of phase difference known points and phase difference unknown points in left search image), Alternatively, (the detection rate of the phase difference known point) = (the number of detected phase difference known points in the right search image) / (1 shot by the right camera
(Total number of pixels in one right image), (rate of detection of phase difference known point) = (number of detected phase difference known points in right search image) / (detection of phase difference known point and phase difference unknown point in right search image) It is represented by the sum of the numbers).

Next, the area cut-out unit 46, from all the attention areas 48, 52 to the left and right attention area images 50, 54 with respect to the left and right images 42, 44 after the image conversion currently stored in the third left and right memories 38, 40. It is determined whether or not the cutting has been completed (reading has been completed) (S9).

If the cutout of all the attention areas 48, 52 for the left and right images 42, 44 after the current image conversion is not completed, the area cutout unit 46 selects one set of the not-yet-finished cutout areas 48, 52. (S10), and then, from the selected attention areas 48 and 52 to the next left and right attention area images 50,
54 is cut out (S4).

For example, in the example of FIG. 12, if the cutting out from the target areas 48 and 52 of y = 3 is finished and the cutting out from the target areas 48 and 52 of y = 6 and 9 is not finished, the cutout is not completed. As the set of attention areas 48 and 52, the attention areas 48 and 52 with y = 6 are selected.

After cutting out all the regions of interest 48 and 52 for the left and right images 42 and 44 after the current image conversion, the evaluation value calculation unit 34 obtains the left and right images 42 and 44 after the current image conversion. Regarding the candidate condition, each attention area 48,
The average value of the detection rates obtained for every 50 is calculated, and this average value is stored in the maximum value detection unit 36 as the evaluation value of the candidate condition (S11).

For example, the left and right images 42 after the current image conversion,
If 44 is obtained according to the candidate condition Δx ₁ , then y
The average value of the detection rates obtained for the respective attention areas 48, 52 of = 3, 6, 9 is obtained as the evaluation value of the candidate condition Δx ₁ .

Next, the condition setting section 28 determines whether or not all candidate conditions for the target species have been selected (S12).

If all candidate conditions of the target species have not been selected, the condition setting section 28 selects the next candidate condition of the target species (S13), and then outputs the selected candidate conditions (S2). ).

For example, among the candidate conditions Δx _{1 to} Δx ₅ of the target species, only Δx ₁ has been selected, and Δx _{2 to} Δx
If the selection for x ₅ has not been completed, one candidate condition Δ selected from the remaining candidate conditions Δx _{2 to} Δx _5.
x ₂ is selected, and the selected candidate condition Δx ₂ is output.

When all the candidate conditions of the target species have been selected, the maximum value detection unit 36 detects the maximum evaluation value from the evaluation values obtained for each candidate condition of the target species and determines the maximum evaluation value. The candidate condition for which the evaluation value is obtained is detected as the correction condition. Then, the maximum value detection unit 28 stores the correction condition of the target species in the condition setting unit 28 (S14). After that, the correction condition detection device 18 ends the process for detecting the correction condition of the target species (end).

For example, all candidate conditions Δx _{1 to} Δ of the target species
When x ₅ is selected, candidate condition Δx ₁ for the target species
The maximum evaluation value is detected from among the evaluation values obtained for each candidate condition of Δx _{5, and} the candidate condition for which the maximum evaluation value is obtained, for example, Δx
₄ is detected as the correction condition for the target species.

The correction condition detecting device 18 has a translation amount in the x-axis direction, a translation amount in the y-axis direction, a rotation angle around the x-axis, a rotation angle around the y-axis, a rotation angle around the z-axis and conversion. The above-described processing of S1 to S14 is performed for each candidate condition of each type of magnification, and the correction condition is detected for each type of candidate condition.

In this embodiment, the left and right search images are focused area images 50, 5 which are part of the left and right images 42, 44.
Therefore, the scanning amount of the image required to detect the correction condition can be reduced, and the detection time of the correction condition can be shortened.

<Third Embodiment of Invention of Claim 1> Next, with reference to FIG. 10, a third embodiment of the invention of Claim 1 will be described. Hereinafter, differences from the second embodiment of the invention of claim 1 will be mainly described, and detailed description of the same points as the second embodiment of the invention of claim 1 will be omitted.

The correction condition detecting device 18 of this embodiment is
Condition setting unit 28, image conversion unit 30, corresponding point search unit 32,
The evaluation value calculation unit 34 and the maximum value detection unit 36, the third left memory 38 and the third right memory 40, and the area cutout unit 4
6 is provided.

The evaluation value calculation unit 34 obtains the detection rate of the phase difference determination point for each of the left and right search images having the same candidate condition, and sequentially determines the detection rates obtained for the left and right search images having the same candidate condition from the top. The detection rate of the number is selected, and the average value of the detection rates obtained by the selection is used as the evaluation value of the candidate condition.

FIG. 13 is a diagram showing the operational flow of the third embodiment of the invention of claim 1. In the same figure, the flow of the operation in the case of focusing on one type out of a plurality of types of candidate conditions and correction conditions and detecting the correction conditions for the focused species using the candidate conditions is shown.

When the operator inputs an operation start signal to the correction condition detecting device 18 (start), the condition setting section 28 first sets one of the plurality of types of candidate conditions as a candidate condition of the target species (S1), Then, the condition setting unit 28 outputs one candidate condition (selected candidate condition) selected from the plurality of candidate conditions of the target species (S2).

Next, the image conversion unit 30 performs image conversion on one of the left image and the right image in accordance with the selection candidate condition input from the condition setting unit 28, and outputs the image-converted left image and right image to the third left memory 38 and the third left memory 38. Stored in the third right memory 40 (S
3).

Next, the area cutout unit 46 cuts out the left and right attention area images from a plurality of attention areas dispersed in the vertical direction (y direction) on the left and right images after the image conversion (S4).

Next, the corresponding point searching unit 32 sets a plurality of types of search conditions by changing the gradation expression of the pixel to be scanned and sets the left and right focused area images as left and right search images, and for each search condition. , Corresponding candidate points in the left and right search images are detected (S5).

After detecting the corresponding candidate points on the horizontal scanning line L over the entire left and right search images for all types of search conditions, the corresponding point searching unit 32 next finds the corresponding candidate points obtained for each search condition. Corresponding candidate points that are a common set in the set are detected as corresponding points (S6).

Next, the corresponding point searching unit 32 detects the phase difference known point and the phase difference unknown point by using the corresponding point detection result, and evaluates the phase difference known point or the number of detected phase difference unknown points. The value is output to the value calculation unit 34 (S7).

Next, the evaluation value calculation unit 32 obtains the detection rate of the phase difference determination points using the number of detected phase difference determination points, and stores the detection rate of the phase difference determination points in the maximum value detection unit 36 ( S
8).

Next, the area cut-out unit 46, for the left and right images 42, 44 after image conversion currently stored in the third left and right memories 38, 40, reads out all the attention areas 48, 52 from the left and right attention area images 50, 54. It is determined whether or not the cutting has been completed (reading has been completed) (S9).

If the cutting out of all the attention areas 48, 52 has not been completed for the current left and right images 42, 44 after the image conversion, the area cutting unit 46 selects a set of not-yet-cut out attention areas 48, 52. (S10), and then, from the selected attention areas 48 and 52 to the next left and right attention area images 50,
54 is cut out (S4).

After cutting out all the regions of interest 48 and 52 for the current left and right images 42 and 44 after the image conversion, the evaluation value calculation section 34 obtains the left and right images 42 and 44 after the current image conversion. Regarding the candidate condition, each attention area 48,
Of the detection rates obtained for every 50, the higher detection rate is selected (S
11).

Next, the evaluation value calculation unit 34 obtains the average value of the selected upper detection rates, and uses this average value as the maximum evaluation value for the candidate conditions for obtaining the left and right images 42, 44 after the current image conversion. It stores in the value detection part 36 (S12).

Here, the left and right images 42 after the current image conversion,
Since the total number of the target areas 48 and 52 set on 44 is the same, if the total number is R, the detection rate of the total number R can be obtained from the total number R of the target areas 48 and 52. Becomes Then, among the detection rates of the total number R, the detection rate of the top r1 (r1 is a natural number such that R>r1> 1) is selected,
The average value of the detection rates of the selected top r1 is set as the evaluation value.
The lower (R-r1) detection rates of (R-r1)> 0 are not used for calculating the evaluation value.

For example, the total number R = 30 target areas 48,
If 52 is set and the detection rate of the total number R = 30 is obtained from these regions of interest, the evaluation value is obtained using the detection rates of the upper r1 = 10 to 20, and the lower (R-r1) = 20 ~
The 10 detection rates are not used for calculating the evaluation value.

Next, the condition setting section 28 determines whether or not all candidate conditions for the target species have been selected (S13).

If all candidate conditions of the target species have not been selected, the condition setting section 28 selects the next candidate condition of the target species (S13), and then outputs the selected candidate conditions (S2). ).

When all the candidate conditions of the target species have been selected, the maximum value detection unit 36 detects the maximum evaluation value from the evaluation values obtained for each candidate condition of the target species, The candidate condition for which the evaluation value is obtained is detected as the correction condition. Then, the maximum value detection unit 28 stores the correction condition of the target species in the condition setting unit 28 (S15). After that, the correction condition detection device 18 ends the process for detecting the correction condition of the target species (end).

The correction condition detecting device 18 has a translation amount in the x-axis direction, a translation amount in the y-axis direction, a rotation angle around the x-axis, a rotation angle around the y-axis, a rotation angle around the z-axis and conversion. The above-described processing of S1 to S15 is performed for each candidate condition of each kind of magnification, and the correction condition is detected for each kind of candidate condition.

Also in this embodiment, as in the second embodiment of claim 1, the left and right search images are changed to the left and right images 42, 44.
The target area images 50 and 54, which are a part of the above, and therefore, the scanning amount of the image necessary for detecting the correction condition can be reduced, so that the detection time of the correction condition can be shortened.

Further, according to this embodiment, when the evaluation value is obtained for each type of candidate condition, the evaluation value is obtained by using the average value of the detection rates of the plurality of upper ranks, and the lower detection rate is calculated as the evaluation value. Since it is not used for the calculation, the detection accuracy of the correction condition can be improved.

FIG. 14 is a diagram for explaining the improvement of the correction condition detection accuracy in this embodiment. The vertical axis of the figure shows the evaluation value of the candidate condition and the horizontal axis shows the candidate condition relating to the amount of translation in the x-axis direction. In FIG. 14A, the total obtained for the candidate condition Δx ₄ is shown. The variation range and the average value of the detection rates of all the numbers are shown for the detection rate of the number R, and in FIG. 14B, the detection of the top r1 is performed for the detection rate of the total number R obtained for the candidate condition Δx _4. The range of variation and the average value are shown.

In the case where the noise riding is different between the left and right search images of the target area due to the quantization error of the image in the left camera 10, the right camera 12, and the image conversion unit 30, the detection rate becomes extremely high in such a target area. descend.

Therefore, as also shown in FIG. 14A, when the detection rate of the total number R obtained for the candidate condition Δx ₄ includes such an extremely small detection rate, the total number If the average value of the R detection rates is used as the evaluation value of the candidate condition Δx ₄ , the evaluation value becomes low due to noise, which makes it difficult to accurately detect the correction condition.

Therefore, as shown in FIG. 14B, the detection rate of the top r1 is selected from the detection rates of the total number R, and
The average value of the detection rates of these top r1 is set as the evaluation value. Since it is highly possible that the lower detection rate not used for evaluation value calculation contains the detection rate whose value is extremely low due to noise, the upper detection rate is used in this way. By detecting the evaluation value without using the detection rate, it is possible to improve the detection accuracy of the correction condition.

<Fourth Embodiment of the Invention of Claim 1> FIG.
FIG. 6 is a functional block diagram provided for explaining a fourth embodiment of the invention of claim 1. Hereinafter, differences from the first embodiment of the invention of claim 1 will be mainly described, and detailed description of the same points as the first embodiment of the invention of claim 1 will be omitted.

The correction condition detecting device 18 of this embodiment is
Condition setting unit 28, image conversion unit 30, corresponding point search unit 32,
The evaluation value calculation unit 34 and the maximum value detection unit 36, the third left memory 38 and the third right memory 40, and the screen selection unit 56.
And

Left camera 1 stored in first left memory 14
The left image from 0 and the right image from the right camera 12 stored in the first right memory 16 are rewritten at a rate of one screen in t seconds.

In this embodiment, the correction condition is not detected for each screen, but the correction condition is detected for every several screens. The screen selection unit 56 displays the first left memory 1 for every several screens.
4. The left image and the right image are read from the first right memory 16 and input to the image conversion unit 30.

The image converting section 30 converts the left image and the right image input from the screen selecting section 56 according to the candidate conditions and stores them in the third left memory 38 and the third right memory 40.
As a result, the correction condition detection device 18 detects the correction condition for the left image and the right image read by the screen selection unit 56 for every several screens.

Further, the image conversion section 30 converts the left image and the right image input from the screen selection section 56 according to the correction conditions and stores them in the second left memory 20 and the second right memory 22. As a result, the corresponding point searching device 24 has the screen selecting unit 56.
The left image and the right image read out every few screens by
The corresponding points will be detected and the phase difference will be detected.

Normally, the left camera 10 and the right camera 12 are installed on the anti-vibration structure, and the epipolar lines of the left image and the right image taken by the left camera 10 and the right camera 12 do not come off with some vibration. There is. Therefore, in the detection of the corresponding points performed by the corresponding point searching device 24, even if the corresponding points are detected by scanning the left image and the right image with the epipolar line based on the correction condition detected for every several screens, the corresponding points are detected. The detection rate can be sufficiently improved in practical use.

The corresponding point searching unit 32 is configured by using a computer so that the processing for detecting the correction condition, for example, the processing for obtaining the evaluation value for each candidate condition can be performed in parallel, but for each several screens. By detecting the correction condition, the number of parallel processings is reduced to reduce the memory capacity required for parallel processing, or the corresponding point searching unit 32 is configured using a computer with a slower processing speed. You can As a result, the device scale of the corresponding point searching unit 32 can be reduced, or the device cost can be reduced.

<Fifth Embodiment of the Invention of Claim 1> FIG.
FIG. 9 is a functional block diagram provided for explaining a fifth embodiment of the invention of claim 1. Hereinafter, differences from the first embodiment of the invention of claim 1 will be mainly described, and detailed description of the same points as the first embodiment of the invention of claim 1 will be omitted.

The correction condition detecting device 58 of this embodiment is
A correction condition detection unit 60, a detection rate calculation unit 62, and a comparison determination unit 64 are provided.

The correction condition detecting unit 60 includes the condition setting unit 28 and the image converting unit 3 as in the first embodiment of the present invention.
0, a corresponding point searching unit 32, an evaluation value calculating unit 34, a maximum value detecting unit 36, a third left memory 38 and a third right memory 40.

The detection rate calculation unit 62 uses the corresponding point searching device 24.
Find the detection rate of corresponding points in. For example, by using the number of detected corresponding points in the corresponding point search device 24 or the number of detected unidentified points in addition to this, (corresponding point detection rate) = (detection of corresponding points in the left image of the second left memory 20) (Number) / (sum of detection numbers of corresponding points and correspondence unknown points in the left image of the second left memory 20), (detection rate of corresponding points) = (detection number of corresponding points in the left image of the second left memory 20) ) / (Sum of detected numbers of corresponding points and correspondence unknown points in the left image of the second left memory 20).

The comparison / determination unit 64 inputs the detection rate of the corresponding points from the detection rate calculation unit 62 and compares the detection rate of the corresponding points with a threshold value.

The correction condition detection unit 60 inputs the comparison result of the detection rate of corresponding points and the threshold value from the comparison determination unit 64, and if the detection rate of corresponding points is less than or equal to the threshold value, a new correction condition is detected and detected. The left image and the right image, which have undergone image conversion according to the new correction conditions, are stored in the second left memory 20 and the second left / right memory 22.

If the detection rate of the corresponding points exceeds the threshold value, the correction condition detecting section 60 does not perform the process of detecting a new correction condition, but the second image is converted into the left image and the right image which have been image-converted according to the already detected correction condition. The data is stored in the left memory 20 and the second right memory 22.

Before operating the corresponding point searching device 24 and the correction condition detecting device 58, the lens optical axes of the left camera 10 and the right camera 12 are mechanically moved and adjusted as is usually done, and these left cameras are operated. The epipolar lines of the left image and the right image captured by 10 and 12 are roughly adjusted, and then the correction condition detection device 58 of this embodiment performs image conversion based on the correction condition to obtain the left image and the right image. It is better to match the epipolar line.

According to this embodiment, since the correction condition is detected only when the detection rate of the corresponding points becomes equal to or less than the threshold value, it is possible to secure a long processing time for detecting the correction condition. The corresponding point searching unit 32 is usually configured by using a computer that can perform the processing for detecting the correction condition in parallel, but the correction is performed only when the detection rate of the corresponding points becomes equal to or less than the threshold value. By detecting the condition, the number of parallel processings is reduced to reduce the memory capacity required for parallel processing, or the corresponding point searching unit 32 is configured by using a computer with a slower processing speed. be able to. As a result, the device scale of the corresponding point searching unit 32 can be reduced, or the device cost can be reduced.

If the epipolar lines of the left image and the right image deviate from each other for some reason, the detection rate of the corresponding points in the corresponding point searching device 24 decreases. Therefore, when the epipolar lines deviate, the correction condition is detected and the epipolar line is detected. You can match the lines.

<First Embodiment of Invention of Claim 5> FIG.
FIG. 9 is a functional block diagram provided for explaining a first embodiment of the invention of claim 5; Hereinafter, differences from the first embodiment of the invention of claim 1 will be mainly described, and detailed description of the same points as the first embodiment of the invention of claim 1 will be omitted.

The correction condition detecting device 18 of this embodiment is
Condition setting unit 28, image conversion unit 30, corresponding point search unit 32,
The evaluation value calculation unit 34 and the maximum value detection unit 36, the third left memory 38, and the third right memory 40 are provided.

Further, the correction condition detecting device 1 of this embodiment
8 includes a gradation conversion unit 66 and a level cutout unit 68, a fourth left memory 70 and a fourth right memory 72.

The condition setting unit 28 sequentially sets a plurality of types of candidate conditions for detecting the correction condition, and the image conversion unit 30
Converts one of the m-gradation left image and the m-gradation right image obtained by the left camera 10 and the right camera 12 according to the candidate condition.

Here, the condition setting unit 28 stores a plurality of types of candidate conditions for detecting the correction conditions and a plurality of types of correction conditions for matching the epipolar lines. The correction condition to be stored is the correction condition detected by the maximum value detection unit 36.

The condition setting section 28 stores candidate conditions relating to parallel movement, rotation and magnification conversion of an image. Therefore, the correction condition detected by the maximum value detection unit 36 is the correction condition relating to the parallel movement, rotation, and magnification conversion of the image, and the image conversion unit 3
Image conversion performed by 0 according to the candidate condition and the correction condition is image parallel movement, rotation, and magnification conversion.

The gradation conversion unit 66 converts the left and right images after image conversion into left and right images of n gradations (m> n), and the level cutout unit 68 selects one or more of the left and right images after the gradation conversion. Plural kinds of gradation levels Here, the left and right images of one kind of gradation level of interest are cut out as left and right attention level images.

For example, m obtained by the left and right cameras 10 and 12.
= 256 gradation left and right images are obtained, the image conversion unit 30
Image-converts one of the left and right images having m = 256 gradations according to the candidate condition, and the gradation conversion unit 66 sets m = 256 after the image conversion.
The left and right images of gradation are converted into left and right images of n = 16 gradations, and the level cutout unit 68 selects one kind of gradation level from the left and right images of 16 gradations after the gradation conversion, for example, the eighth floor. The left and right images of the tone level are cut out as left and right focus level images.

The corresponding point search unit 32 sets a plurality of types of search conditions by making the scanning direction, the gradation expression of the pixel to be scanned, or the color of the pixel to be scanned different, and sets the left and right focus level images. Left and right search images.

Then, the corresponding point searching unit 32 scans the left and right search images according to the search conditions, compares the pixel features of the left search image and the pixel features of the right search image on the corresponding horizontal scanning lines, and compares the pixel feature differences. The pixels of the left and right search images in which is less than or equal to the threshold are detected as corresponding candidate points. Further, the corresponding point searching unit 32 detects, as corresponding points, corresponding candidate points forming a common set in the set of corresponding candidate points detected for each search condition.

Here, the left and right focus level images of the black and white image are the left and right search images. In addition, one pixel of the left and right search images, which is a pixel existing on the corresponding horizontal scanning line in the left and right search images, is a target point, and a pixel of the other image of the left and right search images is a control point. Then, a plurality of types of search conditions with different scanning directions, that is, a plurality of types of search conditions with different target point scanning directions and / or control point scanning directions are set.

Then, according to the search condition, on the corresponding horizontal scanning line L of the left and right search images, the control point on the horizontal scanning line L corresponding to the i-th point of interest sequentially counted in the scanning direction of the point of interest. Are sequentially scanned in the control point scanning direction. When all the control points on the horizontal scanning line L have been scanned without detecting control points whose pixel feature difference is less than or equal to the threshold value for the i-th point of interest, then the i + 1-th point of interest is checked. Do a point scan. When a control point in which the difference in pixel feature is less than or equal to the threshold value with respect to the i-th point of interest is detected, the i-th point of interest and the control point are detected as corresponding candidate points. The control point is scanned for the point of interest.

When the corresponding candidate points are detected for each search condition in this way, the corresponding candidate points forming a common set in the set of corresponding candidate points detected for each search condition are detected as corresponding points.

For example, a first search condition in which the scanning direction of the point of interest is the right direction (direction in which the x coordinate gradually increases) and the scanning direction of the control point is the right direction, and the scanning direction of the target point is the right direction and the control point is A second search condition that sets the scanning direction to the left (direction in which the y coordinate gradually decreases) is set. Then, the left and right search images are scanned according to the first search condition to detect corresponding candidate points, and the left and right search images are further scanned according to the second search condition to detect corresponding candidate points, which are detected according to the first search condition. Corresponding candidate points that are a common set in the set of corresponding candidate points and the set of corresponding candidate points detected by the second search condition are detected as corresponding points.

Next, the corresponding point searching unit 32 detects the identified point and the unknown point of the phase difference in the left and right search images by using the corresponding point detection result. Here, the phase difference is estimated and applied to the unclear point (pixels not detected as the corresponding point) based on the phase difference of the corresponding points. An uncertain point that cannot be fitted with a phase difference is detected as a phase difference unknown point. After that, regarding the phase difference applied to the correspondence unknown point,
Test whether the phase difference is correct. Then, the unknown correspondence point of the phase difference determined to be correct by this test is detected as the unknown phase difference point, and the unknown correspondence point of the phase difference determined to be incorrect is detected as the unknown phase difference point.

The evaluation value producing section 34 obtains the evaluation value of the candidate condition for each kind of candidate condition by using the detection rate of the phase difference known point in the left and right search images. Here, the detection rate of the phase difference determination point in the left and right search images is obtained for each candidate condition, and the detection rate is set as the evaluation value of the candidate condition.

The maximum value detection unit 36 detects, as a correction condition, the candidate condition that has the maximum evaluation value for each type of candidate condition.
Here, the maximum value detection unit 36 stores the detected correction condition in the condition setting unit 28.

FIG. 18 is a diagram showing a flow of operations of the first embodiment of the invention of claim 1. In the same figure, the flow of the operation in the case of focusing on one type out of a plurality of types of candidate conditions and correction conditions and detecting the correction conditions for the focused species using the candidate conditions is shown.

When the operator inputs an operation start signal to the correction condition detection device 18 (start), the condition setting unit 28 first sets one of a plurality of types of candidate conditions as a target condition candidate condition (S1), Then, the condition setting unit 28 outputs one candidate condition (selected candidate condition) selected from the plurality of candidate conditions of the target species (S2).

Next, the image conversion section 30 includes the left and right cameras 10,
One of the left and right images of m gradations photographed by 12 is subjected to image conversion according to the selection candidate condition input from the condition setting unit 28, and the left and right images after the image conversion are processed into the third left and right memories 38, 40.
(S3). The left and right images taken by the left and right cameras 10 and 12 are stored in the first left and right memories 14 and 16, and the image conversion unit 30 reads the left and right images of the first left and right memories 14 and 16, and one of the read left and right images is an image. The left and right images after the image conversion are stored in the third left and right memories 38 and 40 without converting the other image.

Next, the gradation converting unit 66 reads the left and right images after image conversion from the third left and right memories 38 and 40 and converts them into left and right images of n gradations (m> n), and the left and right images after gradation conversion. Is stored in the fourth left and right memories 70 and 72 (S4). For example, m = 256 gradations and n = 16 gradations.

Next, the level cut-out unit 68 scans the left and right images after gradation conversion stored in the fourth left and right memories 70 and 72, detects a pixel of one kind of gradation level of interest, and The left and right images composed of pixels of the target gradation level are cut out as left and right target level images (S5). For example, n =
The eighth gradation level of 16 gradations is set as the focused gradation level.

Next, the corresponding point searching unit 32 sets a plurality of types of search conditions by changing the scanning directions, sets the left and right focus level images as left and right search images, and sets the correspondence candidates in the left and right search images for each search condition. A point is detected (S6). Here, the left and right focus level images corresponding to the entire surface of the left and right images after gradation conversion are set as the left and right search images.

For example, it is assumed that the left and right images photographed by the left and right cameras 10 and 12 are black and white images, so that a black and white image can be obtained as a left and right search image. Furthermore, the pixel of the left search image is the point of interest, and the pixel of the right search image is the control point,
Set a first search condition in which the point-of-interest scanning direction is the right direction and a control point scanning direction is the right direction, and a second search condition in which the point-of-interest scanning direction is the right direction and the control point scanning direction is the left direction. .

Then, according to the first search condition, on the horizontal scanning line L corresponding to the left and right search images, the i-th point of interest sequentially counted in the scanning direction of the point of interest is detected on the horizontal scanning line L corresponding thereto. The control points are sequentially scanned in the control point scanning direction. When all the control points on the horizontal scanning line L have been scanned without detecting the control points where the difference in pixel characteristics, for example, the pixel density, is less than or equal to the threshold value with respect to the i-th point of interest,
The control point is scanned for the + 1st point of interest. When a control point in which the difference in pixel feature is less than or equal to the threshold value with respect to the i-th point of interest is detected, the i-th point of interest and the control point are detected as corresponding candidate points. The control point is scanned for the point of interest. Then, when the scanning of the target point and the control point on the horizontal scanning line L is completed over the entire left and right search images, the detection of the corresponding candidate points under the first search condition is completed. Similarly, the left and right search images are scanned under the second search condition, and the detection of the corresponding candidate points under the second search condition is completed.

When the detection of the corresponding candidate points is completed for all types of search conditions, the corresponding point searching unit 32 then finds the corresponding candidate points that are a common set in the set of corresponding candidate points obtained for each search condition. It is detected as a point (S7).

As described above, the left and right search images on the corresponding horizontal scanning lines L are scanned, and when the difference in pixel feature becomes less than or equal to the threshold value, the next point of interest is scanned to detect the corresponding candidate point. By detecting corresponding candidate points that are a common set in the set of corresponding candidate points obtained for each search condition as corresponding points, it is possible to detect corresponding points that are probabilistically probable.

Next, the corresponding point searching unit 32 detects the phase difference known point and the phase difference unknown point using the corresponding point detection result, and evaluates the phase difference known point or the number of detected phase difference unknown points. The value is output to the value calculator 34 (S8).

The phase difference known point and the phase difference unknown point detected here relate to the entire left and right search images.

Next, the evaluation value calculation unit 32 obtains the detection rate of the phase difference known points by using the number of detected phase difference known points, and (S
The detection rate of the phase difference determination point is stored in the maximum value detection unit 36 as the evaluation value of the selection candidate condition output by the condition setting unit 28 in 2) (S9).

The detection rate of the phase difference determination point obtained here relates to the entire left and right images which are the current left and right search images.

Next, the condition setting section 28 determines whether or not all candidate conditions for the target species have been selected (S10).

When the condition setting section 28 determines that all candidate conditions of the target species have not been selected, the condition setting section 28 selects the next candidate condition of the target species (S11), and then selects. The candidate condition is output (S2).

When the condition setting unit 28 determines that all candidate conditions of the target species have been selected, the maximum value detection unit 36 determines the maximum evaluation value from the evaluation values obtained for each candidate condition of the target species. The candidate condition that has been detected and has the maximum evaluation value is detected as the correction condition. Then, the maximum value detection unit 28 stores the correction condition of the target species in the condition setting unit 28 (S12). After that, the correction condition detection device 18 ends the process for detecting the correction condition of the target species (end).

According to this embodiment, the left and right cameras 10,
The left and right images obtained by photographing with 12 are gradation-converted, and the left and right attention level images consisting of only the pixels of the gradation level of attention are cut out from the left and right images after gradation conversion, and the left and right attention level images are used as left and right search images. Is detected, it is possible to shorten the time required to detect the correction condition used for epipolar line alignment.

In the above description of this embodiment, the operation flow of the gradation conversion unit 66, the level cutout unit 68, and the corresponding point search unit 32 when a plurality of types of search conditions are set by changing the scanning direction. However, it is also possible to set a plurality of types of search conditions by changing the gradation expression of the pixel to be scanned or the color of the pixel to be scanned.

When the gradation expression of the pixel to be scanned is made different, for example, the following may be done. That is, the left and right images are black and white images, the gradation conversion unit 66 converts the left and right images of m gradations after the image conversion into left and right images of n ₁ (m> n ₁ ) gradations, and the level cutout unit 68 converts the gradations. Left and right attention level images are cut out from the subsequent left and right images of n ₁ gradations, and the corresponding point searching unit 32 detects corresponding candidate points for the left and right attention level images. After the detection of the corresponding candidate points for the left and right focused level images of n ₁ gray levels is completed, the gray level conversion unit 66 then converts the m gray level left and right images after image conversion into n ₂ gray level left and right images. The cutout portion 68 is n ₂ (m>
The left and right focused level images are cut out from the left and right images of n ₂ ) gradation, and the corresponding point searching unit 32 detects the corresponding candidate points for the left and right focused level images. At this time, by setting n ₁ = n ₂ , the first and last gray scale widths of the left and right images of n ₁ gray scale are made different from the first and last gray scale widths of the left and right images of n ₂ gray scale,
Different gradation expression. Alternatively, the gradation expression is made different by setting n ₁ ≠ n ₂ . In this way, a plurality of types of search conditions are set to detect the corresponding candidate points for the left and right focus level images cut out from the left and right images of n ₁ gradation and n ₂ gradation in which the gradation expression of the pixel to be scanned is different. Then, the corresponding candidate points may be detected, and the corresponding candidate points forming a common set in the set of corresponding candidate points detected for each search condition may be detected as the corresponding points.

When the color of the pixel to be scanned is made different, the following may be done, for example. That is, with the left and right images as color images, the gradation conversion unit 66 uses m after image conversion.
The left and right images of gradation are converted into left and right images of n (m> n) gradations, and the level cutout unit 68 is composed of only the red pixels of the focused gradation level from the left and right images of n gradations after gradation conversion. The left and right attention level images are cut out, and the corresponding point searching unit 32 detects the corresponding candidate points for the left and right attention level images of the red pixels. When the detection of the corresponding candidate points is completed for the left and right focused level images of the red pixels, the level cut-out unit 68 next examines the left and right focused images of the green level of the focused gray level from the left and right images of n gray levels after the gray level conversion. The level image is cut out, and the corresponding point searching unit 32 detects the corresponding candidate points for the left and right focused level images of the green pixel. When the detection of the corresponding candidate points is completed for the left and right focused level images of the green pixels, the level cutout unit 68 next examines the left and right images of the n gradations after the gradation conversion and focuses the left and right focused pixels including only the blue pixels of the focused gradation level. The level image is cut out, and the corresponding point searching unit 32 detects corresponding candidate points for the right and left focused level images of the blue pixel. In this way, a plurality of types of search conditions are set to detect corresponding candidate points for left and right focus level images in which the color of the pixel to be scanned is different, and the corresponding candidate points are detected and detected for each search condition. Corresponding candidate points that are a common set in the set of corresponding candidate points may be detected as corresponding points.

<Second Embodiment of Invention of Claim 5> Next, a second embodiment of the invention of claim 5 will be described with reference to FIG. Hereinafter, differences from the first embodiment of the invention of claim 5 will be mainly described, and detailed description of the same points as the first embodiment of the invention of claim 5 will be omitted.

The correction condition detecting device 18 of this embodiment is
Condition setting unit 28, image conversion unit 30, corresponding point search unit 32,
The evaluation value calculation unit 34 and the maximum value detection unit 36, the third left memory 38, and the third right memory 40 are provided.

Further, the correction condition detecting device 1 of this embodiment
8 includes a gradation conversion unit 66 and a level cutout unit 68, a fourth left memory 70 and a fourth right memory 72.

In this embodiment, the level cutout unit 68 sequentially cuts out the left and right images of a plurality of different gradation levels of interest from the left and right images after gradation conversion as the left and right attention level images, and the evaluation value calculation unit sets the candidate condition as a candidate condition. The detection rate of the phase difference determination point is obtained for each of the same left and right search images, and the average value of the detection rates obtained for the left and right search images having the same candidate condition is set as the evaluation value of the candidate condition.

FIG. 19 is a diagram showing the operation flow of the second embodiment of the invention of claim 5. In the same figure, the flow of the operation in the case of focusing on one type out of a plurality of types of candidate conditions and correction conditions and detecting the correction conditions for the focused species using the candidate conditions is shown.

When the operator inputs an operation start signal to the correction condition detecting device 18 (start), the condition setting unit 28 first sets one of the plurality of types of candidate conditions as a target condition candidate condition (S1), Then, the condition setting unit 28 outputs one candidate condition (selected candidate condition) selected from the plurality of candidate conditions of the target species (S2).

Next, the image conversion section 30 includes the left and right cameras 10,
One of the left and right images of m gradations photographed by 12 is subjected to image conversion according to the selection candidate condition input from the condition setting unit 28, and the left and right images after the image conversion are processed into the third left and right memories 38, 40.
(S3). The left and right images taken by the left and right cameras 10 and 12 are stored in the first left and right memories 14 and 16, and the image conversion unit 30 reads the left and right images of the first left and right memories 14 and 16, and one of the read left and right images is an image. The left and right images after the image conversion are stored in the third left and right memories 38 and 40 without converting the other image.

Next, the gradation converting unit 66 reads the left and right images after image conversion from the third left and right memories 38 and 40 and converts them into left and right images of n gradations (m> n), and the left and right images after gradation conversion. Is stored in the fourth left and right memories 70 and 72 (S4).

Next, the level cutout unit 68 sequentially cuts out the left and right images of the plurality of types of gradation levels of interest from the left and right images after the gradation conversion as left and right focused level images. The level cutout unit 68 scans the gradation-converted left and right images stored in the fourth left and right memories 70 and 72 to detect pixels of one kind of gradation level among a plurality of kinds of focused gradation levels. Then, the left and right images consisting of pixels of this gradation level of interest are
It is cut out as left and right focus level images (S5). For example, n
= 5th, 8th, and 11th gradation levels out of 16 gradations, the left and right images composed of the pixels of the 5th gradation level among these focused gradation levels Cut out as a level image of interest.

Next, the corresponding point search unit 32 sets a plurality of types of search conditions by changing the scanning directions, sets the left and right focus level images as left and right search images, and sets the correspondence candidates in the left and right search images for each search condition. A point is detected (S6). Here, the left and right focus level images corresponding to the entire surface of the left and right images after gradation conversion are set as the left and right search images.

When the detection of the corresponding candidate points is completed for all types of search conditions, the corresponding point searching unit 32 then sets the corresponding candidate points that are a common set in the set of corresponding candidate points obtained for each search condition. It is detected as a point (S7).

Next, the corresponding point searching unit 32 detects the phase difference known point and the phase difference unknown point by using the corresponding point detection result, and evaluates the phase difference known point or the number of detected phase difference unknown points. The value is output to the value calculator 34 (S8).

The phase difference known point and the phase difference unknown point detected here relate to the entire left and right search images.

Next, the evaluation value calculation unit 32 obtains the detection rate of the phase difference known points by using the number of detected phase difference known points, and (S
The detection rate of the phase difference determination point is stored in the maximum value detection unit 36 as the evaluation value of the selection candidate condition output by the condition setting unit 28 in 2) (S9).

The detection rate of the phase difference determination point obtained here relates to the entire left and right images which are the current left and right search images.

Next, the level cutout unit 68 determines whether or not the cutout of the left and right focus level images has been completed for all the gradation levels of the focus (S10).

If not finished for all the gradation levels of interest, the level cutout unit 68 selects the remaining gradation levels of the interest type (S11), and the left and right focus level images for the selected gradation levels are selected. Cut out (S
5). For example, if the 5th, 8th, and 11th gradation levels among n = 16 gradations are focused and the cutout of the left and right focused level images is completed only for the 5th gradation level, The eighth gradation level is selected as the next gradation level of interest, and the left and right focus level images are cut out for the eighth selected gradation level of interest.

If the processing has been completed for all the gradation levels of interest, then the evaluation value calculation unit 34 obtains the average value of the phase difference determination point detection rates obtained for each of the gradation levels of interest. The average value is stored in the maximum value detection unit 36 as the evaluation value of the candidate condition output by the condition selection unit 28 in (S2) (S1).
2). For example, paying attention to the 5th, 8th, and 11th gradation levels of n = 16 gradations, these 3
If the cutout of the left and right focus level images has been completed for all the target grayscale levels of each type, the average value of the phase difference determination point detection rates obtained for each of these three types of target grayscale levels will be obtained.

Next, the condition setting section 28 determines whether or not all candidate conditions for the target species have been selected (S13).

If all candidate conditions of the target species have not been selected, the condition setting unit 28 selects the next candidate condition of the target species (S14), and then outputs the selected candidate conditions (S2). ).

If all candidate conditions of the target species have been selected, the maximum value detecting unit 36 detects the maximum evaluation value from the evaluation values obtained for each candidate condition of the target species, The candidate condition that has obtained the evaluation value of is detected as the correction condition. Then, the maximum value detection unit 28 stores the correction condition of the target species in the condition setting unit 28 (S15). After that, the correction condition detection device 18 ends the process for detecting the correction condition of the target species (end).

<Third Embodiment of the Invention of Claim 5> Next, with reference to FIG. 17, a third embodiment of the invention of claim 5 will be described. Hereinafter, differences from the first embodiment of the invention of claim 5 will be mainly described, and detailed description of the same points as the first embodiment of the invention of claim 5 will be omitted.

The correction condition detecting device 18 of this embodiment is
Condition setting unit 28, image conversion unit 30, corresponding point search unit 32,
The evaluation value calculation unit 34 and the maximum value detection unit 36, the third left memory 38, and the third right memory 40 are provided.

Further, the correction condition detecting apparatus 1 of this embodiment
8 includes a gradation conversion unit 66 and a level cutout unit 68, a fourth left memory 70 and a fourth right memory 72.

In this embodiment, the level cutout unit 68 sequentially cuts out left and right images of a plurality of types of gradation levels of interest from the left and right images after gradation conversion as left and right attention level images, and the evaluation value calculation unit 34 selects candidates. The detection rate of the phase difference determination point is obtained for each of the left and right search images with the same condition, and a predetermined number of detection rates are sequentially selected from the higher order among the detection rates obtained for the left and right search images with the same candidate condition, and by the selection, The average value of the obtained detection rates is used as the evaluation value of the candidate condition.

FIG. 20 is a diagram showing an operational flow of the third embodiment of the invention of claim 5. In the same figure, the flow of the operation in the case of focusing on one type out of a plurality of types of candidate conditions and correction conditions and detecting the correction conditions for the focused species using the candidate conditions is shown.

When the operator inputs an operation start signal to the correction condition detecting device 18 (start), the condition setting unit 28 first sets one of the plurality of types of candidate conditions as the target condition candidate condition (S1), Then, the condition setting unit 28 outputs one candidate condition (selected candidate condition) selected from the plurality of candidate conditions of the target species (S2).

Next, the image conversion section 30 includes the left and right cameras 10,
One of the left and right images of m gradations photographed by 12 is subjected to image conversion according to the selection candidate condition input from the condition setting unit 28, and the left and right images after the image conversion are processed into the third left and right memories 38, 40.
(S3). The left and right images taken by the left and right cameras 10 and 12 are stored in the first left and right memories 14 and 16, and the image conversion unit 30 reads the left and right images of the first left and right memories 14 and 16, and one of the read left and right images is an image. The left and right images after the image conversion are stored in the third left and right memories 38 and 40 without converting the other image.

Next, the gradation conversion unit 66 reads the left and right images after image conversion from the third left and right memories 38 and 40 and converts them into left and right images of n gradations (m> n), and the left and right images after gradation conversion. Is stored in the fourth left and right memories 70 and 72 (S4).

Next, the level cutout unit 68 sequentially cuts out the left and right images of the plurality of types of gradation levels of interest from the left and right images after the gradation conversion as left and right focus level images. The level cutout unit 68 scans the gradation-converted left and right images stored in the fourth left and right memories 70 and 72 to detect pixels of one kind of gradation level among a plurality of kinds of focused gradation levels. Then, the left and right images consisting of pixels of this gradation level of interest are
It is cut out as left and right focus level images (S5). For example, n
= 5th, 8th, and 11th gradation levels out of 16 gradations, the left and right images composed of the pixels of the 5th gradation level among these focused gradation levels Cut out as a level image of interest.

Next, the corresponding point searching unit 32 sets a plurality of types of search conditions by changing the scanning directions, sets the left and right focus level images as left and right search images, and sets the correspondence candidates in the left and right search images for each search condition. A point is detected (S6). Here, the left and right focus level images corresponding to the entire surface of the left and right images after gradation conversion are set as the left and right search images.

After detecting the corresponding candidate points for all types of search conditions, the corresponding point searching unit 32 then finds a corresponding candidate point that is a common set in the set of corresponding candidate points obtained for each search condition. It is detected as a point (S7).

Next, the corresponding point searching unit 32 detects the phase difference known point and the phase difference unknown point using the corresponding point detection result, and evaluates the phase difference known point or the number of detected phase difference unknown points. The value is output to the value calculator 34 (S8).

The phase difference known point and the phase difference unknown point detected here relate to the entire left and right search images.

Next, the evaluation value calculation unit 32 obtains the detection rate of the phase difference known points by using the number of detected phase difference known points, and (S
The detection rate of the phase difference determination point is stored in the maximum value detection unit 36 as the evaluation value of the selection candidate condition output by the condition setting unit 28 in 2) (S9).

The detection rate of the phase difference determination point obtained here relates to the entire left and right images which are the current left and right search images.

Next, the level cutout unit 68 determines whether or not the cutout of the left and right attention level images has been completed for all the gradation levels of interest (S10).

If not finished for all the gradation levels of interest, the level cutout unit 68 selects the remaining gradation levels of the interest type (S11), and the left and right focus level images for the selected gradation levels of interest. Cut out (S
5). For example, if the 5th, 8th, and 11th gradation levels among n = 16 gradations are focused and the cutout of the left and right focused level images is completed only for the 5th gradation level, The eighth gradation level is selected as the next gradation level of interest, and the left and right focus level images are cut out for the eighth selected gradation level of interest.

If the processing has been completed for all the gradation levels of interest, the evaluation value calculation unit 34 then determines the detection rate of the phase difference determination point obtained for each gradation level of interest for the candidate condition. Among them, the higher detection rate is selected (S12).

Next, the evaluation value calculation unit 34 obtains the average value of the selected upper detection rates, and stores this average value in the maximum value detection unit 36 as the evaluation value for the candidate condition (S1).
3).

Here, assuming that R gradation levels of different types are set as the gradation levels of interest for one candidate condition, the detection rate of the phase difference determination point is determined for each of these gradation levels of interest. Therefore, the detection rate of the total number R is obtained. Then, the detection rate of the top r1 detection rates (r1 is a natural number such that R>r1> 1) is selected from the detection rates of the total number R, and the average value of the selected detection rates of the top r1 is set as the evaluation value. To do. Lower (R-r1)> 0 (R-r
1) The detection rates are not used for calculating the evaluation value.

Next, the condition setting section 28 determines whether or not all candidate conditions for the target species have been selected (S14).

If all candidate conditions of the target species have not been selected, the condition setting section 28 selects the next candidate condition of the target species (S15), and then outputs the selected candidate conditions (S2). ).

If all candidate conditions for the target species have been selected, the maximum value detection unit 36 detects the maximum evaluation value from the evaluation values obtained for each candidate condition for the target species, The candidate condition that has obtained the evaluation value of is detected as the correction condition. Then, the maximum value detection unit 28 stores the correction condition of the target species in the condition setting unit 28 (S16). After that, the correction condition detection device 18 ends the process for detecting the correction condition of the target species (end).

<First Embodiment of the Invention of Claim 9> FIG.
FIG. 10 is a functional block diagram provided for explaining a first embodiment of the invention of claim 9; Hereinafter, differences from the first embodiment of the invention of claim 1 will be described, and detailed description of the same points as the first embodiment of the invention of claim 1 will be omitted.

The correction condition detecting device 18 of this embodiment is
Condition setting unit 28, image conversion unit 30, corresponding point search unit 32,
The evaluation value calculation unit 34 and the maximum value detection unit 36, the third left memory 38 and the third right memory 40, and the phase difference range input unit 74 are further provided.

The condition setting unit 28 sequentially sets the candidate conditions for detecting the height direction correction condition, and the image conversion unit 30 sets one of the left image and the right image obtained by the left camera 10 and the right camera 12, Image conversion is performed according to the candidate conditions.

Here, other types of correction conditions including the height direction correction conditions are also detected, and the condition setting unit 28 selects a plurality of types of candidate conditions for detecting the correction conditions and the epipolar line. A plurality of types of correction conditions for matching are stored. The correction condition to be stored is the correction condition detected by the maximum value detection unit 36. One of the plurality of types of candidate conditions and correction conditions is a height direction candidate condition and a correction condition.

The condition setting section 28 stores candidate conditions relating to parallel movement, rotation and magnification conversion of an image. Therefore, the correction condition detected by the maximum value detection unit 36 is the correction condition relating to the parallel movement, rotation, and magnification conversion of the image, and the image conversion unit 3
Image conversion performed by 0 according to the candidate condition and the correction condition is image parallel movement, rotation, and magnification conversion. Known methods can be used for these parallel movement, rotation, and magnification conversion.

The corresponding point searching unit 32 sets a plurality of types of search conditions by making the scanning direction, the gradation expression of the pixel to be scanned or the color of the pixel to be scanned different, and setting the left and right after image conversion. Let the images be left and right search images. Here, the left and right images of the color image are obtained by the left and right cameras 10 and 12, and the color of the pixel to be scanned is made different to set a plurality of types of search conditions. Further, the left and right images after the image conversion are used as the left and right search images over the entire surface.

Then, the corresponding point searching unit 32 scans the left and right search images according to the search condition, compares the pixel features of the left search image and the pixel features of the right search image on the corresponding horizontal scanning lines, and compares the pixel feature differences. The pixels of the left and right search images in which is less than or equal to the threshold are detected as corresponding candidate points. Further, the corresponding point searching unit 32 detects, as corresponding points, corresponding candidate points forming a common set in the set of corresponding candidate points detected for each search condition. Here, the pixel feature is the pixel density.

For example, a color image is stored in the third left memory 38 and the third right memory 40 as left and right images after image conversion, and the red image of these left and right images after image conversion is the first left and right search image. The search condition and the second search condition in which the blue image of the left and right images after the image conversion is used as the left and right search images are set. Then, the left and right search images of the red image are scanned under the first search condition, and the pixels of the left and right search image where the pixel feature difference on the corresponding horizontal scanning lines is less than or equal to the threshold value To detect as.
Furthermore, the left and right search images of the blue image are scanned under the second search condition, and the pixels of the left and right search images for which the difference in pixel feature on the corresponding horizontal scanning lines is less than or equal to the threshold are identified as the corresponding candidate points according to the second search condition. To detect as. Then, the corresponding candidate points that are a common set in the set of corresponding candidate points detected under the first search condition and the set of corresponding candidate points detected under the second search condition are detected as corresponding points.

Next, the corresponding point search unit 32 detects the identified point and the unknown point of the short-distance phase difference in which the shift amount in the height direction is large in the left and right search images using the corresponding point detection result. The operator uses the phase difference range input unit 74 to set the range of the short-distance phase difference in which the shift amount in the height direction is large, through the corresponding point search unit 3
Enter 2 For example, the range where the phase difference is 80 or more,
It is input as the range of the short-distance phase difference in which the amount of deviation in the height direction is large.

Here, the phase difference is estimated and applied to the unclear point (pixels not detected as the corresponding point) based on the phase difference of the corresponding points. An uncertain point that cannot be fitted with a phase difference is detected as a phase difference unknown point. After that, regarding the phase difference applied to the correspondence unknown point,
Test whether the phase difference is correct. Then, the unknown correspondence point of the phase difference determined to be correct by this test is detected as the unknown phase difference point, and the unknown correspondence point of the phase difference determined to be incorrect is detected as the unknown phase difference point.

Since a plurality of types of search conditions are set by making the color of the pixel to be scanned different, the fitting of the phase difference and the verification of the phase difference are performed for each color of the pixel to be scanned. Pixels for which the phase difference is determined to be correct for all the colors of the pixel to be scanned, and the phase difference is determined to be incorrect for any of the colors of the pixel to be scanned. Is the point of unknown phase difference.

After that, among the phase difference known points, the close distance phase difference known point where the amount of deviation in the height direction becomes large is detected.

The evaluation value calculation unit 34 obtains the detection rate of the phase difference determination point for the short-distance phase difference in which the displacement amount in the height direction in the left and right search images is large, and uses the detection rate to detect the height direction candidate. Calculate the evaluation value of the condition. Here, the detection rate of the phase difference known point is calculated for each height direction candidate condition for the short-distance phase difference in which the amount of deviation in the height direction in the left and right search images is large, and the detection rate is the evaluation value of the height direction candidate condition. And

The maximum value detecting unit 36 detects the height direction candidate condition that has obtained the maximum evaluation value as the height direction correction condition.
Here, the maximum value detection unit 36 stores the detected height direction correction condition in the condition setting unit 28.

FIG. 22 is a diagram showing the operation flow of the second embodiment of the invention of claim 9. In the figure, the flow of the operation in the case of detecting the height direction correction condition using the height direction candidate condition by paying attention to the height direction candidate condition and the correction condition among a plurality of types of candidate conditions and correction conditions Indicates.

When the operator inputs an operation start signal to the correction condition detecting device 18 (start), the condition setting section 28 first selects and sets a candidate condition in the height direction from a plurality of kinds of candidate conditions (S1). After that, the condition setting unit 28 outputs one height direction candidate condition (selected height direction candidate condition) selected from the plurality of height direction candidate conditions (S).
2). The height direction candidate conditions are candidate conditions relating to the parallel movement in the y-axis direction, and for example, five height direction candidate conditions Δ
outputs one height direction candidate condition [Delta] y ₁ selected from among y ₁ ~Δy _5.

Next, the image conversion section 30 includes the left and right cameras 10,
One of the left and right images captured by the condition setting unit 28
The image is converted in accordance with the selected height direction candidate condition input from, and the left and right images after the image conversion are converted into the third left and right memories 38, 40.
(S3). The left and right images captured by the left and right cameras 10 and 12 are stored in the first left and right memories 14 and 16, and the image conversion unit 30 reads the left and right images of the first left and right memories 14 and 16, and one of the read left and right images is The left and right images after the image conversion are stored in the third left and right memories 38 and 40 without image conversion for the other.

Next, the corresponding point searching unit 32 sets a plurality of types of search conditions by making the color of the pixel to be scanned different and sets the entire left and right images after image conversion as the left and right search images. Then, the corresponding candidate points in the left and right search images are detected (S4). Here, since different types of search conditions are set by making the colors of the pixels different, the left and right images photographed by the left and right cameras 10 and 12 are color images.

For example, of the left and right images of the color images after image conversion stored in the third left and right memories 38 and 40, the first search condition for scanning the red left and right images as the left and right search images,
A second search condition for scanning the blue left and right images as the left and right search images is set. Then, the corresponding point search unit 32 scans the red left and right images under the first search condition, cuts out one line of the red left and right images on the corresponding horizontal scanning line L from the third left and right memories 38, 40, and responds. The data is stored in the left and right line memories included in the point search unit 32. Here, in the red left and right images, one pixel is a target point and the other pixel is a reference point, and the i-th target is sequentially counted in a predetermined main scanning direction, for example, the right direction on the corresponding horizontal scanning line L. With respect to the points, the control points on the horizontal scanning line L are sequentially scanned in a predetermined main scanning direction, for example, the right direction. When the scanning of the control point is completed without detecting the control point in which the difference in pixel characteristics, for example, the pixel density, is less than or equal to the threshold value with respect to the i-th point of interest,
The control point is scanned for the + 1st point of interest. When a control point in which the difference in pixel feature is less than or equal to the threshold value with respect to the i-th point of interest is detected, the i-th point of interest and the control point are detected as corresponding candidate points. The control point is scanned for the point of interest. Third left and right memory 38,
When the detection of the corresponding candidate points by the first search condition is completed for the entire left and right images of 40, then the blue left and right images are scanned by the second search condition in the same manner as the first search condition.
Corresponding candidate points are detected by the second search condition.

After detecting the corresponding candidate points for all types of search conditions, the corresponding point searching unit 32 then finds a corresponding candidate point that is a common set in the set of corresponding candidate points obtained for each search condition. It is detected as a point (S5).

As described above, the left and right search images on the corresponding horizontal scanning lines L are scanned, and when the difference in pixel feature becomes less than or equal to the threshold value, the next point of interest is scanned to detect the corresponding candidate point. Corresponding candidate points that are a common set in the set of corresponding candidate points obtained for each search condition are a set of corresponding candidate points that are probabilistically probable.

Next, the corresponding point searching unit 32 detects the phase difference known point and the phase difference unknown point in the short distance phase difference range by using the corresponding point detection result (S6).

The phase difference known point and the phase difference unknown point detected here are for the entire left and right images which are the current left and right search images.

Further, here, the phase difference is estimated and applied to the unclear point (pixels not detected as the corresponding point) based on the phase difference of the corresponding points. An uncertain point that cannot be fitted with a phase difference is detected as a phase difference unknown point. After that, it is verified whether the phase difference applied to the unclear point is correct. Then, the unknown correspondence point of the phase difference determined to be correct by this test is set as the phase difference determination point, and the unknown correspondence point of the phase difference determined to be incorrect is detected as the phase difference unknown point.

Since a plurality of types of search conditions are set by making the color of the pixel to be scanned different, the fitting of the phase difference and the verification of the phase difference are performed for each color of the pixel to be scanned. Pixels for which the phase difference is determined to be correct for all the colors of the pixel to be scanned, and the phase difference is determined to be incorrect for any of the colors of the pixel to be scanned. Is the point of unknown phase difference. Alternatively, a pixel for which the phase difference is determined to be incorrect in all the colors of the pixel to be scanned is set as the phase difference unknown point. Then, in the case where the color that is determined to have the correct phase difference and the color that is determined to have the incorrect phase difference coexist in the pixel to be scanned, it is determined whether or not the phase difference is correct by the principle of majority voting. It may be possible to determine whether the phase difference is known or unknown according to the determination result.

Next, the corresponding point searching unit 32 determines the number of detected phase difference known points in the short distance phase difference range as the evaluation value calculation unit 34.
To (S7). For example, a range having a phase difference of 80 or more is defined as a short range phase difference range.

Next, the evaluation value calculation unit 32 obtains the detection rate of the phase difference determination points by using the number of detected phase difference determination points in the short distance phase difference range, and the selection output by the condition setting unit 28 in (S2). As the evaluation value of the height direction candidate condition, the maximum detection unit 3 detects the detection rate of the phase difference known point in the short distance phase difference range.
6 (S8).

Next, the condition setting section 28 determines whether or not all height direction candidate conditions have been selected (S9).

If all the height direction candidate conditions have not been selected, the condition setting unit 28 selects the next height direction candidate condition (S10), and then outputs the selected height direction candidate condition. Yes (S2).

For example, among the height direction candidate conditions Δy _{1 to} Δy ₅ , only Δy ₁ has been selected, and Δy _{2 to} Δy.
If the selection for y ₅ has not been completed, one candidate condition Δ selected from the remaining candidate conditions Δy _{2 to} Δy _5.
y ₂ is selected, and the selected height direction candidate condition Δy ₂ is output.

If all the height direction candidate conditions have not been selected, the maximum value detecting unit 36 detects the maximum evaluation value from the evaluation values obtained for each height direction candidate condition and determines the maximum value. The height direction candidate condition for which the evaluation value is obtained is detected as the height direction correction condition. Then, the maximum value detection unit 28 stores the height direction correction condition in the condition setting unit 28 (S11). After that, the correction condition detection device 18 ends the process for detecting the height direction correction condition (end).

According to this embodiment, the detection rate of the phase difference determination point in the short distance phase difference range is used as the evaluation value, and the height direction candidate condition that obtains the maximum evaluation value is used as the height direction correction condition. Since the amount of deviation in the height direction is large in the short distance phase difference range, it is more suitable for epipolar line matching by detecting the height direction correction condition with the detection rate of the phase difference known point in the short distance phase difference range as the evaluation value. The height correction condition can be detected.

<Second Embodiment of Invention of Claim 9> FIG.
FIG. 10 is a functional block diagram provided for explaining a second embodiment of the invention of claim 9; Hereinafter, differences from the first embodiment of the invention of claim 9 will be mainly described, and detailed description of the same points as the first embodiment of the invention of claim 9 will be omitted.

The correction condition detecting device 18 of this embodiment is
Condition setting unit 28, image conversion unit 30, corresponding point search unit 32,
The evaluation value calculation unit 34 and the maximum value detection unit 36, the third left memory 38 and the third right memory 40, and the phase difference range input unit 74.
And a region cutout unit 46.

The area cutout unit 46 sequentially cuts out the left and right attention area images from a plurality of attention areas vertically dispersed on the left and right images after the image conversion, and the corresponding point searching unit 32 outputs the left and right attention area images to the left and right. Corresponding candidate points, corresponding points, known phase differences, and unknown points are detected for each of the left and right search images having the same candidate condition as the search image.

The evaluation value calculation unit 34 determines the detection rate of the phase difference determination point for each of the left and right search images having the same height direction candidate condition for the short-distance phase difference in which the shift amount in the height direction in the left and right search images is large. The average value of the detection rates obtained for the left and right search images having the same height direction candidate condition is set as the evaluation value of the height direction candidate condition.

FIG. 24 is a diagram showing the operational flow of the second embodiment of the invention of claim 9. In the same figure, focusing on candidate conditions and correction conditions in the height direction among a plurality of types of candidate conditions and correction conditions, the flow of operation when detecting the height direction correction conditions using the height direction candidate conditions is shown. Show.

When the operator inputs an operation start signal to the correction condition detecting device 18 (start), the condition setting unit 28 first selects and sets a height direction candidate condition from a plurality of types of candidate conditions (S1), Then, the condition setting unit 28 outputs one candidate condition (selected height direction candidate condition) selected from the plurality of height direction candidate conditions (S2).

Next, the image conversion unit 30 performs image conversion of one of the left image and the right image according to the selected height direction candidate condition input from the condition setting unit 28, and the image-converted left image and right image are converted to the third left image. The data is stored in the memory 38 and the third right memory 40 (S3). The left and right images taken by the left and right cameras 10 and 12 are stored in the first left and right memories 14 and 16, and the image conversion unit 30 reads the left and right images of the first left and right memories 14 and 16, and one of the read left and right images is an image. The other image is converted without being converted, and the left and right images after the image conversion are stored in the third left and right memories 38 and 40.

Next, the area cutout unit 46 cuts out the left and right attention area images from the plurality of attention areas dispersed in the vertical direction (y direction) on the left and right images after the image conversion (S4).

For example, as also shown in FIG. 12, the width in the vertical direction of the left and right target area images is set to the width of one horizontal scanning line or the width of a plurality of horizontal scanning lines, here the width of one line, and the area cut-out portion 46 Is a horizontal scanning line L that is obtained by cutting out the left and right target area images on the corresponding horizontal scanning lines L from the left and right images after image conversion stored in the third left and right memories 38 and 40.
The upper left and right attention area images are stored in the left and right line memories of the corresponding point searching unit 32.

Next, the corresponding point search unit 32 sets a plurality of types of search conditions by making the color of the pixel to be scanned different and sets the left and right target area images as the left and right search images, and sets the left and right for each search condition. Corresponding candidate points in the search image are detected (S
5).

After detecting the corresponding candidate points on the horizontal scanning line L over the entire left and right search images for all types of search conditions, the corresponding point searching unit 32 next finds the corresponding candidate points obtained for each search condition. Corresponding candidate points that are a common set in the set are detected as corresponding points (S6).

Next, the corresponding point searching unit 32 detects the phase difference known point and the phase difference unknown point using the corresponding point detection result (S).
7).

The phase difference known point and the phase difference unknown point detected here relate to the one region of interest 48, 52 from which the current left and right search images are read. Also here, the fitting of the phase difference to the unclear point and the verification of the fitted phase difference are performed.

Next, the corresponding point searching unit 32 detects phase difference known points in the short distance phase difference range and outputs the detected number of phase difference known points in the short distance phase difference range to the evaluation value calculation unit 34 ( S8).

Next, the evaluation value calculation unit 32 obtains the detection rate of the phase difference determination point using the number of detected phase difference determination points in the short distance phase difference range, and detects the maximum value of the detection rate of the phase difference determination point. The data is stored in the unit 36 (S9).

Next, the area cut-out unit 46 has finished cutting out (reading out) the left and right attention area images from all the attention areas with respect to the left and right images after image conversion currently stored in the third left and right memories 38 and 40. Determine whether or not (S
10).

If the extraction of all the attention areas has not been completed for the current left and right images after image conversion, the area extraction unit 46 selects a set of attention areas for which the left and right attention area images have not been cut out (S11). After that, the next right and left attention area images are cut out from the selected attention area (S4).

For example, in the example of FIG. 12, if the cutting out from the target areas 48 and 52 of y = 3 is finished and the cutting out from the target areas 48 and 52 of y = 6 and 9 is not finished, the cutout is not completed. As the set of attention areas 48 and 52, the attention areas 48 and 52 with y = 6 are selected.

After cutting out all the target areas for the left and right images 42, 44 after the current image conversion, the evaluation value calculation unit 34 determines the height direction candidate condition for obtaining the left and right images after the current image conversion. The average value of the detection rates obtained for each of the regions of interest is calculated, and this average value is stored in the maximum value detection unit 36 as the evaluation value of the height direction candidate condition (S12).

For example, the left and right images 42 after the current image conversion,
When 44 is obtained according to the candidate condition Δy ₁ , in the example of FIG. 12, the target regions 48, 5 of y = 3, 6, 9 are obtained.
The average value of the detection rates obtained for each 2 is obtained as the evaluation value of the candidate condition Δy ₁ .

Next, the condition setting unit 28 determines whether or not all height direction candidate conditions have been selected (S13).

If all the height direction candidate conditions have not been selected, the condition setting unit 28 selects the next height direction candidate condition (S14), and then outputs the selected height direction candidate condition. Yes (S2).

For example, among the height direction candidate conditions Δy _{1 to} Δy ₅ , only Δy ₁ has been selected, and Δy _{2 to} Δy
If the selection for y ₅ has not been completed, one candidate condition Δ selected from the remaining candidate conditions Δy _{2 to} Δy _5.
Select y ₂ and output the selected candidate condition Δy ₂ .

When all height direction candidate conditions have been selected, the maximum value detection unit 36 detects the maximum evaluation value from the evaluation values obtained for each height direction candidate condition, The height direction candidate condition for which the evaluation value is obtained is detected as the height direction correction condition. Then, the maximum value detection unit 28 stores the height direction correction condition in the condition setting unit 28 (S15). After that,
The correction condition detection device 18 ends the processing for detecting the height direction correction condition (end).

In this embodiment, the left and right search images are the region of interest image which is a part of the left and right images after the image conversion, so that the scanning amount of the image necessary for detecting the height direction correction condition is reduced. Therefore, it is possible to shorten the detection time of the height direction correction condition.

<Third Embodiment of the Invention of Claim 9> Next, with reference to FIG. 23, a third embodiment of the invention of claim 9 will be described. Hereinafter, differences from the second embodiment of the invention of claim 9 will be mainly described, and detailed description of the same points as the second embodiment of the invention of claim 9 will be omitted.

The correction condition detection device 18 of this embodiment is
Condition setting unit 28, image conversion unit 30, corresponding point search unit 32,
The evaluation value calculation unit 34 and the maximum value detection unit 36, the third left memory 38 and the third right memory 40, and the area cutout unit 4
6 is provided.

The evaluation value calculation unit 34 determines the detection rate of the phase difference determination point for each of the left and right search images having the same height direction candidate condition for the short-distance phase difference in which the shift amount in the height direction in the left and right search images is large. Among the detection rates obtained for the left and right search images having the same height direction candidate condition, a predetermined number of detection rates are sequentially selected from the top, and the average value of the detection rates obtained by the selection is used as the evaluation value of the height direction candidate condition. To do.

FIG. 25 is a diagram showing the operational flow of the second embodiment of the invention of claim 9. In the same figure, focusing on candidate conditions and correction conditions in the height direction among a plurality of types of candidate conditions and correction conditions, the flow of operation when detecting the height direction correction conditions using the height direction candidate conditions is shown. Show.

When the operator inputs an operation start signal to the correction condition detecting device 18 (start), the condition setting unit 28 first selects and sets a height direction candidate condition from a plurality of types of candidate conditions (S1), Then, the condition setting unit 28 outputs one candidate condition (selected height direction candidate condition) selected from the plurality of height direction candidate conditions (S2).

Next, the image conversion unit 30 performs image conversion on one of the left image and the right image in accordance with the selected height direction candidate condition input from the condition setting unit 28, and the image-converted left image and right image are converted to the third left image. The data is stored in the memory 38 and the third right memory 40 (S3). The left and right images taken by the left and right cameras 10 and 12 are stored in the first left and right memories 14 and 16, and the image conversion unit 30 reads the left and right images of the first left and right memories 14 and 16, and one of the read left and right images is an image. The other image is converted without being converted, and the left and right images after the image conversion are stored in the third left and right memories 38 and 40.

Next, the area cutout unit 46 cuts out the left and right attention area images from a plurality of attention areas dispersed in the vertical direction (y direction) on the left and right images after the image conversion (S4).

Next, the corresponding point search unit 32 sets a plurality of types of search conditions by making the color of the pixel to be scanned different and sets the left and right target area images as left and right search images, and sets the left and right for each search condition. Corresponding candidate points in the search image are detected (S
5).

When the detection of the corresponding candidate points on the horizontal scanning line L is completed over the entire left and right search images for all types of search conditions, the corresponding point searching unit 32 next finds the corresponding candidate points obtained for each search condition. Corresponding candidate points that are a common set in the set are detected as corresponding points (S6).

Next, the corresponding point searching unit 32 detects the phase difference known point and the phase difference unknown point using the corresponding point detection result (S).
7).

Next, the corresponding point searching unit 32 detects a phase difference known point in the short distance phase difference range and outputs the detected number of phase difference known points in the short distance phase difference range to the evaluation value calculation unit 34 ( S8).

Next, the evaluation value calculation unit 32 obtains the detection rate of the phase difference determination point using the number of detected phase difference determination points in the short distance phase difference range, and detects the maximum value of the detection rate of the phase difference determination point. The data is stored in the unit 36 (S9).

Next, the area cut-out unit 46 has finished cutting out the left and right attention area images from all the attention areas (whether the reading has been completed) with respect to the left and right images after image conversion currently stored in the third left and right memories 38 and 40. Determine whether or not (S
10).

If the extraction of all the attention areas has not been completed for the current left and right images after image conversion, the area extraction unit 46 selects a set of attention areas for which the left and right attention area images have not been cut out (S11). After that, the next right and left attention area images are cut out from the selected attention area (S4).

When the extraction of all the target areas has been completed for the left and right images after the current image conversion, the evaluation value calculation unit 3
Reference numeral 4 relates to the height direction candidate condition for obtaining the right and left images after the current image conversion, and selects a higher detection rate among the detection rates obtained for each attention area (S12).

Next, the evaluation value calculation unit 34 obtains the average value of the selected upper detection rates, and uses this average value as the maximum evaluation value for the height direction candidate condition for obtaining the left and right images after the current image conversion. It stores in the value detection part 36 (S13).

Here, as described with reference to FIG. 12, the attention area 4 set on the left and right images 42 and 44 after the current image conversion.
Since the total number of 8 and 52 is the same, assuming that the total number is R, the detection rate of the total number R is obtained from the R regions of interest 48 and 52. Then, the detection rate of the top r1 (r1 is a natural number of R>r1> 1) is selected from the detection rates of the total number R, and the average value of the detection rates of the selected top r1 is set as the evaluation value. The lower (R-r1) detection rates of (R-r1)> 0 are not used for calculating the evaluation value.

For example, the total number R = 30 target regions 48,
If 52 is set and the detection rate of the total number R = 30 is obtained from these regions of interest, the evaluation value is obtained using the detection rates of the upper r1 = 10 to 20, and the lower (R-r1) = 20 ~
The 10 detection rates are not used for calculating the evaluation value.

Next, the condition setting unit 28 determines whether or not all height direction candidate conditions have been selected (S14).

If all the height direction candidate conditions have not been selected, the condition setting unit 28 selects the next height direction candidate condition (S15), and thereafter outputs the selected height direction candidate condition. Yes (S2).

When all the height direction candidate conditions have been selected, the maximum value detecting unit 36 detects the maximum evaluation value from the evaluation values obtained for each height direction candidate condition and determines the maximum value. The height direction candidate condition for which the evaluation value is obtained is detected as the height direction correction condition. Then, the maximum value detection unit 28 stores the height direction correction condition in the condition setting unit 28 (S16). After that,
The correction condition detection device 18 ends the processing for detecting the height direction correction condition (end).

Also in this embodiment, as in the second embodiment of the ninth aspect, the left and right search images are set as the attention area images which are part of the left and right images after the image conversion, and therefore the height direction correction condition is detected. Since it is possible to reduce the scanning amount of the image required for the adjustment, it is possible to shorten the detection time of the height direction correction condition.

Further, according to this embodiment, when the evaluation value of the height direction candidate condition is obtained, the evaluation value is obtained by using the average value of the plurality of detection rates of the upper rank, and the lower detection rate is calculated. Therefore, it is possible to improve the detection accuracy of the correction condition in the height direction.

<First Embodiment of Invention of Claim 13> Next, a first embodiment of the invention of claim 13 will be described with reference to FIG. Hereinafter, differences from the first embodiment of the invention of claim 1 will be described, and detailed description of the same points as the first embodiment of the invention of claim 1 will be omitted.

The correction condition detecting device 18 of this embodiment is
Condition setting unit 28, image conversion unit 30, corresponding point search unit 32,
The evaluation value calculation unit 34 and the maximum value detection unit 36, the third left memory 38 and the third right memory 40, and the phase difference range input unit 74 are further provided.

The condition setting unit 28 sequentially sets candidate conditions for detecting the tilt angle correction condition, and the image conversion unit 30 selects one of the left image and the right image obtained by the left camera 10 and the right camera 12 as a candidate. Image conversion according to the conditions. The tilt angle is an image rotation angle around the z-axis when the x-axis direction is the horizontal direction, the y-axis direction is the vertical direction (height direction), and the z-axis direction is the depth direction.

Here, the condition setting unit 28 detects the correction conditions of other types including the tilt angle correction condition, and the condition setting unit 28 matches a plurality of types of candidate conditions for detecting the correction conditions with the epipolar line. A plurality of types of correction conditions for storing are stored. The correction condition to be stored is the correction condition detected by the maximum value detection unit 36. One of these plural types of candidate conditions and correction conditions is a tilt angle candidate condition and correction condition.

The condition setting unit 28 stores candidate conditions relating to parallel movement, rotation and magnification conversion of an image. Therefore, the correction condition detected by the maximum value detection unit 36 is the correction condition relating to the parallel movement, rotation, and magnification conversion of the image, and the image conversion unit 3
Image conversion performed by 0 according to the candidate condition and the correction condition is image parallel movement, rotation, and magnification conversion. Known methods can be used for these parallel movement, rotation, and magnification conversion.

The corresponding point searching unit 32 sets a plurality of types of search conditions by making the scanning direction, the gradation expression of the pixel to be scanned, or the color of the pixel to be scanned different, and sets the right and left after image conversion. Let the images be left and right search images. Here, the left and right images of the color image are obtained by the left and right cameras 10 and 12, and the color of the pixel to be scanned is made different to set a plurality of types of search conditions. Further, the left and right images after the image conversion are used as the left and right search images over the entire surface.

Then, the corresponding point searching unit 32 scans the left and right search images according to the search condition, compares the pixel features of the left search image and the pixel features of the right search image on the corresponding horizontal scanning lines, and compares the pixel feature differences. The pixels of the left and right search images in which is less than or equal to the threshold are detected as corresponding candidate points. Further, the corresponding point searching unit 32 detects, as corresponding points, corresponding candidate points forming a common set in the set of corresponding candidate points detected for each search condition. Here, the pixel feature is the pixel density.

For example, a color image is stored in the third left memory 38 and the third right memory 40 as the left and right images after the image conversion, and the red image of the left and right images after the image conversion is the first left and right search image. The search condition and the second search condition in which the blue image of the left and right images after the image conversion is used as the left and right search images are set. Then, the left and right search images of the red image are scanned under the first search condition, and the pixels of the left and right search image where the pixel feature difference on the corresponding horizontal scanning lines is less than or equal to the threshold value To detect as.
Furthermore, the left and right search images of the blue image are scanned under the second search condition, and the pixels of the left and right search images for which the difference in pixel feature on the corresponding horizontal scanning lines is less than or equal to the threshold are identified as the corresponding candidate points according to the second search condition. To detect as. Then, the corresponding candidate points that are a common set in the set of corresponding candidate points detected under the first search condition and the set of corresponding candidate points detected under the second search condition are detected as corresponding points.

Next, the corresponding point searching unit 32 detects the identified point and the unknown point of the long-distance phase difference in which the shift amount of the tilt angle is large in the left and right search images by using the corresponding point detection result. The operator uses the phase difference range input unit 74 to set the range of the long-distance phase difference in which the amount of deviation in the height direction is large, through the corresponding point search unit 3
Enter 2 For example, if the phase difference is less than 80,
Input as the range of the long-distance phase difference in which the amount of tilt angle deviation is large.

Here, the phase difference is estimated and applied to the unidentified point (pixels not detected as the corresponding point) based on the phase difference of the corresponding points. An uncertain point that cannot be fitted with a phase difference is detected as a phase difference unknown point. After that, regarding the phase difference applied to the correspondence unknown point,
Test whether the phase difference is correct. Then, the unknown correspondence point of the phase difference determined to be correct by this test is detected as the unknown phase difference point, and the unknown correspondence point of the phase difference determined to be incorrect is detected as the unknown phase difference point.

Since a plurality of types of search conditions are set by making the color of the pixel to be scanned different, the fitting of the phase difference and the verification of the phase difference are performed for each color of the pixel to be scanned. Pixels for which the phase difference is determined to be correct for all the colors of the pixel to be scanned, and the phase difference is determined to be incorrect for any of the colors of the pixel to be scanned. Is the point of unknown phase difference. Alternatively, a pixel for which the phase difference is determined to be incorrect in all the colors of the pixel to be scanned is set as the phase difference unknown point. Then, in the case where the color that is determined to have the correct phase difference and the color that is determined to have the incorrect phase difference coexist in the pixel to be scanned, it is determined whether or not the phase difference is correct by the principle of majority voting. It may be determined whether the phase difference is known or unknown depending on the determination result.

Then, of the phase difference known points, the long distance phase difference known point where the deviation amount of the tilt angle is large is detected.

The evaluation value calculation unit 34 obtains the detection rate of the phase difference determination point for the long-distance phase difference in which the shift amount of the tilt angle is large in the left and right search images, and uses the detection rate to determine the tilt angle candidate condition. Obtain an evaluation value. Here, the detection rate of the phase difference determination point is calculated for each tilt angle candidate condition for the long-distance phase difference in which the shift amount of the tilt angle is large in the left and right search images, and the detection rate is set as the evaluation value of the tilt angle candidate condition.

The maximum value detecting unit 36 detects the tilt angle candidate condition that has obtained the maximum evaluation value as the tilt angle correction condition.
Here, the maximum value detection unit 36 stores the detected tilt angle correction condition in the condition setting unit 28.

FIG. 26 is a diagram showing the operational flow of the second embodiment of the thirteenth invention. In the same figure, focusing on the tilt angle candidate condition and the correction condition among a plurality of types of the candidate condition and the correction condition, the flow of the operation when the tilt angle correction condition is detected using the tilt angle candidate condition is shown.

When the operator inputs an operation start signal to the correction condition detecting device 18 (start), the condition setting unit 28 first selects and sets a tilt angle candidate condition from a plurality of kinds of candidate conditions (S1), After that, the condition setting unit 28 outputs one tilt angle candidate condition (selected tilt angle candidate condition) selected from the plurality of tilt angle candidate conditions (S).
2). The tilt angle candidate conditions are candidate conditions relating to rotation around the z-axis, and for example, five tilt angle candidate conditions Δγ ₁ to
One tilt angle candidate condition Δγ selected from Δγ ₅
Outputs ₁ .

Next, the image conversion section 30 includes the left and right cameras 10,
One of the left and right images captured by the condition setting unit 28
The image is converted in accordance with the selected tilt angle candidate condition input from, and the left and right images after the image conversion are converted into the third left and right memories 38, 40.
(S3). The left and right images captured by the left and right cameras 10 and 12 are stored in the first left and right memories 14 and 16, and the image conversion unit 30 reads the left and right images of the first left and right memories 14 and 16, and one of the read left and right images is The left and right images after the image conversion are stored in the third left and right memories 38 and 40 without image conversion for the other.

Next, the corresponding point search unit 32 sets a plurality of types of search conditions by making the color of the pixel to be scanned different and sets the entire left and right images after image conversion as the left and right search images. Then, the corresponding candidate points in the left and right search images are detected (S4). Here, since different types of search conditions are set by making the colors of the pixels different, the left and right images photographed by the left and right cameras 10 and 12 are color images.

For example, among the left and right images of the color images after image conversion stored in the third left and right memories 38 and 40, the first search condition for scanning the red left and right images as the left and right search images,
A second search condition for scanning the blue left and right images as the left and right search images is set. Then, the corresponding point search unit 32 scans the red left and right images under the first search condition, cuts out one line of the red left and right images on the corresponding horizontal scanning line L from the third left and right memories 38, 40, and responds. The data is stored in the left and right line memories included in the point search unit 32. Here, in the red left and right images, one pixel is a target point and the other pixel is a reference point, and the i-th target is sequentially counted in a predetermined main scanning direction, for example, the right direction on the corresponding horizontal scanning line L. With respect to the points, the control points on the horizontal scanning line L are sequentially scanned in a predetermined main scanning direction, for example, the right direction. When the scanning of the control point is completed without detecting the control point in which the difference in pixel characteristics, for example, the pixel density, is less than or equal to the threshold value with respect to the i-th point of interest,
The control point is scanned for the + 1st point of interest. When a control point in which the difference in pixel feature is less than or equal to the threshold value with respect to the i-th point of interest is detected, the i-th point of interest and the control point are detected as corresponding candidate points. The control point is scanned for the point of interest. Third left and right memory 38,
When the detection of the corresponding candidate points by the first search condition is completed for the entire left and right images of 40, then the blue left and right images are scanned by the second search condition in the same manner as the first search condition.
Corresponding candidate points are detected by the second search condition.

When the detection of corresponding candidate points is completed for all types of search conditions, the corresponding point searching unit 32 then associates corresponding candidate points that are a common set in the set of corresponding candidate points obtained for each search condition. It is detected as a point (S5).

As described above, the left and right search images on the corresponding horizontal scanning lines L are scanned, and when the difference between the pixel features becomes equal to or less than the threshold value, the next point of interest is scanned to detect the corresponding candidate point. Corresponding candidate points that are a common set in the set of corresponding candidate points obtained for each search condition are a set of corresponding candidate points that are probabilistically probable.

Next, the corresponding point searching unit 32 detects the phase difference known point and the phase difference unknown point in the long-distance phase difference range using the corresponding point detection result (S6).

The phase difference known point and the phase difference unknown point detected here are for the entire left and right images which are the current left and right search images.

Further, here, the phase difference is estimated and applied to the unclear point (pixels not detected as the corresponding point) based on the phase difference of the corresponding points. An uncertain point that cannot be fitted with a phase difference is detected as a phase difference unknown point. After that, it is verified whether the phase difference applied to the unclear point is correct. Then, the unknown correspondence point of the phase difference determined to be correct by this test is set as the phase difference determination point, and the unknown correspondence point of the phase difference determined to be incorrect is detected as the phase difference unknown point.

Since a plurality of types of search conditions are set by making the color of the pixel to be scanned different, a phase difference fitting and a phase difference test are performed for each color of the pixel to be scanned. Pixels for which the phase difference is determined to be correct for all the colors of the pixel to be scanned, and the phase difference is determined to be incorrect for any of the colors of the pixel to be scanned. Is the point of unknown phase difference.

Next, the corresponding point searching section 32 determines the number of detected phase difference known points in the long-distance phase difference range by the evaluation value calculating section 34.
To (S7). For example, the range where the phase difference is less than 80 is defined as the long-distance phase difference range.

Next, the evaluation value calculation unit 32 obtains the detection rate of the phase difference determination points using the number of detected phase difference determination points in the long distance phase difference range, and the selection output by the condition setting unit 28 in (S2). As the evaluation value of the tilt angle candidate condition, the maximum value detection unit 3 uses the detection rate of the phase difference known point in the long-distance phase difference range.
6 (S8).

Next, the condition setting section 28 determines whether or not all the tilt angle candidate conditions have been selected (S9).

If all the tilt angle candidate conditions have not been selected, the condition setting section 28 selects the next tilt angle candidate condition (S10), and thereafter outputs the selected tilt angle candidate condition (S2). ).

For example, among the tilt angle candidate conditions Δγ _{1 to} Δγ ₅ , only Δγ ₁ has been selected, and Δγ _{2 to} Δγ
If the selection for γ ₅ has not been completed, one candidate condition Δ selected from the remaining candidate conditions Δγ _{2 to} Δγ ₅
γ ₂ is selected, and the selected tilt angle candidate condition Δγ ₂ is output.

If all the tilt angle candidate conditions have not been selected, the maximum value detection unit 36 detects the maximum evaluation value from the evaluation values obtained for each tilt angle candidate condition and determines the maximum evaluation value. The obtained tilt angle candidate condition is detected as the tilt angle correction condition. Then, the maximum value detection unit 28 stores the tilt angle correction condition in the condition setting unit 28 (S11). After that, the correction condition detection device 18 ends the process for detecting the tilt angle correction condition (end).

According to this embodiment, the detection rate of the phase difference determination point in the long-distance phase difference range is used as the evaluation value, and the tilt angle candidate condition that gives the maximum evaluation value is used as the tilt angle correction condition. Since the amount of tilt angle shift is large in the long-distance phase difference range, the tilt angle correction condition is detected by using the detection rate of the phase difference determination point in the long-distance phase difference range as an evaluation value, so that a tilt more suitable for epipolar line alignment can be obtained. The corner correction condition can be detected.

<Second Embodiment of Invention of Claim 13> Next, a second embodiment of the invention of claim 13 will be described with reference to FIG. Hereinafter, differences from the first embodiment of the invention of claim 13 will be mainly described, and detailed description of the same points as the first embodiment of the invention of claim 13 will be omitted.

The correction condition detecting device 18 of this embodiment is
Condition setting unit 28, image conversion unit 30, corresponding point search unit 32,
The evaluation value calculation unit 34 and the maximum value detection unit 36, the third left memory 38 and the third right memory 40, and the phase difference range input unit 74.
And a region cutout unit 46.

The area cutout unit 46 sequentially cuts out the left and right attention area images from the plurality of attention areas vertically dispersed on the left and right images after the image conversion, and the corresponding point searching unit 32 outputs the left and right attention area images to the left and right. Corresponding candidate points, corresponding points, known phase differences, and unknown points are detected for each of the left and right search images having the same candidate condition as the search image.

The evaluation value calculation unit 34 obtains the detection rate of the phase difference determination point for each left and right search image having the same tilt angle candidate condition for the long-distance phase difference in which the deviation amount of the tilt angle is large in the left and right search images. The average value of the detection rates obtained for the left and right search images having the same angle candidate condition is set as the evaluation value of the tilt angle candidate condition.

FIG. 27 is a diagram showing the operational flow of the second embodiment of the thirteenth invention. In the same figure, focusing on the tilt angle candidate condition and the correction condition among a plurality of types of candidate conditions and correction conditions, the flow of the operation when the tilt angle correction condition is detected using the tilt angle candidate condition is shown.

When the operator inputs an operation start signal to the correction condition detecting device 18 (start), the condition setting section 28 first selects and sets a tilt angle candidate condition from a plurality of kinds of candidate conditions (S1), and After that, the condition setting unit 28 outputs one candidate condition (selected tilt angle candidate condition) selected from the plurality of tilt angle candidate conditions (S2).

Next, the image conversion unit 30 performs image conversion on one of the left image and the right image in accordance with the selected tilt angle candidate condition input from the condition setting unit 28, and stores the image-converted left image and right image in the third left memory. 38 and the third right memory 40 (S3). The left and right images taken by the left and right cameras 10 and 12 are stored in the first left and right memories 14 and 16, and the image conversion unit 30 reads the left and right images of the first left and right memories 14 and 16, and one of the read left and right images is an image. The other image is converted without being converted, and the left and right images after the image conversion are stored in the third left and right memories 38 and 40.

Next, the area cutout unit 46 cuts out the left and right attention area images from a plurality of attention areas dispersed in the vertical direction (y direction) on the left and right images after the image conversion (S4).

For example, as shown in FIG. 12, the width in the vertical direction of the left and right target area images is set to the width of one horizontal scanning line or the width of a plurality of horizontal scanning lines, and in this case, the width of one area is set. Is a horizontal scanning line L that is obtained by cutting out the left and right target area images on the corresponding horizontal scanning lines L from the left and right images after image conversion stored in the third left and right memories 38 and 40.
The upper left and right attention area images are stored in the left and right line memories of the corresponding point searching unit 32.

Next, the corresponding point searching unit 32 sets a plurality of types of search conditions by making the color of the pixel to be scanned different and sets the left and right target area images as the left and right search images, and sets the left and right for each search condition. Corresponding candidate points in the search image are detected (S
5).

After detecting the corresponding candidate points on the horizontal scanning line L over the entire left and right search images for all types of search conditions, the corresponding point searching unit 32 next finds the corresponding candidate points obtained for each search condition. Corresponding candidate points that are a common set in the set are detected as corresponding points (S6).

Next, the corresponding point searching unit 32 detects the phase difference known point and the phase difference unknown point using the corresponding point detection result (S).
7).

The phase difference known point and the phase difference unknown point detected here are related to the one region of interest 48, 52 from which the current left and right search images are read. Also here, the fitting of the phase difference to the unclear point and the verification of the fitted phase difference are performed.

Next, the corresponding point searching unit 32 detects a phase difference known point in the long distance phase difference range and outputs the detected number of phase difference known points in the long distance phase difference range to the evaluation value calculation unit 34 ( S8).

Next, the evaluation value calculation unit 32 obtains the detection rate of the phase difference determination point using the number of detected phase difference determination points in the long-distance phase difference range, and detects the maximum value of the detection rate of the phase difference determination point. The data is stored in the unit 36 (S9).

Then, the area cut-out unit 46 has finished cutting out (reading out) the left and right attention area images from all the attention areas with respect to the left and right images after image conversion currently stored in the third left and right memories 38 and 40. Determine whether or not (S
10).

If the extraction of all the attention areas has not been completed for the current left and right images after image conversion, the area extraction unit 46 selects a set of attention areas for which the left and right attention area images have not been cut out (S11). After that, the next right and left attention area images are cut out from the selected attention area (S4).

For example, in the example of FIG. 12, if the cutting out from the target areas 48 and 52 of y = 3 is completed and the cutting out from the target areas 48 and 52 of y = 6 and 9 is not completed, the cutout is not completed. As the set of attention areas 48 and 52, the attention areas 48 and 52 with y = 6 are selected.

After cutting out all the target areas for the left and right images 42 and 44 after the current image conversion, the evaluation value calculation unit 34 determines the tilt angle candidate conditions for obtaining the left and right images after the current image conversion. An average value of the detection rates obtained for each of the regions of interest is calculated, and this average value is stored in the maximum value detection unit 36 as the evaluation value of the tilt angle candidate condition (S12).

For example, the left and right images 42 after the current image conversion,
When 44 is obtained according to the candidate condition Δy ₁ , in the example of FIG. 12, the target regions 48, 5 of y = 3, 6, 9 are obtained.
The average value of the detection rates obtained for each 2 is obtained as the evaluation value of the candidate condition Δy ₁ .

Next, the condition setting section 28 determines whether or not all the tilt angle candidate conditions have been selected (S13).

If all the tilt angle candidate conditions have not been selected, the condition setting section 28 selects the next tilt angle candidate condition (S14), and thereafter outputs the selected tilt angle candidate condition (S2). ).

For example, among the tilt angle candidate conditions Δγ _{1 to} Δγ ₅ , only Δγ ₁ has been selected and Δγ _{2 to} Δ
If the selection for γ ₅ has not been completed, one candidate condition Δ selected from the remaining candidate conditions Δγ _{2 to} Δγ ₅
γ ₂ is selected and the selected candidate condition Δγ ₂ is output.

When all the tilt angle candidate conditions have been selected, the maximum value detection unit 36 detects the maximum evaluation value from the evaluation values obtained for each tilt angle candidate condition and determines the maximum evaluation value. The obtained tilt angle candidate condition is detected as the tilt angle correction condition. Then, the maximum value detection unit 28 stores the tilt angle correction condition in the condition setting unit 28 (S15). After that,
The correction condition detection device 18 ends the process for detecting the tilt angle correction condition (end).

In this embodiment, the left and right search images are the area of interest image which is a part of the left and right images after the image conversion, so that the scanning amount of the image necessary for detecting the tilt angle correction condition can be reduced. Therefore, the detection time of the tilt angle correction condition can be shortened.

<Third Embodiment of Invention of Claim 13> Next, with reference to FIG. 23, a third embodiment of the invention of Claim 13 will be described. Hereinafter, differences from the second embodiment of the invention of claim 13 will be mainly described, and detailed description of the same points as the second embodiment of the invention of claim 13 will be omitted.

The correction condition detecting device 18 of this embodiment is
Condition setting unit 28, image conversion unit 30, corresponding point search unit 32,
The evaluation value calculation unit 34 and the maximum value detection unit 36, the third left memory 38 and the third right memory 40, and the area cutout unit 4
6 is provided.

The evaluation value calculation unit 34 obtains the detection rate of the phase difference determination point for each left and right search image having the same tilt angle candidate condition for the long-distance phase difference in which the deviation amount of the tilt angle is large in the left and right search images. Of the detection rates obtained for the left and right search images with the same angle candidate condition, a predetermined number of detection rates are sequentially selected from the top, and the average value of the detection rates obtained by the selection is used as the evaluation value of the tilt angle candidate condition.

FIG. 28 is a diagram showing the operational flow of the second embodiment of the thirteenth invention. In the same figure, focusing on the tilt angle candidate condition and the correction condition among a plurality of types of candidate conditions and correction conditions, the flow of the operation when the tilt angle correction condition is detected using the tilt angle candidate condition is shown.

When the operator inputs an operation start signal to the correction condition detecting device 18 (start), the condition setting section 28 first selects and sets a tilt angle candidate condition from a plurality of types of candidate conditions (S1). After that, the condition setting unit 28 outputs one candidate condition (selected tilt angle candidate condition) selected from the plurality of tilt angle candidate conditions (S2).

Next, the image conversion unit 30 performs image conversion on one of the left image and the right image in accordance with the selected tilt angle candidate condition input from the condition setting unit 28, and stores the image-converted left image and right image in the third left memory. 38 and the third right memory 40 (S3). The left and right images taken by the left and right cameras 10 and 12 are stored in the first left and right memories 14 and 16, and the image conversion unit 30 reads the left and right images of the first left and right memories 14 and 16, and one of the read left and right images is an image. The other image is converted without being converted, and the left and right images after the image conversion are stored in the third left and right memories 38 and 40.

Next, the area cutout unit 46 cuts out the left and right attention area images from a plurality of attention areas dispersed in the vertical direction (y direction) on the left and right images after the image conversion (S4).

Next, the corresponding point searching unit 32 sets a plurality of types of search conditions by making the color of the pixel to be scanned different and sets the left and right target area images as the left and right search images. Corresponding candidate points in the search image are detected (S
5).

After detecting the corresponding candidate points on the horizontal scanning line L over the entire left and right search images for all kinds of search conditions, the corresponding point searching unit 32 next finds the corresponding candidate points obtained for each search condition. Corresponding candidate points that are a common set in the set are detected as corresponding points (S6).

Next, the corresponding point searching unit 32 detects the phase difference known point and the phase difference unknown point using the corresponding point detection result (S).
7).

Next, the corresponding point searching unit 32 detects a phase difference known point in the long distance phase difference range and outputs the detected number of phase difference known points in the long distance phase difference range to the evaluation value calculation unit 34 ( S8).

Next, the evaluation value calculation unit 32 obtains the detection rate of the phase difference determination point using the number of detected phase difference determination points in the long-distance phase difference range, and detects the maximum value of the detection rate of the phase difference determination point. The data is stored in the unit 36 (S9).

Next, the area cutout unit 46 has finished cutting out the left and right attention area images from all the attention areas with respect to the left and right images after image conversion which are currently stored in the third left and right memories 38 and 40 (whether or not they have been read out). Determine whether or not (S
10).

If the cutout of all the attention areas has not been completed for the left and right images after the current image conversion, the area cutout unit 46 selects a set of not-yet-cut out attention areas of the left and right attention area images (S11). After that, the next right and left attention area images are cut out from the selected attention area (S4).

[0498] If the extraction of all regions of interest has been completed for the left and right images after the current image conversion, the evaluation value calculation unit 3
Reference numeral 4 relates to a tilt angle candidate condition for obtaining the right and left images after the current image conversion, and selects a higher detection rate among the detection rates obtained for each attention area (S12).

Next, the evaluation value calculation unit 34 obtains the average value of the selected upper detection rates, and uses this average value as the maximum value as the evaluation value for the tilt angle candidate condition for obtaining the left and right images after the current image conversion. It stores in the detection part 36 (S13).

As described with reference to FIG. 12, the attention area 4 set on the left and right images 42 and 44 after the current image conversion.
Since the total number of 8 and 52 is the same, assuming that the total number is R, the detection rate of the total number R is obtained from the R regions of interest 48 and 52. Then, the detection rate of the top r1 (r1 is a natural number of R>r1> 1) is selected from the detection rates of the total number R, and the average value of the detection rates of the selected top r1 is set as the evaluation value. The lower (R-r1) detection rates of (R-r1)> 0 are not used for calculating the evaluation value.

For example, the total number R = 30 target regions 48,
If 52 is set and the detection rate of the total number R = 30 is obtained from these regions of interest, the evaluation value is obtained using the detection rates of the upper r1 = 10 to 20, and the lower (R-r1) = 20 ~
The 10 detection rates are not used for calculating the evaluation value.

Next, the condition setting section 28 determines whether or not all the tilt angle candidate conditions have been selected (S14).

If all the tilt angle candidate conditions have not been selected, the condition setting section 28 selects the next tilt angle candidate condition (S15), and then outputs the selected tilt angle candidate condition (S2). ).

When all the tilt angle candidate conditions have been selected, the maximum value detection unit 36 detects the maximum evaluation value from the evaluation values obtained for each tilt angle candidate condition and determines the maximum evaluation value. The obtained tilt angle candidate condition is detected as the tilt angle correction condition. Then, the maximum value detection unit 28 stores the tilt angle correction condition in the condition setting unit 28 (S16). After that,
The correction condition detection device 18 ends the process for detecting the tilt angle correction condition (end).

Also in this embodiment, as in the second embodiment of the thirteenth aspect, the left and right search images are the region-of-interest images which are part of the left and right images after image conversion, and therefore the tilt angle correction condition is detected. Since it is possible to reduce the scanning amount of the image necessary for the above, it is possible to shorten the detection time of the tilt angle correction condition.

Further, according to this embodiment, when the evaluation value of the tilt angle candidate condition is obtained, the evaluation value is obtained by using the average value of the plurality of detection rates of the upper order, and the lower detection rate is used for the calculation of the evaluation value. Since it is not used, the detection accuracy of the tilt angle correction condition can be improved.

[0507]

As is apparent from the above description, according to the invention of claim 1, a plurality of candidate conditions are prepared for each type, and a plurality of types of candidate conditions are prepared and obtained by the left and right cameras. One of the left and right images is subjected to image conversion in accordance with the candidate condition, and the known point and the unknown point of the phase difference are detected from the left and right images after the image conversion. Then, for each candidate condition, the evaluation value of the candidate condition is obtained using the detection rate of the phase difference known point in the left and right images after image conversion, and the candidate condition having the maximum evaluation value is used to match the epipolar line. Detect as a correction condition.

In addition, when the point of unknown phase difference and the point of unknown phase difference are detected, a plurality of types of search conditions are set, the left and right images after image conversion are scanned according to the search conditions, and pixel features are displayed on the corresponding horizontal scanning lines. Pixels having a difference of less than or equal to a threshold value are detected as corresponding candidate points. Then, in the set of corresponding candidate points detected for each search condition, corresponding candidate points forming a common set are detected as corresponding points, and the detected point and the unknown point of the phase difference are detected using the corresponding point detection result.

Therefore, according to the first aspect of the present invention, the correction condition used for aligning the epipolar line can be detected even if there is no specific test pattern in the left and right images taken by the left and right cameras.

According to the invention of claim 5, a plurality of candidate conditions are prepared for each type, a plurality of types of candidate conditions are prepared, and one of the left and right images obtained by the left and right cameras is image-converted according to the candidate condition. Then, the known point and the unknown point of the phase difference are detected from the left and right images after the image conversion. Then, for each candidate condition, the evaluation value of the candidate condition is obtained using the detection rate of the phase difference known point in the left and right images after image conversion, and the candidate condition having the maximum evaluation value is used to match the epipolar line. Detect as a correction condition.

[0511] Moreover, when detecting the unknown point or the unknown point of the phase difference, a plurality of types of search conditions are set, the left and right images after image conversion are scanned according to the search conditions, and the pixel features are displayed on the corresponding horizontal scanning lines. Pixels having a difference of less than or equal to a threshold value are detected as corresponding candidate points. Then, in the set of corresponding candidate points detected for each search condition, corresponding candidate points forming a common set are detected as corresponding points, and the detected point and the unknown point of the phase difference are detected using the corresponding point detection result.

Therefore, according to the invention of claim 5, the correction condition used for aligning the epipolar line can be detected even if no specific test pattern is present in the left and right images photographed by the left and right cameras.

According to the fifth aspect of the present invention, the left and right images after the image conversion are converted from m gradations to n gradations, and one or more selected ones of the left and right images of the n gradations after the image conversion. Since the correction condition is detected by scanning only the image of the gradation level of some kind, there is an advantage that the time required to detect the correction condition can be shortened.

According to the ninth aspect of the present invention, a plurality of height direction candidate conditions are prepared, and one of the left and right images obtained by the left and right cameras is image-converted according to the height direction candidate condition to perform image conversion. Detected and unclear points of the phase difference are detected from the left and right images. Then, for each candidate condition, the evaluation value of the height direction candidate condition is obtained by using the detection rate of the phase difference determination point in the short distance phase difference range in which the amount of deviation in the height direction is large, and the maximum evaluation value is obtained. The height direction candidate condition is detected as a height direction correction condition used for matching the epipolar line.

[0515] Moreover, when detecting a phase difference determination point and an unknown point, a plurality of types of search conditions are set, the left and right images after image conversion are scanned according to the search conditions, and pixel features are displayed on corresponding horizontal scanning lines. Pixels having a difference of less than or equal to a threshold value are detected as corresponding candidate points. Then, in the set of corresponding candidate points detected for each search condition, corresponding candidate points forming a common set are detected as corresponding points, and the detected point and the unknown point of the phase difference are detected using the corresponding point detection result.

Therefore, according to the invention of claim 9, the height direction correction condition used for aligning the epipolar line can be detected even if there is no specific test pattern in the left and right images photographed by the left and right cameras. .

Further, according to the invention of claim 9, the evaluation value of the candidate condition in the height direction is obtained by using the detection rate of the phase difference determination point in the short distance phase difference range where the shift amount in the height direction is large. Then, the height direction candidate condition having the maximum evaluation value is detected as the height direction correction condition, so that the height direction correction condition more suitable for the epipolar line alignment can be detected.

According to the thirteenth aspect of the present invention, a plurality of tilt angle candidate conditions are prepared, one of the left and right images obtained by the left and right cameras is subjected to image conversion according to the tilt angle candidate conditions, and after the image conversion, Detected and unknown phase difference is detected from the left and right images. Then, for each candidate condition, the evaluation value of the tilt angle candidate condition is obtained by using the detection rate of the phase difference determination point in the short distance phase difference range where the amount of tilt angle deviation is large, and the tilt that has the maximum evaluation value is obtained. The angle candidate condition is detected as a tilt angle correction condition used to match the epipolar line.

[0519] Moreover, when detecting the unknown point or the unknown point of the phase difference, a plurality of types of search conditions are set, the left and right images after image conversion are scanned according to the search conditions, and the pixel features are displayed on the corresponding horizontal scanning lines. Pixels having a difference of less than or equal to a threshold value are detected as corresponding candidate points. Then, in the set of corresponding candidate points detected for each search condition, corresponding candidate points forming a common set are detected as corresponding points, and the detected point and the unknown point of the phase difference are detected using the corresponding point detection result.

Therefore, according to the thirteenth aspect of the present invention, the tilt angle correction condition used for aligning the epipolar line can be detected even if no specific test pattern is present in the left and right images photographed by the left and right cameras.

Further, according to the invention of claim 13, the evaluation value of the candidate condition of the tilt angle is obtained by using the detection rate of the phase difference determination point in the short distance phase difference range where the deviation amount of the tilt angle is large, Since the tilt angle candidate condition having the maximum evaluation value is detected as the tilt angle correction condition, it is possible to detect the tilt angle correction condition more suitable for epipolar line alignment.

[Brief description of the drawings]

FIG. 1 is a functional block diagram for explaining an embodiment.

FIG. 2 is a diagram showing a flow of operations focusing on one kind of candidate condition.

3A and 3B are diagrams for explaining left and right images stored in a first left and right memory.

4A and 4B are diagrams for explaining left and right images stored in a third left and right memory.

FIG. 5 is a diagram for explaining detection of a correction condition.

FIG. 6 is a diagram for explaining the phase difference fitting.

FIG. 7 is a diagram showing a flow of phase difference verification processing.

8 (A) and 8 (B) are diagrams for explaining a test point and a reference point.

9 (A) and 9 (B) are diagrams for explaining a test point and a reference point.

FIG. 10 is a functional block diagram for explaining the embodiment.

FIG. 11 is a diagram showing a flow of operations focusing on one type of candidate condition.

FIG. 12A and FIG. 12B are diagrams for explaining a target area and a target area image.

FIG. 13 is a diagram showing a flow of operations focusing on one type of candidate condition.

14A and 14B are diagrams for explaining the improvement of detection accuracy of a correction condition.

FIG. 15 is a functional block diagram for explaining the embodiment.

FIG. 16 is a functional block diagram for explaining the embodiment.

FIG. 17 is a functional block diagram for explaining the embodiment.

FIG. 18 is a diagram showing a flow of operations focusing on one kind of candidate condition.

FIG. 19 is a diagram showing a flow of operations focusing on one type of candidate condition.

FIG. 20 is a diagram showing a flow of operations focusing on one type of candidate condition.

FIG. 21 is a functional block diagram for explaining the embodiment.

FIG. 22 is a diagram showing a flow of operations focusing on height direction candidate conditions.

FIG. 23 is a functional block diagram for explaining the embodiment.

FIG. 24 is a diagram showing a flow of an operation focusing on the height direction candidate condition.

FIG. 25 is a diagram showing a flow of operations focusing on height direction candidate conditions.

FIG. 26 is a diagram showing a flow of operations focusing on tilt angle candidate conditions.

FIG. 27 is a diagram showing a flow of operations focusing on tilt angle candidate conditions.

FIG. 28 is a diagram showing a flow of operations focusing on a tilt angle candidate condition.

[Explanation of symbols]

 18: Correction condition detection device 28: Condition setting unit 30: Image conversion unit 32: Corresponding point search unit 34: Evaluation value calculation unit 36: Maximum value detection unit 46: Region cutout unit

Claims (16)

[Claims] 1. A correction condition detection device for detecting a correction condition for matching epipolar lines of left and right images, and a condition setting unit for sequentially setting a plurality of types of candidate conditions for detecting the correction condition, and a left and right image One of them is an image conversion unit that performs image conversion according to a candidate condition, a scanning direction, a gradation expression of pixels to be scanned, or
Different colors of pixels to be scanned are used to set multiple types of search conditions, and the left and right images after image conversion are used as left and right search images.
The left and right search images are scanned according to the search condition, and the pixel features of the left search image and the pixel features of the right search image are compared on the corresponding horizontal scanning lines, and the pixels of the left and right search images in which the difference between the pixel features is less than or equal to a threshold value. Is detected as a corresponding candidate point, a corresponding candidate point that is a common set in the set of corresponding candidate points detected for each search condition is detected as a corresponding point, and the corresponding point detection result is used to detect the phase difference of the left and right search images. By using the corresponding point search unit that detects the unknown point and the unknown point, and the detection rate of the phase difference known point in the left and right search images,
An evaluation value calculation unit that obtains an evaluation value of a candidate condition for each type of candidate condition, and a maximum value detection unit that detects the candidate condition that has the maximum evaluation value for each type of candidate condition as a correction condition A correction condition detection device for a stereo image, characterized by: 2. The stereo image correction condition detection device according to claim 1, wherein the corresponding point search unit sets the entire left and right images after image conversion to the left and right search images, and the evaluation value calculation unit sets the phase difference between the left and right search images. A correction condition detection device for a stereo image, wherein a detection rate of a clarification point is obtained for each candidate condition, and the detection rate is used as an evaluation value of the candidate condition. 3. The stereo image correction condition detecting device according to claim 1, wherein the left and right attention area images are sequentially cut out from a plurality of attention areas vertically dispersed on the left and right images after image conversion. The corresponding point search unit uses the left and right attention area images as the left and right search images, and the evaluation value calculation unit obtains the detection rate of the phase difference determination point for each left and right search image with the same candidate condition, and the candidate condition is A correction condition detection device for a stereo image, wherein an average value of detection rates obtained for the same left and right search images is used as an evaluation value of the candidate condition. 4. The stereo image correction condition detecting device according to claim 1, wherein the left and right attention area images are sequentially cut out from a plurality of attention areas vertically dispersed on the left and right images after image conversion. The corresponding point search unit uses the left and right attention area images as the left and right search images, and the evaluation value calculation unit obtains the detection rate of the phase difference determination point for each left and right search image with the same candidate condition, and the candidate condition is Of the detection rates obtained for the same left and right search images, a predetermined number of detection rates are sequentially selected from the top, and the average value of the detection rates obtained by the selection is used as the evaluation value of the candidate condition. Correction condition detection device. 5. A correction condition detecting device for detecting a correction condition for matching epipolar lines of left and right images, a condition setting section for sequentially setting a plurality of types of candidate conditions for detecting the correction condition, and m gradations. An image conversion unit for converting one of the left and right images of the image according to a candidate condition, a gradation conversion unit for converting the left and right images after the image conversion into left and right images of n gradations (m> n), and A level cutout unit that cuts out, from the left and right images, left and right images of one or more types of gradation levels of interest as left and right attention level images, a scanning direction, a gradation expression of pixels to be scanned, or
A plurality of types of search conditions are set by differentiating the colors of pixels to be scanned, the left and right focus level images are used as left and right search images, the left and right search images are scanned according to the search conditions, and left search is performed on the corresponding horizontal scanning lines. A set of corresponding candidate points detected for each search condition by detecting the pixels of the left and right search images where the difference between the pixel features is equal to or less than the threshold value by comparing the pixel features of the image with the pixel features of the right search image. In the corresponding point search unit that detects a corresponding candidate point that becomes a common set as a corresponding point in the above, and detects a known point and an unknown point of the phase difference in the left and right search images, and the phase difference in the left and right search images. Using the detection rate of the clarification point,
An evaluation value calculation unit that obtains an evaluation value of a candidate condition for each type of candidate condition, and a maximum value detection unit that detects the candidate condition that has the maximum evaluation value for each type of candidate condition as a correction condition A correction condition detection device for a stereo image, characterized by: 6. The stereo image correction condition detection device according to claim 5, wherein the level cutout unit extracts only the left and right images of one type of gradation level of interest from the left and right images after gradation conversion. The level cutout unit and the evaluation value calculation unit obtain the detection rate of the phase difference determination point in the left and right search images for each candidate condition, and use the detection rate as the evaluation value of the candidate condition. Image correction condition detection device. 7. The stereo image correction condition detection device according to claim 5, wherein the level cutout unit sequentially obtains left and right images of a plurality of types of gradation levels of interest from the left and right images after gradation conversion. The evaluation value calculation unit obtains the detection rate of the phase difference determination point for each of the left and right search images having the same candidate condition, and the average value of the detection rates obtained for the left and right search images having the same candidate condition is used as the candidate condition. A correction condition detection device for a stereo image, which is characterized by using an evaluation value. 8. The stereo image correction condition detection device according to claim 5, wherein the level cutout unit sequentially obtains left and right images of a plurality of types of gradation levels of interest from the left and right images after gradation conversion. The evaluation value calculation unit obtains the detection rate of the phase difference determination point for each of the left and right search images with the same candidate condition, and sequentially determines the detection rate for the left and right search images with the same candidate condition from the top. A correction condition detection device for a stereo image, wherein a detection ratio of a number is selected, and an average value of the detection ratios obtained by the selection is used as an evaluation value of a candidate condition. 9. A correction condition detecting device for detecting a correction condition for matching epipolar lines of left and right images, and a condition setting section for sequentially setting candidate conditions for detecting a height direction correction condition, and a left and right image One of them is an image conversion unit that performs image conversion according to a candidate condition, a scanning direction, a gradation expression of pixels to be scanned, or
Different colors of pixels to be scanned are used to set multiple types of search conditions, and the left and right images after image conversion are used as left and right search images.
The left and right search images are scanned according to the search condition, and the pixel features of the left search image and the pixel features of the right search image are compared on the corresponding horizontal scanning lines, and the pixels of the left and right search images in which the difference between the pixel features is less than or equal to a threshold value. Is detected as a corresponding candidate point, a corresponding candidate point that is a common set in the set of corresponding candidate points detected for each search condition is detected as a corresponding point, and the corresponding point detection result is used to determine the height direction in the left and right search images. The corresponding point search unit that detects the unknown point and the unknown point of the short-distance phase difference that causes a large amount of deviation of the phase difference and the phase difference determination point of the short-range phase difference that causes a large amount of deviation in the height direction in the left and right search images. An evaluation value calculation unit that calculates the detection rate and uses the detection rate to calculate the evaluation value of the height direction candidate condition, and the maximum value detection that detects the height direction candidate condition that has the maximum evaluation value as the height direction correction condition. Comprises a section and Fixed condition detection apparatus of the stereo image, wherein the door. 10. The stereo image correction condition detection device according to claim 9, wherein the corresponding point search unit sets the entire left and right images after image conversion to the left and right search images, and the evaluation value calculation unit sets the height in the left and right search images. Stereo characterized by detecting the detection rate of the phase difference determination point for each height direction candidate condition for the short-distance phase difference with a large amount of direction deviation and using the detection rate as the evaluation value of the height direction candidate condition. Image correction condition detection device. 11. The stereo image correction condition detection device according to claim 9, wherein a left and right attention area image is sequentially cut out from a plurality of attention areas vertically dispersed on the left and right images after image conversion. The corresponding point search unit uses the left and right target area images as the left and right search images, and the evaluation value calculation unit determines the height direction candidate condition for the short-distance phase difference in which the height direction shift amount is large in the left and right search images. The detection rate of the phase difference determination point is calculated for each of the left and right search images with the same height direction candidate condition. A stereo image correction condition detecting device. 12. The stereo image correction condition detecting device according to claim 9, wherein the left and right attention area images are sequentially cut out from a plurality of attention areas vertically dispersed on the left and right images after image conversion. The corresponding point search unit uses the left and right target area images as the left and right search images, and the evaluation value calculation unit determines the height direction candidate condition for the short-distance phase difference in which the height direction shift amount is large in the left and right search images. The detection rate of the phase difference determination point is calculated for each of the left and right search images with the same value, and a predetermined number of detection rates are sequentially selected from the top of the detection rates calculated for the left and right search images with the same height direction candidate condition, and the selection rate is obtained. A correction condition detection device for a stereo image, wherein the average value of the detection rates is used as an evaluation value of the height direction candidate condition. 13. A correction condition detection device for detecting a correction condition for matching epipolar lines of left and right images, and a condition setting section for sequentially setting candidate conditions for detecting a tilt angle correction condition, and one of the left and right images. And an image conversion unit for converting an image according to a candidate condition, a scanning direction, a gradation expression of pixels to be scanned, or
Different colors of pixels to be scanned are used to set multiple types of search conditions, and the left and right images after image conversion are used as left and right search images.
The left and right search images are scanned according to the search condition, and the pixel features of the left search image and the pixel features of the right search image are compared on the corresponding horizontal scanning lines, and the pixels of the left and right search images in which the difference between the pixel features is less than or equal to a threshold Is detected as a corresponding candidate point, a corresponding candidate point that is a common set in the set of corresponding candidate points detected for each search condition is detected as a corresponding point, and the tilt angle of the left and right search images is detected using the corresponding point detection result. Corresponding point search unit that detects known and unknown points of long-distance phase difference with large shift amount, and detection rate of phase difference known point with respect to long-distance phase difference with large shift amount of tilt angle in left and right search images And an evaluation value calculation unit that obtains an evaluation value of a tilt angle candidate condition using the detection rate, and a maximum value detection unit that detects the tilt angle candidate condition that has obtained the maximum evaluation value as a tilt angle correction condition. Become Fixed condition detection apparatus of the stereo image, wherein the door. 14. The stereo image correction condition detection device according to claim 13, wherein the corresponding point search unit sets the entire left and right images after image conversion to the left and right search images, and the evaluation value calculation unit sets the tilt angle in the left and right search images. Correction of a stereo image characterized in that the detection rate of the phase difference known point is obtained for each tilt angle candidate condition for a long-distance phase difference with a large deviation amount and the detection rate is used as an evaluation value of the tilt angle candidate condition. Condition detection device. 15. The stereo image correction condition detecting device according to claim 13, wherein the left and right attention area images are sequentially cut out from a plurality of attention areas vertically dispersed on the left and right images after image conversion. The corresponding point search unit uses the left and right target area images as the left and right search images, and the evaluation value calculation unit has the same tilt angle candidate condition for the long-distance phase difference in which the shift amount of the tilt angle is large in the left and right search images. A stereo image characterized in that the detection rate of the phase difference determination point is obtained for each of the left and right search images, and the average value of the detection rates obtained for the left and right search images having the same tilt angle candidate condition is used as the evaluation value of the tilt angle candidate condition. Correction condition detection device. 16. The stereo image correction condition detection device according to claim 13, wherein the left and right attention area images are sequentially cut out from a plurality of attention areas vertically dispersed on the left and right images after image conversion. The corresponding point search unit uses the left and right focus area images as the left and right search images, and the evaluation value calculation unit has the same tilt angle candidate condition for the long-distance phase difference in which the tilt angle shift amount is large in the left and right search images. The detection rate of the phase difference determination point is calculated for each of the left and right search images.A predetermined number of detection rates are sequentially selected from the higher order among the detection rates obtained for the left and right search images with the same tilt angle candidate condition, and the detection rate obtained by the selection is calculated. A correction condition detection device for a stereo image, wherein an average value of the above is used as an evaluation value of the tilt angle candidate condition.