JP2011185720A - Distance obtaining device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a distance obtaining device which can measure a distance with high accuracy based on a stereo image photographed with a stereo camera and reducing a processing time. <P>SOLUTION: The distance obtaining device includes: an image input means 8 for inputting a plurality of stereo image pairs photographed with the stereo camera; a plurality of first parallax detection means 11, 12, 13, and 14 for calculating a parallax from each image pair; a plurality of image interpolation means for interpolating pixels in a region of one image of each image pair containing the pixels used for detecting the parallax when the parallax detected by the first parallax detection means is in a predetermined range; a plurality of second parallax detection means for calculating the parallax, using the pixel interpolated region; a normalization means for normalizing based on a base line length of a stereo imaging system; a parallax similarity evaluation means 15 for making a parallax similarity evaluation, using a normalized parallax; and a distance detection means 16 wherein the parallax evaluation is made by making use of only two results of parallax detection 1 and parallax detection 3. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a distance acquisition device that detects a distance from an image captured using a plurality of stereo imaging systems to a subject.

2. Description of the Related Art A distance acquisition device (stereo camera) using a plurality of stereo imaging systems is known as a detection unit for a distance to a subject, a three-dimensional shape of a subject, and the like in portable device and in-vehicle device.
High accuracy has been demanded in measuring a distance from a camera to a subject from a stereo image taken by a stereo camera.
Patent Document 1 describes an imaging apparatus using a lens array having four lenses.
Non-Patent Document 1 describes a method of taking SSDs (sum of squares of luminance differences) by arranging a large number of cameras in a line.
Non-Patent Document 2 describes a technique for improving ranging accuracy by irregularly arranging lens arrays.
Japanese Patent Application Laid-Open No. 2004-228561 describes a method of performing sub-pixel parallax calculation by performing interpolation by interpolating images.
In Patent Document 3, a matching algo having a high distance resolution at a long distance, a matching algo having a low distance resolution and a high processing speed at a short distance, and an intermediate matching algorithm at an intermediate distance are used to achieve both processing speed and distance resolution. A technique for achieving this is described.
Patent Document 4 describes a method of calculating a subpixel parallax by calculating a positional deviation in units of pixels and then performing interpolation by interpolating images.
Patent Document 5 describes a technique that achieves both a reduction in thickness and resolution of an image sensor by using a microlens array.

It is a well-known technique to calculate a sub-pixel level parallax by parallax calculation. However, since the evaluation value of the evaluation function for calculating the parallax is calculated in pixel units after calculating the parallax in pixel units, the extreme value of the evaluation value is subtracted using an equiangular line or a quadratic function. Estimates pixel level parallax.
It is known that the method includes an error with respect to the original sub-pixel level parallax. Therefore, in Patent Document 4 or 6, after calculating the parallax in pixel units, the image is enlarged by pixel interpolation, It has been attempted to improve the accuracy of distance measurement by calculating sub-pixel level parallax using the enlarged image.
In addition, since the parallax error at the sub-pixel level adversely affects the accuracy of distance measurement as the distance increases, Patent Document 5 describes an algorithm for calculating parallax in an attempt to make the distance measurement accuracy uniform according to the large, medium, and small distances. It has changed.

However, a parallax calculation method for enlarging an image to calculate a sub-pixel level parallax or performing a distance measurement with high accuracy in the case of a long distance requires a large amount of calculation, and costs and time.
By the way, the distance acquisition device according to the present invention is intended to realize an ultra-small stereo camera by mounting a lens array on one image sensor. In this case, the base line length of the stereo camera is several millimeters, and when mounted on an automobile, the subject distance is about several tens of centimeters to about 2 meters.
The parallax in this case is already less than one pixel. The conventional technology has been developed on the assumption that the parallax when measuring distance is at least several pixels or more, so the method of constructing a stereo camera with a single image sensor using a lens array has its parallax calculation accuracy. The distance measurement accuracy calculation speed is not enough.
Furthermore, in the field of machine vision, it is often a specific application scene that recognizes some object such as measuring a slight step on the surface of a coin placed several centimeters ahead, in which case Real-time processing is often required along with distance measurement accuracy.

  The present invention has been made in view of such a situation, and it is an object of the present invention to provide a distance acquisition device that can perform distance measurement with high accuracy from a stereo image captured by a stereo camera and has a short processing time. To do.

In order to achieve the above-described object, the present invention performs high-precision distance measurement only in a predetermined parallax range (distance range) that requires high-precision distance measurement, and achieves both reduction in processing time. It was.
Specifically, the invention described in claim 1 includes an image input means for inputting a plurality of stereo image pairs made up of images picked up using substantially the same plurality of stereo image pickup systems, and parallax from each image pair. In a region of one image of each image pair including a plurality of first parallax detection means to be calculated and a parallax detected by the first parallax detection means within a predetermined range, including pixels used for the parallax detection When there are a plurality of image interpolation means for interpolating pixels, a plurality of second parallax detection means for calculating parallax using the pixel-interpolated region, and a parallax detected by the second parallax detection means Normalization means for normalizing the parallax detected by the first parallax detection means based on the baseline length of the stereo imaging system when there is no parallax detected by the second parallax detection means , Using normalized parallax A parallax similarity evaluation means for performing parallax similarity evaluation, if the disparity is similar, and having a distance detecting means for detecting the distance to the subject, via the use of an average of the normalized parallax.

According to a second aspect of the present invention, in the distance acquisition device according to the first aspect, the disparity similarity evaluation unit evaluates that the disparities are similar when the difference between the normalized disparities is smaller than a predetermined value. It is characterized by that.
According to a third aspect of the present invention, in the distance acquisition device according to the first aspect, the first parallax detection unit and the second parallax detection unit detect the parallax by detecting the extreme value of the correlation value between the images. The parallax similarity evaluation means detects and determines that the parallaxes are not similar when the difference of the normalized parallax is greater than a predetermined value, and the parallax having a value closer to the extreme value. It is determined that the normalized parallax is correct, and the distance detecting unit detects a distance to the subject based on the normalized parallax.
According to a fourth aspect of the present invention, in the distance acquisition device according to any one of the first to third aspects, the stereo imaging system is provided at a position facing the subject, and the plurality of lenses are substantially the same. A lens array having two or more lens pairs having a radius of curvature and different base lengths or base directions, and provided on the image plane side of the lens array, and substantially imaged by each of the plurality of lenses. Imaging means for capturing a compound eye image that is a set of reduced images of the subject, wherein the image input means extracts a plurality of single-eye image pairs from the compound eye image input from the imaging means. And

  The invention according to claim 5 is provided at a position facing the object, and each of the plurality of lenses has two or more lens pairs having substantially the same radius of curvature and different base lengths or base directions. A lens array; an imaging unit that is provided on an image plane side of the lens array, and that captures a compound eye image that is a set of reduced images of the subject that are substantially imaged by each of the plurality of lenses; and an input from the imaging unit Means for extracting a plurality of single-eye image pairs from the compound eye image, a plurality of first parallax detection means for calculating parallax from the plurality of single-eye image pairs, and the parallax detected by the first parallax detection means In the case of a predetermined range, a plurality of image interpolation means for interpolating pixels in one image area of each image pair including the pixels used for the parallax detection, and calculating the parallax using the pixel interpolated area Duplicate If there is a parallax detected by the second parallax detection means and the second parallax detection means, the parallax is detected. If there is no parallax detected by the second parallax detection means, the first parallax is detected. Normalization means for normalizing the parallax detected by the detection means based on a baseline length of a stereo imaging system having the lens array and the imaging means, and parallax similarity for performing parallax similarity evaluation using the normalized parallax When the parallax is similar, the evaluation unit includes a distance detection unit that detects a distance to the subject using an average of the normalized parallaxes.

According to a sixth aspect of the present invention, in the distance acquisition device according to the fifth aspect, the disparity similarity evaluation unit evaluates that the disparities are similar when the difference between the normalized disparities is smaller than a predetermined value. It is characterized by that.
According to a seventh aspect of the present invention, in the distance acquisition device according to the fifth aspect, the first parallax detection unit and the second parallax detection unit detect the parallax by detecting the extreme value of the correlation value between the images. The parallax similarity evaluation means detects and determines that the parallaxes are not similar when the difference of the normalized parallax is greater than a predetermined value, and the parallax having a value closer to the extreme value. It is determined that the normalized parallax is correct, and the distance detecting unit detects a distance to the subject based on the normalized parallax.

According to the present invention, even when a parallax including an error is measured, a distance measurement can be stably performed with high accuracy.
In addition, highly accurate distance measurement using an ultra-small imaging device is possible.

It is an outline top view including a control system of a distance acquisition device concerning one embodiment of the present invention. It is the outline | summary side view seen from the to-be-photographed object direction. It is a figure which shows the relationship between a to-be-photographed object distance and an MTF characteristic. It is a figure which shows the compound eye image by an imaging means. It is a flowchart which shows the production | generation process of a distance image. It is a flowchart of parallax evaluation. It is a flowchart of the parallax evaluation in another Example. It is a flowchart of the parallax evaluation in another Example. It is a flowchart of parallax calculation. It is a flowchart of pixel interpolation. It is a figure which shows the setting of a pixel interpolation area | region. It is explanatory drawing of interpolation pixel value calculation.

Embodiments of the present invention will be described below with reference to the drawings.
1 and 2 show an example of the configuration of the distance acquisition apparatus according to the present embodiment. In FIG. 1, it is assumed that there is a subject in the direction of the arrow, and a schematic cross-sectional view of a configuration in which the subject is imaged by a distance acquisition device is shown. FIG. 2 is a schematic diagram when the distance acquisition device is observed from the direction of the subject, and parts common to those in FIG. 1 are given common numbers.
In FIG. 1, reference numeral 1 denotes a lens array. The lens array 1 comprises two surfaces, a subject side surface and an image surface side surface, and a plurality of lenses are arranged in an array in the surface.
FIG. 1 shows a double-sided lens array in which lens surfaces are provided on both the object side and the image side. Reference numeral 1a indicates a lens provided on the object side surface, and reference numeral 1b indicates a lens provided on the image side surface. 1a and 1b are set to form an image of the object on the image surface. .

Reference numeral 2 denotes a light-shielding partition wall (referred to as a light-shielding wall) for preventing crosstalk (crosstalk) of light rays between adjacent lens sets in the lens array 1. Made of opaque material.
As shown in FIG. 2, the light shielding wall 2 has a rectangular hole corresponding to each lens set of the lens array 1, and the wall between the holes acts as a partition wall. The image side surface of the lens array 1 and the light shielding wall 2 are bonded.
Reference numeral 3 denotes an aperture array in which a circular hole is provided in the plate-like member corresponding to each lens set, and acts as a lens diaphragm.
The aperture array 3 and the lens array 1 are bonded to each other through a protrusion 1c provided on a plane portion of the surface on the subject side of the lens array 1. Reference numeral 4 denotes a CMOS sensor as an image pickup unit for picking up an image of a subject picked up by each lens set in the lens array 1, and is mounted on the substrate 5.
Reference numeral 6 denotes a housing, which is bonded to the object side surface of the lens array 1 to hold the lens array 1, the light shielding wall 2, and the opening array 3, and is bonded to the substrate 5.

The operation of the lens array 1 will be described. Each lens set 111, 121, 131, 141 constituting the lens array 1 has substantially the same radius of curvature, and the radius of curvature is set so as to focus on a subject beyond a predetermined distance. Yes.
Each lens set is configured to function as a stereo camera lens in a pair of two lens sets. For example, in FIG. 2, the lens set indicated by 111 and the lens set indicated by 131 constitute a lens set pair that acts as a stereo camera lens, and the CMOS sensor 4 located at a position corresponding to both lens sets. Together with the pixel area, a stereo camera as a stereo imaging system is configured.
Since the interval between the lens sets in a plane substantially perpendicular to the optical axis constituting the lens set pair is the baseline length of the stereo camera configured by the lens set pair, the baseline length of the stereo camera is B2. Similarly, the lens sets 121 and 141, 111 and 141, and 121 and 131 together with the pixel area of the CMOS sensor constitute stereo cameras with base line lengths of B1, B4, and B3. The baseline lengths B1 to B4 have different values.

FIG. 3 shows an example of through focus MTF by each lens set constituting the lens array. Reference numeral 7 denotes a through focus MTF corresponding to each lens set (in this case, an MTF change associated with a subject distance).
The through focus MTF 7 is designed so that an image is not blurred up to infinity over a certain distance (K).
In addition to setting the radius of curvature, the focal length of each lens set is designed to be the same for all lens sets, so the optical magnification of each lens set is roughly defined only by the subject distance. The image size of the set is constant. Therefore, it is not necessary to consider the difference in image size due to each lens set from the acquired compound eye images, and it is not necessary to consider the difference in image size even when some information recognition is performed using the acquired image.

As shown in FIG. 2, when the apparatus is observed from the direction of the subject, a circle indicated by a solid line in each lens represents an opening (effective diameter), and the diameter thereof is constant. A circle indicated by a broken line represents the diameter of the lens 1a including the invalid area, and a circle indicated by a dotted line represents the diameter of the lens 1b.
Next, an image acquired by the apparatus shown in FIG. 1 will be described. The image by the CMOS sensor 4 is a compound eye image as shown in FIG. In this case, a plane photograph is taken, and the subject is in this case at a certain distance in the distance. When a subject is present nearby, each single-eye image has a parallax in the vertical and horizontal directions depending on its position.
I11 to I41 in FIG. 4 are single-eye images corresponding to the respective lenses of the lens array. Since the number of lens sets is four and the number of lens set pairs is four, the number of single-eye images is four. There are four image pairs.
In FIG. 4, the area that is black around the single-eye image is an area that is shaded by the light shielding wall 2. Since the focal lengths of the lens sets are substantially equal, the image size and the in-focus state in each single-eye image are equal.

The compound eye image of FIG. 4 is captured by an image capture unit 8 as an image input unit, divided into single-eye image data I11 to I41, and transferred to the image correction unit 10. The image correction unit 10 corrects image distortion by using internal parameters and external parameters specific to the camera according to the working accuracy and assembly accuracy of the lens.
Camera-specific internal and external parameters can be obtained using Zhang's technique (“A flexible new technique for camera calibration”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22 (11): 1330-1334, 2000), etc. it can.
In the corrected image, the positional deviation in the vertical direction, the distortion caused by the lens, and the distortion caused by the distortion of the lens attached to the sensor are corrected. Therefore, a stereo pair image having a base line length of B3 or B4 is processed as an image photographed by a stereo camera having a base line length of substantially B2 and B1.

Next, the parallax detection unit 11 detects parallax from the stereo image pairs I21 and I41 (this operation is hereinafter referred to as “parallax detection 1”). In this case, the parallax detection unit 11 functions as a first parallax detection unit.
The parallax detection in the parallax detection units 12, 13, and 14 is referred to as parallax detection 2, parallax detection 3, and parallax detection 4, respectively.
In the images I21 and I41 that have undergone image correction, the corresponding points of the subject in each image are shifted in the horizontal direction according to the distance of the subject, and the amount of shift is detected. The amount of deviation corresponds to parallax.
The parallax detection 1 is demonstrated using the flowchart of FIG. First, a stereo image is input. In this case, the stereo image pair I21 and I41. Next, a target area for calculating parallax is set. This target area is a small area 9 of I21 in FIG.
Next, actual parallax calculation 1 is executed. The parallax calculation 1 uses a technique called block matching. In block matching, in order to search a region corresponding to the small region 9 of I21 set in the parallax calculation target region setting block, the small region 9 ′ is cut out from the vicinity of the region corresponding to the small region 9 of I41 to I21, An evaluation value for evaluating the degree of coincidence of the small areas is calculated.

This is a kind of correlation value. As evaluation values, the sum of luminance differences (SAD: Sum of Absolute Difference) between the two areas, the sum of squares of luminance differences (SSD), normalized cross correlation (NCC) Ask for. In this case, the position of 9 ′ is moved within a predetermined range, and the position shift between images is detected in units of pixels by searching for the position of 9 ′ that gives the minimum value of SAD or SSD or the maximum value of NCC at each position. Can be obtained.
Since parallax in units of pixels often lacks accuracy, parallax less than a pixel (sub-pixel) is also estimated. It is common to interpolate evaluation values using equiangular straight line fitting or parabolic fitting to estimate the true minimum value (in the case of SAD or SSD) or local maximum value (in the case of NCC) of the evaluation value.
Further, the data used for calculating the evaluation value may be a value obtained by differentiating the luminance (for example, DOG) instead of the luminance. Next, it is determined whether or not the calculated parallax is within a predetermined range. In other words, if the calculated parallax is within a predetermined range in order to detect a slight distance difference between objects at a desired distance or accurately detect the distance of an object far away, Perform accurate parallax calculation.
In this case, the parallax detection unit 11 functions as a second parallax detection unit.

Therefore, pixel interpolation is performed. In this case, the parallax detection unit 11 functions as an image interpolation unit. The reason for performing the pixel interpolation is that the parallax of the sub-pixel obtained in the parallax calculation 1 is estimated from the parallax evaluation value in units of pixels, and thus includes a certain amount of error, so that the accuracy is limited.
Rather than that, if you interpolate pixels and interpolate between pixels in units of 1/10 pixel, the parallax calculation is calculated with 1/10 pixel accuracy, and the accuracy of less than 1/10 pixel is estimated by the method of extreme value estimation of the evaluation value Obtainable.
For pixel interpolation, a method using the Sinc function is ideal, but it is more effective to use a method called cubic convolution that uses a polynomial expansion of the Sinc function and uses up to a third-order term. A linear interpolation method is also conceivable, but cubic convolution is better as a method of interpolating pixels.

This pixel interpolation will be described with reference to FIGS. In the flowchart of FIG. 10, first, the parallax D obtained by the parallax calculation 1 is acquired. As shown in FIG. 11, the parallax D means that when the standard block of the stereo right image and the reference block of the left image are shifted by D, the degree of coincidence of both blocks is maximized.
Therefore, a pixel interpolation area is set in order to further improve the accuracy of D. This is the area indicated by the broken line in FIG. This area is an area obtained by slightly expanding the area of the block that gives the extreme value of the evaluation value of the left image. For example, an area expanded by ± 0.5 pixels in each horizontal and vertical direction is set as a pixel interpolation area.
After setting the pixel interpolation area, an interpolation pixel value is calculated. As shown in FIG. 12, the pixel indicated by ■ means the pixel of the original image, and the pixel indicated by □ means the pixel to be interpolated. In this example, the pixels are interpolated in units of 1/10 pixels in the horizontal direction, and the pixels are interpolated in units of 1/5 pixels in the vertical direction.
The image correction unit 10 corrects the stereo image so that the positional deviation between the stereo images exists only in the horizontal direction, but in the case where it cannot be corrected still, the vertical direction is interpolated with 1/5 pixels. Of course, interpolation in the horizontal direction without interpolation in the vertical direction is also effective in terms of reducing the amount of calculation. Interpolation with 1/10 pixel in the vertical direction can also improve the accuracy of parallax detection. This should be determined by how much distance detection accuracy is required.

Next, as shown in FIG. 9, the parallax calculation 2 is executed. Similarly to the parallax calculation 1, the parallax calculation 2 uses block matching. In the parallax calculation 1, the block evaluation is performed by shifting the block position in units of ■ in FIG. 12. In the parallax calculation 2, the block evaluation is performed by shifting the block position in units of □.
In this case, it is possible to calculate parallax with higher accuracy by using POC (Phase Only Correlation), which is said to have high subpixel detection accuracy. However, since this calculation uses Fourier transform, the calculation amount is large.
Therefore, a parallax calculation method with a large amount of calculation that enables high accuracy only in a necessary range after calculating using a method capable of calculating parallax with a certain degree of accuracy at high speed and low cost as in the present invention. By using it, cost performance can be greatly improved. In addition, when parallax detection accuracy that does not require POC is required, a method such as SAD, SSD, or NCC used in the parallax calculation 1 may be used.
In this way, a highly accurate parallax for the region 9 was calculated. A region to be subjected to parallax calculation is sequentially moved and set in units of pixels to detect parallax. In this way, a highly accurate sub-pixel level parallax for a necessary area in the image is calculated and detected as a parallax image. Each pixel of the parallax image corresponds to each pixel of the image I21. Also, the parallax is set to 0 for the pixels for which the parallax could not be detected.

The above is the description of the parallax detection unit 11, but the configuration is the same from the parallax detection unit 12 to the parallax detection unit 14.
Further, parallax images are similarly calculated for three image pairs from stereo image pairs I11 and I31 to I11 and I41.
The outputs from the parallax detection 1 to the parallax detection 4 output not only the parallax image but also the evaluation value in each pixel used for obtaining the parallax.
In this way, since the parallax images in each stereo image pair and the evaluation values in each pixel are calculated, the parallax evaluation unit evaluates how likely each parallax is with the parallax evaluation unit 15 as a parallax similarity evaluation unit. Then, the parallax suitable for calculating the distance to the subject is calculated.
In the parallax evaluation, first, the parallax detected by the parallax detection 1 to the parallax detection 4 is normalized to a value that does not depend on the baseline length of the imaging system that images each stereo image pair.
In this case, the parallax evaluation unit 15 functions as a normalizing unit.
Specifically, the relationship between the parallax and the subject distance is as shown in Expression (1). Therefore, when Expression (1) is modified to Expression (2), normalized parallax independent of the baseline length of the imaging system is obtained. can get. That is, the parallax obtained from the parallax detection 1 to the parallax detection 4 may be divided by (base line length × focal length).
Subject distance = base length × focal length / (parallax × sensor pixel size) (1)
Normalized parallax = parallax / (base length × focal length) = 1 / (sensor pixel size × subject distance) (2)

In the parallax evaluation, the parallax evaluation can be easily performed by using only the result of the parallax detection 1 and the result of the parallax detection 3.
This is because I21 is commonly used, and since the parallax is detected based on I21, there is no positional shift of the corresponding parallax of each stereo image pair, so there is no need to correct the parallax position. .
In other words, it is only necessary to evaluate the parallax of a pixel (x, y) with a parallax detection 1 and the parallax of a pixel (x, y) at the same position of the parallax detection 3. The difference between the two normalized parallaxes is calculated, and when the absolute value of the difference is smaller than a predetermined threshold, the normalized parallax is averaged to obtain the parallax for calculating the distance.
When the absolute value of the normalized parallax difference is larger than a predetermined threshold, the evaluation values used for parallax detection are compared, and the evaluation values are closer to extreme values (when SAD or SSD is used) A parallax value that is a smaller value, which is a larger value when NCC or POC is used) is used as a correct normalized parallax for distance calculation by the distance calculation unit 16 as a distance detection unit.

The distance to the subject can be obtained by Expression (3) obtained by modifying Expression (2).
Subject distance = 1 / (sensor pixel size x normalized parallax) (3)
FIG. 5 shows a processing flow from extracting a single-eye image from a compound eye image to obtaining parallax detection, parallax evaluation, and a distance image using each stereo image pair. FIG. 6 shows an example of a parallax evaluation flow, and FIG. 7 shows another example.
As shown in FIG. 7, the parallax evaluation can be performed using only two results of the parallax detection 2 and the result of the parallax detection 4 to calculate the parallax used for the distance calculation. The parallax in this case is deviated from the parallax calculated using the results of the parallax detection 1 and the parallax detection 3 by (normalized parallax × baseline length × focal length) pixels in the vertical direction.
That is, it can be regarded as a stereo camera of the lens sets 111 and 121 with the distance between the optical centers of the lens set 111 and the lens set 121 as a base line length.
Further, the parallax detection results 1 and 3 and the parallax detection results 2 and 4 are compared with each other to compare the normalized parallaxes, thereby enabling more accurate distance detection. This will be described later with reference to FIG.

FIG. 8 shows an example of parallax evaluation using all parallaxes from parallax detection 1 to parallax detection 4. This will be described. The first half of the parallax evaluation is branched from the start and the ones shown in FIGS. 6 and 7 are performed in parallel. Then, based on the normalized parallax of each pixel calculated in FIG. 6, the corresponding pixel position in FIG. 7 is obtained.
In this case, how much the position is shifted in the vertical direction is calculated. Next, the normalized parallax at each corresponding pixel position is acquired. When the absolute value of the difference between the two normalized parallaxes is smaller than a predetermined threshold, the average of the normalization errors is taken.
When the value is larger than a predetermined threshold, a normalized parallax having an evaluation value closest to the extreme value of the parallax evaluation value is selected and used for subject distance calculation. When the process is executed for all pixels, this process ends.

  INDUSTRIAL APPLICABILITY The present invention can be applied to the field of information imaging devices using an optical system and an image sensor, the field of measuring devices for detecting distances and three-dimensional shapes, and various images such as industrial image sensors, vehicle-mounted image sensors, and monitoring image sensors It can be applied to sensors.

DESCRIPTION OF SYMBOLS 1 Lens array 4 CMOS sensor as an imaging means 8 Image capture part 11, 12, 13, 14 as image input means Parallax detection part 11, 12, 13, 14 as 1st parallax detection means Parallax as image interpolation means Detection unit 11, 12, 13, 14 Parallax detection unit as second parallax detection unit 15 Parallax evaluation unit as normalization unit 15 Parallax evaluation unit as parallax similarity evaluation unit 16 Distance calculation unit as distance detection unit

Japanese Patent Laid-Open No. 2003-143459 JP 2005-250994 A Japanese Patent No. 4053282 JP 2008-304202 A JP 2007-74709 A

`` A Multiple-Baseline Stereo '' Takeo Kanade, IEEE PAMI, Vol.15, No.4, April 1993 "Improvement of image quality and distance accuracy in a compact compound-eye camera using a single image sensor" Osaka University, October 21, 2008, Technical Report of the Institute of Image Information and Television Engineers

Claims (7)

Image input means for inputting a plurality of stereo image pairs consisting of images picked up using a plurality of substantially identical stereo image pickup systems;
A plurality of first parallax detection means for calculating parallax from each image pair;
A plurality of image interpolation means for interpolating pixels in one image region of each image pair including the pixels used for the parallax detection when the parallax detected by the first parallax detection means is in a predetermined range;
A plurality of second parallax detection means for calculating parallax using the pixel-interpolated region;
When there is a parallax detected by the second parallax detection unit, the parallax is detected. When there is no parallax detected by the second parallax detection unit, the parallax detected by the first parallax detection unit is Normalization means for normalization based on the baseline length of the imaging system;
Parallax similarity evaluation means for performing parallax similarity evaluation using normalized parallax;
Distance detection means for detecting the distance to the subject using the average of the normalized parallax when the parallax is similar;
A distance acquisition device comprising: The distance acquisition device according to claim 1,
The distance obtaining apparatus according to claim 1, wherein the disparity similarity evaluating unit evaluates that the disparities are similar when the difference between the normalized disparities is smaller than a predetermined value. The distance acquisition device according to claim 1,
The first parallax detection means and the second parallax detection means detect parallax by detecting the extreme value of the correlation value between images,
The disparity similarity evaluation unit determines that the disparities are not similar when the difference between the normalized disparities is greater than a predetermined value, and correctly normalizes the disparity having the extreme value closer to the extreme value Judged as parallax,
The distance acquisition device, wherein the distance detection means detects a distance to a subject based on the normalized parallax. In the distance acquisition device according to any one of claims 1 to 3,
The stereo imaging system is provided at a position facing a subject, a lens array including a plurality of lens pairs each having a plurality of lenses having substantially the same radius of curvature and different base lengths or base directions. An imaging unit that is provided on the image plane side of the lens array and that captures a compound eye image that is a set of reduced images of the subject that are substantially imaged by each of the plurality of lenses.
The distance acquisition apparatus, wherein the image input unit extracts a plurality of single-eye image pairs from a compound eye image input from the imaging unit. A lens array that is provided at a position facing a subject, and each of the plurality of lenses has substantially the same radius of curvature, and is composed of two or more lens pairs having different baseline lengths or baseline directions;
An imaging unit that is provided on the image plane side of the lens array and that captures a compound eye image that is a set of reduced images of the subject that are substantially imaged by each of the plurality of lenses;
Means for extracting a plurality of single-eye image pairs from a compound eye image input from the imaging means;
A plurality of first parallax detection means for calculating parallax from the plurality of single-eye image pairs;
A plurality of image interpolation means for interpolating pixels in one image region of each image pair including the pixels used for the parallax detection when the parallax detected by the first parallax detection means is in a predetermined range;
A plurality of second parallax detection means for calculating parallax using the pixel-interpolated region;
When there is a parallax detected by the second parallax detection unit, the parallax is detected. When there is no parallax detected by the second parallax detection unit, the parallax detected by the first parallax detection unit is Normalizing means for normalizing based on a baseline length of a stereo imaging system having a lens array and the imaging means;
Parallax similarity evaluation means for performing parallax similarity evaluation using normalized parallax;
Distance detection means for detecting the distance to the subject using the average of the normalized parallax when the parallax is similar;
A distance acquisition device comprising: The distance acquisition device according to claim 5,
The distance obtaining apparatus according to claim 1, wherein the disparity similarity evaluating unit evaluates that the disparities are similar when the difference between the normalized disparities is smaller than a predetermined value. The distance acquisition device according to claim 5,
The first parallax detection means and the second parallax detection means detect parallax by detecting the extreme value of the correlation value between images,
The disparity similarity evaluation unit determines that the disparities are not similar when the difference between the normalized disparities is greater than a predetermined value, and correctly normalizes the disparity having the extreme value closer to the extreme value Judged as parallax,
The distance acquisition device, wherein the distance detection means detects a distance to a subject based on the normalized parallax.

Images