JPH1155691A - Image processing unit, image processing method and transmission medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To attain highly accurate processing at a high speed in the image processing by the stereo method. SOLUTION: Images (1-5) photographed by a camera are respectively given to image input sections 100-1-100-5, stored in a built-in memory and read sequentially to cancel an aberration of the lens of the camera, a difference from a contrast resulting from the characteristic of each camera is corrected and the result is outputted to sum of absolute difference SAD circuits 200-1-200-4. The SAD circuits 200-1-200-4 calculate an absolute value of the difference from pixel values between a reference image 1 outputted from the camera and a reference image outputted from the other two cameras and provide an output. Sum of SAD SSAD circuits 30-1-300-4 provide an output of a result of applying block matching processing to outputs of the SAD circuits. Secondary approximation sections 400-1-400-4 apply secondary approximation to the data outputted from the SSAD circuits and provide an output. Minimum value selection sections 500, 600 and a result memory 700 select a minimum value of data outputted from the secondary approximation sections 400-1-400-4.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus, an image processing method, and a transmission medium, and more particularly, to an image processing apparatus, an image processing method, and a transmission medium for calculating a distance to a target point by a stereo method. About.

[0002]

2. Description of the Related Art The stereo method is a technique for measuring a distance to a target point using images taken from a plurality of viewpoints using the principle of triangulation.

FIG. 6 is a diagram showing an example of the configuration of a conventional image processing apparatus that measures the processing up to a target point using a stereo method.

In FIG. 1, memories 1-1 to 1-5 store images of one frame captured by cameras 1 to 5 (not shown), and read and output pixel data in a predetermined order. It has been made like that. Note that, for example, the camera 1 among the cameras 1 to 5 is set as a reference camera, and a matching process is performed by comparing an image of the reference camera with an image of another camera (reference camera). Is calculated according to the parallax.

[0005] A SAD (Sum of Absolute Difference) circuit 2 calculates a difference in pixel value between the reference image output from the memory 1-1 and each image output from the other memories 1-2 to 1-5. Is calculated. For example, SAD
The circuit 2 includes the memory 1-1, the memory 1-2, and the memory 1-1.
And the memory 1-3, the memory 1-1 and the memory 1-4, and the absolute value of the pixel value difference between the memory 1-1 and the memory 1-5 is output.

[0006] The SSAD (Sum of SAD) circuit 3
Based on the absolute value of the pixel difference between the cameras output from the circuit 2, the result of performing the block matching process between the cameras is output.

The minimum value detection section 4 detects and outputs the minimum value from the results output from the SSAD circuit 3.

The secondary approximation unit 5 approximates (interpolates) the minimum value output from the minimum value detection unit 4 and values before and after the minimum value by a quadratic function, and calculates the minimum value with higher accuracy. I have.

[0009] The memory 6 stores the minimum value output from the second approximation unit 5.

Next, the operation of the above conventional example will be described.

Now, it is assumed that the camera 1 (not shown) is arranged at the center, and the other cameras 2 to 5 (not shown) are arranged so as to surround it. At this time, the image output from the camera 1 is stored in the memory 1-1 as a reference image. The images output from the other cameras 2 to 5 are stored in the memories 1-2 to 1-5, respectively.

The SAD circuit 2 forms a pair of the reference image output from the memory 1-1 and the reference images output from the other memories 1-2 to 1-5, and the pixels between each pair are formed. Calculate and output the absolute value of the difference.

The SSAD circuit 3 performs a matching process on a predetermined pixel block (for example, a 5 × 5 block) from the absolute value of the difference between the pixels for each pair output from the SAD circuit 2. That is, the SSAD circuit 3 calculates a difference for each pixel block, and supplies the obtained result to the minimum value detection unit 4.

The minimum value detector 4 detects the minimum value from the difference values output from the SSAD circuit 3 and supplies the minimum value to the secondary approximation unit 5.

The quadratic approximation unit 5 approximates (interpolates) the minimum value output from the minimum value detection unit 4 and the data at two points before and after the minimum value with a quadratic curve to generate more accurate minimum value data. And output.

The memory 6 stores data output from the secondary approximation unit 5.

[0017]

By the way, in the processing based on the stereo method, it is necessary to solve a corresponding point problem of where each pixel of a reference image corresponds to a reference image. As a method for solving the corresponding point problem, a matching process is generally used. That is, by comparing the reference image and the reference image, portions having the highest matching (similarity) are associated with each other as corresponding points.

When searching for corresponding points by matching processing, the search range is determined in advance by calibration. This search range should theoretically exist along a straight line called an epipolar line, but in practice, it is often a curve due to lens aberrations and the like.

In order to solve such a problem, it is conceivable to list all search points and store them in a lookup table in advance. However, even when the lookup table created in this way is used, the grayscale value of each pixel may differ between cameras due to a difference in the type of lens or a difference in the aperture. Has a problem that the matching process cannot be performed accurately.

Therefore, in order to solve such a problem, a process of compressing the input image to 4 bits and emphasizing a portion where the gray value greatly changes to absorb a difference in the gray value is performed. ("A
Stereo Machine for Video-rate Dense Depth Mapping
and Its New Applications ”Takeo Kanade, AtsushiYos
hida, Kazuo Oda, Hiroshi Kano and Masaya Tanaka, P
roceedings of 15thComputer Vision and Pattern Reco
gnition Conference (CVPR). June 18-20, 1996, San Fr
ancisco. ").

However, in such a method, since the delicate difference in density of the original image is ignored, the delicate difference in density cannot be reflected on the result of the matching processing. Therefore, there is a problem that a correct matching processing result is obtained in a portion where the gray level value largely changes, but a correct matching processing result is not obtained in a portion where the gray level change is small.

Since it is desirable that the distance data obtained as a result of the matching process be obtained with high accuracy, in the above-described conventional example, the minimum value data obtained as a result of searching for the corresponding point is interpolated by a quadratic curve. And more accurate data.

However, in order to perform this interpolation processing, it is necessary to perform division which requires a high calculation cost. Therefore, conventionally, in order to reduce the circuit scale, the minimum value of the result obtained by the block matching is required. Interpolation processing has been performed only on (the output of the minimum value detection unit 4). In such a method, although the hardware scale can be reduced, interpolation processing must be performed after the minimum value is output, so that the entire processing is delayed, and as a result, it is difficult to speed up the processing. There was a problem that becomes.

The present invention has been made in view of the above situation, and enables accurate matching processing and speeds up the processing.

[0025]

According to a first aspect of the present invention, there is provided an image processing apparatus comprising: first input means for inputting an image supplied from one of a plurality of image pickup devices as a reference image; Second input means for inputting an image supplied from the apparatus as a reference image, matching means for performing a matching process between the reference image and the reference image input from the first and second input means, and matching means And a detecting means for detecting a minimum value from the data interpolated by the interpolating means.

According to a fifth aspect of the present invention, in the image processing method, a first input step of inputting an image supplied from one of the plurality of imaging devices as a reference image, and an image supplied from another imaging device. As a reference image, a matching step of performing a matching process between the reference image and the reference image input from the first and second input steps, and data obtained by the matching step. An interpolation step for performing interpolation and a detection step for detecting a minimum value from data interpolated by the interpolation step are provided.

According to a sixth aspect of the present invention, in the transmission medium, a first input step of inputting an image supplied from one of the plurality of image pickup devices as a reference image and an image supplied from another image pickup device are performed. A second input step of inputting as a reference image, a matching step of performing a matching process between the reference image and the reference image input from the first and second input steps, and interpolation of data obtained by the matching step And transmitting a program including an interpolation step for performing the interpolation step and a detection step for detecting a minimum value from the data interpolated by the interpolation step.

[0028] The image processing apparatus according to claim 8, wherein the first storage means for storing an image output from each imaging device;
A second storage unit that stores an order in which pixel data constituting an image stored in the storage unit is read, and a pixel that is stored in the first storage unit according to the order stored in the second storage unit. It is characterized by comprising reading means for reading data and converting means for converting the gradation of pixel data read by the reading means.

According to an image processing method of the present invention, a first storing step of storing images output from the respective image pickup devices and an order of reading out pixel data constituting the images stored in the storing step are stored. A second storage step;
A reading step of reading out the pixel data stored in the first storage step in accordance with the order stored in the second storage step; and a conversion step of converting the gradation of the pixel data read out in the reading step. It is characterized by having.

A transmission medium according to a twelfth aspect stores a first storage step of storing an image output from each imaging device and a first storage step of reading out pixel data constituting the image stored in the storage step. 2, a reading step of reading the pixel data stored in the first storage step in accordance with the order stored in the second storage step, and a gradation of the pixel data read in the reading step Is transmitted.

An image processing apparatus according to claim 1,
The image processing method according to any one of claims 1 to 6, and the transmission medium according to claim 6, wherein an image supplied from one of the plurality of imaging devices is input as a reference image, and an image supplied from another imaging device is input. Is input as a reference image, a matching process is performed between the input reference image and the reference image, data obtained by the matching process is interpolated, and a minimum value is detected from the interpolated data. For example, an image supplied from one of the five imaging devices is input as a reference image, images supplied from the other four imaging devices are input as reference images, and the input reference image and reference image are input. And a matching process is performed using the absolute value of the difference between pixels and block matching, and data obtained by the matching process is interpolated by quadratic approximation, and a minimum value is detected from the interpolated data.

An image processing apparatus according to claim 8, wherein
In the image processing method according to the first aspect and the transmission medium according to the twelfth aspect, an image output from each imaging device is stored, and an order for reading out pixel data constituting the stored image is stored. In accordance with the read order, the stored pixel data is read, and the gradation of the read pixel data is converted. For example, an image output from each imaging device is stored, and pixel data forming the stored image is
The order of reading out the aberrations of the lenses of each imaging device is stored, and the stored pixel data is read out in accordance with the stored order. Is converted so as not to cause.

[0033]

FIG. 1 is a block diagram showing a configuration example of an embodiment of the present invention.

In this figure, an image input unit 100-1
(First input unit) and image input units 100-2 to 100-5 (second input unit)
5 are respectively stored. Then, the data is read out in a predetermined order so as to cancel the aberration of the lens, and the pixel value is subjected to gradation conversion in accordance with the look-up table and output. A detailed configuration example of the image input units 100-1 to 100-5 will be described later with reference to FIG.

Here, the image output from the camera 1 is a reference image, and the images output from the other cameras 2 to 5 are reference images.

The SAD circuits 200-1 to 200-4 (matching means) calculate the absolute value of the pixel difference between the reference image and the two reference images. For example, the SAD circuit 200-1 stores the reference image (memory 10
0-1) and the images output from the cameras 2 and 3 (the images output from the memories 100-2 and 100-3), and the absolute value of the pixel difference between them is input. The value is calculated.
The details of the SAD circuits 200-1 to 200-4 will be described later with reference to FIG.

SSAD circuits 300-1 to 300-4
The (matching means) is configured to calculate and output a difference value for each pixel block composed of 5 × 5 pixels. The SSAD circuit 300-
Details of 1 to 300-4 will be described later with reference to FIG.

The quadratic approximation units 400-1 to 400-4 (interpolating means) receive the three differential values output from the SSAD circuits 300-1 to 300-4, respectively, A curve is obtained, and the minimum value of the quadratic curve is obtained and output. In addition,
Details of the quadratic curve approximation units 400-1 to 400-4 will be described later with reference to FIG.

The minimum value selection section 500 (detection means) selects and outputs the minimum value from the data output from the second approximation sections 400-1 to 400-4.

The minimum value selection section 600 (detection means) compares the new minimum value output from the minimum value selection section 500 with the previous minimum value stored in the result memory 700 (detection means). , The smaller one is selected and output.

The result memory 700 includes a minimum value selector 600
Is temporarily stored, and the stored minimum value is supplied to the minimum value selection unit 600.

By the cooperation of the minimum value selecting section 600 and the result memory 700, data having the minimum value is selected from a series of data output from the minimum value selecting section 500, and the result memory 700 is selected. Will be stored.

Next, a detailed configuration example of the image input unit 100 shown in FIG. 1 will be described with reference to FIG.

In this figure, a memory 101 (first storage means, reading means) stores image data output from a camera at an address corresponding to address data output from a counter 102. When reading the stored data, the memory 10
3 (second storage means) to read and output data from an address corresponding to the address data output from the second storage means. The selector 106 is connected to the counter 102 when the image data is input to the memory 101, and is connected to the memory 103 when the image data is output from the memory 101.

The counter 102 outputs address data when reading image data into the memory 101. For example, the counter 102 outputs address data in ascending order.

The counter 104 outputs address data in a predetermined order. Memory 103
Represents the address data output from the counter 104,
The data is converted and output so as to cancel the aberration of the lens. Note that the data for address conversion stored in the memory 103 is generated by using corresponding points as clues when performing calibration.

Further, a valid bit is added to the data stored in the memory 103. If the search range previously obtained for each camera exceeds the image area, the valid bit is set to "0". , The valid bit signal output from the memory 103 is "0". If the search range is within the area of the image,
The valid bit signal output from the memory 103 is set to "1".

Look-up table 105 (conversion means)
Stores data for performing gradation conversion, and reads and outputs data from an address corresponding to input data. The data stored in the look-up table 105 is such that when the same pattern is captured by the cameras 1 to 5, the outputs of all the cameras are the same. Therefore, this lookup table 1
05 corrects variations in the characteristics of each camera.

Subsequently, referring to FIG. 3, the SA shown in FIG.
A detailed configuration example of the D circuit 200 will be described. In addition,
Since all the SAD circuits 200-1 to 200-4 have the same configuration, the SAD circuit 200-1 will be described below as an example.

In this figure, a subtraction circuit 201-1 is
The pixel value of the reference image output from the camera 2 is subtracted from the pixel value of the reference image output from the camera 1 and output. The subtraction circuit 201-2 uses the camera 1
Is subtracted from the pixel value of the reference image output from the camera 3 from the pixel value of the reference image output from the.

The absolute value circuit 202-1 includes a subtraction circuit 201
The absolute value of the data output from -1 is calculated, and the obtained result is output. Further, the absolute value circuit 202-2 calculates the absolute value of the data output from the subtraction circuit 201-2, and outputs the obtained result.

When the valid bit of the camera 2 is "1", the selector 203-1 outputs the signal to the absolute value circuit 202.
-1 is selected and output. On the other hand, when the valid bit of the camera 2 is "0", the output of the absolute value circuit 202-2 is selected and output.

When the valid bit of the camera 3 is “1”, the selector 203-2 sets the absolute value circuit 202.
-2 is selected and output. On the other hand, when the valid bit of the camera 3 is "0", the output of the absolute value circuit 202-1 is selected and output.

The addition circuit 204 adds the output of the selector 203-1 and the output of the selector 203-2 and outputs the result.

When at least one of the valid bits of the camera 2 and the camera 3 is "1", the selector 206 selects and outputs the output of the adder circuit 204.
When both valid bits are “0”, data (for example, 511) output from the maximum value circuit 205 is selected and output.

Finally, referring to FIG. 4, the SS shown in FIG.
A detailed configuration example of the AD circuit 300 will be described.

In this figure, a delay line 301
Is configured to delay the absolute value of the difference between the input pixel values by the size of the block for block matching multiplied by the number of pixels in one line, and output the delayed value.

The subtraction circuit 302 includes a delay line 301
The data output from the delay line 301 is subtracted from the data that has not passed through, and the obtained result is output.

The addition circuit 303 adds the data output from the subtraction circuit 302 and the data output from the memory 304 and outputs the result. Memory 304
Is configured to delay the data output from the adding circuit 303 by one line and then output the delayed data.

The delay line 305 is connected to an adder 303
Is delayed by the number of pixels constituting one line and output. The subtraction circuit 306 subtracts the output of the delay line 305 from the output of the addition circuit 303 and outputs the result.

The addition circuit 307 adds the output of the subtraction circuit 306 and the output of the memory 308 and outputs the result. The memory 308 delays the output of the adder circuit 307 by the size of the block matching block and outputs it.

Next, the operation of the above embodiment will be described.

When images output from the cameras 1 to 5 are input to the embodiment shown in FIG. 1, the image input units 100-1 to 100-5 temporarily store the images,
The data is read out in a predetermined order, subjected to gradation conversion, and output.

That is, when the image data is supplied to the memory 101 shown in FIG. 2, the data is sequentially stored at the address corresponding to the address data output from the counter 102. In addition, such an operation
The images executed by all of the image input units 100-1 to 100-5 and output from the cameras 1 to 5 are stored in the memory 1
01 is sequentially stored at a predetermined address. At this time, the selector 106 has selected the counter 102 side.

When the storage of the image data is completed, the selector 106 is connected to the memory 103 side. Then, the output of the address data from the counter 104 is started. The memory 103 reads out predetermined data from an address corresponding to the address data output from the counter 104 and supplies the read data to the memory 101. Since the data stored in the memory 103 is set to read image data for each pixel and for each search disparity, as a result, even if the epipolar line to be searched is non-linear, It becomes possible to absorb lens aberrations and the like.

Further, since the valid bits are added to the data stored in the memory 103 as described above, when the search range obtained for each camera exceeds the image area, The valid bit is set to "0". If the search range does not exceed the area of the image, the bit is set to "1".

The image data output from the memory 101 is supplied to the look-up table 105. The lookup table 105 reads out data stored at an address corresponding to the image data output from the memory 101 and outputs the data. This lookup table 10
Since the data stored in 5 is set so as to correct the difference in gradation caused by the difference in the characteristics of each camera, the difference in brightness between the cameras can be absorbed as a result. .

The image data which has been subjected to the correction processing for the lens aberration and the characteristics of each camera in this manner is output from the image input units 100-1 to 100-5, and the SAD
The signals are supplied to the circuits 200-1 to 200-4.

The SAD circuits 200-1 to 200-4 are
The absolute value of the difference between the pixel values between the reference image data output from the image input units 100-1 to 100-5 and the two reference image data is calculated and output. The SAD circuit 2
The operations performed in 00-1 to 200-4 are appropriately changed according to the value of the valid bit.

That is, the corrected image data output from the image input section 100-1 is subtracted from the subtraction circuit 201-
1 and the subtraction circuit 201-2. Also,
The image data output from the image input unit 100-2 and the image input unit 100-3 are supplied to a subtraction circuit 201-1 and a subtraction circuit 201-2, respectively.

The subtraction circuit 201-1 subtracts the pixel value of the image of the camera 2 as the reference image from the pixel value of the reference image and outputs the result. In addition, the subtraction circuit 201-2 subtracts the pixel value of the image of the camera 3 as the reference image from the pixel value of the reference image and outputs the result.

The absolute value circuit 202-1 includes a subtraction circuit 201
The absolute value of the calculation result of -1 is calculated and output. Further, the absolute value circuit 202-2 calculates and outputs the absolute value of the calculation result of the subtraction circuit 201-2. As a result, the absolute value circuit 2
From 02-1, an image input unit 100-2 which is a reference image
, And the absolute value of the difference between the pixel from the image input unit 100-1 as the reference image. Further, the absolute value circuit 202-2 outputs the image input unit 100 as a reference image.
The absolute value of the difference between the pixel of the image from the image input unit 100-1, which is the reference image, and the image of −3 is output.

When the search range of the camera 2 is within the image area, the selector 203-1 sets the valid bit to “1”.
Therefore, the output of the absolute value circuit 203-1 is selected and output to the adder circuit 204. When the search range of the camera 2 exceeds the image area, the valid bit is set to “0”.
03-1 selects the output of the absolute value circuit 202-2 and outputs it to the adder circuit 204.

Similarly, the selector 203-2 is connected to the camera 3
If the search range is within the image area, the valid bit is set to "1", so that the output of the absolute value circuit 203-2 is selected and output to the addition circuit 204. When the search range of the camera 3 exceeds the area of the image, the valid bit is set to “0”. In this case, the selector 203-2 outputs the output of the absolute value circuit 202-1. Is selected and output to the addition circuit 204.

The adding circuit 204 adds the outputs of the selectors 203-1 and 203-2 and outputs the result. Further, the selector 206 selects and outputs the output of the addition circuit 204 or the output of the maximum value circuit 205 according to the state of the valid bits of the cameras 2 and 3. As a result, the output of the selector 206 produces the following output according to the state of the valid bits of the camera 2 and the camera 3.

(1) Both valid bits are "1" (valid)
, The absolute value circuit 202-1 and the absolute value circuit 2
02-2 is added as it is and output. (2) When one of the valid bits is "0" (invalid), the output of the absolute value circuit which is valid is doubled and output. (3) When both valid bits are “0”, the output of the maximum value circuit 205 (for example, the value 511 when the pixel data is 8 bits) is selected and output.

The outputs of the SAD circuits 200-1 to 200-4 are supplied to the SSAD circuits 300-1 to 300-4, respectively.

SSAD circuits 200-1 to 200-4
Performs, for example, a block matching process in units of a pixel block including 5 × 5 pixels, and outputs the obtained result.

That is, the SAD circuits 200-1 to 200-
4 is the absolute value of the pixel difference (| a _ij −b
_ij |) is supplied to the subtraction circuit 302 and also to the delay line 301, and is supplied to the subtraction circuit 302 after being delayed by the size of the block matching × the number of pixels in one line.

The subtraction circuit 302 subtracts the data delayed by the delay line 301 from the output data of the SAD circuit and outputs the result to the addition circuit 303.

The adding circuit 303 adds the data of one line before delayed by the memory 304 and the output of the subtracting circuit 302 and outputs the result. As a result, data (Σ _i | a _ij −b _ij |) subjected to one-dimensional processing of two-dimensional block matching is output from the addition circuit 303.

The delay line 305, the subtraction circuit 306,
The adder circuit 307 and the memory 308 execute the same processing as that of the preceding circuit. As a result, the adder 307 outputs data (デ ー タ_j Σ _i | a _ij −) subjected to two-dimensional block matching processing. b _ij |) will be output.

The data output from the SSAD circuits 300-1 to 300-4 are supplied to second approximation processing units 400-1 to 400-4, respectively.

The quadratic approximation units 400-1 to 400-4
And certain data S ₂ output from the SSAD circuit 300-1 to 300-4, it data S _1, S ₃ may be around 2
Interpolation is performed with the following curve to obtain more accurate minimum value data.

FIG. 5 is a diagram for explaining the operation of the quadratic approximation unit 400.

As shown in this figure, the SSAD circuit 300
When the outputs d _{1 to} d ₃ are input, the second-order approximation circuit 400
Generates a curve connecting d _{1 to} d ₃ by the following quadratic equation.

[0087] ^{_{S 1 = a · d 1 2}} + b · d 1 + c ··· (1) S 2 = a · d 2 2 + b · d 2 + c ··· (2) S 3 = a · d 3 2 + b・ D ₃ + c (3)

Here, if d ₁ = −1, d ₂ = 0, d ₃ = 1, the above equations (1) to (3) are as follows.

S ₁ = a−b + c (4) S ₂ = c (5) S ₃ = a + b + c (6)

When solving this, the minimum value d can be expressed by the following equation.

D = (S ₁ −S ₃ ) / {2 × (S ₁₋₂ · S ₂ + S ₃ )} (7)

Here, the range of d can be expressed as follows.

-0.5≤d≤0.5 (8)

Therefore, when each term is doubled, the following equation is obtained.

-1 ≦ 2 · d ≦ 1 (9)

By substituting equation (7) into equation (9), the following equation is obtained.

−1 ≦ (S ₁ −S ₃ ) / (S ₁ −2 · S ₂ + S ₃ ) ≦ 1 (10)

Normally, when an operation including division as shown in Expression (7) is performed, the calculation cost of hardware increases. However, here, the accuracy to be obtained is as small as 3 to 4 bits after the decimal point, and the range of the obtained result is clear as shown in Expression (10). That is, the position indicating the correct minimum value obtained by the quadratic curve approximation is within a range of ± 0.5 from the position of the obtained minimum value. Therefore, since the calculation can be performed by using the subtraction instead of the division, the calculation can be greatly simplified and the hardware resources can be reduced.

In the above configuration, the SSAD circuit 30
Second order approximation is applied to all data output from 0, but no problem arises because the minimum value selection unit 500 finally selects only the minimum value.

The data approximated quadratic in this way is:
It is supplied to the minimum value selection unit 500. Minimum value selection section 500
Selects the minimum value from the data output from the quadratic approximation units 400-1 to 400-4, and outputs it to the minimum value selection unit 600.

The minimum value selecting section 600 has a result memory 700
And the minimum value selection unit 5
The value compared with the value newly output from 00 is compared, and the smaller value is selected and output. The result memory 700 holds the value output from the minimum value selection unit 600. As a result, among the data obtained from all camera pairs (combinations of cameras corresponding to images input to each SAD circuit), the minimum data is selected and stored in the result memory 700. Become.

Since such minimum data has the highest matching degree, it is possible to obtain the most accurate parallax by using the data.

According to the above embodiment, the SSA
After the output of the D circuit 300 is quadratic approximated by the quadratic approximation unit 400, the minimum value is selected by the minimum value selection unit 500. Therefore, the apparatus can be pipelined and parallel distributed processing can be performed. It becomes possible. Therefore, it is possible to execute the calculation at high speed.

Further, since the memory 103 is provided so that data can be read out non-linearly, it is possible to cancel the influence of aberration of the lens. Further, the look-up table 105 is provided to correct an error in the gray value caused by the difference in the characteristics of each camera. As a result, it is possible to improve the measurement accuracy.

In the above embodiment, the measurement is performed using five cameras. However, it goes without saying that the present invention is not limited to such a case.

In the specification, the transmission medium is F
In addition to information recording media such as D and CD-ROM, network transmission media such as the Internet and digital satellites are included.

[0107]

According to the image processing apparatus of the first aspect, the image processing method of the fifth aspect, and the transmission medium of the sixth aspect, the image data is supplied from one of the plurality of imaging apparatuses. The input image is input as a reference image, an image supplied from another imaging device is input as a reference image, a matching process is performed between the input reference image and the reference image, and data obtained by the matching process. Is interpolated and the minimum value is detected from the interpolated data, so that the distance to the target point can be calculated at high speed in the stereo method.

An image processing apparatus according to claim 8, wherein
According to the image processing method according to the first aspect and the transmission medium according to the twelfth aspect, an image output from each imaging device is stored, and an order for reading out pixel data constituting the stored image is stored. According to the stored order, the stored pixel data is read and the gradation of the read pixel data is converted, so that the distance to the target can be accurately calculated in the stereo method. Becomes

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration example of an embodiment of the present invention.

FIG. 2 is a block diagram showing a detailed configuration example of an image input unit 100 shown in FIG.

FIG. 3 is a block diagram showing a detailed configuration example of a SAD circuit 200 shown in FIG.

FIG. 4 is a block diagram illustrating a detailed configuration example of an SSAD circuit 300 illustrated in FIG. 1;

FIG. 5 is a diagram for explaining processing of a secondary approximation unit 400 shown in FIG. 1;

FIG. 6 is a block diagram illustrating a configuration example of a conventional image processing apparatus.

[Explanation of symbols]

100-1 image input unit (first input means), 100
-2 to 100-5 image input units (second input means),
101 memory (first storage means, reading means),
103 memory (second storage means), 105 lookup table (conversion means), 200-1 to 200
-4 SAD circuit (matching means), 300-1 to 300-4 SSAD circuit (matching means), 5
00,600 Minimum value detection unit (detection means), 700
Result memory (detection means)

Claims (13)

[Claims] An image processing apparatus that calculates a distance to a target point by image processing based on images captured by a plurality of imaging devices, wherein an image supplied from one of the plurality of imaging devices is A first input unit for inputting as a reference image, a second input unit for inputting an image supplied from another imaging device as a reference image, and a reference image input from the first and second input units. A matching unit that performs a matching process with a reference image; an interpolation unit that interpolates data obtained by the matching unit; and a detection unit that detects a minimum value from the data interpolated by the interpolation unit. Characteristic image processing device. 2. The image processing apparatus according to claim 1, wherein said interpolation means performs interpolation by quadratic approximation. 3. The second input means inputs reference images from two or more imaging devices, and the matching means includes a reference image input from the second input means and the reference image. The interpolation means performs data interpolation for each of the reference images, and the detection means detects a minimum value from data of all interpolated reference images. The image processing apparatus according to claim 1. 4. The second input means inputs reference images from two or more imaging devices, and the matching means combines two predetermined reference images input from the second input means. Generating a reference image pair, performing a matching process between each of the reference image pairs and the reference image, interpolating the data for each reference image pair, and detecting the interpolation 2. The image processing apparatus according to claim 1, wherein a minimum value is detected from the data for all the reference image pairs that have been subjected to the processing. 5. An image processing method for calculating a distance to a target point by image processing based on images picked up by a plurality of image pickup devices, wherein an image supplied from one of the plurality of image pickup devices is A first input step of inputting as a reference image, a second input step of inputting an image supplied from another imaging device as a reference image, and a reference image input from the first and second input steps. A matching step of performing a matching process with a reference image; an interpolation step of interpolating data obtained by the matching step; and a detection step of detecting a minimum value from the data interpolated by the interpolation step. Characteristic image processing method. 6. A transmission medium for transmitting a computer program used in an image processing device that calculates a distance to a target point by image processing based on images captured by a plurality of imaging devices, wherein the plurality of imaging devices are A first input step of inputting an image supplied from one of the above as a reference image; a second input step of inputting an image supplied from another imaging device as a reference image; A matching step of performing a matching process between the reference image and the reference image input from the second input step; an interpolation step of interpolating data obtained by the matching step; A transmission medium for transmitting a program comprising a detecting step of detecting a value. 7. An image processing method for storing a program transmitted from the transmission medium according to claim 6, and performing image processing using the program. 8. An image processing apparatus for calculating a distance to a target point by image processing based on images picked up by a plurality of image pickup devices, wherein first storage means for storing images output from each image pickup device. A second storage unit that stores an order in which pixel data constituting an image stored in the storage unit is read; and a first storage unit that stores the order in which the pixel data is stored in the second storage unit. An image processing apparatus, comprising: reading means for reading stored pixel data; and converting means for converting a gradation of pixel data read by the reading means. 9. The image processing apparatus according to claim 8, wherein the data stored in the second storage unit is generated according to a lens aberration of each imaging device. 10. The image processing apparatus according to claim 8, wherein said conversion means is set so that all of said plurality of imaging devices have the same gradation. 11. An image processing method for calculating a distance to a target point by image processing based on images captured by a plurality of imaging devices, wherein a first storage step of storing images output from each imaging device A second storage step of storing an order in which pixel data constituting an image stored in the storage step is read; and a first storage step according to the order stored in the second storage step. An image processing method, comprising: a reading step of reading stored pixel data; and a conversion step of converting a gradation of pixel data read by the reading step. 12. A transmission medium for transmitting a computer program used in an image processing device that calculates a distance to a target point by image processing based on images captured by a plurality of imaging devices, the output being from each of the imaging devices. A first storage step of storing an image to be read, a second storage step of storing an order in which pixel data constituting the image stored in the storage step is read out, and an order stored in the second storage step A transmission medium for transmitting a program, comprising: a read step of reading pixel data stored in the first storage step, and a conversion step of converting the gradation of pixel data read by the read step, according to . 13. An image processing method for storing a program transmitted from the transmission medium according to claim 12, and performing image processing using the program.