JPH09152334A - Object detection apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an object detection apparatus by which the existence of an object can be detected precisely and stably and by which the computing amount of the mapping processing operation of a stereo image is reduced by a method wherein, in a stereo image processing operation which is used as an input method for three-dimensional information, a change, in terms of time, in a distance up to the object is detected. SOLUTION: In an object detection apparatus, the existence of an object is detected on the basis of a change in a distance in terms of time. In the object detection apparatus, the position of a plane such as a road, a floor, a wall or the like is estimated by a plane estimation part 17, the distance of a rectangular region in which a distance cannot be measured in the rectangular region of the plane is computed so as to be supplemented on the basis of a distance up to the plane in the rectangular region in which the distance can be measured. Thereby, the object can be detected stably and surely, and the computing amount of a mapping processing operation in a mapping processing part 7 can be reduced while a parallax search region is limited to a part between an imaging device and the plane.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an object detecting apparatus for monitoring an object, detecting an obstacle, etc. by a stereo image distance measuring method.

[0002]

2. Description of the Related Art FIG. 3 shows an object based on the theory described in "Driving Support System Using 3D Image Recognition Technology" by Keiji Mitsuyoshi et al. In the configuration diagram of the conventional object detection device for detecting,
The operation of this object detection device will be described.

The image taken by the left image pickup device 1 is stored in the left image memory 3 as a reference left image 2 (see FIG. 4). An image captured by the right imaging device 4 arranged so that its optical axis is parallel to the optical axis of the imaging device 1 is a right image memory 6 (see FIG. 5).
Is stored.

Therefore, the associating processing unit 7 first performs the processing shown in FIG.
As shown in FIG. 4, the left image 2 has M (M in the horizontal direction (X-axis direction in FIG. 4)) and N (M in the vertical direction (Y-axis direction in FIG. 4)).
× N) rectangular areas 8 and each rectangular area 8
Is further divided into (m × n) pixels 9 of m in the horizontal direction and n in the vertical direction, and a pixel 9 having the i-th brightness L _i among the (m × n) pixels 9 is divided. The detection is performed for each rectangular area 8.

In the right image 5, a rectangular area 8 of the left image 2 is also included.
Search range in which images similar to other images may exist
In FIG. 10, every time the search rectangular area consisting of (m × n) pixels of m in the horizontal direction and n in the vertical direction is moved by one pixel in the horizontal direction, the i-th brightness in the pixels of the search rectangular area is calculated. Pixels having a magnitude R _i are detected.

Then, each pixel 9 in the rectangular area 8 of the left image 2
And the brightness of the pixel at the position corresponding to the position of each pixel 9 of the rectangular area 8 of the left image 2 in the search rectangular area of the right image 5, and the image of the rectangular area 8 of the left image 2 is compared. The similarity evaluation value C of the right image 5 and the image in the search rectangular area is obtained by (Equation 1).

[0007]

(Equation 1)

As a result, the position of the search rectangular area in the right image 5 when the similarity evaluation value C of both images becomes the minimum (hereinafter referred to as "corresponding area") corresponds to the rectangular area 8 of the left image 2. Then, the parallax d is obtained from (Equation 2) from the deviation between the coordinate X of the rectangular area 8 of the left image 2 and the coordinate X _R of the corresponding area of the right image 5, and is output.

[0009]

D = X−X _R The similarity evaluation value C between the image of the rectangular area 8 of the left image 2 and the image of the search rectangular area of the right image 5 is calculated, and then the left image 2
The operation of obtaining the parallax d from the deviation between the coordinate X of the rectangular area and the coordinate X _R of the corresponding area of the right image 5 is sequentially performed on the rectangular area 8 of all the coordinates of the left image 2.

Therefore, the distance measuring unit 11 measures from the image pickup device 1 or the image pickup device 4 measured for each rectangular area 8 based on the parallax d obtained for each rectangular area 8 of the left image 2 in the association processing unit 7. Distance to object K (X, Y) [where 0 <X ≦
M, 0 <Y ≦ N] is obtained by the generally known (Equation 3), and is converted into an image showing the distance for each rectangular area 8 (hereinafter referred to as “distance image”) and output.

[0011]

(Equation 3)

However, 2a is the distance between the optical axis of the image pickup apparatus 1 and the optical axis of the image pickup apparatus 4, and f is the focal length of the lenses of the image pickup apparatus 1 and the image pickup apparatus 4. Then, the initial distance image memory 12 has a distance measuring unit 11 The distance image output from the measurement reference time t ₀ is stored as an initial distance image, and the distance image memory 13 stores the distance image from the distance measuring unit 11 at the measurement time t (t> t ₀ ) after the measurement reference time t _0. The output distance images are sequentially stored.

The distance difference detection unit 14 includes an initial distance image memory 12
The rectangular area 8 having the same coordinates in the initial distance image stored in the distance image memory 13 and the distance image stored in the distance image memory 13 are compared in the order of coordinates, and the rectangle having the coordinates in which the distance difference between the initial distance image and the distance image is equal to or greater than the threshold value. After the area 8 is identified, it is determined that a new object has appeared in the rectangular area 8 whose coordinates are equal to or greater than the threshold value or the existing object has disappeared, and the image 15 of the detected object is output.

[0014]

As described above, the conventional object detection device determines whether or not there is a rectangular area 8 in which a distance difference occurs between the initial distance image at the measurement reference time t _{0 and} the distance image at the time t. Although the object is detected by determining whether or not, in this case, it is premised that the initial distance image and the distance image are obtained in most of the rectangular area 8.

However, when the distance to an object such as a road, a floor, or a wall that does not have a characteristic shape and has a small change in brightness or brightness is obtained, a rectangular area in which the minimum value of the similarity evaluation value C is unclear. In some cases, 8 increases and an initial range image or range image cannot be obtained. In this case, it becomes difficult to measure the distance, and the object cannot be detected. However, if the initial distance image that is the reference for the distance measurement cannot be obtained, the false detection rate increases and the object is detected from the beginning. There was a serious problem that I could not do it.

Further, the associating processor 7 sets the parallax search range between the maximum value d _max and the minimum value d _{min of} the parallax d according to the distance at which the object whose distance is to be measured exists. 6
And FIG. 7), the parallax search range is set uniformly on the entire surface of the left image 2 and the right image 5.

However, in most cases, the object to be detected exists in front of the plane 16 such as the road, floor, or wall, that is, on the side of the image pickup devices 1 and 4, so that the angles of the image pickup devices 1 and 4 are different from each other. 6 is perpendicular to the plane 16, the minimum value d _min of the parallax d and the parallax d at the position of the plane 16 can be equivalently processed, but the angles of the image pickup devices 1 and 4 are as shown in FIG.
If the object is inclined by the angle θ with respect to the plane 16, the parallax search has to be performed uniformly even at a position farther than the space plane 16 where no object exists. For this reason, the calculation amount in the association processing unit 7, which has the largest calculation amount in the object detection device based on the stereo image distance measurement method, further increases, the distance measurement of the object becomes inefficient, and the false detection rate increases. There was a problem.

[0018]

The present invention has been made to solve the above problems, and the planes such as roads, floors and walls are planes that occupy most of the left image and the right image. It can be assumed that the distance K (X, Y) in the rectangular area of the coordinates where the distance on the plane cannot be measured is calculated from the distance K (X, Y) of a plurality of coordinates partially obtained from the plane by a minimum of two. By estimating and complementing by a multiplication method or the like, the position of the plane can be estimated, the omission of detection of the object is reduced, and the detection of the object can be performed stably.

Further, the parallax search range is set in accordance with the estimated position of the plane, and the parallax search range with less waste is set in the rectangular area of each coordinate, so that the correspondence with the largest amount of calculation in the stereo image processing is performed. The processing amount can be significantly reduced.

In the first aspect of the present invention, the position of the same plane is estimated from the distance to the object measured for each rectangular area obtained by dividing the image in the horizontal and vertical directions, and then the rectangular area of the plane is calculated. In the rectangular area where the distance could not be measured, the distance is calculated and complemented based on the distance to the plane of the rectangular area where the distance can be measured, or the image is divided into a plurality of horizontal and vertical rectangles. The distance to the object measured for each area is converted into a distance image showing the distance, and after estimating the position of the same plane from the distance image, the rectangle for which the distance image was not obtained in the rectangular area of the plane By calculating and supplementing the distance image of the area based on the distance image to the plane of the rectangular area where the distance image was obtained, the rectangular area where the distance could not be measured is reduced, and accurate and stable object detection can be performed. When you can do it To be stable at the same time the amount of processing when performed the object detection is to reduce.

Second, according to the present invention, after estimating the position of the same plane from the distance to the object initially measured for each rectangular area, the distance to the next and subsequent objects is measured by the imaging device and the plane. Between the image pickup device and the plane after estimating the position of the same plane from the distance to the object initially measured for each rectangular area. By detecting the displacement of the image and measuring the distance to the object, the plane can be estimated only at the beginning of the object detection, and the space where the object cannot exist such as the back side of the plane such as a road. Since it is not necessary to perform the associating process for, the calculation amount of the associating process in the associating processing unit can be reduced.

[0022]

Embodiments of the present invention will be described below with reference to the drawings. The same reference numerals as those in FIGS. 3 to 7 denote the same parts.

(Embodiment 1) FIG. 1 is a block diagram of an object detecting device according to a first embodiment of the present invention. The operation of the object detecting device will be described below.

The image photographed by the left imaging device 1 is stored in the left image memory 3 as a reference left image 2 (see FIG. 4). An image captured by the right imaging device 4 arranged so that its optical axis is parallel to the optical axis of the imaging device 1 is a right image memory 6 (see FIG. 5).
Is stored.

Therefore, the associating processing unit 7 first performs the processing shown in FIG.
As shown in, the left image 2 is divided into (M × N) rectangular areas 8 and each rectangular area 8 is divided into (m × n) pixels 9, and (m × n) The pixel 9 having the i-th brightness L _i among the pixels 9 is detected for each rectangular area 8.

In the right image 5, the rectangular area 8 of the left image 2 is also included.
Search range in which images similar to other images may exist
In 10, every time the search rectangular area composed of (m × n) pixels is moved by one pixel in the horizontal direction, the pixel having the i-th brightness R _i is detected from the pixels in the search rectangular area.

Then, each pixel 9 in the rectangular area 8 of the left image 2
And the brightness of the pixel at the position corresponding to the position of each pixel 9 of the rectangular area 8 of the left image 2 in the search rectangular area of the right image 5, and the image of the rectangular area 8 of the left image 2 is compared. The similarity evaluation value C with the image in the search rectangular area of the right image 5 is obtained by the equation (1), and the search rectangular area in the right image 5 when the similarity evaluation value C of both images becomes the minimum Is determined as an area corresponding to the rectangular area 8 of the left image 2 (hereinafter referred to as "corresponding area"), and the coordinate X of the rectangular area 8 of the left image 2 and the coordinate X _R of the corresponding area of the right image 5 are deviated. From parallax d
Is calculated by the equation (2) and is output.

The similarity evaluation value C between the image of the rectangular area 8 of the left image 2 and the image of the search rectangular area of the right image 5 is obtained, and then the coordinate X of the rectangular area of the left image 2 and the right image 5 are calculated. The operation of obtaining the parallax d from the deviation from the coordinate X _{R of the} corresponding area is sequentially performed on the rectangular area 8 of all the coordinates of the left image 2.

Therefore, the distance measuring unit 11 measures from the image pickup device 1 or the image pickup device 4 measured for each rectangular area 8 based on the parallax d obtained for each rectangular area 8 of the left image 2 in the association processing unit 7. Distance to object K (X, Y) [where 0 <X ≦
M, 0 <Y ≦ N] is obtained by a generally known equation (3), and converted into an image (hereinafter referred to as “distance image”) indicating the distance for each rectangular area 8 and output.

The plane estimation unit 17 uses the distance image corresponding to the plane 16 in the distance image of the rectangular area 8 output from the distance measurement unit 11 at the measurement reference time t ₀ , and uses the method such as the least squares method. By estimating the position of the plane 16 on the space, the distance image of the rectangular area 8 of the plane which is not output from the distance measuring unit 11 at the measurement reference time t ₀ is complemented and output.

The initial distance image memory 12 stores the distance image output from the plane estimation unit 17 as an initial distance image,
Further, the distance image memory 13 sequentially stores the distance images output from the distance measuring unit 11 at the measurement time t (t> t ₀ ) after the measurement reference time t ₀ .

Therefore, the distance difference detection unit 14 uses the initial distance image and the distance image memory 13 stored in the initial distance image memory 12.
Rectangular area 8 with the same coordinates as the distance image stored in
Are compared in the order of coordinates to identify the rectangular area 8 having coordinates whose distance difference between the initial distance image and the distance image is equal to or greater than the threshold TH, and whether a new object appears in the rectangular area 8 having coordinates equal to or greater than the threshold TH. It is determined that the existing object has disappeared, and the image 15 of the detected object is output.

As described above, in the stereo image distance measuring method, it is difficult to associate the rectangular areas 8 with each other.
Even with a plane such as a wall having few features, the distance measuring unit 11 estimates the position of the plane in the real space by a method such as the least square method using the distance image corresponding to the plane in the distance image of the rectangular area 8. By complementing the distance image of the rectangular area 8 that is not output at the measurement reference time t ₀ , it is possible to reliably detect the object due to the temporal distance change.

(Second Embodiment) FIG. 2 is a block diagram of an object detecting apparatus according to a second embodiment of the present invention. The operation of the object detecting apparatus will be described below. The same reference numerals as those in FIG. 1 denote the same parts.

As in the above-mentioned association processing unit 7, the association processing unit 18 first processes the left image 2 as shown in FIG.
It is divided into (M × N) rectangular areas 8 and each rectangular area 8 is divided into
After being divided into (m × n) pixels 9, the pixel 9 having the i-th brightness L _i among the (m × n) pixels 9 is detected for each rectangular area 8.

In the right image 5, the rectangular area 8 of the left image 2
Search range in which images similar to other images may exist
In 10, every time the search rectangular area composed of (m × n) pixels is moved by one pixel in the horizontal direction, the pixel having the i-th brightness R _i is detected from the pixels in the search rectangular area.

Then, each pixel 9 in the rectangular area 8 of the left image 2
And the brightness of the pixel at the position corresponding to the position of each pixel 9 of the rectangular area 8 of the left image 2 in the search rectangular area of the right image 5, and the image of the rectangular area 8 of the left image 2 is compared. The similarity evaluation value C with the image in the search rectangular area of the right image 5 is obtained by the equation (1), and the search rectangular area in the right image 5 when the similarity evaluation value C of both images becomes the minimum Is determined as an area corresponding to the rectangular area 8 of the left image 2 (hereinafter referred to as "corresponding area"), and the coordinate X of the rectangular area 8 of the left image 2 and the coordinate X _R of the corresponding area of the right image 5 are deviated. From the above, the parallax d at the measurement reference time t ₀ is obtained by the formula (Equation 2) and is output.

The similarity evaluation value C between the image of the rectangular area 8 of the left image 2 and the image of the search rectangular area of the right image 5 is obtained, and then the coordinates X of the rectangular area of the left image 2 and the right image 5 are calculated. The operation of obtaining the parallax d from the deviation from the coordinate X _{R of the} corresponding area is sequentially performed on the rectangular area 8 of all the coordinates of the left image 2.

Therefore, the distance measuring unit 11 is a correspondence processing unit.
In FIG. 18, the distance K (X, Y) from the image pickup device 1 or the image pickup device 4 to the object measured for each rectangular region 8 based on the parallax d obtained for each rectangular region 8 of the left image 2 [where 0 < X ≦
M, 0 <Y ≦ N] is obtained by a generally known equation (3), and converted into an image (hereinafter referred to as “distance image”) indicating the distance for each rectangular area 8 and output.

Then, the plane estimating unit 19 uses the distance image corresponding to the plane 16 in the distance image of the rectangular area 8 output from the distance measuring unit 11 at the measurement reference time t ₀ , by the method such as the least square method. By estimating the position of the plane 16 in the real space with, the distance measuring unit 11 complements and outputs the distance image of the rectangular area 8 of the plane that was not output at the measurement reference time t ₀ .
The initial distance image memory 20 is stored.

Then, at the measurement time t (t> t ₀ ) after the measurement reference time t ₀ , the associating processing unit 18 sets the parallax search range to the plane in the real space stored in the initial distance image memory 20. With the position limited to 16 positions, the similarity evaluation value C between the image of the rectangular area 8 of the left image 2 and the image of the search rectangular area of the right image 5 is obtained as described above, and then the rectangle of the left image 2 is calculated. The parallax d at the measurement time t is obtained from the deviation between the coordinate X of the area and the coordinate X _R of the corresponding area of the right image 5, and the parallax d is output.

Based on the parallax d obtained for each rectangular area 8 of the left image 2 in the association processing section 18, the distance measuring section 11 measures from the imaging device 1 or the imaging device 4 to the object measured for each rectangular region 8. Distance K (X, Y) [where 0 <X ≦ M, 0 <
Y ≦ N] is obtained by a generally known equation (3), converted into an image showing the distance for each rectangular area 8 (hereinafter referred to as “distance image”) and output, and the distance image memory 13 To memorize.

Therefore, the distance difference detection unit 14 detects the initial distance image and the distance image memory 13 stored in the initial distance image memory 20.
Rectangular area 8 with the same coordinates as the distance image stored in
Are compared in the order of coordinates to identify the rectangular area 8 having coordinates whose distance difference between the initial distance image and the distance image is equal to or greater than the threshold TH, and whether a new object appears in the rectangular area 8 having coordinates equal to or greater than the threshold TH. It is determined that the existing object has disappeared, and the image 15 of the detected object is output.

As described above, according to the present embodiment, since the distance measurement is not performed on the object located at a position farther than the estimated plane 16, the calculation processing amount in the association processing unit 18 can be significantly reduced.

An example has been described in which the position of the plane 16 is estimated by the plane estimation units 17 and 19 after the parallax d output from the association processing unit 7 or 18 is converted into a distance image by the distance measurement unit 11. The position of the plane 16 may be estimated by the plane estimation units 17 and 19 from the parallax d output from the association processing unit 7 or 18 without converting the distance measurement unit 11 into a distance image.

[0046]

As described above, according to the present invention,
The following effects are obtained. That is, roads, floors,
By estimating the position of the plane and complementing the distance on the plane where it is difficult to measure the distance by matching such as walls,
Since the range images of all the rectangular areas on the plane can be obtained, the object can be detected stably and reliably.

Further, by estimating the position of a plane such as a road, floor, or wall, and limiting the parallax search range to the front of the plane, the calculation amount is the highest in the object detecting apparatus using the stereo image processing method. The required calculation amount in the association processing unit can be significantly reduced.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of an object detection device according to a first embodiment of the present invention.

FIG. 2 is a configuration diagram of an object detection device according to a second embodiment of the present invention.

FIG. 3 is a configuration diagram of a conventional object detection device.

FIG. 4 is a diagram showing a relationship between a rectangular area and pixels in a left image.

FIG. 5 is an explanatory diagram showing a search range in the right image.

FIG. 6 is a diagram illustrating a parallax search range when the plane is perpendicular to the optical axis of the imaging device.

FIG. 7 is a diagram illustrating a parallax search range when the plane is inclined with respect to the optical axis of the imaging device.

[Explanation of symbols]

1, 4 ... Imaging device, 2 ... Left image, 3 ... Left image memory, 5 ... Right image, 6 ... Right image memory, 7, 18 ... Correlation processing unit, 8 ... Rectangular area, 9 ... Pixel, 10 ... Search Range, 11 ... Distance measuring unit, 12,20 ... Initial distance image memory, 13 ... Distance image memory, 14 ... Distance difference detection unit,
15 ... Object detection image, 16 ... Plane, 17, 19 ... Plane estimation unit.

Claims (4)

[Claims] 1. A stereo image distance measuring method for measuring a distance to an object on the principle of triangulation from deviations of object images in a plurality of images respectively output from a plurality of image pickup devices arranged at a predetermined interval. In the object detection apparatus using, after estimating the position of the same plane from the distance to the object measured for each rectangular area formed by dividing the image into a plurality of horizontal and vertical directions, the rectangular area of the plane In the object detection device, the distance of the rectangular area in which the distance cannot be measured is calculated and complemented based on the distance to the plane of the rectangular area in which the distance can be measured. 2. The distance to the object measured for each rectangular area formed by dividing the image into a plurality of horizontal and vertical directions is converted into a distance image indicating a distance, and the same distance image is obtained from the distance image. After estimating the position of the plane, the distance image of the rectangular area in which the distance image is not obtained in the rectangular area of the plane is calculated based on the distance image to the plane of the rectangular area in which the distance image is obtained. The object detection apparatus according to claim 1, wherein the object detection apparatus is complemented by the above. 3. After estimating the position of the same plane from the distance to the object initially measured for each of the rectangular areas, the distance to the object thereafter is measured between the imaging device and the plane. The object detection apparatus according to claim 1, wherein the object detection apparatus is performed only within a parallax search range between them. 4. The position of the same plane is estimated from the distance to the object initially measured for each of the rectangular regions, and then the object image is detected in a parallax search range limited between the image pickup device and the plane. The object detection device according to claim 1, wherein a deviation is detected and a distance to the object is measured.