JP2000353300A - Object recognizing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To properly recognize an object without being affected by pitching or rolling in an on-vehicle object recognizing device using a stereocamera. SOLUTION: This device is provided with on-vehicle stereocameras 3 and 3' for photographing a traveling road surface, a measuring means for dividing pictures obtained by the stereocameras 3 and 3' into windows and measuring the distance to an object for each window, an estimated distance storing means for storing an estimated distance to the road surface preliminarily decided for each window, a detecting means for detecting the pitching and rolling angles of the vehicle, and a correcting means for correcting the estimated distance on the basis of the pitching angle and rolling angle detected by the detecting means. Thus, the object nearer than the road surface can be recognized on the basis of the corrected estimated distance.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an optical object recognition apparatus for detecting an object ahead using a stereo camera mounted on a vehicle such as an automobile. The present invention relates to an object recognition device that recognizes an object using a plurality of windows.

[0002]

2. Description of the Related Art In recent years, in order to improve the safety of running a vehicle, the distance and size of an object in front of the vehicle are determined.
There has been proposed a device for appropriately controlling a vehicle in response to this.

A technique related to a method of determining whether a subject whose distance has been detected is an object or a road area (including characters / white lines on a road surface) by using an optical distance measuring device including two light receiving elements is a special technique. In Japanese Unexamined Patent Publication No. Hei 7-225126,
An on-road object determination device capable of correctly recognizing an object ahead of a vehicle is described. This apparatus includes a stereo camera that captures an image of a vehicle on a road surface, divides an image obtained by the camera into a plurality of windows, and calculates a distance to a subject for each window. An object in front of the vehicle is recognized by comparing the distance to the subject with a reference distance determined for each row range of the window.

[0004]

Problems to be Solved by the Invention
No. 6, the road surface is present horizontally in front of the vehicle, the subject imaged by the camera is determined whether the object or road area, the vehicle is pitching or rolling, When traveling on a slope, it is assumed that the vehicle is parallel to the road surface, and a deviation occurs between the estimated road surface determined by parameters such as the camera mounting position and the depression angle, and the actual road surface. In some cases, it is not possible to accurately determine whether the distance value is a distance value to the road surface, and an object may be erroneously determined.

It is an object of the present invention to provide a device which can accurately recognize an object in front even if the host vehicle is tilted by pitching or rolling.

[0006]

In order to solve the above-mentioned problems, an object recognition apparatus according to the first aspect of the present invention is mounted on a vehicle and has a stereo camera for photographing a traveling road surface, and a window for displaying an image obtained from the stereo camera. Measuring means for measuring the distance to the object for each window, estimated distance storage means for storing an estimated distance to a predetermined road surface for each window, and pitching angle detection for detecting the pitching angle of the vehicle Means, correction means for correcting the estimated distance based on the pitching angle detected by the pitching angle detection means, and recognition means for recognizing the object based on the corrected estimated distance.

According to the first aspect of the present invention, the estimated distance to the road surface is corrected according to the pitching angle even when the vehicle is tilted in the front-rear direction due to acceleration, deceleration, etc., that is, when pitching occurs. The object can be properly recognized.

According to a second aspect of the present invention, there is provided an object recognition apparatus mounted on a vehicle, which divides an image obtained from the stereo camera into windows, and measures a distance to an object for each window. Measuring means, an estimated distance storing means for storing an estimated distance to a predetermined road surface for each window, a rolling angle detecting means for detecting a rolling angle of the vehicle, and the rolling angle detecting means. A correction unit configured to correct the estimated distance based on a rolling angle; and a recognition unit configured to recognize the object based on the corrected estimated distance.

According to the second aspect of the present invention, the estimated distance to the road surface is corrected in accordance with the rolling angle even when the vehicle inclines laterally, that is, when the vehicle rolls when traveling on a curve. Can recognize the object.

[0010]

Embodiments of the present invention will now be described with reference to the drawings. FIG. 1 is an overall block diagram of an object recognition apparatus according to one embodiment of the present invention. FIG. 2 is a view for explaining the principle of measuring the distance by the triangular measurement method used in this embodiment. First, a method of measuring a distance using a pair of imaging devices will be described with reference to FIG.

A line sensor 21 and a lens 23 constituting one of a pair of image pickup devices of a stereo camera are connected to a line sensor 22 and a lens 24 constituting the other image pickup device.
And a predetermined interval, that is, an interval in the left-right direction by the base line length B. Line sensors 21 and 22 are
It is typically a one-dimensional CCD, and may be an array of linearly arranged photosensors. Considering night use, it is preferable to use an imaging device using infrared rays. In this case, an infrared transmissive filter is placed in front of the lenses 23 and 24, and the object 20 is periodically emitted using an infrared light source.
It is preferable that the line sensors 21 and 22 detect infrared rays reflected from the object 20.

The line sensors 21 and 22 are arranged at the focal length f of the lenses 23 and 24, respectively. Lens 2
An image of the object located at a distance a from the plane at which the lines 3 and 24 are located is formed at a position shifted by X1 from the optical axis of the lens 23 in the line sensor 21, and is shifted by X2 from the optical axis of the lens 24 in the line sensor 22. Is formed, the distance a from the surfaces of the lenses 23 and 24 to the object 20 is obtained by a = B · f / (X1 + X2) according to the principle of the triangulation measurement method.

In this embodiment, since the image is digitized, the distance (X1 + X2) is digitally calculated. While shifting one or both of the images obtained by the line sensors 21 and 22, the sum of the absolute values of the digital values indicating the luminances of the corresponding pixels of the two images is calculated, and this is used as the correlation value. The shift amount of the image when the correlation value becomes the minimum value indicates the displacement between the two images, that is, (X1 + X2). Ideally, as shown in FIG. 2, the distance over which the two images must be relatively moved to overlap the two images obtained from the line sensors 21 and 22 is (X1 + X2).

Here, for simplicity, the description has been given assuming that the imaging device is one-dimensional line sensors 21 and 22.
As described below, in one embodiment of the present invention, a two-dimensional C
A CD or two-dimensional photosensor array is used as the imager. In this case, when the two-dimensional images obtained from the two imaging devices are relatively shifted and the same correlation calculation as described above is performed, and the shift amount when the correlation value becomes the minimum is obtained, the shift amount becomes (X1 + X2).

The image pickup means 3 in FIG. 1 corresponds to one of the image pickup means comprising the lens 23 and the line sensor 21 in FIG. 2, and the image pickup means 3 'corresponds to the other image pickup means comprising the lens 24 and the line sensor 22 in FIG. Corresponding to the means. In this embodiment, as shown in FIG. 3B, the imaging area is divided into a plurality of windows (small areas) W ₁₁ , W ₁₂ ,..., And the distance is measured for each window. An entire two-dimensional image is required. For this reason, the imaging means 3, 3 '
It consists of a two-dimensional CCD array or a two-dimensional photosensor array.

FIG. 3A shows an example of an image of another vehicle traveling in front of the own vehicle by the imaging means 3 or 3 '. FIG. 3B shows an example of an image. Is conceptually divided into a plurality of windows. FIG. 3B shows rows and columns in the vertical direction and is divided into windows of 10 rows × 15 columns for simplicity. Each window is numbered, for example W ₁₂
Indicates a window in one row and two columns.

The image of the object imaged by the imaging means 3, 3 'is converted into an analog / digital converter (A / D converter) 4,
At 4 ', the image data is converted into digital data.
Is stored in By the window cutout unit 13, an image portion corresponding to the window W ₁₁ is sent to the correlation calculation section 6 cut out respectively from the image memory 5 and 5 '. The correlation calculator 6 shifts the two cut-out images by a predetermined unit and performs the above-described correlation calculation to determine the shift amount when the correlation value is minimized. The shift amount is (X1 + X2). The correlation calculator 6 sends the value of (X1 + X2) thus obtained to the distance calculator 7.

The distance calculation unit 7 calculates a = B · f /
(X1 + X2) using the formula, obtaining the distance a ₁₁ to the object in the window W _11. The distance a ₁₁ thus obtained is stored in the distance storage unit 8. Similar calculation processing is sequentially performed for each window, and the distances a ₁₁ , a ₁₂ ,... Are stored in the distance storage unit 8. Hereinafter, the distance to the target calculated for a certain window is referred to as a measured distance of the window.

Since the resolution of the image data used in the above correlation calculation is determined by the pitch of the elements of the image sensor array, when a light receiving element having a relatively large pitch such as a photo sensor array is used, interpolation calculation between the pitches is performed. It is preferable to perform a process of increasing the density of the image data by performing the correlation calculation on the image data having the increased density.

The estimated distance storage unit 32 stores a distance (hereinafter, referred to as a distance) determined in advance for each window with respect to a horizontal road surface image.
Estimated distance). The road surface removing unit 31
Compare the estimated distance for each window with the distance for each corresponding window actually measured as described above, and determine the window with the measured distance close to the estimated distance, that is, the measured distance that is approximately equal to the distance to the road surface. A window with a measured distance greater than or equal to the estimated distance,
That is, the measured distance value of the window having the measured distance farther than the road surface is deleted from the distance storage unit 8. Thus, data such as a sign character on the road surface is deleted from the distance storage unit 8.

The estimated distance refers to the distance from the vehicle to the road surface when the vehicle is parallel to a horizontal road surface without tilting, and the road surface at this time is referred to as an estimated road surface. The estimated distance is calculated in advance based on the mounting position of the CCD camera, the depression angle, the base line length, the focal length, the CCD size, and the position of the window in the image, and is stored in the estimated distance storage unit 32 for each window.

In the above example, the measured distance value of the window determined to be the road surface is deleted from the distance storage unit 8. However, without deleting these data, for example, a flag for identifying that the road surface has been determined is used. Alternatively, an identification flag may be set on a window determined as a road surface and stored in the distance storage unit 8.

The pitching angle detecting section 9 detects the inclination of the vehicle in the front-rear direction, that is, the pitching, when the rear portion of the vehicle sinks due to acceleration of the vehicle or the vehicle front portion sinks due to deceleration. An angle, that is, a pitching angle is detected. Methods for obtaining the pitching angle include a method for directly obtaining the pitching angle using a vehicle height sensor, a method for obtaining an infinity point by detecting a white line, and obtaining it from its variation, and a method for estimating the distance from the distance to the white line. Alternatively, the correlation between the acceleration and the pitching angle may be obtained in advance as experimental data, and the pitching angle may be obtained based on the detection of the acceleration.

The correcting section 11 corrects the estimated distance based on the pitching angle thus obtained. FIG. 4 shows the correction of the estimated distance to the estimated road surface when the vehicle decelerates and pitching occurs in which the front part sinks. If the pitching angle is θ, the imaging device images the object downward by θ, so the estimated distance for each window to the estimated road surface, which is compared with the measured distance for each window to the object, is also downward by θ. It is necessary to correct as if the image was taken.

Assuming that the estimated distance to the estimated road surface in a certain window is d as shown in FIG. 4, the corrected estimated distance d 'is obtained by the following equation.

[0026]

(Equation 1)

In this equation, h is the mounting height of the camera, l
(L) is the length from the center of the body to the camera mounting position, d
Is the distance to the estimated road surface, d 'is the corrected distance to the estimated road surface, and θ is the pitching angle.

The road surface constituted by the estimated distance corrected in this way is inclined upward by θ from the original estimated road surface as shown by the corrected estimated road surface in FIG.

The rolling angle detecting unit 10 receives a force directed to the outside of the curve by the centrifugal force when the vehicle travels on the curve, and tilts in the lateral direction under the influence of the force, that is, determines the angle (rolling angle) at the time of rolling. To detect.
The rolling angle can be directly obtained using a vehicle height sensor, or the correlation between the vehicle speed and the yaw rate and the rolling angle may be obtained in advance as experimental data, and the rolling angle may be detected from the vehicle speed and the yaw rate. it can. In addition, it can be obtained by estimating from the distance value to the white line.

The correction unit 11 includes a rolling angle detection unit 10
Corrects the estimated distance in response to detecting the rolling angle. FIG. 5 shows that when the vehicle is tilted to the right by α degrees in the drawing, the estimated road surface needs to be tilted to the upper right by the angle α in the same relationship as in FIG. At this time,
The point at a distance from the camera in the horizontal direction x of the estimated road surface must be corrected to a position dy above the estimated road surface.

FIG. 6 is a conceptual diagram showing the effect of rolling as a relative change in estimated road surface. The position of the estimated distance d in the traveling direction with the distance x in the lateral direction from the camera is corrected to the estimated distance d 'by the following equation.

[0032]

(Equation 2)

In this equation, dy is the height of the corrected estimated road surface, α is the rolling angle, and the other parameters are the same as those in the equation (1).

The correcting section 11 corrects the estimated distance stored in the estimated distance storage section 32 according to the outputs from the pitching angle detecting section 9 and the rolling angle detecting section 10 as described above. This correction processing can be executed by a hardware configuration in response to a signal indicating that an angle of a level requiring correction has been detected from the pitching angle detection unit 9 or the rolling angle detection unit 10. Alternatively, the central processing unit (CPU) that controls the object recognition device may perform the correction by executing the correction processing routine every 80 milliseconds, for example, by a software configuration.

The road surface removing unit 31 compares the measured distance for each window obtained from the distance storage unit 8 with the corrected estimated distance for each window obtained from the correcting unit 11, and determines a predetermined value from the corrected estimated distance. Threshold (for example, 20c
m) is determined to be a window having a measured distance equal to or greater than the value obtained by subtracting m), and a window having a measured distance equal to or less than the value obtained by subtracting the predetermined threshold from the corrected estimated distance is determined. It is determined that the window is an image of an object other than the road surface.

In this way, the road surface removing unit 31 converts the data of the window capturing the object other than the road surface into the object recognizing unit 3.
Send to 8. The object recognition unit 38 recognizes one or a plurality of objects based on data of a plurality of windows capturing an object other than a road surface. The object recognition processing is executed by the CPU executing an object recognition routine every 80 milliseconds, for example. Recognition of an object is 3 from 2 cycles
By integrating the results of the cycle recognition process,
The reliability of recognition can be improved.

The object storage unit 39 stores information such as the distance, position, and size of the object recognized by the object recognition unit 38. The object recognizing unit 38 calculates the relative speed between the object and the vehicle from the distance information of the object (vehicle) recognized this time and the distance information of the same object recognized last time, and calculates the relative speed of the object recognized this time. The data is stored in the object storage unit 39 as a part of the data.

The vehicle control unit 45 reads out information about the object sent from the object storage unit 39 and, based on the content, drives an alarm device to issue an alarm indicating that the vehicle is too close to the vehicle in front. It performs a function such as sending a signal to an electronic control unit (ECU) or a brake control unit of the engine to execute a forced braking process. At that time, the vehicle control unit 45 receives the data of the own vehicle speed from the own vehicle speed detecting device 46 and the signal indicating the yaw rate from the yaw rate detecting device 47, determines the traveling area of the own vehicle, and determines the distance to the object. Control your vehicle to be in the appropriate range.

The correlation calculation unit 6, distance calculation unit 7, distance storage unit 8, window cutout unit 13, road surface removal unit 31, estimated distance storage unit 32, object recognition unit 38, object storage unit 39, and vehicle shown in FIG. The control unit 45 includes a central processing unit (CPU), a read-only memory that stores a control program and control data, and a random access memory (RAM) that provides an operation work area of the CPU and can temporarily store various data. Can be configured. Distance storage unit 8, estimated distance storage unit 32, and object storage unit 39
Can be realized using different storage areas of one RAM. Further, a temporary storage area of data required for various calculations can be realized by using a part of the same RAM.

Further, the object recognition device of the present invention may be connected to an electronic control unit (ECU) of the engine, a brake control ECU, and other ECUs via a LAN to utilize the output from the object recognition device for overall control of the vehicle. it can.

[0041]

According to the first aspect of the present invention, an object can be correctly recognized even when the vehicle pitches. According to the second aspect of the present invention, an object can be correctly recognized even when rolling of the vehicle occurs.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an overall configuration of an embodiment of the present invention.

FIG. 2 is a diagram for explaining a principle of measuring a distance by a triangular measurement method.

FIG. 3 shows (a) a captured image according to the present invention;
(B) The figure which shows the image divided | segmented into the small area | region (window) for distance and road area determination.

FIG. 4 is a view for explaining the concept of correcting an estimated distance with respect to pitching.

FIG. 5 is a diagram showing the effect of rolling.

FIG. 6 is a view for explaining the concept of correcting an estimated distance for rolling.

[Explanation of symbols]

 3, 3 'imaging unit (stereo camera) 7 distance calculation unit 8 distance storage unit 31 road surface removal unit

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G06T 7/00 G06F 15/62 415 7/60 15/70 350J F-term (Reference) 2F065 AA06 BB05 BB13 CC40 DD14 EE05 FF05 FF09 FF65 GG21 JJ02 JJ03 JJ05 JJ25 JJ26 NN20 PP01 QQ03 QQ23 QQ24 QQ25 QQ27 QQ29 QQ36 QQ38 QQ41 2F112 AC01 AC06 BA20 CA05 DA19 DA28 FA03 FA21 FA36 FA41 5B057 AA06 DB11 CC04 A13 LL09 5L096 BA04 CA05 EA35 FA34 FA66 HA02

Claims (2)

[Claims] 1. A stereo camera mounted on a vehicle for photographing a running road surface, an image obtained from the stereo camera is divided into windows, and measuring means for measuring a distance to a target object for each window; Estimated distance storage means for storing an estimated distance to a predetermined road surface, pitching angle detection means for detecting a pitching angle of the vehicle, and estimation based on the pitching angle detected by the pitching angle detection means. An object recognition device, comprising: a correction unit that corrects a distance; and a recognition unit that recognizes the target based on the corrected estimated distance. 2. A stereo camera mounted on a vehicle for photographing a running road surface, an image obtained from the stereo camera is divided into windows, and measuring means for measuring a distance to an object for each window; Estimated distance storage means for storing an estimated distance to a predetermined road surface, rolling angle detection means for detecting a rolling angle of the vehicle, and the estimation based on a rolling angle detected by the rolling angle detection means. An object recognition device, comprising: correction means for correcting a distance; and recognition means for recognizing the object based on the corrected estimated distance.