JPH0588745A - Environment recognizing device for mobile vehicle - Google Patents

Abstract

PURPOSE:To provide an environment recognizing device for mobile vehicles which can recognize the obstacles at a high speed in a simple constitution. CONSTITUTION:The 1st density distribution ARY of the direction Y and the 2nd density distribution ARX of the direction X are calculated for an outside gradation image set approximately in the traveling direction of a mobile vehicle. A high frequency position YMAX is calculated from the distribution ARY in regard of the traveling direction of a sample vehicle of the high generating frequency. Then a centroid position XGR of the distribution ARX is calculated. Thus the position of an obstacle is detected based on the positions YMAX and XGR.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an environment recognition device for a moving vehicle, and more particularly to improvement of environment recognition technology by recognizing an obstacle in front of the moving vehicle.

[0002]

2. Description of the Related Art Conventionally, in order to detect an obstacle using an image and estimate its position, a huge amount of preprocessing such as binarization and edge extraction is required. In this pre-processing process, after limiting the number of obstacle candidates in the environment ahead of the traveling direction, the obstacle is detected based on the structure of the obstacle itself, and its position is estimated (for example, (Kaisho 64-26913).

[0003]

However, the above-mentioned preprocessing is the filter calculation in the edge processing process,
Since it requires enormous calculations such as labeling, it lacks real-time performance, which has been a practical obstacle in applying the above-mentioned conventional environment recognition technology to autonomous traveling of a mobile vehicle. Therefore, an object of the present invention is to propose an environment recognition device for a moving vehicle, which has a simple structure and can recognize an obstacle at high speed.

[0004]

The environment recognizing device for a mobile vehicle according to the present invention for achieving the above-mentioned object pays attention to the fact that a shadow is accompanied by a front obstacle. That is, the configuration of the present invention is such that the input means for inputting the grayscale image of the outside world in the substantially vehicle traveling direction, the first density distribution of the grayscale image in the vehicle traveling direction, and the direction substantially orthogonal to the vehicle traveling direction. With respect to the second concentration distribution, and a high-frequency position in the vehicle traveling direction of a sample with a high occurrence frequency from the first concentration distribution to calculate the center of gravity of the second concentration distribution. It is provided with position calculating means for calculating a position and means for detecting an obstacle based on the high frequency position and the position of the center of gravity.

According to this environment recognition device, it is possible to recognize the position of the portion which is considered to be the shadow of the obstacle. Since the position of the shadow is close to the position of the obstacle, it is possible to recognize the front obstacle by regarding this position as the position of the obstacle. In this case, the calculation of the histogram, the calculation of the position of the center of gravity, etc., which is required in this case, has an overwhelmingly small amount of calculation as compared with the conventional preprocessing. Therefore, the device of the present invention has a simple structure and is relatively fast. Enables obstacle recognition.

[0006]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment in which the environment recognition device of the present invention is applied to autonomous traveling will be described below with reference to the accompanying drawings. First
The figure schematically shows the configuration of an autonomous vehicle of the embodiment. This vehicle 100 is equipped with a camera 10 for taking an image 11 of an external (especially front) environment. The image 11 is input to the environment recognition system 12 as a grayscale image, and environment recognition is performed therein. The recognition result is sent to the situation determination system 13 having a knowledge base, and the traveling situation is determined here. The vehicle control system 14 controls, for example, a steering wheel (not shown) or a brake based on the situation determination result by the system 13.

This autonomous vehicle includes a GPS device (12c in FIG. 2) and a clock (12 in FIG. 2) which are not shown in FIG.
d) is provided. FIG. 2 shows an outline of the environment recognition system 12 of FIG. In this environment recognition system, the image processing unit 12a recognizes the position of the front obstacle as the position (X _GR , Y _MAX ) of the shadow of the obstacle. This is because the shadow 201 of the obstacle 200 has a lower density than the obstacle 200 itself as shown in FIG.

By the way, the direction in which the shadow extends differs depending on the position of the sun (light source). Therefore, if the shadow position recognized by the processing unit 12a is not corrected in consideration of the shadow direction, the incorrect position is recognized as an obstacle. Therefore, the environment recognition system 12 of this embodiment has a shadow direction calculation unit 12b as shown in FIG. This calculator 12
b is a GPS (Global Positioning System) device 12c
Based on the vehicle position signal from the vehicle and the current time from the clock 12d mounted on the vehicle, the sunshine database 12e is searched for the sunshine direction. The calculation unit 12b corrects the shadow position based on the sunshine direction to obtain the final obstacle position ( _X'GR , _Y'MAX ) as shown in FIG.

FIG. 5 is a flow chart showing the control procedure in the image processing section 12a. The control procedure for obtaining the obstacle position (X ′ _GR , Y ′ _MAX ) will be described with reference to FIG. In step S2, an image is input from the camera 10. This image is composed of gradation signals having shades. In step S4, a sampling area as shown in FIG. 6 for calculating the threshold value is designated. This is because the present embodiment is based on the assumption that the pixels of the grayscale image that have a density equal to or lower than the predetermined density are in the shadow portion, and therefore a threshold value that means the predetermined density is obtained from the sampling area. .. Normally, there are no obstacles within a certain range in front of the vehicle,
Usually, the density within the range indicates the density of an image of a road or the like. Therefore, such a range is designated in advance as the sampling area, and in step S6, the threshold value TH for determining whether or not the pixel corresponds to a shadow is determined based on the average value of the image density in this area. To do. That is, if the average value of the densities of the image signals in the sampling region is C, the threshold value TH is calculated by TH = k.multidot.C (k is a constant). Therefore, the density at an arbitrary pixel position is D
If it is expressed by (X, Y), it can be estimated that the pixel position where D (X, Y) ≦ TH is a shadow. In FIG. 6, the x marks indicate such pixel positions with a certain density or less.

In step S8, X-direction and Y-direction histograms as shown in FIG. 6 are created. This creation method is as follows. First, the X-direction array area AR _x ,
The Y-direction sequence regions AR _Y, are prepared in a predetermined memory of the image processing unit 12a (not shown). As shown in FIG. 7, AR _x is a data array for each unit length in the X coordinate direction, and each element stores the Y coordinate value. A
R _Y is a data array and a data array taken for each unit length in the coordinate direction, and each element stores a count value.

When creating the histogram, the pixel positions marked with X in FIG. 6 are searched. The search direction is to search from a lower side (that is, from a position closer to the own vehicle) in FIG. 6 at a certain X direction position X _n . Such a pixel D _i
(X _n, Y _i) is the is found to accumulate the discovery number of times which corresponds to the Y-direction coordinate value Y _i of the D _i of the element of the array AR _Y. For AR _x , the X of the pixel D _i is
The Y direction coordinate value Y _i is stored in the element corresponding to the direction coordinate value X _n . Then, X _n indicating the search position is moved from left to right in FIG. By doing so, for example, as shown in FIG. 7, in the array AR _x , the X-direction coordinate position 0,
Threshold value T for each of 1, 2, 3, 4, 5 ... X _N
The Y coordinate position of a pixel having a density of H or less is stored.
In this case, if a plurality of pixels are found at the same Z coordinate position (for example, X = 2), as shown in the figure, each element of the array AR _x (for example, an element corresponding to X = 2 is its Y coordinate value a plurality (e.g., Y _9, Y _10, Y ₁₂₎ are stored. also, the array AR _Y, the histogram as FIG. 6 is formed.

Next, in step S16, Y _MAX which is the Y coordinate value of the shadow is obtained. Since the obstacle (especially the vehicle in front) is usually in front of the own vehicle and its bottom surface is horizontal, its shadow extends in the X-axis direction and its Y-axis as shown in FIG.
The width in the direction is small. Therefore, there is an obstacle in the front, and the Y direction histogram of the image is as shown in FIG.
The frequency with the maximum frequency should occur at the position Y _MAX corresponding to the position of the shadow in the Y direction. That is, the Y coordinate value Y _MAX indicating the maximum frequency of the Y direction histogram is set as the Y direction position of the shadow.
If the maximum frequency occurs at a plurality of consecutive Y-direction positions, the middle thereof is set to Y _MAX .

Next, in step S12, the shadow portion in the X direction
Detect the position. This is done as follows. That is, H direction
Array AR that is a set of directional histograms_x Of the elements of
Y_{MA X} Only those that have as data are extracted.
In the example of FIG. 7, Y_MAX = Y _TenThen, X = 1, 2,
3 pixels are extracted. That is, this continuous pixel is
As is clear from 6, Y_MAX Throw shadow in X direction
It is a shadow. Therefore, the barycentric position of the X-direction projection of the shadow is X
Directional position X_GRAnd in the example of FIG. 7, X_GR= 3
It

Thus, the position of the obstacle (X _GR , Y _MAX )
Was recognized as the position of the shadow of the obstacle. As is clear from the above description, in the image processing of this embodiment, the position of the obstacle could be easily recognized without performing complicated binarization and edge detection. In the control procedure shown in FIG. 5, the position showing the maximum frequency is set as the Y coordinate value of the obstacle in the histogram in the Y direction. This is because, in the field of autonomous traveling technology in which the obstacle is often a forward vehicle, the width of the shadow in the Y direction is narrow as described above, and therefore a peak is generated in the histogram and this peak is detected. It is easy to do. In particular, when the sun is shining from behind the vehicle, the shadow of the vehicle in front should be a thin shadow extending in the X direction.

However, especially when the sun irradiates from the front,
When the elevation angle is low, the length of the shadow becomes long in the Y direction as shown in FIG. In such a case, a large number of positions showing high frequency are continuously found in the Y-direction histogram. In such a case, the center of gravity in the X direction described with reference to FIG. 7 is calculated for each of the Y direction positions having a frequency equal to or higher than the predetermined frequency. In this way, a continuous (X _GR , Y _MAX ) set as shown in FIG. 8 is obtained, and the accuracy of obstacle recognition is eventually reduced.

Therefore, even in such a case, it is possible to accurately perform
A method of recognizing the position of obstacles, that is, due to the direction of sunlight
Then, the correction when the shadow becomes long will be described. this
Shows that the optical axis is deviated from the vertical direction as shown in FIG.
Then, the estimated position of the obstacle (X _GR, Y_MAX ) And the actual position
(X '_GR , Y '_MAXThis is because an error occurs in). this
Therefore, in step S14 of FIG.
In order to detect the
Data). At step S16, the current time
Know the time (Greenwich Mean Time). Actually projected on the road
In order to know the direction of the shadow in the image, the direction of the camera and the road surface
You need to know the inclination. Therefore, in step S18, an example
For example, from the travel control system 14 equipped with a tilt sensor, etc.
Receives a tilt signal of the own vehicle camera 10 with respect to the ground surface. So
Then, in step S20, from the sunshine database, the own vehicle
Based on the position and the current time
Calculate The sunshine database has a calendar and time
It stores information about the positional relationship between the earth's axis and the sun. This
As a result, the angle of the sun with respect to the ground surface at the vehicle position and
You can know the bearing.

The principle of shadow correction is shown in FIGS.
It will be described with reference to the drawings. The elevation angle of the sun at the vehicle position is θ
Assuming _E and the azimuth angle to be θ _D , the rod of length 1 shown in FIG. 10 has Δ _x = Cot θ _E , Δ _Y = Δ _x · Tan θ _D = COT θ _E ·
It becomes the shadow of TAN θ _D. If, as shown in FIG. 11, an obstacle of width X and height Y is tilted by the sun by elevation angle θ _E and azimuth angle θ _D , it is obtained by step S12 (X _GR , Y
If shadows showing the set (see FIG. 8) of the _MAX) is detected as having a, the size of _{b, X + Y · △ x} = a (1 + △ Y) · Y = b satisfies X , Y is recognized as an obstacle. That is, of the set of shaded portions in FIG. 8, the X and Y portions that satisfy the above equation are recognized as the positions of the regular obstacles. (X _GR ,
Relative Y _MAX), the actual position _{(X 'GR, Y' MAX} ) to be because an error occurs.

The present invention can be applied to various modifications of the above embodiment without departing from the spirit of the present invention. For example,
In addition to using the GPS described above, for example, an optical gyro can be used to know the irradiation direction of the sun. The optical gyro device shown in FIG. 12 is based on the positions ΔX and ΔY where the shadow of the marker 201 is detected on the light receiving element 200 which is two-dimensionally spread in the X and Y directions. It is to know the tilt of the axis.

By using this gyro together with the GPS device, the inclination of the vehicle with respect to the ground surface can be detected. The angle at which the sunlight is projected onto the vehicle position can be known from the GPS device or the like. The inclination of the vehicle with respect to the sunlight can be known using an optical gyro. Therefore, the inclination of the vehicle with respect to the ground surface can be known from these.

[0020]

As described above, according to the present invention, the input means for inputting the grayscale image of the outside world in the vehicle traveling direction, the first density distribution of the grayscale image in the vehicle traveling direction, and the vehicle Second about the direction substantially orthogonal to the traveling direction
Distribution calculating means for calculating the concentration distribution of the second concentration distribution, and a high-frequency position of the sample having a high occurrence frequency in the vehicle traveling direction from the first concentration distribution, and a center of gravity position of the second concentration distribution. A position calculation means and a means for detecting an obstacle based on the high frequency position and the center of gravity position are provided.

According to this environment recognition device, it is possible to recognize the position of the portion which is considered to be the shadow of the obstacle. Since the position of the shadow is close to the position of the obstacle, it is possible to recognize the front obstacle by regarding this position as the position of the obstacle. In this case, the calculation of the histogram, the calculation of the position of the center of gravity, etc., which is required in this case, has an overwhelmingly small amount of calculation as compared with the conventional preprocessing. Therefore, the device of the present invention has a simple structure and is relatively fast. Enables obstacle recognition.
In particular, even if the angle of the light source is tilted, it is possible to accurately recognize the obstacle position by performing the correction accordingly.

[Brief description of drawings]

FIG. 1 is a diagram showing a system configuration of an autonomous vehicle of this embodiment.

FIG. 2 is a system block diagram of autonomous traveling control of the autonomous vehicle of FIG.

[Figure 3]

FIG. 4 is a diagram illustrating a method of correcting an error in a recognition position caused by a shadow varying depending on the inclination of a light ray.

FIG. 5 is a flow chart illustrating a control procedure of the present embodiment.

FIG. 6 is a diagram illustrating the principle of a method of recognizing a shadow portion in the present embodiment.

7 is a diagram illustrating the structure of an array AR _x used in the method of FIG.

[Figure 8]

[Figure 9]

[Figure 10]

FIG. 11 is a diagram illustrating a principle of removing an error in a recognition position due to a tilt of a light beam.

FIG. 12 is a diagram showing another example of an apparatus for detecting the inclination of the own vehicle.

Claims (3)

[Claims] 1. An input means for inputting a grayscale image of the outside world in a substantially vehicle traveling direction, a first density distribution of the grayscale image in the vehicle traveling direction, and a first density distribution in a direction substantially orthogonal to the vehicle traveling direction. Distribution calculating means for calculating the second concentration distribution; and a high-frequency position in the vehicle traveling direction of a sample having a high occurrence frequency from the first concentration distribution,
2. An environment recognition device for a mobile vehicle, comprising: a position calculation means for calculating the barycentric position of the density distribution of 1. and a means for detecting an obstacle based on the high-frequency position and the barycentric position. 2. The environment recognizing device for a moving vehicle according to claim 1, wherein the means for calculating the distribution is a means for extracting pixels having a predetermined density or less in an input image, and the number of these pixels is the There is provided means for calculating distributions of occurrence frequencies in a vehicle traveling direction and a direction substantially orthogonal to the vehicle traveling direction and setting these distributions as the first concentration distribution and the second concentration distribution. 3. The environment recognizing device for a moving vehicle according to claim 1, further comprising means for calculating a direction in which a shadow of an obstacle extends, and the position calculating means based on the calculated direction of the shadow. And a correction unit that corrects the data of the determined high frequency position and the center of gravity position.