JPH07160846A - Forward object recognizing device - Google Patents

Abstract

PURPOSE:To detect the reliability of the edge detected from the sharpness of the edge, regarding a forward object recognizing device judging the presence or absence of a forward object on the basis of the existence or absence of the edge which is estimated as the forward object within the image picked up by an image sensor. CONSTITUTION:The front of a vehicle is picked up by a camera (A). Within a picked up image, an edge point (. in a figure) to be estimated as the edge of a preceding vehicle is specified. The edge points are counted in the horizontal direction, a frequency distribution is formed (B) and a peak frequency IYp is determined. The edge points are counted in the vertical direction and peak points XPL and XPR are determined. Reliability Re=IYp/ (XPR-XPL+1) is determined from the length of edge in a horizontal direction and IYp. Re corresponds to the density of the edge point within the edge, that is, the sharpness of the edge and can be recognized as the reliability for the truth of the detected edge to be the preceding vehicle.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a front object recognizing device, and more particularly to whether or not an edge estimated to be an edge of a front object exists in a picked-up image obtained by picking up an image of a front area by an image sensor. The present invention relates to a front object recognition device that determines the presence or absence of a front object on the basis of the above.

[0002]

2. Description of the Related Art Conventionally, there is known a device for determining the presence or absence of a front object using an image sensor. For example, JP-A-6
Japanese Patent Laid-Open No. 3-9813 determines the presence or absence of a preceding vehicle as a front object based on whether or not a horizontal edge equivalent to the width of the vehicle is detected in the obtained captured image by capturing an image of the front of the vehicle with a camera. A device for doing so is disclosed.

The relationship between the preceding vehicle and the preceding vehicle becomes a problem in the vehicle when the inter-vehicle distance is kept at an appropriate distance. When the preceding vehicle is imaged in such a situation, the width of the preceding vehicle in the captured image Should be within an appropriate range, and therefore, it is possible to detect that a horizontal edge of an appropriate length can be detected in the captured image and there is a preceding vehicle, and it can be estimated that there is no preceding vehicle. It is a thing.

[0004]

However, the appearance of the image picked up by the camera greatly changes depending on the ambient brightness and the like. Therefore, the accuracy of the edge detection executed based on the captured image is not constant depending on the environment in which the vehicle is placed.
In particular, in the above-mentioned conventional device that assumes a vehicle as a front object, the outer shape of the vehicle body is mainly targeted for edge detection during the daytime, while the tail lamp is mainly targeted for edge detection during nighttime. Changes.

For this reason, the above-mentioned conventional device functions effectively in an environment assumed in advance, but if the brightness, definition, etc. of the captured image changes due to the change in the environment, the presence or absence of a front object is erroneously detected. There is a problem in that it may be detected and lacks practical reliability.

In this case, it is possible to improve the detection accuracy by performing the process of averaging the brightness of the picked-up image prior to the edge detection. However, such a process requires a complicated configuration in software. In addition to the above, it is not preferable because it takes a long time to average the image brightness, which causes a problem that rapid processing cannot be realized.

The present invention has been made in view of the above points, and the width of the edge estimated to be the edge of the front object,
An object of the present invention is to provide a front object recognition device that solves the above problems by calculating the reliability of a captured image based on the ratio of the edge width in the captured image to the point actually detected as an edge point. And

[0008]

FIG. 1 is a block diagram showing the principle of a front object recognition apparatus which achieves the above object. That is, the above-mentioned object is, as shown in FIG. 1, an image sensor M1 for imaging a front object, and an edge detection for detecting a predetermined direction edge estimated to be an edge of the front object in an image captured by the image sensor M2. Means M2, edge point counting means M3 for counting the number of pixels O actually detected as edge points in the detected predetermined direction edges, and edge length for substantially detecting the length of the predetermined direction edges. Detection means M4
And that the predetermined direction edge is the edge of the front object based on the ratio between the length of the predetermined direction edge detected by the edge length detection means M4 and the edge point counted by the edge point counting means M3. This is achieved by a front object recognition device including a reliability calculation means M5 for calculating the reliability.

Further, in the front object recognizing device having the above structure, the brightness detecting means M6 for detecting the brightness of the image picked up by the image sensor M1 and the reliability calculating means based on the detection result of the brightness detecting means M6. The front object recognition device provided with the reliability correction means M6 for correcting the calculation result of M5 is also effective.

[0010]

In the front object recognizing device according to the present invention, the image sensor M1 images a predetermined monitoring area. Therefore,
When a front object exists in the monitoring area, the image sensor M
1, the front object is imaged together with the background.

The edge detecting means M2 detects a pixel estimated to correspond to the edge of the front object in the captured image as an edge point and connects the detected edge points in a predetermined direction to form a predetermined direction edge. To detect.

Then, the edge point counting means M3 counts the points actually detected as edge points in the thus detected edges in the predetermined direction. in this case,
As the edge of the front object is imaged more clearly, the edge point counting means M3 counts more edge points per unit length.

On the other hand, the edge length detecting means M4 detects the length of the edge in the predetermined direction. The reliability calculation means M5 calculates the ratio of the detection result of the edge point counting means M3 and the edge length detection means M4 to calculate the edge point per unit length within the predetermined direction edge.

In this sense, the calculation result of the reliability calculation means M5 becomes a value corresponding to the sharpness of the edge in the predetermined direction in the image picked up by the image sensor M1,
The larger the value, the higher the possibility that the front object is imaged.

The brightness detecting means M6 detects the brightness of the environment in which the front object recognizing device is placed, based on the brightness of the captured image. Then, the reliability correction means M
In consideration of the fact that the density of the edge points in the imaged predetermined direction edge changes according to the brightness at the time of image capturing,
In order to eliminate the influence, the calculation result of the reliability calculation means M5 is corrected.

[0016]

2 is a block diagram of a front object recognition apparatus 10 according to an embodiment of the present invention. Here, the front object recognition device 10 of the present embodiment is a device mounted on a vehicle for recognizing a preceding vehicle traveling in front of the vehicle.

In FIG. 2, the camera 12 corresponds to the above-mentioned image sensor M1 and picks up an image in front of the vehicle as a monitoring area. The image processing ECU 14 is a central processing unit (CPU)
16. The edge detection means M2, which is mainly composed of 16
Edge point counting means M3, edge length detection means M4, reliability calculation means M5, brightness detection means M6, reliability correction means M7
This is a main part of the present embodiment for realizing the above.

The image processing ECU 14 converts the analog signal supplied from the camera 12 into a digital signal A /
D converter 18, random access memory (RAM) 2
0, a read only memory (ROM) 22 and the CPU 16 are connected by a common bus 24.
The above function is realized by executing a program, which will be described later, stored in M22.

FIG. 3 shows a flow chart of an example of a control routine executed by the image processing ECU 14 to realize the front object recognizing apparatus according to the present invention.

When the routine shown in the figure is activated, first, at step 100, a process of inputting an image from the camera 12 is performed. The front object recognition device 10 of the present embodiment is mounted on a vehicle as described above, and the captured image captures the road surface 30 as shown in FIG. It

In this case, as a method of recognizing the presence of the preceding vehicle in the captured image, a method of detecting an edge estimated to be a boundary between the preceding vehicle and the background in the image is known. That is, if the preceding vehicle is captured in the captured image, a boundary line is inevitably formed between the preceding vehicle and the background, and a brightness difference occurs between the vehicle body side and the background side of the preceding vehicle.

Therefore, if the picked-up image is scanned in the vertical direction or in the horizontal direction and a pixel whose brightness changes abruptly is detected as an edge point, a plurality of contours of the preceding vehicle can be traced as shown in FIG. 4A. Should be detected, and if it is determined whether an edge point that can be estimated as the contour of the preceding vehicle is detected, the presence of the preceding vehicle can be determined accordingly.

From this point of view, in the present embodiment, after the above step 100, in step 102, the edge points in the captured image are detected, and the vertical and horizontal edges are detected as a set of the detected edge points. Perform specified processing. In this sense, this step 102 corresponds to the above-mentioned edge detecting means M2.

The edge points shown in FIG. 4 (A) (points indicated by ・ in the figure) are examples in which the lower end and both side ends of the preceding vehicle are detected as edges. In this case, assuming that an appropriate inter-vehicle distance is maintained between the preceding vehicle and the own vehicle, these edges can be regarded as almost straight lines, and as shown in the same figure in the captured image. When the (X, Y) coordinate is set, the Y coordinate value of the edge point forming the edge representing the lower end of the preceding vehicle is near the specific value Y _P , and the X coordinate of the edge point forming the edge representing the left and right ends of the preceding vehicle. value is,
Concentrate around specific values X _PL and X _PR , respectively.

Therefore, when the preceding vehicle is picked up in the picked-up image and the frequency distribution is created by counting the edge points in the horizontal direction, the frequency distribution having a peak at Y _P as shown in FIG. 4B. Is formed, and when a frequency distribution is created by counting the edge points in the vertical direction, a frequency distribution having peaks at X _PL and X _PR is formed as shown in 4 (C).

At this time, the edge points constituting each of these edges should be detected with higher density as the edges are imaged more clearly, and therefore, the more clearly the edges are imaged, the above-mentioned frequency. Distribution peak values IY _P , IX
_P should be a large value. In other words, the larger the values of IY _P and IX _P , the clearer the imaged edge.

In this case, the sharpness of the imaged edge is
It represents the degree of reliability that the edge represents the outer shape of the vehicle ahead, and the higher the sharpness, the higher the possibility of being the leading vehicle. The lower the sharpness, the higher the possibility of noise. it can. Then, if such a determination is possible, it is possible to determine with high accuracy whether or not the preceding vehicle exists within the monitoring area of the camera 12.

Therefore, in the present embodiment, when the edge detection is completed in step 102, step 1
04 to 112, a process of calculating the reliability Re corresponding to the sharpness of the horizontal direction edge is performed, and the reliability Re
The structure that determines the presence of a front object based on

That is, in step 104, the process shown in FIG.
In order to create a horizontal frequency distribution as shown in (B), the detected edge points are integrated in the horizontal direction. Next, at step 106, the frequency I at the peak point Y _P in the frequency distribution.
Detect Y _P. Then, in step 108, as shown in FIG.
In order to create a vertical frequency distribution as shown in (C), the detected edge points are integrated in the vertical direction, and step 11
At 0, the peak points X _PL and X _PR of the vertical frequency distribution are detected.

In this case, IY _P detected in the above step 106 is the number of edge points in the edge, and X _PL and X _PR detected in the above step 110 are coordinates representing both ends of the edge. The distance between the two corresponds to the length of the edge. In this sense, the step 106 corresponds to the edge point counting means M3, and the steps 108 and 110 correspond to the edge length detecting means M4.

Step 112 is a step of realizing the above-mentioned reliability calculation means M5, and the steps 108, 1 are executed.
The reliability Re is calculated as follows using IY _P and X _PL , X _PR detected in 10.

Re = IY _P / (X _PR −X _PL +1) (1) Here, in this embodiment, the pixel position of the captured image is used as it is as the coordinate value. Therefore, “X _PR −X _PL +1” in the equation (1) corresponds to the length of the horizontal edge, and all the pixels in X _{PL to} X _PR are detected as edge points at the position of Y = Y _P. If so, the above Re should be “1.0”.

That is, the reliability Re is a characteristic value in which the upper limit is "1.0" and the value decreases as the edge point density in the horizontal edge decreases.
In this embodiment, the step 112 corresponds to the reliability calculation means M5 described above.

In this routine, the vehicle shape is generally configured to calculate the reliability Re using the horizontal edge because the vehicle width exceeds the vehicle height. The length of the vertical edge, not the one
And the reliability Re based on the number of edge points in the
May be calculated.

By the way, in FIG. 5, assuming typical daytime (curve indicated by solid line in FIG. 5) and nighttime (curve indicated by broken line in FIG. 5) as a typical environment, the distance L to the preceding vehicle is shown. The result of calculating the reliability Re in the relationship is shown as a reliability curve.

In this case, there is a high possibility that a preceding vehicle is present in front of the own vehicle in the area exceeding the reliability curve corresponding to each environment, and the preceding vehicle is in the area less than the reliability curve. It should most likely not exist. Therefore, if the reliability curve shown in FIG. 5 is stored in advance as a characteristic map, it is possible to accurately determine the presence of a preceding vehicle by comparing the measured reliability Re with the map.

At this time, the reliability curve shown in FIG. 5 indicates that the reliability Re is a function of the distance L from the preceding vehicle. This phenomenon occurs when the inter-vehicle distance is 10
In the area that can exceed 0 m, the reliability R increases as the distance increases.
It is considered that, while e decreases, the edge of the preceding vehicle cannot be regarded as a straight line in the captured image in the region where the inter-vehicle distance is less than 50 m.

Further, the reliability curve shown in FIG.
It also indicates that e is affected by the brightness of the external environment in addition to the distance L. This phenomenon means that the body shape of the preceding vehicle is mainly imaged as an edge in the daytime, while the taillight of the preceding vehicle is mainly imaged as an edge in the nighttime, and the edge point with respect to the horizontal edge is inevitably taken. It is considered that the density is reduced.

Therefore, in order to compare the actually measured reliability Re with the characteristic map as shown in FIG.
In addition, it is necessary to measure the distance L between the own vehicle and the preceding vehicle, and to distinguish whether the external environment is daytime or nighttime.

Therefore, in this routine, after the calculation of the reliability Re is completed in step 112, the process proceeds to step 114 to calculate the inter-vehicle distance L. The calculation of the inter-vehicle distance L can be performed by a known method of processing a captured image of the camera 12. In this embodiment, assuming that the elevation angle of the camera 12 and the ground clearance at the lower end of the preceding vehicle are substantially constant, the above step 1 is performed.
The inter-vehicle distance L is calculated from Y _P obtained in 06.

When the calculation of the inter-vehicle distance L is completed in this way, the routine proceeds to step 116, where the brightness IMB of the input image is calculated. The brightness of the external environment that affects the reliability Re should be reflected in the brightness of the captured image, and if the brightness in an appropriate area in the captured image is detected, it is used as a substitute characteristic value of the external brightness. Because it is possible to grasp. In this sense, this step 116 corresponds to the lightness detecting means M6 described above.

In this case, as the area to be extracted for detecting the lightness, if the preceding vehicle exists as shown in FIG. As shown in FIG. 6B, the upper and lower areas of the captured image that are not affected by the presence or absence of the preceding vehicle are suitable.

In step 116, the brightness IMB
When the calculation of is completed, it is determined in step 118 whether IMB is larger than a predetermined determination value C. This judgment value C
Is a determination value for distinguishing between daytime and nighttime, and is a value set experimentally according to the characteristics of the vehicle.

When IMB> C is satisfied, that is, when it is determined that the external environment is sufficiently bright, it is determined that it is daytime at present, and the process proceeds to step 120. In this step 120, the characteristic map (curve shown by a solid line in FIG. 5) set assuming the daytime is searched for by the actually measured inter-vehicle distance L, and the determined value gd (L) and the actually measured reliability Re are set. Is the step of comparing.

On the other hand, in step 118, the IMB
When it is determined that> C is not satisfied, it is determined that it is currently nighttime, and the process proceeds to step 122. This step 1
22 is a characteristic map set assuming night (see FIG. 5 above).
This is a step of comparing the determined reliability value Re with the determination value gn (L) specified by searching for the actually measured inter-vehicle distance L (curve indicated by the broken line).

In step 120, Re> gd
If it is determined that (L) holds, or step 12
If it is determined that Re> gn (L) is satisfied in step 2, the process proceeds to step 124 and it is determined that there is a vehicle, and if it is determined that the above condition is not satisfied, the process proceeds to step 126 and there is no vehicle. And make a decision.

In this case, the above steps 118 to 122
Substantially corrects the degree to which the reliability Re confirms the existence of the preceding vehicle, and in this sense corresponds to the reliability correction means M7.

In the present embodiment, the above step 1
Only when the presence of the preceding vehicle is recognized in 24
When the routine proceeds to step 128 and the subroutine for alarm / automatic braking is executed, and when it is detected that the inter-vehicle distance L with the preceding vehicle is unduly short, the driver is warned and the automatic braking operation is performed. I am going to do it.

As described above, according to the front object recognition apparatus 10 of the present embodiment, the brightness of the external environment and the distance L to the preceding vehicle.
Based on the above, the presence of the vehicle is detected only when the horizontal edge is detected with the sharpness that should be originally detected when the preceding vehicle is present. Therefore, even if the detection accuracy of the edge in the captured image changes due to a change in the environment, a change in the inter-vehicle distance L, or the like, the presence of the preceding vehicle can always be detected with high accuracy while canceling the change. Can be recognized.

By the way, the front object recognition apparatus 1 of the present embodiment
A value of 0 calculates the reliability Re corresponding to the sharpness of the horizontal edge, and determines the presence of the preceding vehicle based on the value to prevent erroneous detection, and the brightness of the external environment is used for the determination. Although it is realized with the configuration to be reflected, it is not limited to this, and it is effective even when only one of them is adopted.

[0051]

As described above, according to the first aspect of the present invention, by calculating the edge point density in the predetermined direction edge in the image picked up by the image sensor, the sharpness of the predetermined direction edge can be improved. A corresponding characteristic value can be calculated as the reliability.

In this case, the higher the reliability is, that is, the sharper the edge is, the higher the possibility that the front object actually exists, and the lower the reliability, the lower the possibility that the front object exists. It is possible to make a judgment, and it is possible to realize a front object recognition process without erroneous detection by reflecting such possibility.

According to the second aspect of the present invention, the reliability of the recognition result of the front object can be corrected according to the brightness of the environment in which the front object recognition device is placed. In this case, since the brightness of the environment can be reflected in the corrected reliability, it is possible to determine the presence or absence of the front object in consideration of the brightness of the environment. Therefore, according to the front object recognition device of the present invention,
Compared with the invention described in claim 1, it has a feature that effective recognition processing can be performed in a wider environment.

[Brief description of drawings]

FIG. 1 is a principle configuration diagram of a front object recognition device according to the present invention.

FIG. 2 is a block configuration diagram of a front object recognition apparatus that is an embodiment of the present invention.

FIG. 3 is a flowchart of an example of a control routine executed by the image processing ECU.

FIG. 4 is a diagram for explaining a front object recognition method by the front object recognition device of the present embodiment.

FIG. 5 is an example of a characteristic map used in the front object recognition device in the present embodiment.

FIG. 6 is a diagram showing a region suitable for extraction when detecting brightness from a captured image.

[Explanation of symbols]

 M1 image sensor M2 edge detection means M3 edge point counting means M4 edge length detection means M5 reliability calculation means M6 brightness detection means M7 reliability correction means 10 front object recognition device 12 camera 14 image processing ECU 16 central processing unit (CPU) 18 A / D converter 20 Random access memory (RAM) 22 Read only memory (ROM)

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI technical display location G06T 9/20

Claims (2)

[Claims] 1. An image sensor for picking up an image of a front object, edge detecting means for detecting a predetermined direction edge estimated to be an edge of the front object in an image picked up by the image sensor, and the detected predetermined direction edge. An edge point counting means for counting the number of pixels actually detected as an edge point, an edge length detecting means for substantially detecting the length of the edge in the predetermined direction, and an edge length detecting means for detecting the edge length. Reliability calculating means for calculating the reliability of the predetermined direction edge being the edge of the front object based on the ratio of the length of the predetermined direction edge and the edge point counted by the edge point counting means. A front object recognition device. 2. The front object recognizing device according to claim 1, wherein the brightness detecting means for detecting the brightness of the image captured by the image sensor, and the reliability calculating means based on the detection result of the brightness detecting means. A front object recognition device comprising: a reliability correction unit that corrects a calculation result.