JP2002094978A - Lane detector - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a lane detector that is not susceptible to the effect of a noise on image data due to rain drops and dirt stuck on a lens of a camera or the like so as to recognize a lane mark with high accuracy. SOLUTION: A feature quantity is detected from an image received in a step S2 (S3), results on line-segmentation (S4) and labeling (S5) are stored (S6), and by comparing a present labeling result with a past labeling result, coincident data is eliminated as a noise (S7-S12) so as to eliminate data such as rain drops and dirt moved with the camera, thereby detecting a lane mark with high accuracy (S13).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a lane detecting apparatus using image processing applicable to an automatic driving system and a parking assist system.

[0002]

2. Description of the Related Art In recent years, when a white line or the like indicating a driving lane on a road or the like is detected by image processing, various information is provided to a driver based on a recognition result, and further, when a vehicle is automatically driven. There has been proposed a system that uses this recognition result for the control of the image.

[0003] Japanese Patent Laid-Open No.
A technique disclosed in Japanese Patent Application Laid-Open No. 1-195127 is known. According to the publication, it is described that white lines can be recognized with high accuracy by extracting edges in an image, labeling the extracted edges, and detecting an edge having a pair of angles from the extracted edges.

[0004]

Such images are acquired by a camera installed at the front or rear of the vehicle. However, if raindrops or dirt are attached to the front of the camera lens, the image data itself is deteriorated. For this reason, it is difficult to recognize with high accuracy.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a lane detecting device which is less susceptible to noise of image data due to raindrops and dirt attached to a lens of a camera, and which can perform accurate recognition.

[0006]

In order to solve the above-mentioned problems, a lane detecting device according to the present invention is a lane detecting device for detecting lane information in an image taken from a camera. Feature amount detection means for detecting a predetermined feature amount from data; (2) labeling means for extracting and labeling a feature portion in an image based on the detected feature amount; and (3) storage means for storing a labeling result. And (4)
A noise removing unit that compares the current labeling result with the past labeling result and removes data whose labeling result matches at substantially the same position in the frame as noise from the data, and (5) lane information from the noise-removed labeling result. And a detecting means for detecting.

The moving speed of raindrops, dirt, and the like attached to the lens of the camera is smaller than that of the scenery outside the vehicle, and is almost stationary in the frame of the image. In the present invention, by utilizing this fact, the past labeling result and the current labeling result are compared, and data that coincides at substantially the same position in the frame is determined as noise such as dirt and removed to remove the lane. This is to determine the information accurately.

The noise removing means determines that the data is noise when there is data that coincides with the past labeling result at substantially the same position in the frame for a predetermined period. During this predetermined period, the vehicle speed is high. In this case, it is preferable to make the length shorter than that in the case where it is low.

As described above, noise is almost stationary in a frame of an image. Therefore, if there is data that matches the past labeling result over a predetermined period, that is, if there is still data, it can be determined that the data is noise. And, when the vehicle speed is high, the image data such as the landscape moves in the frame at a higher speed.
It is possible to determine whether the data is stationary data in a shorter time.

In comparing the labeling results, the labeling results may be compared a plurality of times in a predetermined period within a predetermined period, or may be compared with the labeling results in a frame before the predetermined period has elapsed.

[0011]

Preferred embodiments of the present invention will be described below in detail with reference to the accompanying drawings. In order to facilitate understanding of the description, the same constituent elements are denoted by the same reference numerals as much as possible in each drawing, and redundant description will be omitted.

FIG. 1 is a schematic configuration diagram showing a basic configuration when a lane detection device according to the present invention is applied to a lane departure warning device, and FIG. 2 is a diagram illustrating a use state of the CCD camera 2.

As shown in FIG. 1, the lane detecting device according to the present invention comprises a lane detecting ECU 1 for receiving and processing an image signal from a CCD camera 2, and has an image processing CPU 10 and a memory 11 therein. That is, the image processing CPU 10 also serves as a feature amount detecting unit, a labeling unit, a noise removing unit, and a lane information detecting unit according to the present invention by a built-in program. The lane detection ECU 1 is connected to the lane departure warning ECU 3 and sends and receives information to and from each other. The lane departure warning ECU 3 includes a vehicle speed sensor 41, a steering angle sensor 4
2. Each output signal of the yaw rate sensor 43 is input.
The lane departure warning ECU 3 predicts the future trajectory of the vehicle from the vehicle state quantity and compares it with the lane position detected by the lane detection ECU 1 to determine that the vehicle departs from the lane. This is to alert the driver by utilizing the information.

The CCD camera 2 is arranged at the center of the front of the cabin of the vehicle 5 and acquires an image of a road surface in front of the vehicle through a windshield 50. A lane (lane) 60 on the road surface 6 on which the vehicle 5 runs is divided by left and right division lines (lane marks) 61R and 61L.

FIG. 3 shows the operation of the lane detecting device according to the present invention.
This will be described with reference to FIGS. FIG. 3 is a flowchart for explaining the operation of this apparatus. FIGS. 4 and 5 show a series of raw images acquired by a CCD camera and the labeling results thereof.

The flow shown in FIG. 3 is repeatedly executed by the image processing CPU 10 in the lane detection ECU 1 at a predetermined interval, for example, in synchronization with the acquisition of an image by the camera after the ignition switch is turned on.

First, in step S1, the lane departure warning E
The vehicle state quantity is read from the vehicle speed sensor 41, the steering angle sensor 42, and the yaw rate sensor 43 via the CU3. Subsequently, in step S2, image data is captured from the CCD camera 2.

In step S3, the acquired image data
For example, an image contour, that is, an edge, is detected from image data as shown in FIGS. In these images, the lane marks 61R, 61L
And the other vehicle 62 ahead of the other lane,
a, Raindrops 71 and 72 attached to the windshield 50 are shown. Each of these contours will be detected.

In step S4, the detected edges are connected to perform line differentiation. After that, the line segments detected in step S5 are numbered, that is, labeled. The methods of edge detection, line differentiation, and labeling from this image data are described in “Image Engineering Handbook” (edited by Kenji Hiwatari, Asakura Shoten, 1986), pages 249 to 256. FIG. 5 shows a state in which edges are detected from the image data shown in FIGS. In the figure, the labeled number is L
It is indicated with a suffix. That is, the raindrop 71 in the original image,
72 are the first and second, respectively, the lane mark 61L is the third, the vehicle front part 5a is the fourth, and the lane mark 61R is the fifth.
The other vehicle 62 is numbered as the sixth.

In step S6, the result of the labeling, that is, the shape and position information of the extracted line segment is stored in the memory 11.

Steps S7 to S11 are performed with raindrops 71 and 72.
This is a step of removing noise such as For example, FIG.
(C), the processing at the time of FIG. Here, the line segments L1 to L6 labeled as Nos. 1 to 6 are detected. This is compared with the past labeling results, one by one, in order from the first, for example, the labeling results at the time points in FIGS. 5A and 5B.

More specifically, first, in step S7, the first data L1 is selected, and in step S8, it is determined whether or not past data has line segment data of a similar shape at substantially the same position. . The data L1 is data of the raindrop 71, which is attached to the windshield 50 of
Since it moves together with the vehicle 5 like the D camera 2, there is almost no moving speed inside the screen. for that reason,
It is determined that the data matches the past data, and the process proceeds to step S9, where this data is removed as noise.

Here, the determination of the presence of the coincident data is made by comparing a case where line segment data of the same shape exists at substantially the same position over a predetermined period as a coincidence, and comparing the data with the data before a predetermined time. There are two types of methods in which a case where line segment data of a similar shape exists at the same position is regarded as a match.
In any case, it is preferable to change the length of the predetermined period and the interval of the predetermined time according to the vehicle motion state from the data such as the vehicle speed and the steering angle obtained in step S1. This change may be stepwise, and may be switched depending on whether or not the speed is equal to or higher than a specific vehicle speed threshold.

When the vehicle is moving at a high speed or when the steering angle is large, the moving speed of the lane mark in the image also increases. Therefore, only the most recent data is used for the past data used for the match determination. Even if it is used, the determination can be made with high accuracy. On the other hand, when the vehicle is moving at a low speed and a small steering angle, the moving speed of the lane mark in the image also becomes slow. It is necessary to use the data at the point of time.

In the next step S11, unless the last label data (here, L6), the process proceeds to step S12, one label number is added, the next data is selected, and the process returns to step S8. The data L2 representing the raindrop 72 and the data L4 representing the front part 5a of the vehicle also have a match, and thus are removed as noise in step S9. In addition,
It is also possible to remove scratches, attached dirt, and the like on the lens of the CCD camera 2 and the windshield 50 by the same method. Since the position and shape of the front part 5a of the vehicle are known, edge detection or the like may be performed by excluding this part at the time of input in step S2.

On the other hand, data L3 and L5 representing the lane marks 61L and 61R and data L representing the other vehicle 62, respectively.
In the case of 6, since there is a movement of the position or the movement of the shape, it is determined in step S8 that there is no match with the stored data, and the process proceeds to step S10. Then, it is left as valid data for the next processing.

In this way, by performing the lane mark detection processing in step S13 only for the data L3, L5, and L6 remaining as valid data, the influence of noise is removed and the lane marks 61L and 61R are accurately recognized. Can be.

If the area occupied by the data excluded as noise in the image is more than a certain amount (for example, 10% or more), it is determined that the image data lacks reliability because the camera has many scratches and dirt. It is preferable to warn the driver of this without performing the lane mark detection process.

In the above description, the embodiment in which the CCD camera 2 is mounted at the front center of the vehicle compartment has been described. However, the installation of the camera is not limited to this position. In addition, the present invention can be applied to a camera that is disposed behind a vehicle compartment and shoots the rear through a rear window. Further, even if a camera that is arranged outside, in front of, on the side of, or behind a vehicle and captures a lane mark at any position is used, lane mark detection without noise can be performed in the same manner.

Although the edge extraction is performed as the feature amount as the information for detecting the lane mark, the feature amount may be extracted by binarization or the like.

Here, an example in which the detected lane mark information is used for the lane departure warning has been described. However, it is also possible to use the detected lane mark information for automatic running of the vehicle, parking assistance, and the like.

[0032]

As described above, according to the present invention, by utilizing the fact that noise such as raindrops and stains move together with the camera, feature amounts are extracted and labeled in the image, and the data which moves little is used. Is removed as noise, it is possible to accurately remove these noises and reliably detect lane marks.

[Brief description of the drawings]

FIG. 1 is a schematic diagram showing a lane departure warning device using a lane detection device according to the present invention.

FIG. 2 is a diagram showing a use state of a CCD camera of the apparatus of FIG.

FIG. 3 is a flowchart illustrating the operation of the lane detection device according to the present invention.

FIG. 4 is a diagram illustrating an example of a change in a raw image acquired by a CCD camera.

FIG. 5 is a diagram illustrating a labeling result in each image of FIG. 4;

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Lane detection ECU, 2 ... CCD camera, 3 ... Lane departure warning ECU, 6 ... Road surface, 10 ... Image processing CPU, 11
... Memory, 41 ... Vehicle speed sensor, 42 ... Steering angle sensor, 4
3: Yaw rate sensor, 50: windshield, 60:
Lane (lane), 61 ... division line (lane mark).

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) B60R 21/00 626 B60R 21/00 626G (72) Inventor Hisashi Satonaka 1st Toyota Town, Toyota City, Aichi Prefecture Toyota Inside the Automobile Co., Ltd. (72) Inventor Toshiaki Kakinami 2-1-1 Asahi-cho, Kariya-shi, Aichi Prefecture Inside Aisin Seiki Co., Ltd. (72) Inventor Shin Shin Takahashi 41 1 Toyota Central Research Laboratory Co., Ltd. F-term (reference) 5C054 AA05 FC01 FC12 FC14 FF06 HA29

Claims (4)

[Claims] 1. A lane detecting device for detecting lane information in an image captured from a camera by image processing, comprising: a characteristic amount detecting means for detecting a predetermined characteristic amount from the captured image data; A labeling unit for extracting and labeling a feature in the image based on the labeling unit, a storage unit for storing the labeling result, and comparing the current labeling result with the past labeling result, and the labeling results match at substantially the same position in the frame. A lane detecting device comprising: a noise removing unit that removes data from data as noise; and a detecting unit that detects lane information from a remaining labeling result. 2. The method according to claim 1, wherein the noise removing unit determines that the data is noise when there is data that coincides with the past labeling result at substantially the same position in the frame for a predetermined period. 2. The lane detecting device according to claim 1, wherein when the distance is high, the distance is shorter than when the distance is low. 3. The lane detecting device according to claim 2, wherein the noise removing unit compares the labeling results a plurality of times in a predetermined cycle within the predetermined period. 4. The lane detecting device according to claim 2, wherein the noise removing unit compares the result with a labeling result in a frame before the lapse of the predetermined period.