JP3655541B2 - Lane detector - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a lane detection apparatus using image processing that can be applied to an automatic driving system or a parking assistance system.
[0002]
[Prior art]
In recent years, a white line or the like indicating a driving lane on a road or the like is detected by image processing, and various information is provided to the driver based on the recognition result, and further, this recognition is used for control when the vehicle is automatically driven. Systems using the results have been proposed.
[0003]
As such a white line recognition technique, a technique disclosed in Japanese Patent Laid-Open No. 11-195127 is known. According to the publication, it is described that white lines can be accurately recognized by extracting edges in an image, labeling the extracted edges, and detecting edges having a pair of angles from the extracted edges.
[0004]
[Problems to be solved by the invention]
These images are acquired by a camera installed at the front or rear of the vehicle. However, if there are raindrops or dirt in front of the camera lens, the image data itself will deteriorate, making it difficult to recognize it accurately. become.
[0005]
SUMMARY OF THE INVENTION An object of the present invention is to provide a lane detection apparatus that is not easily affected by noise of image data due to raindrops or dirt adhering to a camera lens or the like, and that can be recognized accurately.
[0006]
[Means for Solving the Problems]
In order to solve the above problems, a lane detection device according to the present invention is a lane detection device that detects lane information in an image captured from a camera, and (1) detects a predetermined feature amount from the captured image data. Feature quantity detection means, (2) Labeling means for extracting and labeling feature parts in the image based on the detected feature quantity, (3) Storage means for storing the labeling result, and (4) Current labeling result Noise removal means that removes data that matches the labeling result at almost the same position in the frame as noise from the data, and (5) detection means that detects lane information from the labeling result after removing noise And.
[0007]
Raindrops, dirt, and the like adhering to the lens of the camera have a lower moving speed relative to the lens than the scenery outside the vehicle, and are almost stationary in the image frame. In the present invention, the past labeling result and the current labeling result are compared using this fact, and data matching at substantially the same position in the frame is determined as noise such as dirt, and is removed. Information is determined accurately.
[0008]
The noise removing means determines that the data is determined to be noise when there is data that coincides with the past labeling result at substantially the same position in the frame over a predetermined period, and when the vehicle speed is high during the predetermined period. It is preferable to make it shorter than when it is low.
[0009]
As described above, the noise is almost stationary in the frame of the image. Therefore, if there is data that matches the past labeling result over a predetermined period, that is, data that is stationary, it can be determined as noise. When the vehicle speed is high, image data such as landscape moves at a higher speed in the frame, so that it is possible to determine whether or not the data is stationary in a shorter time.
[0010]
In comparing the labeling results, the labeling results may be compared a plurality of times in a predetermined cycle within a predetermined period, or may be compared with the labeling results in a frame before the predetermined period has elapsed.
[0011]
DETAILED DESCRIPTION OF THE INVENTION
DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the invention will be described in detail with reference to the accompanying drawings. In order to facilitate the understanding of the description, the same reference numerals are given to the same components in the drawings as much as possible, and duplicate descriptions are omitted.
[0012]
FIG. 1 is a schematic configuration diagram showing a basic configuration when the lane detection device according to the present invention is applied to a lane departure warning device, and FIG. 2 is a diagram for explaining a use state of the CCD camera 2.
[0013]
As shown in FIG. 1, the lane detection apparatus according to the present invention includes a lane detection ECU 1 that receives and processes an image signal from a CCD camera 2, and has an image processing CPU 10 and a memory 11 inside. That is, the image processing CPU 10 also serves as a feature amount detection unit, a labeling unit, a noise removal unit, and a lane information detection unit according to the present invention by a built-in program. The lane detection ECU 1 is connected to a lane departure warning ECU 3 to exchange information with each other. The output signals of the vehicle speed sensor 41, the steering angle sensor 42, and the yaw rate sensor 43 are input to the lane departure warning ECU 3 as vehicle state quantities. The lane departure warning ECU 3 predicts the future trajectory of the vehicle from the vehicle state quantity, and compares it with the lane position detected by the lane detection ECU 1 to determine that the vehicle departs from the lane. Etc. are used to alert the driver.
[0014]
The CCD camera 2 is arranged at the front center of the vehicle 5 and acquires an image of the road surface in front of the vehicle through the windshield 50. A lane (lane) 60 on the road surface 6 on which the vehicle 5 travels is partitioned by left and right lane markings (lane marks) 61R and 61L.
[0015]
The operation of the lane detection apparatus according to the present invention will be described with reference to FIGS. FIG. 3 is a flowchart for explaining the operation of this apparatus. FIGS. 4 and 5 show a series of raw images acquired by the CCD camera and the labeling results.
[0016]
The flow shown in FIG. 3 is repeatedly executed by the image processing CPU 10 in the lane detection ECU 1 in accordance with a predetermined interval, for example, the image acquisition of the camera after the ignition switch is turned on.
[0017]
First, in step S1, the vehicle state quantity is read from the vehicle speed sensor 41, the steering angle sensor 42, and the yaw rate sensor 43 via the lane departure warning ECU 3. Subsequently, in step S2, image data is taken from the CCD camera 2.
[0018]
In step S3, the outline of the image, that is, the edge is detected as a feature amount from the acquired image data, for example, image data as shown in FIGS. In these images, raindrops 71 and 72 adhering to the front portion 5a of the vehicle 5 and the windshield 50 are shown in addition to the lane marks 61R and 61L and the other vehicle 62 preceding the other lane. Each of these contours will be detected.
[0019]
In step S4, the detected edges are connected to perform line differentiation. Thereafter, the line segments detected in step S5 are numbered, that is, labeled. The methods of edge detection, line differentiation, and labeling from this image data are described in pages 249 to 256 of “Image Engineering Handbook” (Koji Watanabe, Asakura Shoten, 1986). FIG. 5 shows a state in which edges are detected from the image data shown in FIGS. In the figure, the labeled numbers are indicated with L. That is, the raindrops 71 and 72 in the original image are numbered 1 and 2, respectively, the lane mark 61L is numbered 3, the vehicle front 5a is numbered 4, the lane mark 61R is numbered 5, and the other vehicle 62 is numbered 6. Is done.
[0020]
In step S6, the result of labeling, that is, the extracted line segment shape and position information is stored in the memory 11.
[0021]
Steps S7 to S11 are steps for removing noise such as raindrops 71 and 72. For example, consider the processing at the time of FIG. 4 (c) and FIG. 5 (c). Here, line segments L1 to L6 labeled No. 1 to No. 6 are detected. This is compared one by one with the past labeling results in order from No. 1, for example, the labeling results at the time of FIGS. 5 (a) and 5 (b).
[0022]
Specifically, first, in step S7, the first data L1 is selected, and in step S8, it is determined whether or not line data having the same shape exists in substantially the same position in the past data. The data L1 is data of the raindrops 71 and is attached to the windshield 50 of the vehicle 5 and moves together with the vehicle 5 in the same manner as the CCD camera 2, and therefore there is almost no moving speed inside the screen. Therefore, it is determined that the data matches the past data, the process proceeds to step S9, and this data is removed as noise.
[0023]
Here, the determination of the presence of coincidence data is made at a substantially identical position compared to a method in which a line segment data having a similar shape is present at substantially the same position over a predetermined period as compared with data before a predetermined time. There are two types of methods that consider matching when line data of similar shape exists. In any case, it is preferable to change the length of the predetermined period and the predetermined time interval according to the motion state of the vehicle from the data such as the vehicle speed and the steering angle obtained in step S1. This change may be stepwise, and may be switched depending on whether or not it is a specific vehicle speed threshold value or more.
[0024]
When the vehicle is moving at a high speed or when the steering angle is large, the moving speed of the lane mark in the image is also increased, so the past data used for matching determination may be only the most recent data. Judgment with high accuracy is possible. On the other hand, when moving at a low speed and a small steering angle, the moving speed of the lane mark in the image is also slowed down. It is necessary to use data at the time.
[0025]
In the next step S11, unless the data is the final label data (L6 in this case), the process proceeds to step S12, one label number is added, the next data is selected, and the process returns to step S8. Since the data L2 representing the raindrop 72 and the data L4 representing the vehicle front portion 5a are also coincident, they are removed as noise in step S9. In addition, scratches and attached dirt on the lens of the CCD camera 2 and the windshield 50 can be removed by the same method. Since the position and shape of the vehicle front portion 5a are known, edge detection or the like may be performed by excluding this portion at the time of input in step S2.
[0026]
On the other hand, in the case of the data L3 and L5 representing the lane marks 61L and 61R and the data L6 representing the other vehicle 62, there is a movement of the position and a movement of the shape. The process proceeds to S10. Then, it is left as valid data for the next processing.
[0027]
In this way, by performing the lane mark detection process for only the data L3, L5, and L6 remaining as valid data in step S13, it is possible to remove the influence of noise and recognize the lane marks 61L and 61R with high accuracy.
[0028]
Note that if the data excluded as noise occupies more than a certain area (for example, 10% or more), it is determined that the image data is unreliable because there are many scratches and dirt on the camera. It is preferable to warn the driver to that effect without performing the detection process.
[0029]
In the above description, the embodiment in which the CCD camera 2 is mounted in the front center of the passenger compartment has been described. However, the installation of the camera is not limited to this position. The present invention can also be applied to a camera that is arranged behind the camera and shoots behind the rear window. Further, it is possible to detect a lane mark without noise by a similar method even if a camera is used which is disposed in front of, outside, or behind the vehicle and shoots a lane mark at any position.
[0030]
Further, as information for detecting the lane mark, edge extraction is performed as a feature amount. However, a feature amount may be extracted by binarization or the like.
[0031]
Here, an example has been described in which the detected lane mark information is used for a lane departure warning, but it can also be used for automatic driving of a vehicle, parking assistance, and the like.
[0032]
【The invention's effect】
As described above, according to the present invention, noise such as raindrops and dirt moves together with the camera, and features that are extracted from the image and labeled are used to remove data with little movement as noise. Thus, it is possible to accurately detect the lane mark by accurately removing these noises.
[Brief description of the drawings]
FIG. 1 is a schematic diagram showing a lane departure warning device using a lane detection device according to the present invention.
FIG. 2 is a diagram showing a use state of a CCD camera of the apparatus of FIG.
FIG. 3 is a flowchart for explaining the operation of the lane detection apparatus according to the present invention.
FIG. 4 is a diagram showing an example of a raw image change acquired by a CCD camera.
5 is a diagram showing a labeling result in each image of FIG. 4; FIG.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... Lane detection ECU, 2 ... CCD camera, 3 ... Lane departure warning ECU, 6 ... Road surface, 10 ... Image processing CPU, 11 ... Memory, 41 ... Vehicle speed sensor, 42 ... Steering angle sensor, 43 ... Yaw rate sensor, 50 ... Windshield, 60 ... lane (lane), 61 ... lane line (lane mark).

Claims (4)

A lane detection device for detecting lane information in an image captured from a camera by image processing,
Feature quantity detection means for detecting a predetermined feature quantity from the captured image data;
Labeling means for extracting and labeling features in the image based on the detected feature values;
Storage means for storing the labeling results;
A noise removing unit that compares the current labeling result with the past labeling result, and removes data having the same labeling result at substantially the same position in the frame as noise from the data;
Detection means for detecting lane information from the remaining labeling results;
A lane detection device comprising: The noise removing means determines that the data is determined to be noise when there is data that coincides with a past labeling result at substantially the same position in the frame over a predetermined period, and when the vehicle speed is high during the predetermined period. The lane detection device according to claim 1, wherein the lane detection device is shorter than a low case. The lane detection apparatus according to claim 2, wherein the noise removing unit compares the labeling results a plurality of times at a predetermined cycle within the predetermined period. The lane detection apparatus according to claim 2, wherein the noise removing unit compares the result of labeling with a frame before the predetermined period has elapsed.

Images