JPS62221800A - Recognition equipment for lane on road - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention recognizes the lanes drawn on the road surface on which a car is running, such as the outside road line, the roadway center line, and the roadway boundary line. The present invention relates to a lane recognition device on a road surface that issues a warning to a driver based on the relative positional relationship between the road surface and the vehicle.

[Prior Art] Safety devices required when driving a car include a drowsy driving prevention device, a rear visibility device, and the like. It is also desired to monitor the driving condition of a vehicle, issue a warning to the driver when an abnormal driving condition occurs, and execute control to correct the driving condition. Furthermore, there is also a need for a monitoring means that automatically recognizes road signs and the like and notifies the driver of the recognition results so that safer driving operations can be performed.

As a method for monitoring driving conditions, it is considered that the scenery in front of the moving vehicle is photographed, and based on this photographed image, for example, road signs, etc. are recognized. For example, a device as shown in Japanese Unexamined Patent Publication No. 59-128693 has been proposed. This device generates a warning when the vehicle deviates from its lane, but it is difficult to accurately recognize the presence of lanes when there is noise in the lane. is difficult. Moreover, this device recognizes traffic lanes, etc. from a grayscale pattern expressed in white and black, and it is difficult to recognize sign images with colors using this type of technology. .

[Problems to be Solved by the Invention] This invention was made in view of the above-mentioned points. It is possible to judge whether or not the vehicle is traveling accurately based on the lane drawn on the road surface, to effectively inform the driver of the driving condition, and to issue a warning. The system enables more effective safe driving, and in particular, it is equipped with a lane recognition device on the road surface that can accurately recognize the existence of lanes even when there is damage to the painted lanes. This is what we are trying to provide.

[Means for Solving the Problems] That is, in the road lane recognition device according to the present invention, the road surface in front of the vehicle is photographed by a color imaging device such as a color television camera, and the image signal is recorded. Based on this, color image signals corresponding to each color such as R (red), G (green), and B (blue) are formed, and from this color image signal, the features of the image signal of the color representing the lane are extracted. The presence of a lane is determined based on the image signal from which features are extracted. In this case, if there is damage to the lane on the image, the damaged portion is corrected. Then, the relative relationship between the image of the lane and the position of the vehicle is determined and displayed, and the driver is notified by other means.

[Operation] In the lane recognition device as described above, a color image representing a lane is extracted from a color image signal. Therefore, an image of that color is extracted without being affected by the brightness of the road surface, shadows, etc., and the presence of a lane can be recognized from the pattern of this extracted image. Once the position of the lane is recognized on the screen specified in this way, the driving position of the vehicle can be clearly determined from the relationship with the position of the vehicle on the screen. The relative positional relationship between the lane and the vehicle will be displayed in the form of an image, allowing the driver to easily recognize whether the driving condition of his or her vehicle is normal or not. . For example, if the vehicle is in an abnormal state such as straddling lanes, this will be detected and a warning will be issued to the driver. In this case, if there is noise etc. in the recognized image, a correction is made to remove this noise and correct the image, making it possible to recognize lanes more reliably. be.

Examples of the invention] Hereinafter, one embodiment of the present invention will be described with reference to the drawings.

FIG. 1 shows its configuration, which includes a color television camera 11 that photographs the road surface in front of a traveling vehicle. For example, as shown in FIG. 2, this television camera 11 is installed on the front of an automobile 12, and is designed to monitor the road surface in front of the automobile 12 so that the driver pays particular attention while driving. Note that in this figure, an area 13 surrounded by a broken line is effectively photographed. In this figure, 141 is a boundary line with the opposite direction lane represented by, for example, a yellow line, and 142 and 143 are white lanes that demarcate the range of each lane.

A color video signal of an image photographed by such a television camera 11 is supplied to a decoder 15. This decoder 15 converts R (red) from the color video signal.
, G (green), and B (blue) color image signals are separated and extracted, and the RlG and B color image signals are supplied to the image signal processing section 16. This image signal processing section 1
Step 6 recognizes white or yellow marking lines drawn on the road surface and calculates the relative positional relationship between the marking lines and the vehicle.

The recognition result obtained by signal processing in the signal processing unit 1B is displayed on an image display unit I7 such as a CRT or liquid crystal display so that the driver can easily recognize the recognition result.

FIG. 3 shows the configuration of the image signal processing section 16. The R and GSB color image signals obtained by the decoder 15 are supplied to a feature extraction section 161.

The feature extraction unit 161 binarizes the input image signal and extracts only information corresponding to the object to be recognized, in this case, the partition line. As a specific extraction method, extraction conditions for white and yellow that form the partition line are set, and pixels that satisfy the conditional expression of the color expressing the partition line are converted into binarized information among each pixel on the screen. This is to extract and output as .

Here, the conditional expression for extracting the white partition line is as follows.

IR-Gl<ε/10 IG-Bl<ε/10 IB-R1<ε/10 ε/2<R, G, B・・・・・・・・・(1) Here R
, G, and B have a range of values from 0 to ε.

Further, the conditional expression for extracting the yellow lot line is as follows.

RIB, and G>B (2) Then, the binary image information whose features are extracted based on such a conditional expression is stored in the memory 162. This image information storage is executed, for example, every 0.05 seconds. Based on the image information stored in the memory 102, a recognition unit 163 recognizes the presence of a white or yellow lane marking, and a calculation unit 164 calculates the relative positional relationship between this lane and the vehicle. It is.

FIG. 4 shows the flow of the operation of the device configured as described in 1. For example, when the ignition switch of this vehicle is turned on, this process is started.

First, in step 101, settings are made for the scanning part of screen-L,
Initial settings for setting conditions for feature extraction as described above are executed. In the next step 102, R, G, and B image data signals are extracted from the photographed video signal, and in the next step 103, these image data signals are sent to the feature extraction section of the image signal processing section 16, and for example, a partition line is Execute feature extraction of the white and yellow colors being represented. In this feature extraction, as described above, the input image is binarized to extract only information related to the object to be recognized, and the feature-extracted image data is stored in the memory 162 in step 104.

The store of this feature-extracted image data is, for example, 0,
Executes every 0.5 seconds. The stored image data is then sent to the recognition unit 163 in step 105, and the recognition unit 163 determines whether the image data stored in the memory 162 represents a lane. The existence of lanes is determined by determining whether there is a lane or not based on the pattern.

There are various means for determining the existence of this lane, one example of which will be explained below. FIG. 5 shows the state of the image captured by the television camera 11. This image corresponds to the range 13 shown by the broken line in FIG. Specifically, it corresponds to an image of the road surface within a range of 2 m to 6 m in front of the camera 11 position.

Then, a specified setting range 51 is provided on the image shown in FIG. 5, and it is determined whether or not there is a white or yellow pixel from which features have been extracted within this setting range 51. If it is determined that a white or yellow pixel exists within this setting range 51, it is checked whether or not it extends in one direction as a stroke, and this stroke extends to the top of the screen. If it is confirmed that there are lane markings, it is determined that a marking line (lane) exists in this video. however,
This stroke may be a continuous line segment or a broken line.

If the existence of a lane is confirmed in this way, the process proceeds to the next step 107. In this step 107, the image data stored in the memory 162 is sent to the calculation unit 164, and the relative positional relationship between the own vehicle and the confirmed lane is calculated.

Specifically, the distance between the front wheels of the vehicle and the lane is calculated from the stroke position within the setting range 51 shown in FIG. 5 and the inclination of the entire stroke. In other words, the larger the angle of the lane on the screen, the farther away the vehicle is from that lane. For example, in the case of FIG.
If the angles of lanes 52 and 53 are equal and the directions of inclination are opposite, it is determined that the vehicle is running in the center of lanes 52 and 53. This calculation result is then displayed on the display unit 17 in step 107.

In this case, the display method on the display unit 17 is such that, for example, as shown in FIG. 6, a vehicle is shown in the center, and a marking line is displayed in relation to the vehicle using, for example, means such as light emission. Alternatively, the distance between the vehicle and the front wheels may be displayed numerically.

Therefore, the driver can always easily and reliably know the relative positional relationship between the vehicle and the lane on the display unit 17, and the driving condition along the lane can be recognized not only intuitively but also concretely. Become.

Here, when performing the lane recognition operation as described in -1-, it is effective to take measures against noise in lane recognition due to dirt or disappearance of the lane, for example. This process may be executed, for example, in step 103 of feature extraction in the process flow shown in FIG.
03 as shown in FIG.

That is, this step 103 is a step 501 of feature extraction processing to extract white and yellow, a step 502 of performing a diffusion process on 8 neighborhoods of the pixels constituting the lane recognized by the feature extraction, and a shrinkage process of the 8 neighborhoods. 503.

Now, to explain the above-mentioned 8-neighborhood diffusion processing and contraction processing, it is assumed that the image of, for example, a white or yellow lane is as shown in FIG. 8(A). In this figure, each solid-lined square is a pixel that actually indicates a lane, and the broken-lined square indicates pixels that are assumed to exist around the actual pixel. The 8-neighborhood refers to the 8 pixels that are adjacent to one pixel, and therefore the 8-neighborhood diffusion spreads the area to the 8 neighbors surrounding one pixel. Therefore, the image shown in Figure 1 (A) becomes an image formed by adding one pixel each around the actual pixel, as shown in Figure (B). In addition, 8-neighborhood shrinkage removes pixels other than pixels that have areas in all 8 neighborhoods, and the image in Figure 1 (B) becomes the one shown in Figure 13 (C) of the same figure. It is.

In other words, it performs processing for various sizes of local dirt on the lane. For example, if the area of the white line is reduced due to dirt, or if it is partially cut off due to dirt, It becomes difficult to track strokes.

In such a case, after the feature extraction as described above, the area of the white line or yellow line is subjected to repair processing.

Even if such a repair process is executed, if the lane is so dirty that it cannot be repaired, the lane determination in step 105 cannot be performed and the existence of a lane cannot be recognized. However, this means that the image is captured at the next timing and the recognition process is executed, so if an area that is not heavily soiled is captured next, it will be determined whether there is a lane at that point. .

In addition, if the lane area is expanded due to dirt, this will not affect the determination conditions in the lane existence determination step described next, and may cause problems in lane recognition. Must not be.

FIG. 9 shows the lane determination step 105 in the process flow shown in FIG. 4 in more detail.
First, in step 601, a white or yellow area existing within the setting range 51 is extracted. Then, a first determination step 602 is executed based on this extraction result.

In this first determination step 602, small areas existing due to noise or dirt are removed, and the state of the extraction area existing within the set range 51 is determined.

That is, if the area existing within the setting range 51 is part of a lane, its area W satisfies the following relationship.

W≧WoxX.

However, Wo is the number of pixels corresponding to the minimum value of the lane (marking line). Xo is the number of pixels in the vertical direction of the setting range 51. (direction signs, pedestrian crossings, speed signs, etc.), so in the next second determination step 603, the area within the setting range 51 is treated as a stroke (a line segment of a certain width or a certain width), and the two sides of the screen are Check to see if it has extended.

If this stroke is recognized as extending to the top of the screen, this is determined to be a lane.

If it is determined in this second determination step 603 that the stroke is broken midway, then: a) the lane ahead is hidden by the vehicle in front, etc.;

b) The lane is a broken line.

C) Not a lane.

There are three possibilities: Therefore, if such a determination result is obtained, the image is re-captured in step 604, and the features of the image are extracted. Based on this extracted image, the presence or absence of a stroke is determined and stored in step 605, and step 60
In step 6, the number of times of this judgment storage operation is counted and judged. Specifically, in step 606, it is determined whether the number of strokes has reached 50. If the number of strokes has not reached 50, the process returns to step 604 and repeats the process of feature extraction and stroke determination. let Here, this repetition period is performed, for example, at intervals of 100 +ll5Qc.

When the above judgment storage operation is repeated 50 times, the process proceeds to the next third judgment step 607.

In this third determination step 607, if a white or yellow area appears continuously within a predetermined screen, this is determined to be a solid line lane, and if it appears intermittently, this is determined to be a broken line lane. It is.

In the explanation so far, the relative positional relationship between the traveling vehicle and the lane is displayed, but if, for example, the vehicle is traveling out of the lane,
It is also possible to notify the driver of this.

FIG. 10 shows the flow of processing for executing such judgment and notification operations. First, in step 201, initial settings such as feature extraction conditions are performed, as in the previous embodiment, and in step 202. Enter the image data with .

Then, in step 203, feature extraction is performed to extract image data corresponding to the color of the lane, and in step 204, it is determined whether a lane exists. And step 205
to calculate the relative positional relationship between the vehicle and the lane.

In step 206, it is determined whether the vehicle is out of the lane based on the calculation result in step 205. If it is determined that the vehicle is out of the lane, in step 207 the direction indicator is turned on. Based on the operating state, it is determined whether the vehicle is changing lanes. If it is determined that the turn signal is not operated while the vehicle is running out of the lane, the process proceeds to step 208 and the driver is notified of the vehicle running out of the lane. As this notification means, a sound such as a buzzer may be used.

Furthermore, if the operating state of the turn signal is confirmed in step 207, the process proceeds to step 209, where it is determined whether or not the lane that has protruded is yellow. If the lane is yellow, this is a display prohibiting a change of course, so proceed to step 210 and use a buzzer etc. to notify the driver that the driver is about to change direction while the change of direction is prohibited. Make it. Further, this notification means may be a synthetic voice such as "You are running out of the vehicle" or "You are prohibited from changing course."

Therefore, if it is possible to notify such a lane crossing condition, it can be effectively used as a warning means especially for warning of a drowsy driving condition.

It can also be effectively used as a means of caution when driving in areas where course changes are prohibited.

In the explanation so far, we have explained that the television camera can clearly capture the scenery in front of the vehicle, but for example, at night, only the image illuminated by the headlights is received. Become.

Headlights installed on a vehicle have a limited range of light; for example, the area indicated by the broken line in Figure 11 is illuminated by a rad light. A portion of the image will now be captured. Therefore, the range in which the white (yellow) line appears on the captured screen is necessarily limited to the range within the broken line above, and it is effective to perform processing that takes this into account when lane recognition is performed at night. It is true.

FIG. 12 shows the flow of recognition processing in such a case of nighttime. First, in step 301, initial settings are performed as in the previous embodiments, and in step 302, it is determined whether the headlights are on or not. This can be determined from the condition of the headlight switch, etc. If it is determined that the headlights are off and the vehicle is not driving at night, full image tracking processing is executed in step 303. If the lighting state of the headlights is confirmed in step 302, then in step 304 screen processing is executed to limit the tracking range on the screen to the range surrounded by the broken line in FIG. 11.

Then, in step 305, the image data of the screen that has been tracked in step 303 or step 304 is captured, and in step 305, features are extracted from this image data. Thereafter, based on the extracted feature data, the process for determining the presence of lanes, etc., is executed in the same manner as in the previous embodiments.

Additionally, if the road turns sharply, the system can track the stroke of the lane on the screen and calculate its curvature to recognize the curve that appears in front of the vehicle.

FIG. 13 shows the configuration of a lane recognition device including a warning means when such a sharp curve appears. Similar to the case shown in FIG. 1, a video signal captured by a television camera 11 is sent to a decoder 12 The RSG and B signals from the decoder 12 are supplied to the image signal processing section 13 to execute image processing.

In addition, a running vehicle has a speed sensor 18 and a blinker switch 19 that issues a direction indicator operation command, and the signals from this sensor 18 and blinker switch 19 are recognized by the image processing section 16. It is supplied together with the result to a calculation/judgment unit 20 comprised of a CPU or the like. The calculation/judgment unit 20 calculates the curvature (curve radius) of the stroke of the lane extending on the screen;
Judging from the distance to the curve and the vehicle speed, if it is deemed dangerous for safe driving, the display and notification section 21 displays it, and a warning is generated by a buzzer or the like.

In the explanation of the embodiments so far, the imaging means is set to photograph the scenery in front of the vehicle, but the imaging means is set to face rearward and the imaging means is set to photograph the scenery behind the vehicle. If the images from the vehicle are processed in the same manner as in the previous embodiments, it will be possible to effectively recognize, for example, the lane markings of a parking lot that one enters when driving in reverse, and the boundaries of the restricted area of the parking lot can be recognized effectively. The parking position can be accurately recognized based on the correlation with the vehicle body.

[Effects of the Invention] As described above, according to the lane recognition device according to the present invention, the lane drawn on the road surface can be reliably recognized, and the driving position of the vehicle can be easily determined in relation to the lane. Moreover, it can be displayed in a way that the driver can reliably recognize it. Therefore, it is possible to easily and reliably confirm the state of safe driving along the lane, and in particular, it is possible to reliably recognize the relative positional relationship between the lane and the vehicle. Since it is possible to notify the driver, it can be effectively used as a warning means for the driver.

[Brief explanation of drawings]

FIG. 1 is a block diagram illustrating a lane recognition device according to an embodiment of the present invention, FIG. 2 is a diagram illustrating the installation state of an imaging device used in the device described in 1., and FIG. FIG. 4 is a flowchart explaining the operating state of the above device, FIG. 5 is a diagram showing the state of a captured image, and FIG. 6 is a diagram showing the display state. Figure 7 is an explanatory diagram, and Figure 8 (A) to (C) is a diagram explaining in more detail the feature extraction steps in the process flow described in 1.
= 23- is a diagram explaining the recognized lane state and the state after diffusion and contraction processing, respectively. FIG. 9 is a diagram also explaining a more detailed state of lane determination processing, and FIG. FIG. 11 is a flowchart illustrating the flow of processing for warning a lane crossing condition using a device such as this; FIG.
FIG. 2 is a flowchart illustrating an example of nighttime recognition processing using headlights, and FIG. 13 is a configuration diagram illustrating an embodiment of a recognition device that recognizes a sharp curve. 11...Color television camera, 12...Vehicle, 141.142...Lane, 15...Decoder, 1
6... Image signal processing unit, 17... Display unit, 20...
・Calculation judgment part. Applicant's representative Patent attorney Takehiko Suzue Figure 4 Figure 6 Figure 10 Figure 11 ♀

Claims (1)

[Scope of Claims] A color imaging device for photographing a road surface in front of a traveling vehicle, means for converting a video signal photographed by this imaging device into a color image signal corresponding to each color, and a color image signal obtained by this means. a feature extraction means for extracting a color image signal corresponding to the color of the lane from the colored image signal; and a feature extraction means based on the feature image extracted by the feature extraction means.
The vehicle is equipped with a recognition means for recognizing that the vehicle is in a lane and obtaining a corrected recognized image by removing noise, and a means for calculating a relative position between the characteristic image recognized as a lane by this means and the vehicle. , A lane recognition device on a road surface, characterized in that the calculated result is notified to a driver.