JP2020201766A - Lane deviation driving alert system and lane deviation driving alert method - Google Patents

Abstract

To provide a lane deviation driving alert system capable of detecting from an image a state where a driver is deviating from a lane while driving a vehicle with high accuracy.SOLUTION: According to the embodiment, a lane deviation driving alert system includes a camera and a calculation processing unit. The camera captures an image of the front of a vehicle. The calculation processing unit executes calculation processing for determining whether or not the vehicle is deviating from a lane based on the image. The calculation processing unit detects a line segment on the road surface from the image, and determines that the vehicle is deviating from the lane when the line segment is continuously detected for a certain period of time and the slope of the line segment calculated using coordinates on the image does not transition from positive to negative or from negative to positive.SELECTED DRAWING: Figure 1

Description

Embodiments of the present invention relate to a lane deviation travel alert system and a lane deviation travel alert method.

In recent years, various techniques for preventing accidents caused by vehicles such as automobiles have been proposed. As one form of them, for example, a technique of detecting a deviation from a lane and issuing a warning to a driver is known.

Recently, for example, drive recorders for recording the situation at the time of an accident as an image have become widespread. For this reason, drive recorders are also equipped with a function for detecting, for example, lane deviation from an image.

Japanese Unexamined Patent Publication No. 2011-227551 Japanese Unexamined Patent Publication No. 9-39602 Japanese Unexamined Patent Publication No. 2001-176000 Japanese Unexamined Patent Publication No. 11-208306

By the way, in the case of a drive recorder that is retrofitted to a vehicle, it cannot be linked with a vehicle standard device such as a blinker or a steering wheel. Therefore, in the function installed in this type of drive recorder, for example, to detect a lane deviation from an image, the driver intends an event in which the vehicle crosses the lane due to a lane change intended by the driver. There is a possibility of false detection as a deviation from the lane that should be warned.

An object to be solved by the present invention is to provide a lane deviation traveling alert system and a lane deviation traveling alert method capable of detecting a state of traveling deviating from a lane from an image with high accuracy.

According to the embodiment, the lane deviation travel alert system includes a camera and an arithmetic processing unit. The camera captures an image of the front of the vehicle. Based on the image, the arithmetic processing unit executes arithmetic processing for determining whether or not the vehicle is traveling out of the lane. The arithmetic processing unit detects a line segment on the road surface from the image, and when the line segment is continuously detected for a certain period of time, the line segment is calculated using the coordinates on the image. When the inclination does not change from positive to negative or negative to positive, it is determined that the vehicle is traveling out of the lane.

The figure which shows an example of the hardware configuration of the lane deviation driving alert system of embodiment. The flowchart which shows the flow of the basic operation of the lane deviation driving alert system of embodiment. The figure which shows one setting example of various areas set on the image in order to detect a white line from an image in the lane deviation driving alert system of embodiment. The figure which shows the image which detects the white line region from an image in the lane deviation driving alert system of embodiment. The figure for demonstrating the method of calculating the inclination of the white line applied in the lane deviation driving alert system of embodiment. The flowchart which shows the flow of the white line detection processing in the lane deviation driving alert system of embodiment. The flowchart which shows the flow of the steering wheel operation presence / absence determination processing in the lane deviation driving alert system of embodiment. The flowchart which shows the flow of the lane deviation driving alert processing in the lane deviation driving alert system of embodiment. The figure which shows an example of the image which is imaged at the time of lane change in the lane deviation driving alert system of embodiment. The figure which shows the inclination of the white line calculated in the lane deviation driving alert system of embodiment in time series. The figure which shows the presence or absence of the lane deviation driving alert of the lane deviation driving alert system of embodiment according to a situation.

Hereinafter, embodiments will be described with reference to the drawings.

FIG. 1 is a diagram showing an example of the hardware configuration of the lane deviation travel alert system 1 of the present embodiment. Here, it is assumed that the lane deviation travel alert system 1 is realized on a drive recorder of a type that is retrofitted to the vehicle. The drive recorder is an example, and the lane deviation travel alert system 1 can be realized as various devices, not limited to the drive recorder, as long as an image of the front (traveling direction) of the vehicle can be obtained.

Here, lane deviation driving is intended by the driver, except for an event in which the vehicle crosses the lane under the condition that the driver intentionally operates the steering wheel, for example, when changing lanes or driving on a curve. It means the event that the vehicle is traveling out of the lane. That is, in the following, when the vehicle is traveling out of the lane, it means that the vehicle is traveling out of the lane unintentionally, and the driver intentionally operates the steering wheel. It does not include the situation where the vehicle crosses the lane under the situation. In addition, lane deviation driving includes a meandering state called wobbling.

As shown in FIG. 1, the lane deviation travel alert system 1 has, for example, an arithmetic processing unit 11 composed of a CPU (Central Processing Unit). Further, the lane deviation driving alert system 1 includes a camera 12, an image processing unit 13, a GPS (Global Positioning System) receiving unit 14, an acceleration sensor 15, a gyro sensor 16, and a wireless communication unit 17, which are connected to the arithmetic processing unit 11. For example, it has an alert output unit 18 and a memory 19 composed of a speaker and an LED (light-emitting diode). Further, the lane deviation travel alert system 1 has an interface unit (I / F unit) 20 connected to the memory 19.

The arithmetic processing unit 11 is a device that controls the overall control of the drive recorder including the operation as the lane deviation driving alert system 1. For example, by executing various programs stored in the memory 19, various arithmetic processing can be performed. Execute. The various programs include a lane deviation travel alert program 100 for operating the drive recorder as the lane deviation travel alert system 1.

The camera 12 is an imaging unit for continuously capturing an image of the front (traveling direction) of the vehicle in which the drive recorder is installed as a moving image. That is, the drive recorder is installed in the vehicle so that the camera 12 captures an image in front of the vehicle. The image captured by the camera 12 is stored in the memory 19 via, for example, the arithmetic processing unit 11. The camera 12 may be provided separately from the drive recorder main body. In this case, it is only necessary to consider the installation position of the camera 12, and it is possible to increase the flexibility regarding the installation position of the drive recorder main body.

The image processing unit 13 is composed of, for example, a GPU (Graphics Processing Unit), and performs various processes on the image captured by the camera 12. The image processing unit 13 may acquire the image directly from the camera 12, or may indirectly acquire the image stored in the memory 19 via, for example, the arithmetic processing unit 11. The arithmetic processing unit 11 may perform the processing performed on the image by the image processing unit 13. That is, the image processing unit 13 can be omitted.

The GPS receiving unit 14 is a device that receives GPS signals for measuring the position of the vehicle in which the drive recorder is installed. Further, the acceleration sensor 15 and the gyro sensor 16 are devices that acquire acceleration and angular acceleration as auxiliary information for improving the accuracy of position measurement. The arithmetic processing unit 11 executes arithmetic processing for position measurement using the data from the GPS receiving unit 14, the acceleration sensor 15, and the gyro sensor 16. The measured position information is stored in the memory 19 as, for example, image attribute information. The angular acceleration obtained by the gyro sensor 16 is used not only for position measurement but also for determination of lane deviation travel, which will be described later.

The wireless communication unit 17 is a device for wirelessly communicating with a predetermined external device outside the vehicle via, for example, the Internet. Here, it is assumed that the initial setting value necessary for determining the lane deviation travel is acquired from the server which is the predetermined external device. The server manages information (vehicle height, vehicle width, etc.) about the vehicle on which this drive recorder is installed. By acquiring this information from the server and performing the initial settings, the bonnet position and vanishing point of the vehicle in which the drive recorder is installed in the image, which will be described later, are determined. The initial setting is executed when a state in which the initial setting is not performed is detected when the vehicle starts traveling. In other words, if the drive recorder is installed and the vehicle is driven, the initial settings will be automatically executed. In the initial setting, after tentatively applying the initial setting value acquired from the server, fine adjustment to adapt to the installation status of this drive recorder is performed using various data including at least images acquired by this drive recorder. Is executed. The server is, for example, a computer provided by a business operator that provides a support service for driving a vehicle.

The alert output unit 18 is a notification unit for issuing a warning to the driver when it is detected that the vehicle in which the drive recorder is installed is traveling out of the lane. When the alert output unit 18 is composed of a speaker, a warning sound is output, and when the alert output unit 18 is composed of an LED, the lighting or blinking is performed. The alert output unit 18 may be composed of, for example, both a speaker and an LED.

The memory 19 is a non-volatile storage composed of, for example, an HDD (Hard Dick Drive) or an SSD (Solid State Drive). The interface unit 20 is a device for storing various data stored in the memory 19, such as an image captured by the camera 12, in the memory card 2. The interface unit 20 executes communication conforming to the SD (Secure Digital) card standard or the USB (Universal Serial Bus) standard. The memory card 2 is, for example, a non-volatile storage such as an SD card or a USB memory that can be attached to and detached from the drive recorder.

Next, a mechanism by which the main lane deviation driving alert system 1 realized on the drive recorder having the above configuration detects the lane deviation driving will be described. Here, it is assumed that the drive recorder is operated as the lane deviation travel alert system 1 by executing the lane deviation travel alert program 100 by the arithmetic processing unit 11. More specifically, it is assumed that the arithmetic processing unit 11 has a function of detecting the lane deviation travel of the vehicle in which the drive recorder is installed by using the data from the camera 12 and the gyro sensor 16. As described above, this drive recorder has a wireless communication unit 17. Therefore, for example, the data from the camera 12 and the gyro sensor 16 is transmitted to the server by the wireless communication unit 17, the server side executes arithmetic processing for detecting the lane deviation travel, and the wireless communication unit 17 receives the result. Then, it is also possible to give a warning by the alert output unit 18. That is, the lane deviation travel alert system 1 is not limited to being realized by the drive recorder alone, and can be realized, for example, by the cooperation of the drive recorder and the server.

First, with reference to FIG. 2, the basic operation flow of the lane deviation traveling alert system 1 of the present embodiment will be described.

Here, it is assumed that the lane deviation travel alert system 1 starts the operation at the timing when the drive recorder is started and ends the operation at the timing when the drive recorder is stopped. Further, it is assumed that the drive recorder starts at the timing when the vehicle is driven and stops at the timing when the vehicle stops (drive stops). That is, it is assumed that the lane deviation travel alert system 1 starts the operation at the timing when the vehicle is driven and ends the operation at the timing when the vehicle stops (drive stops). Then, the lane deviation travel alert system 1 repeatedly executes each process according to the procedure shown in the flowchart of FIG. 2 during the period from the driving of the vehicle to the stop (driving stop).

First, the arithmetic processing unit 11 determines whether or not there is a steering wheel operation from the white line detection process (step S11) that detects a white line (line segment) from the image captured by the camera 12 and the angular acceleration acquired by the gyro sensor 16. The steering wheel operation presence / absence determination process (step S12) is executed. Here, it is assumed that the line segment indicating the lane on the road surface is white. The white line detection process in step S11 and the handle operation presence / absence determination process in step S12 are preferably executed as parallel processes as shown in FIG. 2, but for example, depending on the performance of the arithmetic processing unit 11, serial processes May be executed as. The processing of the image captured by the camera 12 is partially or entirely executed by the image processing unit 13 under the control of the arithmetic processing unit 11, but in the following, the arithmetic processing unit 11 simply executes the processing. That is. Further, in the following, a line segment indicating a lane may be simply referred to as a lane. That is, in addition to using the "lane" intentionally for the traveling lane, the "lane" may be used intentionally for the line segment indicating the traveling lane.
Here, the line segment indicating the lane on the road surface includes the following cases.
In the case of 2 lanes on each side: the boundary line between the driving lane and the overtaking lane In the case of 3 lanes on each side: the boundary line between the 1st driving lane and the 2nd driving lane, and the boundary line between the 2nd driving lane and the overtaking lane Further, the line segment is not limited to white, and may be any color such as orange and yellow.

Then, the arithmetic processing unit 11 detects a state in which the vehicle is traveling out of the lane and issues a warning based on the processing results of the white line detection process in step S11 and the steering wheel operation presence / absence determination process in step S12. The lane departure driving alert process (step S13) is executed.

Subsequently, the details of each of the white line detection process (step S11), the steering wheel operation presence / absence determination process (step S12), and the lane deviation travel alert process (step S13) will be sequentially described.

First, the white line detection process (FIG. 2: step S11) will be described with reference to FIGS. 3 to 6.

FIG. 3 is an example of an image captured by the camera 12, and is a diagram showing an example of setting various regions set on the image in order to detect a white line from the image.

As described above, in the lane deviation driving alert system 1, the bonnet position a1 in the image captured by the camera 12 is determined by the initial setting using the initial setting value acquired from the server by the wireless communication unit 17 provisionally. Be done. Further, in the initial setting, the arithmetic processing unit 11 sets the rectangular area above the bonnet position a1 in the image as the screen average brightness calculation area a2 based on the bonnet position a1. The screen average brightness calculation area a2 is an area set for obtaining the brightness used for determining, for example, whether it is daytime or nighttime. That is, the lane deviation driving alert system 1 does not obtain the average brightness of the entire image captured by the camera 12 as the screen average brightness, but obtains the average brightness of the region substantially outside the vehicle in the image as the screen average brightness. .. An initial setting value indicating the screen average brightness calculation area a2 may be given by the server.

Further, as described above, in the lane deviation driving alert system 1, the vanishing point a3 in the image captured by the camera 12 is set by tentatively using the initial setting value acquired from the server by the wireless communication unit 17. Is determined. The vanishing point a3 is an infinity point in perspective, and is a point determined by only one point in the image of the front (traveling direction) of the vehicle captured by the camera 12. In the initial setting, the arithmetic processing unit 11 further sets a trapezoidal region below the vanishing point a3 in the image and above the bonnet position a1 as a white line detection region based on the vanishing point a3 and the bonnet position a1. Set to a4. The white line detection region a4 is set so that a pair of non-parallel pairs of opposite sides are located on a line connecting the vanishing point a3 and both side ends of the vehicle, and the lower side is separated from the bonnet position a1 by a certain distance. By narrowing the search range on the image for detecting the lane on the road surface to the white line detection area a4, the lane deviation driving alert system 1 reduces the amount of calculation and does not require high-performance processing power. I have to. FIG. 3 is an image captured by the camera 12 in a state where the vehicle is traveling without deviating from the lane. Although the white line is shown in the image, it is outside the white line detection area a4, so the processing related to this white line is performed. It can be omitted. Further, when the white line which is a lane is detected from the white line detection area a4 set in this way, it can be determined that the vehicle is in a state of straddling the lane. The initial setting value indicating the white line detection area a4 may also be given by the server.

Further, the arithmetic processing unit 11 further sets the road surface luminance calculation area a5 between the lower side of the white line detection area a4 in the image and the bonnet position a1 in the initial setting. The initial setting value indicating the road surface brightness calculation area a5 may also be given by the server.

Next, using the screen average brightness calculation area a2, the white line detection area a4, and the road surface brightness calculation area a5 set in the image as illustrated in FIG. 3, the arithmetic processing unit 11 sets the white line area from the image. The detection method will be described.

FIG. 4 is a diagram showing an image of detecting a white line region from an image captured by the camera 12. In FIG. 4, (A) shows the entire image captured by the camera 12 including the white line detection region a4, and (B) shows the white line region b1 detected from the image captured by the camera 12.

The arithmetic processing unit 11 calculates the road surface brightness using the brightness of each pixel in the road surface brightness calculation area a5. The arithmetic processing unit 11 searches the white line detection area a4 for pixels having a brightness equal to or higher than the threshold value of the road surface brightness, and detects the white line area b1 as shown in FIG. 4 (B). Here, the arithmetic processing unit 11 calculates the screen average brightness calculation using the brightness of each pixel in the screen average brightness calculation area a2, and determines the threshold value for detecting the white line area b1. For example, the threshold is switched between daytime and nighttime. The switching of the threshold value is not limited to two stages such as daytime and nighttime, and may be performed in three or more stages, or the threshold value is changed linearly, for example, according to the calculated screen average brightness. May be good. Further, the arithmetic processing unit 11 detects a region composed of a certain number of pixels (brightness higher than the road surface brightness by a threshold value or more) as a white line region b1.

By the way, the region detected by the difference from the road surface brightness is not necessarily the region corresponding to the white line. Therefore, the arithmetic processing unit 11 bases the detected region on the width (length in the width direction (horizontal direction)) and height (length in the longitudinal direction (vertical direction)), and the region corresponds to the white line. Determine if it is an area. If the width is equal to or greater than the threshold value or the height is less than the threshold value, the arithmetic processing unit 11 determines that the area does not correspond to the white line.

If both the width and the height satisfy the criteria, the arithmetic processing unit 11 further determines whether or not the slope (absolute value) of the white line is equal to or greater than the threshold value. When the inclination is equal to or greater than the threshold value, the arithmetic processing unit 11 determines that the white line is a lane. This threshold value is set according to, for example, the vehicle height and speed. A method of calculating the slope of the white line will be described with reference to FIG.

It is assumed that the image captured by the camera 12 has a resolution of 1280 pixels (horizontal) × 720 pixels (vertical). Further, it is assumed that the upper left of the image is the origin (0,0), the x-axis is taken in the horizontal direction, the y-axis is taken in the vertical direction, and the coordinate value at the lower right of the image is (1280,720). The horizontal direction of the image corresponds to the horizontal direction of the real space within the imaging range of the camera 12, and the vertical direction of the image corresponds to the vertical direction of the real space.

Here, consider a case where a white line is detected with the position (x1, y1) as one end and the position (x2, y2) as the other end, as shown in FIG. In this case, the arithmetic processing unit 11 sets the slope α of the white line.
α = (y2-y1) / (x2-x1) ... (Equation 1)
Calculated by. Then, when the absolute value of the calculated value is equal to or greater than the threshold value, the arithmetic processing unit 11 determines that the white line is a lane.

For example, when (x1, y1) = (660,450) and (x2, y2) = (620,590), the slope α of the white line is
α = (590-450) / (620-660) =-3.5
Will be. Assuming that a value corresponding to 45 degrees or more with respect to the left-right direction (vertical direction to the traveling direction) of the vehicle is set as a threshold value, the absolute value is equal to or more than the threshold value (1 [45 degrees]). Therefore, the arithmetic processing unit 11 determines that this white line is a lane.

In FIG. 5, the coordinates at both ends of the white line are shown as the coordinates for calculating the slope α of the white line, but the coordinates are not limited to this, and the slope α of the white line is calculated from the coordinates of any two points on the white line. It may be calculated.

FIG. 6 is a flowchart showing the flow of the white line detection process (FIG. 2: step S11) in the lane deviation travel alert system 1.

The arithmetic processing unit 11 calculates the screen average brightness for the screen average brightness calculation area a2 set in the image captured by the camera 12 (step S21). Further, the arithmetic processing unit 11 calculates the road surface brightness for the road surface brightness calculation area a5 set in the image captured by the camera 12 (step S22). The process of calculating the screen average brightness in step S21 and the process of calculating the road surface brightness in step S22 may be executed as parallel processing, or may be executed as serial processing in the reverse order of the order shown in FIG. You may.

Next, the arithmetic processing unit 11 detects the white line region b1 for the white line detection region a4 set in the image captured by the camera 12 (step S23). Specifically, using a threshold value determined based on the screen average brightness calculated in step S21, a region having a brightness higher than the threshold value calculated in step S22 is detected from the white line detection area a4. Here, the arithmetic processing unit 11 detects, in the white line detection region a4, a region composed of a certain number of pixels (brightness higher than the road surface brightness than the threshold value) as the white line region b1.

When the white line area b1 is detected from the white line detection area a4 (step S24: YES), the arithmetic processing unit 11 determines whether or not the width and height of the white line area b1 are within the respective threshold values (step S25). .. When the width of the white line region b1 is less than the threshold value and the height is equal to or greater than the threshold value (step S25: YES), the arithmetic processing unit 11 subsequently determines whether or not the slope (absolute value) of the white line region b1 is equal to or greater than the threshold value. (Step S26). For example, it is determined whether or not the temperature is 45 degrees or more with respect to the left-right direction of the vehicle (the direction perpendicular to the traveling direction).

When both the width and the height satisfy the criteria and the slope (absolute value) is equal to or greater than the threshold value (step S26: YES), the arithmetic processing unit 11 determines that there is a white line which is the lane to be detected (step S26: YES). Step S27), the white line detection process is terminated.

On the other hand, when the white line area b1 is not detected from the white line detection area a4 (step S24: NO), or when either the width or the height of the detected white line area b1 does not satisfy the standard (step S25: NO). Or, when the slope (absolute value) of the white line region b1 is less than the threshold value (step S26: NO), the arithmetic processing unit 11 determines that there is no white line in the lane to be detected (step S28), and this The white line detection process ends.

The fact that the white line, which is a lane, is detected from the white line detection area a4 set as described above in the image captured by the camera 12, indicates that the vehicle is in a state of straddling the lane. However, since the white line, which is the lane, can be detected from the white line detection area a4 even when the driver intentionally operates the steering wheel, for example, when changing lanes or driving on a curve, the lane is detected from the white line detection area a4. The fact that the white line is detected does not necessarily indicate the lane deviation driving.

Next, the steering wheel operation presence / absence determination process (FIG. 2: step S12) will be described with reference to FIG. 7. FIG. 7 is a flowchart showing the flow of the steering wheel operation presence / absence determination process.

The arithmetic processing unit 11 calculates the moving average of the angular acceleration in the left-right direction (vertical direction to the traveling direction) of the vehicle based on the data from the gyro sensor 16 (step S31). For example, the moving average of the angular acceleration in the left-right direction for 100 milliseconds is calculated. Further, the arithmetic processing unit 11 calculates the sum of the moving averages of the angular accelerations (step S32).

The arithmetic processing unit 11 determines whether or not the sum of the moving averages of the angular acceleration is equal to or greater than the threshold value (step S33). If it is equal to or greater than the threshold value (step S33: YES), the arithmetic processing unit 11 subsequently determines whether or not the phenomenon in which the sum of the moving averages of the angular accelerations is equal to or greater than the threshold value continues for a certain period of time (step S34). For example, it is determined whether or not the threshold value is continuously exceeded for 500 milliseconds. When the phenomenon continues for a certain period of time (step S34: YES), the arithmetic processing unit 11 determines that there is a handle operation (step S35), and ends the handle operation presence / absence determination process. The phenomenon in which the sum of the moving averages of the angular acceleration exceeds the threshold value continues for a certain period of time, for example, when traveling on a curve. That is, here, it is determined whether or not the driver intentionally operates the steering wheel when traveling on a curve. This steering wheel operation is referred to as a strong steering wheel operation for convenience.

On the other hand, when the sum of the moving averages of the angular acceleration is less than the threshold value (step S33: NO) or when the phenomenon that the sum of the moving averages of the angular acceleration is equal to or more than the threshold value does not continue for a certain period (step S34: NO). The arithmetic processing unit 11 determines that there is no handle operation (step S35), and ends the handle operation presence / absence determination process.

In this way, the lane deviation driving alert system 1 determines whether or not the driver intentionally operates the steering wheel during curve driving without coordinating with a vehicle standard device such as a blinker or a steering wheel. It should be noted that, for example, even when changing lanes, the driver intentionally operates the steering wheel, but the steering wheel operation performed for gently skewing is determined to be no steering wheel operation in this steering wheel operation presence / absence determination process. Ru. For convenience, such a steering wheel operation is referred to as a weak steering wheel operation. In the case of a sudden lane change due to strong steering operation, it is determined that there is steering operation.

Next, with reference to FIGS. 8 to 10, a lane deviation travel alert process (FIG. 2: step S13) executed based on the processing results of the white line detection process and the steering wheel operation presence / absence determination process will be described.

FIG. 8 is a flowchart showing the flow of lane departure driving alert processing.

The arithmetic processing unit 11 determines whether or not the white line is detected by the white line detection process (step S41). When the white line is detected (step S41: YES), the arithmetic processing unit 11 subsequently determines whether or not it is determined by the handle operation presence / absence determination process that there is a handle operation (step S42). Then, when it is determined that there is no handle operation (step S42: NO), the arithmetic processing unit 11 proceeds to step S43. In other words, if the white line is not detected (step S41: NO), or if it is determined that there is a steering wheel operation (step S42: YES), the arithmetic processing unit 11 considers that the vehicle is not in a lane deviation driving state. This lane deviation driving alert process ends. For example, if the vehicle crosses the lane while the driver intentionally operates the steering wheel (strong steering wheel operation) while driving on a curve, no warning is given to the driver.

Note that steps S41 and S42 may be executed in the reverse order of the order shown in FIG. That is, when it is determined that there is no steering wheel operation, it may be determined whether or not the white line is detected.

Next, the arithmetic processing unit 11 determines whether or not the white line is continuously detected for a certain period of time or longer (step S43). In other words, it is determined whether or not the elapsed period from the start of detection of the white line is a certain period or longer. If the detection is continuous for a certain period of time or longer (step S43: YES), the arithmetic processing unit 11 proceeds to step S44. If the fixed period has not been reached (step S43: NO), the arithmetic processing unit 11 does not perform the alert at that time, and ends the lane departure driving alert processing. Therefore, for example, when a vehicle crosses a lane in a short period of time, no warning is given to the driver.

Here, with reference to FIGS. 9 and 10, it is determined whether or not the vehicle is traveling out of the lane based on the detection status of the white line from the image, which is executed in steps S44 to S45. The method of doing this will be described.

FIG. 9 is a diagram showing a transition state of inclination of a white line in an image captured by the camera 12 when a vehicle traveling on a road having two lanes on each side changes lanes.

More specifically, FIG. 9 shows an image at time t1 when a vehicle traveling in the left driving lane of two lanes on each side changes lanes to the right overtaking lane as shown by arrow c1 (temporally than time t1). The images at time t2 (behind the scenes) are arranged in an easy-to-understand and comparable manner.

In the white line detection process described above, the white line c2-1 is detected from the white line detection area a4 from the image at time t1, and the white line c2-2 is detected from the white line detection area a4 from the image at time t2. Further, the slopes of these white lines c2-1 and c2-2 are calculated in the white line detection process described above.

The white line c2-1 in the image at time t1 has a positive slope as the denominator (x2-x1) of the above (Equation 1) is positive (the numerator (y2-y1) is always positive). It is calculated. On the other hand, the white line c2-2 in the image at time t2 is calculated as having a negative slope because the denominator (x2-x1) is negative.

Therefore, when the white line is continuously detected for a certain period of time or longer and the inclination of the white line is transitioning from positive to negative or negative to positive, the arithmetic processing unit 11 basically changes lanes. It is determined that the vehicle straddles the lane (the boundary line between the traveling lane and the overtaking lane) and is not in a state of traveling out of the lane. In addition, the arithmetic processing unit 11 examines whether or not the period from the start of detection of the white line to the transition of the inclination of the white line is within a certain period, and if it is within a certain period, the vehicle is changed to a lane ( It is determined that the vehicle straddles the boundary between the driving lane and the overtaking lane) and is not in a state of traveling out of the lane. In other words, even if the slope of the white line has changed, if the white line has occurred for more than a certain period of time since it started to be detected, it is not due to a lane change, and the lane in the main lane deviation driving alert system 1 It is determined that the vehicle is in a meandering state called wobbling, which is a form of deviant driving. The fixed period for determining the length of the period from the start of detection of the white line to the transition of the slope of the white line is different from the fixed period for determining the length of the period during which the white line is continuously detected. For example, the fixed period of the former is about 1/2 of the fixed period of the latter.

FIG. 10 is a diagram showing the inclination of the white line in the case of lane change (case 1) and the inclination of the white line in the case of lane deviation traveling (case 2) side by side in a time-series and easy-to-understand manner.

In FIG. 10, the column "not detected" indicates that the white line was not detected, and the column in which the numerical value is described indicates that the white line which is the lane is detected and its inclination. Is shown.

When the white line is continuously detected for a certain period (d1) or more, the arithmetic processing unit 11 determines whether or not the vehicle is traveling out of the lane. Here, the arithmetic processing unit 11 determines whether or not the inclination of the white line has transitioned from positive to negative or negative to positive, and if not, the vehicle is traveling out of the lane. Is determined. In the case of Case 2, since the inclination of the white line continues to be positive and no negative transition has occurred, the arithmetic processing unit 11 determines that the vehicle is traveling out of the lane.

On the other hand, in the case of Case 1, the slope of the white line changes from positive to negative (d2). In this case, the arithmetic processing unit 11 determines whether or not the transition occurs within a certain period (d3) after the white line starts to be detected, and if it occurs within a certain period (d3), the lane. It is determined that the change is not in the state of traveling out of the lane. Unlike the case of Case 1, when the transition of the inclination of the white line occurs for more than a certain period (d3) after the white line starts to be detected, the arithmetic processing unit 11 deviates from the lane (wobble / meander). It is determined that the state is.

Therefore, when the vehicle crosses the lane (the boundary between the driving lane and the overtaking lane) while the driver intentionally operates the steering wheel (weak steering wheel operation) when changing lanes, the driver is asked. No warning is given. In addition, when the vehicle crosses a lane (the boundary line between the driving lane and the overtaking lane) due to wobbling or meandering, a warning is given to the driver.

Based on the above, the process returns to FIG. 8 and the explanation of the lane deviation driving alert process is continued.

When the white line is continuously detected for a certain period of time or longer, the arithmetic processing unit 11 determines whether or not the positive / negative of the slope of the white line has changed (step S44). If there is a transition (step S44: YES), the arithmetic processing unit 11 subsequently determines whether or not the transition has occurred within a certain period of time after the white line starts to be detected (step S45).

When the transition occurs within a certain period of time after the white line starts to be detected (step S45: YES), the arithmetic processing unit 11 deviates from the lane because the vehicle crosses the lane due to the lane change. It is determined that the vehicle is not in a driving state, and this lane deviation driving alert processing is terminated.

On the other hand, when the positive / negative of the slope of the white line has not changed (step S44: NO), or the positive / negative of the slope of the white line has changed, the transition occurs within a certain period after the white line starts to be detected. If not (step S45: NO), the arithmetic processing unit 11 determines that the vehicle is traveling out of the lane, and outputs a warning sound or outputs an LED via, for example, an alert output unit 18 composed of a speaker or an LED. A lane departure alert such as lighting or blinking is executed (step S46).

FIG. 11 is a diagram showing the presence / absence of a lane deviation travel alert of the main lane deviation travel alert system 1 when a white line, which is a lane, is continuously detected, according to the situation.

If there is a steering wheel operation (strong steering wheel operation) and there is a transition in the inclination of the white line, the lane deviation driving alert system 1 determines that the lane has changed and does not perform a lane deviation driving alert (first stage). ..

If there is a steering wheel operation (strong steering wheel operation) and there is no transition of the inclination of the white line, it is determined that the vehicle is traveling on a curve, and the lane deviation driving alert is not performed (second stage).

If there is no steering wheel operation (strong steering wheel operation), there is a transition of the inclination of the white line, and the transition occurs within a certain period of time, it is judged that the lane has changed and the lane deviation driving alert is not performed. (3rd stage).

If there is no steering wheel operation (strong steering wheel operation) and there is a transition of the inclination of the white line, but the transition does not occur within a certain period of time, it is judged as lane deviation driving (wobble / meandering) and lane deviation driving alert. (4th step).

If there is no steering wheel operation (strong steering wheel operation) and there is no transition in the inclination of the white line, it is determined that the vehicle has deviated from the lane, and a lane departure alert is issued (fifth stage).

As described above, in the lane deviation driving alert system 1, the transition of the inclination of the white line (lane) detected from the image captured by the camera 12 and the left-right direction (in the traveling direction) of the vehicle obtained by the gyro sensor 16. On the other hand, it is possible to prevent an erroneous alert of lane deviation intended by the driver such as a lane change from the change in the angular speed in the vertical direction.

That is, the lane deviation travel alert system 1 does not need to cooperate with a vehicle standard device such as a blinker or a steering wheel, and can detect a state of traveling out of the lane with high accuracy from an image. In addition, the lane deviation driving alert system 1 reduces the amount of calculation and provides high-performance processing capacity by narrowing the search range (white line detection area a4) for detecting the lane from the image captured by the camera 12. do not need. That is, it has a configuration suitable for implementation on, for example, a type of drive recorder that is retrofitted to a vehicle.

By the way, in the above description, an example of using an image in front of the vehicle (traveling direction) has been shown, but for example, an image in the rear of the vehicle (in the opposite direction of the traveling direction) can also be used. Also, when detecting the white line area, in addition to the method of detecting the white line area based on the brightness value, it is also possible to make a determination based on the R value, B value, and G value (pixel value) of each pixel constituting the image. I do not care. Calculate the average value of R value, G value or B value and the median value of the histogram from the same area as the screen average brightness calculation area a2, and set it to one or more of R value, G value and B value. On the other hand, a threshold value may be set, and the white line region b1 may be detected based on the pixel value to be evaluated among the R value, G value and B value of each pixel. In addition to making a judgment based on the moving average value of the angular velocity in the steering wheel operation presence / absence determination process, the presence / absence of the steering wheel operation is determined based on a value based on the angular velocity such as the transition of the angular velocity and the integral value of the angular velocity for each unit time. It doesn't matter.

Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other embodiments, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are also included in the scope of the invention described in the claims and the equivalent scope thereof.

1 ... Lane deviation driving alert system, 2 ... Memory card, 11 ... Arithmetic processing unit, 12 ... Camera, 13 ... Image processing unit, 14 ... GPS receiving unit, 15 ... Acceleration sensor, 16 ... Gyro sensor, 17 ... Wireless communication unit , 18 ... Alert output unit, 19 ... Memory, 20 ... Interface unit, 100 ... Lane deviation driving alert program.

Claims (10)

A camera that captures an image of the front of the vehicle,
Based on the image, an arithmetic processing unit that executes arithmetic processing for determining whether or not the vehicle is traveling out of the lane, and
Equipped with
The arithmetic processing unit
A line segment on the road surface is detected from the image,
When the line segment is continuously detected for a certain period of time and the slope of the line segment calculated using the coordinates on the image does not transition from positive to negative or from negative to positive, the above. Judge that the vehicle is traveling out of the lane,
Lane deviation driving alert system. Further equipped with a sensor for detecting the angular acceleration of the vehicle,
When the arithmetic processing unit continuously detects a phenomenon in which the sum of the moving averages of the angular accelerations in the left-right direction of the vehicle exceeds the threshold value for a certain period of time, the vehicle is in a state of traveling out of the lane. Judge not,
The lane deviation driving alert system according to claim 1. The lane deviation travel alert system according to claim 1 or 2, wherein the arithmetic processing unit determines that the vehicle is not in a state of traveling out of the lane when the line segment is not detected. The lane according to any one of claims 1 to 3, wherein the arithmetic processing unit determines that the vehicle is not in a state of traveling out of the lane when the inclination of the line segment is changed. Deviation driving alert system. When the period from the detection of the line segment to the transition of the inclination of the line segment is equal to or longer than the threshold value, the arithmetic processing unit determines that the vehicle is traveling out of the lane. The lane deviation driving alert system according to claim 4. The arithmetic processing unit
A part of the area above the hood position of the vehicle on the image is set as the average brightness calculation area.
A part of the area below the vanishing point on the image and above the bonnet position is set as the line segment detection area.
A part of the area below the line segment detection area on the image and above the bonnet position is set as the road surface brightness calculation area.
The line is obtained by extracting a region having a brightness higher than the road surface brightness obtained in the road surface brightness calculation region, which is higher than the threshold value applied according to the average brightness obtained in the average brightness calculation region, from the line segment detection region. Detect minutes,
The lane deviation driving alert system according to any one of claims 1 to 5. The lane deviation driving alert system according to claim 6, wherein the arithmetic processing unit applies either a daytime threshold value or a nighttime threshold value based on the average brightness obtained in the average brightness calculation area. In the arithmetic processing unit, among the regions extracted from the line segment detection region, the length in the longitudinal direction is equal to or greater than the threshold length in the longitudinal direction, and the length in the width direction is the length in the width direction. The lane deviation travel alert system according to claim 6 or 7, wherein a line segment region that is less than the threshold value of is detected as the line segment. The lane deviation travel alert system according to claim 8, wherein the arithmetic processing unit detects the line segment region whose inclination is equal to or greater than a threshold value calculated using the coordinates on the image as the line segment. It has a camera that captures an image of the front of the vehicle and a sensor that detects the angular acceleration of the vehicle, and detects a state in which the vehicle is traveling out of the lane in an electronic device mounted on the vehicle. It is a lane deviation driving alert method that issues a warning.
Detecting line segments on the road surface from the image captured by the camera
When the line segment is continuously detected for a certain period of time and the slope of the line segment calculated using the coordinates on the image does not transition from positive to negative or from negative to positive, the above. Judging that the vehicle is traveling out of the lane,
When the inclination of the line segment is transitioning, or when the phenomenon that the sum of the moving averages of the angular accelerations of the vehicle in the left-right direction detected by the sensor is equal to or greater than the threshold value is continuously detected for a certain period of time. Judging that the vehicle is not traveling out of the lane,
Lane deviation driving alert method equipped with.

Images