JP2009252198A - Travel environment presuming device, method and program, and traffic lane deviation alarm device and steering assisting apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To continue to presume a stable travel environment even when a while line cannot be recognized by presuming the width of traveling road with the same accuracy as that of white line recognition. <P>SOLUTION: This travel environment presuming device includes: an onboard camera 10; a horizontal edge extracting part 12 for extracting an original image horizontal edge e with a camera coordinate system; a coordinate system transforming part 14 for transforming the original image horizontal edge e into a horizontal edge E in a world coordinate system 54 on the basis of a predetermined coordinate coefficient 56; a relation determining part 16 for calculating a target width W and a target distance L from the horizontal edge E and determining a relation 58 between the target width W and the travel width Wr of the travel environment on the basis of the predetermined assumed target width Wp; and a travel road width presuming part 18 for presuming the travel road width Wr on the basis of the relation 58 and the target width W. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

The present invention relates to a technical field that is mounted on a moving body and estimates a driving environment, and more particularly to a driving environment estimation device that estimates a lane width or the like in a driving environment based on an input image from a camera, a method thereof, and a program therefor. .
The present invention also relates to a lane departure warning device for detecting and warning a departure from a lane based on the estimation of the driving environment.
The present invention further relates to a steering assist device that assists steering of a moving body based on estimation of a traveling environment.

Conventionally, by detecting the white line of the road (the line on the road that guides the driver to the road lane as well as the white line), it detects the lane width of the driving environment, warns of deviation from the lane, There is technology to assist.
Patent Document 1 discloses a method of controlling a lane departure warning in consideration of a side slip angle of a vehicle body in order to prevent the driver from feeling uncomfortable with a departure warning even during high-speed turning.
In Patent Document 2, a radar is provided in addition to a camera for the purpose of continuing driving support based on a lane even when a white line of a traveling lane cannot be detected. And a method of calculating a virtual white line of a traveling lane from a movement locus of a preceding vehicle is disclosed.
Patent Document 3 discloses a method for extracting a plurality of feature points from an image ahead of the host vehicle and detecting a travel boundary based on the moving speed and arrangement of each feature point for the purpose of detecting the travel boundary with high accuracy. Is disclosed.

Japanese Patent Laid-Open No. 2002-193055 JP 2007-8281 JP 2007-316685 JP

In the lane departure warning and steering assist based on white line recognition, there are cases where the white line is unclear, or the white line on both roads cannot be detected due to road dirt or weather conditions.
In Patent Document 1, when white line recognition is not possible, the warning of lane departure is stopped, so the commercial value of the departure warning device is lowered.
In Patent Document 2, a preceding vehicle is detected by a radar separately from the white line detection unit, and a deviation is determined. However, in the detection of the preceding vehicle by the radar, the lateral resolution is low, it is difficult to accurately measure the width of the preceding vehicle, and it cannot be determined that the vehicle is the same vehicle. Inaccuracy remains to determine this. For this reason, the driver's distrust of the system is invited and the product value is lowered. In addition, there is a drawback that the cost of the system becomes high by separately mounting a radar of another device.
In Patent Document 3, while enormous image processing such as extraction of feature points of an image, speed calculation for each pixel, and grouping by speed is necessary, how to stably estimate the lane width for various driving environments, Or, it is not disclosed whether to determine unpredictable. In particular, there is an inconvenience that it is difficult for the driver to understand the difference between the driving environment in which the driving path width can be estimated and the driving environment in which the driving path width cannot be estimated.

[Technical Problem 1] Thus, in the above-described conventional example, the road width cannot be estimated without using radar and enormous image processing while maintaining the detection accuracy of the road width equal to that of white line recognition. There was an inconvenience.
[Technical Problem 2] Further, the conventional example described above has a disadvantage that the reliability of the system cannot be increased by making it easy for the driver to understand the process of making the estimation impossible.
[Technical Problem 3] In addition, the above-described conventional example has a disadvantage that the operation rate of the estimation process cannot be increased at a low cost regardless of the state of the white line in the estimation process of the travel path width. It was.

[Object of the Invention] The object of the present invention is to estimate the traveling road width with the same accuracy as white line recognition, and to continue to estimate a stable driving environment even when white line recognition is impossible.
Another object of the present invention is to make it easy for the driver to understand that the travel path width cannot be estimated.

  [Aspects] The inventor of the present invention has focused on the fact that the width of the preceding vehicle is good in order to estimate the traveling road width from information other than the white line with the same accuracy as the white line recognition. And, in order to calculate the width of the preceding vehicle by information processing, the above problem can be solved by extracting the horizontal edge of the image and performing various information processing on the horizontal edge. I came up with an idea.

[Problem Solving Means 1] A first group of the present invention corresponding to Example 1 is a camera that captures a driving environment according to a driver's field of view, and an original image horizontal edge e of the captured image in a camera coordinate system. A horizontal edge extracting unit for extracting, a coordinate system converting unit for converting an original image horizontal edge e in the camera coordinate system into a horizontal edge E in the world coordinate system based on a predetermined coordinate conversion coefficient, and the horizontal The object width W and the object distance L are calculated from the edge E, and the relation between the object width W and the traveling road width Wr of the traveling environment is determined based on a predetermined assumed object width Wp. It has a configuration in which an association determination unit and a traveling path width estimation unit that estimates the traveling path width Wr based on the association and the object width W are provided.
Thereby, the technical problem 1 has been solved.

[Problem Solving Means 2] The second group of the present invention corresponding to the second embodiment is similar to the first group in that the camera, the horizontal edge extracting unit, the coordinate system converting unit, the relation determining unit, the traveling road width, and the like. And an estimation unit.
The problem solving means 2 particularly recognizes a lane in the travel environment from the captured image and calculates a travel line width, a travel path width Wr calculated by the white line recognition unit, and a steering angle. A lane departure time TLC is determined based on Φm, and when a white line is not recognized by the white line recognition unit, a lane departure time TLC is determined from the traveling road width Wr estimated by the traveling road width estimation unit. The lane departure time determination unit and an alarm output unit that outputs an alarm in accordance with the lane departure time TLC are employed.
Thus, the above technical problems 1, 2 and 3 have been solved.

[Problem Solving Means 3] The third group of the present invention corresponding to the third embodiment is similar to the first group in that the camera, the horizontal edge extracting unit, the coordinate system converting unit, the relation determining unit, the traveling road width, and the like. And an estimation unit.
The problem solving means 3 includes a steering amount calculation unit that calculates a steering amount that guides the host vehicle to the estimated traveling road width Wr, a steering drive control unit that steers the host vehicle based on the steering amount, It has the structure of having.
Thereby, the technical problem 1 has been solved.

  The present invention interprets the meaning of the terms described in each claim in consideration of the description of the present specification and the drawings, and certifies the invention according to each claim. There are the following advantageous effects in relation to

[Operation Effect 1 of the Invention] The traveling environment estimation device of the problem solving means 1 calculates the object width of the preceding vehicle in the world coordinate system from the original image horizontal edge e, and the traveling path of the traveling environment from the width of the preceding vehicle. Calculate the width.
Therefore, the traveling road width Wr can be estimated with the same accuracy as the white line without using the information on the white line on the road surface. Further, when the horizontal edge is used, the contrast in the driving environment is high and the luminance difference is large. Therefore, the width information can be obtained with high accuracy by relatively simple image processing without using a radar or the like.

[Operation Effect 2] The lane departure warning device of Problem Solving Means 2 detects the horizontal edge of the preceding vehicle only by image processing when the white line of the road cannot be recognized, and estimates the traveling road width.
Accordingly, since the inoperative state of the departure warning system due to the inability to recognize the white line can be reduced, the driver's reliability to the system can be increased and the convenience can be improved.

[Operation Effect 3 of the Invention] The steering assist device of the problem solving means 3 estimates the travel path width using the object width W of the preceding vehicle, and automatically follows the preceding vehicle.
Accordingly, it is possible to automatically follow a road without drawing a white line on a road in a site such as a factory.

  Three embodiments will be disclosed as the best mode for carrying out the invention. The first embodiment is a driving environment estimation device 100, a method and a program, the second embodiment is a lane departure warning device 102 shown in FIG. 11 and the like, and the third embodiment is a steering assist device 104 shown in FIG. Embodiments including Examples 1 to 3 are referred to as embodiments.

<1.1 Estimating driving environment from horizontal edge>
First, Example 1 of this embodiment is disclosed. In the first embodiment, the driving environment is estimated with the same accuracy (accuracy and precision) as the white line recognition without using the white line of the driving environment.
Referring to FIG. 1, the travel environment estimation device 100 according to the first embodiment includes a camera 10, a horizontal edge extraction unit 12, a coordinate system conversion unit 14, an association determination unit 16, and a travel path width estimation unit 18. ing.

  The camera 10 captures the driving environment according to the driver's (driver) field of view, and inputs the original image 10 a in the camera coordinate system 52. The shooting range should be the same as or wider than the field of view when the driver gazes forward.

  The horizontal edge extraction unit 12 extracts the original image horizontal edge e of the photographed original image 10 a using the camera coordinate system 52. The original image horizontal edge e is a portion where the luminance value changes in the vertical direction in the driver's field of view, and this change continues in the horizontal direction. The camera coordinate system 52 is a coordinate system on the xy plane in which the normal direction of the imaging plane of the camera 10 is the z direction. xy has, for example, the upper left as the origin.

  The coordinate system conversion unit 14 converts the original image horizontal edge e in the camera coordinate system 52 into the original image horizontal edge e in the world coordinate system 54 based on a predetermined coordinate conversion coefficient 56. The coordinate conversion coefficient 56 is a value obtained in advance by calibration of the camera 10. The world coordinate system 54 is a coordinate system for an object to be photographed, and has coordinate values corresponding to actual measured values of the traveling environment. When the coordinate value (p1, p2) of the original image horizontal edge e in the camera coordinate system 52 corresponding to the width of the object is converted into the coordinate value (P1, P2) in the world coordinate system 54, the original image horizontal edge e Is converted into the object width W in the traveling environment.

  The association determination unit 16 calculates the object width W and the object distance L from the horizontal edge E, and based on the predetermined assumed object width Wp, the object width W and the traveling path of the traveling environment A relation 58 with the width Wr is determined. The association 58 takes a value that is related or not related. For example, when it is determined that the horizontal edge E is a preceding vehicle that travels in front of the host vehicle, it is considered relevant.

  The association determination unit 16 can easily calculate the object width W and the object distance L from the coordinate values (P1, P2) of the horizontal edge E of the world coordinate system 54. Then, by comparing a predetermined assumed object width Wp with the object width W, it is possible to determine the relationship 58 regarding whether or not the object width W by the horizontal edge E is due to the vehicle. For example, if the object width W is within a width that can be assumed as a vehicle (assumed object width Wp), the original image horizontal edge e can be determined as a preceding vehicle. In this case, the relation 58 of the original image horizontal edge e is set as the preceding vehicle.

  Then, the traveling road width estimation unit 18 estimates the traveling road width Wr based on the association 58 determined by the association determination unit 16 and the object width W. In other words, the traveling road width estimation unit 18 adds the half lane width W_COM for expanding the width of the preceding vehicle to the traveling road width Wr with respect to the object width W by the horizontal edge E that is regarded as the preceding vehicle as the relation 58. Thus, the traveling road width Wr is estimated.

  FIG. 2 shows a configuration example of hardware resources common to the embodiments. In relation to the first embodiment, the driving environment estimation device 100 includes a controller 30 including one or more CPUs such as an ECU and an image processing dedicated chip, a memory 31 used by the controller 30, and the in-vehicle camera 10. And a vehicle speed sensor 32 and a storage device 42.

  The memory 31 may include a frame memory 31A that stores the original image 10a, an external memory 31B that is shared by a plurality of logical CPUs, and the like. The camera 10 may include an optical system and a CCD area sensor or a CMOS area sensor. In order to improve the processing speed and reduce the frame memory 31A, only odd (or even) line data (640 × 240) of the image (640 × 480) from the camera 10 may be employed. The vehicle speed sensor 32 inputs the host vehicle speed Vm to the controller 30. The storage device 42 is a non-volatile memory, and stores a program for operating the controller 30 as each unit shown in FIG. 1, a coordinate conversion coefficient 56, an assumed object width Wp, and the like.

  In addition, as a hardware resource of the present embodiment, a steering angle sensor 34, a winker control unit 38, and an alarm output unit 40 are provided in relation to Example 2. In addition, a steering angle drive control unit 36 is provided in relation to the third embodiment.

FIG. 3 shows a flowchart of the first embodiment. The camera 10 captures the driving environment according to the driver's field of view (imaging step S1). Subsequently, the horizontal edge extraction unit 12 extracts the original image horizontal edge e of the photographed image with the camera coordinate system 52 (horizontal edge extraction step S2). Further, the coordinate system conversion unit 14 converts the extracted original image horizontal edge e into the world coordinate system 54 based on a predetermined coordinate conversion coefficient 56 and the original image horizontal edge e in the camera coordinate system 52. To the original image horizontal edge e at (coordinate system conversion step S3).
Then, the association determination unit 16 calculates the object width W and the object distance L from the original image horizontal edge e, and based on the predetermined assumed object width Wp and the object width W, A relationship 58 between the object width W and the travel road width Wr of the travel environment is determined (relevance determination step S4). Subsequently, the traveling road width estimation unit 18 estimates the traveling road width Wr based on the relation 58 and the object width W (traveling road width estimation step S8).

  The controller 30 operates as each unit shown in FIG. 1 and the like by executing a program as a CPU, and processes each process shown in FIG. 3 and the like. Hereinafter, information processing examples of each unit and each process will be described.

Horizontal edge extraction processing S2 [Edge strength edge_Prewitt (Prewitt filter)]
The horizontal edge extracting unit 12 may extract the original image horizontal edge e by calculating the magnitude of the luminance gradient of the image by differential processing of the digital image. In order to dynamically capture the driving environment with the in-vehicle camera 10, it is desirable to effectively remove the influence of noise. For example, a Prewitt vertical filter may be used. In the Prewitt differential method, when the luminance value of the pixel at the coordinates (x, y) is f (x, y), the horizontal edge strength edge_Prewitt is obtained by the following equation (1). An edge having a horizontal edge strength edge_Prewitt of a certain level or more is specified as the original image horizontal edge e. For example, if there are a plurality of original image horizontal edges e in one screen, labeling or the like is performed.

Coordinate system conversion processing S3 [horizontal edge width W and distance L]
Referring to FIG. 4, the point p (x, y) in the camera coordinate system 52 corresponds to the point P (X, Y, Z) in the world coordinate system 54. In this case, the projective transformation formula of the following formula (2) is established. Since the world coordinate system is assumed on the road surface, Z = 0 and P (X, Y, Z) = P (X, Y, 0). In the equation (2), α, β, γ, u ₀ , υ ₀ , m are internal parameters of the camera related to the imaging system such as a lens, r ₁₁ , r ₁₂ , r ₂₁ , r ₂₂ , r ₃₁ , r ₃₂ , T _x , T _y and T _z are external parameters related to the installation position and orientation of the camera. These internal and external parameters are coordinate conversion coefficients 56 and are known values obtained by calibration with respect to the installed camera 10.
When the following expression (3) is given to the expression (2), the following expression (4) is obtained. Therefore, the coordinate system conversion unit 14 calculates P (X, Y) by the following equation (5). The coordinate value p (x, y) at both ends in the horizontal direction is extracted from the horizontal edge e of the original image, and the coordinate value P (X, Y) in the world coordinate system 54 is calculated according to equation (5). To do.

Relevance determination processing S4 [object width W and distance L calculation]
Referring to FIG. 5, the original image horizontal edge e can be specified by coordinates P1 (X1, Y1), P2 (X2, Y2) at both ends of the horizontal edge in the world coordinate system 54. From the coordinates P1 (X1, Y1) and P2 (X2, Y2), the object width W of the preceding vehicle is calculated by the following equation (6), and the distance L from the camera 10 is calculated by the following equation (7). Can do.

Related judgment processing S4 [identification of horizontal edge E by distance L]
When there are a plurality of horizontal edges E, it is preferable to select the horizontal edge E whose distance L is closest to the vehicle. When it is determined that there is no relationship in the determination of the relationship 58 based on the object width W, the next horizontal edge e of the original image with the closest distance L is specified. Since the horizontal edge E is specified for estimating the travel path width Wr, it is determined that the horizontal edge E far from the predetermined distance Lp and inappropriate for the estimation of the travel path width Wr is not related. You may do it. This predetermined distance Lp may be variable according to the speed of the vehicle so that a far distance is included in the range if it is fast.

Relevance determination processing S4 [Relation by object width W]
Referring to FIG. 6, the object width W of a preceding vehicle within a certain distance from the host vehicle is related to the travel path width Wr of the travel environment. When the obtained object width W is within the range of the assumed object width Wp, it is determined that the object width W has an association 58 with the travel path width Wr. In cases other than the assumed object width Wp, the horizontal edge E is removed as not being an automobile. The assumed object width Wp may be, for example, within an assumed object minimum width W_MIN = 1.4 [m] (light vehicle) to an assumed object maximum width W_MAX = 2.5 [m] (freight vehicle). A horizontal edge E that goes straight in the traveling direction of the traveling environment such as a stop line may be processed as a stationary object removal by a speed vector, which will be described later, or may be removed by this assumed object width Wp.

Traveling path width estimation process S8 [relation with object width W]
The travel path width estimation unit 18 assumes that the preceding vehicle travels with the center of the travel path width Wr in the travel environment as a guide, and uses a predetermined half lane width W_CON around the center position of Wr. To calculate.

-1.1 Effects of driving environment estimation from the horizontal edge As described above, when the driving environment is estimated from the horizontal edge e of the original image, the accuracy equivalent to that of the white line can be obtained even if there is a preceding vehicle without using the white line information. Thus, the travel path width Wr can be estimated. In addition, when the horizontal edge is used, the contrast in the driving environment is high and the luminance difference is large. Therefore, the width information can be obtained with high accuracy by relatively simple image processing.
In this way, if the road width is estimated by extracting the original image horizontal edge e of the preceding vehicle by image processing, the lateral resolution is high, unlike the forward vehicle detection by radar, so the width of the preceding vehicle is accurately measured. The region to be traveled can be estimated with high accuracy.
Furthermore, hardware resources (parts) necessary for the system configuration are a camera, a controller, and a memory, and it is not necessary to separately add a preceding vehicle detection device such as a radar, so that it can be realized at low cost.

<1.2 Relevance determination by correlation of multiple frames>
Next, an example in which the association 58 is determined using a plurality of frames of original images in the first embodiment will be described. In this example, the camera 10 continuously captures changes in the driving environment at a predetermined time difference td and stores them in the frame memory 31A. Referring again to FIG. 1, in this example, the association determination unit 16 includes an identity determination process 20, a speed calculation process 22, and an association determination process 24.

  Referring to FIG. 3 again, the identity determination processing 20 determines the identity of the horizontal edge E between images that are consecutive with the time difference td (identity determination step S5). This identity determination process 20 is based on the assumption that the horizontal edge E between successive images is the same object on the premise of a change in the relative positional relationship between the host vehicle and the preceding vehicle that occurred during the time difference td. It is determined whether or not.

  The speed calculation process 22 calculates the speed vector V in the world coordinate system 54 of the horizontal edge E (the object corresponding to the horizontal edge E) from the identity of the horizontal edge E (speed calculation step S6). The speed calculation process 22 obtains the speed vector V of the object using the time difference td and the movement distance of the original image horizontal edge e in the world coordinate system. The speed vector V is, for example, a preceding vehicle longitudinal speed Vv and a preceding vehicle lateral speed Vh.

  The relationship determination process 24 determines the relationship 58 between the object width W and the travel road width Wr from a predetermined assumed speed vector Vp and the calculated speed vector V (relevance determination processing step S7). . The assumed velocity vector Vp is preferably parallel to the lane and has predetermined values for the vertical component in the traveling direction and the horizontal component in the horizontal direction. Regarding the preceding vehicle vertical speed Vv, if the speed in the traveling direction of the horizontal edge E is smaller than the predetermined vertical component of the assumed speed vector Vp, it is determined as a stationary object, and the relationship 58 with the traveling road width W is determined. Judged as none. Further, if the preceding vehicle lateral speed Vh is greater than the lateral component of the assumed speed vector Vp, it may be determined that the preceding vehicle changes lanes or turns left or right, and the relationship 58 with the traveling road width Wr is determined to be none. it can.

  The apparatus and method shown in FIGS. 1 and 3 can be realized by a traveling environment estimation program to be executed using a CPU. This program is stored in the storage device 42 shown in FIG. 2, and is executed by the CPU of the controller. This program realizes information processing of each unit shown in FIG. 1 and the like and each process shown in FIG.

  FIG. 7 shows a detailed example of the information processing shown in FIG. Referring to FIG. 7, in the second embodiment, first, an image from the camera 10 is corrected (step S10). Next, the original image horizontal edge e is extracted by using a Previt filter, etc., and converted to the world coordinate system to calculate the horizontal edge E, and the coordinates (X1, Y1), (X2, Y2) of this horizontal edge E are calculated. ) To obtain the object width W (step S11, step S2 in FIG. 3).

Identity determination processing S5 [normal correlation processing]
Subsequently, it is confirmed whether or not the object width W is within the range of the assumed object width Wp (step S12). Here, it is confirmed whether or not the minimum assumed object width W_MIN <the object width W <the maximum assumed object width W_MAX is satisfied. If it is satisfied, it is determined that the horizontal edge E has a relation 58 with the traveling road width Wr, and then it is confirmed whether there is previous frame data (step S13).

  If there is a previous frame, correlation processing is performed (step S14). Referring to FIG. 8, a correlation window 20a of (width, height) = (wx, wy) [pixel] is provided from the position where the horizontal edge E of the current image t is detected. A window of the same size is provided at the position of the horizontal edge E of the image t-dt of the previous frame. The correlation value r is obtained for each pixel in the window according to the normal correlation equation (8) below.

Here, I is the luminance value of each pixel in the window of the time t image, I _ave is the average value thereof, J is the luminance value of each pixel in the window 20a of the time t-dt image, and J _ave is the average value thereof. N = wx * wy. This normalized correlation process is performed while moving the window 20a in the horizontal direction. If TH1 = (r * 100% value is 70% for r of 70% or more of the total number of processes), the same vehicle It is determined that there is (step S16). If the r group of Expression (8) is less than the threshold value, it is determined that there is no correlation between the horizontal edges E of the two frames.

  Further, following the No in step S15, the front edge of the horizontal edge E group that satisfies the minimum expected object width W_MIN <the target object width W <the maximum target object width W_MAX of the previous frame is adopted, Again, the correlation process shown in FIG. 8 may be used to check whether the vehicles are the same. When it is determined that they are not the same vehicle and there are a plurality of edges, the recognition rate of the preceding vehicle can be improved by performing the correlation process again on the horizontal edge in the direction farther away. If there is a correlation, the process proceeds to step S16.

Speed calculation process S6 [Speed calculation]
Subsequently, the speed V of the preceding vehicle is calculated. First, the average speed Vm of the own vehicle speed is calculated by the following equation (9), and the moving distance S of the own vehicle is obtained by the following equation (10). The preceding vehicle longitudinal speed Vv that is the moving speed of the object in the traveling direction is calculated by the following equation (11). Here, L _t and L _t-dt represent the distance to the host vehicle on the time t and t-dt images. Further, the preceding vehicle lateral speed Vh, which is the lateral speed of the preceding vehicle, is obtained by the following equation (12). Here, the preceding vehicle width center Wc _t, Wc _t-dt is the time t, represents the center position in the width direction of the preceding vehicle on t-dt image.

Related judgment processing S7 [Identify horizontal edge by velocity vector (stationary object)]
If the relative speed of the preceding vehicle vertical speed Vv is in the direction toward the host vehicle and is within ± 20% of the host vehicle speed, the vehicle is determined to be a stationary object (step S17), and the relation 58 is omitted. This is a process in which, when a horizontal edge E is detected from characters on the road surface, the traveling road width is not estimated from stationary objects such as characters on the road surface. Here, the character on the road surface is detected as a vehicle when the width is the same as the width of the vehicle, but is removed as a stationary object using the preceding vehicle longitudinal speed Vv. As a condition for removing a stationary object, it may be determined that the vehicle is a preceding vehicle when the speed is equal to or higher than a predetermined speed Vp by paying attention to the speed of the preceding vehicle.

Related judgment processing [Horizontal edge identification by speed vector (lane change, etc.)]
When the horizontal edge E is not a stationary object, it is further determined whether or not the preceding vehicle lateral speed Vh is greater than or equal to a predetermined lateral speed upper limit value Vh_MAX. It is determined as a change or a left / right turn, and the relationship 58 is none. That is, when the horizontal edge E is not laterally moved (step S18), it is determined as a vehicle and stored. In step S18, if the preceding vehicle lateral speed is equal to or higher than a certain level, the traveling road width is not estimated by the horizontal edge. The lateral speed upper limit value Vh_MAX can be calculated by the following equation (13). Referring to FIG. 6, Wr−W = 2 * Wt, and TTLC (Time To Lane Crossing) is the lane departure time of the preceding vehicle.

When the limitation is made by the lateral speed of the preceding vehicle, it is possible to automatically limit the estimation of the traveling path width by the preceding vehicle width when the traveling environment is a curve. By the restriction by the upper limit value of the preceding vehicle lateral speed Vh, a curve smaller than the curve R value calculated in advance according to the preceding vehicle speed can be excluded from the processing target. Due to the restriction by the preceding vehicle lateral speed Vh, the traveling road width is not estimated from the preceding vehicle width in the case of a curve that is largely bent, so that the accuracy of estimating the traveling road width and position from the preceding vehicle width can be ensured.
In addition, when the preceding vehicle makes a right or left turn, the speed of the preceding vehicle decreases, and therefore the estimation of the travel path width can be limited by the value of the preceding vehicle longitudinal speed Vv.

  In the information processing using the images of the two frames shown in FIG. 7, whether the horizontal edge E is a stationary object (step S17), or when moving horizontally (step S18), the previous frame and the current frame. The information is deleted (step S20), and the counter that has not detected the preceding vehicle is counted up (step S21). The value of the non-detection counter can be used when an alarm is given that the traveling width estimation process is not in operation.

Association determination processing [multiple frames]
If five or more frame memories 31A are prepared, the blinker information of the preceding vehicle and the information of the brake lamp can be measured and used by image processing. For example, blinking of the blinker is normally about 1 to 2 times / second, and the state of the blinker can be detected by taking a difference between images at this time interval. When the preceding vehicle blinks the blinker for changing the lane, the lane itself may decrease due to construction or an accident, which is useful for estimating the travel path width and position. Similarly, the brake can be detected by taking the difference between the images. Since the position of the brake lamp indicates the width of the preceding vehicle, it is possible to use such as correcting the width of the preceding vehicle by removing the influence of the shadow of the preceding vehicle.

Runway width guess S8
The horizontal edge E stored as a vehicle in step S19 in FIG. 7 has information on the relative distance of the horizontal edge with respect to the vehicle position, and the time series data is plotted as shown in FIG. Is obtained. The black dot represents the vehicle position, and the vertical line represents the detected horizontal edge E.
The vehicle position of the black spot can be estimated from the operation angle Φm of the steering angle sensor 34 and the vehicle speed Vm of the vehicle speed sensor 32. The estimated lane can be calculated by connecting points spread on both sides by the half lane width W_CON from the center point of the horizontal edge E at each time. The half lane width W_CON may be, for example, the half lane width W_CON = 3.5 / 2 = 1.75 determined by the “Road Structure Ordinance”. In addition, when the white line recognition is used together, the half lane width W_CON may be variable using the width of the white line recognized most recently. When information on a travel route such as a navigation system can be obtained from the outside, the half lane width W_CON may be variable based on the information or information on road width for each road stored in advance.

-1.2 Effect of determining the relationship with the correlation of multiple frames As described above, in the example using two frames, the stationary vehicle is removed, the left and right turn or the lane changing destination vehicle is removed, and the traveling locus of the preceding vehicle is determined. The travel road width Wr is calculated from the object width W of the vehicle ahead. For this reason, if there are two frame images, information processing can be performed, radar and the like are unnecessary, and high-speed processing can be performed at low cost. Further, since the identity of the preceding vehicle is determined by the correlation process shown in FIG. 8 and the traveling locus is specified, the vehicles in the adjacent lane can be accurately distinguished.

<2.1 Horizontal edge processing when white line cannot be recognized>
Next, Example 2 is disclosed.
Referring to FIG. 10, the lane departure warning device 102 according to the second embodiment is similar to the first embodiment in that the camera 10, the horizontal edge extraction unit 12, the coordinate system conversion unit 14, the association determination unit 16, And a width estimation unit 18. The association determination unit 16 may include an identity determination process 20, a speed calculation process 22, and an association determination process 24.
Particularly in the second embodiment, the lane departure warning device 102 includes a white line recognition unit 60, a lane departure time determination unit 62, and a warning output unit 40.

  The white line recognizing unit 60 recognizes a lane in the traveling environment from the captured image and calculates a traveling road width. A lane is a white line of white or other color on the road. The white line recognition unit 60 extracts an edge by differentiation, recognizes a straight line or a curved line (lane) from the feature of the edge, and calculates the traveling road width Wr from a plurality of lanes in the traveling environment.

  The lane departure time determination unit 62 determines the lane departure time TLC based on the travel road width Wr and the steering angle Φm calculated by the white line recognition unit 60. Further, the lane departure time determination unit 62 determines the lane departure time TLC from the travel road width Wr estimated by the travel road width estimation unit 18 when the white line recognition unit 60 does not recognize a white line. That is, in the second embodiment, when the white line is recognized, the traveling road width Wr is calculated from the distance between the lanes by the white line recognition, while the object width W of the preceding vehicle is used when the white line is not recognized. To estimate the road width Wr. Lane departure time TLC is the time it takes to cross the left and right lanes in the current driving state using the current steering angle Φm, the speed Vm of the host vehicle and the road width Wr, and the time is the level of lane departure But there is.

  The warning output unit 40 outputs a warning according to the lane departure time TLC. For example, when the turn signal is not operated, a warning may be given when there is a possibility of crossing a lane within one second. This is controlled by a combination of the winker control signal 38a from the winker control unit 38 shown in FIG.

White line extraction processing [edge strength edge_Sobel (Sobel filter)]
The white line recognizing unit 60 may recognize a straight line that becomes a white line from the original image 10a by the Sobel differential method and the straight line detection by the Hough transform. The luminance value of the pixel at each coordinate (x, y) of the original image 10a is defined as f (x, y), the luminance value change dx in the x direction is expressed by equation (14), and the luminance value change dy in the y direction is expressed by equation (15). Then, the edge strength edge_Sobel is obtained by the following equation (16).

  Referring to FIG. 11A, straight lines from points P1 to P3 are obtained in the original image 10a of the camera coordinate system as the edge strength edge_Sobel. Then, the distance ρ and the angle θ from the origin (lower left in the figure) to the coordinate value P2 can be easily obtained. P1, P2, and P3 are points on one straight line, and the coordinates are expressed as (x, y). If the distance from the origin to the straight line is ρ, and the angle between ρ and the X axis is θ, the following relational expression (17) holds.

In the straight line detection method using the Hough transform, if the coordinates of P1, P2, and P3 are substituted into the equation (17) and plotted in the ρ-θ coordinate system shown in FIG. I can draw. If P1, P2, and P3 are on one straight line, the intersection (ρ ₀ , θ ₀ ) of the curves as shown in the right figure can be obtained. In actual processing, the edge points detected by the Sobel filter are converted according to the equation (16), plotted in the ρ-θ coordinate system, and intersections (ρ ₀ , θ ₀ ) with high appearance frequency are found. For example, when the intersection point is detected, the corresponding linear equation is obtained by the following equation (18) and the following equation (19).

  A curve can also be approximated by a straight line by appropriately dividing the edge, and a lane can be obtained by searching for an intersection by this Hough transform.

Lane Departure Time Judgment Referring to FIG. 12, the lane departure time TLC is the time until the vehicle departs from the lane as shown by the dotted line at the current steering angle at the current speed at the position shown by the solid line. That is, using the current steering angle Φm and own vehicle speed Vm from the current position of the host vehicle, the time TLC until it crosses the left or right lane is calculated, and if TLC <T _Th [sec] Judge that. The TLC was calculated by the following equation (20), and the departure threshold second T _Th was set to 1 second. The departure threshold value T _Th may be variable according to the speed of the host vehicle.

Warning output processing When a deviation from the lane to the left or right within a deviation threshold T _Th seconds is determined, if the driver does not issue a turn signal in that direction, for example, a warning by a buzzer is output. In addition to the buzzer, a desirable human interface technique such as voice guidance when the present invention is implemented can be applied.

  FIG. 13 shows an example of an image. FIG. 13B shows an edge image when a horizontal edge filter is applied to an actual color image shown in gray scale in FIG. As shown in FIG. 13, when the white line on the right side of the driving lane is thin and the edge cannot be detected, the horizontal edge E of the preceding vehicle can be detected in the second embodiment, and is stored as the horizontal edge E of the preceding vehicle. Then, it is possible to estimate the traveling road width Wr using the horizontal edge E and determine the lane departure.

2.1 Effect of horizontal edge processing when white line cannot be recognized As described above, when the white line of the road cannot be detected, the horizontal edge of the preceding vehicle is detected and stored only by image processing, and the continuous vehicle width in time Estimate the road from the horizontal edge. Then, when there is a possibility that the own vehicle will deviate within a certain range, a warning can be issued. As a result, the system downtime due to the inability to recognize the white line on the road can be reduced. Thus, the combined use with the conventional lane departure warning system based on white line detection can greatly increase the system operation time, improving the convenience and reliability of the user. In particular, for example, when driving on an expressway for a long time, running on a monotonous, almost straight road, the attention is likely to be distracted, and fatigue is likely to accumulate. Can be increased.

  As described above, in the second embodiment, since the inoperative state of the departure warning system due to the inability to recognize the white line can be reduced, the reliability of the driver to the system can be increased. In addition, the spread of this system can reduce lane departures and cases where lanes are changed without taking out blinkers. In particular, when a departure warning is given during relatively long and relatively monotonous driving, it will be a support to the driver, and this technology will motivate the purchase of four-wheeled vehicles.

<2.2 Width recognition flag Det and operation status Sta>
Referring to FIG. 14, in the second embodiment, by using the width recognition flag Det, white line recognition processing (S32 to S35, S40) and estimation processing of the traveling road width W based on the object width W of the preceding vehicle (step S41). To S46, S35) can be stably processed in parallel. The width recognition flag Det is turned on (1) when the white line is recognized or the traveling road width Wr can be recognized from the horizontal edge E of the preceding vehicle, and is turned off (0) when it cannot be recognized. Further, the operation status Sta indicates an operation state in accordance with the on / off of the width recognition flag Det. The width recognition flag Det and the operation status Sta are preferably stored in the external memory 31B and can be accessed in common from the white line recognition process and the travel path width estimation process.

  The characteristic processing in FIG. 14 is as follows. That is, first, when the white line recognition unit 60 recognizes the white line (step S34), the width recognition flag Det is turned on (step S35), and when the white line recognition unit 60 is not recognized, the width recognition flag Det is turned off. (Step S40). Then, when the width recognition flag Det is off (step S41), the travel path width estimation unit 18 performs an estimation process of the travel path width Wr (steps S42 to S46). Then, when it can be estimated, the width recognition flag Det is turned on (step S35), and when it cannot be recognized, the width recognition flag Det is turned off (step S47). Then, the alarm output unit 40 warns the recognition operation status Sta according to the on / off of the width recognition flag Det (step S51).

  Thus, when the width recognition flag Det is on (or on for a certain number of times), the warning output unit 40 warns that the lane departure warning device is in operation, and the width recognition flag is off (or When it is off for a certain number of times, an alarm indicating that it is not operating can be output. In the example shown in FIG. 14, the operation status is turned off when the width recognition flag Det is repeatedly turned off by a certain number Cp. For this reason, the undetected counter Cnt is used (steps S48, S49, S39). On the other hand, when the width recognition flag Det is turned on in the non-operating state, the operating state is set (step S38).

In the second embodiment, the controller 30 may include a dedicated image processing chip having a multi-core function. These image processing chips can set the number of CPUs and relative performance by the user. Each logical CPU can access the same external memory 31A. Therefore, in order to process two logics simultaneously, two CPU settings are made, flags indicating the respective processing statuses are set in the external memory 31B, and constantly monitored to prevent the occurrence of a deadlock, By minimizing waits and locks to satisfy the ACID characteristics, high-speed processing according to the CPU capability can be realized. In addition, by setting the performance of each CPU according to the actual processing amount of the two algorithms, it is possible to optimize to minimize the waiting state for processing.
If a data driven image processing chip is used, the CPU will be in an operating state when data is flowing, but otherwise it will be in an inoperable state. Does not consume power.

  Each process is demonstrated according to the flow of FIG. First, the speed Vm of the host vehicle is confirmed by the vehicle speed sensor 32 (step S30). For example, when the vehicle speed Vm ≧ 60 [km / h] is turned on and turned on, the vehicle speed Vm is turned off when Vm ≦ 50 [km / h]. And When turned on, parallel processing of steps S32 and S41 is performed. If it is off, the operation status Sta is set to 0 (off), and a warning is given to the effect that it has become non-operation (step S51). By controlling the operation and non-operation according to the own vehicle speed, it is possible to make the vehicle inoperable when traveling at a low speed in an urban area while warning a lane departure during a long high-speed operation. When the vehicle turns right or left, the system is turned off because the vehicle speed drops.

  In step S31, the image taken by the camera 10 is stored in the frame memory 31A. This image is used for both the white line recognition process and the travel route estimation process. In the white line recognition processing, first, edges are extracted by performing Sobel filter processing (step S32), and lanes that are white lines are extracted by Hough transform or the like (step S33). If this lane is a white line indicating the traveling road width Wr (step S34), the width recognition flag Det is set to 1 (on) (step S35). When the width recognition flag Det is on, the horizontal edge extraction process is not performed (step S41). Subsequently, a deviation is determined (step S36), and if there is a deviation, a warning is issued (step S37). If there is no deviation, the operation status Sta is read and updated to 1 (on, in operation) when it is off (non-operation, 0) (step S38). When the operation status Sta is turned on, the value of a predetermined undetected counter Cnt is initialized to 0 (step S39).

  Subsequently, when the width recognition flag Det is turned on in the current image processing, a warning is given to the effect that the operation state has changed (step S51). For example, as in the case of a decrease in vehicle speed, a display indicating that the system is in a non-operating state or a sound indicating a state change such as “Pon” may be issued once when switching between non-operating and operating in the meter. . After the departure alarm S37, the operation state alarm S51 or no alarm, the next image is waited (steps S30 and S31).

  If no white line is recognized in step S34, the width recognition flag Det is set to 0 (off) (step S40). Subsequently, the next image is waited. At the time of processing the next image, since the width recognition flag Det is off, the answer is YES in step S41, and horizontal edge extraction processing is performed (step S42). Subsequently, a relevance determination process (step S43) and a determination as to whether or not the vehicle is present (step S44). If it is a vehicle, the vehicle edge information (for example, the coordinate value of the object width W and its center position) and the original image 10a from which the original image horizontal edge e is extracted are stored in the frame memory 31A (step S45). Subsequently, the traveling road width Wr is estimated using the data of the previous frame (step S46). If the estimation of the travel path width is successful, the width recognition flag Det is turned on (step S35). Further, the departure is determined based on the travel path width W obtained from the horizontal edge E (step S36).

  If the white line is not recognized in step S34, and the preceding vehicle is not detected in step S44 in the processing of the same or the next image, that is, the white line in the driving environment is worn and there is no preceding vehicle. If the vehicle is far away even if it exists, the road is more than a certain curve, or the vehicle ahead is changing lanes or turning left or right, there is no clue to guess the road width W and the width recognition flag is turned off. (Step S47). In this case, the non-detection counter Cnt is incremented so as to be in the non-operating state when the number of images for which the width recognition flag Det is turned off is more than a certain value (step S48). If the width detection flag Det is on in the most recent image processing, the undetected counter Cnt is initialized to 0 in step S39, and if the step S48 is the first time, the value of the undetected counter is 1. . When the unrecognition of the white line and the non-detection of the vehicle are continuously repeated a predetermined number of times Cp or more, the answer is YES in step S49 and the operation status Sta is turned off (step S50). In the operation state alarm S51, when the operation state is changed to the non-operation state, this operation state is alarmed. Further, in step S51, the operation state may be set in three stages such as good, worse, and impossible according to the presence / absence of white line recognition and the count value of the undetected counter.

2.2 Effect of width recognition flag and operation state As described above, when the width recognition flag or the like is used in the second embodiment, the measurement of the road width by the white line recognition process and the estimation of the road width by the horizontal edge extraction process are performed. Stable parallel processing is possible. For this reason, according to the conversion of the driving environment,
Even if the white line is not detected, it is possible to immediately estimate the traveling road width by the horizontal edge, and it is possible to try the white line extraction process while continuing the estimation of the traveling road width of the horizontal edge. Then, if the width recognition flag is used and stored in the external memory 31A, both CPUs for parallel processing can read them independently when necessary, preventing deadlocks and suppressing unnecessary waiting time. High speed processing can be ensured.

  And the lane departure warning device becomes inoperable because the white line in the driving environment is worn and the driver can visually recognize the deterioration state of the white line, and there is no preceding vehicle running ahead, However, it may be far away, when the vehicle ahead is a lane change or right / left turn, etc., or when the white line cannot be recognized and the traveling road has a certain curve or more. These changes in the driving environment are easy for the driver to understand, and it is easy for the driver to predict how the lane departure warning device will operate or not, and there is a sense of satisfaction when using the lane departure warning device. Can be high. The reliability from the driver can also be improved by improving the operation time and clarifying the reason for non-operation.

<2.3 Road width correction process>
In the second embodiment, correction may be made when estimating the travel path width by extracting the horizontal edge. In this example, referring to FIG. 10 again, the travel path width estimation unit 18 detects the travel path width Wr most recently recognized by the white line recognition unit 60 and the object width W based on the original image horizontal edge e. And a correction process 64 for correcting the travel road width Wr. When the white line extraction by the white line recognition unit 60 becomes impossible, the correction process 64 specifies the travel road width Wr by the white line when there is a white line recognized up to the immediately preceding image. The correction process 64 calculates a predetermined lane width W_COM centered on the center position of Wr from the travel road width Wr recognized most recently. Then, the traveling road width W is obtained by adding to the object width W obtained from the horizontal edge E. That is, the correction process 64 corrects the half lane width W_COM using the travel road width Wr of the latest white line.

  In addition, if there is a road width measured by a magnetic nail up to the last time instead of a white line or a road width obtained from a navigation system or the like, the half-lane width W_COM may be corrected by this road width Wr.

-2.3 Effect of correction processing of travel road width When the travel road width by the horizontal edge is corrected with the latest travel road width in Example 2, the white line recognition is turned off and the travel road width is switched to the travel road width by the horizontal edge. Can be continued without changing discontinuously, and the lane departure determination can be stabilized.

<3.1 Steering assist device>
The steering assist device according to the third embodiment is a device for automatically operating the same traveling path as that of a preceding vehicle in a factory, a distribution site, or the like. That is, the follow-up traveling to the leading vehicle is controlled.
Referring to FIG. 15, the steering assist device of the third embodiment is similar to the first and second embodiments. The camera 10, the horizontal edge extraction unit 12, the coordinate system conversion unit 14, the association determination unit 16, and the travel path A width estimation unit 18. The association determination unit may include an identity determination process 20, a speed calculation process 22, and an association determination process 24.
In the third embodiment, in particular, a steering amount calculation unit 70 that calculates a steering amount for guiding the vehicle to the estimated traveling road width Wr, and a steering drive control unit 36 that steers the vehicle based on the steering amount. And. Referring to FIG. 9 again, the steering amount calculation unit 70 has a steering angle of 0 degree when the forward traveling road width Wr goes straight as shown in the drawing. When the continuation of the front traveling road width Wr becomes a curve, the steering angle of the steering wheel corresponding to the curvature is calculated. The steering drive control unit 36 controls the steering wheel using a motor or the like according to the steering amount. The speed may be a speed at which the distance L between the preceding vehicle and the preceding vehicle is constant, or may be a predetermined speed.

3.1 Effects of the steering assist device When the steering assist device of the third embodiment is used, it is possible to follow the leading vehicle, and it is not necessary to draw a white line on a road in a site such as a factory. Further, for example, the second embodiment may be combined with the third embodiment so that the straight line portion is not drawn with a white line but only with a specific stop position.

It is a block diagram which shows the structural example of one Embodiment of this invention. Example 1 It is a block diagram which shows the structural example of the hardware resource of this embodiment. (Examples 1 to 3) It is a flowchart which shows an example of the information processing of this embodiment. (Examples 1 to 3) It is explanatory drawing which shows the relationship between a camera coordinate system and a world coordinate system. (Examples 1 to 3) It is explanatory drawing which shows an example of the horizontal edge in a world coordinate system. (Examples 1 to 3) It is explanatory drawing which shows an example of the target object width in a world coordinate system, and a travel path width. (Examples 1 to 3) It is a flowchart which shows an example of the information processing which uses the correlation of several frames. (Examples 1 to 3) FIGS. 8A and 8B are explanatory diagrams illustrating an example of the association determination process using a window. (Examples 1 to 3) It is explanatory drawing which shows the relationship between the own vehicle, a preceding vehicle, and a time interval. (Examples 1 to 3) It is a block diagram which shows the structural example of this embodiment. (Example 2) FIGS. 11A and 11B are explanatory diagrams showing an example of white line extraction processing. (Example 2) It is explanatory drawing which shows an example of lane departure time. (Examples 1 to 3) FIGS. 13A and 13B are explanatory diagrams illustrating an example of an original image and a binarized image. (Examples 1 to 3) It is a flowchart which shows an example of the information processing of this embodiment. (Example 2) It is a block diagram which shows the structural example of this embodiment. (Example 3)

Explanation of symbols

e Horizontal edge of the original image
E Horizontal edge
W Object width
Wp Expected object width
Wr Road width
Wt Distance from road to own vehicle
W_MIN Expected minimum object width
W_MAX Expected maximum object width
Wc leading vehicle width center
L Object distance
V velocity vector
Vv preceding vehicle vertical speed
Vh preceding vehicle lateral speed
Vh_MAX Lateral speed upper limit value
Vp assumed velocity vector
Vm Vehicle speed
r Horizontal edge correlation coefficient
t time
td time difference
Det width recognition flag
Sta operation status
Cnt undetected counter
TTLC Lane departure time Φm Steering angle 10 Camera 12 Horizontal edge extraction unit 14 Coordinate system conversion unit 16 Relevance determination unit 18 Traveling road width estimation unit 20 Identity determination process 22 Speed calculation process 24 Relevance determination process 30 Controller 31 Memory 31A Frame memory 31B External memory 32 Vehicle speed sensor 34 Steering angle sensor 36 Steering angle drive control unit 38 Turn signal control unit 40 Alarm output unit 52 Camera coordinate system 54 World coordinate system 56 Coordinate conversion coefficient 58 Related 60 White line recognition unit 62 Lane departure time determination unit 64 Correction processing 100 driving environment estimation device 102 lane departure warning device 104 steering assist device

Claims (8)

A camera that captures the driving environment according to the driver's field of view;
A horizontal edge extraction unit for extracting the original image horizontal edge e of the captured image in the camera coordinate system;
A coordinate system conversion unit that converts an original image horizontal edge e in the camera coordinate system into a horizontal edge E in the world coordinate system based on a predetermined coordinate conversion coefficient;
While calculating the object width W and the object distance L from the horizontal edge E, based on the predetermined assumed object width Wp, the relationship between the object width W and the traveling road width Wr of the traveling environment An association determination unit for determining;
A traveling road width estimation unit that estimates the traveling road width Wr based on the relation and the object width W;
A travel environment estimation device comprising: The camera continuously shoots the change in the driving environment at a predetermined time difference td,
The association determination unit
An identity determination process 20 for determining the identity of the horizontal edge E between consecutive images at the time difference td;
A speed calculation process for calculating a speed vector V in the world coordinate system of the horizontal edge E from the identity of the horizontal edge E;
A relation determination process for determining the relation between the object width W and the travel path width Wr from a predetermined assumed speed vector Vp and the calculated speed vector V;
The travel environment estimation device according to claim 1, further comprising: A shooting process for shooting the driving environment according to the driver's field of view;
A horizontal edge extraction step of extracting the original image horizontal edge e of the captured image in the camera coordinate system;
A coordinate system conversion step of converting the original image horizontal edge e in the camera coordinate system into a horizontal edge E in the world coordinate system based on a predetermined coordinate conversion coefficient;
While calculating the object width W and the object distance L from the horizontal edge E, based on the predetermined assumed object width Wp, the relationship between the object width W and the traveling road width Wr of the traveling environment A related determination step for determining;
A traveling road width estimation step of estimating the traveling road width Wr based on the relation and the object width W;
The association determination step includes
Identity determination processing step for determining the identity of the horizontal edge E between images consecutive at the time difference td,
A speed calculation processing step of calculating a speed vector V in the world coordinate system of the horizontal edge E from the identity of the horizontal edge E;
A relation determination processing step of determining the relation between the object width W and the travel path width Wr from a predetermined assumed speed vector Vp and the calculated speed vector V;
A driving environment estimation method comprising:   A running environment estimation program for executing the method according to claim 3 using a CPU. A camera that captures the driving environment according to the driver's field of view;
A horizontal edge extraction unit for extracting the original image horizontal edge e of the captured image in the camera coordinate system;
A coordinate system conversion unit that converts an original image horizontal edge e in the camera coordinate system into a horizontal edge E in the world coordinate system based on a predetermined coordinate conversion coefficient;
While calculating the object width W and the object distance L from the horizontal edge E, based on the predetermined assumed object width Wp, the relationship between the object width W and the traveling road width Wr of the traveling environment An association determination unit for determining;
A traveling road width estimation unit that estimates the traveling road width Wr based on the relation and the object width W;
A white line recognition unit for recognizing a lane of the driving environment from a photographed image and calculating a traveling road width;
A lane departure time TLC is determined based on the travel road width Wr and the steering angle Φm calculated by the white line recognition unit, and when the white line is not recognized by the white line recognition unit, it is estimated by the travel road width estimation unit. A lane departure time determination unit for determining a lane departure time TLC from the travel road width Wr,
An alarm output unit that outputs an alarm according to the lane departure time TLC;
A lane departure warning device characterized by comprising: The white line recognition unit turns on the width recognition flag Det when recognizing the white line, and turns off the width recognition flag Det when not recognized.
When the width recognition flag Det is off, the travel path width estimation unit performs the estimation process of the travel path width Wr. When the estimation is successful, the width recognition flag Det is turned on. Turn off the flag Det,
6. The lane departure warning device according to claim 5, wherein the warning output unit warns the recognition operating state Sta in accordance with on / off of the width recognition flag Det. The travel road width estimation unit corrects the travel road width Wr based on the travel road width Wr most recently recognized by the white line recognition unit and the object width W by the original image horizontal edge e. Processing
The lane departure warning device according to claim 5 or 6, further comprising: A camera that captures the driving environment according to the driver's field of view;
A horizontal edge extraction unit for extracting the original image horizontal edge e of the captured image in the camera coordinate system;
A coordinate system conversion unit that converts an original image horizontal edge e in the camera coordinate system into a horizontal edge E in the world coordinate system based on a predetermined coordinate conversion coefficient;
While calculating the object width W and the object distance L from the horizontal edge E, based on the predetermined assumed object width Wp, the relationship between the object width W and the traveling road width Wr of the traveling environment An association determination unit for determining;
A traveling road width estimation unit that estimates the traveling road width Wr based on the relation and the object width W;
A steering amount calculation unit for calculating a steering amount for guiding the vehicle to the estimated traveling road width Wr;
A steering drive control unit that steers the vehicle based on the steering amount;
A steering assist device characterized by comprising