JP3039327B2 - Vehicle running state detection device and drowsy driving alarm device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicle running state detecting device for detecting a running state of a running vehicle, and detects a driver's arousal level from the running state of the vehicle to reduce the arousal level. The present invention relates to a drowsy driving alarm device that sometimes issues an alarm to call attention.

[0002]

2. Description of the Related Art In recent years, the number of opportunities to drive a car has been increasing due to the development of a road network and an increase in leisure time. When driving a car, a driver is always desired to have stable mental and physical health. There is a tendency to grasp the steering wheel while ignoring it while being aware of the poor physical condition. When driving a car continuously and without a break for a long time, the driver is likely to accumulate fatigue and reduce concentration, thereby increasing the possibility of causing an accident.

[0003] On the other hand, there are various operating devices that can be operated by a driver, such as a steering wheel, an accelerator pedal, a brake pedal, a shift lever, a turn signal lever, a wiper switch, and a lighting switch. And
When the operation of these operating devices is reduced, the driving becomes monotonous, and the possibility that the driver starts to fall asleep increases.

Therefore, conventionally, for example, Japanese Patent Application Laid-Open No. 58-10 / 1983
As disclosed in Japanese Patent No. 5844, there is a device that counts the elapsed time during driving and issues an alarm to prevent falling asleep when a certain time has elapsed without operation of various operation means. Further, as disclosed in Japanese Patent Application Laid-Open No. 58-175094, when a predetermined time has elapsed without the number of operations of various operating devices being less than a predetermined value, numerical data "1" is added and the added value is added. When a certain value is reached, an alarm is issued to prevent falling asleep.

Further, it is known that there is a relationship between the amount of meandering of the vehicle and the degree of awakening (here, it indicates the degree of drowsiness requiring awakening). Regarding the meandering amount of this vehicle,
For example, JP-A-3-273498 discloses a related technique. In this publication, a video camera captures an image of the road ahead, calculates a reference point in the traveling lane based on the intersection of the lane display lines of the road ahead, and calculates the intersection of the reference point and the lane display line in the current traveling direction of the vehicle. It is determined whether or not a turning instruction has been issued correctly according to the deviation (meandering direction), and an alarm is issued if the turning instruction is incorrect.

[0006]

However, among the above-mentioned prior arts, two of the prior arts which detect the monotony of a driver from the frequency of operation of various operation devices are sampled at regular intervals. Therefore, it is difficult to catch a continuous change in the state of the driver. Also, if a warning is issued when a turn instruction is not issued correctly according to the deviation between the reference point and the intersection of the current lane display line, changes in the amount of meandering do not always correspond to dozing driving properly At present, there is a limit to the determination, and improvement in accuracy is desired.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and a vehicle running state detecting device for detecting a continuous state change of a driver from the start of driving to the present, and a detected state change of the driver. It is an object of the present invention to provide a drowsiness driving alarm device that can accurately detect a driver's drowsing and issue an alarm.

[0008]

According to a first aspect of the present invention, there is provided a vehicle running state detecting device for detecting a running state of a running vehicle. Vehicle speed detecting means for detecting, steering angle detecting means for detecting a steering angle of the steering of the vehicle,
A camera for photographing a road surface in front of or behind the vehicle traveling direction, image data storage means for selectively capturing a road surface image photographed by the camera as an inspection image, and traveling based on the inspection image from the image data storage means. An image processing means for extracting an image of the lane dividing line image to obtain the coordinates of the traveling lane dividing line image; and acquiring the coordinate data of the traveling lane dividing line image from the image processing means for a predetermined number of times or for a predetermined period of time. Meandering amount calculating means for calculating the standard deviation of the deviation on the horizontal coordinate axis of the lane marking image; and setting the meandering rate by dividing the standard deviation calculated by the meandering amount calculating means by a pixel value corresponding to the lane width in the camera. Meandering ratio setting means, and an absolute value of a steering angle detected by the steering angle detecting means to a vehicle speed detected by the vehicle speed detecting means. A steering angle absolute value correction unit that corrects based on the corrected vehicle speed coefficient, and a meandering ratio calculated by the meandering ratio setting unit is corrected according to the corrected steering angle absolute value corrected by the steering angle absolute value correction unit. And a meandering ratio correcting means.

A dozing driving alarm apparatus according to the present invention is a dozing driving alarm apparatus which detects a driving state of a running vehicle and issues an alarm when the awakening degree of the driver is low. When the monotonicity setting means for obtaining a monotonicity by integrating a constant value at predetermined time intervals and various operation means except for the steering are operated, the monotonicity calculated by the monotonicity setting means is calculated. Driving operation detecting means having a monotonicity correcting means for obtaining a corrected monotonicity by subtracting a weight value set in advance corresponding to the operation,
A steering operation detecting means for obtaining a steering amount of the steering within a predetermined time; an image data storage means for selectively capturing a front or rear road surface image from a camera mounted on the vehicle as an inspection image; An image processing means for extracting an image of the traveling lane dividing line image based on the inspection image and obtaining coordinates of the traveling lane dividing line image, and taking in the coordinate data of the traveling lane dividing line image from the image processing means a predetermined number of times or for a predetermined time. The meandering amount calculating means for calculating the standard deviation of the deviation on the horizontal coordinate of the traveling zone dividing line image, and the standard deviation calculated by the meandering amount calculating means are divided by the pixel value corresponding to the lane width in the camera to meander. Meandering ratio setting means for setting the steering ratio, and a steering angle set by the meandering ratio set by the meandering ratio setting means according to the vehicle speed. Traveling position detecting means having meandering rate correcting means for correcting according to the value coefficient; corrected monotonicity determined by the driving operation detecting means; the steering amount determined by the steering operation detecting means; and the traveling position detecting means. Fuzzy inference means for estimating the degree of awakening of the driver based on the membership functions respectively corresponding to the corrected meandering ratio determined by the fuzzy inference means, warning means for promoting the improvement of the degree of awakening of the driver, and the fuzzy inference means. Notification means for driving the warning means according to the driver's arousal level.

[0010]

The image data storage means selectively takes in the road surface image photographed by the camera as an inspection image, and the image processing means extracts the image of the traveling lane dividing line based on the inspection image, obtains its coordinates, and calculates the meandering. The amount calculating means detects the standard deviation of the displacement on the horizontal coordinate axis by taking in the coordinate data of the traveling zone dividing line image a predetermined number of times or for a predetermined time, and the meandering rate setting means uses the standard deviation as a pixel equivalent to the lane width in the camera. While setting the meandering ratio by dividing by the value,
The steering angle absolute value correction means corrects the absolute value of the steering angle detected by the steering angle detection means based on a vehicle speed coefficient corresponding to the vehicle speed detected by the vehicle speed detection means,
The meandering ratio correcting means obtains a corrected meandering ratio by correcting the calculated meandering ratio according to the corrected steering angle absolute value.

The monotonicity setting means calculates the monotonicity by accumulating a constant value at predetermined time intervals while the vehicle is running, and the monotonicity correcting means calculates the monotonicity when various operating means other than the steering are operated. The corrected monotonicity is obtained by subtracting a weighting value set in advance corresponding to this operation from the monotonicity obtained,
Further, the steering operation detecting means obtains the steering amount of the steering within a certain period of time, and the image data storage means selectively takes in a road surface image from a camera mounted on the vehicle as an inspection image. The image of the travel zone division line is extracted based on the image and its coordinates are obtained, and the meandering amount calculating means takes in the coordinate data of the travel zone division line image a predetermined number of times or for a predetermined time, and displays the data on the horizontal coordinate of the travel zone division line image. The standard deviation of the deviation is detected, the meandering ratio setting unit sets the meandering ratio by dividing the standard deviation by the pixel value corresponding to the lane width in the camera, and the meandering ratio correcting unit sets the meandering ratio according to the vehicle speed. The fuzzy inference means calculates the corrected monotonic degree, the steering amount, and the member corresponding to the corrected meandering rate by performing correction in accordance with the steering angle absolute value coefficient of the calculated steering angle. Based on the membership function estimates the alertness of the driver, the notification unit drives the alarm means in accordance with the awakening level of the estimated driver prompts the improvement of awareness of the driver.

[0012]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 1 is a schematic configuration of a drowsy driving alarm device to which a vehicle traveling state detecting device according to one embodiment of the present invention is applied, FIG. 2 is a schematic diagram of an inspection image processed by a traveling position detecting unit,
FIG. 3 illustrates a white line candidate point processed by the traveling position detection unit, FIG. 4 illustrates a characteristic diagram of the white line candidate point processed by the traveling position detection unit after differential filtering, and FIG. 5 illustrates a road for a camera attached to a vehicle. Description showing the effect of the gradient, FIG. 6 is a graph showing the effect of the vehicle speed on the steering angle, FIG. 7 is a graph showing the effect of the steering angle on the meandering ratio, FIG. 8 is a graph showing a correction coefficient for the meandering ratio, FIG. FIG. 10 is a characteristic diagram showing the amount of meandering between when the driver is drowsy and when the driver is awake, FIG. 10 is a diagram illustrating the change in the lane of the vehicle, and FIG. FIG. 12 is a flowchart of a control process by the dozing driving alarm device of the present embodiment, FIG. 13 is a flowchart of a control process by the dozing driving alarm device of the present embodiment, and FIG. 14 is a control process of the dozing driving alarm device of the present embodiment. flow Chart, flow chart of control processing by drowsiness alarm system of the present embodiment in FIG. 15, FIG. 1
6 is a flowchart of a control process by the image processing device, FIG. 17 is a flowchart of a control process for executing the operation of the awakening device by the dozing driving alarm device of the present embodiment, and FIG.
FIG. 2 shows a characteristic diagram showing a monotonicity change as an example in a vehicle with a drowsy driving alarm device.

As shown in FIG. 1, a drowsy driving warning device to which the vehicle running state detecting device of this embodiment is applied is mounted on a front portion of a vehicle and photographs a road surface 1 in front (see FIG. 2). 2, an image processing device 3 connected to the camera 2, a control unit 4 for detecting a driving state of the vehicle and alerting the driver to drowsy, a display 6 and an awakening unit 7 as alarm means. ing. The drowsy driving warning control unit 4 is connected to the image processing device 3 via a communication line, and is configured to be able to exchange signals between the two.

As shown in FIG. 2, the camera 2
Screen A showing the image of road surface 1 ahead in the row direction ₀So that
The road surface image from the camera 2 is attached to an image processing device.
3 is input. This image processing device 3 is a well-known image processing device.
An image that has a function and is particularly a part of the traveling position detection unit A
It has functions of a data storage unit A1 and an image processing unit A2.
You. The image data storage unit A1 stores the route from the camera 2
The surface image data is taken as a still inspection image P1 every predetermined time.
The inspection image P1 is sequentially captured by
The latest inspection image data in the specified image storage device
Can be filed in the area.

On the other hand, the image processing unit A2 performs the image processing of the inspection image P1.
Based on the image data, a white line image
And sequentially execute image processing steps to obtain the coordinates
Is what you do. That is, as shown in FIG.
Download the latest inspection image P1 from the inspection image data storage area.
Data in the Y-axis direction (vertical direction)
), For example, n₁, N_Two, N_Three...
Position is set) and X-axis direction
Each line n lined up (horizontally)₁, N_Two, N_Three···Up
Of all pixels (511 pixels in this embodiment)
The detection signal is A / D converted, and as shown in FIG.
Is converted to Then, each line n ₁, N_Two, N_Three
... The gray value of all pixels above is the change in illuminance of each inspection image
(Shift), each pixel on each line is sequentially
The differential filter processing is performed using the following equation (1).

F (i, j) = − 1 × f (i−3, j) −1 × f (i−2, j) −1 × f (i−1, j) + 0 × f (i, j) + 1 × f (i + 1, j) + 1 × f (i + 2, j) + 1 × f (i + 3, j) (1)

Each line n
₁, N_Two, N_Three... The grayscale values of all the pixels above are smoothed
Then, as shown in FIG. That
Thereafter, the image processing means A2 should detect a white line candidate point.
Here, each defined line n ₁, N_Two, N_ThreeUp·
..Slice level difference preset for all pixels
Then, a white line candidate point pa is searched for in comparison with the file s1.

Each of the lines n ₁ , n ₂ , n ₃ ...
The left and right white lines L based on the data of each of the above white line candidate points pa
_R, performs operations to estimate L _L. Here, the data of each white line candidate point pa on each line n ₁ , n ₂ , n ₃ ... Is processed by the least square method, and the white lines L _{R ′} , L
_L of X, the position on the Y-axis coordinate _{(Y R n = aX R n} + b),
_{(Y L n = cX L n} + d) and outputs an in drowsiness alarm control units 4.

As shown in FIG. 1, the drowsy driving warning control unit 4 includes a vehicle speed sensor 10 for detecting the operation of various operating devices of the vehicle.
Brake operation detection means 11, exhaust detection means 12, shift operation detection means 13, turn signal operation detection means 14,
The wiper operation detecting means 15, the lighting operation detecting means 16, and the steering angle sensor 17 are connected.

The vehicle speed sensor 10 detects the vehicle speed V of the vehicle, the brake operation detecting means 11 detects the operation of a brake pedal, and the exhaust detecting means 12 detects the operation of an exhaust brake switch. The shift operation detecting means 13 detects the operation of the shift lever of the transmission, the turn signal operation detecting means 14 detects the operation of the turn signal lever, and the wiper operation detecting means 15
Detects the operation of the wiper switch. Further, the lighting operation detecting means 16 detects the operation of the lighting switch, and the steering angle sensor 17 detects the steering angle Q of the steering.

The control unit 8 of the drowsy driving warning control unit 4 further includes a timer 18 and counters 19 and 2.
0, the memory 21, the display 6, and the awakening unit 7 are connected. The wake-up unit 7 has four wake-up devices 7a, 7b, 7c, 7d having different strengths of the wake-up action on the driver.
The wake-up device 7a emits an odor or a weak wind during operation, and has the weakest wake-up effect. The awakening device 7b emits light or strong wind during operation, and has a stronger awakening effect than the awakening device 7a. The wake-up device 7c emits sound or vibration during operation, and has a stronger wake-up effect than the wake-up device 7b. Further, the wake-up device 7d emits a plurality of odors, light, wind, sound, and vibration during operation, or emits an electric shock, and has the strongest wake-up effect.

The timer 18 is used for counting various times during operation and for counting a control cycle time. The memory 21 stores a weight value ni for the operation of each operation device.
(I = 1, 2, 3,...), And the weighting value ni is stored in advance in association with each operating device. The memory 21 also stores various set value data. The weight value ni is determined in consideration of the fact that the monotonous feeling eliminated by the execution of the operation differs depending on the type of operation. For example, in the turn signal lever operation for changing lanes, “1” is set as the weight value ni. In the brake operation for stopping or decelerating, the weighting value ni is set to "2.
In other words, one operation of the turn signal lever is detected as it is as the number of operations "1", but one operation of the brake is detected as the number of operations "2.5". Similarly, for other operations, weighting values ni are set in consideration of the amount of monotonicity cancellation.

The drowsy driving warning control unit 4 is mainly composed of a microcomputer, has a function of a meandering amount calculating unit A3 as a traveling position detecting unit A, and has a driving operation detecting unit B and a steering operation detecting unit. C, a fuzzy inference unit D for estimating the degree of awakening of the driver, a reference value setting unit E, and a notification unit F.

The meandering ratio calculating unit A3, which is a part of the traveling position detecting unit A, detects the standard deviation SD of the deviation on the horizontal coordinate axis by taking the coordinate data of the white line image from the image processing unit A2 a predetermined number of times. Meandering amount calculating means A3a;
Meandering ratio setting means A which divides this standard deviation SD by a pixel value corresponding to the lane width in camera 2 to set meandering ratio MR '.
3b, a steering angle absolute value correcting means A3c for correcting the absolute value Sa 'of the steering angle detected by the steering angle sensor 17 according to the vehicle speed V detected by the vehicle speed sensor 10, and correcting the meandering ratio MR'. Meandering ratio corrector A3 for obtaining a corrected meandering ratio MR by correcting the meandering ratio MR according to the angle absolute value Sa
d.

That is, the white lines L _R and L _{L are} output from the image processing device 3.
Position on the X, Y-coordinate _{(Y R n = aX R n} + b), (Y
_{L n = cX L n + d} ) data a predetermined number of times uptake, X-coordinate of the respective white line images _{(X Rn1, X Ln1),} (X Rn2,
_{X Ln2), (X Rn3,} X Ln3) sequentially calculates the deviation of the X coordinates of the respective white line image, i.e., we calculate the standard deviation SD. Then, the meandering ratio MR 'is calculated using the following equation (2). At this time, the standard deviation is represented by SD, and the average value of the pixel values of the camera 2 corresponding to the lane width is represented by W. in this case,
As shown in FIG. 5, when the road surface 1 has a gradient such as an upward slope 1a or a downward slope 1b with respect to the camera 2 attached to the front part of the automobile, an error occurs in the standard deviation SD. Therefore, the mean deviation ratio MR ′ obtained by dividing SD by the pixel value of the camera 2 is used as the standard deviation.

MR ′ = (SD / W) × 100 (2)

On the other hand, a corrected steering angle absolute value Sa obtained by correcting the steering angle absolute value (average value) Sa ′ detected by the steering angle sensor 17 in accordance with the vehicle speed V detected by the vehicle speed sensor 10 is shown in the following (3). Calculate using the formula. Sa = Sa ′ + K _S × (V _S −V) (3) Here, the vehicle speed coefficient K _S is set in advance based on a running test, and as shown in FIG. Is calculated from the ratio of the steering angle to the steering angle. In this embodiment, 0.43 is used. Further, the vehicle speed reference value V _S is the steering angle absolute value Sa '
In the present embodiment, 90 is used. The four lines shown in FIG. 6 represent data with different types of vehicles and different traveling conditions on the traveling road.

The corrected meandering ratio MR 'is corrected according to the corrected steering angle absolute value Sa to calculate the corrected meandering ratio MR using the following equation (4). MR = MR′−K _V × (Sa−Sa _S ) (4) Note that the steering angle coefficient K _V is set in advance based on a running test, and as shown in FIG. It is calculated from the ratio of the meandering ratio to the steering angle absolute value, and in this embodiment,
Use 0.1. Sa _S is the minimum value of the curved road, and 5 is used in this embodiment.

As described above, the meandering ratio calculation unit A3 eliminates the influence of the vehicle speed on the traceability with respect to the road shape. That is, the same even in the road shape, the steering angle with an increase in the vehicle speed is made of large, in terms meandering ratio MR 'obtained in the correction meandering ratio MR in the vehicle speed reference value V _S. In each of the above-described embodiments, each equation (2),
The corrected steering ratio MR is obtained by (3) and (4), but may be obtained by using a correction coefficient K _MR as shown in FIG.

In this case, as shown in FIG. 9, it is clear that the amount of the meandering ratio MR during sleepiness is much larger than the meandering ratio MR during awakening.

The reference value setting section E operates for an initial operation for a certain period of time from the start of the operation.
(4) It has a classification function. (1) A fixed time _ts1 from the start of operation is set as an initial operation. (2) When at least one of the various operating devices except the steering is operated, the weight value ni corresponding to the operating device is read from the memory 21. (3) The read weighting value ni is added by the counter 19 (n = Σni), and the added value n is held as the number of all operations during the initial operation No. Here, the addition of the weighting value ni is considered to be a case in which a plurality of operating devices are operated at the same time, although the actual occurrence rate seems to be low. (4) At the end of the initial operation, the number of all operations No, the steering amount SQ
(Standard deviation) and meandering ratio MR are stored in the memory 21 as reference values.

Further, the driving operation detecting section B functions after the elapse of the initial driving, and has the following functions (1) to (8). (1) The operation time t ₂ and the arousal level determination time t are determined by the timer 18.
Start counting ₃ respectively. (2) As the monotonicity setting means B1, a constant value To is integrated by the counter 19 every 1/10 second which is the control cycle time, and the monotonicity T is set. (3) When at least one of the various operation devices except the steering is operated, the weight value ni corresponding to the operation device is read from the memory 21. (4) The read weight value ni is added by the counter 19 (n = Σni), and the added value n is held as the total number of operations N after the initial operation. (5) By dividing the operation time t ₂ by the total number of operations N, the average value of the elapsed time between operations during operation (= t ₂ /
N). (6) By multiplying the average value over time by a constant value To, the monotonicity increase amount between each operation during driving [= (t ₂ /
N) · To] is calculated. (7) Weight value n (= addition value of n1) for monotonicity increase amount
To obtain a monotonic degree cancellation amount [= (t ₂ / N) · To · n] based on the current operation. (8) The monotonicity correction means B2 corrects the corrected monotonicity T by subtracting the integrated value of the counter 19 by the monotonicity cancellation amount.

The steering operation detector C operates after the initial driving, and has the following functions (1) and (2).

(1) The steering angle data based on the detection result of the steering angle sensor 17 is stored in the memory 21. (2) Based on the steering angle Q data of the latest unit time (currently, one minute is used as a unit), the steering amount SQ in the predetermined time is obtained from the equation (5). In this case, 1/10 of the control cycle time
Sequentially adding seeking previous value of the steering angle data every second (dt) d _{i + 1} and the difference between the current value d _i, to calculate the sum value of the unit time as the steering amount SQ.

The fuzzy inference unit D operates after the initial operation, and has the following functions. That is, the awakening degree of the driver is estimated based on the corrected monotonicity T from the driving operation detecting unit B, the steering amount SQ from the steering operation detecting unit C, and the membership function corresponding to the meandering ratio MR from the traveling position detecting unit A. I do. The fuzzy inference unit D executes a determination cycle every minute to update the arousal level, but data used in each determination cycle is taken in a certain time range (for example, 5 minutes) immediately before each determination cycle. Then, the data range is shifted sequentially by one minute and sequentially taken in.

The notification unit F operates after the initial operation, and has the following functional means. That is, each of the awakening devices 7a, 7b, 7c, 7d is selectively operated according to the arousal level.

Here, the operation of the above-mentioned drowsy driving warning device of the present embodiment will be described with reference to the flowcharts of FIGS.

As shown in FIGS. 12 to 17, when the ignition switch is turned on and the engine is started,
The camera 2, the image processing device 3, and the control unit 4 for drowsy driving warning are activated, and the control unit 8 monitors the vehicle speed V detected by the vehicle speed sensor 10 (step 101). When the vehicle speed V exceeds a predetermined value V ₁ (Yes in step 101), the timer 1
At 8, the counting of the initial operation time t ₁ (second) is started (step 102), and a road surface photographing command is issued to the camera 2 and the image processing device 3 (step 103). As a result, the image processing device 3 enters a process described below. Here, the predetermined value V ₁ of the vehicle speed, the driver assumes a catchy highway monotony, for example, is set to a value of 60Km to 70 Km.

During the operation of the vehicle, various operations are performed by the driver, and the following types are available. That is, there are operation of an accelerator pedal, operation of a brake pedal, operation of an exhaust brake switch, operation of a shift lever, operation of a turn signal lever, operation of a wiper switch, operation of a lighting switch, and operation of steering. When at least one of these operations except the steering operation is performed (Yes in step 104)
s), a predetermined weight value ni for the operating device is read from the memory 21 (step 10).
5). Then, the read weight value ni is added by the counter 19 in consideration of the case where there is a simultaneous operation (step 106), and the added value n is the total operation number No during the initial operation.
(Step 107). n = Σni No = No + n

When the steering is operated, the steering angle Q is detected by the steering angle sensor 17. If this steering operation is performed (Yes in step 108), the steering amount S is calculated based on the detected steering angle Q from the above-described equation (5).
Q is obtained and sequentially stored in the memory 21 (step 109).

In step 110, the positions of the white lines L _R and L _L from the image processing device 3 on the X and Y coordinates (Y _R n = aX
_{R n + b), (Y} L n = cX L n + d) is sequentially taken in ₁ initial operation time t, it is sequentially stored in the memory 21.
Then, the initial operation time t ₁ is equal to a certain time t _s1 (for example, 2
(0 minutes) (step 111), the end of the initial operation.

On the other hand, the image processing device 3 enters white line estimation processing as shown in the flowchart of FIG. Here, input of a road surface photographing command is waited for (step 300), and when the command is inputted, road surface image data from the camera 2 is photographed and taken in as a stationary inspection image P1 (see FIG. 4) every predetermined time. The inspection image P1 data is filed in the latest inspection image data storage area in the predetermined image storage device 3.

In step 302, the data of the latest inspection image P1 is fetched from the storage area of the latest inspection image data, and the gray scale values of all the pixels respectively located at n ₁ , n ₂ , n _3. Find data (Step 3
03). In steps 304 and 305, the grayscale values of all the pixels are subjected to the differential filter processing according to the above-mentioned equation (1), and the grayscale value difference of all the pixels is compared with a preset slice level s1 to search for a white line candidate point pa. And step 30
6, each line n ₁ , n ₂ , n ₃ along the least squares method
... The data of each of the above white line candidate points pa is arithmetically processed, and the positions of the white lines L _R , _{LL on} the X, Y coordinates (Y _R n = aX _R n)
_{+ B), (Y L n} = cX L n + d) determining (Step 3
07), and outputs the same value to the control unit 8 (step 30).
8).

On the other hand, step 111 in the main routine
Then, it is determined that the initial operation has been completed, and the routine proceeds to step 112.
Run. Here, a predetermined number of white lines L_R, L_LPosition of
(Y _Rn = aX_Rn + b), (Y_Ln = cX_Ln + d)
From the data, the X coordinate (X_Rn1, X_Ln1), (X
_Rn2, X_Ln2), (X_Rn3, X_Ln3) Are sequentially calculated, and
Of each white line image at the X coordinate, that is, the operation during the initial operation.
Calculate the standard deviation SD unique to the transferee. And as mentioned above
The meandering ratio MR 'is calculated using the equation (2), and the steering is performed.
Steering steering angle detected by the angle sensor 17
Detected by vehicle speed sensor 10 from paired value (average value) Sa '
Corrected steering angle absolute value Sa corrected according to the corrected vehicle speed V
It is calculated from the above equation (3). And the meandering rate MR '
Is corrected in accordance with the corrected steering angle absolute value Sa,
The meandering ratio MR is calculated by using the above equation (4)._SIs calculated.

In step 113, the respective steering amounts SQ stored in the memory 21 are read, and an added value SQm of the steering amounts during the initial operation is calculated. And counter 1
9 No. of operation times, steering amount (standard value) S at initial operation
Qm, held meandering ratio MR _S in the initial operation in the memory 21 as a reference value, respectively (step 114). In this way, during the initial driving over the fixed time _ts1 , the driving operation data at the individual level for each driver is captured.

After the start of the initial operation, at step 115,
Operation time t by timer 18_Two, Alertness determination time t_ThreeMosquito
The und starts. Then, the vehicle speed V exceeds the fixed value V1.
(Yes in step 116), proceed to step 117
Judge whether the blinker is operating or not, and
Suspends road surface image capture command until turn signal operation is completed
I do. That is, in this step 118, the turn signal operation
During the crop period, for example, as shown in FIG.
The running vehicle changes lanes to the right lane and again
Is changed, the white line image is changed as shown in FIG.
(Y_Rn = aX _Rn + b), (Y_Ln = cX_Ln + d)
Of the center position changes, and in this case, the center position is a line
Line b (right lane image of the right lane)
Degree, the displacement of returning to the a-line occurs.
Do not import road surface image data during operation
are doing.

In step 117, the win
If it is determined that no operation is being performed, the process proceeds to step 119.
Then, the position of the white line image (Y_Rn =
aX _Rn + b), (Y_Ln = cX_Ln + d)
In step 120, a predetermined time (for example, one minute)
X coordinate (X_Rn1, X_Ln1), (X_Rn2,
X_Ln2), (X_Rn3, X_Ln3) Is calculated sequentially, and each white
The standard deviation SD, which is the deviation of the X coordinate of the line, is calculated. Soshi
When the meandering ratio MR 'is calculated from the standard deviation SD,
The steering angle absolute value (average value) Sa '
A corrected steering angle absolute value Sa corrected according to the speed V is calculated,
The corrected meandering rate is calculated from the meandering rate MR 'and the corrected steering angle absolute value Sa.
Calculate MR. The meandering ratio MR is stored in the memory 21.
To accumulate. Even if there is a turn signal operation,
At the time of (for example, 10 seconds)
It is only excluded and the remaining data
The difference SD is calculated, and the corrected meandering ratio MR is calculated.
You. In step 121, the steering angle sensor 17
Based on the detected steering angle Q data, the steering is performed according to the above equation (5).
The steering amount SQ is obtained, and the steering amount data is stored in the memory 21.
Is done.

Further, at step 122, the counter 19
, The constant value To is integrated, and the monotonicity T is obtained. T = T + To The integration of the constant value To is repeatedly executed every 1/10 second, which is the control cycle time. When at least one operation other than the steering operation is performed during driving of the vehicle (Yes in step 123), the weighting value ni predetermined for the operating device is stored in the memory 2.
1 and the weight value ni is added (step 124). Then, the addition value n is integrated by the counter 19 as the total number of operations N during operation (step 12).
5). n = Σni N = N + n The operation time t ₂ is divided by the total number of operations N, and an average value [= t ₂ / N] of the lapse of time between operations after the initial operation has elapsed is obtained. Further, the average value of the lapse of time is multiplied by the monotonicity increase amount per second (= 10 · To), and the monotonicity cancellation amount based on the current operation [= (t ₂ / N) · 10 · T
on · n] is required. The integrated value of the counter 19 is reduced by this monotonic degree cancellation amount, and the monotonic degree T is corrected (step 126). T = T − [(t ₂ / N) · 10 · To · n] Therefore, as can be seen from the experimental data of FIG. 18, the monotonicity T continues to increase without any operation, and increases each time the operation is performed. It is canceled by the amount corresponding to the operation.

When the monotonicity T becomes a negative value (Yes in step 127), the monotonicity T is corrected to zero (step 128). Thus until awareness determination time t ₃ reaches a predetermined time (step 129), the coefficient of monotonousness T,
The accumulation of the steering amount SQ and the meandering ratio MR are repeated.

If the vehicle speed V falls below the fixed value V1 in step 116, the process proceeds to step 13
Then, the steering amount SQ stored in the memory 21 is cleared. At the same time, the awareness determination time t ₃ is returned to zero is cleared (step 131). When the vehicle speed V decreases to zero (Yes in step 132), that is, when the vehicle stops, the display of the arousal level described later is turned off (step 133). However, in this case, the monotonicity T
And the X coordinate of the previous white line is held.

On the other hand, in step 129, if the arousal level determination time t ₃ exceeds the predetermined time (Y in step 129).
es), each white line image (Y _R n) stored at the predetermined time
_{= AX R n + b),} (Y L n = cX L n + d X coordinate deviation average value MR1 meandering ratio MR is in) is determined (step 134). Similarly, based on the steering angle Qn data stored at the predetermined time, the steering amount at the latest predetermined time is added, and SQn (= ΣQn) is stored in the memory 21 (step 135). When the process reaches step 136, the meandering ratio MR, the monotonicity T, and the steering amount SQm (SQ
The three membership functions for n) are stored in memory 21
Is stored in

These membership functions are set in advance, and basically, each of the reference values (average meander rate MRs, monotonicity T, steering amount SQm, total number of operations No) obtained in the initial operation and initial operation Are determined in accordance with the total number of operations N obtained after the lapse of. Note that membership functions and fuzzy control are general,
The description here is omitted.

Accordingly, when the required amount of the arousal degree X is obtained by the fuzzy control, it is displayed on the display 6 (step 137). By looking at this display, the driver can grasp his own driving state. Also, the passenger may talk to the driver to prevent him from falling asleep,
Gives attention and takes appropriate action. Thereafter, as long as the operation is not stopped, the awakening degree determination time t ₃ is forcibly set to 60 seconds (step 139), and step 1 is executed.
Returning to 16, the operation is stopped and the system is stopped.

On the other hand, as shown in the flowchart of FIG. 17, the required amount of the awakening degree X obtained by the fuzzy inference and the set value “4” stored in the memory 21 in advance.
"3", "2" and "1" are sequentially compared (step 2
01-204). Here, the set value “4” is a level almost corresponding to a dozing state. The setting value “3” is a level at which the driver can fall asleep and sometimes meander. “2” is a level that causes considerable sleepiness, such as frequent yawning.
The set value “1” is a level that causes a slight drowsiness. If the required amount of the arousal degree X is larger than the set value “4” (Yes in step 210), the awakening device 7d operates for a predetermined time (step 205), and light, wind, sound, and vibration are emitted together, Or an electric shock is applied to the driver's body. If the required amount of the arousal degree X is larger than the set value “3” (Yes in step 202), the awakening device 7c operates for a predetermined time (step 206), and a sound or vibration is generated by the driver's body. Is added to

If the required amount of the awakening degree X is larger than the set value "2" (Yes in step 203), the awakening device 7b operates for a predetermined time (step 207), and light is emitted or strong wind is generated by the driver. It is sprayed on the body. If the required amount of the alertness X is larger than the set value "1" (step 204)
Yes), the awakening device 7a operates for a predetermined time (step 208), and an odor is emitted or a weak wind is blown to the driver's body. When the required amount of the arousal degree X is smaller than the set value “1” (No in step 204), the awakening device does not operate.

As described above, by giving the driver a wake-up action of a level corresponding to the required amount of wakefulness X, the driver can be prevented from falling asleep beforehand and safety is improved. In particular, the continuous state change of the driver is captured as monotonicity,
In addition, the features of steering operation that leads to drowsy driving are added, the meandering rate according to the amount of deviation of the white line position on the front road surface captured by the camera is captured, and these are used as fuzzy inference inputs, and there is no worry about drowsy driving yet. At the time of initial driving, individual-level driving operation data is captured for each driver and incorporated as the reference value for fuzzy inference, so that the driver's drowsiness can be accurately grasped at all times, and the reliability is excellent. Becomes Further, the awakening means can be activated stepwise according to the degree of awakening, and optimal awakening processing can be performed.

The wake-up devices 7a, 7b, 7c,
7d is not limited to that of the above-described embodiment, and various types can be used, such as applying vibration to a sheet. Others
The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the scope of the invention.

[0059]

As described above in detail with reference to the embodiments, according to the vehicle traveling state detecting device of the present invention, the image data storage means selectively selects the road surface image photographed by the camera as the inspection image. The image processing means extracts the image of the travel zone division line based on the inspection image and obtains its coordinates, and the meandering amount calculation means takes in the coordinate data of the travel zone division image a predetermined number of times or for a predetermined time. While detecting the standard deviation of the deviation on the horizontal coordinate axis of the traveling zone dividing line image, the meandering ratio setting means divides the calculated standard deviation by a pixel value corresponding to the lane width in the camera to set the meandering ratio, while setting the steering angle. The absolute value correcting means corrects the absolute value of the steering angle detected by the steering angle detecting means based on a vehicle speed coefficient corresponding to the vehicle speed detected by the vehicle speed detecting means, and calculates a meandering ratio. Since the corrective means corrects the calculated meandering rate according to the corrected corrected steering angle absolute value, the meandering rate corresponding to the vehicle speed, the road gradient, and the curvature is accurately calculated, and from the start of driving. It is possible to accurately detect a continuous state change of the driver up to the present.

Further, according to the drowsy driving warning device of the present invention, the driving operation detecting means integrates a constant value at predetermined time intervals during the running of the vehicle to obtain monotonicity, and the operation of various operating means except for the steering can be performed. When performed, a preset weighting value is subtracted from the monotonicity to obtain a corrected monotonicity, the steering operation detecting means obtains a steering amount within a fixed time, and the traveling position detecting means Based on the inspection image which selectively captures the road surface image from the camera mounted on the vehicle, an image of the traveling zone dividing line is extracted and its coordinates are obtained, and the coordinate data of the traveling zone dividing line image is determined a predetermined number of times or a predetermined time. By taking in the time, the standard deviation of the deviation on the horizontal coordinate of the traveling zone dividing line image is calculated, and the standard deviation is divided by a pixel value corresponding to the lane width in the camera to set a meandering rate. The corrected meandering rate is obtained by correcting according to the steering angle absolute value coefficient of the steering set according to the vehicle speed, and the fuzzy inference means uses the corrected monotonicity, the steering amount, and the membership function corresponding to the corrected meandering rate, respectively. The awakening degree of the driver is estimated, and the notifying means drives the warning means in accordance with the estimated degree of awakening of the driver to promote the improvement of the awakening degree of the driver. The driver's drowsiness can be accurately detected from the steering amount and the meandering ratio, and the driver's state change is detected from the detected driver's state change while taking into account the driver's continuous state change from the start of driving to the present. A drowsiness can be accurately detected and an alarm can be issued, and safety during vehicle running can be improved.

[Brief description of the drawings]

FIG. 1 is a schematic configuration diagram of a drowsy driving alarm device to which a vehicle driving state detection device according to one embodiment of the present invention is applied.

FIG. 2 is a schematic diagram of an inspection image processed by a traveling position detection unit.

FIG. 3 is an explanatory diagram of white line candidate points processed by a traveling position detection unit.

FIG. 4 is a characteristic diagram of a white line candidate point processed by a traveling position detection unit after a differential filter process.

FIG. 5 is an explanatory diagram showing an influence of a road gradient on a camera attached to a vehicle.

FIG. 6 is a graph showing an influence of a vehicle speed on a steering angle.

FIG. 7 is a graph showing an influence of a steering angle on a meandering ratio.

FIG. 8 is a graph showing a correction coefficient for a meandering ratio.

FIG. 9 is a characteristic line diagram showing each meandering amount when the driver is drowsy and when the driver is awake.

FIG. 10 is an explanatory diagram of lane change of a vehicle.

FIG. 11 is a characteristic diagram illustrating a temporal change of a center position of a white line when a lane of a vehicle is changed.

FIG. 12 is a flowchart of a control process by the dozing driving alarm device of the present embodiment.

FIG. 13 is a flowchart of a control process by the dozing driving alarm device of the present embodiment.

FIG. 14 is a flowchart of a control process by the dozing driving alarm device of the present embodiment.

FIG. 15 is a flowchart of a control process by the dozing driving alarm device of the present embodiment.

FIG. 16 is a flowchart of a control process performed by the image processing apparatus.

FIG. 17 is a flowchart of a control process for executing the operation of the awakening device by the dozing driving alarm device of the present embodiment.

FIG. 18 is a characteristic diagram illustrating a monotonicity change as an example in a vehicle with a dozing driving alarm device.

[Explanation of symbols]

 2 Camera 3 Image processing device 4 Control unit for drowsy driving warning 6 Display unit 7 Warning unit A Running position detection unit A1 Image data storage unit A2 Image processing unit A3 Meander ratio calculation unit A3a Meander amount calculation unit A3b Meander ratio setting unit A3c Steering Angle absolute value correction unit A3d meandering ratio correction unit B driving operation detection unit C steering operation detection unit D fuzzy inference unit F notification unit

──────────────────────────────────────────────────の Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI B60R 21/00 626A (72) Inventor Yukie Hirata 5-33-8 Shiba, Minato-ku, Tokyo Inside Mitsubishi Motors Corporation (72 ) Inventor Shinichi Higuchi 4-21-1, Shimomaruko, Ota-ku, Tokyo Inside Mitsubishi Automotive Engineering Co., Ltd. (56) References JP-A-58-105844 (JP, A) JP-A-58-175094 (JP, A) JP-A-3-273498 (JP, A) JP-A-7-9879 (JP, A) JP-A-7-9880 (JP, A) JP-A-7-144555 (JP, A) JP-A-6-156262 (JP, A) JP-A-5-319135 (JP, A) JP-A-5-319134 (JP, A) JP-A-59-17695 (JP, A) International publication 95/18433 (WO, A1) Hirata Yukie Others, "Driver arousal and vehicle For the amount of meandering of the relationship ", Mitsubishi automobile Technical Review 1995NO. 7, 4 May 1, 1995, P25-P31 (58) investigated the field (Int.Cl. ^7, DB name) B60K 28/06 B60R 21/00 G08G 1/16

Claims (2)

(57) [Claims] 1. A vehicle running state detecting device for detecting a running state of a running vehicle, a vehicle speed detecting unit detecting a running speed of the vehicle, and a steering angle detecting unit detecting a steering angle of a steering of the vehicle. A camera for photographing a road surface in front of or behind the traveling direction of the vehicle, image data storage means for selectively capturing a road surface image photographed by the camera as an inspection image, based on the inspection image from the image data storage means. An image processing means for extracting an image of the traveling zone division line image to obtain the coordinates of the traveling zone division line image; and acquiring the traveling zone division line image coordinate data from the image processing means for a predetermined number of times or for a predetermined period of time. Meandering amount calculating means for calculating a standard deviation of a deviation on the horizontal coordinate axis of the band dividing line image; and a standard deviation calculated by the meandering amount calculating means. Meandering rate setting means for setting a meandering rate by dividing by a pixel value corresponding to a lane width in a camera; and an absolute value of a steering angle detected by the steering angle detecting means according to a vehicle speed detected by the vehicle speed detecting means. Steering angle absolute value correction means for correcting based on the calculated vehicle speed coefficient, and meandering for correcting the meandering rate calculated by the meandering rate setting means in accordance with the corrected steering angle absolute value corrected by the steering angle absolute value correcting means. A vehicle traveling state detection device comprising a rate correction means. 2. A dozing driving alarm device for detecting a driving state of a running vehicle and issuing an alarm when the driver's awakening degree is decreasing, wherein a constant value is integrated every predetermined time while the vehicle is running. When the operation of the monotonicity setting means for obtaining the monotonicity and various operation means other than the steering is performed, the monotonicity obtained by the monotonicity setting means is set in advance corresponding to the operation. Driving operation detecting means having monotonicity correcting means for obtaining a corrected monotonicity by subtracting a weight value, steering operation detecting means for obtaining a steering amount of the steering within a predetermined time, forward from a camera mounted on the vehicle Alternatively, image data storage means for selectively capturing a rear road surface image as an inspection image, and an image of a traveling lane dividing line is extracted based on the inspection image from the image data storage means, and An image processing means for obtaining the coordinates of the traveling zone lane marking image, and a standard deviation of the horizontal coordinate deviation of the traveling lane lane marking image obtained by taking in the coordinate data of the traveling lane marking image from the image processing part a predetermined number of times or for a predetermined time. The meandering amount calculating means for calculating the meandering amount, the meandering rate setting means for setting the meandering rate by dividing the standard deviation calculated by the meandering amount calculating means by the pixel value corresponding to the lane width in the camera, and the meandering rate setting means A traveling position detecting means having a meandering rate correcting means for correcting the meandering rate according to the steering angle absolute value coefficient of the steering set according to the vehicle speed, and a corrected monotonic degree obtained by the driving operation detecting means. Memberships corresponding to the steering amount obtained by the steering operation detecting means and the corrected meandering ratio obtained by the traveling position detecting means, respectively. Fuzzy inference means for estimating the degree of arousal of the driver based on the fuzzy function, warning means for encouraging the improvement of the degree of awakening of the driver, and the warning means in accordance with the degree of awakening of the driver estimated by the fuzzy inference means A drowsy driving alarm device, comprising: a notifying device for driving the vehicle.