JP2018133007A - Alarm apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve a problem such that a wrong alarm is output caused by unfavorable conditions.SOLUTION: An alarm apparatus 100 for a driver driving a vehicle includes: an alarm output unit 63A for outputting an alarm to a driver based on a feature quantity output from a feature quantity extraction device for extracting the feature quantity of the driver from image data generated by photographing an object so as to include a position of the face of the driver driving the vehicle; a feature quantity comparison unit 62 for acquiring an amount of change in a feature quantity output by a feature quantity extraction device 5 and the feature quantity output by the feature quantity extraction device 5 that has been output before the feature quantity; and an alarm suppression unit 63B for suppressing an output of an alarm by the alarm output unit 63A when the amount of change is smaller than a threshold for a prescribed period continuously.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an alarm device for a driver who is driving a vehicle.

  A technology has been developed that alerts the driver by outputting an alarm when an undesired driving state such as a side view of the driver driving the vehicle is detected by a camera. For example, Patent Document 1 describes that a driver's face direction and line of sight are monitored, and a warning is output when it is determined that the driver is looking aside.

JP-A-6-243367

  When traveling on the bridge, for example, the sun light is repeatedly blocked by the truss structure of the truss bridge, the pillar material of the Rose bridge, etc., so the brightness in the vehicle may change abruptly. There has been a problem that when an image recognition is hindered due to such a rapid change in brightness or an adverse condition such as high temperature or high humidity, the alarm device erroneously outputs an alarm.

  Therefore, the present invention has been made in view of these points, and an object thereof is to provide an alarm device capable of suppressing an alarm from being erroneously output due to an adverse condition.

  An alarm device according to a first aspect of the present invention is an alarm device for a driver who is driving a vehicle, and is generated by photographing a subject so as to include the position of the driver's face during driving. A warning output unit for outputting a warning to the driver based on the feature value output by the feature value extraction device for extracting the feature value of the driver from the image data, and output by the feature value extraction device. A feature amount comparison unit that obtains a change amount between the feature amount and the feature amount output by the feature amount extraction device before the feature amount, and the change amount continuously exceeds a threshold value for a predetermined period. And an alarm suppression unit that suppresses an alarm output from the alarm output unit when the alarm output unit is small.

  The alarm suppression unit may suppress output of an alarm by the alarm output unit when the feature amount extraction device has not output the feature amount for a certain period of time.

  The alarm suppression unit may suppress the alarm until a predetermined output stop period after the alarm output is suppressed.

  Based on the feature quantity output by the feature quantity extraction device, the system further comprises a state detection unit for detecting first and second alarm-required states for outputting an alarm to the driver. An alarm is output when the first alarm required state continues after the elapse of a first determination time shorter than the predetermined period, and the second determination time elapses after a second determination time longer than the predetermined period. The alarm suppression unit outputs an alarm when the alarm requiring state continues, and the alarm suppression unit suppresses the output of the alarm after outputting the alarm in the first alarm requiring state, and outputs the alarm in the second alarm requiring state. The output of the alarm may be suppressed before the output.

  The alarm suppression unit suppresses an alarm based on the first alarm-needed state until the first output stop period elapses after suppression of an alarm output in the first alarm-needed state, After the suppression of the alarm output in the alarm-needed state, the alarm based on the second alarm-needed state may be suppressed until a second output stop period longer than the first output stop period elapses.

  An operation control start unit that outputs a trigger for starting the operation control of the vehicle based on the feature value output by the feature value extraction device, and the change amount is continuously smaller than a threshold value for a predetermined period. In this case, an operation control stop unit that stops the output of the trigger by the operation control start unit may be further provided.

  ADVANTAGE OF THE INVENTION According to this invention, the alarm device which can suppress outputting an alarm accidentally due to bad conditions can be provided.

3 is a diagram for describing a functional configuration of an alarm device according to Embodiment 1. FIG. It is a figure which shows an example of the image data produced | generated by the camera. It is a time chart which shows an example of operation | movement when a feature-value extraction apparatus outputs a fixed value. It is a flowchart which shows an example of operation | movement of an alarm device.

<Embodiment 1>
[Outline of alarm device 100]
FIG. 1 is a diagram illustrating a functional configuration of an alarm device 100 according to the first embodiment. The alarm device 100 is a device that monitors the driving state of the driver and is mounted on the vehicle. This vehicle includes an automatic driving device such as an automatic brake device that detects the approach of a pedestrian and decelerates the vehicle. The alarm device 100 outputs an alarm for alerting the driver that the driver is in an undesirable driving state. Undesirable driving conditions include, for example, a sidewalk, a nap, a dull driving that is driving while thinking, a driving in an inappropriate posture, and a driving difficulty caused by health. An inappropriate posture is, for example, a posture that is not suitable for driving such as warping or placing an elbow on a window frame.

  The alarm device 100 includes a speaker 1, a display 2, a camera 3, a storage unit 4, a feature amount extraction device 5, and a control unit 6. The control unit 6 includes a state detection unit 61, a feature amount comparison unit 62, and an alarm unit 63. The speaker 1 is an in-vehicle speaker for outputting sound. The display 2 is an image display device for displaying display data input from the control unit 6, and is arranged on, for example, a dashboard. The camera 3 is a photographing device for photographing a subject, and is disposed on a dashboard, for example.

  FIG. 2 is a diagram illustrating an example of image data generated by the camera 3. The camera 3 is arranged at a position and orientation for photographing the face and upper body during driving when the driver is driving in a normal driving state. For this reason, the camera 3 shoots the subject so as to include the position where the driver's face exists except when the driver loses his / her posture due to health reasons or the like. The camera 3 inputs image data generated by photographing the subject to the feature amount extraction device 5.

  Returning to the description of FIG. The feature quantity extraction device 5 is an image recognition device mounted on a vehicle. The feature quantity extraction device 5 extracts the driver's feature quantity from the image data generated by the camera 3 by image recognition. For example, if the front direction of the vehicle is the front, the feature quantity extraction device 5 is a roll angle of the driver's face angle with the front-rear direction as an axis, a yaw angle with a face angle with the vertical direction as the axis, and a left-right direction The face orientation including the pitch angle of the face orientation angle with respect to is extracted as a feature amount. For example, the feature quantity extraction device 5 detects the face outline, eyes, nose, mouth, and the like from the image data, and extracts the driver's face orientation angle based on their relative positional relationship.

  In addition, the feature quantity extraction device 5 includes, as other feature quantities, a pitch angle of rotation about the left and right direction of the driver's line of sight, a yaw angle of rotation about the vertical direction of the line of sight, and the degree of eye opening. The face position is extracted in a certain degree of eye opening, in the front-rear direction, the up-down direction, and the left-right direction with respect to the front direction of the vehicle. For example, the feature quantity extraction device 5 extracts feature quantities at predetermined time intervals and outputs the feature quantities to the state detection unit 61 and the feature quantity comparison unit 62. The feature amount extraction device 5 can extract each feature amount described above by using a known image recognition technique.

  The storage unit 4 includes a storage medium such as a ROM (Read Only Memory), a RAM (Random Access Memory), and a hard disk. The storage unit 4 stores a program executed by the control unit 6. The control unit 6 is, for example, a CPU (Central Processing Unit), and functions as a state detection unit 61, a feature amount comparison unit 62, and an alarm unit 63 by executing a program stored in the storage unit 4.

  The state detection unit 61 detects a warning-required state that is a driving state that needs to alert a driver or a passenger with an alarm. Specifically, the state detection unit 61 detects an alarm state requiring alarm such as a sidewalk, a nap, a dull driving, a driving in an inappropriate posture, and a driving difficulty state.

  The state detection unit 61 detects an alarm required state based on the feature amount output by the feature amount extraction device 5. For example, the state detection unit 61 compares the yaw angle, pitch angle, and roll angle of the face direction angle with threshold values. The state detection unit 61 detects a side call-required alarm state when any of the yaw angle, pitch angle, and roll angle of the face orientation angle is greater than the threshold value for a predetermined time or longer. The threshold is, for example, the maximum value of the yaw angle, pitch angle, and roll angle of the face orientation angle assumed in a normal driving state.

  In addition, the state detection unit 61 detects a state of needing to fall asleep based on the degree of eye opening. For example, by calculating the accumulated time of the time when the eyes were closed in a predetermined doze determination time, and comparing the ratio of the accumulated time of the eyes closed with respect to the whole doze determination time with a threshold, To detect. This threshold value is the maximum value of the rate assumed as the rate of time for closing eyes in normal operation. The state detection unit 61 detects a state of needing to fall asleep when the ratio of the accumulated time of the time when the eyes are closed is larger than the threshold.

  While the vehicle is traveling on the bridge, when the sun is repeatedly blocked by the truss structure of the truss bridge, the pillar material of the Rose bridge, or when entering or exiting the tunnel, the brightness in the vehicle changes rapidly. In addition to such a rapid change in the brightness of the vehicle, adverse conditions such as the feature quantity extraction device 5 operating at a temperature or humidity outside the guaranteed operating range, and the processing load on the feature quantity extraction device 5 being excessive. As a result, the feature quantity extraction device 5 may not be able to extract the correct feature quantity from the image data. In this case, the feature quantity extraction device 5 stops outputting the feature quantity or outputs a fixed value as the feature quantity.

  The feature quantity comparison unit 62 monitors the feature quantity output from the feature quantity extraction device 5. The feature quantity output from the feature quantity extraction device 5 always fluctuates due to the fact that it is practically difficult for the driver to remain stationary, and due to the characteristics of image recognition and the like. Therefore, the feature amount comparison unit 62 acquires a change amount between the feature amount output from the feature amount extraction device 5 and the feature amount output by the feature amount extraction device 5 before the feature amount. The feature amount comparison unit 62 determines that the feature amount extraction device 5 outputs a fixed value when the acquired change amount is less than the threshold, for example, when both feature amounts match. This threshold value is the minimum value of the fluctuation of the feature amount assumed based on the characteristics of image recognition. On the other hand, the feature amount comparison unit 62 determines that the feature amount extraction apparatus 5 does not output a fixed value as the feature amount when the obtained amount of change in the feature amount is equal to or greater than the threshold value.

  The alarm unit 63 controls alarm output or suppression. The alarm unit 63 includes an alarm output unit 63A and an alarm suppression unit 63B. The warning output unit 63A outputs a warning to the driver. The alarm output unit 63A outputs an alarm by, for example, outputting sound by the speaker 1 or displaying a warning on the display 2.

  Further, the warning output unit 63A outputs a warning to the driver based on the feature amount extracted by the feature amount extraction device 5. For example, in a state in which the state detection unit 61 detects a state of needing to fall asleep based on the feature amount output by the feature amount extraction device 5, the alarm output unit 63A requires the need to fall asleep until a predetermined dozing determination time has elapsed. An alarm is output when the alarm state continues.

  The alarm suppression unit 63B suppresses the alarm by the alarm output unit 63A. For example, when the warning output unit 63A displays a warning on the display 2, the warning display is stopped. Further, the alarm suppression unit 63B stops the audio output when the alarm output unit 63A outputs the alarm by the speaker 1, but the alarm volume may be reduced. Further, the alarm suppression unit 63B determines that the feature extraction is hindered by an adverse condition when the feature extraction device 5 has not output the feature for a certain period of time, and the alarm output unit 63A issues an alarm. May be suppressed.

[Time chart for fixed value output]
With reference to FIG. 3, the operation of the alarm output unit 63A and the alarm suppression unit 63B will be described. FIG. 3 is a time chart showing an example of the operation when the feature quantity extraction device 5 outputs a fixed value. FIG. 3A shows an example of side detection by the state detection unit 61 when the feature quantity extraction device 5 outputs a fixed value. FIG. 3B shows the feature quantity extraction device 5 that outputs a fixed value. An example of dozing detection by the state detection unit 61 is shown.

  In a state where the feature quantity extraction device 5 outputs a fixed value as the yaw angle of the face orientation angle due to an adverse condition, the state detection unit 61 indicates the driver's face orientation. Misunderstood to be. When the fixed value output by the feature quantity extraction device 5 is equal to or smaller than the threshold for detecting a side, the state detection unit 61 does not detect a side. On the other hand, in a state where the fixed value output by the feature quantity extraction device 5 is larger than the threshold value for sidewalk detection, the state detection unit 61 issues a sidewalk warning on condition that the output of the fixed value continues for a predetermined time or longer. Detect state. That is, as shown in FIG. 3A, the state detection unit 61 detects a side-requiring alarm state after a predetermined time has elapsed since the start of outputting a fixed value. Thereafter, the alarm output unit 63A outputs a sidewalk alarm when the sidewalk-required alarm state continues until a predetermined sidewalk determination time elapses after the fixed value output starts.

  The alarm suppression unit 63B suppresses output of an alarm by the alarm output unit 63A when a predetermined fixed value detection period has elapsed in a state where the feature quantity extraction device 5 continues to output a fixed value. The fixed value detection period is a period for determining with a predetermined accuracy that the feature quantity extraction device 5 is outputting a fixed value as a feature quantity.

  When the alarm suppression unit 63B suppresses the alarm output by the alarm output unit 63A, the alarm suppression unit 63B suppresses the alarm by the alarm output unit 63A until a predetermined output stop period after the alarm output is suppressed. . This output stop period is, for example, 10 seconds.

  In a state in which the feature amount extraction device 5 outputs a fixed value as a feature amount for detecting falling asleep, the state detection unit 61 indicates that the fixed value output by the feature amount extraction device 5 indicates the driving state of the driver. Misunderstood as a thing. For this reason, in a state where the fixed value output by the feature quantity extraction device 5 is larger than the threshold value, as shown in FIG. 3B, the state detection unit 61 is provided on condition that the output of the fixed value continues for a predetermined time or more. Detecting the state of alarm that requires dozing.

  On the other hand, as in FIG. 3A, the alarm suppression unit 63B is configured to output an alarm output unit when a predetermined fixed value detection period has elapsed in a state in which the feature quantity extraction device 5 continues to output a fixed value. The alarm output by 63A is suppressed.

  While the side-view determination time shown in FIG. 3A is shorter than the fixed value detection period, the dozing determination time shown in FIG. 3B is longer than the fixed value detection period. For this reason, in the case of FIG. 3A, the alarm suppression unit 63B suppresses the output of the alarm after outputting the alarm in the side-required alarm state, but in the case of FIG. The unit 63B suppresses the output of the alarm before the output of the alarm in the alarm state requiring a doze. That is, in the case of FIG. 3B, the alarm output unit 63A does not output an alarm. Similarly to FIG. 3A, the alarm suppression unit 63B suppresses the alarm by the alarm output unit 63A until the output stop period after the alarm output is suppressed.

  FIG. 4 is a flowchart illustrating an example of the operation of the alarm device 100. First, the camera 3 captures a subject so as to include the position of the face of the driving driver (S1). Next, the feature amount extraction device 5 detects the contour of the face, eyes, nose, mouth, and the like from the image data, and extracts the driver's face orientation angle as a feature amount based on their relative positional relationship ( S2).

  The feature amount comparison unit 62 compares the yaw angle, pitch angle, and roll angle of the driver's face orientation angle with threshold values (S3). If any of the yaw angle, the pitch angle, and the roll angle of the face angle is larger than the threshold (NO in S3), the state detection unit 61 determines that the driver is looking aside.

  Further, the alarm suppression unit 63B determines whether or not the feature quantity extraction device 5 outputs a feature quantity (S4). When the feature quantity extraction device 5 outputs the feature quantity (YES in S4), the feature quantity comparison unit 62 outputs the feature quantity output by the feature quantity extraction apparatus 5 and the feature quantity extraction device before the feature quantity. 5 is acquired. The feature amount comparison unit 62 determines that the face orientation angle is not a fixed value when the acquired change amount is equal to or greater than the threshold (NO in S5). In this case, the warning output unit 63A alerts the driver by outputting a warning and ends the process (S6).

  In step S3, if any of the yaw angle, pitch angle, and roll angle of the face angle is equal to or less than the threshold value (YES in S3), the state detection unit 61 detects that the driver is not looking aside and performs the process. finish. If the feature quantity extraction device 5 does not output the feature quantity in the determination of step S4 (NO in S4), the alarm suppression unit 63B stops the alarm by the alarm output unit 63A and ends the process (S7). .

  In the determination in step S5, the feature amount comparison unit 62 determines that the amount of change between the feature amount output by the feature amount extraction device 5 and the feature amount output by the feature amount extraction device 5 before the feature amount is less than the threshold value. If it is, it is determined that the feature quantity extraction device 5 is outputting a fixed value as the face orientation angle (YES in S5). When the feature quantity extraction device 5 continues to output a fixed value as the face angle until the fixed value detection period elapses, the alarm suppression unit 63B stops the alarm by the alarm output unit 63A and performs processing. The process ends (S7).

  According to the present embodiment, when the change amount of the feature amount acquired by the feature amount comparison unit 62 is continuously smaller than the threshold value for a predetermined period, the alarm suppression unit 63B outputs an alarm output by the alarm output unit 63A. Suppress. For this reason, when the feature quantity extraction device 5 outputs a fixed value as a feature quantity due to an adverse condition, the alarm output unit 63A can be prevented from erroneously outputting a warning.

  Further, the alarm suppression unit 63B suppresses the output of the alarm by the alarm output unit 63A when the feature amount extraction device 5 has not output the feature amount for a certain period of time. For this reason, it is possible to prevent the alarm output unit 63A from erroneously outputting an alarm when the feature amount extraction device 5 does not output the feature amount due to an adverse condition.

  Further, the alarm suppression unit 63B suppresses the alarm until a predetermined output stop period after the alarm output is suppressed. For this reason, it is possible to suppress frequent repetition of the alarm output by the alarm output unit 63A and the alarm suppression by the alarm suppression unit 63B.

  In the present embodiment, an example has been described in which the output stop period after the sidewalk alarm stop by the alarm suppression unit 63B is the same as the output stop period after the doze alarm stop by the alarm suppression unit 63B. However, the present invention is not limited to this. For example, the configuration may be such that the output stop period after the sidewalk alarm stop by the alarm suppression unit 63B is shorter than the output stop period after the doze alarm stop. In this case, the alarm suppression unit 63B stops the alarm based on the detection of the side effect by the state detection unit 61 until the side output stop period elapses after the output of the alarm is stopped in the side alarm required alarm state, and the need for a doze After stopping the alarm output in the alarm state, the alarm based on the detection of dozing by the state detection unit 61 may be stopped until the dozing output stop period elapses.

  In the present embodiment, the configuration in which the alarm device 100 includes the camera 3 has been described. However, the present invention is not limited to this. For example, the alarm device 100 may be configured not to include the camera 3 but to monitor the driving state of the driver using a camera mounted in the vehicle.

  Further, the alarm device 100 may be configured to be interlocked with an automatic driving device mounted on a vehicle. An example of a configuration in which the alarm device 100 is interlocked with the automatic driving device will be described below. The control unit 6 of the alarm device 100 further includes an operation control start unit and an operation control stop unit. The driving control start unit outputs a trigger for starting driving control of the vehicle to the automatic driving device. For example, the operation control start unit instructs the automatic brake device mounted on the vehicle to start deceleration or stop by the brake. The driving control start unit outputs a trigger for starting driving control of the vehicle based on the feature amount extracted by the feature amount extraction device 5. For example, the driving control start unit operates in conjunction with the alarm output unit 63A in FIG. 1 to generate a trigger for starting the driving control of the vehicle at the same timing as the alarm output unit 63A outputs an alarm. Output to.

  The operation control stop unit stops the trigger output from the operation control start unit. More specifically, when the change amount of the feature value acquired by the feature value comparison unit 62 continues to be smaller than the threshold value until the fixed value detection period has elapsed, the operation control stop unit outputs a trigger by the operation control start unit. To stop. This threshold value is the minimum value of the fluctuation of the feature amount assumed based on the characteristics of image recognition. For example, the operation control stop unit stops the trigger output by the operation control start unit at the same timing as the alarm suppression unit 63B suppresses the alarm output by the alarm output unit 63A by interlocking with the alarm suppression unit 63B. To do.

  According to the present embodiment, when the amount of change of the feature amount acquired by the feature amount comparison unit 62 continues to be smaller than the threshold until the fixed value detection period elapses, the operation control stop unit Stops the trigger output by the control start unit. Therefore, when the feature amount extraction device 5 outputs a fixed value as a feature amount due to a rapid change in the brightness of the vehicle, an automatic driving device such as an automatic brake device is generated by a trigger generated by the driving control start unit. Can be prevented from operating erroneously.

  The operation control stop unit stops the trigger output from the operation control start unit in conjunction with the alarm output suppression by the alarm suppression unit 63B. For this reason, it can suppress that suppression of the alarm output by the alarm suppression part 63B and the stop by the driving control stop part of the output of the trigger for starting the driving control of a vehicle do not correspond.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

DESCRIPTION OF SYMBOLS 1 ... Speaker 2 ... Display 3 ... Camera 4 ... Memory | storage part 5 ... Feature-value extraction apparatus 6 ... Control part 61 ... State detection part 62 ... Feature-value comparison part 63 ... alarm unit 63A ... alarm output unit 63B ... alarm suppression unit 100 ... alarm device

Claims (6)

An alarm device for a driver driving a vehicle,
Based on the feature amount output by the feature amount extraction device that extracts the feature amount of the driver from image data generated by photographing the subject so as to include the position of the face of the driver who is driving, An alarm output unit for outputting an alarm to the driver;
A feature amount comparison unit that acquires a change amount between the feature amount output by the feature amount extraction device and the feature amount output by the feature amount extraction device before the feature amount;
An alarm device comprising: an alarm suppression unit that suppresses output of an alarm by the alarm output unit when the change amount is continuously smaller than a threshold value for a predetermined period.   The alarm device according to claim 1, wherein the alarm suppression unit suppresses output of an alarm by the alarm output unit when the feature amount extraction device has not output the feature amount for a predetermined time.   The alarm device according to claim 1, wherein the alarm suppression unit suppresses the alarm until a predetermined output stop period after the alarm output is suppressed. Based on the feature amount output by the feature amount extraction device, further comprising a state detection unit for detecting first and second alarm required states for outputting an alarm to the driver,
The alarm output unit outputs an alarm when the first alarm required state continues after the elapse of a first determination time shorter than the predetermined period, and has a second determination time longer than the predetermined period. When the second alarm required state continues after the lapse of time, an alarm is output,
The alarm suppression unit suppresses output of the alarm after output of an alarm in the first alarm-required state, and suppresses output of the alarm before output of an alarm in the second alarm-required state. 4. The alarm device according to any one of items 1 to 3.   The alarm suppression unit suppresses an alarm based on the first alarm-needed state until the first output stop period elapses after suppression of an alarm output in the first alarm-needed state, The alarm based on the second alarm required state is suppressed until a second output stop period longer than the first output stop period elapses after the alarm output in the alarm required condition is suppressed. The alarm device described. An operation control start unit that outputs a trigger for starting the operation control of the vehicle based on the feature amount output by the feature amount extraction device;
The operation control stop unit that further stops the output of the trigger by the operation control start unit when the change amount is continuously smaller than a threshold value for a predetermined period, according to any one of claims 1 to 5. The alarm device described.

Images