JP2017151694A - Safety confirmation diagnostic system and safety confirmation diagnostic method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable safety confirmation behavior to be accurately diagnosed by finely detecting a driving scene of a vehicle and to reduce a manufacturing cost.SOLUTION: A safety confirmation diagnostic system comprises: a camera (2) which images a driver's face; a face direction detection section (3) which detects a direction of the driver's face on the basis of image data on the driver's face; a driving scene detection section (9) which detects a driving scene of a vehicle; an evaluation zone separation section (10) which separates the driving scene into evaluation zones with different evaluation criteria; and a safety confirmation behavior evaluation section (11) which evaluates safety confirmation behavior of the driver on the basis of the direction of the driver's face and the evaluation zones of the driving scene of the vehicle.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a safety confirmation diagnosis system and a safety confirmation diagnosis method for diagnosing whether or not a driver's safety confirmation behavior is appropriately performed.

  Detecting driving scenes that require safety confirmation based on vehicle behavior, detecting driver's gaze, and driving safety according to driving scenes that require safety confirmation based on the detected driver's gaze A system for evaluating whether a confirmation action has been executed has been proposed (see, for example, Patent Documents 1 and 2).

JP 2012-247871 A JP, 2015-141432, A

  For example, let us consider a case where a driving scene for changing lanes is detected. Even if it is a driving scene of lane change, there is a difference in the appropriate safety confirmation behavior when the road shape is merged and when the road shape is branched. Did not consider. In addition, even if you are traveling straight on the main road of the expressway, if there is a merge ahead, you need to drive the vehicle in the merge path with caution, and sometimes you may not be able to meet Although it is necessary to confirm, the system having the above-described conventional configuration does not evaluate whether or not such a confirmation action is executed.

  Furthermore, the safety confirmation behavior at the time of lane change differs between the safety confirmation behavior before the lane change and during the lane change, but such a difference is not considered in the system of the conventional configuration. Further, the gaze detection device that detects the driver's gaze has a problem that it is quite expensive. In addition, when the driver is wearing glasses, there is a problem that the driver's line of sight cannot be accurately detected.

  An object of the present invention is to provide a safety confirmation diagnosis system and a safety confirmation diagnosis method capable of detecting a vehicle driving scene in detail, diagnosing safety confirmation behavior with high accuracy, and reducing manufacturing costs. There is.

  The invention of claim 1 includes a camera (2) that captures the face of the driver of the vehicle, a face direction detector (3) that detects the face direction of the driver based on image data of the driver's face, A driving scene detection unit (9) for detecting a driving scene of the vehicle, an evaluation section dividing unit (10) for dividing the driving scene into evaluation sections having different evaluation criteria, and evaluation of the driver's face direction and the driving scene of the vehicle It is a system provided with the safety confirmation action evaluation part (11) which evaluates a driver | operator's safety confirmation action based on a section.

  The invention of claim 8 includes a camera (2) that captures the face of the driver of the vehicle, a face direction detector (3) that detects the face direction of the driver based on image data of the driver's face, A safety confirmation diagnosis method comprising a driving scene detection unit (9) for detecting a driving scene of a vehicle and an evaluation section dividing unit (10) for dividing the driving scene into evaluation sections having different evaluation criteria. In this method, the driver's safety confirmation behavior is evaluated based on the face direction and the evaluation section of the driving scene of the vehicle.

The block diagram of the safety check diagnostic system which shows 1st Embodiment of this invention Flowchart of driving scene detection control Flow chart of face angle detection control Flow chart of evaluation section division control Flow chart of safety confirmation diagnosis control Diagram showing the relationship between driving scenes such as expressways, judgment conditions, etc. Diagram showing an example of driving scene A diagram showing a general road driving scene as a table The figure explaining division control of an evaluation section The figure explaining the example of the division | segmentation control of the evaluation area of another driving scene Figure showing evaluation section division information such as expressways as a list (A) is a characteristic diagram showing the relationship between the face orientation angle and the face orientation frequency, and (b) is a characteristic diagram showing the relationship between the face orientation time and the face orientation frequency. The figure which shows the measurement result of face orientation distribution A diagram showing the face orientation distribution of multiple driving scenes The block diagram of the safety check diagnostic system which shows 2nd Embodiment of this invention Flow chart of control of mode change of automatic travel device The block diagram of the safety check diagnostic system which shows 3rd Embodiment of this invention Flow chart of control for detecting line of sight Figure showing a list of evaluation section division information for general roads

  Hereinafter, a first embodiment of the present invention will be described with reference to FIGS. The safety confirmation diagnosis system 1 of this embodiment is an in-vehicle system that is mounted on a vehicle (not shown). As shown in FIG. 1, the safety confirmation diagnosis system 1 includes a camera 2, a face orientation detection unit 3, a GPS receiver 4, a map data storage unit 5, a blinker signal detection unit 6, and a vehicle speed detection unit 7. The steering state detection unit 8, the driving scene detection unit 9, the evaluation section division unit 10, the safety confirmation behavior evaluation unit 11, the evaluation criterion learning unit 12, the advice presentation unit 13, and the manager transmission unit 14 It is configured with.

  The camera 2 photographs the face of the driver of the vehicle and outputs image data of the photographed driver's face to the face orientation detection unit 3. The face orientation detection unit 3 inputs image data from the camera 2, extracts a driver's face image from the image data, detects the driver's face orientation based on the face image, and detects the face orientation. The detection result is output to the safety confirmation action evaluation unit 11.

  The GPS receiver 4 detects the current position of the vehicle and outputs a position detection result to the driving scene detection unit 9. The map data storage unit 5 includes a DVD, a hard disk, a semiconductor memory, and the like, and stores road map data including road map information and various data. The driving scene detection unit 9 can refer to the traveling road on which the vehicle is traveling, the location of the current position of the vehicle, and the like based on the road map data in the map data storage unit 5.

  The winker signal detection unit 6 detects the lighting state of the winker and outputs a detection signal to the driving scene detection unit 9. The vehicle speed detection unit 7 detects the speed of the vehicle based on a signal from a vehicle speed sensor or the like, and outputs a vehicle speed detection signal to the driving scene detection unit 9. The steering state detection unit 8 detects the steering state of the vehicle based on signals from a steering angle sensor, a gyro, and the like, and outputs a steering state detection signal to the driving scene detection unit 9.

  The driving scene detection unit 9 acquires signals from the GPS receiver 4, the map data storage unit 5, the blinker signal detection unit 6, the vehicle speed detection unit 7, and the steering state detection unit 8, for example, “lane change on a highway” ”Is detected and the detection result is output to the evaluation section dividing unit 10.

The evaluation section dividing unit 10 acquires the detection result from the driving scene detecting unit 9, divides the driving scene into evaluation sections having different evaluation criteria, and outputs the divided result to the safety confirmation action evaluating unit 11.
The safety confirmation action evaluation unit 11 acquires the detection result from the face direction detection unit 3 and the division result from the evaluation section dividing unit 10, and evaluates the safety confirmation action for each driving scene and evaluation section. It is determined whether or not safety confirmation actions have been appropriately performed. The safety confirmation action evaluation unit 11 determines the determination result (that is, either the result of determining that the driver has appropriately performed the safety confirmation action or the result of determination that the driver has not appropriately performed the safety confirmation action, or both. To the advice presenting unit 13 and the transmitting unit 14 for the manager.

  The evaluation criterion learning unit 12 acquires face orientation information for each driving scene from the safety confirmation behavior evaluating unit 11, learns an evaluation criterion for each driving scene and for each evaluation section, and determines the face orientation angle threshold value and the face for each evaluation section. A threshold value for the direction time is determined, and the determined threshold value is output to the safety confirmation action evaluation unit 11.

  The advice presenting unit 13 inputs a safety confirmation behavior determination signal from the safety confirmation behavior evaluation unit 11 and, for example, when a determination result determined that the driver did not appropriately perform the safety confirmation behavior is input to the driver Give advice to pay attention to safety confirmation actions and suggest breaks. In this case, it is preferable that the advice presenting unit 13 be configured to display a message corresponding to the advice on the display device or to output a message corresponding to the advice by voice, for example.

  When the safety confirmation behavior evaluation signal is input from the safety confirmation behavior evaluation unit 11, the transmission unit 14 for the manager receives information on the evaluation of the driver's safety confirmation behavior and the vehicle when the safety confirmation behavior is evaluated. Information on the position and information on the time at which the evaluation of the safety confirmation action occurred is acquired from the safety confirmation action evaluation unit 11, and the acquired information is transmitted to the vehicle such as a truck or bus via wireless communication (for example, mobile phone communication). To the management office that manages commercial vehicles. Such information transmission processing is performed when the vehicle is a commercial vehicle.

  Further, instead of wireless communication, each information is stored in a recording medium such as a memory card, and the stored information is transferred to the management office by removing the stored recording medium from the vehicle and carrying it to the management office. You may comprise as follows. Further, in the management office, when the vehicle driving instruction is given to the driver, each information of the transmitted safety confirmation driving evaluation can be used.

  Next, the control, that is, the operation of the driving scene detection unit 9 will be described with reference to the flowchart of FIG. In step S10 of FIG. 2, the driving scene detection unit 9 receives signals from the GPS receiver 4, the map data storage unit 5, the winker signal detection unit 6, the vehicle speed detection unit 7, and the steering state detection unit 8, that is, Get information. Then, it progresses to step S20 and the driving scene detection part 9 measures the present position of a vehicle based on the position detection result from the GPS receiver 4. FIG.

  In step S30, the driving scene detection unit 9 refers to the vehicle position information, vehicle speed information, steering state information (for example, angular velocity information), and map data, and is further used in the navigation system. Using the technology, the travel road and the travel location where the vehicle travels are specified. Then, the driving scene detection unit 9 refers to the identified driving road and the driving location and the detection information of the blinker lighting signal, so that various driving scenes as shown in the table of FIG. “Change lane”, “Main line merge”, “Leave main line”, “Straight with merging (1st lane)”, “Straight with merging (other than 1st lane)”, “Straight with branch”, “Single It is possible to specify “straight ahead”. In addition, the figure which specifically illustrated each driving scene of the above-mentioned vehicle with a highway or a vehicle exclusive road is shown in FIG.

  In this case, for example, if the driving scene is “lane change”, the starting point of the driving scene is set to 3 seconds before the blinker is turned on, and the end point of the driving scene is set to the time when the blinker is turned off. Also, if the driving scene is “straight ahead with merging”, the driving scene is divided according to the driving lane, with the driving scene starting point 300 m before the merging end point and the merging end point as the ending point of the driving scene. As for the merge end point, when the road link detected from the vehicle position and the map data passes through the node where the main link and the merge link are connected, the switching between the main link and the main link occurs in FIG. In this case, the merge end point can be obtained from the distance difference between the preset merge end point and the link switching point. The lane recognition is based on the method of recognizing the white line with the camera, the number of occurrences of right lane change and left lane change from the main line merge, and the number of lanes for each road link stored in the map data. There are methods for recognizing lanes, and these methods are preferably used as appropriate.

  FIG. 6 is a table showing driving scenes specified on an expressway or a vehicle-only road. On the other hand, the table | surface shown about the driving scene specified in a general road is shown in FIG. The driving scene on the general road shown in FIG. 8 can also be detected in substantially the same manner as the above-described control.

  Subsequently, the process proceeds to step S40 in FIG. 2, and the driving scene detection unit 9 includes information on the type of the detected driving scene and various information related to the detected driving scene (for example, the time and point of the driving scene start point). Information, information on the time and point of the driving scene end point, information on the speed and angular velocity of the vehicle, etc.) and history information of these information are stored in the internal memory. Thereby, the control of FIG. 2 is terminated.

  Next, the control, that is, the operation of the face direction detection unit 3 will be described with reference to the flowchart of FIG. In step S110 of FIG. 3, image data from the camera 2 is input. Then, it progresses to step S120 and the face direction detection part 3 extracts a driver | operator's face image from the input image data. Then, the process proceeds to step S130, and the face direction detection unit 3 performs matching processing on the extracted face image and many face-oriented template images created in advance, calculates a correlation value, and obtains the highest correlation value. The face orientation angle of the driver is detected (ie, estimated) from the face orientation angle of the template image having a high height. As a number of face-oriented template images created in advance, for example, face-oriented template images of every 15 degrees from -45 degrees to 45 degrees may be created and stored in the internal memory. preferable.

  Further, as a face orientation detection technique, there are configurations described in Japanese Patent Laid-Open Nos. 2000-97676 and 2003-44853, and these configurations may be used as appropriate.

  Next, the process proceeds to step S140 in FIG. 3, and the face orientation detection unit 3 stores information on the face orientation angle detected as described above (that is, face orientation data) in an internal memory. Thereby, the control of FIG. 3 is terminated.

  Next, the control, that is, the operation of the evaluation section dividing unit 10 will be described with reference to the flowchart of FIG. In step S210 of FIG. 4, the evaluation section dividing unit 10 receives each of the driving scene, the blinker signal, the vehicle speed, the steering state (for example, the steering angle information of the steering angle sensor or the angular velocity information of the gyro sensor) from the driving scene detection unit 9. Get information. Then, it progresses to step S220 and the evaluation area division part 10 detects the change point of driving action based on each acquired said information. And it progresses to step S230 and the evaluation area division part 10 divides | segments a driving scene into a some evaluation area based on the detected change point, and produces | generates evaluation area division information.

  Here, when the driving scene is, for example, “lane change”, a process of dividing into a plurality of evaluation sections will be described with reference to FIG. 9. When performing “lane change”, it is preferable to check the surrounding safety situation to determine whether or not the lane change is possible before the turn signal is turned on. After turning on the blinker, confirm the safety of the lane change destination and start turning the steering wheel. This period is defined as section 2. Then, while confirming the status of the lane to which the lane is changed from the lane before the lane is changed, the vehicle enters the changed lane and turns the steering wheel. This period is defined as section 3. Subsequently, the steering wheel angle is returned to the original, and the blinker is turned off. This period is referred to as section 4.

  In this configuration, the division reference points to be divided from the section 1 to the section 4 are the blinker lighting time point p1, the steering wheel turning start time point p2, and the steering wheel turning back time point p3. These time points p1, p2, and p3 are changing points of driving behavior, and can be obtained based on the blinker signal and angular velocity information.

  Specifically, the steering wheel turning start time point p2 and the steering wheel turning back time point p3 calculate an angular acceleration Δω that is a time difference between the angular velocities ω, and based on the calculated positive or negative peak of the angular acceleration Δω. Can be sought.

The steering wheel turning start time point p2 can be regarded as the first time point that satisfies the following conditions after the blinker lights up.
When changing to the right lane, when the peak value is convex below Δω <−Δω1 When changing to the left lane, when the peak value is convex above Δω> Δω1, the turning point p3 of the steering wheel is It can be regarded as the time when the following conditions are met.

When changing to the right lane, when the peak value is convex above Δω> Δω2 When changing to the left lane, when the peak value is convex below Δω <-Δω2, Δω1 and Δω2 are set according to conditions such as the vehicle and vehicle speed. Is the set value. The peak is determined by calculating an angular jerk ΔΔω that is a time difference value of the angular acceleration Δω. The point at which the angle jerk ΔΔω changes from positive to negative is an upwardly convex peak, and the point at which the angle jerk ΔΔω changes from negative to positive is a downwardly convex peak.

  Note that the start time point p0 of the section 1 is a time point, for example, 3 seconds before the blinker lighting time point p1. Also, the end point of section 4 (that is, the steering wheel return point) p4 is a point in time when the blinker is turned off.

  In the graph in FIG. 9, the solid curve K1 indicates the angular velocity ω, the broken curve K2 indicates the angular acceleration Δω, and the dashed-dotted curve K3 indicates the angular jerk ΔΔω. In the present embodiment, the lane change behavior is detected from the change pattern of the angular acceleration Δω to divide the section. Also, the solid line S indicates the lane change state, “0” before the signal (that is, before the blinker lights), “1” after the signal (that is, after the blinker lights), and “2” that is turning (that is, the steering wheel) “During operation”, “3” indicates from the end of the turn to the end of the signal (ie, blinker extinction).

  In the above embodiment, the lane change behavior of the vehicle is detected from the change pattern of the angular acceleration Δω and the sections 1 to 4 are divided. However, the present invention is not limited to this. A white line that divides the lane of the road is detected by a camera (that is, a white line detection unit), and a lane change behavior (that is, a change point of driving behavior) is detected from the positional relationship between the detected position of the white line and the position of the vehicle. You may comprise so that the division | segmentation of 1-4 may be performed.

  Further, in the division of the evaluation section when the driving scene is, for example, “straight with merge”, as shown in FIG. For example, the interval is divided into three for every 100 m, and the divided sections are set as a section 1, a section 2, and a section 3 from the front.

  Each driving scene is divided into evaluation sections in this way, and the divided evaluation sections, that is, evaluation section division information is shown in a list in FIG. As shown in FIG. 11, in each evaluation section of each driving scene, the condition data of the start point and end point of the section, identification data indicating whether or not it is an evaluation target, threshold value data of the face orientation angle, Are associated with threshold value data of the face orientation time. A data table of evaluation section division information shown in FIG. 11 is created in advance and stored in a memory inside the evaluation section division section 10. When the evaluation section dividing unit 10 receives the information of the driving scene detected from the driving scene detecting unit 9, the evaluation section dividing unit 10 acquires evaluation section dividing information corresponding to the detected driving scene based on the data table of FIG. Information of the detected driving scene and the evaluation section division information are output to the safety confirmation action evaluation unit 11. Thereby, the control of FIG. 4 is terminated.

  Next, the control, that is, the operation of the safety confirmation action evaluation unit 11 will be described with reference to the flowchart of FIG. In step S <b> 310 of FIG. 5, the safety confirmation action evaluation unit 11 performs the evaluation corresponding to the detected driving scene output from the evaluation section dividing unit 10 and the information of the driving scene detected by the driving scene detection unit 9. The section division information and information on the driver's face orientation angle detected by the face orientation detector 3 are acquired.

  Then, it progresses to step S320 and the safety confirmation action evaluation part 11 determines whether the nth evaluation area (initial value of n is "1") from the near side in the detected driving scene is an evaluation object area. to decide. In the case of this embodiment, as shown in the table of FIG. 11, an evaluation target section is set. In step S320, when the evaluation section is not the evaluation target section (NO), the process proceeds to step S370, and the safety confirmation action evaluation unit 11 determines whether or not all the evaluation sections in the driving scene have been evaluated. Here, when all the evaluation sections have not been evaluated (NO), n is counted up, the process returns to step S320, and it is determined whether or not the next evaluation section is an evaluation target section.

  In step S320, when the evaluation section is an evaluation target section (YES), the process proceeds to step S330, and the safety confirmation action evaluation unit 11 refers to the face angle angle threshold of the evaluation section. Then, it progresses to step S340 and it is judged whether the driver | operator's face direction angle detected in the evaluation area exceeds the said threshold value.

  In step S340, if the detected driver's face angle exceeds the threshold (YES), the process proceeds to step S350, and the safety confirmation action evaluation unit 11 performs the safety confirmation action by the driver in the corresponding evaluation section. It is determined that it is properly performed, and the determination result is stored in an internal memory. In step S340, when the detected driver's face angle does not exceed the threshold value (NO), the process proceeds to step S360, and the safety confirmation action evaluation unit 11 performs safety confirmation by the driver in the corresponding evaluation section. It determines with action not being performed appropriately, and memorize | stores a determination result in internal memory.

  Subsequently, it progresses to step S370 and the safety confirmation action evaluation part 11 judges whether all the evaluation areas in the driving scene were evaluated. Here, when all the evaluation sections have not been evaluated (NO), n is counted up, and the process returns to step S320 to determine whether or not the next evaluation section is an evaluation target section. Repeatedly.

  In step S370, when all the evaluation sections have been evaluated (YES), the process proceeds to step S380, and the safety confirmation action evaluation unit 11 performs the safety confirmation action by the driver in all the evaluation sections in the driving scene. Determine if it is done properly. Here, when the safety confirmation action by the driver is appropriately performed in all the evaluation sections (YES), the process proceeds to step S390, and the safety confirmation action evaluation unit 11 performs the safety confirmation action by the driver for the corresponding driving scene. It is determined that it is properly performed, and the determination result is stored in an internal memory. Thereby, the control of FIG. 5 is terminated.

  In step S380, when there is even one evaluation section in which the safety confirmation action by the driver is not properly performed (NO), the process proceeds to step S400, and the safety confirmation action evaluation unit 11 performs driving for the corresponding driving scene. It is determined that the safety confirmation action by the person is not properly performed, and the determination result is stored in the internal memory. Thereby, the control of FIG. 5 is terminated.

  Next, a method for determining the face angle threshold and the face time threshold for each evaluation section of each driving scene will be described. Data on the face angle for each evaluation section of each driving scene is collected from the actual driving by an exemplary driver approved by an automobile driving instructor or a transportation company, and the figure is displayed for each evaluation section of each driving scene. A characteristic diagram showing the relationship between the face orientation angle and the face orientation frequency as shown in FIG. From FIG. 12A, the face orientation angle B1 that is the face orientation angle other than the front and has the most distribution (that is, the peak of the face orientation frequency) is set as the threshold value of the face orientation angle. FIG. 12B is a characteristic diagram showing the relationship between the face orientation time at the face orientation angle B1 with the most distribution (ie, the time during which the face orientation is held at the face orientation angle B1) and the face orientation frequency. Create From FIG. 12B, the average value T1 of the face direction time is set as a threshold value of the face direction time. These threshold values B1 and T1 are stored as data in the evaluation section division information shown in FIG. 11, and are stored in the memories inside the safety confirmation action evaluation unit 11 and the evaluation reference learning unit 12. Such processing for determining the face angle threshold and the face time threshold for each evaluation section of each driving scene (that is, processing for determining or learning evaluation criteria) is performed by an information center or a manufacturer. The process for determining the threshold value may be executed by the evaluation criterion learning unit 12. In the present embodiment, the evaluation criterion is configured to be determined or learned from the face angle of the model driver. However, the present invention is not limited to this, and the driver who drives the vehicle instead of the model driver. It may be configured to determine or learn from the face orientation angle of the driver detected in the normal driving.

  Now, for example, for the evaluation sections 1 to 4 where the driving scene is “lane change”, an example of data obtained by measuring the face angle of the model driver (that is, the measurement result of the face orientation distribution) is shown in FIG. . FIG. 13B shows an example of data obtained by measuring the driver's face orientation angle for which safety confirmation is insufficient (that is, the measurement result of the face orientation distribution).

  In FIG. 13, the face orientation distribution of each section is a distribution of detection results of face orientation angles for each section in a plurality of right lane changes. Changes in the face orientation distribution from section 1 to section 4 are known, and the difference between the model driver and the driver whose safety is insufficient is clearly understood. From FIG. 13, it can be seen that the facial orientation distribution of a general driver may be evaluated with respect to the facial orientation distribution of each section in a plurality of driving scenes with reference to the facial orientation distribution of the model driver. As an evaluation method, there are a method of comparing average values, comparing a value obtained by adding the average value and standard deviation σ, and comparing peak positions of face orientation distributions.

  Further, the results of measuring the relationship between the face angle and the face direction frequency (ie, the face direction distribution) in the case where the driving scene is “lane change”, “straight forward with merging”, and “straight forward” are shown in FIG. Shown in FIG. 14A shows a model driver's face orientation distribution, and FIG. 14B shows a driver's face orientation distribution for which safety confirmation is insufficient. 14 (a) and 14 (b), the one-dot chain line curve C1 is the case of “lane change”, the dashed curve C2 is “straight with merging”, and the solid curve C3 is “single road” This is the case of “straight ahead”. Note that the curve C1 of FIG. 14 can be obtained based on the measurement data of FIG.

  In FIG. 14, each driving scene is not divided into sections, but even when compared for each driving scene, there is a clear difference between the model driver and a driver whose safety is insufficient. For this reason, it turns out that a general driver's face direction distribution may be evaluated on the basis of the model driver's face direction distribution shown in Drawing 14 (a).

  According to this embodiment having such a configuration, a driving scene of the vehicle is detected by the driving scene detection unit 9, and the detected driving scene is divided into evaluation sections having different evaluation criteria by the evaluation section dividing unit 10, so that the safety confirmation action Since the evaluation unit 11 is configured to evaluate the safety confirmation behavior of the driver based on the direction of the driver's face and the evaluation section of the vehicle driving scene, the vehicle driving scene is divided into evaluation sections and detected in detail. In addition, the safety confirmation behavior can be diagnosed with high accuracy. Moreover, in this embodiment, since a line-of-sight detection apparatus is not used, manufacturing cost can be reduced.

  In the present embodiment, the evaluation section dividing unit 10 detects driving action change points p1, p2, and p3 based on driving scene information, blinker signal information, vehicle speed information, and steering state information. Since the driving scene is divided into the evaluation sections based on the detected change points p1, p2, and p3, it is possible to accurately divide into the evaluation sections having different evaluation criteria. In this configuration, it is preferable to use the blinker lighting time point p1, the steering wheel turning start time point p2, and the steering wheel turning back time point p3 as the change points of the driving behavior.

  Furthermore, in the present embodiment, the driving scene detection unit 9 is configured to detect the driving scene as a different driving scene when the lane in which the vehicle travels is different, so that the driving scene can be detected in detail, and the safety confirmation The diagnostic accuracy of behavior can be further improved.

  Further, in the driving scene of “lane change” shown in FIG. 9 of the present embodiment, it is possible to evaluate the safety confirmation behavior with respect to the blinker lighting time. Specifically, section 2 shown in FIG. 9 is a section from blinker lighting to the start of turning the steering wheel. Based on the length (that is, time) of this section 2, evaluation of the goodness or badness of the blinker lighting operation by the driver is evaluated. can do.

  In the evaluation section dividing process shown in FIG. 9, if the blinker is turned on too late or the blinker is turned off early, a situation may occur in which the start point of the steering wheel or the peak point of the steering wheel return is not found. There is. In such a case, it is preferable that a section that cannot be detected is stored in a memory inside the evaluation section dividing unit 10 as an evaluation invalid section or an abnormal driving action section.

(Second Embodiment)
15 and 16 show a second embodiment of the present invention. In addition, the same code | symbol is attached | subjected to the same structure as 1st Embodiment. In 2nd Embodiment, it applied to the vehicle provided with the automatic travel apparatus which can perform automatic driving | running | working, such as an auto cruise, a lane keep assist, and a vehicle line change. As a technique for automatically changing the lane, for example, a configuration described in JP-A-11-339186 is known.

  Even when automatic driving such as auto cruise or lane keeping assist is being executed, the driver is always required to confirm safety as long as the driver is responsible for safe driving. Therefore, also in the second embodiment, the safety confirmation action is evaluated for each evaluation section of each driving scene, and if it is determined that the safety confirmation action by the driver is not properly performed, the safety confirmation action is performed. To provide advice and to provide safety confirmation directions. Further, in the second embodiment, when it is repeatedly determined that the safety confirmation action by the driver is not properly performed, the operation mode of the automatic travel device is changed, for example, the vehicle speed is controlled to a set value or less. Or change the car line.

  As shown in FIG. 15, the safety confirmation diagnostic system 21 of the second embodiment includes an automatic traveling device operation detection unit 22 and an automatic traveling device mode change control unit in addition to the configuration of the safety confirmation diagnostic system 1 of the first embodiment. 23. The automatic travel device operation detection unit 22 detects the operation state of the automatic travel device that executes auto cruise, lane keep assist, vehicle line change, and the like, and transmits a detection signal to the safety confirmation action evaluation unit 11.

  Whether the automatic traveling device mode change control unit 23 receives the safety confirmation behavior determination signal and the detection signal of the operating state of the automatic traveling device from the safety confirmation behavior evaluation unit 11 and needs to change the operation mode of the automatic traveling device. If it is necessary to make a change, a change control signal is output to the automatic travel device.

  Next, the control, that is, the operation of the automatic travel device mode change control unit 23 according to the second embodiment will be described with reference to the flowchart of FIG. In step S410 of FIG. 16, the automatic travel device mode change control unit 23 acquires information such as a safety confirmation behavior determination signal and a detection signal of the operating state of the automatic travel device from the safety confirmation behavior evaluation unit 11. Then, it progresses to step S420 and the automatic travel device mode change control part 23 judges whether the safety confirmation is completed for every evaluation area of the detected driving scene. Here, when the safety confirmation is completed for all the evaluation sections, the process proceeds to “YES” in step S420, and the control of FIG.

  In step S420, when safety confirmation has not been performed for at least one evaluation section (NO), the process proceeds to step S430, and the automatic travel device mode change control unit 23 performs the evaluation section and driving scene for which safety confirmation has not been performed. The operation mode changing process of the automatic travel device is executed based on the above. In this case, the automatic travel device mode change control unit 23 controls, for example, the vehicle speed to be equal to or lower than a set value, or disables the change of the lane, for example, when it is repeatedly determined that safety cannot be confirmed. A change instruction signal for instructing to change the operation mode of the automatic traveling device is transmitted to the automatic traveling device. Thereby, the control of FIG. 16 is terminated.

  The configurations of the second embodiment other than those described above are the same as the configurations of the first embodiment. Therefore, in the second embodiment, substantially the same operational effects as in the first embodiment can be obtained. In particular, according to the second embodiment, even in a vehicle equipped with an automatic travel device, it is possible to accurately determine the safety confirmation behavior by the driver, and when the safety confirmation behavior is not executed during automatic travel, The operation mode of the automatic travel device can be changed to correspond.

(Third embodiment)
17 and 18 show a third embodiment of the present invention. In addition, the same code | symbol is attached | subjected to the same structure as 1st Embodiment. In the third embodiment, the gaze detection unit 24 that detects the gaze of the driver is provided, and the gaze detection result detected by the gaze detection unit 24 is output to the safety confirmation action evaluation unit 11.

  The line-of-sight detection unit 24 receives the driver's face image data from the face orientation detection unit 3 and estimates (ie, detects) the driver's line of sight based on the input driver's face image data. In this case, the gaze detection unit 24 extracts the left and right eye images from the driver's face image, and acquires the positional relationship between the eyeball and the iris from the extracted left and right eye images. Further, the gaze detection unit 24 is configured to detect the driver's gaze direction based on the acquired positional relationship between the eyeball and the iris and the driver's face orientation angle detected by the face orientation detection unit 3. ing.

  Then, the line-of-sight detection unit 24 outputs the estimated detection result of the driver's line-of-sight direction to the safety confirmation action evaluation unit 11. The safety confirmation action evaluation unit 11 is configured to determine whether or not the driver's safety confirmation action has been appropriately performed in consideration of the detection result of the driver's line-of-sight direction from the line-of-sight detection unit 24.

  Next, the control, that is, the operation of the line-of-sight detection unit 24 according to the third embodiment will be described with reference to the flowchart of FIG. In step S <b> 510 of FIG. 18, the line-of-sight detection unit 24 inputs driver face image data from the face orientation detection unit 3. Then, it progresses to step S520 and the gaze detection part 24 acquires the information of a driver | operator's face direction angle from the face direction detection part 3. FIG.

  Next, the process proceeds to step S530, and the line-of-sight detection unit 24 detects (that is, estimates) the driver's line-of-sight direction based on the extracted driver's face image. In step S540, the line-of-sight detection unit 24 stores (that is, stores) the estimated detection result of the driver's line-of-sight in the internal memory and outputs the result to the safety confirmation action evaluation unit 11. Thereby, the control of FIG. 18 is completed.

  The configuration of the third embodiment other than that described above is the same as the configuration of the first embodiment. Therefore, in the third embodiment, substantially the same operational effects as in the first embodiment can be obtained. In particular, according to the third embodiment, the safety confirmation action evaluation unit 11 is configured to determine the driver's safety confirmation action in consideration of the detection result of the driver's line of sight by the line of sight detection unit 24. The determination accuracy of the confirmation action can be improved.

(Fourth embodiment)
FIG. 19 shows a fourth embodiment of the present invention. In addition, the same code | symbol is attached | subjected to the same structure as 1st Embodiment. In the first embodiment, the driver's safety confirmation behavior is evaluated for a driving scene on a highway or a vehicle-only road. However, in the fourth embodiment, the driver's safety is also evaluated for a driving scene on a general road. It was configured to evaluate confirmation behavior.

  Specifically, as shown in FIG. 19, on a general road, for each driving scene of “lane change”,..., “Straight line straight”, the driving scene is divided into a plurality of evaluation sections. In the same manner as in the first embodiment, the driver's safety confirmation behavior is evaluated (ie, determined).

  The configurations of the fourth embodiment other than those described above are the same as the configurations of the first embodiment. Therefore, also in the fourth embodiment, substantially the same operational effects as in the first embodiment can be obtained.

  In each embodiment described above, in each evaluation section of each driving scene, it is configured to evaluate the driver's safety confirmation behavior based on whether the face angle or the face time exceeds each threshold. Alternatively, the safety confirmation action may be evaluated based on the shape of the face angle distribution in each evaluation section of each driving scene.

  In the drawings, 1 is a safety confirmation diagnostic system, 2 is a camera, 3 is a face direction detection unit, 6 is a blinker signal detection unit, 9 is a driving scene detection unit, 10 is an evaluation section division unit, 11 is a safety confirmation action evaluation unit, 12 is an evaluation criterion learning unit, 13 is an advice presenting unit, 14 is a transmission unit for managers, 21 is a safety confirmation diagnostic system, 22 is an automatic travel device operation detection unit, 23 is an automatic travel device mode change control unit, and 24 is a line of sight. It is a detection unit.

Claims (11)

A camera (2) that captures the face of the driver of the vehicle;
A face direction detection unit (3) for detecting the direction of the driver's face based on the image data of the driver's face;
A driving scene detector (9) for detecting a driving scene of the vehicle;
An evaluation section dividing unit (10) for dividing the driving scene into evaluation sections having different evaluation criteria;
A safety confirmation diagnosis system comprising a safety confirmation action evaluation unit (11) that evaluates a driver's safety confirmation action based on a driver's face direction and an evaluation section of a vehicle driving scene.   The evaluation section dividing unit (10) is a driving action change point based on the information on the driving scene, the information on the blinker signal, the information on the vehicle speed, and the information on the steering state detected by the driving scene detection unit (9). The safety confirmation diagnosis system according to claim 1, wherein the driving scene is divided based on the detected change point.   The safety confirmation diagnosis system according to claim 2, wherein the change points of the driving behavior are a blinker lighting time point, a steering wheel turning start time point, and a steering wheel turning back time point. It has a white line detector that detects the white line of the road,
The evaluation section dividing unit (10) is configured to divide the driving scene by detecting a change point of driving action from a positional relationship between the detected position of the white line and the position of the vehicle. The safety confirmation diagnostic system according to claim 1.   The safety confirmation diagnosis system according to claim 1, wherein the driving scene detection unit (9) is configured to detect a driving scene as a different driving scene when the lane in which the vehicle travels is different.   The safety confirmation diagnosis system according to claim 1, further comprising an evaluation criterion learning unit (12) for learning an evaluation criterion of the safety confirmation action evaluation unit (11) based on a driver's normal driving or an exemplary driver's driving.   The advice presenting part (13) for proposing advice to the driver based on the evaluation result of the safety confirming action from the safety confirming action evaluating part (11). Safety confirmation diagnostic system.   Based on the evaluation result of the safety confirmation action from the safety confirmation action evaluation unit (11), information on the evaluation result of the driver's safety confirmation action, information on the position of the vehicle when the safety confirmation action is evaluated, and 8. The system according to claim 1, further comprising a transmission unit for an administrator (14) that transmits information on a time when the safety confirmation action is evaluated to a management office that manages the vehicle. Safety confirmation diagnostic system. An automatic traveling device operation detecting unit (22) for detecting an operating state of the automatic traveling device;
An automatic traveling device mode change control unit (23) for determining whether or not it is necessary to change the operation mode of the automatic traveling device based on the evaluation result of the safety confirmation behavior by the safety confirmation behavior evaluating unit (11). The safety confirmation diagnostic system according to any one of claims 1 to 8, further comprising: A gaze detection unit (24) for detecting the gaze of the driver,
The safety according to any one of claims 1 to 9, wherein the safety confirmation action evaluation unit (11) is configured to evaluate a driver's safety confirmation action in consideration of a detection result of the driver's line of sight. Confirmation diagnostic system. A camera (2) that captures the face of the driver of the vehicle, a face direction detection unit (3) that detects the face direction of the driver based on image data of the driver's face, and a driving scene of the vehicle A safety confirmation diagnosing method comprising a driving scene detecting unit (9) and an evaluation section dividing unit (10) for dividing a driving scene into evaluation sections having different evaluation criteria,
A safety confirmation diagnosis method for evaluating a driver's safety confirmation behavior based on a driver's face direction and an evaluation section of a vehicle driving scene.

Images