JP2019114104A - Oversight detection device, oversight detection method, and program - Google Patents

Abstract

To provide an oversight detection device that appropriately detects oversight of a driver's risk depending on a surrounding situation.SOLUTION: An oversight detection device 1 comprises: a surrounding area monitoring sensor 10 for monitoring a surrounding state of a vehicle; an in-vehicle camera 13 for photographing a driver; a risk level calculation unit 14 for obtaining a risk level in each direction viewed from the vehicle on the basis of data on the surrounding condition obtained by monitoring by the surrounding area monitoring sensor 10; a gaze ratio calculation unit 15 that detects a gaze ratio according to each direction by calculating a gaze target that the driver gazes within a predetermined time and a gaze time thereof on the basis of an image captured by the in-vehicle camera 13 and by specifying a direction in which the gaze target is present; and an oversight detection unit 16 for calculating a risk of an oversight on the basis of the risk level according to the direction and the gaze ratio obtained by the gaze ratio calculation unit 15, and for detecting an oversight of the risk when a possibility of the oversight risk is greater than or equal to a predetermined threshold value in either direction.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a technology for detecting whether a driver has overlooked a risk around a vehicle.

  2. Description of the Related Art Conventionally, there is known a device for raising the driver's attention when there is an object to be noted by the driver around the vehicle. The device described in Patent Document 1 calculates individual risk levels when there are a plurality of risk targets to be watched by the driver, and selects a risk target having the highest risk level from among the plurality of risk targets. Then, a display indicating the direction of the risk object with the highest risk level is performed.

  The device described in Patent Document 2 determines the degree of reaction of the driver to the attraction display after performing the attraction display for guiding the driver's gaze in order to make the driver practice safety confirmation. Provide appropriate feedback.

  Patent document 3 calculates | requires the direction which a driver should look at based on the surrounding condition detected by the surrounding condition detection part, and the apparatus which guides a driver's gaze to the direction, The apparatus which reduces a driver's eyes burden Is disclosed. In this device, when a driver does not view the direction in a predetermined direction for a certain period of time after detecting a new object, the detected object is highlighted.

JP, 2016-197407, A JP, 2017-142621, A JP, 2015-125686, A

  Although Patent Documents 1 and 2 described above guide the driver's gaze in the direction of the target of the risk found around the vehicle, they are not concerned with the driver's driving behavior. Patent Document 3 performs control based on the behavior of the driver in that the display of the gaze guidance is not performed when the driver has already viewed the target of the risk. However, according to the invention described in Patent Document 3, the visual guidance is only displayed in response to the fact that the direction of the new object has not been seen for a certain period of time after detection.

  Since there are various traffic participants with different speeds on the actual road, and the situation changes from moment to moment, the target for the driver to pay attention changes depending on the positional relationship with the traffic participants. If a uniform process like Patent Document 3 is performed, the alerting to the driver may be delayed or, conversely, the alert may be alerted even though the driver has already recognized the risk. there is a possibility.

  SUMMARY OF THE INVENTION In view of the above background, the present invention has an object to provide an oversight detection device that appropriately detects oversight of a driver's risk according to the surrounding situation.

  The oversight detection device of the present invention is a vehicle based on data of a surrounding condition obtained by monitoring with a surrounding area monitoring sensor that monitors the surrounding condition of the vehicle, a camera that photographs the driver, and the surrounding area monitoring sensor. A risk level calculator for obtaining a risk level for each direction viewed from the point of view, and a gaze target at which the driver gazes within a predetermined time based on the image captured by the camera and the gaze time thereof; Possibility of overlooking the risk based on the gaze ratio calculation unit that detects the gaze ratio according to the direction by specifying the direction to go, and the gaze level calculated by the gaze ratio calculation unit and the risk level according to the direction And an oversight detector configured to calculate the gender and detect oversight of the risk if the possibility of oversight of the risk is greater than or equal to a predetermined threshold value in any direction.

  With this configuration, an oversight is detected based on the risk level and the possibility of risk oversight calculated based on the driver's gaze ratio at a predetermined time, so that based on the surrounding situation and the situation of the driver's attention, Appropriately detect oversight of risks. Here, “possibility of oversight” is an index determined by the risk level by direction (which may be called the magnitude of the risk) and the rate at which the driver is gazing in that direction, a serious risk Is defined to be a large value when not looking at the direction in which For example, it can be defined by the possibility of risk oversight = risk level / gaze ratio.

  In the missed detection device of the present invention, the risk level calculation unit may calculate the risk levels of the own lane and the lanes or the sidewalks located to the left and right of the own lane as the risk level by direction. The left and right lanes are, for example, an opposite lane, a passing lane, a traveling lane, etc. In the case of one lane on one side, the left side of the own lane is a sidewalk. The driver's own lane is the leading vehicle, the overtaking lane or the traveling lane is the parallel vehicle, the opposite lane is the oncoming vehicle, and the sidewalk is a bicycle or a pedestrian. Since the moving speed and direction of the object are different, it is one desirable method to calculate the risk level for each of the own lane and the lanes or sidewalks to the left and right of the own lane.

  In the overlook detection system according to the present invention, when the gaze ratio calculation unit can not identify the gaze target gazed by the driver, the lane line may be determined based on one or both of the face direction and the line of sight of the driver. The gaze ratio of the lane or the sidewalk on either side of the own lane may be detected. As a result, even if the gaze target can not be identified, the processing for missing detection can be continued.

  In the overlook detection system according to the present invention, when the risk level can not be evaluated based on the data of the surrounding situation obtained by the surrounding area monitoring sensor, the risk level calculating unit sets the risk level in the direction to the maximum. It is also good. By estimating the maximum risk and making a judgment on the safe side, it is possible to reduce the risk of overlooking the driver's risk.

  The oversight detection apparatus according to the present invention guides the driver's gaze in a direction in which the possibility of oversight of risk is equal to or more than a predetermined threshold when oversight of risk is detected by the oversight detection unit. A gaze guidance unit may be provided. With this configuration, it is possible to guide the driver's gaze in the direction in which the risk exists and to prevent an accident in advance.

  The missed detection device according to the present invention may further include a deceleration instruction unit that outputs a deceleration instruction to the vehicle when the missed detection unit detects that a risk is missed. This configuration makes it easier to deal with risks.

  The oversight detection method according to the present invention comprises the steps of monitoring the surrounding condition of the vehicle by the surrounding area monitoring sensor, photographing the driver with the camera, and data of the surrounding condition obtained by monitoring by the surrounding area monitoring sensor. Determining a risk level for each direction viewed from the vehicle based on the image captured by the camera, and detecting a gaze target and a gaze time that the driver gazes within a predetermined time as a gaze ratio The possibility of risk oversight is calculated based on the risk level according to the direction and the gaze ratio, and the risk oversight is considered when the possibility of oversight is above a predetermined threshold in any direction. And detecting.

  The program according to the present invention is a program for detecting whether the driver has overlooked the risk around the vehicle, and the computer receives the data of the surrounding situation of the vehicle obtained by monitoring with the surroundings monitoring sensor. Based on the steps, the step of receiving video data obtained by photographing the driver with a camera, and the data of the surrounding situation obtained by monitoring with the surroundings monitoring sensor, the risk level according to the direction viewed from the vehicle is determined Detecting the gaze target and the gaze time that the driver gazes within a predetermined time as a gaze ratio based on the video data received from the camera, and based on the risk level according to the direction and the gaze ratio Calculate the possibility of missed risks, and detect missed risks if the possibility of missed risks in any direction is greater than or equal to a predetermined threshold To perform the steps that.

  According to the present invention, there is an effect that the oversight of risk can be appropriately detected based on the surrounding situation and the situation of the driver's gaze.

It is a figure showing composition of an overlook detection device of an embodiment. It is a figure which shows the example of the driving | running | working scene where the oversight detection apparatus of this Embodiment is used. (A) It is a figure which shows the model which calculates the risk level of the own lane. (B) It is a figure which shows the model which calculates the risk level of an opposite lane. (C) It is a figure which shows the model which calculates the risk level of a sidewalk. It is a figure which shows the process of a gaze ratio calculation part. It is a figure for demonstrating a gaze ratio. (A) It is a figure which shows the example of the risk level according to the direction calculated in the risk level calculation part. (B) It is a figure which shows the example of the gaze ratio calculated | required by the gaze ratio calculation part. It is a figure which shows the example which calculated the possibility of an oversight. It is a figure which shows the example of a structure of a gaze guidance part. It is a figure which shows operation | movement of the omission detection apparatus of embodiment. It is a figure which shows operation | movement of the oversight detection apparatus which concerns on a modification.

  Hereinafter, an overlook detection apparatus according to an embodiment of the present invention will be described with reference to the drawings. The oversight detection device according to the embodiment is mounted on a vehicle and detects whether a driver driving the vehicle has overlooked a risk. It should be noted that the oversight detection device is not only for a completely manually operated vehicle, but also for automatic operation level 1 (a state where the system supports single acceleration, steering, or braking alone), automatic operation level 2 (system The present invention can also be applied to a vehicle in which the system performs a plurality of operations simultaneously among acceleration, steering, and braking while observing the driving environment.

  FIG. 1 is a diagram showing the configuration of the missed detection apparatus 1 according to the embodiment. The oversight detection device 1 includes a periphery monitoring sensor 10 for monitoring the surroundings of the vehicle, an in-vehicle camera 13 for capturing an image of a driver, a risk level calculator 14 for obtaining a risk level for each direction viewed from the vehicle, and a predetermined time The gaze ratio calculation unit 15 calculates the gaze ratio according to the driver's direction in and the overlook detection unit 16 that detects the omission of the risk from the risk level and the gaze ratio.

  The surrounding area monitoring sensor 10 identifies objects such as other vehicles, pedestrians and obstacles around the vehicle (hereinafter, also simply referred to as “near objects”) and detects position information of the objects around them as the surrounding situation. . Although the camera 11 outside the vehicle and the radar 12 are used as the surroundings monitoring sensor 10 in the present embodiment, sensors other than these can also be used.

  The in-vehicle camera 13 is a near infrared camera, and can stably capture the driver even if the vehicle interior becomes dark. The in-vehicle camera 13 is mounted at a position where the driver is photographed from the front, for example, under a meter visor. It is also possible to use a “driver status monitor” camera developed by Denso Corporation.

  The risk level calculator 14 calculates the risk level in each direction viewed from the vehicle, based on the data of the surrounding situation obtained by monitoring by the surrounding area monitoring sensor 10.

  FIG. 2 is a diagram showing an example of a driving scene in which the oversight detection device 1 of the present embodiment is used. In the example shown in FIG. 2, the right side of the own lane in which the vehicle equipped with the oversight detection device 1 is traveling is the oncoming lane, and the left side is the sidewalk. In this case, the oversight detection device 1 calculates the risk level separately for the own lane, the oncoming lane, and the direction of the sidewalk. In addition, the overlook detection apparatus 1 of this Embodiment can be used also for the driving | operation scene different from FIG. 2 like one side several lanes.

  FIG. 3A to FIG. 3C are diagrams showing an example of a risk level calculation method by the risk level calculation unit 14. FIG. 3A shows an example of a model for calculating the risk level of the own lane. The risk level of the own lane is a model that results in "the risk of collision with the preceding vehicle" due to "relationship with the preceding vehicle" and "possibility of interruption from the left and right lanes". The probability that the result will occur for the cause can be obtained in advance by learning. Examples of the “relation with the preceding vehicle” include the vehicle speed of the preceding vehicle (difference with the vehicle speed of the own vehicle), the change in the vehicle speed of the preceding vehicle (sudden deceleration etc.), the inter-vehicle distance with the preceding vehicle, and the like. As "the possibility of interruption from the left and right lanes", there are blinkers and forward and backward movements of a vehicle traveling in the left and right lanes. In the driving scene as shown in FIG. 2, there is no input to the node of “probability of interruption from the left and right lanes”.

  FIG. 3 (b) shows an example of a model for calculating the risk level of the oncoming traffic lane. The risk level of the oncoming traffic lane is a model that results from the "front collision risk with oncoming vehicles" as a result of "movement of oncoming vehicles etc." As "movement of oncoming vehicles etc." possibility that the oncoming vehicle is interrupting to the own lane (such as the blinker and speed of the oncoming vehicle), and when the oncoming vehicle is not moving, it is possible for a person to jump out from behind the oncoming vehicle There is sex.

  FIG. 3C shows an example of a model for calculating the risk level of the sidewalk. The risk level of the sidewalk is a model that is based on the "boundary between the sidewalk and the lane" and "the movement of the pedestrian" and the result is "the risk of the pedestrian's jumping out". The "boundary between a sidewalk and a lane" includes the presence or absence of an object that blocks a sidewalk such as a guardrail and the driveway, and the presence or absence of an object that promotes crossing of pedestrians such as a pedestrian crossing. "Pedestrian's movement" includes the direction or movement of the pedestrian (is it moving toward the road).

  FIG. 6A is a diagram showing an example of the risk level by direction calculated by the risk level calculator 14. Risk levels are calculated for each of the own lane, the opposite lane, and the sidewalk. In this example, the risk level of the own lane is the highest, followed by the sidewalk and the oncoming lane.

  In the present embodiment, an example of using a model to calculate the risk level by direction is given. However, for calculation of the risk level, for example, a score is added to the risk factor and calculation is performed by the method of accumulating For example, a method different from the method described in the present embodiment may be adopted.

  Referring back to FIG. The gaze ratio calculation unit 15 calculates the gaze target that the driver gazes within a predetermined time based on the image captured by the in-vehicle camera 13 and the gaze time thereof, and identifies the direction in which the gaze target exists. Detect the gaze ratio of

  FIG. 4 is a diagram showing the process of the gaze ratio calculation unit 15. As shown in FIG. 4, the gaze ratio calculation unit 15 specifies a target (preceding vehicle, oncoming vehicle, pedestrian, etc.) existing outside the vehicle and its position. The identification of targets outside the vehicle may use data of targets detected when calculating the risk level in the risk level calculation unit 14. Further, the gaze ratio calculation unit 15 detects the face direction and the gaze direction of the driver from the image obtained by the in-vehicle camera 13 (S11). The gaze ratio calculation unit 15 detects the direction in which the driver is facing the eyes by detecting the direction in which the driver's eyes are black, taking into account the direction of the driver's face in consideration of the driver's face direction. Detect if you are facing.

  The gaze ratio calculation unit 15 detects a target outside the vehicle present in the direction in which the driver is looking as a gaze target (S12). Subsequently, the gaze ratio calculation unit 15 identifies the direction in which the driver is gazing based on the gaze target (S13), and stores the identified gaze direction (S14). For example, if the gaze target is a leading vehicle, the driver is gazing at his own lane, for example, if the gaze target is a pedestrian, the driver is gazing at the sidewalk. The gaze ratio calculation unit 15 can obtain the gaze ratio by repeatedly performing the above processing.

  FIG. 5 is a diagram for explaining the gaze ratio. In FIG. 5, the time of passing through the place where the pedestrian is present is 0 seconds, and the gaze ratio data obtained every 4 seconds is shown going back to 12 seconds. For example, in a section C of 4 seconds to 0 seconds, it can be seen that the leading vehicle is watched for about 4 seconds to 2.4 seconds, and the pedestrian is watched for 2.4 seconds to 0 seconds. In this case, it is determined that the gaze ratio of the preceding vehicle is 40% (1.6 seconds) and the gaze ratio of the pedestrian is 60% (2.4 seconds). The gaze ratio calculation unit 15 identifies the gaze direction as the own lane when the gaze target is a preceding vehicle, and identifies the gaze direction as a sidewalk when the gaze target is a pedestrian.

  The reason why the gaze direction is obtained from the gaze target in the present embodiment is that the gaze direction can be identified with high accuracy. For example, if the driver is looking diagonally forward to the left, if there is a pedestrian in that direction, it can be determined that the pedestrian is looking at the sidewalk, and if there is no pedestrian and there is a leading vehicle, one is looking at the own lane It can be determined that However, when the gaze target can not be specified, the gaze ratio calculation unit 15 may estimate the gaze direction from the direction in which the driver is looking.

  The oversight detection unit 16 calculates the possibility of risk oversight based on the risk level for each direction obtained by the risk level calculation unit 14 and the gaze ratio obtained by the gaze ratio calculation unit 15. In the present embodiment, a value obtained by dividing the risk level by direction by the gaze ratio by direction is used as an index indicating the possibility of missing.

  FIG. 6 (b) is a diagram showing an example of the gaze ratio calculated by the gaze ratio calculator 15 in the section corresponding to FIG. 6 (a). As shown in FIG. 6 (b), in this section, the gaze ratio to the own lane occupies most, and the gaze ratio to the opposite lane and the sidewalk is small.

  FIG. 7 is a diagram showing an example in which the possibility of missing is calculated based on the risk level shown in FIG. 6 (a) and the gaze ratio shown in FIG. 6 (b). In the example shown in FIG. 7, the risk of overlooking the risk in the direction of the sidewalk is the highest, and although the risk level was high for the own lane (FIG. 6 (a)), the possibility of overlooking the risk is small. . This is because the rate at which the driver gazes at his own lane is high. The oversight detection unit 16 detects that a risk is overlooked when the possibility of oversight of the risk is equal to or more than a predetermined threshold value in any direction of the own lane, the opposite lane, and the sidewalk. In the example illustrated in FIG. 7, since the possibility of overlooked sidewalk risk is equal to or higher than the threshold, it is detected that there is an overlooked risk.

  The overlook detection apparatus 1 according to the present embodiment, in addition to the above-described overlook detection function, guides a direction of a target of a risk at which the driver's gaze has been overlooked when a miss of the risk is detected. A gaze guidance unit 17 and a deceleration instructing unit 18 for instructing the operation control unit 20 to decelerate are provided.

  FIG. 8 is a diagram showing an example of the configuration of the sight-line guiding unit 17. As shown in FIG. The line-of-sight guidance unit 17 is configured of a linear light emitting area 31 disposed in an instrument panel 30 of the vehicle. The linear light emitting region 31 linearly extends along the width direction WD of the vehicle, and includes a large number of light emitting elements linearly arranged. The light emitting spot 32 can be displayed by causing at least a part of the large number of light emitting elements of the linear light emitting region 31 to emit light. The gaze guidance unit 17 displays the light emission spot 32 at a position corresponding to the direction in which the driver's gaze is desired to be guided. In addition, even if the gaze guidance unit 17 moves the light emission spot 32 in a direction in which the driver's gaze is desired to be guided (for example, from right to left when it is desired to guide the driver's gaze left). Good.

  The configuration of the oversight detection apparatus 1 according to the present embodiment has been described above. The hardware of the oversight detection apparatus 1 is, for example, an ECU, a RAM, a ROM, a hard disk, a display, a keyboard, a mouse, a communication interface, etc. Computer with A program having a module for realizing the risk level calculating unit 14, the gaze ratio calculating unit 15, and the oversight detecting unit 16 is stored in the RAM or the ROM, and the ECU executes the program by the ECU. The function of the detection device 1 is realized. Such programs are also included in the scope of the present invention.

  FIG. 9 is a diagram showing the operation of the missed detection apparatus 1 of the embodiment. The overlook detection device 1 according to the embodiment monitors the surroundings of the vehicle by the surroundings monitoring sensor 10 to acquire data of the surrounding situation (S20), and based on the acquired data of the surrounding situation, the risk level by direction is acquired Is calculated (S21). Further, the oversight detection device 1 shoots the driver with the in-vehicle camera 13 (S22), and calculates the gaze ratio in each direction in a predetermined time based on the captured video of the driver (S23).

  Subsequently, the oversight detection apparatus 1 divides the risk level by the gaze ratio according to the direction, calculates the possibility of the oversight according to the direction, and detects the oversight of the risk based on the possibility of the oversight (S24) . If there is no oversight of the risk (if the possibility of oversight of the risk is not higher than the threshold in any direction) (NO in S25), the process returns to the first (START) and repeats the above operation. If there is an oversight (YES in S25), the sight-line guiding unit 17 of the oversight detection device 1 guides the driver's line of sight in the direction of the target of the overlooked risk (S26). Then, the process returns to the beginning (START) to repeat the above operation.

  Heretofore, the configuration and operation of the omission detection apparatus 1 according to the present embodiment have been described. Since the oversight detection device 1 of the present embodiment detects oversight based on the risk level and the possibility of oversight of the risk calculated based on the driver's gaze ratio at a predetermined time, the surrounding situation and the driver are detected. The risk oversight can be detected appropriately based on the situation of the gaze of

  For example, as shown in FIGS. 6 (a), 6 (b) and 7, even when the risk level of the own lane is high, in the case where the driver is closely watching the own lane And the risk is low. In this case, it is only annoying for a driver who is already paying sufficient attention to alerting, so that the risk of being overlooked is unlikely to be overlooked and alerting is not conducted unnecessarily. On the contrary, even if the risk level is medium, as in the sidewalk, if the driver's gaze ratio on the sidewalk is small, there is a possibility that the risk may be overlooked. I do.

  Thus, unlike the prior art which determines whether or not to alert the driver depending on whether the overlooked object detection device 1 of the present embodiment simply looks at the object that is the object of risk or the direction thereof, It is possible to properly detect oversight based on the magnitude of risk and the driver's driving behavior.

  As mentioned above, although the embodiment of the omission detection device of the present invention has been described in detail, the omission detection device of the present invention is not limited to the above embodiment.

  The above embodiment has not described the case where the risk level calculation unit 14 can not calculate the risk level based on the surrounding situation data acquired by the surrounding area monitoring sensor 10. For example, when the surrounding area monitoring sensor 10 can not obtain sufficient surrounding situation data, for example, when there is a large blind spot due to heavy rain, fog, or a large car, the risk level may be calculated based on the surrounding situation data. Although this may not be possible, in such a case, the risk level calculator 14 sets the risk level in the direction to the maximum. This makes it possible to detect missed risks on the safety side.

  In the embodiment described above, an example was described in which when the oversight detection unit 16 detects a risk oversight, it guides the driver's gaze in the direction of the risk target, but the oversight detection unit 16 detects an oversight At times, the operation control unit 20 may be instructed to reduce the speed of the vehicle.

  Further, in accordance with the possibility of the oversight detected by the oversight detection unit 16, the visual line guidance and the deceleration may be selected. FIG. 10 is a diagram showing the operation of the missed detection apparatus according to the modification. The basic operation of the oversight detection apparatus shown in FIG. 10 is the same as the flow shown in FIG. 9, but the operation after detecting oversight (YES in S25) is different. In the oversight detection apparatus according to the modification, when detecting oversight by the oversight detection unit 16, it is determined whether or not the possibility of oversight is more than a threshold value in two or more directions. (S27). If the possibility of an oversight is more than the threshold value in only one direction (NO in S27), the oversight detection device guides the driver's gaze in that direction (S28). For example, if there is a possibility of risk oversight in two or more directions such as a sidewalk and an opposite lane (YES in S27), the oversight detection device causes the operation control unit 20 to decelerate the vehicle. (S29).

  In the above-described embodiment, the gaze ratio calculation unit 15 calculates the gaze ratio by the length of time in which each direction is gazed at a predetermined time interval, but weights are given to gaze at the time interval closer to the current time. You may For example, in the section C of 4 seconds to 0 seconds in FIG. 5 described above, the closer two seconds (2 seconds to 0 seconds) are given twice the weight of the distant 2 seconds (4 seconds to 2 seconds) It is also good. In the example of FIG. 5, the pedestrian is seen for 2 seconds to 0 seconds, the time for looking at the leading vehicle is 1.6 seconds, and the time for seeing the pedestrian is 0.4 seconds for 4 seconds to 2 seconds. It is a second. In this case, the pedestrian's gaze is 2 seconds × 2 (weight) +0.4 seconds × 1 (weight) = 4.4, and the gaze of the preceding vehicle is 1.6 seconds × 1 (weight) = 1.6. Desired. Accordingly, the gaze ratio of the pedestrian is 4.4 / (4.4 + 1.6) × 100 = 73.3%, and the gaze ratio of the preceding vehicle is 1.6 / (4.4 + 1.6) × 100 = 26.7 It becomes%.

  In addition, the gaze ratio calculation unit 15 may omit the time when eye gaze measurement could not be performed due to blinking or the like from the calculation time.

  In addition, the gaze ratio calculation unit 15 may determine the gaze ratio by the number of times instead of the time when gazed in each direction in a predetermined time interval. For example, in the section C of 4 seconds to 0 seconds shown in FIG. 5, since the pedestrian and the leading vehicle are viewed once, the gaze ratio between the pedestrian and the leading vehicle may be 50%. Further, at this time, the number of times may be counted only when the gaze of a pedestrian or a leading vehicle is watched for a predetermined time (for example, 0.1 seconds) or more.

1 Overlook Detection Device 10 Peripheral Monitoring Sensor 11 Outside Camera 12 Radar 13 In-Car Camera 14 Risk Level Calculator 15 Gaze Ratio Calculator 16 Oversight Detection Unit 17 Line-of-sight Guidance Unit 18 Deceleration Command Unit 20 Operation Control Unit 30 Instrument Panel 31 Linear luminous area 32 luminous point

Claims (8)

A perimeter monitoring sensor that monitors the surrounding status of the vehicle;
With the camera to shoot the driver,
A risk level calculation unit for obtaining a risk level in each direction viewed from the vehicle based on data of the surrounding condition obtained by monitoring by the surrounding area monitoring sensor;
The gaze target at which the driver gazes within the predetermined time and the gaze time thereof are calculated based on the video taken by the camera, and the gaze ratio by direction is detected by specifying the direction in which the gaze target exists. The gaze ratio calculation unit,
Calculate the possibility of risk oversight based on the risk level according to the direction and the gaze ratio determined by the gaze ratio calculator, and the possibility of risk oversight in any direction is greater than or equal to a predetermined threshold An overlooked detection unit that detects overlooked risks in certain cases, and
Overlook detection device equipped with.   The missing detection device according to claim 1, wherein the risk level calculation unit calculates the risk levels of the own lane and the lanes or the sidewalks located to the left and right of the own lane as the risk levels classified by direction.   When the gaze ratio calculation unit can not identify the gaze target the driver is gazing at, based on one or both of the face direction and the line of sight of the driver, the own lane and lanes or sidewalks to the left and right of the own lane The overlook detection apparatus according to claim 2, wherein the gaze ratio of is detected.   4. The risk level calculation unit according to any one of claims 1 to 3, wherein, when the risk level can not be evaluated based on the data of the surrounding situation obtained by the surrounding area monitoring sensor, the risk level in the direction is set to the maximum. Overlook detection device.   The eye gaze guidance unit for guiding the driver's gaze in a direction in which the possibility of risk oversight is equal to or more than a predetermined threshold when the oversight of the risk is detected in the oversight detection unit. The missed detection device according to any one of 4.   5. The oversight detection apparatus according to any one of claims 1 to 4, further comprising a deceleration instruction unit that outputs a deceleration instruction to the vehicle when the oversight detection unit detects an oversight of a risk. Monitoring the surroundings of the vehicle by the surroundings monitoring sensor;
Shooting the driver with a camera,
Determining a risk level for each direction viewed from the vehicle based on data of the surrounding condition obtained by monitoring by the surrounding area monitoring sensor;
Detecting the gaze ratio by direction based on a gaze target that the driver gazes within a predetermined time and a gaze time of the gaze based on the video taken by the camera;
Calculate the possibility of risk oversight based on the risk level according to the direction and the gaze ratio, and detect the oversight of risk when the possibility of oversight of the risk is higher than a predetermined threshold in any direction Step to
Overlook detection method comprising. It is a program for detecting whether the driver overlooks the risk around the vehicle, and the computer
Receiving the data of the surroundings of the vehicle obtained by monitoring by the surroundings monitoring sensor;
Receiving video data obtained by photographing a driver with a camera;
Determining a risk level for each direction viewed from the vehicle based on data of the surrounding situation received from the surrounding area monitoring sensor;
Detecting the gaze ratio by direction based on a gaze target the driver gazes within a predetermined time and the gaze time thereof based on the video data received from the camera;
Calculate the possibility of risk oversight based on the risk level according to the direction and the gaze ratio, and detect the oversight of risk when the possibility of oversight of the risk is higher than a predetermined threshold in any direction Step to
A program that runs

Images