JP2018147021A - Dangerous state prediction device, dangerous state prediction method, dangerous state prediction program, dangerous state prediction data acquisition device, dangerous state prediction data acquisition method, and dangerous state prediction data acquisition program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a dangerous state prediction device, etc. for preventing occurrence of a dangerous state in a monitored person.SOLUTION: A dangerous state prediction device predicts occurrence of a dangerous state in a monitored person. The dangerous state prediction device includes a symptom behavior detection section for monitoring a behavior of a monitored person and detecting a behavior by a monitored person that coincides with a symptom behavior indicating a symptom that a state of a monitored person is shifted to a dangerous state.SELECTED DRAWING: Figure 6

Description

  The present invention relates to a dangerous state prediction device, a dangerous state prediction method, a dangerous state prediction program, a dangerous state prediction data acquisition device, a dangerous state prediction data acquisition method, and a dangerous state prediction data acquisition program.

The following methods are conceivable as methods for determining that a driver who is driving a vehicle is in a state of danger such as reduced attention and sleepiness.
(1) A method of determining that the driver's attention during driving is decreasing from sudden braking or steering wheel operation, vehicle acceleration sensor data, or a photographed image of the driver (2) During driving Whether the driver is drowsy is determined from changes in the amount of steering wheel operation, driving data such as how often the white line of the road protrudes from the tires, changes in vital data such as pulse and breathing, etc. To judge from the processed image

However, the determination result of the occurrence of the dangerous state obtained from the travel data, vital data, and the captured image of the driver has a large variation among individual drivers, and it is not always possible to determine the appropriate occurrence of the dangerous state. There's a problem.
Based on the time from when the driver is notified of the occurrence of drowsiness, which is one of the dangerous situations, until the manual release, the dozing operation can be easily changed according to the driver's preference and characteristics. A driving alarm device has been proposed (see, for example, Patent Document 1).

JP 2002-183900 A

  However, like the method described in Patent Document 1, there is a problem that it is too late even if the driver is warned after a dangerous state occurs.

  In one aspect, an object of the present invention is to provide a dangerous state prediction apparatus that can prevent a dangerous state from occurring in a monitored person.

  In one embodiment, the dangerous state prediction device is a dangerous state prediction device that predicts that a dangerous state will occur in the monitored person, and monitors the operation of the monitored person, and the monitored person is in a dangerous state. A sign action detection unit that detects an action performed by the monitored person that coincides with a sign action indicating a sign of transition to

  In one aspect, it is possible to provide a dangerous state prediction device that can prevent a dangerous state from occurring in a monitored person.

FIG. 1 is a hardware configuration diagram illustrating an example of a dangerous state prediction apparatus according to the present invention. FIG. 2 is a functional block diagram showing an example of the dangerous state prediction apparatus of the present invention. FIG. 3 is a schematic diagram illustrating an example of an assumed model of a driver's dangerous state. FIG. 4A is a functional block diagram illustrating an example of a dangerous state detection process. FIG. 4B is a diagram illustrating an example of a dangerous state determination threshold value table used in the dangerous state detection process. FIG. 4C is a diagram illustrating an example of information in which the driver's dangerous state and the driver's moving image data stored in the dangerous state moving image data storage unit are associated with each other. FIG. 5A is a functional block diagram illustrating an example of determination data extraction processing. FIG. 5B is a diagram illustrating an example of a symptom operation used for determination data extraction processing and a criterion for determining the symptom operation. FIG. 5C is a diagram illustrating an example of moving image data stored in the dangerous state moving image data storage unit. FIG. 6 is a functional block diagram illustrating an example of the sign action detection process. FIG. 7 is a flowchart illustrating an example of the flow of a dangerous state detection process. FIG. 8 is a flowchart illustrating an example of the flow of the determination data extraction process. FIG. 9 is a flowchart illustrating an example of the flow of detection processing of the sign operation. FIG. 10 is a diagram illustrating an example of a result of the sign action detection process. FIG. 11 is a diagram illustrating another example of the result of the sign action detection process.

Hereinafter, although one embodiment of the present invention is described, the present invention is not limited to these embodiments.
Note that the control performed by the sign motion detection unit or the like in the “dangerous state prediction apparatus” of the present invention is synonymous with the implementation of the “dangerous state prediction method” of the present invention, and therefore the “dangerous state prediction apparatus” of the present invention. The details of the “dangerous state prediction method” of the present invention will also be clarified through the description of FIG. Further, since the “dangerous state prediction program” of the present invention is realized as the “dangerous state prediction device” of the present invention by using a computer or the like as a hardware resource, the “dangerous state prediction device” of the present invention The details of the “dangerous state prediction program” of the present invention will be clarified through the explanation.
The control performed by the recording unit in the “dangerous state prediction data acquisition apparatus” of the present invention is synonymous with the implementation of the “dangerous state prediction data acquisition method” of the present invention. The details of the “dangerous state prediction data acquisition method” of the present invention will be clarified through the description of the “prediction data acquisition device”. The “dangerous state prediction data acquisition program” of the present invention is realized as the “dangerous state prediction data acquisition device” of the present invention by using a computer or the like as a hardware resource. The details of the “dangerous state prediction data acquisition program” of the present invention will be clarified through the description of the “dangerous state prediction data acquisition device”.
Furthermore, since the “dangerous state prediction data acquisition device” of the present invention is synonymous with the recording unit in the “dangerous state prediction device” of the present invention, the description will be given through the description of the recording unit of the “dangerous state prediction device” of the present invention. The details of the “dangerous state prediction data acquisition device” of the invention will also be clarified.

(Dangerous state prediction device and dangerous state prediction method)
The dangerous state prediction apparatus of the present invention is a dangerous state prediction device that predicts that a dangerous state will occur in a monitored person, preferably has a sign operation detection unit, and preferably has a recording unit and a warning unit. It has other parts as needed.

  The dangerous state prediction method of the present invention is a dangerous state prediction method for predicting the occurrence of a dangerous state in a monitored person, preferably including a sign operation detection step, and preferably including a recording step and a warning step. Other processes are included depending on the situation.

  The dangerous state prediction method of the present invention can be preferably implemented by the dangerous state prediction apparatus of the present invention, the sign operation detection step can be performed by the sign operation detection unit, and the recording step can be performed by the recording unit. The warning process can be performed by the warning unit, and the other processes can be performed by other means.

  According to the dangerous state prediction apparatus and the dangerous state prediction method of the present invention, the monitored person is monitored, and the monitored person is detected by detecting an action performed by the monitored person that coincides with the indication action. Since it is possible to predict in advance that a dangerous state will occur, it is possible to prevent a dangerous state from occurring.

<Sign operation detection step and indication operation detection unit>
The indication operation detection step is a step of monitoring an operation of the monitored person and detecting an operation performed by the monitored person that coincides with an indication operation indicating an indication that the monitored person enters a dangerous state. Implemented by the sign operation detection unit.

-Monitored person-
The “monitored person” in the “dangerous state prediction device” or the like is not particularly limited as long as it is a person who needs to be monitored in order to prevent the occurrence of a dangerous state, and is appropriately selected according to the purpose. For example, a driver of a vehicle, an elderly person living alone, a certified care recipient, a worker at a construction work site, a patient who suffers a disturbance of consciousness due to a seizure, a patient who suffers a movement disorder due to a seizure, and the like.

The driver of the vehicle is always driving during driving in order to prevent the occurrence of dangerous conditions such as reduced driver's attention, driver drowsiness, driver's drunkenness, etc. People need to be monitored.
There is no restriction | limiting in particular as said "vehicle", According to the objective, it can select suitably, For example, a train, a train, a vehicle, a ship, an airplane, etc. are mentioned. Among these, a vehicle is preferable.
There is no restriction | limiting in particular as said vehicle, According to the objective, it can select suitably, For example, a large sized car (bus, truck, etc.), a medium sized car, or a normal car etc. are mentioned.

Workers at the construction work site need to be constantly monitored in order to prevent the occurrence of dangerous conditions such as heat stroke during work in midsummer.
The above-mentioned elderly living alone needs to be constantly monitored in order to prevent the occurrence of a dangerous state such as whether they are eating meals or are bedridden.
The certified care recipient needs to be constantly monitored in order to prevent the occurrence of a dangerous state such as falling from a bed or going out alone.
Examples of patients who cause consciousness disorder or movement disorder due to the seizure include cardiac illnesses such as arrhythmia, angina pectoris, myocardial infarction, and epileptic patients. These patients need to be monitored at all times because they do not know when the seizure will occur.

-Dangerous state-
The dangerous state of the monitored person varies depending on the type of the monitored person and cannot be defined unconditionally. For example, the state of reduced attention of the monitored person, the state of reduced cognitive power of the monitored person, Examples include the state of sleepiness of the monitored person, the state of jealousy of the monitored person, the state of consciousness disorder due to the seizure of the monitored person, and the state of movement disorder due to the seizure of the monitored person. These may be used individually by 1 type and may use 2 or more types together.

-Sign operation-
The sign behavior of the monitored person differs depending on the type of the monitored person and cannot be defined unconditionally. For example, the change in the head movement of the monitored person, the change in facial expression of the monitored person A change in the movement of the line of sight of the monitored person, the number of times the head of the monitored person is shaken, the number of vertical movements of the shoulder of the monitored person, the number of blinks of the monitored person, the eyes of the monitored person Examples include the number of times of rubbing, the number of times the monitored person reaches out, the number of times the monitored person yawns, and the number of times the monitored person sits down. These may be used individually by 1 type and may use 2 or more types together.

<Recording process and recording section>
When detecting that a dangerous state has occurred in the monitored person from a predetermined action of the monitored person,
A recording unit that extracts the symptom movement from the movement of the monitored person a predetermined time before the occurrence of the dangerous state and records it in a determination data storage unit, and the symptom movement detection unit is included in the determination data storage unit; It is preferable to detect an action performed by the monitored person that coincides with the recorded symptom action.
By having the recording unit, the sign operation can be extracted and accumulated in the determination data storage unit, so that it is possible to accurately predict in advance that a dangerous state will occur in the monitored person. The occurrence of a state can be prevented beforehand.
Here, the configuration of the recording unit corresponds to the “dangerous state prediction data acquisition device” of the present invention, and the determination data extraction process for implementing the recording unit is the “dangerous state prediction data acquisition method” of the present invention. The program that causes the computer to execute the processing of the recording unit corresponds to the “dangerous state prediction data acquisition program” of the present invention.

-Predetermined behavior of monitored person-
The predetermined operation of the monitored person is at least one of a change in the operation of the monitored person, a change in the biological information of the monitored person, and a change in the behavioral state of the monitored person, and these changes It is preferable that the operation exceeds the threshold value for determining the dangerous state.
Since the threshold value for determining the dangerous state of the monitored person is different for each monitored person, it is preferable to obtain the threshold value for each monitored person in advance and register it in the dangerous state detection unit.

<Warning process and warning section>
When the sign operation detection unit detects an operation that matches the sign operation, it is preferable that the sign operation detection unit includes a warning unit that warns the monitored person that there is a high possibility that a dangerous state will occur.
Examples of the warning unit include a vibrator, a speaker, a light, a display, a speed reduction unit, and an automatic operation switching unit. These may be used individually by 1 type and may use 2 or more types together.

<Other processes and other parts>
There is no restriction | limiting in particular as said other part, According to the objective, it can select suitably, For example, a communication part, an input part, etc. are mentioned.
There is no restriction | limiting in particular as said other process, According to the objective, it can select suitably, For example, a communication process, an input process, etc. are mentioned.

  The communication unit is not particularly limited as long as it can communicate with the outside of the vehicle, and a known one can be used as appropriate. Examples thereof include a transceiver.

  The input unit is not particularly limited as long as it can accept various requests for the dangerous driving prediction device, and a known one can be used as appropriate. Examples thereof include a keyboard, a mouse, a touch panel, and a microphone.

  In the following embodiments, the “dangerous driving prediction apparatus” in which the “monitored person” in the “dangerous state prediction apparatus” of the present invention is “the driver of the vehicle” will be specifically described as an example. The present invention is not limited to this embodiment.

<Dangerous driving prediction device>
A dangerous driving prediction device as one embodiment of the dangerous state prediction device of the present invention is a dangerous driving prediction device that predicts that a dangerous state occurs in a driver of a vehicle, and has a sign operation detection unit, and is recorded It is preferable to have a part and a warning part, and further have other parts as necessary.

  According to the dangerous driving prediction device, it is predicted in advance that a dangerous state will occur in the driver by monitoring the operation of the driver and detecting an operation performed by the driver that matches the sign operation. Therefore, it is possible to prevent a dangerous state from occurring.

−Sign motion detection unit−
The sign operation detection unit is a unit that monitors the operation of the driver and detects an operation performed by the driver that coincides with a sign operation indicating a sign that the driver shifts to a dangerous state.

--- Driver's hazardous condition ---
Examples of the driver's dangerous state include a driver's attention reduced state, a driver's drowsiness occurrence state, a driver's heel state, and the like. For example, drunk driving.

--- Driver's indication behavior--
The driver's symptom behavior includes one detected as a state change during driving and one detected as a change in the number of occurrences during driving.
The sign action detected as the state change during driving is not particularly limited and can be appropriately selected according to the purpose.For example, a change in the driver's head movement, a change in the driver's facial expression, Examples include changes in the driver's line of sight. These may be used individually by 1 type and may use 2 or more types together.
The sign action detected as a change in the number of occurrences during driving is not particularly limited and can be appropriately selected according to the purpose, for example, the number of times the driver's head is shaken, the vertical movement of the driver's shoulder The number of times, the number of blinks of the driver, the number of times that the driver's eyes are rubbed, the number of times that the driver's hand is stretched, the number of times that the driver yawns, and the number of times that the driver sits down. These may be used individually by 1 type and may use 2 or more types together. In addition, since the frequency | count in the symptom operation | movement detected as a change of the frequency | count of the generation | occurrence | production during a driving | operation changes for every driver | operator, it is preferable to obtain | require separately for every driver | operator.

-Recording section-
When the recording unit detects that a dangerous state has occurred in the driver from the predetermined operation of the driver, the recording unit extracts the indication operation from the operation of the driver a predetermined time before the dangerous state has occurred. It is preferable that a recording unit for recording in the determination data storage unit is provided, and the sign operation detection unit detects an operation performed by the driver that matches the sign operation recorded in the determination data storage unit.

--- Driver's prescribed action-
The predetermined action of the driver is at least one of a change in the driving operation of the driver, a change in the biological information of the driver, and a change in the driving state of the driver. It is preferable that the operation exceeds the state determination threshold.
Since the determination threshold value of the driver's dangerous state is different for each driver, it is preferable that the determination threshold value is obtained in advance for each driver and registered in the determination threshold value database of the dangerous state detection unit.
The change in the driver's driving operation includes, as vehicle data, for example, a driver's sudden braking tendency, a steering operation amount change, a sudden acceleration tendency, a vehicle body vibration amount change, and the like.
The change of the driver's biometric information includes, for example, the driver's pulse, respiration, heartbeat, brain wave, blood pressure, sweating, and body movement as biometric data.
The change in the driving state of the driver includes, for example, a driver's facial expression, body movement, face movement, and the like as a camera captured image.

−Warning section−
Preferably, the warning unit warns the driver that there is a high possibility of a dangerous state when the sign operation detection unit detects an operation that matches the sign operation.
The warning unit is not particularly limited as long as it can warn the driver, and can be appropriately selected according to the purpose.For example, a vibrator, a speaker, a light, a display, a speed reduction means, an automatic driving switching means, etc. Can be mentioned. These may be used individually by 1 type and may use 2 or more types together.

-Other parts-
There is no restriction | limiting in particular as said other part, According to the objective, it can select suitably, For example, a communication part, an input part, etc. are mentioned.

Example 1
Hereinafter, a dangerous driving prediction apparatus according to a first embodiment of the present invention will be described.
Here, FIG. 1 is a configuration diagram of a dangerous driving prediction apparatus 100 according to an embodiment of the present invention. A dangerous state prediction program is recorded in a memory (not shown) of the dangerous driving prediction apparatus 100, and a later-described CPU 402 reads out and executes the program, whereby a later-described dangerous state detection unit 10 and indication operation detection unit are performed. It operates as 16.

  The dangerous driving prediction apparatus 100 includes a brake electronic control unit (brake ECU) 101, a steering electronic control unit (steering ECU) 102, an accelerator electronic control unit (accelerator ECU) 103, a sensor electronic control unit (sensor ECU) 104, , A vehicle control electronic control unit (vehicle control ECU) 401, a first vital sensor 201, a first camera 301, and a warning unit 501, and further include other units as necessary.

-Brake ECU, Steering ECU, Accelerator ECU, Sensor ECU-
The brake ECU 101 is a unit used for acquiring brake operation information.
The steering ECU 102 is a unit used for acquiring steering operation information.
The accelerator ECU 103 is a unit used for acquiring accelerator operation information.
The sensor ECU 104 is a unit used for acquiring sensor information such as an acceleration sensor.

  Information obtained from the brake ECU 101, the steering ECU 102, the accelerator ECU 103, and the sensor ECU 104 is collected as vehicle data via the vehicle control ECU 401, and a sudden braking tendency, a change in steering operation amount, a sudden acceleration tendency, It is a change in the amount of vibration, and it can be determined that a dangerous state has occurred in the driver based on such information.

The first vital sensor 201 is used to acquire biological data such as pulse, breath, heartbeat, brain wave, blood pressure, sweating, and body movement, and detects, for example, the occurrence of sleepiness during driving from a sleepiness detection algorithm.
Moreover, you may prepare multiple vital sensors used for biometric data acquisition like the 2nd vital sensor 202 according to the biometric information to acquire.
As the drowsiness detection algorithm, for example, an algorithm described in JP-A-2015-21703 can be used.
Information obtained from the vital sensor is collected as biometric data via the vehicle control ECU 401, and it is possible to determine that a dangerous state has occurred in the driver from the occurrence of drowsiness.

The first camera 301 captures the driving state of the driver (for example, facial expression during driving, body movement, facial movement, eye movement, etc.), and for example, from the side-by-side detection algorithm, Is detected.
As the aside look detection algorithm, for example, those described in Japanese Patent Application Laid-Open Nos. 2005-182452 and 2004-133749 can be used.
There is no restriction | limiting in particular as said 1st camera 301, According to the objective, it can select suitably, For example, a monocular camera, a stereo camera, etc. are mentioned, A commercial item can be used.
In addition, a plurality of cameras used for imaging may be prepared like the second camera 302 according to the driver who performs imaging.
Information obtained from the camera is collected as a camera-captured image of the driver via the vehicle control ECU 401, and it can be determined that a dangerous state has occurred in the driver from the occurrence of looking aside. Further, the driver's image captured by the first camera 301 is also stored as moving image data in the data storage device 403 of the vehicle control ECU 401, and is used for detection of a sign action indicating a sign that the driver shifts to a dangerous state. It is done.

The vehicle control ECU 401 includes a CPU (Central Processing Unit) 402 used for processing and a data storage device 403 that stores data.
The CPU 402 includes a dangerous state detection unit 10, a symptom operation detection unit 16, a processing unit 11, and the like in the functional blocks of the dangerous driving prediction apparatus illustrated in FIGS. 4A, 5A, and 6.
The data storage device 403 includes a recording unit in the functional block of the dangerous driving prediction device shown in FIGS. 4A, 5A, and 6. Examples of the recording unit include a moving image data storage unit 13, a dangerous state moving image data storage unit 14, and a determination data storage unit 15.
The recording unit corresponds to the dangerous state prediction data acquisition apparatus of the present invention. When the recording unit is executed, the dangerous state prediction data acquisition method of the present invention and the dangerous state prediction data acquisition program of the present invention are also executed. Is done.

The CPU 2 of the vehicle control ECU 401 determines that the driver has shifted to a dangerous state from the vehicle data, the biological data, and the camera captured image, and from the moving image data of the driver stored in the data storage device 403 in advance. Extract actions taken by the driver that match the signs action.
Further, the CPU 2 monitors the driver's moving image data imaged by the first camera 301 or the like, and if an operation performed by the driver that coincides with the sign operation is detected, there is a possibility that a dangerous state occurs in the driver. It is determined that the value is high, and the driver is warned through the warning unit 501 that a dangerous state has occurred.

  The data storage device 403 of the vehicle control ECU 401 is not particularly limited as long as various kinds of information can be stored, and can be appropriately selected according to the purpose. For example, a solid state drive, a hard disk drive, a CD (Compact Disc) drive , DVD (Digital Versatile Disc) drive, BD (Blu-ray (registered trademark) Disc) drive, and the like. The data storage device 403 may be a part of a cloud that is a computer group on the network.

  The warning by the warning unit 501 includes, for example, tactile notification using a vibrator, notification as sound using a speaker, and visual notification using a display device such as a display. Further, the warning includes warning the outside of the vehicle according to the degree of the dangerous state, and then reducing the speed of the vehicle or forcibly stopping the vehicle. Further, in the case of a vehicle with an automatic driving function, switching from manual driving to automatic driving is also included.

  FIG. 2 shows a functional block diagram of the dangerous driving prediction apparatus. The function of the dangerous driving prediction apparatus in FIG. 2 is divided into three processes: process A, process B, and process C.

The process A is a dangerous state detection process and is performed by the dangerous state detection unit 10 shown in FIG. 4A.
In the dangerous state detection process (Process A), the driver is constantly operating while driving the vehicle, the driver's driving operation change (change in vehicle data), the driver's biometric information change (vital) A change in data) and a change in the driving state of the driver (a change in the image captured by the camera). Detect if not.
The determination threshold value is obtained in advance and may be a fixed value, but it is desirable that the determination threshold value be appropriately adjusted in accordance with the traveling environment.
When it is determined that a dangerous state has occurred in the driver, the dangerous state detection unit 10 warns the driver through the warning unit 501, and the moving image data of the driver a predetermined time before the dangerous state has occurred. And the dangerous state are stored in association with each other and used for the determination data extraction process (process B).
The predetermined time is not particularly limited and may be appropriately selected depending on the intended purpose, but is preferably 1 minute or more, and more preferably 1 minute to 5 minutes.

The process B is a process for extracting determination data, and is performed by the recording unit shown in FIG. 5A. The determination data extraction process corresponds to the dangerous state prediction data acquisition method of the present invention, and the recording unit corresponds to the dangerous state prediction data acquisition device of the present invention.
The determination data extraction process (process B) is a process that operates at predetermined intervals. The determination data (in the detection process of the symptom movement of process C) is detected from the moving image data of the driver in response to an instruction from process A ( This is a process of extracting the sign operation). Note that this process does not necessarily have to be performed during operation, and the process B stops when the determination data extraction process is completed.
Note that the determination data is an operation image of the driver that coincides with a sign operation indicating a sign that the driver shifts to a dangerous state.

Process C is a sign action detection process, and is performed by the sign action detection unit 16 shown in FIG.
The sign action detection process (process C) is always in progress while the driver is driving the vehicle. The driver uses the camera-captured image and the determination data (sign sign action) extracted in process B. It is monitored whether there is any action performed by the driver that coincides with the sign action indicating the sign of transition to a dangerous state during driving. When the sign action detection unit 16 detects an action performed by the driver that matches the sign action from the camera-captured image, the driver is warned through the warning part 501 that a dangerous state is likely to occur. To do.

Here, FIG. 3 shows an example of an assumed model of the dangerous state of the driver of the driving vehicle. In FIG. 3, a dangerous state of “occurrence of drowsiness” (such a driver's dangerous state is sometimes referred to as “drowsiness driving”) is defined. A case will be described in which a symptom operation that “the movement of the line of sight stops” occurs before time. It should be noted that the danger state of “sleepiness” includes “decrease in attention”.
Functional blocks for each process of the dangerous state detection process (Process A), the determination data extraction process (Process B), and the sign action detection process (Process C) using the assumed model of FIG. It shows in FIG. 5A and FIG.

  As shown in FIG. 4A, in the dangerous state detection process (Processing A), the dangerous state detection unit 10 always detects at least one of the driver's vehicle data, vital data, and a camera image, Based on these data, it is monitored whether or not the driver is in danger of “sleepiness” (Processing A-1).

Here, it is determined that a dangerous state of “sleepiness” has occurred in the driver based on a predetermined action of the driver, a change in the driving operation of the driver (change in vehicle data), a biological condition of the driver. Whether the change corresponds to an operation that exceeds any of the change threshold of the danger state of the change of information (change of vital data) and the change of the driving state of the driver (change of the captured image of the camera) of the driver. Depending on what is done.
Since the determination threshold value of the driver's dangerous state is different for each driver, it is preferable that the determination threshold value is obtained in advance for each driver and registered in the determination threshold value database of the dangerous state detection unit.
Specific examples of a method for determining that a dangerous state of “sleepiness” has occurred in the driver includes the following.
In the change in the steering wheel operation amount that is vehicle data, it is determined that the dangerous state has occurred because the change exceeds the dangerous state determination threshold.
In the change of the fluctuation component of the pulse wave obtained from the vital sensor, it is determined that the dangerous state has occurred due to the change exceeding the dangerous state determination threshold.
In the change in the operation in which the driver closes his eyes for several seconds or more from the image of the driver, it is determined that the dangerous state has occurred due to the change exceeding the dangerous state determination threshold.

FIG. 4B is a diagram illustrating an example of a determination threshold value table for a dangerous state. In the table of FIG. 4B, “detection data”, “determination threshold”, and “dangerous state” are stored in association with each other.
In the present embodiment, “detection data” is “vehicle data”, “vital data”, or “camera captured image”. These contents are as described above.
The “determination threshold” indicates a determination threshold corresponding to the detection device, and the determination threshold is obtained in advance. The determination threshold may be a fixed value, but is preferably adjusted according to the driving environment or the like.
The “dangerous state” is “decrease in attention” or “occurrence of drowsiness” in this embodiment.

When the processing unit 11 receives a notification from the dangerous state detection unit 10 that a dangerous state of “sleepiness” has occurred to the driver (processing A-2), the processing unit 11 generates a dangerous state for the driver. The warning unit 501 is instructed to warn that it has been performed (Processing A-3), and the driver's moving image data stored in the moving image data storage unit 13 has a predetermined danger state of “sleepiness” has occurred. The driver's moving image data before the time is instructed to be stored in the dangerous state moving image data storage unit 14 in association with the dangerous state “sleepiness” (process A-4).
The predetermined time is not particularly limited and may be appropriately selected depending on the intended purpose, but is preferably 1 minute or more, and more preferably 1 minute to 5 minutes.
Note that the moving image data of the driver photographed by the camera is stored in the moving image data storage unit 13 for a certain period, but is overwritten with new moving image data after the certain period.

Here, FIG. 4C shows an example of the moving image data stored in the dangerous state moving image data storage unit 14.
In the table of FIG. 4C, “time information”, “detected dangerous state”, and “moving image data” are stored in association with each other.
“Time information” is information on the date and time when the dangerous state occurred in the driver.
In the present embodiment, the “detected dangerous state” is “decrease in attention” or “occurrence of drowsiness”.
“Movie data” is associated with “dangerous state”.

As shown in FIG. 5A, in the determination data extraction process (process B), the processing unit 11 says that “the movement of the line of sight stops” for the moving image data stored in the dangerous state moving image data storage unit 14 at regular intervals. Whether or not there is a sign operation is detected using a motion detection algorithm or the like (Process B-1).
As the motion detection algorithm, for example, an algorithm described in JP-A-10-304384 can be used.

Detection of the symptom movement “the movement of the line of sight is stopped” from the moving picture data of the driver stored in the dangerous state moving picture data storage unit 14 is performed using, for example, the determination reference table of the symptom movement shown in FIG. 5B.
In the table of FIG. 5B, “indication action”, “determination criterion”, and “remarks” are stored.
"Signal motion" includes detection as a change in driving state such as a change in driver's driving posture and a change in line of sight, and detection as a change in the number of occurrences during driving such as yawning and re-sitting. There is. In order to easily detect the sign operation, the sign operation may be prepared in advance.
In this embodiment, the “determination criterion” is defined by “occurrence of sign operation / state” or “number of occurrences of sign operation”.
In the present embodiment, “remarks” stores the target moving image data in the determination data storage unit 15 when the determination criterion is “occurrence of sign operation / state”. When the determination criterion is “number of occurrences of sign operation”, the determination data storage unit 15 stores that the corresponding sign operation has occurred, not the moving image data.

When the processing unit 11 determines that there is an indication motion “stops line-of-sight movement” from the moving image data of the driver stored in the dangerous state moving image data storage unit 14, this indication operation data is stored in the determination data storage unit 15. Store (process B-2). Note that the stored symptom motion data is obtained by extracting only the symptom motion image from the moving image data.
Here, as shown in FIG. 5C, when the detected dangerous state is “occurrence of drowsiness”, “data A”, “data B”, and “data” stored in the dangerous state moving image data storage unit 14 Among “C”, “data A” and “data B” corresponding to the symptom action of “the movement of the line of sight” is stored in the determination data storage unit 15.

As shown in FIG. 6, in the sign motion detection process, the sign motion detection unit 16 agrees with the sign motion that “the movement of the line of sight stops” stored in the determination data storage unit 15 with respect to the image captured by the driver. It is monitored whether there is an action performed by the driver who performs the process (Process C-1).
When the sign action detection unit 16 detects an action performed by the driver that matches the sign action “stops the movement of the line of sight” using a motion detection algorithm or the like for the camera captured image of the driver, the processing part 11 Notify Then, the processing unit 11 instructs the warning unit 501 to warn the driver that there is a high possibility that the dangerous state “sleepiness” will occur (processing C-2).
When warning, the warning level may be changed according to the number of occurrences of the symptom action. For example, the warning level for the occurrence of yawning may be set higher than the warning level for the occurrence of yawning.
Examples of warning levels include changing the color, changing the sound, and changing the magnitude of vibration.

  As described above, the sign action detection unit detects an action performed by the driver that coincides with the sign action indicating that the movement of the line of sight stops indicating the sign of transition to a dangerous state of `` sleepiness ''. It is possible to warn a person in advance that there is a high possibility that a dangerous state “sleepiness” will occur, and it is possible to prevent a dangerous state “sleepiness” from occurring. In addition, since the sign action detection unit can detect the sign action for each driver, it can accurately detect the dangerous state according to the feature of each driver.

  FIG. 7 is a flowchart illustrating an example of the flow of a dangerous state detection process. Hereinafter, with reference to FIG. 7 and FIGS. 4A to 4C, the flow of the detection process of the dangerous state “sleepiness” will be described.

In step S101, the dangerous state detection unit 10 determines whether or not a dangerous state of “sleepiness” has occurred while the driver is driving the vehicle with at least one of vehicle data, vital data, and a camera image. These data are monitored based on a threshold value for determining a dangerous state.
If the dangerous state detection unit 10 determines that the dangerous state “occurrence of drowsiness” has not occurred, the process returns to S101. If determined that the dangerous state “occurrence of sleepiness” has occurred, the process proceeds to S102. Transition.

  In step S <b> 102, when the processing unit 11 receives a notification from the dangerous state detection unit 10 that the dangerous state “occurrence of drowsiness” has occurred, the driver 11 indicates that the dangerous state “occurrence of drowsiness” has occurred. If the warning unit 501 is instructed to warn, the process proceeds to S103.

In step S <b> 103, the processing unit 11 obtains, from the driver's moving image data stored in the moving image data storage unit 13, the driver's moving image data of a predetermined time before the danger state “occurrence of drowsiness” occurs in the driver “ When it is instructed to be stored in the dangerous state moving image data storage unit 14 in association with the dangerous state “sleepiness”, this process is terminated.
The predetermined time is not particularly limited and may be appropriately selected depending on the intended purpose, but is preferably 1 minute or more, and more preferably 1 minute to 5 minutes.

  FIG. 8 is a flowchart illustrating an example of the flow of the determination data extraction process. Hereinafter, the flow of the determination data extraction process will be described with reference to FIG. 8 and FIGS. 5A to 5C.

  In step S <b> 201, the processing unit 11 performs an indication operation “stops line-of-sight movement” from the moving image data of the driver stored in the dangerous state moving image data storage unit 14 with respect to the dangerous state moving image data storage unit 14. When a process for extracting the presence is instructed, the process proceeds to S202.

  In step S202, when the processing unit 11 determines that there is no indication motion “stops the movement of the line of sight” in the moving image data stored in the dangerous state moving image data storage unit 14, the process returns to S201, If it is determined that there is a symptom action “stops”, the process proceeds to S203.

  In step S203, when the processing unit 11 determines that there is an indication operation that “the movement of the line of sight is stopped”, the processing unit 11 associates the indication operation that “the movement of the line of sight” stops with a danger state “the occurrence of drowsiness”. When the data is stored in 15, the process is terminated.

  FIG. 9 is a flowchart illustrating an example of the flow of detection processing of the sign operation. Hereinafter, with reference to FIG. 9 and FIG. 6, a description will be given of the flow of the detection process of the symptom action that the driver's “gaze movement stops”.

  In step S <b> 301, the sign action detection unit 16 has an action performed by the driver that matches the sign action “stops line-of-sight movement” stored in the determination data storage unit 15 in the driver's camera image. If it is monitored, the process proceeds to S302.

  In step S302, when the sign action detection unit 16 determines that the driver's camera captured image does not include an action performed by the driver that matches the sign action indicating that “the movement of the line of sight stops”, the process returns to S301. If an operation performed by the driver that coincides with the symptom operation that the movement of the line of sight stops is detected, the process proceeds to S303.

  In step S303, when the processing unit 11 receives a notification from the symptom motion detection unit 16 that there is a high possibility that a dangerous state of “sleepiness” will occur, a dangerous state of “sleepiness” may occur. If the warning unit 501 is instructed to warn the driver that the vehicle is highly likely, the process is terminated.

As described above, in the first embodiment, as shown in FIG. 10, there are four occurrences of “decrease in attentiveness”, which is a dangerous state, whereas the driver agrees with the symptom action that “the movement of the line of sight stops”. Since it was possible to detect the action performed by 2 out of 4 cases, the coincidence rate of both was 50%. Note that the warning level can be appropriately changed according to the match rate and the like.
Therefore, by monitoring the driver who is driving and detecting the action performed by the driver that matches the symptom action that `` the movement of the line of sight stops '', it is warned that the driver is likely to be in a dangerous state Therefore, it has been found that the occurrence of a dangerous state of “sleepiness” can be predicted in advance, and the occurrence of the dangerous state can be prevented in advance.

(Example 2)
In the second embodiment, the dangerous state is detected in the same manner as in the first embodiment, except that the symptom movement that “the movement of the line of sight stops” is replaced with the symptom movement that the driving posture is tilted. Therefore, the description of the same contents as those in the first embodiment is omitted.
In Example 2, as shown in FIG. 11, there were four occurrences of “decrease in attentiveness” which is a dangerous state, whereas the driver who coincided with the symptom action “driving posture tilted” was performed. Since two out of four cases were able to detect an action, the agreement rate between the two was 50%. Note that the warning level can be appropriately changed according to the match rate and the like.
Therefore, by monitoring the driver who is driving and detecting the action performed by the driver that coincides with the symptom action that the driving attitude is tilted, it is warned that the driver is likely to be in a dangerous state Therefore, it has been found that the occurrence of a dangerous state of “sleepiness” can be predicted in advance, and the occurrence of the dangerous state can be prevented in advance.

Regarding the above embodiment, the following additional notes are disclosed.
(Appendix 1)
A dangerous state prediction device for predicting that a dangerous state will occur in a monitored person,
A sign action detecting unit that monitors the action of the monitored person and detects an action performed by the monitored person that coincides with a sign action indicating a sign that the monitored person enters a dangerous state. Hazardous state prediction device.
(Appendix 2)
When detecting that a dangerous state has occurred in the monitored person from a predetermined action of the monitored person,
A recording unit that extracts the sign operation from the operation of the monitored person a predetermined time before the dangerous state occurs and records it in a determination data storage unit;
The danger state prediction device according to supplementary note 1, wherein the sign action detection unit detects an action performed by the monitored person that matches the sign action recorded in the determination data storage part.
(Appendix 3)
3. The supplementary note 1 or 2 according to any one of appendices 1 to 2, further comprising: a warning unit that warns the monitored person that a dangerous state is likely to occur when the sign operation detection unit detects an operation that matches the sign operation. Hazardous state prediction device.
(Appendix 4)
The monitored person is at least one of a vehicle driver, an elderly person living alone, a certified care recipient, a worker at a construction work site, a patient who suffers a disturbance of consciousness due to a seizure, and a patient who suffers a movement disorder due to a seizure. The dangerous state prediction apparatus according to any one of appendices 1 to 3.
(Appendix 5)
The dangerous state is a state of reduced attention to the monitored person, a state of reduced cognitive ability of the monitored person, a state of drowsiness of the monitored person, a state of jealousy of the monitored person, and an awareness by the seizure of the monitored person The dangerous state prediction apparatus according to any one of supplementary notes 1 to 4, which is at least one of a failure state and a movement disorder state due to the seizure of the monitored person.
(Appendix 6)
The symptom movement includes a change in the movement of the head of the monitored person, a change in facial expression of the monitored person, a change in the movement of the line of sight of the monitored person, the number of times the head of the monitored person is shaken, The number of times the monitor's shoulders move up and down, the number of blinks of the monitored person, the number of rubs the eyes of the monitored person, the number of reaching of the person to be monitored, the number of yawns of the monitored person, and The dangerous state prediction apparatus according to any one of appendices 1 to 5, including at least one of the number of times the monitored person sits down.
(Appendix 7)
The predetermined operation of the monitored person is at least one of a change in the operation of the monitored person, a change in the biological information of the monitored person, and a change in the behavior state of the monitored person, and these changes The dangerous state predicting device according to any one of supplementary notes 2 to 6, wherein the dangerous state is an operation that exceeds a threshold value for determining a dangerous state.
(Appendix 8)
The dangerous state prediction apparatus according to any one of appendices 1 to 7, wherein the monitored person is a vehicle driver.
(Appendix 9)
The danger state prediction device according to appendix 8, wherein the driver's danger state is at least one of the driver's attention reduction state, the driver's drowsiness occurrence state, and the driver's drowsiness state.
(Appendix 10)
The dangerous state prediction device according to any one of supplementary notes 8 to 9, wherein the dangerous state of the driver is any one of drowsy driving, side-viewing driving, random driving, and drunk driving.
(Appendix 11)
The dangerous state prediction apparatus according to any one of appendices 3 to 10, wherein the warning unit is at least one of a vibrator, a speaker, a light, a display, a speed reduction unit, and an automatic operation switching unit.
(Appendix 12)
A dangerous state prediction method for predicting that a dangerous state will occur in a monitored person,
A sign action detecting step of monitoring the action of the monitored person and detecting an action performed by the monitored person that coincides with a sign action indicating a sign that the monitored person enters a dangerous state. Hazardous state prediction method.
(Appendix 13)
When detecting that a dangerous state has occurred in the monitored person from a predetermined action of the monitored person,
Including a recording step of extracting the symptom movement from the movement of the monitored person a predetermined time before the dangerous state occurs and recording it in the determination data storage unit,
The dangerous state prediction method according to appendix 12, wherein the sign action detection step detects an action performed by the monitored person that matches the sign action recorded in the determination data storage unit.
(Appendix 14)
14. The appendix 12 to 13, further comprising a warning step that warns the monitored person that a dangerous state is likely to occur when an operation that matches the sign operation is detected by the sign operation detection step. Hazardous state prediction method.
(Appendix 15)
A dangerous state prediction program for predicting that a dangerous state will occur in a monitored person,
Monitoring the operation of the monitored person, and causing the computer to execute a process of detecting an operation performed by the monitored person that coincides with a sign operation indicating a sign that the monitored person enters a dangerous state. Hazardous condition prediction program.
(Appendix 16)
When detecting that a dangerous state has occurred in the monitored person from a predetermined action of the monitored person,
Extracting the symptom action from the action of the monitored person a predetermined time before the dangerous state occurred, and recording it in the determination data storage unit,
The dangerous state prediction program according to appendix 15, which causes a computer to execute a process of detecting an action performed by the monitored person that matches the sign action recorded in the determination data storage unit.
(Appendix 17)
The dangerous state prediction program according to any one of supplementary notes 15 to 16, which causes the computer to execute a process of warning the monitored person that a dangerous state is likely to occur when an operation that matches the sign operation is detected.
(Appendix 18)
A dangerous state prediction data acquisition device for acquiring data for predicting that a dangerous state will occur in a monitored person,
When the operation of the monitored person is monitored and it is detected from the predetermined operation of the monitored person that a dangerous state has occurred in the monitored person, the operation of the monitored person before the predetermined time when the dangerous state occurs The data acquisition device for risk condition prediction, further comprising: a recording unit that extracts and records in the determination data storage unit a sign action indicating a sign that the monitored person shifts to a risk condition.
(Appendix 19)
A method for acquiring data for predicting a dangerous state for acquiring data for predicting that a dangerous state will occur in a monitored person,
When the operation of the monitored person is monitored and it is detected from the predetermined operation of the monitored person that a dangerous state has occurred in the monitored person, the operation of the monitored person before the predetermined time when the dangerous state occurs And a recording step of extracting a symptom operation indicating a sign that the monitored person enters the dangerous state and recording it in the determination data storage unit.
(Appendix 20)
A dangerous state prediction data acquisition program for acquiring data for predicting that a dangerous state will occur in a monitored person,
When the operation of the monitored person is monitored and it is detected from the predetermined operation of the monitored person that a dangerous state has occurred in the monitored person, the operation of the monitored person before the predetermined time when the dangerous state occurs A risk condition prediction data acquisition program that causes a computer to execute a process of extracting a sign action indicating a sign that the monitored person enters a risk condition and recording the action in a determination data storage unit.

DESCRIPTION OF SYMBOLS 10 Danger state detection part 11 Processing part 13 Moving image data storage part 14 Danger state moving image data storage part 15 Judgment data storage part 16 Indication motion detection part 100 Danger state prediction apparatus (dangerous driving prediction apparatus)
401 Vehicle control electronic control unit (vehicle control ECU)
402 CPU
403 Data storage device 501 Warning section

Claims (12)

A dangerous state prediction device for predicting that a dangerous state will occur in a monitored person,
A sign action detecting unit that monitors the action of the monitored person and detects an action performed by the monitored person that coincides with a sign action indicating a sign that the monitored person enters a dangerous state. Hazardous state prediction device. When detecting that a dangerous state has occurred in the monitored person from a predetermined action of the monitored person,
A recording unit that extracts the sign operation from the operation of the monitored person a predetermined time before the dangerous state occurs and records it in a determination data storage unit;
The danger state prediction apparatus according to claim 1, wherein the sign action detection unit detects an action performed by the monitored person that matches the sign action recorded in the determination data storage unit.   3. The warning unit according to claim 1, further comprising: a warning unit that warns the monitored person that a danger state is likely to occur when the sign operation detection unit detects an operation that matches the sign operation. Danger state prediction device.   The dangerous state is a state of reduced attention to the monitored person, a state of reduced cognitive ability of the monitored person, a state of drowsiness of the monitored person, a state of jealousy of the monitored person, and an awareness by the seizure of the monitored person The danger state prediction device according to any one of claims 1 to 3, wherein the device is at least one of a failure state and a movement disorder state caused by the seizure of the monitored person.   The symptom movement includes a change in the movement of the head of the monitored person, a change in facial expression of the monitored person, a change in the movement of the line of sight of the monitored person, the number of times the head of the monitored person is shaken, The number of times the monitor's shoulders move up and down, the number of blinks of the monitored person, the number of rubs the eyes of the monitored person, the number of reaching of the person to be monitored, the number of yawns of the monitored person, and The dangerous state prediction apparatus according to any one of claims 1 to 4, comprising at least one of the number of times the monitored person sits down.   The predetermined operation of the monitored person is at least one of a change in the operation of the monitored person, a change in the biological information of the monitored person, and a change in the behavior state of the monitored person, and these changes The dangerous state prediction apparatus according to claim 2, wherein the operation is an operation that exceeds a threshold value for determining a dangerous state.   The danger according to any one of claims 1 to 6, wherein the dangerous state is at least one of a state of reduced attention to the monitored person, a state of drowsiness of the monitored person, and a state of jealousy of the monitored person. State prediction device. A dangerous state prediction method for predicting that a dangerous state will occur in a monitored person,
A sign action detecting step of monitoring the action of the monitored person and detecting an action performed by the monitored person that coincides with a sign action indicating a sign that the monitored person enters a dangerous state. Hazardous state prediction method. A dangerous state prediction program for predicting that a dangerous state will occur in a monitored person,
Monitoring the operation of the monitored person, and causing the computer to execute a process of detecting an operation performed by the monitored person that coincides with a sign operation indicating a sign that the monitored person enters a dangerous state. Hazardous condition prediction program. A dangerous state prediction data acquisition device for acquiring data for predicting that a dangerous state will occur in a monitored person,
When the operation of the monitored person is monitored and it is detected from the predetermined operation of the monitored person that a dangerous state has occurred in the monitored person, the operation of the monitored person before the predetermined time when the dangerous state occurs The data acquisition device for risk condition prediction, further comprising: a recording unit that extracts and records in the determination data storage unit a sign action indicating a sign that the monitored person shifts to a risk condition. A method for acquiring data for predicting a dangerous state for acquiring data for predicting that a dangerous state will occur in a monitored person,
When the operation of the monitored person is monitored and it is detected from the predetermined operation of the monitored person that a dangerous state has occurred in the monitored person, the operation of the monitored person before the predetermined time when the dangerous state occurs And a recording step of extracting a symptom operation indicating a sign that the monitored person enters the dangerous state and recording it in the determination data storage unit. A dangerous state prediction data acquisition program for acquiring data for predicting that a dangerous state will occur in a monitored person,
When the operation of the monitored person is monitored and it is detected from the predetermined operation of the monitored person that a dangerous state has occurred in the monitored person, the operation of the monitored person before the predetermined time when the dangerous state occurs A risk condition prediction data acquisition program that causes a computer to execute a process of extracting a sign action indicating a sign that the monitored person enters a risk condition and recording the action in a determination data storage unit.