WO2023204218A1

WO2023204218A1 - Cognitive ability estimation apparatus and program

Info

Publication number: WO2023204218A1
Application number: PCT/JP2023/015517
Authority: WO
Inventors: 幸男篠崎
Original assignee: 幸男篠崎
Priority date: 2022-04-18
Filing date: 2023-04-18
Publication date: 2023-10-26
Also published as: TW202404532A

Abstract

A cognitive ability estimation apparatus including: a biological information acquisition means that acquires biological information from which at least a heartbeat of a subject who performs driving or operation of a moving body can be specified; a video information acquisition means that acquires video information of the subject; a state estimation means that estimates the state of an autonomous nerve of the subject on the basis of the heartbeat to be specified from the biological information; and a cognitive ability estimation means that estimates an cognitive ability of the subject on the basis of an estimation result of the autonomous nerve by the state estimation means and the video information.

Description

Cognitive ability estimation device and program

The present invention relates to a cognitive ability estimation device and a program.

There are various types of moving objects that are driven or operated by people. Such moving objects are heavy, and if an accident occurs due to inappropriate driving or maneuvering, the damage is likely to be severe. For example, in a car, the driver's cognitive ability is estimated, and based on the estimation results, the driver is stimulated as necessary to support safe driving. This has also been done (for example, see Patent Document 1). Here, for the sake of explanation and convenience, the moving object will be referred to as a car, the subject will be referred to as a driver, and the operation for moving the moving object will be referred to as driving.

The driver's cognitive ability can be estimated using video information from a camera that captures an image of the driver sitting in the driver's seat (for example, see Patent Document 1). However, changes in the driver's visually verifiable characteristic values, such as the frequency of blinking, the range of movement of the line of sight, and the speed of movement thereof, appear as a result of drowsiness. Therefore, there is a time lag between when sleepiness actually occurs and when a change in the feature amount is confirmed. Because of the existence of such a time difference, sleepiness has also been estimated from the state of autonomic nerves (for example, see Patent Document 2).

JP 2019-200544 Publication Japanese Patent Application Publication No. 2009-039167 Japanese Patent Application Publication No. 2008-125801

Conventionally, a driver who is estimated to be drowsy is given some kind of stimulation using an on-vehicle device to improve his or her drowsiness (level of alertness). Stimuli include, for example, blowing air from an air conditioner and sound warnings.
Drivers who notice their own drowsiness usually consciously try to improve their drowsiness. Due to this awareness, the activity of the sympathetic nerves may increase even if the person is sleepy. That is, an abnormal state may occur in which both the sympathetic nerve and the parasympathetic nerve are activated (for example, see Patent Document 3).

Because drivers must drive cars safely, it is thought that their autonomic nerves are relatively likely to become abnormal. For this reason, it can be said that it is not always possible to estimate a driver's drowsiness, or cognitive ability, with a high degree of accuracy based on the state of the autonomic nervous system. It seems important to be able to estimate cognitive ability with higher accuracy.

An object of the present invention is to provide a cognitive ability estimation device that can more accurately estimate the cognitive ability of a subject who drives or controls a mobile object.

A cognitive ability estimation device according to an aspect of the present disclosure includes a biological information acquisition unit that acquires biological information that can identify at least the heartbeat of a subject who drives or steers a mobile object, and an image that acquires video information of the subject. an information acquisition means, a state estimation means for estimating the state of the autonomic nerves of the subject based on the heartbeat identified from the biological information, an estimation result of the state of the autonomic nerves by the state estimation means, and based on the video information, A cognitive ability estimation means for estimating the cognitive ability of a subject.

According to the present invention, the cognitive ability of a subject who drives or controls a mobile object can be estimated with higher accuracy.

FIG. 2 is a diagram illustrating an example of a mechanism in which a cognitive ability estimating device according to an embodiment of the present invention estimates a subject's cognitive ability, and an example of control performed according to the estimated cognitive ability. FIG. 1 is a block diagram showing an example of the hardware configuration of a cognitive ability estimation device according to an embodiment of the present invention. FIG. 1 is a functional block diagram showing an example of a functional configuration implemented on a cognitive ability estimation device according to an embodiment of the present invention. It is a flow chart which shows an example of safe driving support processing. 3 is a flowchart illustrating an example of safe driving support processing (continued).

Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. Note that the embodiment described below is an example, and the technical scope of the present invention is not limited thereto. The technical scope of the present invention also includes various modifications.

FIG. 1 is a diagram illustrating an example of a mechanism in which a cognitive ability estimating device according to an embodiment of the present invention estimates a subject's cognitive ability, and an example of control performed according to the estimated cognitive ability.
The target person whose cognitive ability is estimated by the cognitive ability estimating device 1 is a person who drives or operates a mobile object. Therefore, a moving body is equipped with a power source such as an engine that enables movement.

In FIG. 1, it is assumed that the moving object is a car, and the subject is a driver 3 who is sitting in the driver's seat 2 of the car. Therefore, unless otherwise specified, the description will be made assuming that the moving object is a car and the subject is the driver 3. Note that the moving object may be a train, a ship, an airplane, or the like. The target person may be a pilot or the like.

In this embodiment, biological information, video information, and state information are used to estimate the cognitive ability of the driver 3. Among these pieces of information, state information can be omitted.
The biological information is information representing the heartbeat of the driver 3, or information that allows the heartbeat to be specified. FIG. 1 shows a steering wheel 5 equipped with a pulse sensor and a smart watch 6 worn on the arm of a driver 3 as examples of devices that generate biological information.

The pulse sensor is, for example, a capacitance type, an optical type, or a radio wave type, and processes a signal output from the sensor for sensing to identify a heartbeat from the pulsation of blood flow. The identification result is transmitted from the pulse sensor as biological information. This biological information is input to the cognitive ability estimation device 1 via, for example, an ECU (Electronic Control Unit). Note that the pulse sensor may be provided not on the steering wheel 5 but on the driver's seat 2. The type of sensor, installation location, etc. are not particularly limited as long as the heartbeat can be identified.

The smart watch 6 is equipped with, for example, an optical pulse sensor. The smart watch 6 can detect the pulsation of the blood flow of the driver 3 confirmed by its pulse sensor as a heartbeat, and can transmit the detection result as biological information. The biological information wirelessly transmitted from the smart watch 6 is directly received by the cognitive ability estimation device 1, for example.

The camera 4 is installed inside the car to take an image of the face of the driver 3 sitting in the driver's seat 2. The video information obtained by the imaging is transmitted to the cognitive ability estimating device 1 via the ECU, similarly to the pulse sensor provided on the steering wheel 5.
It is also possible to identify the heart rate of the driver 3 from the video information. It is also possible to identify breathing. Therefore, video information may be positioned as biological information.

The camera 7 and the radar 8 are devices that generate status information or generate information used to generate status information. This status information is information representing the status of the automobile, which is a moving object.
The camera 7 is used, for example, to capture an image in the direction in which the automobile is traveling. The video information obtained by the imaging is used, for example, to confirm the state of the road in the direction of travel of the vehicle. Specifically, the road conditions that are checked include the way the road curves, the presence or absence of lines drawn on the road, the presence or absence of obstacles, and the types of obstacles. All of these are identified by video analysis using video information. Therefore, the camera 7 is a device for generating state information.

The radar 8 is for measuring the distance to an object existing in the direction of travel, and for example, generates and outputs distance information representing the measured distance as state information. Therefore, the radar 8 is a device that can generate state information.
Note that the status information is not limited to what can be generated by the camera 7 or the radar 8. Measured position information, vehicle speed information, steering angle of the steering wheel 5, and the like can also be used as the status information. The state information to be used may be determined depending on the type of mobile object, its purpose, etc.

The distance information output from the radar 8 and the video information output from the camera 7 are processed by the corresponding ECU 9. The ECU 9 recognizes, for example, the presence or absence of a line drawn on the road, the type of line, the presence or absence of an object existing in the direction of travel, the type of object, etc. from the video information. Based on such recognition results, it is determined whether the vehicle body is swaying, whether the vehicle body is protruding from the line, etc. In addition, in combination with distance information, it also determines whether the distance between the vehicle and another vehicle traveling in front is appropriate, whether an obstacle exists, and whether it is possible to avoid the obstacle or not. . Thereby, the ECU 9 outputs, for example, video information, each recognition result, and each determination result to the cognitive ability estimation device 1 as state information. The state information (distance information) generated by the radar 8 is converted into a processing result using the state information.

The video information and status information received by the cognitive ability estimation device 1 are used for the video analysis process in step S1. This video analysis process is a process for evaluating multiple indicators that can be used to estimate cognitive ability. The types of multiple indicators and their combinations are not particularly limited, but it is necessary to use at least drowsiness. This is because sleepiness has a very large effect on cognitive ability. As other indicators, fatigue and attention may be used, for example, as in Patent Document 1. Here, it is assumed that the plurality of indicators are sleepiness, fatigue, and attentiveness.

As the evaluation method for each index, well-known methods can be adopted. Specifically, for example, the method described in Patent Document 1 may be adopted. For this reason, detailed explanation will be omitted here. Note that the drowsiness of the driver 3 is generally estimated by focusing on eye movements, specifically, blink frequency, eyeball movement, and the like. From the eye movements, it is possible to check the speed of line of sight movement, the range of line of sight movement, etc.

The faster the vehicle travels, the narrower the range of movement of the driver's 3 line of sight tends to be. Further, the driver 3 who is trying to avoid an obstacle ahead may visually move his/her line of sight significantly in order to confirm the direction in which the vehicle should proceed. When turning the car left or right, the driver 3 usually moves his/her line of sight relatively widely. For this reason, the state information is useful for estimating the cognitive ability of the driver 3 with higher accuracy.

The biological information received by the cognitive ability estimation device 1 is used in the autonomic nerve state analysis process in step S2. This state analysis process evaluates the state of the autonomic nerves, that is, the activity levels of the sympathetic and parasympathetic nerves, and uses the evaluation results to estimate the levels of drowsiness and stress.

It is known that a spectrum obtained by frequency analysis of heartbeat intervals has a single peak structure. It is said that the activity level can be relatively evaluated by comparing the power in the range of 0.05-0.15 Hz for the sympathetic nerve and 0.15-0.45 Hz for the parasympathetic nerve. Sleepiness and stress can be evaluated from their relative activity levels. Therefore, biological information that represents heartbeat or that can identify heartbeat can be used to estimate the state of autonomic nerves, that is, drowsiness and stress. Therefore, in this embodiment, biological information is information necessary for estimating the state of the autonomic nerve. Since a well-known method may be adopted as a method for estimating the state of the autonomic nervous system from the heartbeat, a more detailed explanation will be omitted.

The structure that exists in the range of 0.05-0.15Hz is called the blood pressure fluctuation component (NWSA: Myer Wave Sinus Arrhythmia), and the structure that exists in the range of 0.15-0.45Hz is called the blood pressure fluctuation component (NWSA). This is called the fluctuation component (RSA: Respiratory Sinus Arrhythmia). Information regarding respiration and blood pressure changes can also be obtained from the spectrum of heart rate fluctuations.

The states of the driver 3 estimated from the activity levels of the autonomic nerves, that is, the sympathetic nerves, and the parasympathetic nerves, are broadly classified into the following four types (see, for example, Patent Document 3).
(1) State in which sympathetic nerves are dominant (2) State in which parasympathetic nerves are dominant (3) State in which the activity levels of sympathetic nerves and parasympathetic nerves are both increased (4) Each activity of sympathetic nerves and parasympathetic nerves Both levels are decreasing

(1) is a state in which it is estimated that the driver 3 is not feeling sleepy. (1) also includes a state where the stress level is high, that is, a state where the driver 3 is excited. Hereinafter, this will be referred to as the first state.
(2) is a state in which the driver 3 is estimated to be drowsy or fatigued. If the driver 3 is very relaxed, this state may be assumed. Hereinafter, this will be referred to as the second state.
(3) is a state that appears when the driver 3 tries to overcome sleepiness. This is an abnormal condition that does not normally occur. The driver 3 who is aware of sleepiness is conscious of trying to at least suppress the sleepiness. Due to this awareness, it is considered that this condition is relatively likely to occur for driver 3. Hereinafter, this will be referred to as the third state.
(4) is a state that is likely to be assumed when the driver 3 is in a depressed state. It can be inferred that the state of the autonomic nervous system is unstable. Hereinafter, this will be referred to as the fourth state.

Even if the autonomic nervous system is roughly divided into four states in this way, it does not mean that the level of each state has been estimated. Furthermore, it is not always possible to estimate the level of cognitive ability with high accuracy.
In the first state, driver 3 is considered to have high cognitive ability. If the driver 3 is feeling strong stress, there is a possibility that his or her attentiveness is decreasing due to the stress. Decreased attentiveness refers to a state in which the actual cognitive ability is low because the driver is not paying attention to driving or is looking elsewhere. However, it is possible that even when under stress, people are able to control themselves and keep their cognitive abilities high. For these reasons, it is not always possible to estimate the level of cognitive ability appropriately.

In the second state, it is highly likely that the driver 3 has low cognitive ability due to drowsiness, fatigue, or the like. However, depending on the road on which the car is traveling or the situation at that time, there is a possibility that the person is simply relaxing. For this reason, even in the second state, there is a possibility that the driver 3 can drive the car safely.
The third state is a state seen when the driver 3 is trying to overcome sleepiness, as described above. Since the driver 3 is aware of his own drowsiness and is conscious of trying to overcome the drowsiness, it is possible that his cognitive ability itself has not deteriorated much.
Even in the fourth state, it is possible that the driver 3 appropriately recognizes the situation. Therefore, it is not always possible to estimate the level of cognitive ability with high accuracy.

When focusing on the state of autonomic nerves, changes in the level of cognitive ability can be estimated at earlier timing than video analysis (for example, Patent Document 3). However, it is possible that the level of cognitive ability cannot be estimated with high accuracy based on the state of the autonomic nervous system. For this reason, in this embodiment, each index evaluated from video information and the results obtained by autonomic nerve state analysis are used in a complementary manner to estimate the cognitive ability of the driver 3 with higher accuracy. I try to do that. In order to estimate complementary cognitive ability, the actual drowsiness level analysis process in step S3 and the stress influence analysis process in step S4 are executed by the cognitive ability estimation device 1.

The drowsiness that occurs in the driver 3 in the second state has a particularly serious effect on the decline in cognitive ability. This is because not only is sleepiness a major cause of cognitive decline, but sleepiness typically continues for a long period of time. In other words, sleepiness is likely to cause a state of extremely low cognitive ability to continue for a long period of time. For this reason, in this embodiment, the actual level of sleepiness estimated from the state of the autonomic nerves is evaluated as the actual sleepiness level by the actual sleepiness level analysis process in step S3. The driver 3 to be evaluated is the driver 3 whose autonomic nervous system is estimated to be in the second state. This actual drowsiness level analysis process uses video information and state information.

The driver 3 who is trying to overcome sleepiness, for example, moves his body intentionally within a range that does not interfere with safe driving and tries to provide stimulation to his body. Examples of such body movements include intentionally moving your head or shoulders to provide stimulation to your body, intentionally moving your eyes (blinking, etc.), operating air conditioning equipment, etc. Alternatively, you can make your body more susceptible to sound stimulation, open the windows and let outside air into the car, etc. Even if the driver 3 who moves in this manner feels sleepy, it can be estimated that the decrease in the level of cognitive ability is relatively small. In other words, the actual sleepiness level is relatively low, and it can be estimated that a sufficient cognitive level is maintained.

However, if such movements cannot be confirmed or cannot be confirmed at all, even if you are conscious of trying to overcome your sleepiness, that is, even if you can confirm that your sympathetic nervous system is activated, you are not taking any action to wake up your sleepiness. It turns out. Therefore, it can be estimated that the actual drowsiness level of driver 3 is high and that his cognitive ability level is not sufficient.

As mentioned above, the movements of the driver 3 are affected by the state of the vehicle. Furthermore, the movements of the driver 3 affect the state of the vehicle. For example, when the vehicle is traveling on a road with lines drawn on it, if the driver 3 does not operate the steering wheel 5 appropriately, there is a risk of the vehicle body protruding from the lines. There is also a risk that the vehicle will be driven at a speed that significantly exceeds the speed limit. Such information can also be used as state information to estimate the actual drowsiness level of the driver 3. The presence or absence of a line drawn on the road, its type, and whether the vehicle body protrudes from the line can all be confirmed using video information from the camera 7. When a sign or the like indicating a speed limit is imaged by the camera 7, the speed limit can be confirmed from the video information.

Additionally, there is a relationship between attentiveness, which is evaluated as an index, and sleepiness. For example, the driver 3, who is estimated to have low attentiveness due to frequently moving his line of sight over a relatively large area, is unlikely to be feeling sleepy. This is because such movements would not occur without brain function. For this reason, video information and state information are used in the actual drowsiness level analysis process.

In the above explanation, for convenience, the actual drowsiness level is divided into two levels: high and low. However, in reality, the actual sleepiness levels of each of these stages can be further divided into two or more stages.
At a high actual drowsiness level, that is, in a situation where the driver 3 is estimated to be feeling strongly drowsy, it is not necessary to divide into multiple stages in consideration of safety. That is, as a numerical value representing the actual sleepiness level, a numerical value representing that safe driving cannot be expected may be provided, and a level at which safe driving can be expected may be represented by two or more numerical values. Here, for the sake of explanation and convenience, it is assumed that the numerical value indicating that safe driving cannot be expected is 3, and that the estimation result of the actual drowsiness level is expressed as an integer between 0 and 3. The smaller the value, the lower the actual drowsiness level, that is, the less the driver 3 feels drowsy.

In the stress impact analysis process in step S4, the impact of stress felt by the driver 3 on himself or herself is evaluated. Video information and status information are used to evaluate this influence. The results of autonomic nerve state analysis processing are used to determine whether or not it is necessary to evaluate this influence. As a result, the only driver 3 whose influence is to be evaluated is the driver 3 who is in the first state. More specifically, a driver whose stress level evaluated in the autonomic nerve state analysis process in step S2 is equal to or higher than a set value, that is, a driver who is estimated to be feeling stress equal to or higher than the stress level that is considered to have a negative impact on safe driving. There are only 3.

In stress impact analysis processing, the impact of stress on attention is mainly evaluated. As a result, movements of the driver's 3 arms, upper body, head, etc. are not given importance unless they are at a level that adversely affects visual recognition in a direction that should be visually recognized or driving operation. This is because if the stress is not at that level, it is considered that stress does not actually affect the decline in attention, or that the degree of stress is relatively small. If the degree of influence is within a small range, it can be estimated that the driver 3 is in a state where he or she can drive safely.

On the other hand, when the driver 3 moves his or her arms, upper body, head, etc. violently or greatly, the driver 3 must properly direct his/her line of sight in the direction that should be seen, properly recognize the image that has entered his/her eyes, or change the driving direction that should be carried out. It becomes difficult to perform operations immediately and appropriately. It is conceivable that such movements are caused by the effects of stress manifesting in the movements of the driver's 3 body. If such actions can be confirmed repeatedly, it can be evaluated that the driver's 3's attentiveness is low. It is highly likely that driver 3 is unable to control his stress. Therefore, it can be estimated that the stress has such an effect on the driver 3 that he is unable to drive safely.

The degree of influence is also evaluated, for example, in multiple stages. In evaluating the degree of influence, as described above, it is also determined whether safe driving is possible. Accordingly, the degree of influence may also include, for example, a numerical value indicating that safe driving cannot be expected. In this case, the level at which safe driving can be expected may be expressed, for example, by two or more numerical values. Here, for the sake of explanation and convenience, it is assumed that the numerical value indicating that safe driving cannot be expected is 3, and that the evaluation result of the degree of influence is expressed as an integer between 0 and 3. The smaller the value, the lower the degree of influence, that is, the stress has virtually no effect on the driver 3.

As mentioned above, the movements of the driver 3 are affected by the state of the vehicle. For example, in a situation where the car is stopped, there are no restrictions on safe driving on the actions of the driver 3. The driver 3 can make arbitrary movements. Even when the automobile is running, the permissible actions of the driver 3 vary depending on the driving speed and the like. For this reason, video information and status information are also used in stress effect analysis processing.

The results of the stress influence analysis process and the actual drowsiness level analysis process are passed to the cognitive ability estimation process in step S5 and the device control process in step S6, respectively.
The cognitive ability estimation process is a process for estimating the cognitive ability of the driver 3 as well as changes in the cognitive ability. Cognitive ability is estimated using the actual sleep level, the evaluation results of each index except for sleepiness, and the degree of influence of stress. Again, for convenience, it is assumed that cognitive ability is evaluated on a four-level scale from 0 to 3, with 3 representing the worst condition. Under these assumptions, the smaller the number, the higher the cognitive ability. Since cognitive ability is estimated numerically, the target of estimation will also be referred to as "cognitive ability level" from now on.

As mentioned above, when the actual drowsiness level and the degree of influence are evaluated in four stages, and the value representing the worst condition in terms of safe driving is set to 3, if the actual drowsiness level or the degree of influence is 3, for example, The cognitive ability level may also be set to 3. If both the actual sleepiness level and the degree of influence are not 3, the cognitive ability level may be calculated, for example, using a calculation formula that uses the actual sleepiness level and the evaluation results of each index excluding sleepiness as variables. For example, each variable may be multiplied by a coefficient determined depending on the degree of influence on safe driving.

Changes in cognitive ability are evaluated using, for example, actual sleepiness level, degree of influence, stress level, and each index other than sleepiness. These are taken into consideration in controlling one or more devices that constitute the device group 20. For this reason, they will hereinafter be collectively referred to as "control indicators" to distinguish them from the above-mentioned indicators.

In FIG. 1, only an air conditioning equipment control device 21 and a steering control device 22 are shown as devices constituting the device group 20.
The air conditioner control device 21 is a device that controls air conditioners capable of blowing hot air and cold air, and is also capable of controlling the temperature of the air and the direction of air blowing. Through the control of the control device 21, the driver 3 can be stimulated by the wind. The control device 21 is shown in the device group 20 because it is a device necessary for controlling the air conditioning equipment.

The steering control device 22 is a device that performs control to generate a driving force that allows the steering wheel 5 to be turned more easily. It is also part of the equipment that makes autonomous driving possible.
The control device 22 can generate vibrations or the like in the hand holding the handle 5 by using the driving force. The driver 3 can be stimulated by controlling the control device 22 to generate vibrations or the like in the steering wheel 5. Like the control device 21, the control device 22 is also shown in the equipment group 20 because it is a device necessary for stimulation by the handle 5.

Note that the devices to be controlled are not limited to those shown in FIG. A navigation device that performs navigation by display and voice, interior lighting, power windows, etc. may also be controlled objects. The navigation device can be used for audio output or message display. In FIG. 1, the cognitive ability estimating device 1 is shown to give instructions directly to the equipment group 20. However, in reality, control is requested to the corresponding ECU. The reason why the ECU 9 is shown in FIG. 1 is because the ECU 9 generates status information.

In this embodiment, desirable stimuli are given to the driver 3 in a timely manner so that the driver 3 can drive safely. Stimuli that are considered desirable for the driver 3 may differ depending on the driver's 3 condition. The content of the information to be provided to the driver 3 also differs depending on the condition of the driver 3. For example, the driver 3 who has low attentiveness may be made more aware of his low attentiveness. Therefore, it is conceivable to convey the fact that the attention level is low through voice or the like. However, for the driver 3 who is feeling drowsy or fatigued, it is considered that providing such information through voice or the like is insufficient. It is also considered necessary to provide stimulation to the driver 3 to alleviate the level of sleepiness or fatigue that the driver 3 feels.

Note that if information is provided by voice or the like, information will also be provided to fellow passengers in the same vehicle. As a result, it can be expected that the fellow passenger will encourage the driver 3. For this reason, it is desirable to provide information in such a way that anyone other than the driver 3 can easily recognize it.

In this way, it is desirable that the content of the information provided to the driver 3 and the stimulation to be provided be selected in consideration of the driver's 3 condition. However, even if information and stimulation that are considered desirable are given to the driver 3, the condition of the driver 3 is not necessarily improved. Therefore, in this embodiment, changes in each control index are evaluated and the evaluation results are reflected in the control of the equipment group 20, so that the condition of the driver 3 is improved more reliably. . In the cognitive ability estimation process, it is determined whether there is a need to control the device group 20, and if it is determined that there is a need, a device to be controlled is selected from the device group 20, and the selected device is controlled. The content etc. are also determined.

The device control process in step S6 is a process for controlling the device to be controlled among the device group 20. The processing results of the actual drowsiness level analysis processing, the stress influence analysis processing, and the cognitive ability estimation processing are passed to the device control processing.
In the device control process, the results of the actual drowsiness level analysis process and the stress influence analysis process are passed, so that a case where the actual drowsiness level or degree of influence becomes 3 is handled. Start controlling devices to wake up sleepiness or control stress. After the start of control, and in a situation where both the actual drowsiness level and the degree of influence are not 3, the device control process controls the device according to the instruction content passed as a processing result of the cognitive ability estimation process. Control of the device also includes termination of control of the device.

In the cognitive ability estimation process, if no improvement in the driver's condition is confirmed even after stimulation is applied to the driver 3, stronger stimulation is applied in stages. Therefore, the actual drowsiness level or degree of influence becomes 3, and after starting to control the device, the device is controlled in accordance with the instruction content passed from the cognitive ability estimation process.

In this embodiment, the state of the driver 3 estimated from the autonomic nervous system is confirmed from video information, and the actual state of the driver 3 is estimated. Therefore, the actual condition of the driver 3 can be estimated with higher accuracy. It is possible to more appropriately respond to individual differences. Status information obtained from the vehicle improves estimation accuracy, making it possible to take more appropriate measures. As a result, information can be provided to the driver 3 according to the driver's condition, and furthermore, stimulation to the body can be provided in a timely manner and more appropriately. By providing stimulation or the like, it can be expected that the condition of the driver 3 will be improved. For this reason, support for the driver 3 can be provided more appropriately so that the driver 3 can drive the car safely.

FIG. 2 is a block diagram showing an example of the hardware configuration of a cognitive ability estimation device according to an embodiment of the present invention. In FIG. 2, various sensors related to the cognitive ability estimating device 1 and each device constituting the device group 20 are also shown. Note that this configuration example is just an example, and the hardware configuration of the cognitive ability estimation device 1 and the information processing device that can be used as the estimation device 1 is not limited to this.

As shown in FIG. 2, the cognitive ability estimation device 1 includes, for example, a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, an SSD (Solid State Drive) 14, and an IFC (InterFace). controller) 15, and a communication section 16. They are connected to the bus.

The CPU 11 executes, for example, a program recorded in the ROM 12 and/or a program recorded in the SSD 14 to realize various processes. Both programs are loaded into the RAM 13 and executed. The programs loaded from the SSD 14 into the RAM 13 include, for example, an OS (Operating System) and various application programs that run on the OS. The various application programs include one or more programs developed to cause the information processing device to function as the cognitive ability estimation device 1. Hereinafter, this developed application program will be referred to as a "developed application."

The developed application may be recorded on removable media and distributed. It may also be possible to distribute it via a network such as the Internet. For this reason, the recording medium on which the developed application is recorded must be installed or attached to an information processing device that is directly or indirectly connected to a network, or installed or attached to an externally accessible device. It may be attached.

The IFC group 15 enables connection with various external devices. The IFC group 15 may include IFCs for audio output and image output. An IFC for directly connecting in-vehicle equipment may also be included. An IFC that enables reception of biometric information or video information may also be included.
The RAM 13 also appropriately stores data necessary for the CPU 11 to execute various processes. The data includes data used by various programs executed by the CPU 11. Video information, biological information, and status information are also included in the data.

When each ECU including the ECU 9 is connected to an in-vehicle LAN (Local Area Network), the communication unit 16 enables communication with each ECU via the in-vehicle LAN. When making it compatible with the reception of biometric information from the smart watch 6, the communication unit 16 may be capable of wireless communication.

In FIG. 2, a camera 4 and a pulse sensor 31 are shown as a sensor group 30 capable of acquiring biological information. The pulse sensor 31 is provided on the handle 5, for example. This pulse sensor 31 may be provided not on the steering wheel 5 but on the driver's seat 2.
In addition to the sensor groups, two

sensor groups

40 and 50 are shown in FIG. 2 for acquiring status information. The sensor group 40 is particularly used to check the state of the vehicle, and the sensor group 50 is used to check the environment in which the vehicle is placed. As described above, among the ECUs, there is also the ECU 9 that processes information output from one or more sensors included in the

sensor groups

40 and 50, respectively, and generates and outputs status information. In FIG. 2, this ECU 9 is omitted.

The sensor group 40 includes a steering angle sensor 41, a brake/accelerator sensor 42, a G sensor 43, and a speedometer 44.
The steering angle sensor 41 detects the angle at which the steering wheel 5 is rotated as a steering angle. The break/accelerator sensor 42 detects the amount of operation of an accelerator pedal and a brake pedal (not shown). G sensor 43 is an acceleration sensor. Detects the acceleration generated in the vehicle. The speedometer 44 measures the running speed of the vehicle. All of these detection results are treated as vehicle body status information. The vehicle body status information will be referred to as "vehicle information" hereinafter.

As the sensor group 5, in addition to the camera 7 and the radar 8, a locator 51 is shown.
The locator 51 outputs position information representing the position of the vehicle specified by positioning. With this position information, it is possible to confirm the road on which the vehicle is traveling, the presence or absence of a curve in the road in the traveling direction, the degree of the curve (radius), etc.
Video information from the camera 7, status information generated by the ECU 9 from the video information and distance information from the radar 8, and position information from the locator 51 will be referred to as "environmental information" hereinafter.
The type, number, and combination of sensors constituting each sensor group 30 to 50 are not particularly limited. What is shown in FIG. 2 is one example.

As the equipment group 20, in addition to an air conditioning equipment control device 21 and a steering control device 22, a message output control device 23 and an automatic driving device 24 are shown.
The message output control device 23 makes it possible to provide information to the driver 3 through audio output. This control device 23 may be mounted on a navigation device.
The automatic driving device 24 enables automatic driving of the vehicle. If the automatic driving device 24 cannot confirm that the driver's condition has improved even after stimulation is applied, specifically, if it considers that the driver cannot be expected to drive safely due to drowsiness or stress, The automatic driving device 24 is requested to switch to automatic driving.

FIG. 3 is a functional block diagram showing an example of a functional configuration implemented on a cognitive ability estimation device according to an embodiment of the present invention. Next, an example of the functional configuration implemented on the cognitive ability estimation device 1 will be described in detail with reference to FIG. 3.

On the CPU 11 of the cognitive ability estimation device 1, as shown in FIG. An analysis section 115, an index evaluation section 116, an actual sleep level evaluation section 117, a stress influence evaluation section 118, a cognitive ability estimation section 119, and a device control processing section 120 are realized. The CPU 11 can transmit and receive information (data) to and from each ECU group 60 including the ECU 9 via the communication unit 16.

Each sensor included in the sensor groups 30 to 50 is connected to one of the ECUs forming the ECU group 60. The same applies to each device constituting the device group 20. Therefore, reception of information obtained by each sensor and control of each device are performed via one of the ECUs.

Functional components on the CPU 11 are realized by the CPU 11 executing various programs including development applications. As a result, the SSD 14 includes a biological information storage section 141, a state information storage section 142, a state analysis result storage section 143, an index evaluation result storage section 144, a level evaluation result storage section 145, an impact evaluation result storage section 146, and a cognitive ability storage section 144. Estimation result storage section 147, condition evaluation information storage section 148, index evaluation information storage section 149, level evaluation information storage section 150, impact evaluation information storage section 151, estimation information storage section 152, and device control information storage section 153 are information storage units. This area is reserved for use.

Note that information that should be temporarily stored may be stored in the RAM 13 instead of the SSD 14. Various information stored in the SSD 14 is stored in the RAM 13, and then transferred to and stored in the SSD 14. Here, for convenience, the process of storing information is ignored, and only the SSD 14 is assumed as the information storage destination.

The biological information acquisition unit 111 is responsible for acquiring biological information necessary for autonomic nerve state analysis. The biological information here includes not only information obtained by the pulse sensor 31 but also video information obtained by the camera 4. The biometric information acquired by the biometric information acquisition unit 111 is stored in the biometric information storage unit 141 secured in the SSD 14.

The vehicle information acquisition unit 112 and the environmental information acquisition unit 113 correspond to acquisition of vehicle information and environmental information, respectively. Both the acquired vehicle information and environmental information are stored in the status information storage section 142 secured in the SSD 14.
Note that information acquisition by the biological information acquisition unit 111, the vehicle information acquisition unit 112, and the environmental information acquisition unit 113 is performed at predetermined timings, for example, at predetermined time intervals. This is because pulse rate and the like are unlikely to repeat rapid changes within a short period of time.

The autonomic nerve state analysis unit 114 estimates the state of the autonomic nerves through analysis using the biological information stored in the biological information storage unit 141. Through the analysis, the activity levels of sympathetic and parasympathetic nerves are evaluated, and the evaluation results are used to estimate the levels of sleepiness and stress. This estimation result is stored as a state analysis result in the state analysis result storage section 143 secured in the SSD 14. The autonomic nerve state analysis process shown as step S2 in FIG. 1 is executed by the autonomic nerve state analysis unit 114.

The state evaluation information storage unit 148 secured in the SSD 14 stores information for estimating the state of the autonomic nerve as state evaluation information. For example, as described above, this evaluation information evaluates the activity levels of the sympathetic and parasympathetic nerves from the spectrum obtained by frequency analysis of the heartbeat interval, and from the evaluation results, the levels of sleepiness and stress are determined. This is information for further evaluation. The autonomic nerve state analysis unit 114 refers to this evaluation information and estimates the state of the autonomic nerves.

The video information analysis unit 115 generates motion feature information representing the characteristics of the movement of the driver 3 by analysis using video information stored in the biometric information storage unit 141 as biometric information, for example.
The index evaluation unit 116 evaluates each index through analysis using the generated motion characteristic information and the state information stored in the state information storage unit 142. The evaluation results of each index are stored in the index evaluation result storage section 144 secured in the SSD 14.

The index evaluation information storage unit 149 secured in the SSD 14 stores, for each index, information for evaluating that index as index evaluation information. This evaluation information is prepared for each category of status represented by status information, for example. The classification is a classification of vehicle conditions, taking into consideration, for example, the type of road (including whether it is an expressway or not), the driving area, and the driving speed. The index evaluation unit 116 evaluates each index by specifying evaluation information to be referred to from the state information and referring to the specified evaluation information using the operation characteristic information.
For this reason, the video analysis process shown as step S1 in FIG. 1 is realized by the video information analysis section 115 and the index evaluation section 116.

The actual drowsiness level evaluation unit 117 evaluates the actual drowsiness level. This evaluation includes, for example, the condition analysis results stored in the condition analysis result storage section 143, the condition information stored in the condition information storage section 142, the evaluation results of each index except for drowsiness stored in the index evaluation result storage section 144, This is performed using actual drowsiness level evaluation information stored in the level evaluation information storage unit 150.

The actual drowsiness level evaluation information is information for evaluating the actual drowsiness level. This evaluation information is prepared for each category of state represented by the state information, for example, similar to the index evaluation information. Thereby, the state information is used to specify which of the actual drowsiness level evaluation information stored in the level evaluation information storage section 150 should be referred to. Each piece of actual drowsiness level evaluation information is information representing the correspondence between each index other than drowsiness, that is, each evaluation result of fatigue and attentiveness, and the actual drowsiness level. The actual drowsiness level evaluated with reference to such actual drowsiness level evaluation information is stored in the level evaluation result storage section 145 secured in the SSD 14. The actual drowsiness level analysis process shown as step S3 in FIG. 1 is realized by the actual drowsiness level evaluation unit 117.

The stress impact evaluation unit 118 evaluates the actual degree of impact of stress. This evaluation uses the stress level stored as an analysis result in the condition analysis result storage section 143, the condition information stored in the condition information storage section 142, and the stress impact evaluation information stored in the impact evaluation information storage section 151. will be carried out.

Stress impact evaluation information is information for evaluating the degree of impact of stress. For people with low stress tolerance, even if the stress level is low, the effects of stress may be strongly reflected in their actual behavior. For this reason, it is thought that there are relatively large individual differences in the effects of stress. Therefore, in this embodiment, the degree of influence is positioned as the stress level that actually appears in the behavior of the driver 3, and the degree of influence is evaluated by referring to stress influence evaluation information.

Even if there are individual differences, if the level of stress that is estimated to be felt is small, the impact of that stress on the behavior of the driver 3 is considered to be small. For this reason, the target person may be, for example, only the driver 3 whose estimated stress level is 2 or higher. This also applies to the actual drowsiness level evaluation section 117.

For example, like the index evaluation information, the stress impact evaluation information is also prepared for each state category represented by the state information. Thereby, the status information is similarly used to specify which stress impact evaluation information stored in the impact evaluation information storage section 151 should be referred to.

The effects of stress may appear in behaviors that do not or are difficult to affect attention. Examples of such actions include relatively small continuous movements of the upper body, arms, or head. There may also be expressions of anger or sadness. Because of this, the degree of influence is now being evaluated from a different perspective than attentiveness, etc. Each stress impact evaluation information enables such an evaluation. The degree of impact evaluated with reference to such stress impact evaluation information is stored in the impact evaluation result storage section 146 secured in the SSD 14. The stress impact analysis process shown as step S4 in FIG. 1 is realized by the stress impact evaluation unit 118.

The cognitive ability estimating unit 119 estimates the cognitive ability level of the driver 3 from the control index, that is, the actual sleep level, the evaluation results of each index excluding drowsiness, and the degree of influence of stress. It is determined whether or not there is a need to control 20.
The cognitive ability estimating unit 119 checks changes in each control index, selects a device to be controlled from among the devices constituting the device group 20, and determines the content of the control. For this purpose, the cognitive ability estimation information stored in the estimation information storage section 152 secured in the SSD 14 is referred to.

The cognitive ability estimation information is information in which the type of equipment to be controlled and the content of the control are defined, for example, by control index, the level represented by the control index, and the content of changes in the level of the control index. . Therefore, the cognitive ability estimating unit 119 refers to the cognitive ability estimation information using each control index, and outputs the type of equipment to be controlled and the details of the control as the cognitive ability estimation result. This estimation result is stored in the cognitive ability estimation result storage section 147 secured in the SSD 14. The cognitive ability estimation process shown as step S5 in FIG. 1 is realized by the cognitive ability estimation unit 119.

The device control processing unit 120 performs processing for controlling the device group 20 according to the type of device to be controlled and the content of the control stored as the estimation result in the cognitive ability estimation result storage unit 147. The device control processing shown as step S6 in FIG. 1 is realized by the device control processing section 120.

The control of each device constituting the device group 20 is actually performed by the corresponding ECU. Therefore, the device control processing unit 120 transmits a control request including control information representing the specified device, actual control contents, etc. to the corresponding ECU, depending on the type of device to be controlled and its control content. This control information is determined by referring to the device control information stored in the device control information storage section 153 secured in the SSD 14. For this purpose, the device control information is such that control information to be transmitted is defined for each device and each control content, for example. A control request including control information is transmitted by the device control processing section 120 via the communication section 16. Note that the device group 20 is controlled as needed, and the device group 20 is not always controlled.

Each of the above-mentioned parts 111 to 120 operates while the vehicle is in a drivable state. For example, it operates every time a predetermined time interval elapses. Thereby, timely support can be provided so that the driver 3 can drive safely. This time interval may be changed depending on the situation, for example, depending on the actual drowsiness level estimated by the driver 3.

4 and 5 are flowcharts illustrating an example of safe driving support processing. This safe driving support processing controls necessary equipment by estimating cognitive ability based on analysis of video information and analysis of the state of autonomic nerves using biological information, in order to support safe driving of driver 3. This is the process to be executed. For example, it is executed every time a predetermined time interval elapses. Each unit 114 to 120 is realized by executing this process. Next, the safe driving support process will be described in detail with reference to FIGS. 4 and 5. It is assumed that the main body that executes the processing is the CPU 11.

First, in step S11, the CPU 11 analyzes the state of the autonomic nerve using biological information. In the following step S12, the CPU 11 evaluates each index by video analysis using video information. After that, the process moves to step S13.
In step S13, the CPU 11 determines whether or not there is an abnormal result that may impair safe driving in any of the autonomic nerve state analysis results or the evaluated indicators. For example, if drowsiness or stress is estimated by autonomic nerve state analysis, or if any of drowsiness, fatigue, or decreased attention is found by video analysis, it is determined that there is an abnormality and the determination in step S13 is made. If the answer is YES, the process moves to step S16. On the other hand, if this is not the case, it is assumed that there is no abnormality, the determination in step S13 is NO, and the process moves to step S14.

In step S14, the CPU 11 determines whether or not the device is currently being controlled. If the device is under control, the determination in step S14 is YES and the process moves to step S15. When the device is not under control, that is, when the driver 3 maintains a state in which he can drive safely, the determination in step S14 becomes NO, and the safe driving support process ends here.

In step S15, the CPU 11 ends the control of the device being controlled. Termination of the control is realized by transmitting a request to the corresponding ECU. After transmitting the request, the safe driving support process ends. Note that the automatic operation device 24 is not included in the target equipment here.

In step S16, the CPU 11 determines whether automatic driving is currently in progress. If automatic driving is in progress, the determination in step S16 is YES, and the safe driving support process ends here. If automatic operation is not in progress, that is, if manual operation is in progress, the determination in step S16 is NO and the process moves to step S17.

In step S17, the CPU 11 determines whether drowsiness is detected by analyzing the state of the autonomic nerves. If drowsiness is detected, that is, estimated, the determination in step S17 is YES and the process moves to step S18. If drowsiness is not detected, the determination in step S17 is NO, and the process moves to step S31 in FIG. 5.

In step S18, the CPU 11 evaluates the actual drowsiness level using the video analysis results. In the following step S19, the CPU 11 determines whether the evaluation result of the actual sleepiness level is equal to or higher than the setting. If the evaluation result is higher than the setting, for example, the actual drowsiness level is 2 or higher, the determination in step S19 is YES and the process moves to step S20. If the evaluation result is less than the setting, the determination in step S19 is NO and the process moves to step S31 in FIG.

In step S20, the CPU 11 determines whether or not devices other than the automatic driving device 24 are being controlled. If any device is being controlled, the determination in step S20 is YES and the process moves to step S22. If no device is being controlled, the determination in step S20 is NO and the process moves to step S21.

In step S21, the CPU 11 controls the device to be selected according to the actual drowsiness level. After that, the safe driving support process ends. Note that the device selected here is for providing a relatively small stimulus to the driver 3. If the actual drowsiness level is not improved, a stronger stimulus will be given to the driver 3.

In step S22, the CPU 11 determines whether or not the effect of improving the actual sleepiness level has been observed. If the effect is recognized, the determination in step S22 becomes YES and the process moves to step S21. Thereby, the driver 3 continues to be stimulated as before. On the other hand, if the effect is not recognized, the determination in step S22 is NO and the process moves to step S23. Note that since it takes a certain amount of time for the effect to appear on the driver 3, that time is taken into consideration when determining whether or not the effect is observed. This also applies to step S38, which will be described later.

In step S23, the CPU 11 determines whether there are other options for controlling the device. Other options are options for devices that provide stronger stimulation to the driver 3 or options for control content. If there are other options, the determination in step S23 is YES and the process moves to step S24. If there are no other options, the determination in step S23 is NO and the process moves to step S25.

In step S24, the CPU 11 selects one of the other options. What is selected here is, for example, the one that provides the least stimulus among the other options. After the selection, the process moves to step S21. As a result, the device will be controlled based on the selection result.
On the other hand, in step S25, the CPU 11 requests the automatic driving device 24 to switch to automatic driving. In this way, manual operation is transitioned to automatic operation. After that, the safe driving support process ends. For this reason, autonomous driving is treated separately from other options as an option that should be ultimately selected.

In step S31 of FIG. 5, the CPU 11 determines whether stress has been detected by analyzing the state of the autonomic nerves. If stress is detected, the determination in step S31 is YES and the process moves to step S32. If stress is not detected, the determination in step S31 is NO and the process moves to step S34.

In step S32, the CPU 11 evaluates the degree of influence of stress. In the following step S33, the CPU 11 determines whether the evaluated degree of influence is greater than or equal to a setting. If the evaluated degree of influence is equal to or higher than the setting, for example, if the degree of influence is 2 or more, the determination in step S33 is YES and the process moves to step S36. If the degree of influence is less than the setting, the determination in step S33 is NO and the process moves to step S34.

In step S34, the CPU 11 estimates cognitive ability, determines whether or not it is necessary to control the device according to the estimation result, and if it is determined that it is necessary to control the device, the CPU 11 estimates the cognitive ability. Determine the equipment and its control details. In the subsequent step S35, the CPU 11 performs processing according to the determination result of necessity, the equipment to be controlled, and the details of the control. After that, the safe driving support process ends.
In addition, when proceeding to step S34, it is considered that the influence of the state of the autonomic nerves on the driving of the driver 3 is relatively small. Therefore, in step S34, cognitive ability may be estimated using only the evaluation results of each index.

In step S36, the CPU 11 determines whether or not devices other than the automatic driving device 24 are being controlled. If any device is being controlled, the determination in step S36 is YES and the process moves to step S38. If no device is being controlled, the determination in step S36 is NO and the process moves to step S37.

In step S37, the CPU 11 controls the device to be selected according to the degree of influence. After that, the safe driving support process ends. Note that the device selected here is intended to provide a relatively small stimulus to the driver 3 to suppress the influence of stress. If the degree of influence is not improved, a stronger stimulus will be given to the driver 3.

In step S38, the CPU 11 determines whether or not the effect of improving the degree of influence has been observed. If the effect is recognized, the determination in step S38 becomes YES and the process moves to step S37. Thereby, the driver 3 continues to be stimulated as before. On the other hand, if the effect is not recognized, the determination in step S38 is NO and the process moves to step S39.

In step S39, the CPU 11 determines whether there are any other options for controlling the device. Other options are options for devices or control contents that provide stronger stimulation to the driver 3 and suppress the effects of stress. If there are other options, the determination in step S39 is YES and the process moves to step S40. If there are no other options, the determination in step S39 is NO, and the safe driving support process ends here. As a result, the same stimulation as before continues to be given to the driver 3.

In step S40, the CPU 11 selects one of the other options. What is selected here is, for example, the one that provides the least stimulus among the other options. After the selection, the process moves to step S37. As a result, the device will be controlled based on the selection result.

In this way, in this embodiment, stimulation is applied to the driver 3 who is estimated to be in an undesirable state for safe driving, and if the state does not improve even after the stimulation is applied, the driver 3 is given a stimulus in a step-by-step manner. , to give a stronger stimulus. If the driver 3 is so sleepy that no improvement is seen even after stimulation, the driver 3 is forced to shift to automatic driving. Note that the stimulation includes providing information through audio output. If no improvement is observed by providing information through voice output, body stimulation other than voice output will be performed at the same time. The process of applying stronger stimulation step by step is based on the idea that there is a low possibility that an inappropriate control index will change while the device is being controlled. This is because, for example, if the driver 3 is feeling very sleepy, it is unlikely that he will feel strong stress in a relatively short period of time.

In the functional configuration exemplified in FIG. 4, the biometric information acquisition unit 111 corresponds to biometric information acquisition means and video information acquisition means. The autonomic nerve state analysis unit 114 corresponds to state estimation means. The video information analysis section 115, the index evaluation section 116, the actual drowsiness level evaluation section 117, the stress influence evaluation section 118, and the cognitive ability estimation section 119 correspond to cognitive ability estimation means.
Furthermore, the vehicle information acquisition section 112 and the environmental information acquisition section 113 correspond to state information acquisition means. The device control processing section 120 corresponds to control processing means.

Note that in this embodiment, a car is assumed as the mobile device, but the mobile device is not limited to a car. The mobile device may be a flying vehicle such as a train, an airplane, or a helicopter, or a ship.
Such moving objects differ not only in their characteristics but also in the constraints on which a subject can drive or operate them. For example, trains can only travel on rails. Furthermore, there is usually no need to consider the presence of obstacles. For this reason, it is necessary to determine the evaluation method for each index and the estimation method for cognitive ability depending on the type of moving object. However, this embodiment can also be applied to moving objects other than automobiles.

Furthermore, in this embodiment, as described above, each index evaluated from video information and the results obtained by analyzing the state of autonomic nerves are used in a complementary manner to each other. However, drowsiness and the like can be estimated more quickly by analyzing the state of the autonomic nervous system. For this reason, the control may be divided into control using the estimation result from autonomic nerve state analysis and control using both the estimation result and the evaluation result using video information. As a result, for example, if drowsiness is estimated by autonomic nerve state analysis, a warning will be issued by audio output, and if drowsiness is also estimated by evaluation using video information, physical stimulation will be provided by controlling the device. You can also give it to them. Various modifications including this are possible.

1. Cognitive ability estimation device, 2. Driver's seat, 3. Driver (subject), 4, 7. Camera, 6. Smart watch, 8. Radar, 9. ECU, 11. CPU, 14. SSD, 16. Communication department, 20. Equipment group, 21. Air conditioning equipment. Control device, 22 Steering control device, 23 Message output control device, 24 Automatic driving device, 31 Pulse sensor, 41 Steering angle sensor, 42 Brake access sensor, 43 G sensor, 44 Speedometer, 111 Biological information acquisition unit, 112 Vehicle Information acquisition unit, 113 Environmental information acquisition unit, 114 Autonomic nerve state analysis unit, 115 Video information analysis unit, 116 Index evaluation unit, 117 Actual sleep level evaluation unit, 118 Stress impact evaluation unit, 119 Cognitive ability estimation unit, 120 Equipment control processing section.

Claims

biological information acquisition means for acquiring biological information that can at least identify the heartbeat of a subject who drives or steers a mobile object;
video information acquisition means for acquiring video information of the subject;
a state estimating means for estimating the state of the subject's autonomic nerves based on the heartbeat identified from the biological information;
Cognitive ability estimating means for estimating the cognitive ability of the subject based on the estimation result of the autonomic nerve state by the state estimating means and the video information;
A cognitive ability estimation device comprising:
The cognitive ability estimating means evaluates the condition of the subject using a plurality of indicators including sleepiness based on the video information, and evaluates the condition of the subject represented by the evaluation results of each indicator and the estimation result of the state of the autonomic nervous system. estimating the cognitive ability using the sleepiness level of
The cognitive ability estimation device according to claim 1.
The cognitive ability estimating means calculates an actual sleepiness level that is the actual sleepiness level of the subject using the sleepiness level of the subject represented by the estimation result of the state of the autonomic nervous system and the evaluation results of the plurality of indicators. evaluate,
The cognitive ability estimation device according to claim 2.
The cognitive ability estimating means evaluates the condition of the subject using one or more indicators based on the video information, and evaluates the condition of the subject expressed by the evaluation results of each indicator and the estimation result of the autonomic nerve condition. using the stress level to estimate the degree to which stress at the stress level affects each evaluation result of the plurality of indicators;
The cognitive ability estimation device according to claim 1.
further comprising a state information acquisition means capable of acquiring state information representing the state of the mobile object,
The cognitive ability estimating means estimates the cognitive ability of the subject based on the estimation result of the autonomic nerve state by the state estimating means, the video information, and the state information.
The cognitive ability estimation device according to claim 1.
control processing means for performing processing for controlling a first device capable of giving the subject a stimulus for improving the cognitive ability, based on the estimation result of the cognitive ability by the cognitive ability estimating means; , further comprising;
The cognitive ability estimation device according to claim 1.
The control processing means performs processing for controlling a second device that can notify a person other than the target person.
The cognitive ability estimation device according to claim 6.
When the mobile body is equipped with an automatic driving function, the control processing means controls the movement by the automatic driving function based on the estimation result of the cognitive ability by the cognitive ability estimation means after controlling the device. Performs processing for self-driving the body,
The cognitive ability estimation device according to claim 6.
In the information processing device,
Acquire biological information that can at least identify the heartbeat of a subject who drives or steers a mobile object,
Obtain video information of the subject;
Estimating the state of the subject's autonomic nerves based on the heartbeat identified from the biological information,
estimating the cognitive ability of the subject based on the estimation result of the state of the autonomic nerve and the video information;
A program that executes processing.