JP2018075255A - State estimation device, and state estimation method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a state estimation device or the like for achieving highly accurate state estimation while securing resistance to disturbance.SOLUTION: A wearable device 10 functions as a state estimation device for estimating a user's state. The wearable device 10 acquires blinking detection information that detects the user's blinking, and calculates a plurality of feature amounts relating to the user's blinking from the blinking detection information. Further, the wearable device 10 estimates user environment around the user, and selects a feature amount corresponding to the user environment from among the plurality of feature amounts. The user's state is estimated by using the feature amount selected in this manner.SELECTED DRAWING: Figure 3

Description

  The disclosure according to this specification relates to a state estimation device and a state estimation method for estimating a user's state.

  Conventionally, in order to estimate a user's state such as drowsiness, for example, a technique for detecting the opening degree of a user's eyelids and blinks is known. For example, Patent Document 1 discloses a face detection device that identifies the positions of upper and lower eyelids that have blinked from the user's face image captured using an imaging unit and detects the opening degree of the user's eyelids. Yes.

  In addition, the face detection device of Patent Document 1 detects the angle of the face direction of the user in the left-right direction, and detects the eyelid opening degree of both eyes when the angle of the user face direction is equal to or less than the threshold value. When the face angle exceeds the threshold, the process switches to processing for detecting the eyelid opening degree of one eye. By such processing, a decrease in detection accuracy due to the user's face orientation is avoided.

Japanese Patent No. 4992823

  The inventor of the present disclosure has focused on the fact that the detection accuracy is reduced not only by the user's face direction but also by disturbances such as wind and light around the user. Here, in patent document 1, the fall of the detection accuracy by the disturbance resulting from the environment around a user is not considered at all. Therefore, when the state of the user is estimated using the technique disclosed in Patent Document 1, there may be a concern that the detection accuracy is deteriorated due to the influence of disturbance.

  The present disclosure has been made in view of the above problems, and aims to provide a state estimation device and a state estimation method capable of realizing highly accurate state estimation while ensuring resistance to disturbance. To do.

  In order to achieve the above object, the disclosed first aspect includes a blink information acquisition unit (31) that acquires blink detection information for detecting a blink of the user, and a blink of the user from the blink detection information. Among the plurality of feature amounts calculated by the feature amount calculation unit (32) that calculates a plurality of related feature amounts, the environment estimation unit (33) that estimates the user environment around the user, and the feature amount calculation unit A state in which a feature amount selection unit (34) for selecting a feature amount corresponding to the user environment estimated by the environment estimation unit and a state in which the user state is estimated using the feature amount selected by the feature amount selection unit And an estimation unit (35).

  Further, the disclosed second aspect is a state estimation method for estimating a user's state by processing of at least one processor (21, 251 and 351). Obtaining (S111), calculating a plurality of feature quantities related to the user's blink from the blink detection information (S112), estimating a user environment around the user (S102), Then, a feature amount corresponding to the estimated user environment is selected (S113), and the state of the user is estimated using the selected feature amount (S115).

  In these aspects, after estimating the user environment around the user, a feature amount corresponding to the estimated user environment is selected from a plurality of feature amounts calculated from the blink detection information. Therefore, when the user is in an environment with little disturbance, the state of the user is estimated using many feature amounts. As a result, it is possible to estimate the state with high accuracy. On the other hand, when there is a user in an environment where a strong disturbance exists, a feature amount that is easily affected by the disturbance can be excluded from the target used for estimating the user's state. As a result, deterioration in accuracy of state estimation due to the influence of disturbance is reduced. According to the above, it is possible to realize highly accurate user state estimation while ensuring resistance to disturbance.

  Note that the reference numbers in the parentheses merely show an example of a correspondence relationship with a specific configuration in an embodiment described later, and do not limit the technical scope at all.

It is a perspective view of the wearable device by a first embodiment. It is a block diagram which shows the electrical structure of a wearable device. It is a figure which shows each functional block constructed | assembled in a state estimation circuit. It is a figure which shows typically the detail of each feature-value calculated from an electrooculogram signal. It is a flowchart which shows the detail of an environment estimation process. It is a flowchart which shows the detail of a state estimation process. It is a block diagram which shows the electrical structure of the state estimation system by 2nd embodiment. It is a block diagram which shows the electric constitution of the state estimation system by 3rd embodiment.

  Hereinafter, a plurality of embodiments of the present disclosure will be described based on the drawings. In addition, the overlapping description may be abbreviate | omitted by attaching | subjecting the same code | symbol to the corresponding component in each embodiment. When only a part of the configuration is described in each embodiment, the configuration of the other embodiment described above can be applied to the other part of the configuration. Moreover, not only the combination of the configurations explicitly described in the description of each embodiment, but also the configuration of a plurality of embodiments can be partially combined even if they are not explicitly described, as long as there is no problem in the combination. And the combination where the structure described in several embodiment and the modification is not specified shall also be disclosed by the following description.

(First embodiment)
The function of the state estimation apparatus according to the first embodiment of the present disclosure is realized by the eyewear-type wearable device 10 illustrated in FIGS. 1 and 2. Wearable device 10 is provided with frame 10a formed in the shape of glasses, and can measure the user's electrooculogram when worn by the user. The wearable device 10 can estimate the degree of sleepiness, tension, concentration, and the like as the user state based on the measured electrooculogram. The estimation result of the user state estimated by the wearable device 10 is transmitted to an electronic device such as a smartphone by wireless communication, and is used for providing information to the user or calling attention.

  The wearable device 10 includes a battery 11, a blink measurement unit 12, an environment sensor group 17, a communication module 18, and a state estimation circuit 20 as an electrical configuration.

  The battery 11 is a power source that supplies power to the blink measurement unit 12, the communication module 18, the state estimation circuit 20, and the like. The battery 11 may be a primary battery such as a lithium battery or a secondary battery such as a lithium ion battery.

  The blink measurement unit 12 measures the user's electrooculogram in order to detect the user's blink. The blink measurement unit 12 includes a center electrode 13, a left electrode 14, a right electrode 15, and the like. Each electrode 13-15 is formed with metal materials, such as stainless steel provided with high rust prevention property and electroconductivity. Each electrode 13-15 is provided in the flame | frame 10a in the arrangement | positioning spaced apart from each other. Each of the electrodes 13 to 15 is connected to the state estimation circuit 20 through a wiring embedded in the frame 10a.

  Each electrode 13-15 is arrange | positioned near a user's eyeball by a user wearing the wearable device 10, and contacts a user's skin. The center electrode 13 is provided at the bridge portion of the frame 10 a and contacts the portion between the eyebrows of the user wearing the wearable device 10. The left electrode 14 and the right electrode 15 form left and right nose pads of the glasses-like frame 10a, and contact the user's nose from both the left and right sides.

  The blink measurement unit 12 supplies the electrooculogram signal measured using the electrodes 13 to 15 to the state estimation circuit 20 as blink detection information. The number and arrangement of the electrodes provided in the blink measurement unit 12 can be changed as appropriate. For example, the blink measurement unit 12 may further include a ground electrode that supplies a ground potential to the state estimation circuit 20.

  The environmental sensor group 17 has a plurality of sensors. The plurality of sensors measure physical quantities necessary for estimating the environment around the user wearing the wearable device 10 (hereinafter, “user environment”). Specifically, the environmental sensor group 17 includes a clock 17a, a GNSS (Global Navigation Satellite System) receiving unit 17b, and an illuminance sensor 17c. The clock 17a, the GNSS receiver 17b, and the environment sensor group 17 are each connected to the state estimation circuit 20.

  The clock 17a sequentially provides time information indicating the current date and time to the state estimation circuit 20. The GNSS receiving unit 17b generates current position information of the wearable device 10 based on positioning signals received from a plurality of artificial satellites. The GNSS receiver 17b sequentially provides the generated position information and strength information indicating the received strength of the positioning signal to the state estimation circuit 20. The illuminance sensor 17c measures the brightness around the wearable device 10 by converting the light incident on the light receiving element into a current. The illuminance sensor 17c sequentially provides brightness information indicating brightness around the user to the state estimation circuit 20.

  The communication module 18 can transmit and receive information to and from an electronic device such as a smartphone by wireless communication using, for example, Bluetooth (registered trademark) and a wireless LAN. The communication module 18 receives environment information that can be used for estimation of the user environment by wireless communication, and transmits the estimation result of the user state to another electronic device or the like.

  The state estimation circuit 20 includes a processor 21, a RAM 22, a storage medium 23, a power interface, a signal interface, and the like. A storage area 37 (see FIG. 3) to be described later is secured in the storage medium 23. The signal interface includes, for example, an amplifier unit that amplifies signals input from the electrodes 13 to 15 and a filter unit that removes signal noise.

  The state estimation circuit 20 causes the processor 21 to execute a state estimation program stored in the storage medium 23. By executing the state estimation program, the state estimation circuit 20 includes the blink information acquisition unit 31, the feature amount calculation unit 32, the environment estimation unit 33, the feature amount selection unit 34, and the state estimation unit 35 illustrated in FIG. Built as.

  The blink information acquisition unit 31 acquires blink detection information that has detected a user's blink from the blink measurement unit 12. The blink detection information includes at least a measurement result (an electrooculogram signal) obtained by measuring the electrooculogram of the user. The electrooculogram signal is measurement data used for estimating eye movement by the so-called EOG method.

  The feature amount calculation unit 32 calculates a plurality of feature amounts related to the user's blink from the electrooculogram signal as the blink detection information acquired by the blink information acquisition unit 31. As shown in FIG. 4, when the user performs blinks, a large amplitude appears in the electrooculogram signal. The feature amount calculation unit 32 (see FIG. 3) determines that one blink has occurred when the amplitude in the waveform of the electrooculogram signal exceeds both the threshold values th1 and th2 set to positive and negative, respectively. The feature amount calculation unit 32 calculates, for each detected blink, a blink strength and a time required for the blink (blink time). In addition, the feature amount calculation unit 32 generates the number of blinks generated per predetermined time (for example, 1 minute) (hereinafter, “number of blinks”), the blink occurrence interval, the deviation of the blink interval, and the blink strength deviation. Then, the blink time deviation, the number of occurrences of the blink blink, and the like are calculated. It should be noted that the number of occurrences of cluster blinks is counted as one occurrence of cluster blinks when multiple blinks occur in a very short specific time (for example, 1 second). As shown in FIG. 4, the feature amount calculation unit 32 sequentially provides information on each calculated feature amount, that is, a calculated value, to the environment estimation unit 33 and the feature amount selection unit 34.

  The environment estimation unit 33 performs environment estimation processing (see FIG. 5) for estimating the user environment. The environment estimation process is started when the user wears the wearable device 10 and continues repeatedly until the user removes the wearable device 10. In the environment estimation process, the environment estimation unit 33 acquires environment information necessary for environment estimation from the environment sensor group 17, the communication module 18, and the feature amount calculation unit 32 (see S101 in FIG. 5). And the environment estimation part 33 estimates a user environment based on the acquired environment information (refer FIG. 5 S102).

  In the environment estimation process, the environment estimation unit 33 can determine whether or not the user feels dazzled based on, for example, environment information related to brightness around the user. In addition, the environment estimation unit 33 can determine whether or not the user is in an environment receiving wind based on environment information related to the wind around the user.

  Specifically, the environment information related to the brightness corresponds to brightness information acquired from the illuminance sensor 17c, time information acquired from the clock 17a, and the like. The environment estimation unit 33 can determine whether or not the user feels dazzling based on the brightness information of the illuminance sensor 17c. When the illuminance of the user environment detected by the illuminance sensor 17c is higher than a preset illuminance threshold, the environment estimation unit 33 determines that the user feels dazzling.

  In addition, the environment estimation unit 33 estimates whether the current time is a daytime zone or a nighttime zone based on, for example, time information. If the current time is a daytime time zone, the environment estimation unit 33 determines that the user feels dazzled by natural light. On the other hand, if the current time is a night time zone, the environment estimation unit 33 determines that the user does not feel dazzling or is difficult to feel.

  Moreover, the strength information of the positioning signal acquired from the GNSS receiving unit 17b corresponds to the environment information related to the wind. The environment estimation unit 33 estimates whether the user is indoors or outdoors based on the strength information of the positioning signal. When the received intensity of the positioning signal is lower than a preset sensitivity threshold, the environment estimation unit 33 determines that the user is not indoors because the user is indoors. On the other hand, when the received intensity of the positioning signal is higher than a preset sensitivity threshold, the environment estimation unit 33 determines that the user is outside and is in an environment receiving wind.

  Furthermore, the environment estimation unit 33 can estimate the user environment based on changes in the feature amounts using the feature amounts calculated by the feature amount calculation unit 32 as environment information. For example, among each feature amount, the number of blinks, the occurrence interval of blinks, and the deviation of the occurrence interval are easily affected by glare. Therefore, when significant changes occur in these values, the environment estimation unit 33 determines that the user environment has changed to an environment in which glare is felt. Similarly, among the feature amounts, the number of blinks, the blink generation interval, and the blink time deviation are easily affected by the wind. Therefore, when significant changes occur in these values, the environment estimation unit 33 determines that the user environment has changed to an environment that receives wind.

  The feature amount selection unit 34 acquires the calculated values of the plurality of feature amounts calculated by the feature amount calculation unit 32. The feature amount selection unit 34 selects a feature amount corresponding to the user environment estimated by the environment estimation unit 33 from the plurality of feature amounts calculated by the feature amount calculation unit 32. When the user does not feel dazzling and is not in an environment that receives wind, the feature amount selection unit 34 uses all the feature amounts calculated by the feature amount calculation unit 32 as the feature amounts used for state estimation. Select (see selection result “SR1”).

  On the other hand, when a user environment that feels dazzling is estimated, the feature amount selection unit 34 selects a plurality of feature amounts that are set in advance as being easily affected by glare from the state estimation use target. exclude. Specifically, when a user environment that feels dazzling is estimated, the feature amount selection unit 34 determines the number of blinks, the occurrence interval of blinks, and the deviation of the occurrence interval among the feature amounts. , Exclude from state estimation use. As a result, the blink strength, the blink time, the blink strength deviation, the blink time deviation, and the number of occurrences of the cluster blink are selected as the state estimation use targets (see selection result “SR2”).

  Furthermore, when a user environment that receives wind is estimated, the feature amount selection unit 34 excludes a plurality of feature amounts that are set in advance as being easily affected by wind from the use targets of state estimation. Specifically, when the user environment that receives wind is estimated, the feature amount selection unit 34 calculates the above-described deviation of the number of blinks, the occurrence interval of blinks, and the blink time among the feature amounts. , Exclude from state estimation use. As a result, the blink strength, the blink time, the blink strength deviation, the occurrence interval deviation, and the number of occurrences of the cluster blink are selected as the state estimation use targets (see selection result “SR3”).

  The state estimation unit 35 can estimate the degree of sleepiness, tension, concentration, and the like as the user state using the feature amount selected by the feature amount selection unit 34. The state estimation unit 35 is provided with a storage area 37. A large number of discriminators 36 are stored in the storage area 37 in advance. Each discriminator 36 is generated with contents corresponding to a combination of feature amounts selected by the feature amount selection unit 34. In addition, the discriminator 36 is individually set for each specific user state to be estimated such as sleepiness and tension. Each discriminator 36 is generated in advance by using machine learning such as a random forest, and is optimized for each user environment. Each discriminator 36 can digitize and output the degree of sleepiness, the degree of tension, the degree of concentration, etc., using each numerical value of the selected feature amount.

  The state estimation unit 35 performs state estimation processing (see FIG. 6) together with the blink information acquisition unit 31 and the feature amount calculation unit 32. The state estimation process is started by the user wearing the wearable device 10 in the same manner as the environment estimation process, and continues in parallel with the environment estimation process until the user removes the wearable device 10.

  In S111 of the state estimation process, an electrooculogram signal is acquired as blink detection information, and the process proceeds to S112. In S112, a plurality of feature amounts are calculated from the electrooculogram signal acquired in S111, and the process proceeds to S113. In S113, the feature quantity corresponding to the user environment estimated by the environment estimation process is selected, and the process proceeds to S114.

  In S114, one discriminator 36 corresponding to the combination of the feature amounts selected in S113, that is, the feature amount setting results SR1 to SR3 is selected, and the process proceeds to S115. In S114, when there are a plurality of user states to be estimated, the discriminator 36 corresponding to each estimation item is selected one by one. That is, for example, when estimating the sleepiness level and the tension level, a discriminator 36 for estimating the sleepiness level and a discriminator 36 for estimating the tension level are selected.

  In S115, each numerical value of the feature amount selected in S113 is input to the discriminator 36 selected in S114, so that a numerical value obtained by estimating the user state such as the drowsiness level and the tension level is acquired, and again in S111. Return.

  In the first embodiment described so far, the user environment around the user is estimated, and a feature quantity corresponding to the estimated user environment is selected from a plurality of feature quantities calculated from the blink detection information. Is done. Therefore, when the user is in an environment with little disturbance such as natural light and wind, the state of the user is estimated using many feature amounts. As a result, it is possible to estimate the state with high accuracy. On the other hand, when there is a user in an environment where a strong disturbance exists, a feature quantity that is easily affected by the disturbance can be excluded from the target used for state estimation. As a result, deterioration in accuracy of state estimation due to the influence of disturbance is reduced. According to the above, it is possible to realize highly accurate state estimation while ensuring resistance to disturbance.

  In addition, in the first embodiment, a plurality of discriminators 36 are stored in the storage area 37 in advance. With such a configuration, the individual discriminators 36 can be generated in advance with contents optimized for a specific user environment. Therefore, according to the process of selecting one from the plurality of discriminators 36 and estimating the user state, the accuracy of state estimation in each user environment is further improved.

  Further, according to the first embodiment, when the user feels dazzling, feature quantities that are easily affected by dazzling are excluded from the state estimation usage targets. Similarly, when the user is in an environment that receives wind, feature quantities that are easily affected by wind are excluded from use targets of state estimation. As described above, by estimating the user environment that significantly affects blinking such as light and wind, and selecting the feature amount corresponding to each user environment, tolerance to disturbance of state estimation is reliably ensured. The

  Furthermore, in the first embodiment, the change in the user environment is estimated based on the change in the feature amount calculated by the feature amount calculation unit 32. As described above, when the user environment is displaced, a particular feature amount is significantly changed. Therefore, by using the change in the feature amount as the environment information, the user environment estimation accuracy by the environment estimation process can be further improved.

  In addition, the feature amount calculation unit 32 of the first embodiment calculates the blink strength as one of the feature amounts. Since the blink strength is hardly affected by disturbance, it is not excluded from the state estimation usage target even in an environment with disturbance. Therefore, if the configuration can calculate the blink strength, the accuracy of state estimation can be reliably maintained even in an environment where disturbance occurs. In the first embodiment, the wearable device 10 corresponds to a “state estimation device”.

(Second embodiment)
The second embodiment of the present disclosure shown in FIG. 7 is a modification of the first embodiment. In the second embodiment, the state estimation system 200 is configured by the wearable device 210, the smartphone 240, and the like. The wearable device 210 and the smartphone 240 are paired with each other and can transmit and receive information by wireless communication. The environment estimation process and the state estimation process in the state estimation system 200 are performed by the smartphone 240 among the wearable device 210 and the smartphone 240.

  The wearable device 210 is a glasses-type motion sensor device similar to the first embodiment (see FIG. 1). The wearable device 210 includes the battery 11, the blink measurement unit 12, the communication module 18, and the blink measurement circuit 220 that are substantially the same as those of the first embodiment. The blink measurement circuit 220 has a configuration mainly composed of a processor as in the state estimation circuit 20 (see FIG. 2) of the first embodiment. The blink measurement circuit 220 acquires the electrooculogram signal measured by the blink measurement unit 12 and wirelessly transmits it as blink detection information from the communication module 18 to the smartphone 240.

  The smartphone 240 is a mobile terminal owned by the user. The smartphone 240 includes a communication module 248, an environment sensor group 247, a state estimation circuit 250, and a battery 41 that supplies power to these components as an electrical configuration.

  The communication module 248 can wirelessly communicate with the communication module 18 of the wearable device 210. The communication module 248 receives the blink detection information transmitted from the communication module 18 and sequentially supplies it to the state estimation circuit 250. In addition, the communication module 248 enables the smartphone 240 to connect to the Internet.

  The environmental sensor group 247 has a configuration corresponding to the environmental sensor group 17 (see FIG. 2) of the first embodiment, and includes a clock 17a, a GNSS receiver 17b, and an illuminance sensor 17c. The environmental sensor group 247 considers that the smartphone 240 is in substantially the same environment as the user, and acquires environment information around the smartphone 240. The environmental sensor group 247 sequentially supplies the acquired environmental information to the state estimation circuit 250.

  The state estimation circuit 250 is configured to perform substantially the same function as the state estimation circuit 20 (see FIG. 2) of the first embodiment. The state estimation circuit 250 is mainly configured by a microcomputer having a processor 251, a RAM 252, and a storage medium 253. The state estimation circuit 250 executes a state estimation program by the processor 251 based on the application activation by the user. As a result, the state estimation circuit 250 includes a blink information acquisition unit 31, a feature amount calculation unit 32, an environment estimation unit 33, a feature amount selection unit 34, and a state estimation unit 35 that are substantially the same as those in the first embodiment. (See FIG. 3).

  The environment estimation unit 33 (see FIG. 3) built in the state estimation circuit 250 can estimate the user environment using environment information acquired from the Internet. For example, the environment estimation unit 33 can determine whether or not the user feels dazzling by combining the weather information with the time information. Moreover, the environment estimation part 33 can estimate more accurately whether the user is indoors or outdoors by combining position information and map information with the intensity information of the positioning signal. As a result, the environment estimation unit 33 can more accurately determine whether or not the user is in an environment receiving wind.

  As in the second embodiment described so far, the feature amount corresponding to the user environment can be selected even in the embodiment in which the environment estimation process and the state estimation process are performed by the smartphone 240. As a result, as in the first embodiment, it is possible to reduce the influence on the state estimation due to the disturbance, and to realize a highly accurate state estimation in a situation without the disturbance. In the second embodiment, the smartphone 240 corresponds to a “state estimation device”.

(Third embodiment)
The third embodiment of the present disclosure shown in FIG. 8 is a modification of the second embodiment. A state estimation system 300 according to the third embodiment includes a driver monitor system (DMS) 310, an in-vehicle control device 340, and the like. The state estimation system 300 is mounted on a vehicle and can estimate the state of a passenger (particularly a driver) of the vehicle as a user. The DMS 310 and the in-vehicle control device 340 are directly connected by a communication cable, for example, and can transmit / receive information to / from each other. The environment estimation process and the state estimation process in the state estimation system 300 are performed by the in-vehicle control device 340 out of the DMS 310 and the in-vehicle control device 340.

  The DMS 310 includes a blink measurement unit 312, an imaging control circuit 320, a communication module 318, and the like. The blink measurement unit 312 is a camera unit having a near infrared light source 313 and a near infrared camera 314. The blink measurement unit 312 is installed in front of the driver in the passenger compartment of the vehicle. The near-infrared light source 313 images the driver's face irradiated with near-infrared light by the near-infrared light source 313 with the near-infrared camera 314.

  The imaging control circuit 320 is configured mainly with a processor, and controls imaging of the driver's face by the blink measurement unit 312. In addition, the imaging control circuit 320 detects the opening degree of the driver's eyes by analyzing the face image captured by the blink measurement unit 312. The imaging control circuit 320 causes the communication module 318 to transmit the detection result of the eye opening of the driver as blink detection information to the in-vehicle control device 340.

  The in-vehicle control device 340 is one of a plurality of electronic control units mounted on the vehicle, and is configured to integrally control a user interface for a driver, for example. The in-vehicle control device 340 includes a communication module 348 and a state estimation circuit 350 as an electrical configuration.

  The communication module 348 can communicate with the communication module 318 of the DMS 310. The communication module 348 receives blink detection information transmitted from the communication module 318. In addition, the communication module 348 is connected to a communication bus 360 of an in-vehicle network mounted on the vehicle. The communication module 348 can acquire various vehicle information output to the communication bus 360. For example, the communication module 348 acquires the operation information of the air conditioner mounted on the vehicle, the opening / closing information indicating the opening / closing state of the window of the vehicle, and the like as the environment information indicating the user environment. The communication module 348 sequentially supplies the acquired instantaneous detection information and environment information to the state estimation circuit 350.

  The state estimation circuit 350 is configured to perform substantially the same function as the state estimation circuit 250 (see FIG. 7) of the second embodiment. The state estimation circuit 350 is connected to an environmental sensor group 347 mounted on the vehicle. The GNSS receiving unit 347b of the environmental sensor group 347 is configured to be incorporated in a locator, for example. The illuminance sensor 347c of the environmental sensor group 347 is configured to detect the brightness outside the vehicle, for example, in order to switch the vehicle headlamp on and off.

  The state estimation circuit 350 is mainly configured by a microcomputer having a processor 351, a RAM 352, and a storage medium 353. The state estimation circuit 350 executes a state estimation program by the processor 351 when power supply from a power source mounted on the vehicle is started, for example. As a result, the state estimation circuit 350 includes the blink information acquisition unit 31, the feature amount calculation unit 32, the environment estimation unit 33, the feature amount selection unit 34, and the state estimation unit 35 that are substantially the same as those in the second embodiment. (See FIG. 3).

  The feature amount calculation unit 32 (see FIG. 3) constructed in the state estimation circuit 350 calculates each feature amount using the detection result of the eye opening. In the third embodiment in which the blink detection information is acquired using the camera unit, the blink strength and the intensity deviation are not calculated.

  The environment estimation unit 33 (see FIG. 3) built in the state estimation circuit 350 can estimate the user environment by acquiring the environment information received by the communication module 348. The environment estimation unit 33 can determine whether or not the user is in an environment receiving wind by using the air conditioner operation information and the window opening / closing information. For example, when the air conditioner is blowing air to the user, the environment estimation unit 33 determines that the user is in an environment that receives wind. In addition, when the window on the side of the driver's seat is opened, the environment estimation unit 33 determines that the user is in an environment that receives wind.

  As in the third embodiment described so far, even in an embodiment in which the environment estimation process and the state estimation process are performed by the in-vehicle control device 340, it is possible to select a feature amount corresponding to the user environment. As a result, as in the second embodiment, it is possible to achieve both of ensuring resistance to disturbance and estimating the user state with high accuracy. In the third embodiment, the in-vehicle control device 340 corresponds to a “state estimation device”.

(Other embodiments)
Although a plurality of embodiments according to the present disclosure have been described above, the present disclosure is not construed as being limited to the above embodiments, and can be applied to various embodiments and combinations without departing from the gist of the present disclosure. can do.

  In the above embodiment, the user state has been described mainly focusing on sleepiness for convenience, but the state estimation method according to the present disclosure is not limited to the estimation of the user state such as sleepiness, tension, and concentration exemplified in the above embodiment. It is also applicable to estimation of other user states. The state estimation method according to the present disclosure is applied not only to the driver as in the third embodiment, but also to a dex worker who performs work in an office, a factory worker who performs work in a factory, and the like. Can be estimated. In the above-described embodiment, each function provided by a processor such as a wearable device can be provided by hardware and software different from the above-described configuration, or a combination thereof. That is, the specific configuration for performing the state estimation method is not limited to the wearable device, the smartphone, and the vehicle-mounted control device, and can be changed as appropriate.

  For example, the environment estimation process and the state estimation process may be performed by a processor of a processing circuit provided in the DMS 310 (see FIG. 8) of the third embodiment. Moreover, the environment estimation process and the state estimation process may be performed in a distributed manner by a plurality of configurations. Specifically, blink information acquisition and feature value calculation are performed by wearable devices and DMS, etc., and user environment estimation, feature value selection, and state estimation are performed by smartphones and in-vehicle control devices, etc. May be implemented. Furthermore, as the storage medium for storing the state estimation program, a non-transitional physical storage medium such as a flash memory and a hard disk drive may be appropriately employed.

  The state estimation unit 35 (see FIG. 3) of the above embodiment selects a single discriminator and then estimates the state using the selected discriminator. However, the state estimation unit may be configured to perform state estimation using one discriminator regardless of the selection result of the feature amount based on the user environment. Further, the machine learning technique for constructing the classifier is not limited to the random forest described above, and may be a support vector machine, a neural network, or the like.

  In the above embodiment, an example has been described in which the feature amount to be used is changed in view of the influence of light and wind around the user. However, the user environment estimated by the environment estimation unit may be changed as appropriate according to various factors that change the user's blink mode. For example, based on the knowledge that the mode of blinking changes before and after a meal, it may be determined whether or not the user has just eaten, and the feature amount used for state estimation may be changed.

  In the above embodiment, the environment estimation unit can detect the change in the user environment by monitoring the change in the specific feature amount calculated by the feature amount calculation unit. However, the environment estimation based on the change in the feature amount may not be performed. Alternatively, the user environment may be estimated based only on the change in the feature amount.

  In the above-described embodiment, blink detection and calculation of each feature amount are performed using the electrooculogram signal or the detection result of the eye opening. However, the configuration of the measurement unit that measures blinks is not limited to the electrodes and the near-infrared camera of the above-described embodiment, and can be changed as appropriate. In addition, blink detection and feature amount calculation may be performed using both the electrooculogram signal and the eye opening detection result. In addition, the number and content of feature amounts (parameters) calculated by the feature amount calculation unit can be changed as appropriate. Furthermore, the method for calculating the feature amount from the blink detection information is not limited to the method shown in FIG. 4 and can be changed as appropriate.

DESCRIPTION OF SYMBOLS 10 Wearable device (state estimation apparatus), 21,251,351 processor, 31 blink information acquisition part, 32 feature-value calculation part, 33 environment estimation part, 34 feature-value selection part, 35 state estimation part, 36 identifier, 240 Smartphone (state estimation device), 340 In-vehicle control device (state estimation device)

Claims (7)

A blink information acquisition unit (31) for acquiring blink detection information for detecting a user's blink;
A feature amount calculation unit (32) that calculates a plurality of feature amounts related to the user's blinks from the blink detection information;
An environment estimation unit (33) for estimating a user environment around the user;
A feature amount selection unit (34) for selecting the feature amount corresponding to the user environment estimated by the environment estimation unit from among the plurality of feature amounts calculated by the feature amount calculation unit;
A state estimation unit (35) for estimating the state of the user using the feature amount selected by the feature amount selection unit;
A state estimation device comprising: In the state estimation unit, a plurality of discriminators (36) respectively corresponding to combinations of the feature amounts selected by the feature amount selection unit are stored in advance.
The state estimation unit estimates the state of the user using the classifier corresponding to the selection result of the feature amount selection unit and the feature amount selected by the feature amount selection unit. The state estimation apparatus described in 1. The environment estimation unit determines whether or not the user feels dazzled based on environment information related to brightness around the user,
When the environment estimation unit determines that the user is in an environment where the user feels glare, the feature amount selection unit uses the feature amount set in advance as being easily affected by the glare. The state estimation apparatus according to claim 1 or 2, which is excluded from a target. The environment estimation unit determines whether or not the user is in an environment receiving wind based on environment information related to wind around the user,
The state estimation device according to any one of claims 1 to 3, wherein the feature amount selection unit excludes the feature amount set in advance as being easily affected by the wind from a state estimation use target.   The state estimation device according to any one of claims 1 to 4, wherein the environment estimation unit estimates the user environment based on at least one change in the feature amount calculated by the feature amount calculation unit. . The blink information acquisition unit acquires the blink detection information including at least a measurement result obtained by measuring the electrooculogram of the user,
The state estimation device according to claim 1, wherein the feature amount calculation unit calculates a value related to the blink strength of the user as the feature amount from the measurement result of electrooculogram. A state estimation method for estimating a user's state by processing of at least one processor (21, 251 and 351),
The blink detection information for detecting the blink of the user is acquired (S111),
A plurality of feature quantities related to the user's blink are calculated from the blink detection information (S112),
Estimating a user environment around the user (S102);
The feature amount corresponding to the estimated user environment is selected from the calculated plurality of feature amounts (S113),
A state estimation method for estimating the state of the user using the selected feature amount (S115).

Images