JP2018130342A - Wakefulness estimation device, wakefulness estimation method and wakefulness estimation system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To estimate drowsiness with little erroneous determination.SOLUTION: A wakefulness estimation device includes an eye opening acquisition part for acquiring an opening degree of an eye of a subject. The wakefulness estimation device includes a variation calculation part for calculating variation in opening degree of an eye in a prescribed time. The wakefulness estimation device includes a drowsiness determination part for determining drowsiness using the calculated variation and a prescribed threshold.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a wakefulness estimation device, a wakefulness estimation method, and a wakefulness estimation system.

  A technique is known in which a region including the eyes of a determination target person is imaged to determine the sleepiness state of the determination target person. For example, a technique for determining the sleepiness state of the determination target person based on a plurality of types of blink feature quantities is known. In the above technique, the blink feature amount is different from the blink feature amount related to the cluster blink, the blink feature amount related to the cluster blink, the length of the open eye state, the length of the closed eye state, etc. Use.

JP 2010-68848 A

  By the way, as sleepiness progresses, the blink occurrence rate increases, and a conflict state (hereinafter sometimes referred to as “wakefulness effort”) is likely to occur against sleepiness. When the awakening effort begins, humans become conscious of trying to open their eyes, and the blinking characteristics of high and low arousals are mixed. In the above technique, there is a case where the awakening state is erroneously determined in a situation where the blinking characteristics at the time of high awakening and the blinking at the time of low awakening are mixed.

  In one aspect, an object is to provide a wakefulness estimation device, a wakefulness estimation method, and a wakefulness estimation system that can estimate sleepiness with few erroneous determinations.

  In one aspect, the arousal level estimation device acquires the eye opening of the subject, calculates a variation in the eye opening within a predetermined time, and determines sleepiness using the calculated variation and a predetermined threshold. I do.

  According to one aspect, sleepiness estimation with few misjudgments can be performed.

FIG. 1 is a diagram illustrating an example of the arousal level estimation system according to the first embodiment. FIG. 2 is a flowchart illustrating an example of the arousal level estimation process according to the first embodiment. FIG. 3 is a diagram illustrating an example of a sleepiness estimation result in the background art. FIG. 4 is a diagram illustrating an example of sleepiness estimation results according to the first embodiment. FIG. 5 is a flowchart illustrating an example of the arousal level estimation process according to the second embodiment. FIG. 6 is a diagram illustrating an example of a sleepiness estimation result in the second embodiment. FIG. 7 is a diagram illustrating an example of a relationship between the backlight ratio and the face-captured image in the third embodiment. FIG. 8 is a diagram illustrating an example of the arousal level estimation system according to the third embodiment. FIG. 9 is a flowchart illustrating an example of the arousal level estimation process according to the third embodiment. FIG. 10 is a diagram illustrating an example of a computer that executes an arousal level estimation program.

  Hereinafter, embodiments of the arousal level estimation device, the arousal level estimation method, and the arousal level estimation system disclosed in the present application will be described in detail with reference to the drawings. Note that the present invention is not limited to the embodiments. Moreover, you may combine suitably each Example shown below in the range which does not cause contradiction.

[Function block]
In the present embodiment, a configuration will be described in which the driver's arousal level during driving of the vehicle is estimated based on a numerical value calculated using the eye opening. First, an example of the arousal level estimation system in the present embodiment will be described with reference to FIG. FIG. 1 is a diagram illustrating an example of the arousal level estimation system according to the first embodiment. As shown in FIG. 1, the arousal level estimation system 1 in the present embodiment includes a wakefulness level estimation device 100 and a sensor group 200. The arousal level estimation device 100 and the sensor group 200 are connected to each other by a network N such as a CAN (Controller Area Network) (not shown) so that they can communicate with each other. The arousal level estimation device 100 and the sensor group 200 are mounted on a moving body such as an automobile, for example.

  The sensor group 200 is a sensor for grasping the opening degree of the driver's eyes and the like. As an example of the sensor group 200, there is an in-vehicle camera that photographs a driver's eyes. Further, the sensor group 200 may be a camera that is mounted on a head mounted display (HMD (Head Mounted Display)) worn by the driver and photographs the eyes of the driver. The sensor group 200 may be a sensor such as a camera, a gyro sensor, or a myoelectric potential measuring device that acquires information related to the movement of the eyeball and the head direction of the driver.

  Next, the arousal level estimation apparatus 100 includes a communication unit 110, a storage unit 120, and a control unit 130. The communication unit 110 acquires data from the sensor group 200 and outputs the data to the control unit 130.

  The storage unit 120 stores various data such as a program executed by the control unit 130, for example. The storage unit 120 includes an eye opening degree storage unit 121, a variation storage unit 122, and a threshold value storage unit 123. The storage unit 120 corresponds to a semiconductor memory device such as a random access memory (RAM), a read only memory (ROM), and a flash memory, and a storage device such as a hard disk drive (HDD).

  The eye opening degree storage unit 121 stores data related to the eye opening calculated from various sensor data acquired from the sensor group 200 in association with the time. Information stored in the eye opening degree storage unit 121 is input by an eye opening degree acquisition unit 131 described later.

  Next, the variation storage unit 122 stores the variation of the stored eye opening value. The variation storage unit 122 stores a dispersion value of the eye opening value calculated for a predetermined period such as 5 minutes in association with the time. Note that the information stored in the variation storage unit 122 is input by the variation calculation unit 132 described later. In the following, the eye opening value may be referred to as “eye opening”.

  Next, the threshold value memory | storage part 123 memorize | stores the threshold value used in order to identify the level of arousal level from the dispersion | variation in the stored eye opening degree. In this embodiment, the threshold is, for example, a value obtained by multiplying a minimum value of the dispersion value of the eye opening within a predetermined period stored in the variation storage unit 122 by a predetermined coefficient, but is not limited to this. The configuration may be set by the user. Note that information stored in the threshold storage unit 123 is input by a threshold setting unit 133 described later.

  Next, the control unit 130 is a processing unit that performs overall processing of the arousal level estimation device 100. The control unit 130 is realized, for example, by executing a program stored in an internal storage device using a RAM as a work area by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like. Further, the control unit 130 may be realized by an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). The control unit 130 includes an eye opening degree acquisition unit 131, a variation calculation unit 132, a threshold setting unit 133, a drowsiness determination unit 134, and a warning output unit 135. The eye opening degree acquiring unit 131, the variation calculating unit 132, the threshold setting unit 133, the drowsiness determining unit 134, and the warning output unit 135 are an example of an electronic circuit included in the processor and an example of a process executed by the processor.

  The eye opening degree obtaining unit 131 obtains sensor data related to the eye opening degree such as image data obtained by photographing the driver's eyes from the sensor group 200 through the communication unit 110. The eye opening degree acquisition unit 131 calculates the opening degree of the driver's eyes by analyzing image data acquired using a known image analysis technique or the like. Then, the eye opening degree acquiring unit 131 stores the calculated eye opening degree in the eye opening degree storage unit 121 in association with the time when the image data is acquired.

  The variation calculation unit 132 refers to the eye opening storage unit 121 and calculates the variation in the driver's eye opening within a predetermined period. The variation calculation unit 132 acquires the eye opening of the driver within a predetermined period such as 5 minutes, and calculates a dispersion value of the eye opening as the variation in the eye opening. Then, the variation calculation unit 132 stores the calculated variation in the eye opening in the variation storage unit 122 in association with the time.

  Next, the threshold value setting unit 133 acquires information on the variation stored in the variation storage unit 122, and sets a threshold value used for arousal level estimation. The threshold setting unit 133 stores, for example, a value obtained by multiplying a minimum value among the obtained dispersion values of the eye opening within a predetermined period by a predetermined coefficient in the threshold storage unit 123 as a threshold. Note that the number of thresholds set by the threshold setting unit 133 varies depending on, for example, the number of levels of arousal level to be determined. For example, when the awakening level is classified into three levels, the threshold setting unit 133 sets two thresholds: a first threshold and a second threshold that is larger than the first threshold. In the following, the level of wakefulness may be referred to as “sleepiness level” or simply “level”. In the following, the drowsiness level 1 indicates that the driver is in a high arousal state, i.e., no drowsiness at all. The higher the drowsiness level, the lower the awake state, i.e., the stronger the drowsiness.

  Next, the drowsiness determination unit 134 determines the driver's arousal level using the information regarding the variation in the eye opening acquired from the variation storage unit 122 and the threshold acquired from the threshold storage unit 123. For example, the drowsiness determination unit 134 obtains information related to the variation in the eye opening from the variation storage unit 122 in real time and performs the determination process. For example, when the obtained variation in the eye opening exceeds the second threshold, the sleepiness determination unit 134 determines that the arousal level is “sleepiness level 3”. Similarly, the sleepiness determination unit 134 determines that the arousal level is “sleepiness level 2” when the obtained variation in the eye opening exceeds the first threshold value and is equal to or less than the second threshold value, and is acquired. If the variation in the measured eye opening is equal to or less than the first threshold, it is determined that the arousal level is “sleepiness level 1”. Then, the drowsiness determination unit 134 outputs the determination result to the warning output unit 135.

  Next, the warning output unit 135 outputs a warning to the driver using the determination result of the arousal level output from the drowsiness determination unit 134. For example, the warning output unit 135 determines whether or not the output result of the arousal level output from the sleepiness determination unit 134 is “sleepiness level 3”. The warning output unit 135 outputs a warning to the driver when it is determined that the output result of the arousal level is “sleepiness level 3”. The warning output by the warning output unit 135 is, for example, an alarm sound or a text message. In addition, the awakening level at which the warning output unit 135 outputs a warning is not limited to this. For example, when it is determined that the awakening level is “drowsiness level 2”, the warning output unit 135 outputs a warning. Alternatively, different warnings may be output depending on the arousal level.

[Process flow]
Next, wakefulness estimation processing by the wakefulness estimation system 1 in the present embodiment will be described with reference to FIG. FIG. 2 is a flowchart illustrating an example of the arousal level estimation process according to the first embodiment. As shown in FIG. 2, the eye opening degree obtaining unit 131 of the wakefulness estimation device 100 waits until an instruction to start the wakefulness estimation process is received from a user (not shown) through an operation unit (not shown), for example (S100: No). . If it is determined that the start process of the arousal level estimation process has been acquired (S100: Yes), the eye opening degree acquiring unit 131 includes, for example, information on the eye opening degree of the driver from the sensor group 200 through the communication unit 110. Get image data. Then, the eye opening degree acquiring unit 131 calculates the opening degree of the driver's eyes using the acquired image data, and stores it in the eye opening degree storage unit 121 (S101).

  Next, the variation calculation unit 132 refers to the eye opening degree storage unit 121, calculates a dispersion value A that is a dispersion value of the eye opening during a predetermined period, and stores the calculated value in the variation storage unit 122 (S <b> 102). Next, the threshold value setting unit 133 refers to the variance value A stored in the variation storage unit 122, calculates the first threshold value and the second threshold value, and stores them in the threshold value storage unit 123 (S103).

  Next, the drowsiness determination unit 134 determines whether or not the variance value A stored in the variation storage unit 122 exceeds the second threshold stored in the threshold storage unit 123 (S110). When it is determined that the variance A exceeds the second threshold (S110: Yes), the sleepiness determination unit 134 determines that the driver's arousal level is “sleepiness level 3” (S111). And warning output part 135 outputs a warning to a driver (S112), and shifts to S130.

  On the other hand, when it is determined that the variance value A is equal to or less than the second threshold value (S110: No), the drowsiness determination unit 134 determines whether the variance value A exceeds the first threshold value (S120). When it is determined that the variance A exceeds the first threshold (S120: Yes), the sleepiness determination unit 134 determines that the driver's arousal level is “sleepiness level 2” (S121), and proceeds to S130. . On the other hand, when it is determined that the variance A is equal to or less than the first threshold (S120: No), the sleepiness determination unit 134 determines that the driver's arousal level is “sleepiness level 1” (S123), and S130. Migrate to

  Next, the eye opening degree obtaining unit 131 determines whether or not an instruction to end the arousal level estimation process has been received from the user, for example, through the operation unit (S130). If the eye opening degree obtaining unit 131 determines that an instruction to end the arousal level estimation process is not received (S130: No), the process returns to S101 and repeats the process. On the other hand, if the eye opening degree obtaining unit 131 determines that an instruction to end the arousal level estimation process has been received (S130: Yes), the awakening level estimation process ends.

[effect]
As described above, the arousal level estimation apparatus according to the present embodiment acquires the eye opening of the subject, calculates the variation in the eye opening within a predetermined time, and calculates the calculated variation and the predetermined threshold value. The drowsiness is determined using. Thereby, even when the blink pattern at the time of low awakening and the blink pattern at the time of high awakening are mixed, such as when an awakening effort occurs in the low arousal state, sleepiness estimation with less misjudgment can be performed. That is, it is possible to prevent a warning from being leaked to the driver even when the driver is waking up. Moreover, since the arousal level can be estimated using the variance value of the eye opening as the feature amount, the amount of calculation for estimating the arousal level can be reduced.

  The effect in the present embodiment will be described with reference to FIGS. FIG. 3 is a diagram illustrating an example of a sleepiness estimation result in the background art. In FIG. 3, a graph 3000 shows a time-series change in the driver's eye opening. In graph 3000, the horizontal axis indicates the passage of time by the number of frames of the captured image, and the vertical axis indicates the eye opening of the driver. On the vertical axis of the graph 3000, “0” indicates the closed eye state, “100” indicates the standard open eye state, and the larger the number, the larger the eye opening. Periods 3011 and 3012 show examples of periods during which it is determined that the driver's eye opening is low.

  In FIG. 3, a graph 3100 shows a time-series change in the driver's blink feature amount calculated based on the driver's eye opening shown in the graph 3000. FIG. 3 shows an example in which PERCLOS (percentage of eyes closed for 1 minute) is used as the feature amount of the driver's blink.

  Next, in FIG. 3, a graph 3200 shows a time-series change in the sleepiness correct value that is a result of measuring the driver's sleepiness. The sleepiness correct value is, for example, a value that is subjectively determined by visually observing the facial expression of the driver and the movement of the eyeball, such as the NEDO evaluation value, but is not limited thereto, and the driver's brain wave or electrooculogram, You may calculate using the value which measured heart rate etc.

  In the graph 3200, sleepiness correct values are shown in three stages of sleepiness level 1 to sleepiness level 3. For example, in FIG. 3, frames 3001, 3002, and 3003 indicate time periods in which the driver's drowsiness correct value is “sleepiness level 3”, that is, the driver's sleepiness is very strong.

  Next, in FIG. 3, a graph 3300 shows a time-series change in the result of determining the driver's sleepiness using PERCLOS shown in the graph 3100. Similarly to the graph 3100, the graph 3300 shows sleepiness in three stages of “sleepiness level 1” to “sleepiness level 3”. Graph 3300 shows an example in which the driver's sleepiness is determined to be sleepiness level 1 when PERCLOS in graph 3100 is equal to or lower than the first threshold value indicated by straight line 3101 in graph 3100. Similarly, graph 3300 shows an example in which the driver's sleepiness is determined to be sleepiness level 3 when PERCLOS in graph 3100 exceeds the second threshold value indicated by straight line 3102 in graph 3100. Graph 3300 shows an example in which the driver's sleepiness is determined to be sleepiness level 2 when PERCLOS in graph 3100 exceeds the first threshold and is equal to or less than the second threshold.

  Here, PERCLOS in the periods 3111 and 3112 corresponding to the periods 3011 and 3012 is equal to or less than the first threshold indicated by the straight line 3101 as shown in the graph 3100. That is, PERCLOS does not necessarily increase even during a period in which the sleepiness correct value is “sleepiness level 3”. In this case, as shown in periods 3311 and 3312 corresponding to the periods 3011 and 3012, the driver's sleepiness is determined to be “sleepiness level 1”. However, in the graph 3200, the sleepiness correct value in the periods 3211 and 3212 corresponding to the periods 3011 and 3012 is “sleepiness level 3”, and the determination result and the sleepiness correct value are different.

  In the periods 3111 and 3112 of the graph 3100, the driver's awakening effort has started, and the driver is actually trying to open his eyes even though the driver's sleepiness is very strong. Can be guessed. In such a case, when PERCLOS or the like is used as in the background art, the arousal level may be erroneously estimated.

  Next, the result of estimating the driver's arousal level in the present embodiment will be described with reference to FIG. FIG. 4 is a diagram illustrating an example of sleepiness estimation results according to the first embodiment. In FIG. 4, a graph 3000 shows a time-series change in the driver's eye opening as in FIG. 3. Next, a graph 3500 in FIG. 4 shows a time-series change in the variance value of the driver's eye opening shown in the graph 3000. That is, the graph 3500 shows a time-series change in the value of how much the driver's eye opening has changed in a predetermined period such as 5 minutes.

  In the graph 3500, straight lines 3501 and 3502 indicate the first threshold value and the second threshold value, respectively, similarly to the straight lines 3101 and 3102 in the graph 3100 of FIG. Next, a graph 3600 shows a result of determining the driver's sleepiness using the variance value of the driver's eye opening and the threshold values 3501 and 3502. Similarly to the graphs 3100 and 3300, the graph 3600 shows sleepiness in three stages of “sleepiness level 1” to “sleepiness level 3”.

  For example, in the background art, as shown in the graph 3100 of FIG. 3, in the periods 3111 and 3112 corresponding to the periods 3011 and 3012 of the graph 3000, the PERCLOS is smaller than the first threshold value. It is erroneously determined as “sleepiness level 1”. On the other hand, in a period in which the sleepiness correct value is “sleepiness level 3”, the variance value of the eye opening is often high, and the variance value of the eye opening tends to increase as the sleepiness level increases. For example, in periods 3511 and 3512 corresponding to the periods 3011 and 3012, the dispersion value of the eye opening degree is maintained at least larger than the first threshold 3501. For this reason, in the periods 3611 and 3612 corresponding to the periods 3011 and 3012 of the graph 3000, the sleepiness determination result in this embodiment is “sleepiness level 2” or “sleepiness level 3”. That is, in this embodiment, the arousal level can be estimated more appropriately without being erroneously determined to be “sleepiness level 1” as in the determination results in the periods 3311 and 3312 in the background art.

  As described above, the arousal level estimation apparatus according to the present embodiment estimates a state where the dispersion value of the eye opening over a long period of time, such as 5 minutes, exceeds a predetermined threshold value as a drowsiness state. Misjudgments can be suppressed with a small amount of calculation even during awakening efforts where blink patterns are mixed with low awakenings.

  In Example 1, although the structure which estimates a driver | operator's alertness using the dispersion | variation value of the opening degree of a driver | operator's eye was demonstrated, embodiment is not restricted to this. In the present embodiment, the arousal level estimation device that corrects the driver's arousal level estimated in addition to the variance value A of the driver's eye opening using a variance value B that is a variance value of the variance value A. Will be described.

[Function block]
First, an example of the arousal level estimation system 2 in the present embodiment will be described. In the following embodiments, the same parts as those shown in the drawings described above are denoted by the same reference numerals, and redundant description is omitted. The awakening level estimation system 2 is not shown.

  The awakening level estimation system 2 in the present embodiment includes a waking level estimation device 500 and a sensor group 200. The awakening level estimation device 500 includes a communication unit 110, a storage unit 520, and a control unit 530.

  The storage unit 520 stores various data such as a program executed by the control unit 530, for example. The storage unit 520 includes an eye opening degree storage unit 121, a variation storage unit 522, and a threshold value storage unit 523.

  The variation storage unit 522 further stores the variation in the variation in the eye opening calculated within the predetermined period in addition to the variation in the eye opening. Note that information stored in the variation storage unit 522 is input by a variation calculation unit 532 described later.

  Next, the threshold storage unit 523 specifies the arousal level from the variation in the eye opening variation in addition to the threshold value used for identifying the arousal level from the stored variation in the eye opening degree. The threshold value used for the is further stored. In this embodiment, the threshold value regarding the variation in the variation in the eye opening is, for example, a value obtained by multiplying the minimum value of the dispersion value B within the predetermined period stored in the variation storage unit 522 by a predetermined coefficient. The configuration is not limited to this, and may be set in advance by the user. Information stored in the threshold storage unit 523 is input by a threshold setting unit 533 described later.

  Next, the control unit 530 is a processing unit that performs overall processing of the arousal level estimation device 500. The control unit 530 includes an eye opening degree acquisition unit 131, a variation calculation unit 532, a threshold setting unit 533, a drowsiness determination unit 534, and a warning output unit 135.

  The variation calculation unit 532 refers to the eye opening storage unit 121 and the variation storage unit 522, and further calculates the variation in the eye opening variation in addition to the variation in the driver's eye opening within the predetermined period. The variation calculation unit 532 acquires, for example, a variance value of the driver's eye opening within a predetermined period such as 10 minutes from the variation storage unit 522 as the variation of the dispersion value of the eye opening, and acquires the acquired eye opening. The variance value of the variance value of is calculated. Then, the variation calculation unit 532 stores the calculated variation in the eye opening variation in the variation storage unit 522 in association with the time. Note that the period for acquiring the eye opening for calculating the dispersion value of the variation may be set wider than the period for acquiring the eye opening for calculating the variation.

  Next, the threshold setting unit 533 sets a threshold relating to variation in variation in eye opening used for arousal level estimation in addition to a threshold relating to variation in eye opening. The threshold setting unit 533 uses, for example, a value obtained by multiplying a minimum value among the dispersion values of the eye opening dispersion values within a predetermined period acquired by a predetermined coefficient as a threshold relating to variations in eye opening dispersion. Store in the threshold storage unit 523. In the present embodiment, the threshold regarding the variation in the variation of the eye opening is not used for the arousal level estimation process itself, but is used for correcting the estimated arousal level, and is not necessarily a determination target. It does not always change according to the number of awakening levels. In the present embodiment, an example will be described in which the threshold setting unit 533 sets only one threshold relating to the variation in the variation in eye opening.

  Next, the drowsiness determination unit 534 determines the driver's arousal level and uses the threshold value regarding the variation in the variation in the eye opening and the variation in the variation in the eye opening to determine the driver's arousal level. Correct. For example, when it is determined that the arousal level is “sleepiness level 2”, the drowsiness determination unit 534 acquires information on the variation in the variation in the eye opening from the variation storage unit 522. The sleepiness determination unit 534 corrects the wakefulness level to “sleepiness level 3” when, for example, the obtained variation in the variation in the eye opening exceeds the third threshold value. The drowsiness determination unit 534 outputs the corrected determination result to the warning output unit 135. The sleepiness determination unit 534 determines that the degree of arousal is “sleepiness level 1” or “sleepiness level 3”, or determines that the variation in the obtained variation in eye opening is equal to or less than the third threshold value. In this case, the determination result is output to the warning output unit 135 without correction.

[Process flow]
Next, the flow of processing in the present embodiment will be described. FIG. 5 is a flowchart illustrating an example of the arousal level estimation process according to the second embodiment. In the following description, the same reference numerals as those shown in FIG. 2 are the same steps, and detailed description thereof is omitted.

  The threshold value setting unit 533 of the awakening level estimation device 500 calculates the first threshold value and the second threshold value, and stores them in the threshold value storage unit 523 (S103). Next, the variation calculation unit 532 acquires a dispersion value A that is a variation in the eye opening within a predetermined period from the variation storage unit 522. Then, the variation calculation unit 532 calculates a dispersion value B, which is a dispersion value of the dispersion value A, as a variation in the variation in the eye opening, and stores it in the variation storage unit 522 (S104). Next, the threshold value setting unit 533 calculates the third threshold value with reference to the dispersion value B stored in the variation storage unit 522, and stores it in the threshold value storage unit 523 (S105).

  Moving to Step S121, when the sleepiness determination unit 534 determines that the arousal level is “sleepiness level 2”, the sleepiness determination unit 534 determines whether or not the variance value B exceeds the third threshold (S125). If the drowsiness determination unit 134 determines that the variance B is equal to or less than the third threshold (S125: No), the process proceeds to S130. On the other hand, if it is determined that the variance B exceeds the third threshold (S125: Yes), the sleepiness determination unit 534 corrects the driver's arousal level to “sleepiness level 3” (S126), and the corrected determination The result is output to the warning output unit 135. And warning output part 135 outputs a warning to a driver (S112), and shifts to S130.

[effect]
As described above, the arousal level estimation apparatus according to the present embodiment further calculates the variance value of the calculated variation, and performs sleepiness determination using the variance value of the variation. Thereby, in the case where the variation of the eye opening is large, sleepiness estimation with less erroneous determination can be performed.

  The effect in the present embodiment will be described with reference to FIG. FIG. 6 is a diagram illustrating an example of a sleepiness estimation result in the second embodiment. In FIG. 6, a graph 3500 and a graph 3600 are sleepiness estimation results using the variance value of the driver's eye opening shown in FIG. 4 and the variance value of the driver's eye opening in the first embodiment, respectively.

  On the other hand, in FIG. 6, a graph 3700 shows a time-series change in the dispersion value of the dispersion value of the eye opening shown in the graph 3500. That is, the graph 3700 shows a time-series change in the value of how much the variance value of the driver's eye opening has changed in a predetermined period such as 10 minutes. In the graph 3700, a straight line 3701 indicates a third threshold value.

  Next, the graph 3800 shows the result of correcting the determination result of the driver's sleepiness shown in the graph 3600 using the variance value of the variance value of the driver's eye opening and the threshold value 3701. In addition, periods 3511, 3611, 3711, and 3811 in FIG. 6 indicate the same period. Similarly, the periods 3512, 3612, 3712, and 3812 in FIG. 6 also indicate the same period.

  For example, in the drowsiness determination result in the first embodiment shown in the graph 3600, there is a point in time periods 3611 and 3612 when the variance of the eye opening shown in the graph 3500 is less than or equal to the second threshold and exceeds the first threshold. To do. At this time, the driver's sleepiness is determined to be “sleepiness level 2”.

  On the other hand, in the periods 3711 and 3712 of the graph 3700 corresponding to the periods 3611 and 3612, there is a time point when the variance value of the variance value of the eye opening exceeds the third threshold value. At such time, the sleepiness determination result in this embodiment is corrected from “sleepiness level 2” to “sleepiness level 3”. That is, as shown in the periods 3811 and 3812 of the graph 3800, in this example, when the sleepiness correct value is “sleepiness level 3”, the time zone in which it is accurately determined that the sleepiness level 3 is “sleepiness level 3”. Can be expanded. Thereby, in the case where the variation of the eye opening is large, sleepiness estimation with less erroneous determination can be performed.

  In addition, although the present Example demonstrated the structure used for determination whether the threshold value regarding the dispersion | variation in eye opening variation | mutation is corrected for the determined arousal level, embodiment is not restricted to this. For example, when the sleepiness determination unit 534 determines that the wakefulness is “sleepiness level 2” when the dispersion value A shown in FIG. 5 is equal to or less than the second threshold and exceeds the first threshold, The configuration may be such that it is determined whether or not the value B exceeds the third threshold value. In other words, a configuration may be used in which the threshold value regarding the variation in the variation in the eye opening is used for the determination of the arousal level.

  When estimating the degree of arousal using the driver's eye opening, various disturbance factors can be considered. For example, in backlighting, the driver may narrow his eyes compared to in direct lighting. In this case, an erroneous determination may occur in the estimation of the arousal level using the variation in the eye opening. Therefore, in this embodiment, a configuration for estimating the driver's arousal level in consideration of disturbance factors during backlighting will be described.

  First, the influence of the backlight on the eye opening of the driver will be described with reference to FIG. FIG. 7 is a diagram illustrating an example of a relationship between the backlight ratio and the face-captured image in the third embodiment. FIG. 7 shows an example in which images of the same driver driving the same vehicle in different backlight conditions are compared. An image 8101 in FIG. 7 is an example of an image when the backlight ratio is “67%”. Similarly, an image 8201 in FIG. 7 is an example of an image when the backlight rate is “100%”, and an image 8301 is an example of an image when the backlight rate is “8.7%”.

  In the image 8301 in the case where the backlight ratio is low, as shown in the eye detection region 8311, the driver keeps the same eye opening as usual. On the other hand, in the image 8101 in the case where the backlight ratio is higher, as shown in the eye detection region 8111, the driver's eye opening is smaller than that in the image 8301. In this case, since it becomes difficult for the driver to widen his / her eyes due to the influence of backlight, the variation in the eye opening may not be calculated appropriately. In addition, when the backlight rate changes in a short period, the variation in the eye opening may be larger than in a normal case.

  When the backlight rate as shown in the image 8101 is high, if the same arousal level estimation process as that when the backlight rate as shown in the image 8301 is low is performed, the variation in the eye opening may not be calculated appropriately. In some cases, misjudgment of arousal level may occur. Therefore, in this embodiment, a wakefulness level estimation device that performs a wakefulness level estimation process that reflects a disturbance factor caused by backlighting will be described.

  When the backlight rate is extremely high, the eye opening may not be clearly specified in the first place, as shown in the eye detection area 8211 of the image 8201. In this case, since the variation in the eye opening may not be calculated correctly, a configuration in which the arousal level estimation process itself is stopped may be used.

[Function block]
First, an example of the arousal level estimation system 3 in the present embodiment will be described. In the following embodiments, the same parts as those shown in the drawings described above are denoted by the same reference numerals, and redundant description is omitted. FIG. 8 is a diagram illustrating an example of the arousal level estimation system according to the third embodiment.

  The awakening level estimation system 3 in the present embodiment includes a waking level estimation device 600 and a sensor group 900. The arousal level estimation device 600 and the sensor group 900 are connected to each other via a network N such as a CAN (not shown) so as to communicate with each other. The arousal level estimation device 600 and the sensor group 900 are mounted on a moving body such as an automobile.

  The sensor group 900 includes a sensor for specifying a backlight state in an image capturing the driver, in addition to a sensor for grasping the opening degree of the driver's eyes and the like. As an example of a sensor for specifying a backlight state, there is an illuminance sensor installed on a front window, a sun visor, a headrest or the like of an automobile. In addition, the sensor group 900 includes a GPS (Global Positioning System) positioning sensor, an RTC (Real-Time Clock), a weather sensor, and the like for estimating a traveling direction, a sun direction, weather, and the like used for estimating a backlight state. May be included. Instead of using an independent sensor for measuring luminance or illuminance, a backlight detection unit 636, which will be described later, analyzes a photographed driver's image, and locally analyzes a portion corresponding to the driver's eye. The configuration may be such that the backlight ratio is determined using high brightness.

  The awakening level estimation device 600 includes a communication unit 110, a storage unit 620, and a control unit 630. The storage unit 620 stores various data such as a program executed by the control unit 630, for example. In addition, the storage unit 620 includes an eye opening degree storage unit 621, a variation storage unit 622, a threshold storage unit 623, and a backlight state storage unit 624.

  The eye opening degree storage unit 621 stores data relating to the eye opening degree calculated from various sensor data acquired from the sensor group 900 in association with the time. In this embodiment, the eye opening degree storage unit 621 stores, for example, a weight set according to the backlight state in association with the eye opening degree. Moreover, the eye opening degree memory | storage part 621 may memorize | store the eye opening degree corrected according to the backlight state, for example. Information stored in the eye opening degree storage unit 621 is input by an eye opening degree obtaining unit 631 described later.

  Next, the variation storage unit 622 stores the stored variation in the eye opening value. In the present embodiment, the variation storage unit 622 may store variations calculated using, for example, a weighted or corrected eye opening according to the backlight state. Information stored in the variation storage unit 622 is input by a variation calculation unit 632 described later.

  Next, the threshold storage unit 623 stores a threshold used to specify the level of arousal level from the stored variation in eye opening. Note that information stored in the threshold storage unit 623 is input by a threshold setting unit 633, which will be described later, or is corrected according to the backlight state.

  Next, the backlight state storage unit 624 stores time-series changes in the backlight state. The backlight state storage unit 624 stores, for example, the backlight ratio, the backlight disturbance degree calculated based on the backlight ratio, and the like in association with the time. Information stored in the backlight state storage unit 624 is input by the backlight detection unit 636 described later.

  Next, the control unit 630 is a processing unit that performs overall processing of the arousal level estimation device 600. The control unit 630 includes an eye opening degree acquisition unit 631, a variation calculation unit 632, a threshold setting unit 633, a drowsiness determination unit 634, a warning output unit 135, and a backlight detection unit 636.

  The eye opening degree acquiring unit 631 calculates the opening degree of the eyes of the driver and stores the eye opening degree storing unit 621 in the same manner as the eye opening degree acquiring unit 131 in the first embodiment. The eye opening degree acquiring unit 631 refers to, for example, the backlight state storage unit 624, acquires the backlighting rate at the time corresponding to the eye opening degree, and corrects the eye opening degree according to the backlighting rate. The result is stored in the eye opening degree storage unit 621.

  For example, as shown in FIG. 7, when the backlight rate increases, the driver's eye opening may decrease. Therefore, the eye opening acquisition unit 631 may correct the eye opening itself using the backlight ratio, for example, by dividing the calculated eye opening by the backlight ratio.

  In addition, when trying to calculate the dispersion value using the eye opening when the backlight rate is high, the dispersion value of the eye opening is not properly calculated, and the dispersion value of the eye opening and the sleepiness correct value diverge. There is a case. Therefore, the eye opening degree acquisition unit 631 may store the calculated eye opening degree in the eye opening degree storage unit 621 in association with weighting using the backlight disturbance degree. For example, in the case where the backlight disturbance level is set in three stages from “1” to “3”, the eye opening degree obtaining unit 631 sets the weight to “1” when the backlight disturbance level is “1”. Similarly, the eye opening degree acquiring unit 631 sets the weight to “0.5” when the backlight disturbance degree is “2”, and sets the weight to “0.2” when the backlight disturbance degree is “3”. To do.

  In this case, for example, when the variation calculation unit 632 calculates the variation in the eye opening, the degree of contribution of the eye opening at the time when the backlight disturbance level is high is reduced. For example, the degree of contribution of the eye opening at the time when “0.5” is weighted because the backlight disturbance degree is high is 1 / of the contribution of the eye opening at the time when the backlight disturbance degree is low and not weighted. 2. In this case, it is possible to suppress calculation of an inappropriate dispersion value of the eye opening as compared with a case where weighting using the backlight disturbance degree is not performed.

  For example, when the backlight ratio is very high and the eye opening itself cannot be detected, or when the detected eye opening is not appropriate, the weight may be set to “0”. Thereby, it can suppress that the dispersion | distribution value of an eye opening is calculated using an inappropriate eye opening. In addition, since the variation calculation unit 632 may not obtain a sufficient number of samples of the eye opening within a predetermined period when the opening is weighted, the eye opening storage unit over a period longer than the predetermined period. The eye opening may be acquired from 621.

  The variation calculation unit 632 refers to the eye opening degree storage unit 621 and calculates the variation of the driver's eye opening within a predetermined period. For example, the variation calculation unit 632 weights the eye opening using the weight stored in the eye opening storage unit 621 in association with the eye opening, thereby reflecting the effect of the backlight disturbance degree. Calculate the degree of variance. The variation calculation unit 632 stores the calculated variation in the eye opening in the variation storage unit 622 in association with the time.

  Next, the threshold setting unit 633 acquires information regarding the variation stored in the variation storage unit 622, sets a threshold used for estimating the arousal level, and stores the threshold in the threshold storage unit 623. Further, the threshold setting unit 633 may correct the set threshold using, for example, a backlight disturbance degree.

  Next, the drowsiness determination unit 634 determines the driver's arousal level using the information regarding the variation in the eye opening and the threshold value, and outputs the determination result to the warning output unit 135. In addition, the drowsiness determination unit 634 may correct the determination result according to the backlight disturbance level. For example, when the backlight disturbance level exceeds a predetermined threshold, the sleepiness determination unit 634 decreases the sleepiness level as a determination result by one level.

  Next, the backlight detection unit 636 detects the backlight disturbance level using the sensor data acquired from the sensor group 900 through the communication unit 110. The backlight detection unit 636 acquires, for example, an image such as the image 8101 in FIG. 7 as sensor data, and detects the luminance of each pixel in the eye detection region 8111. The backlight detection unit 636 calculates, as the backlight ratio, the ratio of pixels whose luminance exceeds a predetermined threshold in the eye detection region 8111, and calculates the backlight disturbance degree based on the backlight ratio.

  The backlight ratio is calculated by, for example, the following formula (1).

  Backlight factor = 100 × (overThreCnt / allCnt) /0.5 (1)

  In Expression (1), “Thre” represents a predetermined threshold value. In addition, “overThreCnt” indicates the number of pixels whose luminance is “Thre” or more, and “allCnt” indicates the number of all pixels included in the target detection area. The predetermined threshold “Thre” is calculated by, for example, the following formula (2).

  Thre = maximum luminance value−0.15 × (maximum luminance value−minimum luminance value) Expression (2)

  In Equation (2), “maximum luminance value” indicates the luminance value of the pixel having the highest luminance in the target detection region, and “minimum luminance value” indicates the highest luminance in the target detection region. Represents the luminance value of a low pixel. That is, in the present embodiment, the backlight ratio is calculated by the formula (1) based on the ratio of the number of pixels having a high luminance value of 15% from the maximum luminance. Note that the backlight ratio in the present embodiment is normalized so that the backlight ratio becomes “100%” when the ratio of the pixels whose luminance is “Thre” or more is “50%” or more. However, the value for normalizing the backlight ratio and the threshold “Thre” may be arbitrarily changed.

  Next, the backlight detection unit 636 calculates the backlight disturbance level using the calculated backlight ratio at each time and the change thereof. For example, the backlight detection unit 636 calculates the average value of the backlight ratio within a predetermined time, the ratio of the time when the backlight ratio is equal to or greater than a predetermined value within the predetermined time, and the like as the backlight disturbance degree. For example, the backlight detection unit 636 specifies that the backlight disturbance degree is “1” when the average value of the backlight ratio in “5 seconds” is less than “50%”. Similarly, when the average value of the backlight ratio in “5 seconds” is “50%” or more and less than “75%”, the backlight detector 636 determines that the backlight disturbance degree is “2” or “75%” or more. The backlight disturbance level is set to “3”. Then, the backlight detection unit 636 stores the calculated backlight disturbance degree in the backlight state storage unit 624 in association with the time.

[Process flow]
Next, the flow of processing in the present embodiment will be described. FIG. 9 is a flowchart illustrating an example of the arousal level estimation process according to the third embodiment. In the following description, the same reference numerals as those shown in FIG. 2 are the same steps, and detailed description thereof is omitted.

  The backlight detection unit 636 of the arousal level estimation apparatus 600 acquires, from the sensor group 900, the image data including, for example, information related to the luminance of the peripheral region of the driver's eyes through the communication unit 110 (S301). Next, the backlight detection unit 636 calculates the backlight rate using the luminance information included in the acquired image data, calculates the backlight disturbance degree Y based on the backlight rate, and stores it in the backlight state storage unit 624 ( S302).

  Next, the eye opening degree obtaining unit 631 obtains, for example, image data including information related to the opening degree of the driver's eyes from the sensor group 900 through the communication unit 110 and calculates the opening degree of the driver's eyes. It stores in the eye opening degree storage unit 621 (S303). Next, the eye opening degree obtaining unit 631 obtains the backlight disturbance degree Y at the time corresponding to the stored eye opening degree from the backlight state storage unit 624, calculates the weight, and stores it in association with the eye opening degree. (S304).

  Next, the variation calculation unit 632 calculates a dispersion value A, which is a dispersion value of the eye opening in a predetermined period, using a value obtained by multiplying the eye opening stored in the eye opening storage unit 621 and the weight. And stored in the variation storage unit 622 (S305). Next, the threshold setting unit 633 refers to the variance value A stored in the variation storage unit 622, calculates the first threshold value and the second threshold value, and stores them in the threshold value storage unit 623 (S306).

  Next, when it is determined that “sleepiness level 3” is determined in step S111, or when it is determined that “sleepiness level 2” is determined in step S122, the sleepiness determination unit 634 refers to the backlight state storage unit 624. Then, the average value Ya of the backlight disturbance degree Y is calculated (S320). For example, the drowsiness determination unit 634 calculates an average value Ya of the backlight disturbance degree Y for 5 seconds before the determination time. Then, the sleepiness determination unit 634 determines whether or not the average value Ya exceeds the fourth threshold value (S321). If the sleepiness determination unit 634 determines that the average value Ya exceeds the fourth threshold (S321: Yes), the sleepiness determination unit 634 lowers the determined sleepiness level (S322), and proceeds to S330. On the other hand, if the sleepiness determination unit 634 determines that the average value Ya is equal to or less than the fourth threshold value (S321: No), it outputs a warning to the driver (S323), and proceeds to S330.

  Then, the threshold setting unit 633 refers to the backlight state storage unit 624 and determines whether or not a change in the backlight disturbance level Y is detected (S330). When it is determined that the change in the backlight disturbance level Y has been detected (S330: Yes), the threshold setting unit 633 corrects the threshold stored in the threshold storage unit 623 (S331), and proceeds to S130. On the other hand, if the threshold setting unit 633 determines that the change in the backlight disturbance level Y is not detected (S330: No), the process proceeds to S130.

[effect]
As described above, the arousal level estimation apparatus according to the present embodiment detects a backlight disturbance state with respect to the subject, and performs drowsiness determination by further using the detected backlight disturbance state. Thereby, the disturbance factor by backlight can be considered in sleepiness estimation processing.

  In the present embodiment, an example in which the dispersion value of the eye opening is used as the variation used for the drowsiness determination is not limited to this, but the dispersion value of the dispersion value of the eye opening shown in Embodiment 2 and the like are further increased. The structure used may be used.

  Although the embodiments of the present invention have been described so far, the present invention may be implemented in various different forms other than the embodiments described above. For example, in each of the above-described embodiments, the configuration in which the alertness estimation system 1 estimates the alertness of the driver of the car has been described. However, the present invention is not limited to this, and the driver of other mobile objects such as a ship, a railway, and an aircraft It may be configured to estimate the arousal level. Moreover, the structure which estimates not only a driver | operator but the arousal level of the operator etc. in a business establishment may be sufficient.

  When the awakening effort begins, humans often try to open their eyes larger than usual. Therefore, in each of the above embodiments, the arousal level estimation device may further determine whether the average value or maximum value of the eye opening within a predetermined period is equal to or greater than a predetermined threshold value. In this case, the arousal level estimation apparatus according to the present embodiment determines that the magnitude of variation is equal to or greater than a predetermined threshold value and that the average or maximum value of the eye opening is equal to or greater than the predetermined threshold value. The subject is determined to be sleepy. As a result, it is possible to more accurately determine that the driver's awakening effort has started.

  In this case, the wakefulness level estimation device can calculate the average or maximum eye opening degree within a predetermined period of the driver even when the wakefulness level is estimated to be “sleepiness level 1” or “sleepiness level 2”. If it is determined that the value is greater than or equal to a predetermined threshold, a warning may be output. As a result, when the driver's awakening effort starts, a warning can be output accurately.

  Further, as variation used by the arousal level estimation device for drowsiness determination, the dispersion value of the eye opening is shown in Example 1, and the dispersion value of the dispersion value of the eye opening is shown in Example 2. Is not limited to this. For example, the arousal level estimation device may be a frequency at which the deviation of the eye opening, the difference between the maximum value and the minimum value of the eye opening, and the difference in the eye opening expand beyond a predetermined threshold. .

  The sleepiness determination unit 134 may store the determination result in a determination result storage unit (not shown). Thereby, for example, the threshold value can be calibrated by comparing the accumulated determination result and the drowsiness correct value.

[system]
Each component of each part shown in the figure is functionally conceptual and does not necessarily need to be physically configured as shown. In other words, the specific form of distribution / integration of each unit is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed / integrated in arbitrary units according to various loads or usage conditions. Can be configured. For example, a configuration in which the eye opening degree acquisition unit 131 or the warning output unit 135 of the arousal level estimation device 100 is realized by another device may be used. Further, the awakening level estimation device 100 may not have the threshold value setting unit 133 and may use a preset threshold value. Furthermore, various processing functions performed by each device may be executed entirely or arbitrarily on a CPU (or a microcomputer such as an MPU or MCU (Micro Controller Unit)). In addition, various processing functions may be executed in whole or in any part on a program that is analyzed and executed by a CPU (or a microcomputer such as an MPU or MCU) or on hardware based on wired logic. Needless to say, it is good.

  For example, the function of the arousal level estimation apparatus 100 may be implemented by a server outside the vehicle that can communicate with the automobile. In this case, the arousal level estimation apparatus 100 acquires sensor data transmitted by the sensor group 200 mounted on the automobile through the communication unit 110, and performs the arousal level estimation process.

  Moreover, the structure which the sensor group 200 is also mounted in the arousal level estimation apparatus 100 mounted in the motor vehicle may be sufficient. For example, the drive recorder may be configured to take an image of a driving driver and perform arousal level estimation processing.

[Awakening degree estimation program]
The various processes described in the above embodiments can also be realized by executing a program prepared in advance on a computer system such as a personal computer or a workstation. Therefore, in the following, an example of a computer system that executes a program having the same function as in the above embodiment will be described. FIG. 10 is a diagram illustrating an example of a computer that executes an arousal level estimation program. In the following, the arousal level estimation apparatus 100 according to the first embodiment will be described, but the awakening level estimation apparatuses 500 and 600 can be realized by a similar computer.

  As illustrated in FIG. 10, the computer 300 includes a CPU 310, an HDD 320, and a RAM 340. These units 300 to 340 are connected via a bus 400.

  The HDD 320 includes a wakefulness level estimation program 320a that performs the same functions as the eye opening degree acquisition unit 131, the variation calculation unit 132, the threshold value setting unit 133, the drowsiness determination unit 134, and the warning output unit 135 of the wakefulness level estimation device 100 described above. Stored in advance. The arousal level estimation program 320a may be separated as appropriate.

  The HDD 320 stores various information. For example, the HDD 320 stores eye opening degree data 320b, variation data 320c, and threshold value data 320d stored in each storage unit illustrated in FIG.

  Then, the CPU 310 reads the arousal level estimation program 320a from the HDD 320, expands it in the RAM 340, and executes each process using the eye opening degree data 320b, the variation data 320c, and the threshold value data 320d stored in the HDD 320. That is, the arousal level estimation program 320a performs the same operations as the eye opening degree acquisition unit 131, the variation calculation unit 132, the threshold setting unit 133, the drowsiness determination unit 134, and the warning output unit 135.

  Note that the awakening level estimation program 320a is not necessarily stored in the HDD 320 from the beginning.

  For example, the program is stored in a “portable physical medium” such as a flexible disk (FD), a CD-ROM, a DVD disk, a magneto-optical disk, or an IC card inserted into the computer 300. Then, the computer 300 may read and execute the program from these.

  Furthermore, the program is stored in “another computer (or server)” connected to the computer 300 via a public line, the Internet, a LAN, a WAN, or the like. Then, the computer 300 may read and execute the program from these.

  The following supplementary notes are further disclosed with respect to the embodiments including the above examples.

(Supplementary note 1) an eye opening degree obtaining unit for obtaining an eye opening degree of the subject;
A variation calculating unit for calculating variation in the opening of the eye within a predetermined time;
A wakefulness estimation device comprising: a drowsiness determination unit that performs drowsiness determination using the calculated variation and a predetermined threshold value.

(Supplementary note 2) The variation calculation unit, as the variation, the dispersion value of the eye opening, the deviation of the eye opening, the difference between the maximum value and the minimum value of the eye opening, The awakening level estimation apparatus according to appendix 1, wherein at least one of the frequencies at which the difference in opening is increased to a predetermined threshold value or more is calculated.

(Supplementary Note 3) The variation calculating unit further calculates a variance value of the calculated variation,
The awakening level estimation apparatus according to appendix 1 or 2, wherein the drowsiness determination unit performs the drowsiness determination by further using the variance value of the variation.

(Additional remark 4) The said dispersion | variation calculation part calculates the dispersion | distribution value of the said dispersion | variation using the opening degree of the said subject's eye acquired in the time longer than the said predetermined time, The additional remark 3 characterized by the above-mentioned. Wakefulness estimation device.

(Additional remark 5) The memory | storage part which memorize | stores the dispersion | variation in the opening degree of the said eye acquired in the past,
Using a variation in the opening of the eye stored in the storage unit, further comprising a threshold setting unit for setting the predetermined threshold;
The drowsiness determination unit determines drowsiness of the target person when the magnitude of the variation is equal to or greater than the predetermined threshold that is set. Wakefulness estimation device.

(Additional remark 6) The said threshold value setting part sets the said predetermined threshold value by multiplying the predetermined value with the average value of the said opening degree of the eye in a predetermined period, The alertness degree of Additional remark 5 characterized by the above-mentioned Estimating device.

(Additional remark 7) The alertness estimation apparatus as described in any one of additional remarks 1 thru | or 6 which further has a warning output part which outputs a warning, when it determines with it being a drowsiness state.

(Additional remark 8) It further has a backlight detection part which detects the backlight disturbance state with respect to the said subject,
The arousal level estimation apparatus according to any one of appendices 1 to 7, wherein the drowsiness determination unit performs the drowsiness determination by further using the detected backlight disturbance state.

(Additional remark 9) The said opening degree acquisition part weights the acquired opening degree of the said eye using the said disturbance determination state,
The wakefulness estimation device according to appendix 7, wherein the variation detection unit calculates the variation in the eye opening within the predetermined time using the weighted eye opening.

(Additional remark 10) The said threshold value setting part correct | amends the said threshold value using the said disturbance determination result detected, The awakening degree estimation apparatus of Additional remark 8 or 9 characterized by the above-mentioned.

(Additional remark 11) The said backlight detection part is related with the luminance value estimated using the luminance value of the acquired image of the subject's eye, the illuminance value acquired by the illuminance sensor, and the traveling direction, time, and weather information of the subject. The awakening level estimation device according to any one of appendices 8 to 10, wherein the backlight disturbance state is detected using at least one of the information.

(Supplementary note 12) The supplementary notes 8 to 11, further comprising a warning output unit that outputs a warning when the drowsiness state is determined and the backlight disturbance state is determined to be equal to or less than a predetermined threshold value. The arousal level estimation apparatus according to any one of the above.

(Additional remark 13) The said drowsiness determination part further determines whether the average value or the maximum value of the said eye opening within a predetermined period is more than a predetermined threshold value, and the magnitude | size of the said dispersion | variation is set to the said predetermined value And when it is determined that the average or maximum value of the eye opening is equal to or greater than the predetermined threshold, it is determined that the subject is in a sleepy state. The arousal level estimation apparatus according to any one of 1 to 12.

(Appendix 14) Obtaining the eye opening of the subject,
Calculate the variation of the eye opening within a predetermined time,
A wakefulness level estimation method in which a computer executes a process of performing drowsiness determination using the calculated variation and a predetermined threshold value.

(Additional remark 15) In the arousal level estimation system which has a vehicle-mounted apparatus and a server,
The in-vehicle device is
A transmission unit that transmits an image of the eye of the subject to the server;
The server
An eye opening obtaining unit that calculates an eye opening using an image of the eye of the subject received from the in-vehicle device;
A variation calculating unit for calculating variation in the opening of the eye within a predetermined time;
A sleepiness level estimation system comprising: a sleepiness determination unit that performs sleepiness determination using the calculated variation and a predetermined threshold value.

1, 2, 3 Arousal level estimation system 100, 500, 600 Arousal level estimation device 110 Communication unit 120, 520, 620 Storage unit 121, 621 Eye opening storage unit 122, 522, 622 Variation storage unit 123, 523, 623 Threshold value Storage unit 624 Backlight state storage unit 130, 530, 630 Control unit 131, 631 Eye opening acquisition unit 132, 532, 632 Variation calculation unit 133, 533, 633 Threshold setting unit 134, 534, 634 Sleepiness determination unit 135 Warning output unit 636 Backlight detection unit 200, 900 Sensor group

Claims (7)

An eye opening obtaining unit for obtaining the eye opening of the subject;
A variation calculating unit for calculating variation in the opening of the eye within a predetermined time;
A wakefulness estimation device comprising: a drowsiness determination unit that performs drowsiness determination using the calculated variation and a predetermined threshold value. The variation calculation unit further calculates a variance value of the calculated variation,
The wakefulness estimation device according to claim 1, wherein the sleepiness determination unit performs the sleepiness determination by further using the variance value of the variation. A storage unit for storing variation of the opening degree of the eye acquired in the past;
A threshold setting unit that sets the predetermined threshold using the variation in the opening of the eye stored in the storage unit;
3. The arousal level estimation according to claim 1, wherein the drowsiness determination unit determines drowsiness of the subject when the magnitude of the variation is equal to or greater than the predetermined threshold value that is set. apparatus. Further comprising a backlight detection unit for detecting a backlight disturbance state for the subject,
The wakefulness estimation device according to any one of claims 1 to 3, wherein the sleepiness determination unit performs the sleepiness determination by further using the detected backlight disturbance state.   The drowsiness determination unit further determines whether an average value or maximum value of the eye opening within a predetermined period is equal to or greater than a predetermined threshold, and the magnitude of the variation is equal to or greater than the predetermined threshold set. And determining that the subject is in a sleepy state when it is determined that an average value or a maximum value of the opening degree of the eye is equal to or greater than the predetermined threshold value. The arousal level estimation apparatus according to any one of the above. Get the eye opening of the subject,
Calculate the variation of the eye opening within a predetermined time,
A wakefulness level estimation method in which a computer executes a process of performing drowsiness determination using the calculated variation and a predetermined threshold value. In a wakefulness estimation system having an in-vehicle device and a server,
The in-vehicle device is
A transmission unit that transmits an image of the eye of the subject to the server;
The server
An eye opening obtaining unit that calculates an eye opening using an image of the eye of the subject received from the in-vehicle device;
A variation calculating unit for calculating variation in the opening of the eye within a predetermined time;
A sleepiness level estimation system comprising: a sleepiness determination unit that performs sleepiness determination using the calculated variation and a predetermined threshold value.

Images