JP2007241937A - Awaking degree estimation device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To present a determination result of reliability for detection of eye opening together with a determination result of awaking in an awaking degree estimation device. <P>SOLUTION: This device comprises an eye opening detection means 101 detecting an eye opening contained in an image; an eye opening transition detection means 104 detecting, for a plurality of images continued in time series, time-series transition of the eye opening detected by the detection means 101; an awaking degree estimation means 105 determining an awaking based on the time-series transition of the eye opening detected by the detection means 104; and a reliability determination means 106 determining reliability for the detection result of eye opening by the detection means 101 based on the time-series transition of the eye opening detected by the detection means 104. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an arousal level estimation device, and more particularly, to an improvement in awakening level estimation based on an eye opening state in an image of a face of an occupant such as a driver of a vehicle.

  Conventionally, it has been attempted to forcibly increase the driver's arousal level by, for example, sounding a warning sound when the awakening state of a vehicle occupant, particularly the driver, is detected and the awakening state is low. ing. In such an attempt, an arousal level estimation device for detecting the arousal level is used.

This awakening degree estimation device detects awakening degree based on the opening degree of the passenger's eyes, for example, detects eyes based on an image of the passenger's face taken by a camera or the like installed in the passenger compartment. Then, by tracking the opening of the eye in time series, a histogram relating to the blinking time (the length of time required for one blink, etc.) is created, and the user is not sleepy immediately after driving (high arousal level) Based on the blink time histogram in the state), the standard blink time in the state where the arousal level is high is obtained, and is longer than the standard blink time in the total number of blinks detected in a certain period during driving thereafter. The ratio of the number of blinks in the blink time is obtained, and the arousal level is estimated based on this ratio (Patent Document 1).
JP-A-9-277849

  By the way, the state of ambient light in the passenger compartment may change greatly while the vehicle is running, and this change in ambient light may affect the illuminance change on the face of the occupant being photographed, It affects as incident light, and a face image may not be captured in an appropriate exposure state, or may be noise when detecting the opening of the eye from the face image.

  Further, when noise is superimposed on a signal line of an image signal input from a camera or a signal line inside the arousal level estimation device, it can be a noise regarding a face image.

  Then, using the blink time histogram created based on the face image on which such noise is superimposed, even if a standard blink time that is the basis of all the standards is set, a correct wakefulness estimation result is obtained. However, providing such an erroneous estimated awakening level to an external dozing prevention device or the like may cause malfunction of the dozing prevention device or the like, which may impair reliability.

  The present invention has been made in view of the above circumstances, and provides an arousal level estimation device that can present a determination result of a degree of arousal together with a determination result of a degree of reliability related to detection of an eye opening. It is the purpose.

  In the wakefulness estimation device according to the present invention, the reliability determination means determines the reliability of the detection result of the eye opening according to the state of the time series transition of the eye opening. Together with the determination result, the determination result of the reliability related to the detection of the opening degree of the eye is also presented.

  That is, the wakefulness estimation device according to the present invention detects eye opening detection means for detecting an eye opening included in an image and a plurality of images that are connected in time series by the eye opening detection means. The eye opening degree transition detecting means for detecting the time series transition of the eye opening and the degree of arousal based on the time series transition of the eye opening detected by the eye opening degree transition detecting means. Based on the time-series transition of the eye opening detected by the awakening degree determining means and the eye opening transition detecting means, the reliability of the detection result of the eye opening by the eye opening detecting means And reliability determination means for determining the degree.

  Here, the “time-series transition of the eye opening” used when the reliability determination unit determines the reliability is the “time-series transition of the eye opening” used when the arousal level determination unit determines the arousal level. The process for determining the reliability by the reliability determination unit and the process for determining the arousal level by the wakefulness determination unit are different from each other because the purpose thereof is different. .

  According to the arousal level estimation apparatus according to the present invention configured as described above, the eye opening degree detection means detects the eye opening degree for each image for a plurality of images that are connected in time series, and The degree-of-opening detection means detects the time-series transition of the eye opening (time-series change of the eye opening) based on the eye opening obtained for each image, and the degree of arousal The determination means determines the arousal level based on the time-series transition of the eye opening.

  On the other hand, the reliability determination unit determines the reliability of the detection result of the eye opening by the eye opening detection unit according to the time-series transition of the eye opening detected by the eye opening transition detection unit. .

  As a result, the wakefulness level estimation device according to the present invention can present the wakefulness level and the reliability with respect to the detection result of the eye opening.

  According to the arousal level estimation apparatus according to the present invention, it is possible to provide, together with the arousal level, a determination material (reliability) for determining whether or not this arousal level can be used.

  Therefore, for example, whether the wakefulness level output from the wakefulness level estimation device of the present invention is used for a device such as a device for preventing a drowsiness for a vehicle occupant, and whether the eye opening is the basic data for deriving the wakefulness level Can be easily determined according to the reliability presented for.

  Hereinafter, the best embodiment of the arousal level estimation apparatus according to the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing an arousal level estimation apparatus 100 as an embodiment of the arousal level estimation apparatus according to the present invention, and FIG. 2 is a state in which the awakening level estimation apparatus 100 shown in FIG. It is a schematic diagram which shows.

  The arousal level estimation apparatus 100 shown in the figure is based on an image of the face of the driver 10 of the vehicle 200 photographed by a CCD camera 201 mounted in the passenger compartment of the vehicle 200. The eye opening degree detecting means 101 for detecting the opening degree and the time series transition of the eye opening degree detected by the eye opening degree detecting means 101 are detected for a plurality of face images successively connected in time series. Eye opening degree transition detecting means 104, wakefulness degree judging means 105 for judging the degree of wakefulness based on the time series transition of the eye opening degree detected by the eye opening degree transition detecting means 104, and eye opening degree transition detecting means And a reliability determination unit 106 that determines the reliability of the detection result of the eye opening by the eye opening detection unit 101 based on the time-series transition of the eye opening detected by 104. is there.

  Here, the camera 201 may be another imaging element such as a CMOS in addition to the CCD, and outputs images sequentially photographed in time series.

  Note that the camera 201 is not provided as a component of the arousal level estimation apparatus 100, but is provided in the passenger compartment in advance for other purposes. However, the cameras 201 provided in advance are provided. If the camera 201 does not exist, the camera 201 may be separately provided as a component of the arousal level estimation device 100.

  The eye opening detection unit 101 includes an eye detection unit 102 that detects an eye, and an eye opening detection unit 103 that detects the eye opening based on the eye detected by the eye detection unit 102.

  The eye detection unit 102 performs vertical scanning (vertical direction) on the facial image density distribution input from the CCD camera 201 in the vertical direction (vertical direction) and sub-scanning in the horizontal direction (horizontal direction). A density part that becomes relatively dark, in which the density fluctuation in the direction satisfies a predetermined condition, is extracted, and an eye is detected based on the density part that becomes relatively dark.

  In addition, the eye detection unit 102 detects a range corresponding to a range near the eye position detected in the immediately preceding image that is connected in time series to the detection target image with respect to the detection target image. A range (tracking region) is set, and eyes are detected for the set tracking region.

  However, if there is no immediately preceding image in time series, or if the eye cannot be detected by the eye detection process performed for the set tracking area, the entire range of the detection target image Detection is performed.

  The eye opening degree detection unit 103 obtains the height along the vertical direction of the eye detected by the eye detection unit 102, and detects the obtained height as the eye opening degree. The aspect ratio of the rectangle circumscribing the eye detected by the eye detection unit 102 may be obtained, and the obtained aspect ratio may be detected as the opening of the eye, or may be detected by the eye detection unit The curvature radius of the upper edge of the eye may be obtained, and the obtained curvature radius may be detected as the opening degree of the eye.

  The reliability determination unit 106 time-differentiates the time-series transition of the eye opening P detected by the eye-opening transition detection unit 104, and according to the distribution of the obtained time-differential value K of the time-series transition. The reliability is determined and the reliability is output.

  Specifically, with respect to the distribution of the time differential value K of the time series transition, the frequency at which the absolute value | K | of the time differential value K exceeds a predetermined threshold is obtained, and the reliability is determined according to the obtained frequency. Is.

  Further, the vehicle 200 emits a predetermined warning sound according to the arousal level output from the arousal level determination unit 105 of the arousal level estimation device 100 and the reliability level output from the reliability level determination unit 106. A drowsiness prevention device (alarm buzzer or the like) 202 for an occupant is provided.

  Next, the operation of the arousal level estimation apparatus 100 of the present embodiment will be described with reference to the flowcharts shown in FIGS.

  First, as shown in FIG. 3, an image photographed by the CCD camera 201 is input to the eye opening degree detecting means 101 (step 301 (hereinafter referred to as S301)).

  The eye detection unit 102 of the eye opening degree detection unit 101 first determines whether or not an eye tracking area is set (S302). Immediately after the processing is started (when an image is input for the first time in time series), there is no image input before that, and therefore no eye tracking area is set. Is detected as an eye detection area (S303).

  For example, as shown in FIG. 7, the eye detection process performed by the eye detection unit 102 is the main image scanning in the vertical direction (Y-axis direction, vertical direction) and the horizontal direction (X-axis direction, horizontal direction). ), The points where the density fluctuation in the vertical direction satisfies the predetermined condition are extracted on each vertical line (main scanning line) for the entire image. Then, the extracted points are shown on the two-dimensional coordinates as shown in FIG.

  In FIGS. 7 and 8, symbols Xa, Xb, Xc, and Xd are X coordinates of arbitrary vertical lines. Further, reference numeral A1 in FIG. 8 indicates that the vertical line is scanned on each vertical line (scanning in the direction of increasing the Y coordinate (coordinate (X, 0) → coordinate (X, 479))). First, the extraction point that satisfies the predetermined condition, the symbol A2 is the second extraction point that satisfies the predetermined condition on each vertical line, and hereinafter, the symbol A3 is the third and the symbol A4 is the fourth. Represents each.

  Next, as shown in FIG. 9, between adjacent vertical lines, the extracted points are close to each other in the vertical direction, and the extracted points are grouped together.

  This grouping is a process for detecting density characteristics having a large density fluctuation and extending in the horizontal direction, that is, density edges extending in the horizontal direction, and the characteristics G1, G2, G3 are detected by a relatively simple calculation process. G4, G5, and G6 can be detected.

  Thereafter, the features G1, G2,..., G6 are made to correspond to the zones L, C, R divided into three in the horizontal direction shown in FIG. Based on the relative positional relationship between the features G1, G2,..., G6 (representing the center (centroid) coordinates of G1, G2,..., G6 as representative points), each feature G1, G2,. It is determined which element of the eye corresponds, and an eye which is one of the elements is specified and its position is detected. In this embodiment, the sign G1 can be identified as the right eyebrow of the passenger, the sign G2 is the left eyebrow, the sign G3 is the right eye, the sign G4 is the left eye, the sign G5 is the nose, and the sign G6 is the mouth. Can be detected.

  When the center position 1101 (coordinates (Xk, Yk)) of the eye G4 (which may be the eye G3, and so on) is detected (S303), the eye detection unit 102 is shown in FIG. In this manner, a predetermined rectangular area 1103 having the center position 1101 of the eye G4 as the center coordinates (Xk, Yk)) is set.

  This rectangular area 1103 limits the eye detection target range (tracking area) in an image (the (N + 1) th frame) input next time in series in time series to a small area, and the image of the Nth frame. A rectangular area with X axis direction ± m and Y axis direction ± n is set as the tracking area with the center 1101 of the eye G3 detected from the center coordinates as the central coordinates.

  The rectangular area is specified by the coordinates 1102 (Xk−m, Yk−n) of the upper left corner.

  The eye position is detected within the tracking area 1103 set in step 304 (S304) (S305). This process is originally set as a tracking area in the (N + 1) -th frame image input next time in a time series, but the entire area is set as a detection target area (in the determination of S302). Also for the image of the Nth frame (when “N” is selected), the eye detection process is performed again in this tracking area 1103, so that the vertical line interval in FIG. Detection processing can be performed to further improve the detection accuracy of the center position of the eye.

  Then, as shown in FIG. 11B, when the eye G4 is detected in the image of the (N + 1) -th frame that is input next time in time series, the tracking area is centered on a center coordinate ( A rectangular area with X axis direction ± m and Y axis direction ± n, which is Xk−Δx, Yk−Δy), is set as a new tracking area 1106. That is, the tracking area is updated to the coordinates 1105 (Xk−Δx−m, Yk−Δy−n), and hereinafter, according to the center position (S305, S306) of the eye G4 detected in the sequentially input images. The tracking area is sequentially updated by the same processing (S309).

  Before updating the tracking region of the eye G4 (S309), it is determined whether or not the eye G4 has been successfully detected in the tracking region of the eye G4 (S306), and the detection of the eye G4 has failed (S306). Deletes the set tracking region of the eye G4, returns to step 302, and executes the eye G4 detection process (S303) using the entire region of the image as the detection target region.

  Note that the timer control (S308) is control used for calculation of the reliability of eye opening / closing determination and determination of arousal level, and will be described in each determination flow.

  When the detection is successful in the determination of whether or not the eye G4 has been successfully detected (S306), the eye opening degree detection unit 103 detects the opening degree of the eye G4 after the tracking area is updated (S309) (S310). .

  Here, as an example of the detection processing of the opening degree of the eye G4, three types of opening detection processing will be described below.

  First, in the first opening degree detection process, as shown in FIGS. 12A and 12B, the height (vertical width) H along the vertical direction of the detected eye G4 is obtained, and this vertical width H Is calculated as an eye opening value.

  In this process, as shown in FIG. 12, the binarization process is performed to clearly reveal the outline shape of the eye G4, and black pixels along the vertical direction (black and white pixels are separated by the binarization process). The number of consecutive black pixels) is counted, and the maximum value of the count is calculated as the opening value of the eye G4.

  Whether or not the eye G4 is open can be determined by the distance between the lower edge of the upper eyelid and the upper edge of the lower eyelid, that is, the height along the vertical direction of the eye G4, while the lower edge of the upper eyelid As described above, the upper edge of the lower eyelid can be detected by simple arithmetic processing that scans the density distribution of the facial image along the vertical direction.

  Therefore, by defining the opening degree of the eye G4 as the height along the vertical direction of the eye G4, the opening degree of the eye G4 can be obtained by a simple calculation process. In addition, when the vertical width H is large, the eye is open. As the vertical width H becomes smaller, the eye opening degree changes in the direction in which the eyes are closed.

  Next, in the second opening degree detection process, as shown in FIGS. 13A and 13B, a rectangular aspect ratio H / W circumscribing the detected eye G4 (W represents a lateral width). ) And the obtained aspect ratio H / W is detected as the eye opening. When the aspect ratio H / W is large, the eye is open, and as the aspect ratio H / W decreases, the eye opening degree changes in the direction of closing the eyes.

  Since this process defines the rectangular aspect ratio H / W circumscribing the binarized eye G4 as the opening of the eye G4, when the image magnification differs between images, that is, FIG. As shown in (b), even if the occupant 10 to be imaged is the same between images, even if there is a difference in the distance between the occupant 10 and the camera 201 ((a) is more distance than (b)). ) Can be eliminated, and the opening degree of the eye G4 can be detected with high accuracy.

  In the third opening degree detection process, as shown in FIGS. 14A and 14B, the curvature radius of the detected upper edge of the eye G4 (the contour line of the upper eyelid) is obtained, and the obtained curvature radius. Is detected as the opening degree of the eye G4.

  Here, the radius of curvature is calculated by approximating a contour line (a set of extracted points) with a parabola. When the value of the radius of curvature is small, the eye is open (FIG. 14 (a)). As the value of the radius of curvature increases, the eye closes (FIG. 14 (b)). Assume that the degree of eye opening changes.

  As shown in the photograph of FIG. 14 (c), when the contrast of the eye part (the part surrounded by a rectangular frame in the figure) is shaded or the like is lowered, the vertical width H or the vertical and horizontal directions of the eye G4 described above. Depending on the detection process according to the ratio H / W, it is difficult to accurately detect the opening state of the eye G4, whereas this detection process based on the radius of curvature is not easily affected by a decrease in contrast and the contrast of the image is low. Even in this case, the opening degree of the eye G4 can be detected with high accuracy.

  After the opening degree of the eye G4 is detected by the above-described processing (S310), the detected opening degree of the eye G4 is input to the eye opening degree transition detecting means 104, and the eye opening degree transition detecting means 104 is time-series. Then, the opening degree of the eye G4 respectively obtained from a plurality of consecutive images is sequentially stored, and a time-series transition of the stored opening degree of the eye G4 is detected.

  Here, in order to obtain the time-series transition, a plurality of eye G4 opening degrees acquired in time series are necessary, and also in calculating a differential value of the eye G4 opening described later, A plurality of openings of the eye G4 acquired in time series are necessary.

  Therefore, in the next step (S312), it is determined whether or not the opening value of the eye G4 for calculating the differential value of the opening of the eye G4 is stored in the eye opening transition detecting means 104.

  When it is determined that it is not stored, the opening of the eye G4 is stored (S311), the next image that is continuous in time series is input (S301), and the above processing (S311 to S310) is performed. repeat. Note that the eye opening degree transition detecting unit 104 repeats the above processing, and the eye opening degree of the eye G4 is sequentially stored in the eye opening degree transition detecting unit 10 based on the time series eye opening degree of the eye G4. Obtain the time series of the opening.

  On the other hand, when it is determined that the opening degree of the eye G4 is stored for the number of data necessary for the calculation of the differential value (S312), among the stored opening degrees of the plurality of eyes G4, the oldest memory time is stored. The opening degree of the eye G4 is updated with the opening degree of the eye G4 based on the latest image (S313).

  Then, the differential value by the opening degree of several eyes G4 including the opening degree of the newest eye G4 memorize | stored is calculated (S314). Thereafter, the update of the opening degree of the eye G4 (S313) and the calculation of the differential value (S314) are repeated, whereby the calculation of the differential value with respect to the latest opening degree of the eye G4 is continued.

  Next, the process of determining the reliability of the detection result of the opening degree of the eye G4 by the reliability determination unit 106 will be described with reference to the flowchart shown in FIG.

  First, it is determined whether at least one of the timer TA for reliability determination and the timer TB for arousal determination is in a stopped state (S401), and when either one of the timers TA or TB is in a stopped state Activates the stopped timer TA or TB, respectively (S402).

  Next, the process waits for the reliability determination timer TA to reach a predetermined time (S403, S404). The predetermined time compared with the timer TA is a time for determining the reliability of the detection result of the opening degree of the eye G4, and based on the occurrence frequency of blinks in a normal state (a state of high arousal level), The abnormality of the time-series transition of the opening degree of the eye G4 is determined.

  Specifically, this will be described with reference to FIGS. 15 and 16. FIG. 15 outputs the vertical width H of the eye G4 as the opening degree P of the eye. FIG. 15A shows the opening degree P (vertical width H) of the eye G4 on the vertical axis and time series on the horizontal axis. A graph showing the frame number (number of frames) of the image and showing the time-series transition of the opening degree P of the eye G4, (b) was obtained by time-differentiating the opening degree P of the eye G4 in (a). It is a graph which shows the time-sequential transition of the time differential value K of the opening degree P of the eye G4.

  FIG. 15A shows a state where the output of the opening degree of the eye G4 is ideally formed, the change width of the opening degree P of the eye G4 in the open eye state is small, and the opening degree P is large downward. The three places which are changing show the opening degree transition due to blinking.

  As is clear from this figure, when the detection state of the opening degree P of the eye G4 is normal, the number of blinks occurring within a predetermined time (even if the individual difference of the occupant 10 as the subject is taken into consideration) Frequency) is limited.

  On the other hand, as factors that adversely affect the opening detection processing of the eye G4 by the above-described image recognition, changes in ambient light, superimposition of noise on the output signal (image signal) from the CCD camera 201, or this When noise is superimposed on the image signal in the arousal level estimation device 100 and the signal processing state becomes unstable due to these factors, the time series transition of the opening degree P of the eye G4 and the opening of the eye G4 are performed. The time-series transition of the time differential value K of degree P is as shown in FIGS. 16A and 16B which are completely different from the ideal detection state of FIG.

  That is, as shown in FIG. 16A, the change width of the opening P of the eye G4 becomes large, and it becomes difficult to distinguish between blink and noise. Therefore, the total number of blinks and noises occurring within a predetermined time exceeds the limited number (frequency) of blinks occurring within a predetermined time.

  Furthermore, if a specific numerical value is illustrated, the number of blinks generated in 10 seconds can be defined as 10 or less. The predetermined time monitored by the timer TA means this time interval (= 10 seconds).

  In order to determine this state, the reliability determination means 106 obtains a time-series transition of the time differential value K of the opening P of the eye G4. That is, when the timer TA has not reached the predetermined time (S403), the output count of the absolute value | K | of the differential value K exceeding a predetermined range (a range where the absolute value | K | Count up (S404).

  That is, when the blink in the normal state shown in FIG. 15 is set as the detection target, the number of times that the absolute value | K | of the time differential value K exceeds the predetermined range depends on the positive and negative peaks generated in pairs per blink. In contrast to the six times, in the state in which the opening detection of the eye G4 shown in FIG. 16A is unstable, in addition to blinking, the absolute value | K | of the differential value K is within a predetermined range (the differential value K The number of times of output exceeding the absolute value | K | of the range of 5 or less, for example, is extremely high (FIG. 16B).

  Therefore, as a result of monitoring the predetermined time by the timer TA (S403), when the predetermined time has passed (S403 → S405), the number of times that the absolute value | K | of the time differential value K counted up in step 404 exceeds the predetermined range Accordingly, it is possible to determine whether the reliability for detecting the opening degree of the eye G4 is high (S407) or low (S408).

  Note that before the reliability determination (S406), the timer TA is stopped and reset (S405), and after the reliability determination (S406), the counted-up counter value is reset (S409).

  Here, the predetermined time monitored by the timer TA and the reference value (threshold value) of the number of times that the absolute value | K | of the time differential value K occurs within the predetermined time exceeds the predetermined range. Whether it is preferable to set the value will be described with reference to FIGS. 17 and 18.

  FIG. 17 shows the occurrence frequency of blinks in each predetermined time for 26 different passengers A to Z when the predetermined time is set to 30 seconds and when the predetermined time is set to 60 seconds. Are summarized in a list.

  According to the results shown in this table, it can be seen that the blink occurrence frequency has a large individual difference. However, the maximum value of the frequency depends on the occupant E, and as shown in a column 1701, it is 27 times in the 30-second section and 52 times in the 60-second section.

  Therefore, when the number of times exceeds this (the number exceeding 27 times in the 30-second interval, or the number exceeding 52 times in the 60-second interval), it is not based on normal blink measurement (noise is superimposed → reliability is Low).

  A specific example of this determination process will be described with reference to FIG. FIGS. 18A and 18B are time-series transitions of the differential value K of the opening degree P of the eye G4. For example, the state shown in FIG. The number of points 1801 where the absolute value | K | of the differential value K of the opening degree P exceeds the predetermined range (= 5) is 6 times, and this number of times is shown in the list of FIG. 27 times ", that is, less than the standard of" 9 times in the section converted into 10 seconds ". As a result, it can be determined that there is no noise superposition and that blinks are properly detected.

  On the other hand, in the state shown in FIG. 18B, the number of times of the point 1802 where the absolute value | K | of the differential value K of the opening degree P of the eye G4 exceeds the predetermined range (= 5) in about 10 seconds is 40 times. The number of times is much higher than the standard of “about 9 times in the section converted to 10 seconds”, and as a result, normal detection of the opening degree P of the eye G4 such as noise superposition is performed. Judge that it is not.

  In FIG. 18, for the sake of easy understanding of the differential value K of the opening degree P of the eye G4, the explanation was made in the 10 second section for convenience. However, from the viewpoint of the actual blink occurrence state, in the short time section, FIG. The length of time and the number of occurrences are not necessarily in direct proportion.

  That is, the number of occurrences in the 30-second interval is not simply three times the number of occurrences in the 10-second interval. However, the error decreases as the time interval becomes longer, and the length of the time interval and the number of occurrences exhibit a substantially direct relationship.

  Therefore, this time interval may be changed as appropriate, or may be set according to individual circumstances.

  Next, with reference to the flowcharts shown in FIGS. 5 and 6, the arousal level determination process by the arousal level determination unit 105 will be described.

  Here, FIGS. 19 and 20 are graphs in which (a) shows a time-series transition of the opening degree P of the eye G4, and (b) is obtained by time-differentiating the opening degree P of the eye G4 in (a). 19 is a graph showing a time-series transition of the time differential value K of the opening degree P of the obtained eye G4, and FIG. 19 shows a case where one normal blink in a state where the arousal level is high is normally detected. Indicates a case where one blink that is longer than normal in a state in which the awakening level starts to decrease is normally detected.

  Next, with reference to the flowchart shown in FIG. 5, the output state of the differential value K of the opening degree P of the eye G4 corresponding to blinking in the normal state (high arousal level) shown in FIG. 19 will be described as an example. To do.

  First, based on the time series transition of the opening degree P of the eye G4 output from the eye opening degree transition detection means 104, the arousal level determination unit 105 performs time series of the time differential value K of the opening degree P of the eye G4. A transition is obtained, and it is determined whether or not the absolute value | K | of the differential value K is within a predetermined range (≦ L).

  The predetermined range L is a region 1901 in FIG. 19, and specifically, for example, a range of ≦ 5.

  This region 1901 (absolute value | K | ≦ 5) is set as a range whose magnitude relationship is opposite to the range (absolute value | K | ≧ 5) for counting the points 1801 and the like when determining the reliability. However, since L is an index for determining the degree of arousal and an index for determining the reliability have different purposes in the first place, the indices do not necessarily have the same value. 1901 may be set as (absolute value | K | ≦ 6) or the like, and a range for counting the points 1801 or the like when determining the reliability may be set as (absolute value | K | ≧ 4) or the like.

  Since the initial differential value K in FIG. 19B is a negative value (−1) and is in the area 1901, the positive flag (which defines that the differential value K is positive) is set to OFF ( S502 → S507 → S508).

  If the positive flag is already OFF, the process of step 508 is bypassed and the process proceeds to step 509.

  Next, with respect to the graph of the time-series transition of the differential value K (FIG. 19B), the integration of the area values in the negative range and the integration of the area values in the positive range are performed separately (S509).

  However, since only the first derivative K does not constitute a time-series transition graph (section), that is, neither the positive range nor the negative range constitutes an area (area = 0), the integration process (S509) is bypassed. Then, the process proceeds to the next step 510.

  In step 510, triggered by the fact that the differential value K is switched from positive to negative, whether or not the integrated value of the area in the positive range of the time-series transition of the previous differential value K has exceeded a predetermined threshold value. Determine whether.

  When the predetermined threshold value is exceeded (S510), the areas of the respective positive ranges are integrated, the positive integrated values are stored (S513), and the positive range of the time-series transition of the differential value K is determined. The area value is reset (S511). Note that what is reset is an area value in a positive range for integration, and the stored positive integration value is held as it is.

  If the predetermined threshold value is not exceeded (S510), the area value in the positive range of the time-series transition of the differential value K is reset as it is (S511). In this case as well, the area value in the positive range that does not exceed the predetermined threshold value is reset, and the already stored positive integrated value is held as it is.

  Here, only the first derivative value K does not constitute a time-series transition graph as described above, and therefore the area of the positive range and the area of the negative range are 0, and the above-described steps 510, 513, and 511 are performed. The process of step 510 is step 510 → step 511.

  In step 510, the area of the positive range of the time series transition of the differential value K is compared with the predetermined threshold value because the area of the positive range of the time series transition of the differential value K is the predetermined threshold value. If the area exceeds a predetermined threshold, the integration is performed in step 513. If the area of the positive range of the time-series transition of the differential value K is smaller than a predetermined threshold, the integration is performed. This is because they are ignored and not passed through (not via step 513).

  Since the second differential value K is also a negative value (−1) and is in the area 1901, the positive flag is set to OFF (S502 → S507 → S508), and the time series transition of the differential value K (FIG. 19). For (b)), the integration of the area values in the negative range and the integration of the area values in the positive range are performed separately (S509).

  Here, since the area value in the positive range is 0, the integrated value remains 0 (S509), but the area value in the negative range is the second absolute value from the first absolute value K. Since the graph (section) of the time series transition up to K is constructed, the area value (negative value) in this range can be calculated (S509).

  Also here, since the integrated value of the area of the positive range of the time series transition of the differential value K does not exceed the predetermined threshold (S510), the positive range of the time series transition of the differential value K as it is. The area value is reset (S511).

  Since the third differential value K is a positive value (1) and is in the area 1901, the positive flag is turned ON (S501 → S502 → S503).

  Next, with respect to the graph of the time-series transition of the differential value K, the integration of the area values in the negative range and the integration of the area values in the positive range are performed separately (S504).

  Since the area of the negative range 1902 in FIG. 19 is closed by this processing, it is determined whether or not the integrated value of the area of the negative range of the differential value K exceeds a predetermined value (S505).

  When the predetermined threshold value is exceeded (S505), the areas of the respective negative ranges are integrated, the negative integrated values are stored (S514), and the negative range of the time-series transition of the differential value K is determined. The area value is reset (S506). Note that what is reset is the area value in the negative range for integration, and the stored negative integration value is held as it is.

  If the predetermined threshold value is not exceeded (S505), the area value in the negative range of the time-series transition of the differential value K is reset as it is (S506). Also in this case, the area value in the negative range that does not exceed the predetermined threshold value is reset, and the negative accumulated value that has already been stored is held as it is.

  In step 505, the area of the negative range of the time series transition of the differential value K is compared with the predetermined threshold value because the area of the negative range of the time series transition of the differential value K is the predetermined threshold value. In the case of a large area exceeding the threshold value, the integration is performed in step 514. However, in the case where the area in the negative range of the time-series transition of the differential value K is smaller than a predetermined threshold, the integration is performed. This is because it is ignored and not passed through (not via step 514).

  Since the fourth differential value K is 0 and is in the area 1901, it is determined whether or not the positive flag is ON (S501 → S502 → S507 → S512).

  Since the positive flag is set to ON in step 503 during the process resulting from the third derivative value K, the determination (S512) results in transition to step 504, and the time-series transition of the derivative value K In the graph, integration of area values in the negative range and integration of area values in the positive range are performed separately (S504).

  Here, since the area of the negative range existing before the process caused by the fourth differential value K is reset in step 506 during the process caused by the third differential value K, step The processes of 505, 514, and 506 are substantially passed through.

  As described above, every time the differential value K is switched between positive and negative values, the positive range area value exceeds the predetermined value, or the negative range area value exceeds the predetermined value is integrated. And remember each one.

  In addition, when there is no range in which the time-series transition of the differential value K is negative between the ranges 1905 and 1906 in which the time-series transition of the differential value K shown in FIG. After the cumulative transition is integrated for the area of the positive range 1905 (S504), the positive flag is determined via the step 507 (S512), and the differential value K is integrated for the area of the positive range 1905. Since (S504) is performed continuously, the integrated value of the areas in the positive range is treated as the sum of the areas in the positive range 1905 and the subsequent positive range 1906.

  In the state where the arousal level shown in FIG. 19 is not lowered and only a normal blink occurs, the time series transition of the opening degree in the open eye state is relatively small. It is rare that the integrated value of the positive and negative areas is stored through the processing, and even if the integrated value of the area is accidentally stored, the negative differential value K caused by blinking shown in FIG. In the range 1910 of (MIN: −9) and the range 1911 of the positive differential value K (MAX: 6), the absolute value | K | of the differential value K exceeds the predetermined range L in the branch determination in step 501, respectively. The process proceeds to step 606 in FIG. 6, where the sum of the integrated value of the area in the positive range of the time-series transition of the differential value K and the integrated value of the area in the negative range of the time-series transition of the differential value K is Reset.

  Further, before reaching the negative differential value K range 1910, after the process of step 506 or 511, the process proceeds to the process of step 601 shown in FIG. Until the timer TB reaches a predetermined time (a period set for the determination of the degree of arousal), the process returns to step 301 shown in FIG.

  When the timer TB reaches a predetermined time (S601), after the timer TB is stopped and reset (S602), the integrated value of the positive range area (S513) and the integrated value of the negative range area are separately accumulated. (S514) is summed, and it is determined whether or not the sum obtained by this summing exceeds a predetermined threshold (S603).

  Here, in a state where the arousal level is not lowered and only a normal blink occurs, it is determined that the awakening level is high because the total value does not exceed a predetermined threshold value. (S604).

  Then, the integrated value of the positive and negative areas of the differential value is cleared at the end of the arousal level determination period (S606).

  On the other hand, when the arousal level decreases, the time series transition of the opening degree P of the eye G4 increases the occurrence frequency of long blinks instead of normal blinks as shown in FIG. Therefore, the differential value K of the opening degree P of the eye G4 does not exceed the index L in the process of step 501 (see FIG. 5), and the blinking speed is also slowed. The area (integrated value) of the closed range 2001 in which the differential value K is negative shown in FIG. 20B, which is a time-series transition of FIG. 20B, is generated in a normal blink shown in FIG. 19B. The differential value K is sufficiently larger than the areas (integrated values) of the negative closed ranges 1904, 1907, etc., and the area of the differential range K positive closed range 2002 shown in FIG. (Integrated value) indicates that the differential value K generated in a normal blink shown in FIG. Since range sufficiently larger than the area of such 1903,1905 (integrated value) were both greater than the predetermined value in the processing of the processing and step 510 in step 505 of FIG. 5.

  As a result, in each process of steps 514 and 513, a large area is set as a closed range, and the area is integrated and stored.

  Therefore, in the determination process of step 603 in FIG. 6, the sum of the integrated value of the positive range area and the integrated value of the negative range area, which is integrated separately, exceeds a predetermined threshold value, and the arousal level is low. It is determined that it is in a state (S605).

  Then, the integrated value of the positive and negative areas of the differential value is cleared at the end of the arousal level determination period (S606).

  In the process of step 308 shown in the flowchart of FIG. 3, when eye tracking (detection) fails (S306), the entire range of the image is set as a detection target, and the eye position is determined from the entire range of the image. Although the time required for detecting the eye position from the entire range is naturally longer than the time required for detecting the eye position from the tracking area, the time required for detecting the eye position from the entire range of the image. The timer TA and the timer TB are stopped for the purpose of excluding from the time of the reliability determination period and the arousal determination period.

  Thereafter, when the eye is successfully detected by the eye detection process from the entire range of the image and the tracking area is set again, the timer TA and the timer TB are restarted (step 402 in the flowchart of FIG. 4). .

  As described above, according to the arousal level estimation apparatus 100 according to the present embodiment, the eye opening degree detection unit 101 targets a plurality of images that are continuous in time series, and the eye opening degree P for each image. Based on the eye opening degree P obtained for each image, the eye opening degree transition detecting means 104 detects the time series transition of the eye opening degree P (time series of the eye opening degree). Change) and the wakefulness determination means 105 determines the wakefulness based on the time-series transition of the eye opening P.

  On the other hand, the reliability determination unit 106 responds to the detection result of the eye opening P by the eye opening detection unit 101 according to the time-series transition of the eye opening P detected by the eye opening transition detection unit 104. Determine reliability.

  As a result, the arousal level estimation apparatus 100 according to the present invention can also present the arousal level and the reliability of the detection result of the eye opening P, which is a material for determining whether or not the arousal level can be used. .

  Therefore, for example, an eye that is basic data for deriving whether or not the wakefulness level output from the wakefulness level estimation device 100 of the present embodiment is used for a device such as the drowsiness prevention device 202 for a vehicle occupant. It can be easily determined according to the reliability presented for the opening degree P.

  FIG. 21A shows a time-series transition of the opening degree P of the eye in a so-called prone eye state (a state of high arousal but with the line of sight directed downward), and FIG. 21B shows a time derivative of FIG. FIG. Even if the degree of wakefulness is high, there is a risk of erroneous detection that a long blink has occurred due to a decrease in the wakefulness level.

  In general, so-called down eyes are not completely closed, and thus can be distinguished from a completely closed state.

  However, although it is not a complete closed eye state, a state in which the degree of arousal is reduced may also occur. Therefore, if it is determined that only the completely closed eye state is a state in which the arousal level is reduced, false detection of the prone eyes is prevented. Although it is possible, there is a possibility that a state in which the degree of arousal is not detected but is not completely closed is missed.

  Conventionally, this trade-off relationship could not be solved, but the arousal level estimation apparatus according to the present invention can solve this problem. The action of reliably distinguishing the bound eyes from the state where the arousal level is actually lowered will be described.

  FIG. 21 shows a state in which the degree of arousal is high and a prone eye can occur, where (a) represents the time-series transition of the eye opening degree P, and (b) represents the time derivative of (a).

  According to FIG. 21, it turns out that it is blinking at the moment of seeing a head-on again when it becomes a bound state. This is because when the focal length has changed significantly (for example, when it has changed from a bounded state (a state in which a distance is viewed) to a state in which it is viewed in front (a state in which a distance is viewed)), The human eye's habit of closing once appears, and it occurs with a high probability without special consciousness.

  And the arousal level estimation apparatus 100 of embodiment mentioned above can catch this phenomenon easily and exactly by the time-sequential transition of the differential value K of the opening degree P of eyes.

  That is, in the negative ranges 2101 and 2102 of the time-series transition of the differential value K of the eye opening P shown in FIG. 21B, the absolute value | K | In step 501, the absolute value | K | exceeds the predetermined range L, and the integrated value of the positive and negative areas of the differential value is reset at each end of the arousal level determination section (step 606 in FIG. 6).

  Therefore, it is possible to avoid outputting an erroneous estimation result that the degree of arousal is low. That is, it is possible to eliminate erroneous determination caused by the bound-eye state.

  In addition, in the arousal level estimation apparatus 100 according to the present embodiment, the eye detection unit 102 may detect an image of the (N + 1) th frame that is a detection target, and an image of the (N + 1) th frame that is a target of detection of the eye. A range corresponding to the vicinity range of the eye position detected in the image of the Nth frame immediately preceding in time series is set as detection target ranges 1103 and 1106 (FIG. 11), and the set detection target range is set. Perform eye detection.

  In a plurality of images that are connected in time series, the eye position does not differ greatly between successive images, and the eye position in any image is usually detected in the immediately preceding image that is connected in time series. It exists in the range near the position of the eye.

  Therefore, a range corresponding to the vicinity range of the eye position detected in the immediately preceding image (such as the Nth frame with respect to the (N + 1) th frame) connected in time series is set as the target range (tracking region) for detecting the eye. As a result, the processing time can be significantly reduced as compared with the case where the eye detection process is performed with the entire image as the target range.

  On the other hand, it is rare that the image of the Nth frame immediately before in time series does not exist or the eye cannot be detected by the eye detection operation performed on the set detection target ranges 1103 and 1106 and the like. It can happen.

  In such a case, a situation occurs in which eyes existing in any part of the image cannot be detected only by performing processing with only the set range as an eye detection target.

  However, the arousal level estimation apparatus 100 according to the present embodiment first performs eye detection on a range corresponding to the vicinity range of the eye position detected in the immediately preceding image that is continuous in time series, and the detection target range. When the eye cannot be detected, eye detection is performed for the entire range of the image, thereby making it possible to reduce both the detection processing time and the detection probability.

  In addition, the arousal level detection apparatus 100 according to the present embodiment is obtained by the reliability determination unit 106 performing time differentiation of the time series transition of the eye opening degree P detected by the eye opening degree transition detection unit 104. The reliability is determined according to the distribution of the time differential value K of the time series transition.

  When noise is generated in the image signal including a change in the state of ambient light in the passenger compartment, the time-series transition of the eye opening P changes rapidly in a short time.

  Therefore, when the reliability determination unit 106 obtains the distribution of the time derivative K of the time-series transition of the eye opening P and determines the reliability according to the distribution, the noise is generated in the image signal. When the noise is not generated, it can be clearly distinguished, and the reliability can be accurately determined by simple arithmetic processing.

  Furthermore, in the arousal level detection apparatus 100 according to the present embodiment, the reliability determination unit 106 determines that the absolute value | K | of the time differential value K exceeds a predetermined threshold for the distribution of the time differential value K of the time series transition. The frequency is obtained, and the reliability is determined according to the obtained frequency.

  When noise occurs in the image signal, the absolute value | K | of the time differential value K of the opening degree P of the eye increases, so that the absolute value | K | of the time differential value K exceeds a predetermined threshold value. By detecting, it is possible to detect when noise is generated in the image signal, but even when no noise is generated in the image signal, the absolute value | K | of the time differential value K exceeds a predetermined threshold value. It is possible.

  However, the number of times that the absolute value | K | of the time differential value K exceeds a predetermined threshold is clearly different between when the noise is generated in the image signal and when no noise is generated.

  Therefore, by determining the reliability according to the frequency with which the absolute value | K | of the time differential value K exceeds a predetermined threshold, when the noise is generated in the image signal, and when the noise is not generated, The reliability can be determined clearly and reliably.

  Note that the reliability determination unit 106 applies a reference time interval for obtaining a frequency (threshold value) serving as a reference for determining reliability, or a reference frequency value for determining reliability according to the frequency. Further, correction according to individual differences of the driver 10 (subject) reflected in the image to be detected may be performed.

  Here, the correction amount corresponding to the individual difference may be obtained in advance by experiments or the like.

  According to the arousal level estimation apparatus 100 configured as described above, it is possible to eliminate the influence of individual differences among subjects, and thus it is possible to accurately determine the reliability regardless of individual differences.

  Note that the combination of the determination of the reliability based on the time-series transition of the differential value K of the eye opening P and the determination of the arousal level is a wakefulness level when the tendency to suppress a general false alarm is used. If the reliability is low even if it is determined that the vehicle is low, due to the low wakefulness, the alarm output that the vehicle occupant prevention device (alarm buzzer etc.) 202 issues is limited, It is also possible to change the type of notification / notification (change the volume, pitch, type of sound, and rhythm).

  Further, in the arousal level detection apparatus 100 according to the present embodiment, (1) the reliability determination result by the reliability determination unit 106 is high, and (2) the eye opening H detected by the eye opening transition detection unit 104 is high. Of the time differential value K distribution of the time series transition obtained by time differentiation of the time series transition of the time, the time differential value K is in a negative range, or the time differential value K is in a positive range , The area of each area exceeds the predetermined value, and the area of the integrated area range in which the absolute value of the time differential value K is within the predetermined reference range (| K | ≦ L) is sequentially integrated (3 ) When the integrated value obtained by sequentially integrating the areas of the integrated area range exceeds a predetermined value within a predetermined period (wakefulness determination period), the wakefulness determination means 105 reduces the wakefulness. When it is determined that there is a high degree of wakefulness, normal blinking Among the ranges where the time differential value K is positive, even if the absolute value of the area of the range is often below the predetermined value or exceeds the predetermined value, the time differential value K is predetermined. Therefore, the state of reduced arousal level can be accurately determined by a simple calculation process.

It is a block diagram which shows the arousal level estimation apparatus which is one Embodiment of the arousal level estimation apparatus which concerns on this invention. It is a schematic diagram which shows the state by which the arousal level estimation apparatus shown in FIG. 1 was mounted in the vehicle. 3 is a flowchart (part 1) for explaining the operation of the arousal level estimation apparatus shown in FIG. 4 is a flowchart (part 2) for explaining the operation of the arousal level estimation apparatus shown in FIG. 6 is a flowchart (part 3) for explaining the operation of the arousal level estimation apparatus shown in FIG. 6 is a flowchart (part 4) for explaining the operation of the arousal level estimation apparatus shown in FIG. It is a figure explaining an example of the eye detection process by an eye detection part, and the process of detecting the density fluctuation part (feature) of a predetermined condition by performing a main scan in the vertical direction and a sub-scan in the horizontal direction for a two-dimensional gray image It is explanatory drawing. It is the figure which extracted only each characteristic part in FIG. It is a figure explaining the grouping of the characteristic in FIG. It is a figure explaining the process which specifies each feature corresponding to the element of a face. It is a figure explaining a tracking area | region, (a) shows the tracking area | region set in the image of the Nth frame, (b) shows the tracking area | region set in the image of the (N + 1) th frame, respectively. It is explanatory drawing of prescribing | regulating eye height as an opening degree, (a) shows the opening degree in an open eye state, (b) shows the opening degree in a closed eye state, respectively. It is explanatory drawing of prescribing | regulating the aspect ratio of an eye with an opening degree, (a) shows the opening degree in a state with a large image magnification, (b) shows the opening degree in a state with a small image magnification, respectively. It is explanatory drawing of prescribing | regulating the curvature radius of the upper edge of eyes as the opening degree P, (a) is a state with a small curvature radius, (b) is a state with a large curvature radius, (c) Image of an example of a closed eye state (D) The figure showing the eye of a closed eye state detected from the image of (c), (e) The figure showing the eye of the open eye state detected from the image which is not shown in figure, respectively is represented. In a state of high reliability, (a) shows a time-series transition of the eye opening, in which the vertical axis represents the eye opening and the horizontal axis represents the frame number (number of frames) of time-series images. A graph and (b) are time-sequential transitions of the time differential value of the eye opening obtained by time differentiation of the eye opening in (a). In a state with low reliability, (a) is a graph showing the time-series transition of the eye opening, and (b) is a graph showing the time-series transition of the time differential value of the eye opening. It is a figure which shows the list which investigated and summarized the occurrence frequency of the blink within each predetermined time about 26 passengers. It is a time-series transition of the differential value of the opening degree of the eye, (a) is a diagram showing an appropriate detection state of blinking, (b) shows a state where appropriate detection is not performed such as noise superposition. FIG. In the case where one normal blink is normally detected in a state where the degree of arousal is high, (a) is a graph showing a time-series transition of the eye opening, and (b) is a time of the eye opening. It is a graph which shows the time-sequential transition of a differential value. In the case where one blink which is longer than usual in a state in which the degree of arousal starts to decrease is normally detected, (a) is a graph showing the time-series transition of the eye opening, and (b) is the eye It is a graph which shows the time-sequential transition of the time differential value of the opening degree. FIG. 5A is a graph showing a time-series transition of the eye opening, and FIG. 5B is a graph showing a time-series transition of the time differential value of the eye opening.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Arousal degree estimation apparatus 101 Eye opening degree detection means 102 Eye detection part 103 Eye opening degree detection part 104 Eye opening degree transition detection part 105 Arousal degree judgment means 106 Reliability judgment means 200 Vehicle 201 CCD camera 202 Dozing prevention apparatus

Claims (10)

Eye opening degree detecting means for detecting an eye opening degree included in the image;
Eye opening transition detecting means for detecting a time series transition of the eye opening detected by the eye opening detecting means for a plurality of time series images,
Arousal degree determination means for determining arousal level based on a time-series transition of the opening degree of the eye detected by the eye opening degree transition detection means;
Reliability determination means for determining the reliability of the detection result of the eye opening by the eye opening detection means based on the time-series transition of the eye opening detected by the eye opening change detection means And a wakefulness level estimation device.   The eye opening detection means includes an eye detection unit that detects the eye, and an eye opening detection unit that detects the opening of the eye based on the eye detected by the eye detection unit. The arousal level estimation apparatus according to claim 1, wherein   The eye detection unit is configured to detect a range corresponding to a vicinity range of an eye position detected in an immediately preceding image connected to the detection target image in time series with respect to the target image to be detected. The eye is detected for the set detection target range, or there is no immediately preceding image connected in time series, or the eye detection performed for the set detection target range is performed. 3. The arousal level estimation apparatus according to claim 2, wherein when the eye cannot be detected by a detection operation, the eye is detected for the entire range of the detection target image.   The eye opening degree detection unit obtains a height along the vertical direction of the eye detected by the eye detection unit, and detects the obtained height as the eye opening degree. The arousal level estimation apparatus according to claim 2 or 3.   The eye opening detection unit obtains a rectangular aspect ratio circumscribing the eye detected by the eye detection unit, and detects the obtained aspect ratio as the eye opening. The arousal level estimation apparatus according to claim 2 or 3.   The eye opening detection unit obtains a curvature radius of the upper edge of the eye detected by the eye detection unit, and detects the obtained curvature radius as the eye opening. The arousal level estimation apparatus according to claim 2 or 3. The reliability determination means includes
Time-differentiating the time-series transition of the eye opening detected by the eye-opening transition detecting means, and determining the reliability according to the distribution of the time-differential value of the obtained time-series transition The awakening level estimation apparatus according to any one of claims 1 to 6, wherein The reliability determination means includes
Regarding the distribution of time differential values of the time-series transition, the frequency that the absolute value of the time differential value exceeds a predetermined threshold is determined, and the reliability is determined according to the obtained frequency. The arousal level estimation apparatus according to claim 7, wherein The reliability determination means includes
A reference time interval for obtaining the frequency serving as a reference for determining the reliability, or a reference frequency value for performing the reliability determination according to the frequency is reflected in the image to be detected. The wakefulness estimation apparatus according to claim 8, wherein correction according to individual differences of subjects is performed. (1) The reliability determination result by the reliability determination means is high,
(2) With respect to the distribution of the time differential value of the time series transition obtained by time differentiation of the time series transition of the eye opening detected by the eye opening transition detection means, the time differential value is negative. Of the integrated area range in which the absolute value of the area of the range exceeds a predetermined value and the time differential value is within a predetermined reference range. Accumulate area sequentially,
(3) When the integrated value obtained by sequentially integrating the areas of the integrated area range exceeds a predetermined value within a predetermined period,
The arousal level estimation device according to any one of claims 1 to 9, wherein the arousal level determination unit determines that the arousal level is lowered.

Images