JP2009078006A - Sleepy state determining device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a sleepy state determining device capable of determining with higher accuracy a sleepy state which occurs to a driver or the like driving an automobile or the like. <P>SOLUTION: The sleepy state determining device includes: a camera 12 imaging the eyes of a person 11; an initial pupil diameter computing section 25 computing a pupil diameter in the initial state of the person based on image data obtained from an imaging signal from the camera; a storage section 24 storing the pupil diameter in the initial state; a pupil diameter characteristic value computing section 26 computing pupil diameter characteristic values of the person within a predetermined period based on the image data obtained from subsequent imaging signals; a comparative determining section 27 comparing the pupil diameter in the initial state with the pupil diameter characteristic values within the predetermined period to determine whether the pupil diameter has a tendency to become gradually small on the basis of the pupil diameter in the initial state; and a sleepy sign state determining section (a sleepy state determining section 28) determining a sleepy sign state when the pupil diameter is determined to have a tendency to become gradually small. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a drowsiness state determination apparatus, and more particularly to a drowsiness state determination apparatus suitable for extracting in advance with high accuracy a predictive state of drowsiness caused by a driver or the like driving an automobile or the like.

There are techniques disclosed in Patent Documents 1 to 3 as conventional techniques for determining whether a driver who drives an automobile or the like is in a drowsiness state. In the driver state detection device disclosed in Patent Document 1, the driver's gaze direction and pupil diameter are detected, the gaze direction frequency distribution (gaze behavior) and the change rate of the pupil diameter are calculated, and the gaze direction frequency distribution and the pupil are calculated. The driver state is detected by combining the rate of change in diameter. In the moving body control device disclosed in Patent Document 1, the change speed of the driver's pupil diameter is detected to determine the state of the driver, and reflected in the control of the moving body such as an automobile. The wakefulness detection device disclosed in Patent Literature 3 detects the circularity of the pupil region related to the human eye and monitors the blinking state based on the change in the circularity to determine a decrease in the human wakefulness state. It is configured.
JP 2002-367100 A JP 2000-351337 A JP-A-6-270711

  All of the devices disclosed in Patent Documents 1 to 3 above illuminate the human eye (pupil) to be evaluated using an illuminating device (infrared irradiation or the like), and the image of the eye with a camera or the like. Is acquired and image processing is performed.

  In addition, it has been reported that when a human becomes sleepy, the pupil diameter greatly fluctuates at a low frequency, and the fluctuation width may change greatly from the initial pupil diameter to the maximum pupil diameter. Therefore, simply calculating the pupil diameter change rate, the pupil diameter change speed, and the circularity of the pupil region cannot accurately determine the human drowsiness state. Therefore, the conventional determination technique is not so high in accuracy, and a determination technique with higher accuracy is required for the drowsiness state of the driver or the like.

  In view of the above problems, an object of the present invention is to provide a drowsiness state determination apparatus that can perform a higher accuracy determination on a drowsiness state generated by a driver or the like driving an automobile or the like.

  The drowsiness state determination apparatus according to the present invention is configured as follows to achieve the above object.

  A first drowsiness state determination device (corresponding to claim 1) includes an imaging unit that images a region including a human eye, and a pupil in an initial state of a human based on image data obtained from an imaging signal output from the imaging unit An initial pupil diameter calculating means for calculating the diameter, a storage means for storing the pupil diameter in the initial state, and a predetermined period based on the image data obtained from the imaging signal output from the imaging means after calculating the pupil diameter in the initial state The pupil diameter characteristic value calculation means for calculating the human pupil diameter characteristic value in the inside is compared with the pupil diameter characteristic value in the initial state and the pupil diameter characteristic value within the predetermined period, and the pupil diameter is determined based on the pupil diameter in the initial state. A comparison determination unit that determines whether or not the pupil diameter gradually decreases, and a drowsiness predictor state determination that determines that a human is in a drowsiness predictor state when the comparison determination unit determines that the pupil diameter has a tendency to gradually decrease. Means and Thus constructed.

  In the above drowsiness state determination device, the fluctuation of the pupil diameter uses the “monotonic contraction pupil section” that is a temporal change characteristic when the drowsiness sign state occurs, and the “low frequency fluctuation section” that appears subsequently. A drowsiness sign state is determined with high accuracy based on accurately extracting a change in pupil diameter and comparing the change characteristics of the pupil diameter.

  In the above configuration, the second drowsiness state determination device (corresponding to claim 2) is preferably characterized in that the pupil diameter characteristic value in the predetermined period is an average value of the pupil diameter in the predetermined period.

  According to the present invention, with respect to fluctuations in the pupil diameter of a target person, a “monotonic reduced pupil section” that is a temporal change characteristic when a drowsiness predictive state occurs, a “low frequency fluctuation section” that appears subsequently, and the like are utilized. Since it is configured to accurately extract changes in the pupil diameter, it is possible to determine a drowsiness predictor state with high accuracy based on comparing the change characteristics of the pupil diameter with the reference pupil diameter, and driving an automobile or the like It is possible to reliably prevent a drowsiness sign state that occurs in a person.

  DESCRIPTION OF EMBODIMENTS Preferred embodiments (examples) of the present invention will be described below with reference to the accompanying drawings.

  FIG. 1 shows a block configuration of a sleepiness state determination apparatus according to an embodiment of the present invention, and FIG. 2 shows a procedure of processing for determining a sleepiness state.

  In FIG. 1, the code | symbol 11 shows the subject's head which needs to be evaluated for a sleepiness state. The target person is a driver who is riding a car or truck and performing a driving operation, or a driver who is driving a motorcycle. For the target person 11, for example, a camera 12 is installed toward the front of the face. The camera 12 captures an area including the eyes of the target person 11 in particular. The camera 12 images the eye (pupil) of the target person 11 that needs to be evaluated, and uses the imaging information to perform data processing, image processing, By performing the determination process, the change in the pupil diameter of the target person 11 is monitored, and the sleepiness state or the drowsiness sign state of the target person is determined.

  The camera 12 is installed at an appropriate location in the front of the driver in the case of an automobile or the like, and is attached to a location such as a driver's helmet in the case of a motorcycle. A normal imaging device is used for the camera 12. In this case, in principle, when natural ambient light is present, the eye area of the target person 11 is not illuminated. The camera 12 images the eye of the target person in the natural light state in the environment where the target person 11 exists. At this time, the state of the pupil diameter of the eye (pupil) of the target person 11 is determined in a natural state. However, it is desirable to irradiate light such as infrared rays at night and the like when there is no natural ambient light.

  The imaging signal output from the camera 12 is input to the sleepiness state determination device 13. The sleepiness state determination device 13 is configured by a computer, and its internal functional elements (data processing function, image processing function, determination processing function, etc.) are realized by a program or the like created based on a predetermined algorithm.

  As basic functional elements constituting the sleepiness state determination device 13, an interface 21 that captures an imaging signal output from the camera 12, an A / D conversion unit 22 that converts an imaging signal that is an analog signal into a digital image signal, On the basis of a real-time imaging signal continuously supplied from the camera 12, a pupil image data extraction unit 23 that extracts image data of eyes or pupil portions from a digital image signal, a storage unit 24 that stores image data of pupil portions, and thereafter An initial pupil diameter calculator 25 that calculates the initial pupil diameter of the target person 11 based on the obtained pupil image data, the storage unit 24 that stores data relating to the initial pupil diameter, and the initial pupil diameter. After calculation, the object within a predetermined period based on the image data obtained as described above by the imaging signal output from the camera 12 A pupil diameter characteristic value calculation unit 26 for calculating a pupil diameter characteristic value (a later-described feature amount used for determination) between the pupil diameter 24a in the initial state and the pupil diameter characteristic value within a predetermined period, and comparing the initial state The determination unit 27 determines whether or not the pupil diameter has a tendency to gradually decrease (trend) with the pupil diameter 24a as a reference, and the comparison determination unit 27 determines that the pupil diameter has a tendency to gradually decrease (trend). The target person 11 has a drowsiness state determination unit 28 that determines that the target person 11 is in a drowsiness sign state. This drowsiness state determination unit 28 is a broad sense drowsiness state determination unit, and determines a drowsiness state (low frequency fluctuation state) as will be described later in addition to the functional part (drowsiness sign state determination unit) that determines the above drowsiness sign state. Functional part (narrowly defined sleepiness state determination unit). The result of the determination is output from the sleepiness state determination unit 28.

  In the sleepiness state determination device 13 configured to include the functional elements (21 to 28) as described above, a process for determining the sleepiness state of the target person 11 based on the processing procedure illustrated in FIG. Execute. Before describing the flowchart shown in FIG. 2, the concept of sleepiness state determination will be described.

  FIG. 3 shows a time change characteristic 30 of the pupil diameter by the driving simulator when a drowsiness state (decrease in arousal state) occurs in the driving state driver. In FIG. 3, the horizontal axis represents time (minutes), and the vertical axis represents pupil diameter (mm). When a drowsiness occurs in a driving driver, the pupil diameter of the driver's pupil changes as shown in FIG. In the time variation characteristic 30 of the pupil diameter, the monotonic pupil segment 31 first occurs in a period of about 0 to 4 minutes, and the low frequency fluctuation segment 32 subsequently occurs in a period of about 4 to 10 minutes. It turns out. The monotonic reduced pupil section 31 indicates a drowsiness sign state in which the pupil diameter tends to gradually decrease. Further, the low frequency fluctuation section 32 indicates a state of drowsiness. In addition to the monotonic reduced pupil section 31 and the low frequency fluctuation section 32, a normal awakening section exists.

  Therefore, in the drowsiness state determination of the present invention, the pupil diameter state based on the imaging information of the eye of the target person 11 captured by the camera 12 is based on the temporal change characteristic 30 of the pupil diameter shown in FIG. Calculate the amount. As described above, in order to determine the monotonic reduced pupil section 31, the low frequency fluctuation section 32, and the wakefulness section, for example, using data every 40 seconds (hereinafter referred to as “one segment data”) every 20 seconds. Calculate the amount.

  As feature quantities used for the determination, (1) the initial pupil diameter (PD1) at the start of the determination process, (2) the difference (ΔPD) between the initial pupil diameter (PD1) and the pupil diameter average value (PD2) of each segment (3) The regression linear coefficient (α) with respect to the time change of the pupil diameter in one segment, its standard deviation (SD1), and the standard deviation (SD2) after removal of the trend component of the pupil diameter change. For these feature quantities, thresholds (TH1, TH2, TH3) with high generalization that do not depend on the target person 11 are set, and each section that is found in pupil diameter variation is determined based on the following determination algorithm.

(1) Judgment of awakening interval When a high arousal level is maintained, the ΔPD is kept at a small value. Therefore, a segment in which ΔPD is equal to or smaller than a predetermined threshold TH1 is determined as a “wakeful segment”.

(2) Determination of Low Frequency Fluctuation Section 32 In the low frequency fluctuation section, data variation increases due to fluctuations as compared with the awakening section and the monotonic reduced pupil section. Therefore, a segment that is not determined to be an awakening section (when ΔPD is larger than the threshold value TH1) and the SD1 is equal to or greater than the predetermined threshold value TH2, or the segment where the SD2 is equal to or greater than the predetermined threshold value TH3. Is determined as a “low frequency fluctuation segment”.
If the previous segment is a low frequency fluctuation segment, the threshold value is manipulated so that the current segment is easily determined to be a low frequency fluctuation segment.

(3) Determination of monotonic reduced pupil section 31 A segment in which α is negative and not determined as a low frequency fluctuation section is determined as a “monotonic reduced pupil segment”.

(4) Correction of erroneous determination In the determination for the monotonic reduced pupil section 31, when the previous or previous segment is a monotone reduced pupil segment, the section that has not been determined to be a monotonic reduced pupil is also re-determined as a monotonic reduced pupil section.

  The sleepiness state determination process executed by the sleepiness state device 13 will be described according to the flowchart of FIG.

  When the drowsiness state determination process is started, a pupil image is first acquired (step S11). The pupil image is acquired from the imaging signal output from the camera 12 described above. Next, an image binarization process is executed for the pupil image (step S12). When the pupil image is binarized, a circular pupil region and an annular iris region can be identified on the pupil image. Calculated (step S13). The pupil diameter calculated in the initial state immediately after the start of the drowsiness state determination process is stored in the storage unit 24 as the initial pupil diameter PD1 (step S14). The imaging signal continuously output from the camera 12 continues to be supplied to the sleepiness state determination device 13 thereafter. Therefore, the drowsiness state determination device 13 continues to acquire the current pupil image (step S15). The current pupil image is similarly binarized (step S16), and the pupil diameter is further calculated (step S17). Note that pupil diameter data obtained with respect to continuously acquired pupil images is stored in the storage unit 24 for a required period. As a result, in the next step S18, an average pupil diameter PD2 in an arbitrary time interval is calculated. Here, the arbitrary time interval is one segment described above.

  In the next step S19, a difference ΔPD between the initial pupil diameter PD1 and the average pupil diameter PD2 in an arbitrary time interval (one segment) is calculated. It is determined whether or not the calculated difference ΔPD is equal to or less than a threshold value TH1 (step S20). If YES in determination step S20, it is determined that the user is in an awake state (“wake segment”) (step S27), and then the process proceeds to determination step S30. When the determination step S20 is NO, the process proceeds to step S21.

  In step S21, a regression line coefficient α of pupil diameter-time in an arbitrary time section (one segment) is calculated. As a result, a regression line (change characteristic) for specifying the temporal change of the pupil diameter in an arbitrary time interval is determined by linear approximation.

  Next, in step S22, the standard deviation SD1 of the pupil diameter in an arbitrary time interval (one segment) is calculated. The calculation of the standard deviation SD1 is performed on the basis of a determined regression line that identifies the time change of the pupil diameter. In an arbitrary time period, that is, one segment, one segment is 40 seconds, and 30 frames of pupil images are acquired per second, so 1200 pupil diameter data are obtained. The standard deviation of these 1200 pupil diameter data is calculated as SD1 based on the determined regression line.

  In the next step S23, the standard deviation SD2 after the removal of the trend of pupil diameter change in an arbitrary time interval (one segment) is calculated. The standard deviation SD2 is calculated by calculating the difference between the actual pupil value and the calculated value approximated by the regression line for each of the above-mentioned 1200 pupil diameter data, collecting 1200 of the difference data, and calculating the 1200 difference. The standard deviation of the data is SD2.

  In the subsequent determination step S24, for the two standard deviations SD1 and SD2 obtained as described above, it is determined whether SD1 is equal to or greater than a predetermined threshold TH2 or whether SD2 is equal to or greater than a predetermined threshold TH3. Is done. If YES in the determination step S24, the current arbitrary time interval (one segment) is determined to be the low frequency fluctuation interval 32, that is, the “low frequency fluctuation segment”, and the thresholds TH1 and TH3 are adjusted. After (step S28), it is determined that “they are drowsy (sleepy)” (step S29). Thereafter, the process proceeds to determination step S30. If NO in the determination step S24, it is determined in the next determination step S25 whether or not the regression linear coefficient α is less than 0. If YES, it is “a drowsiness sign state”, that is, monotonous. It is determined as the miosis segment 31 (“monotonic miosis segment”). If NO in determination step S25, the process proceeds to determination step 30.

  In determination step S30, it is determined whether or not to continue the drowsiness state determination process. In the determination step S30, in the case of YES, the process returns to step S15 and the above process is repeated, and in the case of NO, the sleepiness state determination process is ended.

  The sleepiness state determination apparatus 13 determines the sleepiness state, sleepiness sign state, and arousal state of the target person 11 according to the determination process shown in FIG. Regarding how to use the judgment result, especially “monotonic miosis” is a predictive signal that the user wants to sleep a few minutes later (for example, three minutes later). Be careful with ventilation, etc. " Alternatively, an air conditioner or audio may be activated automatically, or a nearby service area can be guided to encourage rest.

  In addition, when “low frequency fluctuation” is occurring, the person is consciously drowsy, so in addition to the above various warnings, a warning such as “Wake up for ventilation, etc.” is displayed. It may be emitted.

  Although the application example of the drowsiness state determination device according to the present embodiment has been described with respect to an example of an automobile and a two-wheeled vehicle, it is not limited to this. The drowsiness state determination apparatus according to the present invention can also be applied to the case of monitoring an operator such as a mobile body such as a ship or an aircraft, a personal computer, or a television. For example, when this method is applied to a TV, the TV viewer's arousal level is judged, and when the viewer's arousal level is low, and the TV is likely to fall asleep soon, the TV will be automatically turned off. Can be used. At the same time, the lighting of the living space can be interlocked so that it is dimmed or turned off.

  The configurations, shapes, sizes, and arrangement relationships described in the above embodiments are merely schematically shown to the extent that the present invention can be understood and implemented, and the numerical values and the compositions (materials) of the respective components Is just an example. Therefore, the present invention is not limited to the described embodiments, and can be variously modified without departing from the scope of the technical idea shown in the claims.

  A drowsiness state determination apparatus according to the present invention is used to accurately determine a drowsiness sign state of a driver of an automobile or the like and reliably prevent the occurrence of a drowsiness state.

It is a block diagram which shows the apparatus structure of the drowsiness state determination apparatus which concerns on embodiment of this invention. It is a flowchart which shows the process of the sleepiness state determination performed with the sleepiness state determination apparatus which concerns on this embodiment. It is a graph which shows the time change characteristic of the pupil diameter by a driving simulator when a sleepiness state (decrease in arousal state) occurs in a driving state driver.

Explanation of symbols

11 Target human (head)
DESCRIPTION OF SYMBOLS 12 Camera 13 Drowsiness state determination apparatus 23 Pupil image data extraction part 24 Memory | storage part 25 Initial pupil diameter calculation part 26 Pupil diameter characteristic value calculation part 27 Comparison determination part 28 Drowsiness state determination part

Claims (2)

An imaging means for imaging an area including the human eye;
An initial pupil diameter calculating means for calculating a pupil diameter in the initial state of the human based on image data obtained from an imaging signal output from the imaging means;
Storage means for storing the pupil diameter in the initial state;
After calculating the pupil diameter in the initial state, pupil diameter characteristic value calculating means for calculating the human pupil diameter characteristic value within a predetermined period based on image data obtained from the imaging signal output from the imaging means;
Comparison determining means for comparing the pupil diameter in the initial state with the pupil diameter characteristic value in the predetermined period and determining whether or not the pupil diameter has a tendency to gradually decrease with reference to the pupil diameter in the initial state. When,
A drowsiness sign state determination unit that determines that the human is in a drowsiness sign state when the comparison determination unit determines that the pupil diameter has a tendency to gradually decrease;
A drowsiness state determination device comprising:   The sleepiness state determination apparatus according to claim 1, wherein the pupil diameter characteristic value within the predetermined period is an average value of the pupil diameters during the predetermined period.

Images