JP2019122446A - Examinee state estimation device and examinee state estimation method - Google Patents

Abstract

To determine the state of an examinee including sleepiness, a fatigue degree, and a tension degree utilizing information relating to blinking action of the eyes of the examinee.SOLUTION: An examinee state determination device (driver state determination device 10) includes: a blinking action detection unit 11 detecting an eye blinking action that the examinee (e.g., a driver during driving a vehicle) performs unconsciously; a blinking interval detection unit 12 which detects a blinking interval including information of a time from a certain blinking action to the next blinking action among a plurality of times of blinking actions performed continuously; and an examinee state determination unit 13 which determines the state of the examinee on the basis of the blinking interval detected by the blinking interval detection unit.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a subject state determination device and a subject state determination method for determining the state of a human (subject) including drowsiness, fatigue and tension. In addition, a human state is a mental or sensory state of a human, and refers to the state which a human feels in a brain. Further, in the subject state determination device and the subject state determination method according to the present invention, it is possible to determine the driver's state, in particular, with the driver (driver) driving the vehicle as the subject.

  As a prior art which determines the state of the driver which drives a vehicle, the technique using the blink (blink, blink) of the driver obtained from the image which imaged the face of the driver is known. For example, Patent Documents 1 and 2 below disclose a technique for detecting blinks of a driver from an image obtained by capturing an image of the driver's face, and determining the awakening degree of the driver based on the occurrence frequency of long blinks longer than usual. It is done.

JP, 2011-62465, A JP-A-9-277849

  The techniques disclosed in Patent Literatures 1 and 2 both measure the blink time (for example, the time from the beginning of the closing of the eyelid to the reopening of the eyelid), and the frequency of short blinks is high when awakening; The driver's awakening level is determined using the tendency of human blinks that the frequency of long time flickering is high when the awakening level is low.

  However, although the techniques disclosed in Patent Documents 1 and 2 can obtain a certain effect in the determination of the driver's alertness, they have the problem that measurement of the individual blink time is susceptible to disturbances and the environment in the vehicle. In addition, as to whether or not the blink time is long, a simple binary judgment using a preset threshold value is performed, so that the determination of the driver's arousal level largely depends on the validity of the preset threshold value. It has the problem of being dependent. Furthermore, the driver's state other than the driver's awakening level can not be determined, and there is a problem that the application range is narrow.

  SUMMARY OF THE INVENTION In view of the various problems described above, the present invention is a new technique for determining the condition of a subject using information related to the blink (blink, blink: meaning eyelid opening and closing movement) motion of the subject's eyes. An object of the present invention is to provide a subject state determination device and a subject state determination method used.

In order to achieve the above object, the subject condition determination apparatus of the present invention is a subject condition determination apparatus that determines the condition of the subject including drowsiness, fatigue and tension of the subject,
A blink motion detection unit that detects a blink motion of the eye performed by the subject unconsciously;
A blink for detecting a blink interval including time information from one blink operation to the next blink operation for a plurality of continuously performed blink events detected by the blink operation detection unit. Interval detection unit,
A subject state determination unit that determines the state of the subject based on the blink interval detected by the blink interval detection unit;
Have.

Further, in order to achieve the above object, the subject condition determination method of the present invention is a subject condition determination method for determining the condition of the subject including drowsiness, fatigue and tension of the subject,
A blink action detection step of detecting an eye blink action performed by the subject unconsciously;
A blink for detecting a blink interval including time information from one blink operation to the next blink operation for a plurality of successive blink operations detected in the blink operation detection step. Interval detection step,
A subject state determination step of determining the state of the subject based on the blink interval detected in the blink interval detection step;
Have.

  According to the present invention, a subject state determination device and a subject state determination using a new technology that determines the state of a subject including drowsiness, fatigue and tension using information related to blink behavior of the subject's eyes It becomes possible to provide a method.

FIG. 3 is a view schematically showing an example of a state in the vehicle in a case where the driver in driving the vehicle is a subject in the embodiment of the present invention. It is a block diagram which shows an example of the function of the test subject state determination apparatus (driver state determination apparatus) in embodiment of this invention. It is a figure for demonstrating the eye-opening degree of the driver's eye in an embodiment of the present invention, (a) is a figure showing an example of the state which opened the eye the most, (b) is a figure showing an example of the state closed most. (C) is a figure which shows an example of the eye-opening degree in one reciprocation blink operation. It is a graph which shows typically an example of eye opening degree data detected by the blink operation detection part of the test subject state judging device (driver state judging device) in an embodiment of the present invention. It is a block diagram which shows an example of the function of the driver state determination part in embodiment of this invention. It is a figure for demonstrating the outline | summary of the process performed by the non-linear analysis part of the driver state determination part in embodiment of this invention. It is a flowchart which shows an example of the process performed by the frequency spectrum analysis part of the driver state determination part in embodiment of this invention. It is a block diagram which shows an example of the function of the test subject state determination apparatus (driver's state determination apparatus) in another embodiment of this invention. It is a graph which shows typically an example of eye-opening degree data detected by the blink operation detection part of the test subject state judging device (driver's state judging device) in another embodiment of the present invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In addition, although the case where a driver (driver) in driving a vehicle is a subject is described below as an example, a subject targeted by the present invention is not limited to the driver in driving a vehicle, and any state may be used. And it is possible to make a human subject who is performing action.

  FIG. 1 is a view schematically showing an example of a state in a vehicle in a case where a driver in driving the vehicle is a subject in the embodiment of the present invention. FIG. 1 schematically shows that the driver is seated at the driver's seat in the vehicle. In the vehicle, the camera 1 is installed at a position where the driver's face can be imaged.

  The camera 1 is an imaging device for imaging the face of the driver, in particular the eyes of the driver, and can use, for example, a camera used in existing technologies such as an eye mark recorder and a driver status monitor. The camera 1 may image both eyes of the driver or may image either eye.

  The camera 1 performs imaging at a constant cycle to obtain an image capable of detecting a blink of a driver (blink, blink: means opening and closing movement of the eyelid), in particular, a periodic blink performed unknowingly It has a function. The speed of human blinks (the time required for one reciprocation of opening and closing of the eyelid in one blink) is generally said to be 100 to 150 milliseconds, and it is a high-speed camera that captures still images at high speed. In the case of using a (high speed camera) or a video camera that captures a moving image at a high frame rate (for example, 120 fps or more), it is possible to easily observe the blink operation. On the other hand, when the human eye is imaged by the low frame rate camera 1, there is a possibility that the blinked frame may be dropped. However, if the frame rate is about 15 to 30 fps or more, it is possible to detect the blink operation at a high recognition rate by using the image processing technology.

  In the present invention, it is found that the fluctuation related to the blink of a subject (driver) obtained by the present inventor represents the state of the subject (driver) (for example, the state including drowsiness, fatigue and tension). Is based on In the present specification, the case of detecting the blink operation from the image taken by the camera 1 will be described, but in the present invention, it is only necessary to detect the blink operation of the driver (and the fluctuation related to this blink operation). The technology relating to detection is not particularly limited. For example, when an image of the driver's eye is continuously acquired and blink is detected using an image processing technique, the degree of eye openness (open eye / closed eye state) is determined based on the edge strength (edge contrast) of the eye. The determination may be made to detect the blink operation, or the blink operation may be detected based on the size of the area of the black eye. Further, based on the reflected light from the black eye, a blink detection may be detected using a light detection technique. Furthermore, although the above-mentioned image processing technology and light detection technology detect the blink operation without contacting the driver, the electrodes and the magnetic induction element are arranged around the driver's eye (for example, the driver However, it is also possible to employ a technique for detecting the blinking operation by capturing potential fluctuations or magnetic fluctuations by putting on special glasses or the like).

  An electrical signal (imaging signal including imaged image data) obtained by imaging with the camera 1 is supplied to a driver state determination device (subject state determination device) 10. The driver state determination device 10 can be realized by a general-purpose or dedicated computer provided with a CPU (Central Processing Unit: central processing unit) and a GPU (Graphics Processing Unit: graphic processing device). The driver state determination device 10 may be connected to the camera 1 via a wired or wireless connection in order to receive an imaging signal from the camera 1, or the imaging signal is once imaged from the camera 1 to a storage device (not shown). It may be stored as image data, and may be configured to be connected to the storage device and to read out captured image data. The driver state determination device 10 may be installed at any place in the vehicle as shown in FIG. 1 or at a remote place, and picks up an imaging signal from the camera 10 via a communication network or the like. It may be received.

  FIG. 2 is a block diagram showing an example of the function of the driver state determination device according to the embodiment of the present invention. The driver state determination device 10 illustrated in FIG. 2 includes a blink operation detection unit 11, a blink interval detection unit 12, and a driver state determination unit 13.

  The blink operation detection unit 11 has a function of detecting the blink operation of the eye of the driver. The blink operation detection unit 11 generates data (eye opening degree data) indicating the blink operation by performing image processing on, for example, captured image data of the eye of the driver captured by the camera 1. Here, as an example, the case where the blink operation detection unit 11 detects the blink operation of the driver by performing image processing on the captured image data of the eye of the driver will be described, but as described above, the blink operation of the driver Can use any detection technology including image processing technology, light detection technology, potential fluctuation detection technology, magnetic fluctuation detection technology and the like.

  Here, the eye opening degree data detected by the blink operation detection unit 11 based on the captured image data received from the camera 1 will be described. FIG. 3 is a view for explaining the degree of eye opening of the driver according to the embodiment of the present invention, and FIG. 3 (a) shows an example of a state in which the eye is most opened, FIG. 3 (b) 1 shows an example of the state in which the eyes are most closed, and FIG. 3 (c) shows an example of the degree of eye opening in one reciprocation blink operation.

  The blink operation detection unit 11 performs processing related to the image of the driver's eye included in the captured image data received from the camera 1, and outputs the image recognition processing of the driver's eye and eye opening degree data indicating the eye opening degree as an example. And the function to The blink operation detection unit 11 recognizes, for example, the most open state shown in FIG. 3A, sets the degree of eye opening in the most open state to 100%, and is shown in, for example, FIG. Recognize the most closed state, and set the eye opening degree of the most closed state to 0%.

  For the most opened state and the most closed state, for example, image recognition is performed on an image of the eye of the driver for a predetermined time as a step before outputting the eye opening degree data, and the most open eye detected during this predetermined time The closed state and the most closed state may be set as an eye opening degree of 100% and 0%, respectively. Here, the eye opening degree is determined by measuring the distance between the edge of the upper eyelid of the eye and the edge of the lower eyelid of the eye, but the black eye portion of the eye (the pupil and iris surrounded by the corneal ring) The eye open / closed state may be quantified based on any criteria, such as measuring the distance in the longitudinal direction of the included portion to determine the eye open degree. Furthermore, the present invention only needs to be able to capture the interval of the blink action of the subject (driver) (the time interval for closing the eyes with a crack), and may recognize only the closed eye state. For example, when the occupancy rate of the skin color of the eyelids is equal to or greater than a predetermined value (the upper eyelid is closed and the skin color spreads over the predetermined area over the eye area), the occupancy rate of the black eye portion is predetermined A state below the value (upper eyelid closed and part of the black eye is not visible), and a state where the edge of the upper eyelid and the edge of the lower eyelid overlap (state where the eye is closed) is recognized as a closed eye state Thus, the time interval between closed eyes (closed eye interval) may be defined as the blinking interval.

  As described above, for example, when the eye opening degree in the most open state is set to 100% and the eye opening degree in the most closed state is set to 0%, the blink operation detection unit 11 is shown in FIG. As shown, it is possible to represent the degree of eye opening that changes with the blinking operation as a numerical value between 0% and 100%. It should be noted that the normal state of the eye is looking at a slightly closed state (for example, about 80% of the eye opening degree) than the state in which the eye is most open, and the closed state of the eye in periodic blinks performed unconsciously is It is in a slightly opened state (for example, about 10% of the degree of eye opening) than the state in which the eye is most closed.

  FIG. 4 is a graph schematically showing an example of the degree of eye opening degree data detected by the blink operation detection unit of the driver state determination device according to the embodiment of the present invention. The blink operation detection unit 11 outputs the eye opening degree as a numerical value, and when the numerical value indicating the eye opening degree is plotted on the time axis, a graph as shown in FIG. 4 is obtained. In the graph shown in FIG. 4, the vertical axis represents the degree of eye opening, and the horizontal axis represents time. FIG. 4 is a graph drawn for the purpose of explaining the degree of eye opening in the blink operation, and the scale of time on the horizontal axis is not necessarily exactly described. The time for one blink operation is generally 100 to 150 milliseconds as described above, and the blink operation interval (blink interval) is generally on the order of several seconds.

  The blink interval detection unit 12 has a function of detecting an interval (blink interval) of the blink operation based on the eye opening degree data output from the blink operation detection unit 11. The blink interval refers to the time from when one blink operation, which is the opening and closing motion of the eyelid, is performed to when the next blink operation is performed next. For example, as shown by the dotted line in the graph shown in FIG. 4, based on the timing at which eye closure is started in eye blink operation (eye closure start timing), eye closure start of the next blink operation from eye closure start timing of one blink operation It is possible to set the blink interval between blinks to the timing as the blink interval. The definition of the blink interval is not limited to the above, and for example, the blink interval is based on an arbitrarily determined rule, such as based on the timing when eye opening is started from the closing eye and the central timing during the closing eye. It may be defined.

  FIG. 5 is a block diagram showing an example of the function of the driver state determination unit according to the embodiment of the present invention. For example, as illustrated in FIG. 5, the driver state determination unit 13 includes a non-linear analysis unit 131, a power spectrum analysis unit 132, a DFA (Detrended Fluctuation Analysis) analysis unit 133, and a determination result output unit. It consists of 134. The driver state determination unit 13 does not have to include all the non-linear analysis unit 131, the power spectrum analysis unit 132, and the DFA analysis unit 133, and any of the non-linear analysis unit 131, the power spectrum analysis unit 132, and the DFA analysis unit 133. One or more may be provided.

  The non-linear analysis unit 131 has a function of receiving information including the blink interval from the blink interval detection unit 12 and performing non-linear analysis processing. Specifically, the non-linear analysis unit 131 sequentially receives information related to the blink interval detected by the blink interval detection unit 12, generates an attractor based on the received information related to the blink interval, and generates the attractor generated based on the generated attractor. It has the function of performing processing to calculate the Lyapunov index.

  An outline of processing in the non-linear analysis unit 131 is shown in FIG. Information including blink intervals from the blink interval detection unit 12 is, for example, information arranged in time series so as to be represented as P [1], P [2], P [3], P [N],. Handled. The nonlinear analysis unit 131 plots a set of N data (P [1], P [2],..., P [N]) adjacent to the time series on an N-dimensional space, and sequentially sets the data. While shifting, as in (P [2], P [3], ..., P [N + 1]), (P [3], P [4], ..., P [N + 2]), ..., in the N-dimensional space Plot sequentially on the coordinates. In addition, in FIG. 6, an example in the case of N = 3 is represented typically. A line connecting coordinates in an N-dimensional space in which a set of blink interval data is plotted draws a locus to form an attractor. It is possible to calculate the Lyapunov index by analyzing the stable tendency with respect to the past attractor group from the attractor generated by the information related to the blink interval. The calculation of the Lyapunov index can be performed in any cycle, and as a result, Lyapunov index time series data is generated. The inventor has found that the Lyapunov Index obtained from the information on blink intervals is correlated with the state of physical function and brain function. For example, since the lower the Lyapunov index tends to lower the driver's physical function and brain function, the Lyapunov index can be used as an indicator that represents the driver's drowsiness and fatigue level.

  The frequency spectrum analysis unit 132 acquires the Lyapunov index calculated by the non-linear analysis unit 131, determines the power spectrum density of the Lyapunov index time series data based on the frequency spectrum analysis technology, and determines the low frequency band from the power spectrum density. And the function of calculating the value of the power spectral density of each of the high frequency bands. For example, the frequency spectrum analysis unit 132 obtains the power spectrum density of the Lyapunov exponent time series data calculated by the non-linear analysis unit 131, and calculates the integral LF of the low frequency band and the integral HF of the high frequency band. Is possible.

  An example of processing in the frequency spectrum analysis unit 132 will be described with reference to FIG. The frequency spectrum analysis unit 132 first acquires time-series data x (t) of Lyapunov exponents calculated by the non-linear analysis unit 131 (step S201). Subsequently, the frequency spectrum analysis unit 132 calculates the power spectral density PSD (f) of the acquired Lyapunov exponent time-series data x (t) (step S202). The power spectral density is described, for example, in FIG. 7 as an example of the formula (for example, the range of the frequency f is set to 0 Hz to 1 Hz), and the calculation method thereof is performed using the conventional frequency spectrum analysis technique. The same method as the method to be used may be used.

  Then, the frequency spectrum analysis unit 132 calculates the integrated value LF of the low frequency band of the obtained power spectral density PSD (f) (step S203), and further integrates the high frequency band of the power spectral density PSD (f) The value HF is calculated (step S204). Although the low frequency band exists on the lower frequency side than the high frequency band, both bands may have an overlap, and as an example, the low frequency band is defined as 0.04 Hz to 0.15 Hz, and the high frequency band Can be defined as 0.15 Hz to 0.4 Hz.

  Further, the frequency spectrum analysis unit 132 shifts the time series data x (t) of the power spectrum density to be sampled along the time series, and integrates the integrated value LF of the low frequency band of the power spectrum density PSD (f), the power spectrum density By calculating the integrated value HF of the high frequency band of PSD (f), output of a predetermined sampling rate interval is performed. In the present inventor, the activity state of the driver's sympathetic nerve has a correlation with the ratio (LF / HF) of the integral value LF to the integral value HF, and the activity state of the parasympathetic nerve of the driver has a correlation with the integral value HF. I have found that I have. For example, the present inventor normalizes the time series data of LF / HF and HF, and obtains the normalized time series data of LF / HF and normalized HF, respectively. Difference between normalized HF differential time series data and normalized LF / HF differential time series data (differentiation time series data of normalized HF-differential time series data of normalized LF / HF) is driver fatigue It is found that it can be used as an indicator of degree or tension. That is, the integral value LF and the integral value HF can be used as an index indicating the degree of fatigue or tension (relaxation) of the driver.

The DFA analysis unit 133 has a function of performing DFA analysis based on the information related to the blink interval, and outputting the DFA analysis result (scaling index α). Note that DFA analysis is a method for analyzing long-term correlation characteristics of non-stationary time-series data, and is a widely known technique, so detailed description will be omitted here. In DFA analysis, new time-series data created by integrating input data is divided into boxes of length n, and in each box, an approximate straight line is calculated by the least squares method to obtain a local trend, and a trend is obtained from the signal After removal, the variance is averaged to calculate a value (F (n)). F (n) determined by this DFA analysis can be represented by F (n) ∝n ^α and is characterized by the scaling index α. The scaling index α is a numerical value indicating the state or characteristics of the driver. As an example, it is determined whether the scaling index α is apart from a predetermined value (for example, α = 1.0), and if it is largely separated from the predetermined value (for example, α <0.8 or the like), It can be used as an indicator that can determine that an abnormality has occurred in the state of the driver. It is also possible to refer to the scaling index to determine the threshold in the determination using the Lyapunov index.

  The determination result output unit 134 is a Lyapunov exponent output from the non-linear analysis unit 131, an integrated value of a low frequency band of the power spectral density PSD (f) output from the DFA analysis unit 133 output from the power spectrum analysis unit 132 It has a function of using the integrated value HF of LF and high frequency band and the scaling index α output from the DFA analysis unit 133 as an index indicating the state of the driver to determine the state of the driver and outputting the determination result. When the Lyapunov index is used as an index indicating the state of the driver, for example, the drowsiness or fatigue degree of the driver when the Lyapunov index is lower than a predetermined threshold (the threshold may be set depending on the scaling index α) May be determined to be high. In addition, when the scaling index α is used as an index indicating the state of the driver, it is assumed that the driver's body function or brain function is abnormal when the scaling α is lower than a predetermined threshold (for example, α = 0.6) You may judge. In addition, when using the integrated value LF of the low frequency band of the power spectral density PSD (f) or the integrated value HF of the high frequency band as an index indicating the state of the driver, “differentiation time series data of normalized HF-normalized LF When the numerical value of “/ HF differential time-series data” is higher than a predetermined threshold value, it may be determined that the driver's degree of fatigue or tension is high.

  Further, in the above embodiment, the blink interval detection unit 12 is configured to detect the blink interval based on the eye opening degree data output from the blink operation detection unit 11, but another embodiment of the present invention As shown in FIG. 8, instead of the blink interval detection unit 12, a blink closed eye time detection unit 21 may be provided to detect the time during which the eye is closed by the blink operation (blink closed eye time). The eyelid closing time refers to the time during which the eyelid is closed in one eyelid operation which is the opening and closing movement of the eyelid. For example, as indicated by a dotted line in the graph shown in FIG. 9, let the time during which the eye opening degree is less than a predetermined eye opening degree (40% in FIG. Is possible. In addition, the definition of blink closing time is not limited to the above, and based on the time from the timing when the eye closing was started to the timing when the eye opening was completed, the blink closing time is based on an arbitrarily defined rule It may be defined.

  The blink eye closed time detection unit 21 has a function of outputting the sum of the blink eye closed times detected within a predetermined time window (for example, 30 seconds or 1 minute). Furthermore, the blink closed eye time detection unit 21 has a function of outputting a predetermined time window while shifting each predetermined time (for example, 10 seconds). Specifically, for example, the blink closed eye time detection unit 21 calculates and outputs the sum of the blink closed eye times within the time window ranging from time t1 seconds to time t1 seconds + 30 seconds, and then, the time window is set to 10 A process of calculating and outputting the sum of blink closed eye times within a time window ranging from time t1 seconds + 10 seconds to time t1 seconds + 40 seconds is sequentially performed repeatedly with a time shift of seconds. As a result, the blink closed eye time detection unit 21 outputs the sum of the blink closed eye times in different time windows every 10 seconds.

  The driver state determination unit 13 has a function of determining the state of the driver based on the sum of the blink eye closing time output from the blink eye closing time detection unit 21 instead of the blink interval output from the blink space detection unit 12. doing. The process in the driver state determination unit 13 is the same as the above-described process related to the blink closed eye time, and can be understood by replacing the wording of the blink interval with the sum of the blink closed eye time.

  In addition, although the case where the subject while driving the vehicle is the subject is described as an example in the present specification, the subject targeted by the present invention is not limited to the driver while driving the vehicle. It is possible to use a human subject who is performing any state and action. For example, the user state such as the user's drowsiness, fatigue, and tension may be determined from the blink operation of the user who performs desk work.

  Although each function of the hardware configuration illustrated in the drawings is illustrated by a block, each of these functions may be processed by hardware such as an IC (Integrated Circuit) chip and / or It can be realized by a computer executing a program.

  The present invention is applicable to a technique for determining the condition of a subject including drowsiness, fatigue and tension, and in particular, for determining the condition of a driver while driving the vehicle as a subject. Applicable to technology.

1 Camera 10 Driver state determination device (subject state determination device)
11 blink detection unit 12 blink interval detection unit 13 driver state determination unit 131 nonlinear analysis unit 132 frequency spectrum analysis unit 133 DFA analysis unit 134 determination result output unit

Claims (12)

A subject state determination device that determines the state of the subject including the subject's sleepiness, fatigue and tension,
A blink motion detection unit that detects a blink motion of the eye performed by the subject unconsciously;
A blink for detecting a blink interval including time information from one blink operation to the next blink operation for a plurality of continuously performed blink events detected by the blink operation detection unit. Interval detection unit,
A subject state determination unit that determines the state of the subject based on the blink interval detected by the blink interval detection unit;
Subject state determination device having.   The subject state determination unit is configured to calculate a Lyapunov index related to the blink interval by performing non-linear analysis processing on time-series data of the blink interval, and to determine the state of the subject based on the Lyapunov index. The subject state determination apparatus according to claim 1.   The subject state determination unit calculates a power spectrum density of time series data of the Lyapunov exponent, and calculates an integrated value of a predetermined low frequency band at the calculated power spectral density and an integrated value of a predetermined high frequency band. The test subject state determination apparatus according to claim 2, wherein the test subject state determination apparatus is configured to calculate and determine the state of the subject based on an integral value of the predetermined low frequency band and an integral value of the predetermined high frequency band.   The subject state determination unit is configured to calculate a scaling index related to the blink interval by performing a DFA analysis process on time-series data on the blink interval, and determine the state of the subject based on the scaling index. The subject state determination apparatus according to any one of claims 1 to 3.   The blink interval detection unit according to claim 1, wherein the blink interval detection unit is configured to detect the blink operation performed by the subject unconsciously from a captured image obtained by capturing images of the subject's eyes continuously in time. The subject state determination device according to any one of 4.   The subject state determination apparatus according to any one of claims 1 to 5, wherein the subject is a driver driving a vehicle. A subject state determination method for determining the state of the subject including the subject's sleepiness, fatigue and tension,
A blink action detection step of detecting an eye blink action performed by the subject unconsciously;
A blink for detecting a blink interval including time information from one blink operation to the next blink operation for a plurality of successive blink operations detected in the blink operation detection step. Interval detection step,
A subject state determination step of determining the state of the subject based on the blink interval detected in the blink interval detection step;
The subject state determination method to have.   The subject state determination step has a step of calculating a Lyapunov index related to the blink interval by performing non-linear analysis processing on time-series data of the blink interval, and determining the state of the subject based on the Lyapunov index The subject state determination method according to claim 7.   The subject state determination step calculates a power spectrum density of time series data of the Lyapunov index, and calculates an integrated value of a predetermined low frequency band at the calculated power spectral density and an integrated value of a predetermined high frequency band. 9. The method according to claim 8, further comprising the steps of: calculating and determining the state of the subject based on the integrated value of the predetermined low frequency band and the integrated value of the predetermined high frequency band.   The subject state determination step has a step of calculating a scaling index related to the blink interval by performing a DFA analysis process on time-series data of the blink interval, and determining the state of the subject based on the scaling index The subject state determination method according to any one of claims 7 to 9.   The blink interval detection step has a step of detecting the blink operation performed by the subject unconsciously from a captured image obtained by imaging the subject's eyes continuously in time. The subject state determination method according to any one.   The subject state determination method according to any one of claims 7 to 11, wherein the subject is a driver driving a vehicle.

Images