JP3158494U - Mobile communication terminal - Google Patents

Abstract

A mobile communication terminal having a wide range of practical conditions for detecting a sign before a driver of a vehicle is aware of drowsiness is provided. In a mobile communication terminal, a vehicle movement representing a head movement is detected by a vehicle movement detecting unit, an eye movement is detected by an eye movement detecting unit, and a vehicle movement is detected by an ideal eye movement angular velocity calculating unit. An ideal eye movement angular velocity is calculated based on the head movement data detected by the detection unit 11. Furthermore, the eyeball rotation angular velocity calculation unit 14 calculates the eyeball rotation angular velocity based on the eyeball movement data detected by the eyeball movement detection unit 12, and the drowsiness sign determination unit 15 calculates the vestibular oculomotor reflex from the ideal eyeball movement angular velocity and the eyeball rotation angular velocity. And a sign of sleepiness before the driver of the vehicle is aware of sleepiness is determined based on the vestibulo-oculomotor reflex. Then, when the drowsiness sign determination unit 15 detects a drowsiness sign, the warning sound generation unit 16 generates a warning sound. [Selection] Figure 1

Description

The present invention relates to a portable communication terminal capable of detecting a sign before a driver of a vehicle is aware of drowsiness using vestibulo-oculomotor reflexes induced by head movement.

It is effective to detect the arousal level objectively as a measure for preventing a decrease in the arousal level of the driver of the vehicle, that is, an accident caused by sleepiness. Conventionally, as a drowsiness detection device that detects drowsiness of a driver of a vehicle, a device that detects drowsiness by using biological signals such as brain waves, blinks, and eye movements caused by drowsiness as an index has been proposed. For example, in Patent Document 1, a blink having a closing time longer than the evaluation time is detected based on a blink evaluation time set from a blink closing time specific to the driver at awakening. A drowsiness detection device that detects drowsiness from the ratio of a long blink with respect to is disclosed.

Japanese Patent Laid-Open No. 10-272960

When detecting the degree of arousal objectively, it is effective as an accident prevention measure to detect a wakefulness decline rather than increasing the degree of arousal once and prevent the decline of arousal by appropriate measures. Yes. However, the above-described technique has a problem that it is possible to detect a lowered arousal level, that is, drowsiness, but it is impossible to detect a sign of a reduced arousal level. On the other hand, as a suggestion of the possibility of detecting signs of drowsiness, a method of detecting a decrease in arousal level focusing on fluctuations in the pupil of a vehicle driver has been proposed. When the subject is aware of sleepiness, large low frequency fluctuation (LLFF) occurs in the pupil,
Since monotonic miosis (GM) occurs before the drowsiness awareness, it has been suggested that LLFF can be an indicator of drowsiness and GM can be an indicator of drowsiness before awareness. However, the size of the pupil changes depending on the brightness of the external environment and the distance to the point of gaze, so it is considered that the pupil functions effectively on a straight road without night oncoming vehicles. There is a risk that highly reliable sleepiness cannot be detected in nighttime driving situations where there are many.
In addition, there is a problem that an expensive dedicated device is necessary because of a large calculation load and a device for imaging the pupil.

Therefore, an object of the present invention is to realize a mobile communication terminal that can easily detect a sign before a driver of a vehicle is aware of drowsiness using a function of the mobile communication terminal.

In order to achieve the above object, according to the first aspect of the present invention, in the device according to claim 1, a vehicle motion detection unit that detects the motion of the vehicle, an eye movement detection unit that detects an eye movement of the driver of the vehicle, and the eyeball A drowsiness sign determination unit that detects a drowsiness sign of a vehicle driver based on data acquired by the motion detection unit and the vehicle motion detection unit, and a warning sound when the drowsiness sign determination unit detects a drowsiness sign A warning sound generating unit that generates a warning sound, and an ideal eye movement angular velocity of a driver of the vehicle based on the vehicle motion data measured by the vehicle motion detection unit in the drowsiness sign determination unit The eye movement is detected based on the eyeball image of the vehicle driver imaged by the eye movement detection unit, the eyeball rotation angular velocity is calculated based on the detected eye movement data, and the ideal eyeball movement is calculated. Detecting the vestibular ocular reflex (VOR) from an angular velocity and the eyeball rotation angular velocity, determining the sign of drowsiness based on the vestibular ocular reflex, using technical means that.

As a measure for preventing accidents caused by a decrease in the arousal level of a vehicle driver, it is important to detect the arousal level objectively. As an objective indicator for detecting human drowsiness, the applicant uses reflective eye movements to rotate the eyeball in the opposite direction at almost the same speed as the head movement, and to obtain a blurred vision. Paying attention to a certain vestibulo-ocular reflex (VOR), the effectiveness was confirmed by experiments. According to the first aspect of the present invention, the drowsiness sign determination unit provided in the mobile communication terminal calculates the ideal eye movement angular velocity of the driver of the vehicle based on the vehicle movement data measured by the vehicle movement detection unit. Here, since the linear acceleration and the rotational angular velocity according to the linear acceleration and the rotational angular velocity acting on the vehicle act on the driver's body, the vehicle motion data detected by the vehicle motion detector is generated on the driver's head. The ideal eye movement angular velocity of the driver of the vehicle can be calculated using the head movement data as representative data. Then, the eye movement is detected based on the eyeball image of the driver of the vehicle imaged by the eye movement detection unit, the eyeball rotation angular velocity is calculated based on the detected eye movement data, and the ideal eye movement angular velocity and the eyeball rotation are calculated. A vestibular movement reflex (VOR) is detected from the angular velocity, and a sign of sleepiness before the driver of the vehicle is aware of drowsiness can be determined based on the vestibular movement reflex. Then, when the drowsiness sign determination unit detects a drowsiness sign, the warning sound generation unit can generate a warning sound, so that the driver can be aware that sleepiness will occur in the future. Can be a trigger for awakening and resting. That is, it is possible to catch a sign that the driver's arousal level is reduced and prevent the reduction of the arousal level.
Vestibulo-oculomotor reflexes are not easily affected by the external environment such as ambient brightness, so that the practical condition range can be widened. In addition, since the calculation load is small, real-time measurement and determination are possible, and processing can be performed at a practical speed with a CPU generally mounted in a mobile communication terminal.

According to a second aspect of the present invention, in the mobile communication terminal according to the first aspect, the drowsiness predictor determining unit approximates the eyeball rotation angular velocity with the following equation that is a linear expression of the ideal eye movement angular velocity. The technical means of calculating the VOR gain defined in (1) and the rate of decrease of the VOR gain and determining that it is a sign of drowsiness when the rate of decrease of the VOR gain exceeds a preset threshold value is used.
e (t) = G h (t-τ) + dc + ε (t)
e (t): Eye rotation angular velocity, G: VOR gain, h (t): Ideal eye movement angular velocity, τ: Delay time of eye movement relative to head movement, dc: Constant term, ε (t): Remaining regression model difference

Since the decrease in VOR gain occurs before awareness of sleepiness, it is effective as an index indicating a sign of sleepiness. As described in claim 2, the drowsiness predictor determination unit is preferably used for determination of a drowsiness sign in order to determine a drowsiness sign when the rate of decrease in the VOR gain exceeds a preset threshold. be able to.

According to a third aspect of the present invention, in the portable communication terminal according to the first or second aspect, the drowsiness predictor determination unit approximates the eyeball rotation angular velocity with the following equation that is a linear expression of the ideal eye movement angular velocity. The approximate residual defined by ε (t) and the residual standard deviation are calculated, and when the increase rate of the residual standard deviation exceeds a preset threshold, it is determined as a sign of sleepiness The technical means is used.
e (t) = G h (t-τ) + dc + ε (t)
e (t): Eye rotation angular velocity, G: VOR gain, h (t): Ideal eye movement angular velocity, τ: Delay time of eye movement relative to head movement, dc: Constant term, ε (t): Remaining regression model difference

Since the increase in the residual standard deviation occurs before awareness of sleepiness, it is effective as an index indicating a sign of sleepiness. As the invention according to claim 3, the drowsiness sign determination unit determines that it is a drowsiness sign when the increase rate of the residual standard deviation exceeds a preset threshold value, and is therefore suitable for determination of a drowsiness sign Can be used. In particular, when used together with the device according to claim 2, it is possible to improve the accuracy of determining a drowsiness sign.

It is a block diagram of a portable communication terminal. It is a flowchart which shows the drowsiness sign detection method. 1 is a schematic diagram of an experimental system that simulates driving of a vehicle. It is explanatory drawing which shows a test result. FIG. 4A is a plot in which the ideal eye movement angular velocity and the eyeball rotation angular velocity at the time of a typical subject's arousal (during mental calculation task) are overlapped. FIG. 4B is a plot of the data of FIG. 4A with the eyeball rotation angular velocity on the vertical axis and the ideal eye movement angular velocity on the horizontal axis. 4 (c) and 4 (d) are test results in a section in which the subject felt sleepy. FIG. 4 (c) is shown in FIG. 4 (a), and FIG. 4 (d) is shown in FIG. 4 (b). It corresponds. It is explanatory drawing which compared VOR gain and SDres with the pupil diameter change. It is explanatory drawing which shows the relationship between the VOR gain reduction | decrease rate (DELTA) VOR (t) and SDres increase rate (DELTA) SDres (t), and the drowsiness precursor. FIG. 6A is a plot of the VOR gain reduction rate ΔVOR (t) and the SDres increase rate ΔSDres (t) in the six experimental data with the vertical axis and the horizontal axis, respectively. FIG. 6B is a plot of ΔVOR (t) and ΔSDres (t) from the end of the mental arithmetic task to the perception of sleepiness.

A mobile communication terminal according to the present invention will be described with reference to the drawings. The mobile communication terminal 10 is arranged at a position where a driver's eyeball image can be captured, for example, in the vicinity of the instrument panel.

The mobile communication terminal 10 is represented by a mobile phone, for example. As illustrated in FIG. 1, the mobile communication terminal 10 includes a vehicle movement detection unit 11 that detects movement of a vehicle, an eye movement detection unit 12 that detects eye movement of a driver of the vehicle, a vehicle movement detection unit 11, and an eyeball. A drowsiness sign determination unit 15 that detects a drowsiness sign of the driver of the vehicle based on data acquired by the motion detection unit 12, and a warning sound is generated when the drowsiness sign determination unit 15 detects a drowsiness sign A warning sound generator 16.

The vehicle movement detection unit 11, eye movement detection unit 12, drowsiness sign determination unit 15 and warning sound generation unit 16 are each composed of a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory) not shown. ). A program for controlling each unit is stored in a storage area provided in the drowsiness sign determination unit 15 or the like, or a ROM.

The drowsiness sign determination unit 15 includes an ideal eye movement angular velocity calculation unit 13 that calculates an ideal eye movement angular velocity based on the vehicle movement data detected by the vehicle movement detection unit 11, and eye movement data detected by the eye movement detection unit 12. And an eyeball rotation angular velocity calculation unit 14 that calculates the eyeball rotation angular velocity based on the above.

The vehicle motion detection unit 11 detects linear acceleration and rotational angular velocity acting on the vehicle, and uses a three-axis acceleration sensor built in the mobile communication terminal 10 or a gyroscope that detects rotational angular velocity, and The linear acceleration in the traveling direction, the up-down direction, and the left-right direction, and the rotational angular velocity in each of the rolling, pitching, and yawing directions of the driver can be detected. Since the linear acceleration and the rotational angular velocity corresponding to the linear acceleration and the rotational angular velocity acting on the vehicle act on the driver's body, the linear acceleration and the rotational angular velocity detected by the vehicle motion detector 11 are generated on the driver's head. It can be used as a value representative of linear acceleration and rotational angular velocity. Thereby, the head movement is detected, and the acceleration data and the rotational angular velocity data are sent to the ideal eye movement angular velocity calculator 13.

The eye movement detection unit 12 captures an eyeball image, and a built-in camera such as a video camera that is widely used in the mobile communication terminal 10 can be suitably used.

The ideal eye movement angular velocity calculation unit 13 calculates an ideal eye movement angular velocity that is an ideal eye movement angular velocity for compensating head movement based on the vehicle movement data detected by the vehicle movement detection unit 11.

The eyeball rotation angular velocity calculation unit 14 calculates an eyeball rotation angular velocity based on the eye movement data detected by the eye movement detection unit 12.

The drowsiness sign determination unit 15 calculates a VOR gain G and a residual standard deviation SDres, which will be described later, as parameters relating to vestibulo-ocular reflex (VOR) from the ideal eye movement angular velocity and the eyeball rotation angular velocity, and at least one of them. The sign of sleepiness is determined based on the parameters.

The warning sound generation unit 16 generates a warning sound when the drowsiness sign determination unit 15 detects a drowsiness sign. The warning sound can be selected from a ringing tone or music stored in a ROM or the like. As the warning sound generating unit 16, for example, a speaker provided in the mobile communication terminal 10 can be used.

Next, a method for detecting drowsiness signs will be described. First, the drowsiness sign detection program is started by operating the operation buttons and icons of the mobile communication terminal 10, and the eye movement detection unit 12 such as a video camera confirms that the driver's eye is being imaged on the display. Place it at a predetermined position such as near the instrument panel.

As shown in FIG. 2, in step S <b> 1, the vehicle motion detection unit 11 detects linear acceleration and rotation angular velocity acting on the vehicle, and sends the linear acceleration data and rotation angular velocity data to the ideal eye movement angular velocity calculation unit 13. Here, the vertical acceleration data in the vertical direction and the rotational angular velocity with respect to the pitching direction are used, but other components can also be used.

Next, in step S 2, the ideal eye movement angular velocity calculation unit 13 calculates the ideal eye movement angular velocity based on the vehicle movement detected in step S 1, and sends the ideal eye movement angular velocity data to the drowsiness sign determination unit 15.

In step S3, the eye movement detection unit 12 captures the eye movement, measures the amount of movement of the pupil center coordinates, and stores the eye movement data for each of the rolling, pitching, and yawing directions of the driver to the eye rotation angular velocity calculation unit 14. Send it out.

In step S4, the eyeball rotation angular velocity calculation unit 14 calculates the eyeball rotation angular velocity based on the eye movement data captured in step S3, and sends the eyeball rotation angular velocity data to the drowsiness sign determination unit 15.

In step S5, the drowsiness sign determination unit 15 obtains the following VOR gain G and residual standard deviation SDres as parameters relating to the vestibulo-oculomotor reflex based on the ideal eye movement angular velocity data and eyeball rotation angular velocity data. Here, the vestibulo-ocular reflex (VOR) is a reflective eye movement for rotating the eyeball in the opposite direction at substantially the same speed as the head movement to obtain a blur-free field of view.

The VOR gain is obtained by least-squares estimation as a regression model coefficient G in which the objective variable is an eyeball rotation angular velocity e (t) and the explanatory variable is an ideal eye movement angular velocity h (t) and a constant term dc. Here, ε (t) is the residual of the regression model, and τ is the delay time of the eye movement with respect to the ideal eye movement. Here, the VOR gain may be calculated in at least one direction of the driver's traveling direction, vertical direction, and horizontal direction.

(Equation 1)
e (t) = G h (t-τ) + dc + ε (t)

The residual standard deviation SDres is calculated by the following equation. Here, N is the number of data points to be measured.

Since the decrease in the VOR gain and the increase in the residual standard deviation occur before the sleepiness is recognized, it is effective as an index indicating a sign of sleepiness. Therefore, in step S6, in order to quantify the increase in the VOR gain and the increase in the residual standard deviation, the following index is calculated by the drowsiness sign determination unit 15. That is, since the initial value and the amount of change of the VOR gain and the residual standard deviation SDres vary depending on individual differences, the average value of the high arousal state is obtained and the rate of change relative to the average value is calculated. The VOR gain decrease rate ΔVOR (t) and the SDres increase rate ΔSDres (t) at a certain time are respectively defined and calculated as follows.

In step S7, the drowsiness sign determination unit 15 compares the VOR gain decrease rate ΔVOR (t) and the SDres increase rate ΔSDres (t) obtained in step S6 with preset threshold values th1 and th2, respectively. If both the VOR gain decrease rate ΔVOR (t) and the SDres increase rate ΔSDres (t) exceed the threshold (S7: YES), it is determined that a sign of sleepiness has been detected, and the process proceeds to the subsequent step S8. To do. If at least one of the VOR gain decrease rate ΔVOR (t) and the SDres increase rate ΔSDres (t) is equal to or less than the threshold (S7: NO), it is determined that no sign of drowsiness is detected, and the series of processing ends.

In step S8, the drowsiness sign determination unit 15 outputs a drowsiness sign detection signal to the warning sound generation unit 16, the warning sound generation unit 16 generates a warning sound, and the series of processing ends.

As described above, according to the mobile communication terminal 10 of the present invention, a sign of sleepiness before the driver of the vehicle is aware of sleepiness can be determined based on the vestibulo-oculomotor reflex. Vestibulo-oculomotor reflexes are not easily affected by the external environment such as ambient brightness, so that the practical condition range can be widened. In addition, since the calculation load is small, real-time measurement and determination are possible. In addition, the warning sound can make the driver aware that sleepiness will occur in the future, so that the driver can recover the arousal level or trigger a break.

In the drowsiness sign detection method described above, the processing order of steps S1 and S2 and steps S3 and S4 is arbitrary.

The data measured and calculated in the above steps can also be stored in a storage area provided in the drowsiness sign determination unit 15. According to this, by analyzing each data later, for example, it is possible to grasp the situation in which the driver has a drowsiness sign and contribute to safe driving.

(Example)
The effect of the drowsiness sign determination method used in the mobile communication terminal 10 of the present invention was experimentally verified. FIG. 3 shows a schematic diagram of an experimental system that simulates driving of a vehicle. The driving simulator (DS) system 21 includes a projector that projects a DS image, and a driver seat that includes a screen, a steering, an accelerator, and a brake. The driver seat was placed at the position where the subject's head and the center of the screen face each other. The distance between the front surface of the subject's head and the screen is 2470 mm, and the screen size is 100 inches (left / right viewing angle ± 39.1 deg, vertical viewing angle ± 26.3 deg). A vibration device 22 for inducing vestibulo-oculomotor reflexes was installed below the driver seat. The brightness and contrast of the projector were adjusted so that the pupil diameter of the subject was approximately the middle diameter of the pupil movable range (diameter of about 6 mm).

The eyeball image was measured at 29.97 fps by an eyeball rotation imaging device 23 corresponding to the eye movement detection unit 12 of the mobile communication terminal 10. For head movement, a three-axis acceleration sensor (not shown) and three gyroscopes are mounted on the eyeball rotation imaging device 23,
The linear acceleration and rotational angular velocity in each of the three axis directions were measured. The head movement data was synchronized with the eyeball image, AD-converted by the AD / DA converter 24 at a sampling frequency of 1 kHz, and recorded in the data storage PC 25. Further, the eyeball image and head movement data were branched and input to the arithmetic processing PC 26, and the head movement, eye movement and pupil diameter change were observed in real time.

Here, the three-axis acceleration sensor and the gyroscope are in the vehicle motion detection unit 11, the eyeball rotation imaging device 23 is in the eyeball motion detection unit 12, and the arithmetic processing PC 26 is the ideal eyeball motion angular velocity calculation unit 13, eyeball rotation angular velocity calculation unit. 14 and drowsiness sign determination unit 15 respectively.

The test subject was seated on the driver seat in a natural posture after wearing the eyeball rotation imaging device 23, the acceleration sensor, and the gyroscope, and was subjected to the experimental task described below.

First, in order to measure the position of a stationary eyeball for 20 seconds from the start of measurement, the upper part of the number plate of the preceding vehicle was pointed at the point of sight and instructed to fixate without driving the vibration device 22. Next, after 20 seconds had elapsed, the vibration device 22 was driven to induce VOR. During this time, the subject kept staring at the gazing point and operated the steering so as not to deviate from the driving lane. Also, a simple mental arithmetic task was imposed for 3 minutes after driving the vibration device in order to maintain the awake state. After 3 minutes, the mental arithmetic task was stopped according to the examiner's instructions, and the DS operation and vibration stimulation were continued until the experiment was completed. The DS operation time was 15 minutes including the mental arithmetic task section, and after the experiment, the subject was given a reflection report regarding the presence or absence of sleepiness during the mental arithmetic task and the subsequent monotonous running.

(Data analysis)
The ideal eye movement angular velocity h (t) can be calculated from the linear acceleration and rotational angular velocity generated in the head. In this example, since the contribution of linear acceleration was small, it was calculated based on the rotational angular velocity of the head. The ideal eye movement angular velocity h (t) was calculated by the arithmetic processing PC 26 after removing noise by a digital bandpass filter, re-sampling at a sampling frequency of 29.97 Hz in order to match the eye movement data and the number of data points. The extraction of the eye movement angle and the pupil diameter was calculated by calculating the eyeball image by the eyeball rotation imaging device 23 by the calculation processing PC 26. The eyeball rotation angular velocity e (t) was obtained by differentiating the eyeball movement angle. Note that the ideal eye movement angular velocity h (t) can also be calculated taking into account the linear acceleration generated in the head.

From the ideal eye movement angular velocity h (t) and the eyeball rotation angular velocity e (t), the VOR gain and the residual standard deviation (SDres) were calculated by the calculation processing PC 26. The VOR gain and residual standard deviation (SDres) are 40-second data that can be obtained with sufficient estimation accuracy even after rapid phase removal from eye movement data, and each segment is 10 seconds with an overlap of 30 seconds. The value in each segment was calculated.

(Test results)
FIG. 4 is an explanatory diagram showing test results. FIG. 4A is a plot in which the ideal eye movement angular velocity and the eyeball rotation angular velocity at the time of a typical subject's arousal (during mental calculation task) are overlapped. It can be confirmed that the normal VOR in which the eye movement reversely occurs at almost the same speed with respect to the head movement is induced. FIG. 4B shows the rotation of the eyeball with the data of FIG. The plot is made by plotting the angular velocity and the ideal eye movement angular velocity on the horizontal axis, and the straight line is obtained by fitting the regression line of Equation 1. In this example, the VOR gain was 0.802 and SDres was 1.017.

The test results in the section in which the subject felt sleepy are shown in FIGS. 4 (c) corresponds to FIG. 4 (a), and FIG. 4 (d) corresponds to FIG. 4 (b). The VOR gain was 0.673 and the SDres was 2.964. The VOR gain was decreased and the SDres was increased as compared with the awake state.

FIG. 5 is an explanatory diagram comparing the VOR gain and SDres with changes in pupil diameter that have been confirmed to be effective as sleepiness and its predictive detection index. From 1 minute to about 2 minutes after the mental arithmetic task is stopped, the VOR gain gradually decreases, and SDres gradually increases in contrast. Furthermore, a monotonic miosis (GM) can be confirmed in the pupil diameter change in a section in which the VOR gain and SDres start to decrease and increase, respectively. In this GM section, it has been shown from previous studies that the subject is not yet aware of drowsiness. That is, a characteristic change such as a decrease in VOR gain or an increase in SDres in this interval is a sign signal of sleepiness.

About 2 minutes after the start of monotonic pupil reduction, the VOR gain and SDres further decrease and increase, respectively, and large low frequency fluctuation (LLFF) can be confirmed in the pupil diameter. LLFF is known to be a phenomenon that occurs when the subject perceives subjective sleepiness, and is consistent with the introspection report of subjects who have reported feeling sleepy from the middle to the second half of the experiment. That is, a decrease in the VOR gain and an increase in SDres are also a drowsiness predictor signal and a drowsiness index.

(Drowsiness sign determination method)
In the experimental task under the same condition as the above-mentioned experiment, the subject was informed about introspection about sleepiness every 2 minutes, and a sign of sleepiness was determined from changes in VOR gain and SDres. For the determination, the VOR gain decrease rate ΔVOR (t) and the SDres increase rate ΔSDres (t) defined by Equations 3 and 4 were used. FIG. 6 is an explanatory diagram showing the relationship between the VOR gain reduction rate ΔVOR (t) and the SDres increase rate ΔSDres (t) and the signs of drowsiness. FIG. 6A shows the VOR gain reduction in the six experimental data. The rate ΔVOR (t) and the increase rate of SDres ΔSDres (t) are plotted on the vertical axis and the horizontal axis, respectively. Black circles indicate data of an awake state, and X indicates data of a section where sleepiness is present. In the upper part and the right part of the figure, the distribution of each state is plotted with weighting so that the area becomes 1. Here, a threshold value is set for each of ΔVOR (t) and ΔSDres (t) so as to include all the data of the arousal state, an area below the two threshold values is set as an arousal area, and an area above the threshold value is set as a sleepiness area. FIG. 6B is a plot of ΔVOR (t) and ΔSDres (t) from the end of the mental arithmetic task to the perception of sleepiness. FIG. 6A shows that 95% of the data reported as having sleepiness is plotted in the sleepiness area, and it can be confirmed that the presence or absence of sleepiness can be separated. As can be seen from the respective distributions shown in FIG. 6 (b), most of the data exists in the arousal area, while some data also extends to the sleepiness area.

Here, in a state in which sleepiness is not perceived, a signal that is plotted on a sleepiness area and appears continuously for 40 seconds or more is defined as a predictive signal of sleepiness. This is because the effect of overlap may be included in the case of continuation within 30 seconds. As a result, 5 data (83.3%) showed signs of sleepiness. One data in which no signs of drowsiness were observed was also plotted in the sleepiness area, but was not recognized as a sign of drowsiness because it did not appear continuously for more than 40 seconds.

As a result, it was confirmed that a drowsiness symptom can be easily detected in most subjects by setting a subject-independent threshold for the high arousal state of each subject defined in advance.

[Effect of the embodiment]
(1) According to the mobile communication terminal 10 of the present invention, the vehicle movement detection unit 11 detects a vehicle movement that represents head movement, the eye movement detection unit 12 detects an eye movement, and an ideal eye movement angular velocity calculation unit. 13 calculates the ideal eye movement angular velocity based on the vehicle movement data detected by the vehicle movement detection unit 11, and the eye rotation angular velocity based on the eye movement data detected by the eye movement detection unit 12 by the eye rotation angular velocity calculation unit 14. And the vestibular ocular reflex (VOR) is detected from the ideal eye movement angular velocity and the eyeball rotation angular velocity by the drowsiness sign determination unit 15, and drowsiness before the vehicle driver is aware of drowsiness based on the vestibular ocular reflex. Can be determined. And when the drowsiness sign determination unit 15 detects a drowsiness sign, the warning sound generation unit 16 can generate a warning sound, so that the driver can be aware that sleepiness will occur in the future. The driver can use it as a trigger for recovery of arousal and a break. That is, it is possible to catch a sign that the driver's arousal level is reduced and prevent the reduction of the arousal level. Vestibulo-oculomotor reflexes are not easily affected by the external environment such as ambient brightness, so that the practical condition range can be widened. Further, since the calculation load is small, real-time measurement and determination are possible, and processing can be performed at a practical speed by a CPU generally mounted in the mobile communication terminal 10.

(2) The decrease in the VOR gain and the increase in the residual standard deviation are effective as an index indicating a sign of sleepiness because they occur before the sleepiness is recognized. The drowsiness sign determination unit 15 calculates the VOR gain decrease rate ΔVOR (t) and the SDres increase rate ΔSDres (t), respectively, and compares them with the preset thresholds L1 and L2, respectively, to obtain the VOR gain decrease rate ΔVOR (t ) And SDres increase rate ΔSDres (t) both exceed the threshold, it can be determined that a sign of sleepiness has been detected. This confirmed that most subjects could easily detect drowsiness signs.

[Other Embodiments]
In step S7, it is determined whether or not both the VOR gain decrease rate ΔVOR (t) and the SDres increase rate ΔSDres (t) exceed the threshold value. For simplicity, only one of them is calculated and the threshold value is set. A sign of drowsiness may be determined based on whether or not the number is exceeded.

DESCRIPTION OF SYMBOLS 10 Mobile communication terminal 11 Vehicle movement detection part 12 Eye movement detection part 13 Ideal eye movement angular velocity calculation part 14 Eyeball rotation angular velocity calculation part 15 Sleepiness sign determination part 21 Driving simulator (DS) system 22 Vibrating device 23 Eyeball rotation imaging device 26 Arithmetic processing PC

Claims (3)

A vehicle motion detector for detecting the motion of the vehicle;
An eye movement detector that detects the eye movement of the driver of the vehicle;
A drowsiness sign determination unit that detects a drowsiness sign of a driver of the vehicle based on data acquired by the eye movement detection unit and the vehicle movement detection unit;
A warning sound generating unit that generates a warning sound when the drowsiness sign determination unit detects a drowsiness sign; and
A mobile communication terminal comprising:
In the sleepiness sign determination unit,
Based on the vehicle movement data measured by the vehicle movement detector, the ideal eye movement angular velocity of the driver of the vehicle is calculated,
Detecting eye movement based on the eyeball image of the vehicle driver imaged by the eye movement detection unit;
Calculate the eye rotation angular velocity based on the detected eye movement data,
A portable communication terminal characterized by detecting a vestibular eye movement reflex (VOR) from the ideal eye movement angular velocity and the eyeball rotation angular velocity, and determining a sign of sleepiness based on the vestibular eye movement reflex. The drowsiness sign determination unit calculates a VOR gain defined by G and a rate of decrease of the VOR gain when the eyeball rotation angular velocity is approximated by the following equation that is a linear expression of the ideal eye movement angular velocity. 2. The mobile communication terminal according to claim 1, wherein when the gain reduction rate exceeds a preset threshold value, it is determined as a sign of drowsiness.
e (t) = Gh (t-τ) + dc + ε (t)
e (t): Eye rotation angular velocity, G: VOR gain, h (t): Ideal eye movement angular velocity, τ: Delay time of eye movement relative to head movement, dc: Constant term, ε (t): Remaining regression model difference The drowsiness sign determination unit calculates an approximate residual and a residual standard deviation defined by ε (t) when the eyeball rotation angular velocity is approximated by the following equation that is a linear expression of the ideal eye movement angular velocity, 3. The mobile communication terminal according to claim 1, wherein when the rate of increase of the residual standard deviation exceeds a preset threshold value, the mobile communication terminal is determined to be a sign of drowsiness.
e (t) = Gh (t-τ) + dc + ε (t)
e (t): Eye rotation angular velocity, G: VOR gain, h (t): Ideal eye movement angular velocity, τ: Delay time of eye movement relative to head movement, dc: Constant term, ε (t): Remaining regression model difference

Images