WO2020188629A1 - Dispositif de détermination d'état de vigilance, système d'éveil, et procédé de détermination d'état de vigilance - Google Patents

Dispositif de détermination d'état de vigilance, système d'éveil, et procédé de détermination d'état de vigilance Download PDF

Info

Publication number
WO2020188629A1
WO2020188629A1 PCT/JP2019/010829 JP2019010829W WO2020188629A1 WO 2020188629 A1 WO2020188629 A1 WO 2020188629A1 JP 2019010829 W JP2019010829 W JP 2019010829W WO 2020188629 A1 WO2020188629 A1 WO 2020188629A1
Authority
WO
WIPO (PCT)
Prior art keywords
period
control unit
state
awakening
state determination
Prior art date
Application number
PCT/JP2019/010829
Other languages
English (en)
Japanese (ja)
Inventor
あゆみ 竹本
Original Assignee
オムロン株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by オムロン株式会社 filed Critical オムロン株式会社
Priority to JP2021506797A priority Critical patent/JPWO2020188629A1/ja
Priority to PCT/JP2019/010829 priority patent/WO2020188629A1/fr
Publication of WO2020188629A1 publication Critical patent/WO2020188629A1/fr

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/18—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state for vehicle drivers or machine operators

Definitions

  • the alertness state regarding a person's alertness changes over time.
  • the arousal state includes a normal state, a drowsiness state in which the arousal level is lower than the normal state, and a drowsiness state in which the arousal level is further lower than the drowsiness state. If you fall into a drowsy or drowsy state with reduced alertness, you may reduce the work efficiency of a person or cause a serious accident. Therefore, it is required to determine the awake state of a person, for example, whether or not the awake state of a person is in the awake state.
  • Patent Document 1 various arithmetic processes are performed in order to detect minute changes such as a small amplitude or a frequency of occurrence of saccades in the vertical direction from the captured image of the driver and perform threshold determination.
  • minute changes such as a small amplitude or a frequency of occurrence of saccades in the vertical direction from the captured image of the driver and perform threshold determination.
  • a driver drives a vehicle or the like, there are various factors that reduce the accuracy of imaging, and it is easy to implement an imaging means or a calculation means capable of detecting minute changes as described above. Difficult to do.
  • the awakening state determination device sequentially acquires image data from an imaging unit that captures an image including the human eye and generates image data, and sequentially acquires image data from the imaging unit, and moves the human eye based on the acquired image data. It is provided with a control unit for detecting.
  • the control unit measures the frequency of saccades in which the eyes generate saccades based on the eye movements detected during the period to be determined, and measures the saccade period in which the eye movements stay based on the detected eye movements. Then, based on the measured saccade frequency and the period of stay, it is determined whether or not the awakening state of the person is in a vague state during the period to be determined.
  • Another aspect of the present disclosure provides an awakening system including the awakening state determination device of the above aspect and an output unit that executes a predetermined operation based on the determination result of the awakening state determination device.
  • the saccade state determination method includes a step in which the control unit sequentially acquires image data generated by capturing an image including the human eye, and a control unit determines the movement of the human eye based on the acquired image data. Based on the step to detect, the step to measure the frequency of saccades in which the eyes generate saccades, based on the eye movements detected by the control unit during the period to be determined, and the steps to measure the frequency of saccades detected by the control unit. Based on the step of measuring the dwell period in which the eye movement is stagnant and the measured frequency of occurrence of saccade and the dwell period, whether or not the awakening state of the person is in a vague state during the period to be determined. Includes a step to determine.
  • the figure for demonstrating the application example of the awakening state determination apparatus which concerns on this disclosure The figure which illustrates the structure of the awakening system which concerns on embodiment of this disclosure.
  • Flow chart exemplifying the operation flow of the awakening system A flowchart illustrating the detailed flow of the initial processing step of FIG. Diagram for explaining an example of a saccade detection method
  • Diagram for explaining an example of a method for measuring the degree of eye opening A flowchart illustrating the detailed flow of the state determination step of FIG.
  • FIG. 1 is a diagram for explaining an application example of the awakening state determination device 100 according to the present disclosure.
  • FIG. 1 illustrates an awakening system 1 equipped with an awakening state determination device 100.
  • the awakening system 1 is embodied as, for example, a vehicle control system that controls the vehicle 2.
  • the awakening state determination device 100 includes a camera 3 attached to the vehicle 2.
  • the camera 3 is an example of the “imaging unit” of the present disclosure.
  • the camera 3 captures the driver's face, especially the eyes.
  • the camera 3 is mounted in front of the driver, such as near the steering column cover, dashboard, and room mirror.
  • the position of the camera 3 is not limited to this, and may be any position as long as it can capture the driver's face.
  • the camera 3 may be a spectacle-type camera or a head-mounted camera mounted on the driver's head.
  • the camera 3 has a frame rate of, for example, about 20 to 60 fps, for example, 30 fps. In other words, the camera 3 has a time resolution of 20-60 Hz.
  • the camera 3 may be a high-speed camera having a frame rate of 100 fps or more, but the awakening state determination device, the awakening system, and the awakening state determination method according to the present disclosure are realized by using a normal, that is, a non-high-speed camera. it can.
  • the awakening state determination device 100 measures the driver's eye movement or eye opening degree based on the image captured by the camera 3. At the time of measurement, data indicating the positions of the driver's pupil center, upper eyelid, and lower eyelid obtained by analyzing the image captured by the camera 3 is used. Specifically, the arousal state determination device 100 measures the driver's saccade frequency, eyeball retention period, and eye opening degree.
  • the awakening state determination device 100 determines whether or not the driver is in a drowsy state or a drowsy state based on the measurement result.
  • the rambling state is a state in which the driver's arousal level is lower than that at the time of awakening. For example, in a state of rambling, the driver does not concentrate on driving and becomes vague.
  • the drowsiness state is a state in which the arousal level is further lowered from the drowsiness state.
  • a drowsiness state is a state in which the driver is drowsy.
  • the inventor found that when the driver falls into a state of rambling, the number of saccades becomes about 50% of the time of awakening, and the eyeball retention period also becomes about 70% of the time of awakening.
  • the inventor has found that when the driver falls into a drowsy state, the time rate at which the degree of eye opening, which indicates the degree of eye opening, decreases to about 50% of the time of awakening increases.
  • the awakening state determination device 100 can determine whether the driver is in an awake state, a drowsiness state, or a drowsiness state by measuring the saccade frequency, the eyeball retention period, and the degree of eye opening.
  • the awakening state determination device 100 determines that the driver is in a drowsy state or a drowsy state
  • the awakening state determination device 100 drives the output unit 20 to improve the awakening degree of the driver.
  • the output unit 20 is, for example, a speaker, and when the driver is in a drowsy state or a drowsy state, the output unit 20 outputs a voice to awaken the driver.
  • the camera 3 is an imaging device that images the face of the subject, especially the eyes, to form an captured image.
  • the camera 3 is, for example, a camera that forms an image captured by a solid-state image sensor such as CMOS (Complementary MOS) or CCD (Charge Coupled Device).
  • CMOS Complementary MOS
  • CCD Charge Coupled Device
  • the camera 3 has a frame rate of, for example, about 20-60 fps, for example 30 fps.
  • the camera 3 may be a high-speed camera having a frame rate of 100 fps or more.
  • the awakening state determination device 100 may include an infrared irradiator 4.
  • the infrared irradiator 4 is a light source device that irradiates an infrared ray toward the subject's face, particularly the eyes. Infrared rays reflected by the subject's face, cornea, etc. are incident on the camera 3. In this case, the camera 3 is configured to have sensitivity in the infrared region.
  • the configuration of the awakening state determination device 100 is not limited to the above, and a camera 3 having sensitivity in the same wavelength range as the wavelength of the light reflected by the subject's face may be provided.
  • the awakening state determination device 100 may be configured to include a visible light irradiator and a visible light camera, or an ultraviolet irradiator and an ultraviolet camera.
  • the awakening state determination device 100 does not include a light source device such as an infrared irradiator 4, a visible light irradiator, and an ultraviolet irradiator, and the camera 3 captures sunlight or fluorescent light reflected by the subject's face. It may be an image.
  • the storage unit 5 stores the information of the program or the like by electrical, magnetic, optical, mechanical or chemical action so that the computer or other device, the machine or the like can read the information of the recorded program or the like. It is a medium to do.
  • the storage unit 5 includes, for example, an auxiliary storage device such as a hard disk drive or a solid state drive.
  • the control unit 10 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), and the like, and controls each component according to information processing.
  • the control unit 10 is composed of, for example, an ECU (electronic control unit).
  • the control unit 10 interprets and executes, for example, a program expanded in the RAM by the CPU.
  • the control unit 10 includes, for example, an image processing unit 11, a measurement unit 12, a state determination unit 13, and a drive control unit 14.
  • the output unit 20 executes a predetermined operation based on the determination result of the awakening state determination device 100.
  • the output unit 20 includes, for example, a display unit 21.
  • the display unit 21 is a display device such as a liquid crystal display, an organic EL display, and a projector.
  • the display unit 21 may be, for example, a head-up display that projects an image onto the windshield of the vehicle 2 or a dedicated glass panel.
  • the control unit 10 of the awakening state determination device 100 that operates as the drive control unit 14 transmits a control signal indicating the determination result of the state of the target person to the display unit 21, and the display unit that receives the control signal. 21 displays the determination result.
  • the output unit 20 includes, for example, a speaker 22.
  • the speaker 22 is a voice output device that notifies the target person of various information by voice.
  • the control unit 10 of the awakening state determination device 100 that operates as the drive control unit 14 causes the speaker 22 to output a voice corresponding to the determination result of the state of the target person.
  • the warning sound output from the speaker 22 improves the arousal level of the subject, and eliminates the drowsiness state and the drowsiness state.
  • the output unit 20 may include a conversation system 23 that has a conversation with the target person, and the conversation system 23 may output a voice according to a determination result about the state of the target person. For example, when the control unit 10 determines that the subject is in a drowsy state or a drowsy state, the control unit 10 causes the conversation system 23 to output a voice to recommend a break.
  • the output unit 20 includes, for example, an air conditioning system 24.
  • the air conditioning system 24 is a system that adjusts the temperature, humidity, air flow, and the like of the air around the subject.
  • the control unit 10 of the awakening state determination device 100 that operates as the drive control unit 14 controls the air conditioning system 24 according to the determination result of the state of the target person to lower the temperature around the target person or the target person. Or blow the wind on. As a result, the arousal level of the subject is improved, and the drowsiness state and the drowsiness state are eliminated.
  • the output unit 20 includes, for example, a lighting system 25.
  • the lighting system 25 is, for example, a light source device that adjusts the illuminance and the brightness of the space around the subject.
  • the control unit 10 of the awakening state determination device 100 that operates as the drive control unit 14 controls the lighting system 25 according to the determination result of the state of the target person to increase the illuminance or blink the light source. As a result, the arousal level of the subject is improved, and the drowsiness state and the drowsiness state are eliminated.
  • the output unit 20 includes, for example, a vibration unit 26.
  • the vibrating unit 26 is attached to, for example, a seat on which the subject sits, and can transmit vibration to the subject.
  • the vibrating unit 26 may be attached to, for example, a steering handle and transmit the vibration to the target person's hand.
  • the control unit 10 of the awakening state determination device 100 that operates as the drive control unit 14 vibrates the vibrating unit 26 according to the determination result of the state of the target person. As a result, the arousal level of the subject is improved, and the drowsiness state and the drowsiness state are eliminated.
  • the output unit 20 may be, for example, a brake 27.
  • the control unit 10 of the awakening state determination device 100 that operates as the drive control unit 14 controls the brake 27 according to the determination result of the state of the target person, and stops the vehicle 2 driven by the target person.
  • a collision accident or the like caused by a decrease in arousal level such as a drowsy state or a drowsy state of the subject.
  • the output unit 20 may be, for example, a steering wheel 28.
  • the control unit 10 of the awakening state determination device 100 that operates as the drive control unit 14 controls the steering 28 according to the determination result of the state of the target person, and controls the traveling direction of the vehicle 2 driven by the target person. ..
  • a collision accident or the like caused by a decrease in arousal level such as a drowsy state or a drowsy state of the subject.
  • FIG. 3 is a flowchart illustrating the flow of operation of the awakening system 1. Each process of the flowchart of FIG. 3 is executed by the control unit 10 of the awakening state determination device 100 of the awakening system 1.
  • the control unit 10 executes the initial processing step S1 and the state determination step S2, turns on the drowsiness state flag when the target person is in a drowsiness state, and turns on the drowsiness state flag when the target person is in a drowsiness state. To. Details of steps S1 and S2 will be described later.
  • the control unit 10 that operates as the drive control unit 14 determines whether or not the drowsiness state flag is ON (S3).
  • the control unit 10 operating as the drive control unit 14 performs drive control for the drowsiness state with respect to the output unit 20 (S4).
  • the drive control for the drowsiness state is a drive control for increasing the arousal level of the subject and eliminating the drowsiness state.
  • control unit 10 controls the voice output of the conversation system 23, the vibration application by the vibration unit 26, the vehicle stop by the brake 27, the adjustment of the vehicle traveling direction by the steering 28, and the like as the drive control for the drowsiness state.
  • step S3 when it is determined in step S3 that the drowsiness state flag is OFF (No in S3), the control unit 10 operating as the drive control unit 14 determines whether or not the drowsiness state flag is ON (S5). .. When it is determined that the rambling state flag is ON (Yes in S5), the control unit 10 operating as the drive control unit 14 performs drive control for the rambling state to the output unit 20 (S6).
  • the drive control for the rambling state is a drive control for increasing the arousal level of the subject and eliminating the rambling state.
  • control unit 10 controls the output of a warning sound from the speaker 22, the adjustment of the temperature and the air flow by the air conditioning system 24, the increase in the illuminance by the lighting system 25, and the like as the drive control for the rambling state.
  • step S4 When step S4 is completed, when step S6 is completed, and when it is determined in step S5 that the vague state flag is OFF (No in S5), the process returns to step S2.
  • steps S2 to S6 may be performed immediately when each step is completed, or may be performed at predetermined time intervals.
  • the initial processing step S1 is executed only once.
  • the operation of the awakening system 1 is not limited to this. For example, every time a predetermined period elapses, the process may return to step S1 and the initial process may be performed again.
  • FIG. 4 is a flowchart illustrating the detailed flow of the initial processing step S1 of FIG.
  • the control unit 10 acquires an image of the target person's eyes captured by the camera 3 (S101).
  • Step S101 may include irradiating the subject's eyes with infrared rays by the infrared irradiator 4 before acquiring the image.
  • control unit 10 that operates as the image processing unit 11 performs image processing on the acquired image of the eye, and detects the position of the center of the pupil and the positions of the upper eyelid and the lower eyelid (S102).
  • the detected data indicating the positions of the pupil center, the upper eyelid, and the lower eyelid are stored in the storage unit 5.
  • the position of the center of the pupil is used to detect saccades and saccades observed as eye movements.
  • the stagnation is sometimes referred to as fixation.
  • the control unit 10 does not detect eye movements such as saccades when the relative position of the eyeball with respect to the face does not change even when the entire face of the subject moves significantly.
  • the position relative to the reference point on the subject's face or the like is detected as the position of the center of the pupil. In this sense, it can be said that the detection of the position of the center of the pupil performed by the control unit 10 operating as the image processing unit 11 is the detection of the line of sight of the subject.
  • the control unit 10 detects the position of the center of the pupil by using the corneal reflex method.
  • the corneal reflex method infrared rays emitted from the infrared irradiator 4 which is a point light source are reflected by the cornea of the subject and incident on the camera 3.
  • the infrared rays reflected from the cornea are called Pulkinier images.
  • the Pulkinier image serves as a reference point for detecting the position of the center of the pupil. That is, when eye movement occurs, the position of the center of the pupil relative to the Purkinje image changes. Utilizing this, the control unit 10 can determine the position of the center of the pupil of the subject based on the position of the Pulkinier image.
  • control unit 10 may detect the relative position of the center of the pupil of the subject with at least one of the inner and outer corners of the eye (outer canthus) of the subject as a reference point. In this case, it is not necessary to use the infrared irradiator 4.
  • Steps S101 and S102 are repeated from the start of the initial processing step S1 until a predetermined period T0 elapses (S103). As a result, data indicating the positions of the subject's pupil center, upper eyelid, and lower eyelid are accumulated in the storage unit 5.
  • the control unit 10 operating as the measurement unit 12 has an initial value n0 (S104) of the saccade frequency n of the subject and an eyeball retention period t.
  • the initial value ta0 (S105) of the average value ta and the initial value La0 (S106) of the average value La of the eye opening degree L are calculated.
  • data indicating the positions of the center of the pupil, the upper eyelid, and the lower eyelid stored in the storage unit 5 is used.
  • Steps S104, S105, and S106 are in no particular order.
  • the initial values n0, ta0, and La0 calculated in steps S104, S105, and S106 are stored in the storage unit 5.
  • the initial value n0 of the saccade frequency n is calculated.
  • the saccade frequency n is the number of saccades per unit time.
  • FIG. 5 is a diagram for explaining an example of a saccade detection method.
  • FIG. 5 shows points pi to pi + 2 representing the position of the center of the pupil detected in step S102.
  • Figure 5 is moved, the position of the pupil center, from the measured p i at time t i, the p i + 1 at time t i + 1 is later ⁇ t seconds at time t i + 2 is after addition thereof ⁇ t seconds p i + 2 It shows that it was done.
  • the distance between the position of the pupil center at a certain time (current frame) and the position of the pupil center in the frame immediately before that (that is, the frame ⁇ t seconds before the current frame) is a predetermined threshold value d1. If it exceeds th , it is determined that a saccade has occurred between the previous frame and the current frame. In the example shown in FIG. 5, since the distance between the position p i + 1 of the pupil center at time t i + 1, and the position p i + 2 of the pupil center at time t i + 2 is greater than the threshold d1 th, time t i + 1 and the time t i + 2 It is judged that a saccade has occurred between.
  • the initial value ta0 of the average value of the eyeball retention period is calculated.
  • the control unit 10 the distance between the position of the pupil center at a certain time (current frame) and the position of the pupil center in the frame immediately before that (that is, the frame ⁇ t seconds before the current frame) is a predetermined threshold value d2.
  • a predetermined threshold value t th it is determined that "staying" has occurred.
  • the control unit 10 counts this period t as the eyeball staying period.
  • the control unit 10 record a N / F as eye fixation duration.
  • the initial value ta0 of the average value of the eyeball retention period is a value obtained by dividing the total of the eyeball retention periods generated from the start of the initial processing step S1 to the elapse of the predetermined period T0 by the number of retentions.
  • FIG. 6 is a diagram for explaining an example of a method for measuring the eyeball retention period.
  • FIG. 6 shows the positions p j to p j + 11 of the center of the pupil detected at times t j to t j + 11 in step S102, respectively.
  • the frame rate F of the camera 3 is 30 fps and the predetermined time t th is set to 250 msec.
  • the distance between p j and p j + 1 and the distance between p j + 10 and p j + 11 exceed a predetermined threshold value d2 th , but between other frames. It is assumed that the distances between the positions of the center of the pupil are all equal to or less than the predetermined threshold value d2 th .
  • the control unit 10 determines that "staying" has occurred, and calculates the eyeball staying period as 300 msec.
  • the predetermined threshold value d1 th which is a criterion for determining the presence or absence of saccade
  • the predetermined threshold value d2 th which is a criterion for determining the presence or absence of saccade
  • the d1 th and the d2 th may have different values.
  • step S106 the initial value La0 of the average value of the eye opening degree L is calculated.
  • the average value of the eye opening degree L is a time average value of the eye opening degree L.
  • FIG. 7 is a diagram for explaining an example of a method for measuring the degree of eye opening L.
  • the degree of eye opening L is an index of the degree of opening of the eyes.
  • the degree of eye opening L is represented by y / x.
  • x is the lateral width of the eye, for example, the distance from the inner canthus 91 to the outer canthus 92.
  • y is the eye opening height, and is, for example, the maximum value of the distance from the lower end 93 of the upper eyelid to the upper end 94 of the lower eyelid in the direction perpendicular to the line segment connecting the inner and outer eye angles 92.
  • the eye opening height y is reduced by the action of closing the eyes or the blinking.
  • FIG. 8 is a flowchart illustrating the detailed flow of the state determination step S2 of FIG.
  • the control unit 10 acquires an image of the target person's eyes captured by the camera 3 (S201).
  • the control unit 10 that operates as the image processing unit 11 performs image processing on the acquired image of the eye, and detects the position of the center of the pupil and the positions of the upper eyelid and the lower eyelid (S202).
  • the detected data indicating the positions of the pupil center, the upper eyelid, and the lower eyelid are stored in the storage unit 5.
  • Steps S201 and S202 are repeated from the start of the state determination step S2 until a predetermined period T1 elapses (S203). As a result, data indicating the positions of the subject's pupil center, upper eyelid, and lower eyelid are accumulated in the storage unit 5.
  • the control unit 10 operating as the measurement unit 12 calculates the current value La1 of the average value La of the eye opening degree L (S204). At the time of calculation, the data indicating the positions of the upper eyelid and the lower eyelid stored in the storage unit 5 from the start of the state determination step S2 to the elapse of the predetermined period T1 are used.
  • the current value La1 of the average value of the eye opening degree obtained in step S204 is a predetermined value Lax and the initial value La0 obtained in step S106 of FIG. It is determined whether or not the product of Lax ⁇ La0 is smaller than (S205).
  • Lax ⁇ La0 is an example of the “third threshold” of the present disclosure.
  • Lax is, for example, 0.2 to 0.8, for example 0.5 or 0.7.
  • step S206 it is determined that the subject is in a drowsy and drowsy state. After finishing step S206, the control unit 10 finishes the state determination step S2 and proceeds to step S3 of FIG.
  • the control unit 10 operating as the measurement unit 12 calculates the current value n1 of the saccade frequency n (S207). At the time of calculation, the data stored in the storage unit 5 indicating the position of the center of the pupil from the start of the state determination step S2 to the elapse of the predetermined period T1 is used.
  • the current value n1 of the saccade frequency obtained in step S207 is the product of a predetermined value nx and the initial value n0 obtained in step S104 of FIG. It is determined whether or not it is smaller than nx ⁇ n0 (S208).
  • nx ⁇ n0 is an example of the “first threshold” of the present disclosure.
  • nx is, for example, 0.2 to 0.8, for example 0.5.
  • control unit 10 finishes the state determination step S2 and proceeds to step S3 in FIG. In this case, it is determined that the subject is not in a drowsy state or a drowsy state.
  • the control unit 10 When it is determined that n1 is smaller than nx ⁇ n0 (Yes in S208), the control unit 10 operating as the measurement unit 12 calculates the current value ta1 of the average value ta of the eyeball retention period t (S209). At the time of calculation, the data stored in the storage unit 5 indicating the position of the center of the pupil from the start of the state determination step S2 to the elapse of the predetermined period T1 is used.
  • the current value ta1 of the average value of the eyeball retention period obtained in step S209 is a predetermined value tax and the initial value ta0 obtained in step S105 of FIG. It is determined whether or not the product is smaller than the product tax ⁇ ta0 (S210).
  • tax ⁇ ta0 is an example of the “second threshold” of the present disclosure.
  • the tax is, for example, 0.2 to 0.8, for example 0.7.
  • the control unit 10 finishes the state determination step S2 and proceeds to step S3 in FIG. In this case, it is determined that the subject is not in a drowsy state or a drowsy state.
  • step S211 it is determined that the subject is in a vague state.
  • the control unit 10 finishes the state determination step S2 and proceeds to the step S3 of FIG.
  • the awakening state determination device 100 determines the awake state of the subject in the awake state.
  • the awakening state determination device 100 includes a camera 3 that captures an image including the eyes of the subject and generates image data, and a control unit 10.
  • the control unit 10 sequentially acquires image data from the camera 3 and detects the movement of the eyes of the target person based on the acquired image data.
  • the control unit 10 measures the frequency of saccades in which the eyes cause saccades based on the movements of the eyes detected during the period T1 to be determined.
  • the control unit 10 measures the eyeball retention period during which the eye movement is stopped, based on the detected eye movement. Based on the measured saccade frequency and eyeball retention period, the control unit 10 determines whether or not the awakening state of the subject is in a vague state during the determination target period T1.
  • this awakening state determination device 100 it is possible to determine whether or not the subject is in a vague state based on the saccade frequency and the eyeball retention period.
  • the saccade frequency and the eyeball retention period can be measured mainly by a simple configuration including the camera 3 and the control unit 10 and a simple process.
  • the control unit 10 detects the eye movement as a saccade. For example, the control unit 10 measures the saccade frequency based on the saccade detected during the determination target period T1. In this way, the saccade can be measured by a simple configuration and a simple process.
  • the camera 3 does not have to be a high-speed camera having a frame rate of 100 fps or more, for example.
  • the control unit 10 changes the awakening state of the subject. It may be determined that the state is in a saccade state.
  • control unit 10 detects the movement of the position of the center of the pupil as a distance.
  • the present disclosure is not limited to this, as long as the control unit 10 can detect the movement of the position of the center of the pupil.
  • the control unit 10 may detect the movement of the position of the center of the pupil as the movement of the viewpoint (gaze point) corresponding to the angle conversion of the line of sight.
  • FIG. 9 is a diagram for explaining an example of the saccade detection method in the modified example 1.
  • a ⁇ x axis and a ⁇ y axis are provided.
  • ⁇ x represents the angle of the line of sight in the horizontal direction
  • ⁇ y orthogonal to ⁇ x represents the angle of the line of sight in the vertical direction.
  • (0,0) means that the angle of the line of sight is about 34.5 degrees to the left and downward when the coordinates (0.5,0.5) are the front (0 degrees, 0 degrees).
  • (1,1) corresponds to a state in which the line-of-sight angle is about 34.5 degrees to the right and about 34.5 degrees to the top.
  • Step S205 shown in FIG. 8 is for determining whether or not the subject is in a drowsy state, and is a step for the control unit 10 to determine whether or not La1 is smaller than Lax ⁇ La0.
  • the method for determining whether or not the subject is drowsy is not limited to the above.
  • the control unit 10 operating as the state determination unit 13 may determine that the subject is in a drowsy state and proceed to step S206.
  • the threshold value Uth is, for example, 0.2 to 0.8, for example, 0.6.
  • the awakening system 1 may be used not only for in-vehicle use illustrated above, but also for awakening factory workers, for example, workers engaged in simple work.
  • the camera 3 captures the eyes of an inspector who inspects the appearance of a product flowing through a factory line.
  • the control unit 10 determines whether or not the inspector is in a state of drowsiness or drowsiness, and if so, outputs voice from the speaker 22 or the conversation system 23 to the inspector. Improves alertness.
  • the control unit 10 may stop the operation of the factory line.
  • the control unit (10) measures the degree of eye opening by the human eye based on the acquired image data.
  • the time during which the degree of eye opening is smaller than the third threshold value is equal to or longer than a predetermined ratio in the period to be determined, it is determined that the person's awake state is drowsy.
  • the third threshold value is defined by the degree of eye opening of the person in the past fixed period.

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Pathology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Hospice & Palliative Care (AREA)
  • Psychiatry (AREA)
  • Psychology (AREA)
  • Social Psychology (AREA)
  • Physics & Mathematics (AREA)
  • Developmental Disabilities (AREA)
  • Biophysics (AREA)
  • Child & Adolescent Psychology (AREA)
  • Biomedical Technology (AREA)
  • Educational Technology (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Traffic Control Systems (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

La présente invention concerne un dispositif de détermination d'état de vigilance (100) pourvu : d'une unité d'imagerie (3) pour générer des données d'image en capturant les images comprenant un œil d'une personne ; et d'une unité de commande (10) pour acquérir séquentiellement les données d'image de l'unité d'imagerie et détecter le mouvement de l'œil de la personne sur la base des données d'image acquises. L'unité de commande (10) mesure la fréquence des saccades de l'œil sur la base du mouvement de l'œil détecté dans une période de détermination, mesure une période d'arrêt dans laquelle le mouvement de l'œil est stoppé sur la base du mouvement détecté de l'œil, et détermine si l'état de vigilance de la personne est ou non un état sans vigilance dans la période de détermination sur la base de la fréquence mesurée des saccades et de la période d'arrêt.
PCT/JP2019/010829 2019-03-15 2019-03-15 Dispositif de détermination d'état de vigilance, système d'éveil, et procédé de détermination d'état de vigilance WO2020188629A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2021506797A JPWO2020188629A1 (fr) 2019-03-15 2019-03-15
PCT/JP2019/010829 WO2020188629A1 (fr) 2019-03-15 2019-03-15 Dispositif de détermination d'état de vigilance, système d'éveil, et procédé de détermination d'état de vigilance

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2019/010829 WO2020188629A1 (fr) 2019-03-15 2019-03-15 Dispositif de détermination d'état de vigilance, système d'éveil, et procédé de détermination d'état de vigilance

Publications (1)

Publication Number Publication Date
WO2020188629A1 true WO2020188629A1 (fr) 2020-09-24

Family

ID=72519190

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2019/010829 WO2020188629A1 (fr) 2019-03-15 2019-03-15 Dispositif de détermination d'état de vigilance, système d'éveil, et procédé de détermination d'état de vigilance

Country Status (2)

Country Link
JP (1) JPWO2020188629A1 (fr)
WO (1) WO2020188629A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022085276A1 (fr) * 2020-10-20 2022-04-28 日本電気株式会社 Système de traitement d'informations, système de mesure de l'état de l'œil, procédé de traitement d'informations, et support non transitoire lisible par ordinateur

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7107725B2 (ja) * 2018-04-10 2022-07-27 浜松ホトニクス株式会社 眼球運動特徴量算出システム、眼球運動特徴量算出方法及び眼球運動特徴量算出プログラム

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH11276461A (ja) * 1998-03-27 1999-10-12 Suzuki Motor Corp 注意力測定装置及びこれを用いた情報提示装置
JP2008212298A (ja) * 2007-03-01 2008-09-18 Toyota Central R&D Labs Inc 眠気判定装置及びプログラム
JP2015116376A (ja) * 2013-12-19 2015-06-25 株式会社デンソー 状態推定装置、および状態推定プログラム
JP2017023519A (ja) * 2015-07-24 2017-02-02 株式会社デンソー 状態推定装置および状態推定プログラム
JP2017199212A (ja) * 2016-04-27 2017-11-02 株式会社デンソー 運転支援装置

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180055354A1 (en) * 2016-08-31 2018-03-01 Alcohol Countermeasure Systems (International) Inc. Novel non-intrusive approach to assess drowsiness based on eye movements and blinking
US20190077409A1 (en) * 2017-07-31 2019-03-14 Alcohol Countermeasure Systems (International) Inc. Non-intrusive assessment of fatigue in drivers using eye tracking

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH11276461A (ja) * 1998-03-27 1999-10-12 Suzuki Motor Corp 注意力測定装置及びこれを用いた情報提示装置
JP2008212298A (ja) * 2007-03-01 2008-09-18 Toyota Central R&D Labs Inc 眠気判定装置及びプログラム
JP2015116376A (ja) * 2013-12-19 2015-06-25 株式会社デンソー 状態推定装置、および状態推定プログラム
JP2017023519A (ja) * 2015-07-24 2017-02-02 株式会社デンソー 状態推定装置および状態推定プログラム
JP2017199212A (ja) * 2016-04-27 2017-11-02 株式会社デンソー 運転支援装置

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022085276A1 (fr) * 2020-10-20 2022-04-28 日本電気株式会社 Système de traitement d'informations, système de mesure de l'état de l'œil, procédé de traitement d'informations, et support non transitoire lisible par ordinateur

Also Published As

Publication number Publication date
JPWO2020188629A1 (fr) 2020-09-24

Similar Documents

Publication Publication Date Title
US10849496B2 (en) Motion sickness estimation device, motion sickness prevention device, and motion sickness estimation method
JP5255063B2 (ja) 眠気予兆検出装置
JP7099037B2 (ja) データ処理装置、モニタリングシステム、覚醒システム、データ処理方法、及びデータ処理プログラム
JP4772935B2 (ja) 注意状態判定装置、方法およびプログラム
JP5974915B2 (ja) 覚醒度検出装置、および覚醒度検出方法
JP6470334B2 (ja) 車両の表示システム及び車両の表示システムの制御方法
US9526448B2 (en) State estimation device and state estimation program
JP4899059B2 (ja) 眠気検知装置
WO2014073645A1 (fr) Dispositif de mesure d'informations biologiques et dispositif d'entrée l'utilisant
WO2015027598A1 (fr) Procédé de rappel et dispositif de rappel
JP2018156172A (ja) 車両の表示システム及び車両の表示システムの制御方法
JP5600256B2 (ja) 情報表示装置
WO2020188629A1 (fr) Dispositif de détermination d'état de vigilance, système d'éveil, et procédé de détermination d'état de vigilance
JP4977856B2 (ja) 覚醒状態検知方法および覚醒状態検知装置
JP2015207163A (ja) 状態推定装置、および状態推定プログラム
JP7018634B2 (ja) 眼球運動測定装置及び眼球運動解析システム
WO2019155914A1 (fr) Dispositif de traitement de données, système de surveillance, système de vigilance, procédé de traitement de données, programme de traitement de données et support de stockage
WO2019176492A1 (fr) Système de calcul, dispositif de traitement d'informations, système d'aide à la conduite, procédé de calcul d'indice, programme informatique et support de stockage
WO2019155913A1 (fr) Dispositif de traitement de données, système de surveillance, système d'alerte, méthode de traitement de données, programme de traitement de données et support de stockage
JP2009125518A (ja) ドライバの瞬き検知方法及びドライバの覚醒度判断方法並びに装置
JP7207518B2 (ja) 認知症検知装置及び認知症検知方法
JP7099036B2 (ja) データ処理装置、モニタリングシステム、覚醒システム、データ処理方法、及びデータ処理プログラム
JP2011206072A (ja) 有効視野測定システムおよび有効視野測定方法
JP7255691B2 (ja) 表示システム、表示方法、及びプログラム
JP3158494U (ja) 携帯通信端末

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19919814

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021506797

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19919814

Country of ref document: EP

Kind code of ref document: A1