JP2010273800A - Blinking measuring apparatus and blinking measuring method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a blinking measuring apparatus and a blinking measuring method having excellent reliability. <P>SOLUTION: The blinking measuring apparatus 1 determines the biological condition of a subject to be measured on the basis of blinking movement of the subject to be measured. The blinking measuring apparatus comprises a photographing section 5 for photographing the eye of the subject to be measured and an eyelid opening/closing measuring section 7 which calculates a first characteristic amount regarding continuous voluntary blinking movement on the basis of an eye image photographed by the photographing section 5 and determines the biological condition of the subject to be measured on the basis of the calculated first characteristic amount. Because the blinking measuring apparatus 1 evaluates the continuous voluntary blinking movement which often reflects the biological condition of the subject to be measured by calculating the characteristic amount of the continuous voluntary blinking movement, the apparatus determines the biological condition of the subject more accurately compared to a determination on the basis of a characteristic amount of a single voluntary blinking movement. As a result, reliability of the determination result of the biological condition is improved. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a blink measurement apparatus and a blink measurement method for determining a biological state of a measurement target person based on a blink action of the measurement target person.

  In recent years, attention has been drawn to the relationship between eye fatigue caused by VDT work or the like, or fatigue caused by work such as driving a car, and eye movement. For example, Patent Document 1 describes that the speed and acceleration at the start of blinking are used as parameters for estimating the state of eye fatigue. Patent Document 2 describes that the number and frequency of blinks are used as parameters for estimating the state of eye fatigue. Further, Non-Patent Document 1 discloses a time period from the start of blinking to the reopening of eyesight through the closed eye state as a parameter for detecting / estimating the driver's state of consciousness drowsiness (ie, one blink operation). It is described that the time to complete) is used.

JP-A-8-289327 Japanese Patent No. 3348956 Kazumasa Adachi, 3 others, “Blink measurement by moving image processing for detection of driver consciousness decline”, IEEJ Transactions C (Electronics, Information and Systems Division), IEEJ, 2004, Vol. 124, Vol. 3, p. 776-783

  Incidentally, the devices described in Patent Documents 1 and 2 and Non-Patent Document 1 calculate each parameter based on a face image acquired by a CCD camera. However, current general CCD cameras can only perform imaging at a low speed of 30 frames / second. In this case, the required time per frame is about 33 milliseconds. On the other hand, the time required for a human eye blink (from the start of the eye blink until the eye is reopened after the eye is closed) is about 100 to 300 milliseconds, and the CCD camera can only obtain an image of 3 to 10 frames. It is difficult to accurately measure the movement. Therefore, the devices described in Patent Documents 1 and 2 or Non-Patent Document 1 have a problem that reliability regarding determination of a biological state such as a fatigued state or an arousal state is low. Another problem is that most of the previous cases were for natural blinks. However, natural blinks are infrequent and take time to measure or the effects of statistical variation. There were problems such as being easy to receive.

  Therefore, the present invention has been made in view of the above-described problems, and an object thereof is to provide a highly reliable blink measurement apparatus and blink measurement method.

  The inventors have heretofore known a method for efficiently determining a biological state of a measurement target person by measuring a blink operation using an imaging device having an extremely high frame rate of 1 kHz or higher. I found out the fact that was not there. Specifically, as the blink feature amount related to the blink operation, the maximum speed when the eye is closed, the time from the open state to the closed state, the maintenance time of the closed state, the maximum speed when the eye is opened, the closed state to the open state Time to reach, eye position before and after blinking, position of eyelid movement from open to closed state, and movement of upper eyelid edge from closed to open state As a result of calculating the amount and comparing the correlation with a biological state such as a fatigue state, it was found that the determination method needs to be changed depending on the type of blink operation. In the present invention, among such types of blink operation, measurement is performed for two tasks, a single voluntary blink operation that is a single blink operation and a continuous voluntary blink operation that is a continuous blink operation. Thus, a blink measurement device and a blink measurement method that can obtain a highly reliable determination result are proposed.

  The present invention relates to a blink measurement device for determining a living state of a measurement subject based on the blink operation of the measurement subject, the imaging means for imaging the eye of the measurement subject, and an image taken by the imaging means. Based on the first feature quantity calculated by the first feature quantity calculation means and the first feature quantity calculation means for calculating the first feature quantity related to the continuous voluntary blink operation based on the eye image. And a biological condition determining means for determining the biological condition of the measurement subject.

  According to this blink measurement device, when evaluating the feature quantity of a single voluntary blink action by calculating and evaluating the feature quantity of a continuous voluntary blink action in which the biological state of the measurement subject is easily reflected Compared to the above, since it is possible to determine the biological state of the measurement subject with higher accuracy, it is possible to improve the reliability related to the determination result of the biological state.

  In addition, the apparatus further includes a second feature amount calculation unit that calculates a second feature amount related to the single voluntary blink operation based on the eye image captured by the imaging unit, and the biological state determination unit includes the first feature. It is preferable to determine the biological state of the measurement target person based on the first feature amount calculated by the amount calculation unit and the second feature amount calculated by the second feature amount calculation unit.

  According to such a configuration, the first and second feature quantities are calculated and evaluated for the blink action having different characteristics such as the single voluntary blink action and the continuous voluntary blink action. Since the biological state can be determined with higher accuracy, the reliability of the biological state determination result can be improved.

  Moreover, it is preferable to further include an instruction means for issuing an instruction for causing the measurement subject to perform a single voluntary blink operation or a continuous voluntary blink operation. In this case, it is not necessary for the measurement operator who operates the blink measurement device to issue an instruction to perform a single voluntary blink operation or a continuous voluntary blink operation to the measurement target, so that the measurement work time can be shortened and The labor of the measurement operator can be reduced. In addition, it is possible to automatically issue an instruction at an appropriate timing when the blink measurement device is ready for measurement, which is advantageous for automation of measurement work.

  Further, the first feature amount calculating means calculates the upper eyelid position at the time of eye opening in the continuous voluntary blink operation as the first feature amount, and the second feature amount calculating means is the single voluntary instantaneous moment as the second feature amount. It is preferable to calculate the upper eyelid position at the time of eye opening in the eye movement. When the measurement subject is in a fatigued state, the upper eyelid position at the time of eye opening tends to vary in multiple single voluntary blink operations and continuous voluntary blink operations. Thus, the biological state of the measurement subject can be determined with higher accuracy.

  Further, the first feature quantity calculating means calculates the eye closing speed in the continuous voluntary blink operation as the first feature quantity, and the second feature quantity calculating means in the single voluntary blink action as the second feature quantity. It is preferable to calculate the eye closing speed. When the measurement subject is in a fatigued state, the closed eye speed is likely to decrease in single voluntary blink operation and continuous voluntary blink operation. The biological state can be determined.

  Further, the first feature quantity calculating means calculates the eye opening speed in the continuous voluntary blink operation as the first feature quantity, and the second feature quantity calculating means is the single feature voluntary blink action as the second feature quantity. It is preferable to calculate the eye opening speed. When the measurement subject is in a fatigued state, the eye opening speed is likely to decrease in single voluntary blink operation and continuous voluntary blink operation, so by calculating the eye opening velocity as a feature amount, The biological state can be determined.

  The present invention relates to a blink measurement method for determining a biological state of a measurement target person based on a blink action of the measurement target person, the imaging step for imaging the eye of the measurement target person, and an imaging unit. Based on the first feature value calculated in the first feature value calculation step and the first feature value calculation step for calculating the first feature value related to the continuous voluntary blink operation based on the eye image. And a biological state determination step of determining the biological state of the measurement subject.

  According to this blink measurement method, when evaluating the feature quantity of a single voluntary blink action by calculating and evaluating the feature quantity of continuous voluntary blink actions that easily reflect the biological state of the measurement subject Compared to the above, since it is possible to determine the biological state of the measurement subject with higher accuracy, it is possible to improve the reliability related to the determination result of the biological state. In addition, the combination of infrared illumination and wavelength selection mirrors enables the real-time monitoring of the state in which a person is looking at the outside world using this device. It is possible to cope with fatigue level measurement.

  ADVANTAGE OF THE INVENTION According to this invention, a reliable blink measurement apparatus and blink measurement method can be provided.

It is the schematic which shows one Embodiment of the blink measurement apparatus by this invention. It is a figure which shows the internal structure of the imaging part used in this embodiment. It is a figure which shows the electrical connection relationship between a photon detection part, an amplifier, an A / D converter, and a soot extraction process part. (A) (b) It is a figure which shows the detection method of the edge part of the eyelid based on image data in a eyelid extraction process part. (A) is a figure which shows the time change of the position of the edge part of an upper eyelid, (b) is a figure which shows the time change of the moving speed of the edge part of an upper eyelid. It is a figure for demonstrating the other example of the blink feature-value. (A) is a figure which shows the time change of the position of the edge part of the upper eyelid in single voluntary blink operation | movement, (b) is a figure which shows the time change of the moving speed of the edge part of the upper eyelid in single voluntary blink operation | movement. It is. (A) is a figure which shows the time change of the position of the edge part of the upper eyelid in continuous voluntary blink operation | movement, (b) is a figure which shows the time change of the moving speed of the edge part of the upper eyelid in continuous voluntary blink operation | movement. It is. (A) is a graph which shows the relationship between the VDT work time in a single voluntary blink operation, and a closed eye / opening speed, (b) is the relationship between the VDT work time and a closed eye / opening speed in continuous voluntary blink operation. It is a graph which shows. It is a flowchart regarding operation | movement of a blink measurement apparatus. It is the schematic which shows other embodiment of the blink measurement apparatus by this invention.

  DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. In addition, in each figure, the same code | symbol is attached | subjected to the same or an equivalent part, and the overlapping description is abbreviate | omitted.

  FIG. 1 is a schematic diagram showing a first embodiment of a blink measuring device according to the present invention. The blink measurement device 1 according to the present embodiment measures a single voluntary blink operation and a continuous voluntary blink operation performed by the measurement target person, thereby determining a biological state such as a fatigue state or an arousal state of the measurement target person. To do. Here, the single voluntary blink operation is a single blink operation performed every certain time among voluntary blink operations performed by the measurement subject (for example, about one blink operation per second). The continuous voluntary blink operation is a blink operation performed continuously for a certain period of time (for example, a blink operation performed continuously at a frequency of about four times per second).

  Referring to FIG. 1, the blink measurement device 1 includes an illumination 2, a dichroic mirror 3, a condenser lens 4, and an imaging unit (imaging unit) 5. Further, the blink measuring device 1 includes a wrinkle extraction processing unit 6, a wrinkle opening / closing measurement unit (first feature amount calculation means, second feature amount calculation means, biological state determination means) 7, and an instruction unit (instruction means). 10 and a speaker 11. The condensing lens 4, the imaging unit 5, the eyelid extraction processing unit 6, and the eyelid opening / closing measuring unit 7 are housed inside the camera 8.

  The illumination 2 is illumination means for illuminating the measurement subject's eye 100 and its surroundings, and is preferably composed of, for example, a plurality of LEDs that generate infrared light. When the illumination 2 irradiates the eye 100 and its periphery with the infrared light L1, the infrared light L1 is reflected at the eye 100 and its periphery to generate a light image L2. The illumination 2 is not limited to the infrared LED, but other light sources can be used. At the time of high-speed imaging, it is dazzling for the subject to emit a sufficient amount of visible light to illuminate the state of the eye 100 and the like. It is preferable to use an infrared light source.

  The dichroic mirror 3 is disposed so as to transmit the light image L2 from the eye 100 and its periphery and to make the light image L2 enter the light detection unit 51 of the imaging unit 5. In addition, the dichroic mirror 3 optically couples the visual target 9 and the eye 100 so that the measurement target can visually recognize the visual target 9 installed beside the optical axis connecting the eye 100 and the imaging unit 5. is doing. The visual target 9 may be configured to generate a bright spot pattern by combining, for example, an LED that generates visible light and a pinhole mask. The dichroic mirror 3 reflects the visible light L3 emitted from the visual target 9 toward the eye 100. Accordingly, the eye 100 and its periphery can be imaged while presenting the visual target 9 to the measurement subject and maintaining the corneal position of the eye 100 constant. It should be noted that a lens 9a for focus adjustment is preferably provided between the visual target 9 and the dichroic mirror 3.

  The condensing lens 4 is a lens for condensing the light image L2 to form an image on the light detection unit 51 of the imaging unit 5. The condenser lens 4 is disposed between the dichroic mirror 3 and the imaging unit 5. The condensing lens 4 of the present embodiment is configured such that the entire orbit of the eye 100 falls within the angle of view of the imaging unit 5 so that the imaging unit 5 described later can acquire an image including the entire eye 100.

  The imaging unit 5 is means for capturing the light image L2 from the eye 100. The imaging unit 5 includes a light detection unit 51 including a plurality of pixels arranged in a two-dimensional shape, and converts the light image L2 incident on the light detection unit 51 into an electrical signal in each pixel. Image data indicating the amount of incident light for each pixel relating to the image L2 is generated. The imaging unit 5 outputs the generated image data to output means such as a display device or a video output terminal, and provides the image data to the eyelid extraction processing unit 6.

  Based on the image data provided from the imaging unit 5, the eyelid extraction processing unit 6 detects the eyelid edge 101 in the image data and generates eyelid position information. The eyelid opening / closing measuring unit 7 also uses the eyebrow feature, which is a feature amount related to the eyeblink operation, based on the time change of the eyelid position information provided from the eyelid extraction processing unit 6, and the amount of change in the eyelid position and the moving speed thereof. Calculated as a quantity (first feature quantity, second feature quantity). The cocoon extraction processing unit 6 and the cocoon opening / closing measurement unit 7 are realized as software in a computer having a central processing unit and a memory, in addition to an electric circuit, for example.

  The instruction unit 10 is electrically connected to the camera 8, and outputs various instructions to the measurement subject by sound output from the speaker 11. The instruction unit 10 issues an instruction for causing the measurement subject to perform a continuous voluntary blink operation (an instruction for performing blinks as frequently as possible) at a predetermined measurement timing of the continuous voluntary blink operation. Moreover, the instruction | indication part 10 issues the instruction | indication (instruction which makes a blink be performed at a fixed interval) for a measurement subject to perform a single voluntary blink operation at the measurement timing of a predetermined single voluntary blink operation. .

  FIG. 2 is a diagram illustrating an internal configuration of the imaging unit 5 used in the present embodiment. The imaging unit 5 of the present embodiment is an imaging device having an extremely high frame rate of 1 kHz or more, and examples of such an imaging device include an intelligent vision system (IVS) camera manufactured by Hamamatsu Photonics. The imaging unit 5 includes a light detection unit 51, an amplification unit 53, an A / D conversion unit 55, and a switch unit 57. In the present embodiment, the light detection unit 51 is a so-called MOS type imaging device, and includes a plurality of pixels 51a arranged in a two-dimensional shape (m rows × n columns). Each of the plurality of pixels 51a generates a charge Q corresponding to the amount of incident light.

  The amplifying unit 53 includes m amplifiers 53 a corresponding to the number of rows of the light detecting unit 51. The m amplifiers 53a are electrically connected to the corresponding rows of the pixels 51a of the light detection unit 51, respectively, and sequentially receive the charges Q from the n columns of pixels 51a. The amplifier 53a amplifies the charge Q and converts the charge Q into an image signal S1 that is a voltage signal. Each of the m amplifiers 53a is electrically connected to a control unit (not shown), and changes the amplification factor when converting the charge Q into the image signal S1 according to the amplification factor control signal S3 from the control unit. Can be made.

  The A / D converter 55 has m A / D converters 55 a corresponding to the number of rows of the light detector 51. The m A / D converters 55a are electrically connected to the corresponding m amplifiers 53a, respectively, and receive an image signal S1 that is a voltage signal (analog signal) from the amplifier 53a, and thus an image that is a digital signal. Convert to data S2. In the present embodiment, the image data S2 converted into a digital signal is used as the image data from the imaging unit 5, but the image signal S1 that is an analog signal may be used as the image data from the imaging unit 5.

  The switch unit 57 includes m switches 57 a corresponding to the number of rows of the light detection unit 51. The m switches 57a are provided between the corresponding m A / D converters 55a and the soot extraction processing unit 6, and the connection / non-connection between the A / D converter 55a and the soot extraction processing unit 6 is provided. Control the connection. When the switch 57a enters the connected state, the image data S2 from the A / D converter 55a is provided to the soot extraction processing unit 6. Each of the m switches 57a is electrically connected to the soot extraction processing unit 6, and connection / disconnection is individually controlled.

  Here, the light detection unit 51 and its peripheral circuits will be described in more detail. FIG. 3 is a diagram illustrating an electrical connection relationship between the light detection unit 51, the amplifier 53 a, the A / D converter 55 a, and the soot extraction processing unit 6. Referring to FIG. 3, the light detection unit 51 includes a plurality of pixels 51 a configured by photoelectric conversion elements such as photodiodes. The light detection unit 51 includes a plurality of capacitors 51b and a plurality of readout switches 51c corresponding to the plurality of pixels 51a. The photoelectric conversion element of the pixel 51a and the capacitor 51b are connected in parallel to each other, and one end of a readout switch 51c is connected to one end of the photoelectric conversion element and the capacitor 51b. The other end of the read switch 51c is connected to the input end of the amplifier 53a together with the other end of the other read switch 51c included in the same row. Each of the read switches 51c is electrically connected to a control unit (not shown), and connection / disconnection is individually controlled according to a control signal S4 from the control unit. The output terminal of the amplifier 53a is electrically connected to the input terminal of the A / D converter 55a, and the output terminal of the A / D converter 55a is electrically connected to the input terminal of the soot extraction processing unit 6. .

  The operation of the light detection unit 51 and its peripheral circuits shown in FIG. 3 is as follows. When the light image L2 is incident on the light detection unit 51, a charge corresponding to the incident light amount of the light image L2 for each pixel 51a is accumulated in the capacitor 51b. When the read switches 51c are sequentially connected in each row in accordance with an instruction from the control unit, the charges Q accumulated in the capacitors 51b are sequentially sent to the amplifier 53a. The charge Q is converted into a voltage signal by the amplifier 53a and amplified to become an image signal S1. The image signal S1 is converted from an analog signal to a digital signal by the A / D converter 55a to become image data S2. The image data S2 is provided to the soot extraction processing unit 6.

  In order to perform the calculation in the cocoon extraction processing unit 6 at high speed, for example, the imaging unit 5 preferably further includes a parallel calculation circuit corresponding to each of the pixels 51a. Such a parallel arithmetic circuit is connected to, for example, a subsequent stage of the A / D converter 55a.

  4A and 4B are diagrams illustrating a method for detecting the edge of the eyelid based on the image data S2 in the eyelid extraction processing unit 6. FIG. FIG. 4A shows the image data S2, and the image data S2 includes an image D1 corresponding to the eye 100 shown in FIG. The haze extraction processing unit 6 according to the present embodiment first extracts the darkest dark area D2 in the image data S2. The dark area D2 is an area corresponding to the cornea (or pupil) of the eye 100 in the image data S2. For example, as shown in FIG. 4A, the dark part region D2 is extracted by setting regions A1 and A2 whose longitudinal direction is the eyelid opening / closing direction (that is, the vertical direction of the face), and luminance histograms (luminances) in the regions A1 and A2. It may be based on mutual comparison of distributions. FIG. 4B is a graph showing an example of a luminance histogram in the areas A1 and A2. In FIG. 4B, a graph G1 shows a luminance histogram in the region A1, and a graph G2 shows a luminance histogram in the region A2. The haze extraction processing unit 6 can determine the dark region D2 included in the graph G1 by comparing the graphs G1 and G2 in FIG.

  Subsequently, the eyelid extraction processing unit 6 performs the eyelid edge 101 (see FIG. 1) on the basis of the luminance histogram of the region that passes through the dark region D2 and whose opening / closing direction of the eyelid is the longitudinal direction (region A1 in this embodiment). The image portions D3 and D4 corresponding to are detected. The wrinkle extraction processing unit 6 of the present embodiment detects such image portions D3 and D4 by extracting edge portions E1 and E2 whose luminance changes rapidly in the graph G1 shown in FIG. 4B. Then, the positions of the image portions D3 and D4 in the eyelid opening / closing direction are provided to the eyelid opening / closing measuring unit 7 as eyelid position information. Note that since the momentum of the upper eyelid is larger than that of the lower eyelid during blinking, it is preferable that the eyelid extraction processing unit 6 detects the edge (image portion D3) of the upper eyelid to generate the eyelid position information. Thereby, eyelid position information can be generated more accurately.

  As another determination method related to the eyelid position, for example, in the graphs G1 and G2 shown in FIG. 4B, the positions of the edge portions E1 and E2 where the luminance histogram changes sharply are compared with each other, and both are in an arbitrary range. When it falls within, the wrinkle extraction processing unit 6 may determine the position as the wrinkle position. In this case, it is possible to extract the upper eyelid position and the lower eyelid position by comparing the change in the luminance value histogram (G1, G2) with a preset threshold th. In addition, as another calculation method for obtaining the eyelid position, a method for obtaining a moved position by comparing luminance histograms in the time direction, a block matching method, or an optical flow method may be used.

  FIG. 5 is a diagram illustrating an example of a calculation method used when the eyelid opening / closing measurement unit 7 calculates the blink feature amount based on the time change of the eyelid position information. In FIG. 5, (a) shows the time change of the position of the edge of the upper eyelid, and (b) shows the time change of the moving speed of the edge of the upper eyelid (that is, the time differential value of the graph shown in (a)). ). First, as an eyebrow feature amount, a case of calculating an eye closing speed that is a speed from the open eye state to the eye closing state, an eye opening speed that is a speed from the eye closing state to the eye opening state, and numerical values corresponding thereto will be described.

  In FIG. 5, time t1 indicates the eye closing start timing that can be placed in the blink of an eye, time t2 indicates the eye closing completion timing, time t3 indicates the eye opening start timing, and time t4 indicates the eye opening completion timing. The period T6 is a closed eye period, and the eyelid is in a closed state.

  For example, as shown in FIGS. 5A and 5B, the eyelid opening / closing measuring unit 7 performs eyelid opening / closing in a period T5 from the start of eye closure (time t1) to the time of eye closure completion (time t2). The maximum moving speed of the edge is calculated and presented as the closing eye speed. Similarly, the eyelid opening / closing measuring unit 7 calculates the maximum movement speed of the edge of the eyelid during a period T7 from the start of eye opening (time t3) to the time of eye opening completion (time t4) at the time of blinking. Present as speed. Specifically, eyelid opening / closing measuring unit 7 obtains transition information (FIG. 5B) regarding the movement speed of the edge of the upper eyelid by differentiating the eyelid position information shown in FIG. Then, the maximum value Vmax1 of the moving speed in the transition information period T5 is presented to the outside as the closed eye speed. Similarly, the maximum value Vmax2 of the moving speed in the transition information period T7 is presented to the outside as the eye opening speed. The eye closing start timing (time t1) and the eye opening completion timing (time t4) can be known, for example, when the speed of the upper eyelid falls below (or exceeds) the threshold speed Vth shown in FIG. The eye closing completion timing (time t2) and the eye opening start timing (time t3) can be known, for example, when the upper eyelid speed is lower (or higher) than the threshold speed Vth shown in FIG.

  Further, the eyelid opening / closing measuring unit 7 calculates the ratio between the eye closing speed and the eye opening speed, that is, [the eye opening speed] / [the eye closing speed] or [the eye closing speed] / [the eye opening speed] as numerical values corresponding to the eye closing speed and the eye opening speed. Alternatively, the difference between the eye closing speed and the eye opening speed, that is, [eye opening speed] − [eye closing speed] or [eye closing speed] − [eye opening speed] may be calculated. By adopting such numerical values, it is possible to calculate a blink feature amount that suppresses influences such as individual differences and measurement variations.

Another example of the blink feature amount will be described with reference to FIG. The graph G3 shown in FIG. 6 is the transition of the heel position (left scale), and the graph G4 is the transition of the heel speed (right scale). The blink measurement apparatus 1 according to the present embodiment can output detailed position information of eyelids at the time of blinking as a measurement value, and thus can calculate the following blink feature amount.
1. 1. Upper eyelid position before blinking (upper eyelid position when opening eyes): upper eyelid position before closing eyes, that is, upper eyelid position at time when eyelid speed before closing eyes is Vth or less 2. Upper eyelid position when the eye is closed: Upper eyelid position between the eye closing operation and the eye opening operation, that is, the upper eyelid position at a time when the eyelid speed sandwiched between the eye closing completion timing and the eye closing start timing is Vth or less. 3. Upper eyelid position after blinking (upper eyelid position when opening eye): upper eyelid position after eye opening, that is, upper eyelid position at time when eyelid speed after eye opening is Vth or less 4. Upper eyelid change amount when closed (H1): difference between eyelid position before blinking and eyelid position when closed 5. Upper eyelid change amount at opening (H2): difference between eyelid position when closed and eyelid position after blinking 6. Closed maximum speed time (ta): Time at which the absolute value of the closed eye speed is maximized 7. Closed / open eye minimum speed time (tb): Time when the eyelid position and eyelid speed become minimum when moving from the closed eye movement to the eye opening movement 8. Eye opening maximum speed time (tc): Time at which the absolute value of the eye opening speed is maximized 10. 10% time when eyes closed (td10): Time when eyelid position reaches 10% of distance from eyelid position before eyes closed to eyelid position after eyes closed 10. Time when eye is closed (td50): Time when eyelid position reaches 50% of distance from eyelid position before eye closure to eyelid position after eye closure 9. 90% time when the eye is closed (td90): Time when the eyelid position reaches 90% of the distance from the eyelid position before the eye is closed to the eyelid position after the eye is closed 10. 10% time at eye opening (te10): Time when eyelid position reaches 10% of distance from eyelid position before eye opening to eyelid position after eye opening. Time of eye opening 50% (te50): Time when eyelid position reaches 50% of distance from eyelid position before eye opening to eyelid position after eye opening 14. 90% time at eye opening (te90): Time when the eyelid position reaches 90% of the distance from the eyelid position before eye opening to the eyelid position after eye opening

Further, the following blink feature amount can be calculated from each time.
15. Closed eye time: (time difference between 10% time (td10) when closed and 90% time (td90) when closed)
16. Eye opening time: (the time difference between the 10% time (te10) when the eye is opened and the 90% time (te90) when the eye is opened)

In addition, in order to show the features when the eyes are closed / open, the following blink feature amount can be calculated.
17. Closed eye ratio: (td50-td90) / (td10-td50)
18. Ratio at the time of eye opening: (te50-te90) / (te10-te50)
19. Closed eye difference: (td50-td90)-(td10-td50)
20. Difference in eye opening: (te50-te90)-(te10-te50)
Further, the index relating to the speed is used as the blink feature quantity so far, but the feature quantity relating to the acceleration that is the time derivative of the speed can also be used.

Ｘstd＝｛Σ（Ｘi−Ｘave）2／n(n-1)｝(i=1,・・・,n)・・・（１） The eyelid opening / closing measurement unit 7 calculates a representative value for the blink feature amount related to n blink operations (n is a natural number of 2 or more) calculated as described above. As such representative values, the average value Xave of the blink feature amounts X1 to Xn for n times (n is a natural number of 2 or more) when the subject blink feature amount is X, the blink feature amounts X1 to X1. Examples include a maximum value Xmax of Xn, a minimum value Xmin of blink feature amounts X1 to Xn, and a variation Xstd of blink feature amounts X1 to Xn represented by the following equation (1).

Xstd = {Σ (Xi−Xave) 2 / n (n−1)} (i = 1,..., N) (1)

  When the representative value is calculated at the measurement timing of the predetermined continuous voluntary blink operation, the eyelid opening / closing measurement unit 7 determines that the representative value is the representative value of the first feature value regarding the continuous voluntary blink operation. recognize. The eyelid opening / closing measuring unit 7 evaluates the first feature value by comparing a preset evaluation pattern for the first feature value with the calculated representative value of the first feature value.

  Similarly, when the representative value is calculated at the measurement timing of a predetermined single voluntary blink operation, the eyelid opening / closing measurement unit 7 determines that the representative value is the second feature amount related to the single voluntary blink operation. recognize. The eyelid opening / closing measuring unit 7 evaluates the second feature value by comparing the preset evaluation pattern for the second feature value with the calculated representative value of the second feature value.

  FIG. 7A shows the time change of the position of the edge of the upper eyelid in the single voluntary blink operation, and FIG. 7B shows the time of the movement speed of the edge of the upper eyelid in the single voluntary blink operation. The change (that is, the time differential value of the graph shown in (a)) is shown. FIG. 8A shows the temporal change in the position of the edge of the upper eyelid in the continuous voluntary blink operation, and FIG. 8B shows the moving speed of the edge of the upper eyelid in the continuous voluntary blink operation. The time change of is shown. 7 and 8, the graph shapes are compared by superimposing the graphs of the measurement results of a plurality of times on the basis of the maximum speed time ta when the eyes are closed. 7 and 8, the solid line shows the measurement results of a plurality of blink operations before the VDT work (without fatigue), and the broken line shows a plurality of blink actions after the VDT work for about 1 hour. The measurement results are shown.

  As shown in FIGS. 7 and 8, in both the single voluntary blink operation and the continuous voluntary blink operation, the blink operation before the VDT work is highly reproducible, and the blink after the VDT work is the edge of the upper eyelid. It can be seen that in both the position displacement and the velocity displacement of the part, the variation in the graph shape becomes large and the reproducibility becomes low. In particular, the upper eyelid position at the time of eye opening (upper eyelid position before blinking, the upper eyelid position after blinking), the upper eyelid position at the time of closing eye, and the depth (the amount of change in the upper eyelid H1 when the eye is closed) in the continuous voluntary blink operation Regarding the variation in the amount H2), the difference before and after the VDT operation becomes significant. Moreover, in both the single voluntary blink operation and the continuous voluntary blink operation, it is possible to find a decrease in the maximum speed (eye closing speed) Vmax1 when the eyes are closed and an increase in the variation of the maximum speed (eyes opening speed) Vmax2 when the eyes are opened.

  As described above, since there is a difference in tendency between the first feature amount in the continuous voluntary blink operation and the second feature amount in the single voluntary blink operation, the comparison with the evaluation pattern corresponding to each tendency is performed. This makes it possible to obtain a reliable evaluation result of the blink feature amount. In addition, as the blink feature amount, measurement is performed by calculating the eye closing speed, the eye opening speed, and the upper eyelid position at the time of opening (upper eyelid position before blinking, upper eyelid position after blinking) where the difference between before and after the VDT operation is noticeable. The determination accuracy for the biological state of the subject can be improved.

  The eyelid opening / closing measuring unit 7 determines the biological state of the measurement target person based on the evaluation results of the first feature value and the second feature value. Examples of such a biological state include a state of concentration of consciousness, a disease state such as an eye movement disease, Alzheimer's disease, and the like in addition to a fatigue state and an arousal state for sleepiness. Hereinafter, the case where the fatigue state is determined as the biological state of the measurement subject will be described with reference to FIG.

  FIG. 9 is a graph showing the relationship between the VDT work time and the eye closing / opening speed. FIG. 9A shows the eye closing / opening speed immediately after the VDT work (without fatigue) and immediately after the VDT work was continued for 60 minutes in a single blink. FIG. 9B shows a measurement result in continuous blinking. The error notation shown at the front end of the bar graph indicates the standard deviation (variation) of the blink for a plurality of times. Referring to FIG. 9, it can be seen that all the numbers change greatly before and after VDT. For example, in FIG. 9A, the speed is about 1/3 before and after VDT, and the variation increases 3 to 4 times. The same tendency was seen also in FIG.9 (b). Therefore, by using an index relating to the closed / open eye calculated as the blink feature amount, it is possible to index the degree of fatigue of the measurement subject.

  Note that the eye-opening operation and the eye-closing operation in the blink operation are extremely high-speed phenomena in which the required time is about 50 milliseconds to 250 milliseconds. In order to measure these blink feature amounts, Measurement is required at a sampling period of about 1/10 times (5 milliseconds, that is, 200 Hz when capturing a phenomenon of 50 milliseconds). For this reason, with a conventional CCD camera (30 frames / second), it has been difficult to accurately measure the eye-opening operation and the eye-closing operation with high accuracy. In the above embodiment, the MOS type imaging device is exemplified as the imaging device having a high frame rate, but the present invention is not limited to this. For example, a high frame rate can be realized by using a partial reading function (high-speed partial reading) or a collective reading (binning function) of a CCD camera.

The blink measurement apparatus 1 presents the biological state of the measurement subject determined in this manner to the outside. Examples of the presentation destination include a display device (display) and various types of fatigue reduction devices (devices that adjust the contrast, brightness, etc. of VDT).

  FIG. 10 is a flowchart regarding the operation of the blink measurement apparatus 1. As shown in FIG. 10, in the blink measurement apparatus 1, first, in the imaging process, the imaging unit 5 captures the eye 100 and its surroundings shown in FIG. 1 to acquire image data (step S1). Next, the dark spot area D2 (black eye part) shown in FIG. 4A is extracted by the eyelid extraction processing unit 6, and the luminance histogram of the area A1 passing through the dark part area D2 and having the eyelid opening / closing direction as the longitudinal direction (FIG. 4). (B)) is created. The eyelid extraction processing unit 6 detects an image portion D3 corresponding to the edge portion 101 (see FIG. 1) of the eyelid based on the luminance histogram, and creates eyelid position information indicating the position of the image portion D3 ( Step S2).

  When the eyelid position information is created in step S2, the instruction unit 10 recognizes that it is the measurement timing of the single voluntary blink operation, and issues an instruction for causing the measurement subject to perform the continuous voluntary blink operation ( Step S3). When the measurement subject performs a continuous voluntary blink operation, the eyelid opening / closing measuring unit 7 calculates the first feature amount based on the temporal change of the eyelid position information in the first feature amount calculating step (step S4). . The heel opening / closing measuring unit 7 calculates a representative value of these first feature values after calculating the first feature value a predetermined number of times (step S5).

  When the representative value of the first feature value is calculated in step S5, the instruction unit 10 recognizes that it is the measurement timing of the continuous voluntary blink operation and causes the measurement target person to perform the single voluntary blink operation. Is issued (step S6). When the measurement subject performs a single voluntary blink operation, the eyelid opening / closing measuring unit 7 calculates the second feature amount based on the temporal change of the eyelid position information in the second feature amount calculating step (step S7). . Thereafter, the eyelid opening / closing measuring unit 7 calculates a representative value in the second feature amount (step S8).

  Subsequently, the eyelid open / close measuring unit 7 evaluates the representative value of the first feature value and the representative value of the second feature value (step S9). The eyelid opening / closing measuring unit 7 determines the biological state of the measurement subject based on the evaluation results of the representative value of the first feature value and the representative value of the second feature value (step S10).

  According to the blink measuring apparatus 1 described above, by calculating and evaluating the first and second feature amounts for the blink operations having different characteristics, that is, the single voluntary blink operation and the continuous voluntary blink operation, respectively, Compared with the case where only the feature value of the voluntary blink operation is evaluated, it is possible to determine the biological state of the measurement subject with higher accuracy, and thus it is possible to improve the reliability related to the determination result of the biological state. . Further, by measuring the single voluntary blink operation and the continuous voluntary blink operation, it is possible to improve the reliability related to the determination result of the biological state with a relatively simple configuration and procedure.

  In addition, since the blink measuring device 1 includes the instruction unit 10 that gives an instruction to the measurement subject to perform a single voluntary blink operation or a continuous voluntary blink operation, a measurement operation for operating the device. Since it is not necessary for the operator to give an instruction to perform a single voluntary blink operation or a continuous voluntary blink operation to the measurement subject, the measurement work time can be shortened and the labor of the measurement worker can be reduced. Further, since it is possible to automatically issue an instruction at an appropriate timing when the blink measurement device 1 is ready for measurement, it is advantageous for automation of measurement work.

  Note that the blink measurement apparatus 1 does not necessarily measure both the single voluntary blink operation and the continuous voluntary blink operation, and can determine the biological state only by measuring the continuous voluntary blink operation, for example. In this case, the biological state is determined only by measuring the single voluntary blink movement by calculating and evaluating the first feature amount of the continuous voluntary blink movement in which the biological state of the measurement subject is easily reflected. Compared to the above, it becomes possible to determine the biological state of the measurement subject with higher accuracy.

  The blink measurement apparatus 1 according to the present embodiment can be applied as follows. For example, the eyes of a person engaged in VDT work, driving a car, product inspection, etc. are measured, and when the eye opening speed or a numerical value corresponding to the eye opening speed exceeds a predetermined threshold, a warning is issued to encourage rest. It can be applied as a degree monitor. In addition, according to the eye opening speed measured by the blink measurement device 1 or a numerical value corresponding to the eye opening speed, environmental parameters (for example, the contrast and brightness of the screen during VDT work, the brightness of the room during work, etc.) It is also possible to adjust the work environment according to the worker's fatigue level by adjusting the. In addition, since the blink measurement device 1 can be configured in a small size and can perform non-contact measurement, it can be used as a simple health measurement device in combination with a camera function of a mobile phone or a WEB camera of a personal computer.

  The blink measuring device according to the present invention is not limited to the above-described embodiment, and various other modifications are possible. For example, depending on the application, it is necessary for the subject to measure the blink at the same time while looking at the normal outside world. In this case, the subject is mounted in the form of a head-mounted display with the configuration shown in FIG. It is also possible.

  In the above embodiment, an imaging device (IVS camera) having a frame rate of 1 kHz or more is used as the imaging means, but an area sensor or a line sensor may be used. When using a line sensor, in order to detect a change in the position of the eyelid accompanying blinking, the line sensor may be installed so that the longitudinal direction of the line sensor is along the opening / closing direction of the eyelid. Further, the blink feature amount is not limited to that described above. The eye opening speed calculated as the blink feature amount or a numerical value corresponding to the eye opening speed is the above-described three numerical values (maximum moving speed, moving distance in a predetermined period including the time when the maximum moving speed is reached, and half-value width of the moving speed). It is not limited, and other numerical values may be calculated / presented.

  In addition, the instruction to the measurement subject by the instruction unit 10 is not limited to that by voice, but is an aspect in which an instruction is given by external stimulation such as vibration, electrical stimulation, wind stimulation, etc. in addition to blinking of an indicator and blinking of an illumination light source. May be. Further, the order of measurement of the continuous voluntary blink operation and the single voluntary blink operation is not limited to the above-described order, and either may be measured first.

  DESCRIPTION OF SYMBOLS 1 ... Blink measuring device, 2 ... Illumination, 3 ... Dichroic mirror, 4 ... Condensing lens, 5 ... Imaging part (imaging means), 6 ... Haze extraction process part, 7 ... Haze opening / closing measurement part (1st feature-value) Calculation means, second feature quantity calculation means, biological state determination means), 8 ... camera, 9 ... target, 10 ... instruction unit (instruction unit), 11 ... speaker, 51 ... light detection unit, 51a ... Pixel, 53 ... Amplifier, 53a ... Amplifier, 55 ... Converter, 55a ... Converter, 57 ... Switch, 57a ... Switch, D2 ... Dark part, E1, E2 ... Edge part, L1 ... Infrared light, L2 ... light image, L3 ... visible light.

Claims (7)

A blink measurement device for determining the biological state of the measurement subject based on the blink operation of the measurement subject,
Imaging means for imaging the eye of the measurement subject;
First feature amount calculating means for calculating a first feature amount related to continuous voluntary blink operation based on the image of the eye imaged by the imaging means;
A biological state determination unit that determines a biological state of the measurement subject based on the first feature amount calculated by the first feature amount calculation unit;
A blink measuring device characterized by comprising: A second feature quantity calculating means for calculating a second feature quantity related to the single voluntary blink operation based on the image of the eye imaged by the imaging means;
The biological state determination means is based on the first feature quantity calculated by the first feature quantity calculation means and the second feature quantity calculated by the second feature quantity calculation means. The blink measurement apparatus according to claim 1, wherein the measurement target person's biological state is determined.   3. The blink according to claim 1, further comprising an instruction unit that issues an instruction for causing the measurement subject to perform the single voluntary blink operation or the continuous voluntary blink operation. Measuring device. The first feature amount calculating means calculates an upper eyelid position at the time of eye opening in the continuous voluntary blink operation as the first feature amount,
The said 2nd feature-value calculation means calculates the upper eyelid position at the time of eye opening in the said one-time voluntary blink operation | movement as said 2nd feature-value. The blink measurement apparatus of description. The first feature amount calculating means calculates an eye closing speed in the continuous voluntary blink operation as the first feature amount,
5. The instantaneous feature value according to claim 1, wherein the second feature value calculating unit calculates an eye closing speed in the one-time voluntary blink operation as the second feature value. Eye measuring device. The first feature amount calculating means calculates an eye opening speed in the continuous voluntary blink operation as the first feature amount,
6. The instantaneous feature value according to claim 1, wherein the second feature value calculating means calculates an eye opening speed in the single voluntary blink operation as the second feature value. Eye measuring device. A blink measurement method for determining a biological state of the measurement subject based on a blink operation of the measurement subject,
An imaging step of imaging the eye of the measurement subject;
A first feature amount calculating step of calculating a first feature amount related to continuous voluntary blink operation based on the image of the eye imaged by the imaging means;
A biological state determination step of determining a biological state of the measurement subject based on the first feature amount calculated in the first feature amount calculation step;
A blink measurement method characterized by comprising:

Images