JP5825588B2 - Blink measurement device and blink measurement method - Google Patents

Description

  The present invention relates to a blink measurement device and a blink measurement method.

  It is known that humans blink about 20 times per minute (blink). In the past, it was said that “to protect the eyeball from dryness”, but there are many unknown parts about why blinks occur, such as the number of blinks hardly changing due to changes in environmental humidity and wind speed. Under such circumstances, in recent years, the form of blinking has a biological state (eye strain (VDT fatigue), fatigue, concentration, arousal (sleepiness), lifestyle-related diseases, mental illness, neurological diseases, dementia, etc. It is reported that it can be an indicator related to

  For example, Patent Document 1 describes that the speed and acceleration at the start of blinking are used as parameters for estimating the state of eye fatigue (degree of eye fatigue). Patent Document 2 describes that the number and frequency of blinks are used as parameters for estimating the degree of eye fatigue. In Non-Patent Document 1, as a parameter for detecting / estimating a driver's consciousness reduction state (especially sleepiness) from a blink start to a closed eye state until the eye is opened again (that is, blink complete). It is described that the time is used.

  On the other hand, the present inventors have been developing a high-speed blink measurement device so far, and proposed a blink measurement device that calculates the fatigue degree based on the correlation between the fatigue degree and the blink feature amount. Various verifications are performed using such an apparatus (see Patent Document 3) (see Non-Patent Document 2).

JP-A-8-289327 Japanese Patent No. 3348956 JP 2009-125154 A

Kazumasa Adachi, 3 others, “Blink measurement by moving image processing for detection of driver consciousness decline”, IEEJ Transactions C (Electronics, Information and Systems Division), IEEJ, 2004, Vol. 124, Vol. 3, p. 776-783 Yoshiko Nakamura and 6 others, “Measurement of Spontaneous Blinks Using High-Speed Blink Analyzer”, Journal of the Japanese Ophthalmological Society 112 (12), 2008, p. 1059-1067

  The devices described in Patent Documents 1 and 2 and Non-Patent Document 1 calculate each parameter based on a face image acquired by a CCD camera. However, current general CCD cameras can only perform imaging at a low speed of 30 frames / second. In this case, the required time per frame is about 33 milliseconds. On the other hand, the time required for human blinking (from blinking start to eye opening again after closing the eye) is about 100 to 300 milliseconds, and the CCD camera can only obtain 3 to 10 frame images. Although it is possible to measure macro feature amounts, it is difficult to accurately measure qualitative feature amounts related to detailed movements of blinks. Therefore, there is doubt about the reliability of whether the parameters described in Patent Documents 1 and 2 or Non-Patent Document 1 can accurately estimate the fatigue level. It is hard to say that it is based on an accurate correlation between degree and eye movement.

  On the other hand, the devices described in Patent Literature 3 and Non-Patent Literature 2 are expected to discriminate in more detail and more accurately the mode of blinking that can be an index related to a biological state.

  This invention is made in view of such a situation, and makes it a subject to provide the blink measurement apparatus and blink measurement method which can discriminate an irregular blink as a parameter | index relevant to a biological condition. .

In order to solve the above-described problem, the blink measuring apparatus of the present invention includes an imaging unit that captures an eye of a measurement target person, and a single blink performed by the measurement target person based on an eye image captured by the imaging unit. Eyeblink discriminating means for discriminating eyeblinks such that three or more true extreme values appear in the temporal change in acceleration of eyelids that close and open eyelids, and the eyeblink discriminating means comprises: The eyelid acceleration data indicating the temporal change in the acceleration of the eyelid that closes and opens in a single blink is obtained, and the acceleration of the eyelid that closes and opens in a single blink based on the eyelid acceleration data The extreme values appearing in the time change of the time are extracted, the acceleration difference between each of the target points extracted as extreme values and the target point before and after is calculated, and both of the absolute values of the acceleration differences are equal to or less than a predetermined threshold Judge extreme values other than the extreme values as true extreme values More if polarity appear consecutively same true extremum, by selecting the a larger absolute value of the acceleration of the true extremum successive extracts true extremum, the true extremum When three or more points are extracted, the blink is determined to be an irregular blink.

Further, the blink measurement method of the present invention includes an imaging step in which the imaging unit images the eye of the measurement target person, and a blink determination unit that is based on the eye image captured in the imaging step. A blink determination step for determining a blink that has three or more true extreme values in the temporal change in the acceleration of the eyelid that is closed and opened in one blink, as an irregular blink, In the determination step, the blink determination unit obtains eyelid acceleration data indicating temporal changes in the acceleration of the eyelid that closes and opens in one blink, and based on the eyelid acceleration data, Extract the extreme values that appear in the temporal change in the acceleration of the eyelids that close and open in eyelids, calculate the difference in acceleration from the previous and subsequent target points for each of the target points extracted as extreme values, and calculate the absolute value of the difference in acceleration extreme both other than extreme fake equal to or less than a predetermined threshold If it is judged as a true extremum and true extremums with the same polarity appear in succession, the true extremum is selected by selecting the consecutive true extremums with the larger absolute value of acceleration. When a value is extracted and three or more true extremum values are extracted, the blink is determined to be the irregular blink.

In these blink measurement devices and blink measurement methods, blinks whose true extreme values appear at three or more points in the temporal change in acceleration of the eyelids to be closed and opened are determined to be irregular blinks. As a result, an irregular blink accompanied by intermittent movement of the eyelids (fine movement of the eyelids) when the eyes are closed or opened (that is, the eyelids stop or open during the closing of the eyes, or the eyelids stop or close during the opening of the eyes) It is possible to detect anomalous blinks in which wrinkles appear. Thus, according to these blink measurement devices and blink measurement methods, it is possible to determine an irregular blink as an index related to the biological state. Note that the closed eye means a series of operations for closing the eyes in one blink, and the opening of the eye means a series of operations for opening the eyes in one blink. Moreover, closing means the movement of the eyelid in the closing direction, and opening means the movement of the eyelid in the opening direction.

  Here, the blink determination unit obtains eyelid position data indicating temporal changes in the position of the eyelid that closes and opens in one blink, and based on the eyelid position data, in one blink. It is preferable to discriminate blinks from irregular blinks when three or more extreme values appearing in the temporal change in the position of the eyelids to close and open are extracted.

  The blink determination means acquires eyelid speed data indicating temporal changes in the speed of eyelids that close and open in a single blink, and closes in a single blink based on the eyelid speed data. It is preferable to discriminate blinks from irregular blinks when three or more extreme values appearing in the temporal change in the speed of eyelids and eyelids to be opened are extracted.

  According to these, it is possible to detect an irregular blink associated with slight eyelid movement when the eye is closed or when the eye is opened based on at least one of the eyelid position data, the eyelid velocity data, and the eyelid acceleration data.

  According to the present invention, an irregular blink can be determined as an index related to a biological state.

It is a block diagram of the blink measurement apparatus of one Embodiment of this invention. It is a block diagram of the imaging part of the blink measurement apparatus of FIG. It is a figure which shows the electrical connection relationship in the imaging part of the blink measurement apparatus of FIG. It is a figure which shows the production | generation method of the eyelid position information by the eyelid extraction part. It is a graph which shows the time change of the position, speed, and acceleration of the eyelid in a normal blink. It is a graph which shows the time change of the position, speed, and acceleration of the eyelid in an irregular blink. It is a graph which shows the time change of the position, speed, and acceleration of the eyelid in an irregular blink. It is a graph which shows the time change of the position, speed, and acceleration of the eyelid in an irregular blink.

  DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. In addition, in each figure, the same code | symbol is attached | subjected to the same or an equivalent part, and the overlapping description is abbreviate | omitted.

  As shown in FIG. 1, the blink measurement device 1 includes an illumination 2, a dichroic mirror 3, a condenser lens 4, an imaging unit (imaging unit) 5, an eyelid extraction unit 6, and a blink determination unit (blink determination unit). 7 is provided. The condenser lens 4, the imaging unit 5, the eyelid extraction unit 6, and the blink determination unit 7 are housed inside the camera 8.

  The illumination 2 is illumination means for illuminating the measurement subject's eye 100 and its surroundings, and is preferably composed of, for example, a plurality of LEDs that generate infrared light. When the illumination 2 irradiates the eye 100 and its periphery with the infrared light L1, the infrared light L1 is reflected at the eye 100 and its periphery to generate a light image L2. The illumination 2 is not limited to the infrared LED, but other light sources can be used. At the time of high-speed imaging, the measurement target person emits a sufficient amount of visible light to illuminate the state of the eye 100 and the like. Since it is dazzling, it is preferable to use an infrared light source.

  The dichroic mirror 3 is disposed so as to transmit the light image L2 from the eye 100 and its periphery and to make the light image L2 enter the light detection unit 51 of the imaging unit 5. In addition, the dichroic mirror 3 optically couples the visual target 9 and the eye 100 so that the measurement target can visually recognize the visual target 9 installed beside the optical axis connecting the eye 100 and the imaging unit 5. doing. The visual target 9 may be configured to generate a bright spot pattern by combining, for example, an LED that generates visible light and a pinhole mask. The dichroic mirror 3 reflects the visible light L3 emitted from the visual target 9 toward the eye 100. Accordingly, the eye 100 and its periphery can be imaged while presenting the visual target 9 to the measurement subject and maintaining the corneal position of the eye 100 constant. It should be noted that a lens 9 a for visual acuity adjustment is preferably provided between the visual target 9 and the dichroic mirror 3.

  The condensing lens 4 is a lens for condensing the light image L2 to form an image on the light detection unit 51 of the imaging unit 5. The condenser lens 4 is disposed between the dichroic mirror 3 and the imaging unit 5. The condensing lens 4 is configured such that the entire orbit of the eye 100 falls within the angle of view of the imaging unit 5 so that the imaging unit 5 described later can acquire an image including the entire eye 100.

  The imaging unit 5 captures the eye 100 of the measurement subject by capturing the light image L2 from the eye 100. The imaging unit 5 includes a light detection unit 51 including a plurality of pixels arranged in a two-dimensional shape, and converts the light image L2 incident on the light detection unit 51 into an electrical signal in each pixel. Image data indicating the amount of incident light for each pixel relating to the image L2 is generated. The imaging unit 5 outputs the generated image data to output means such as a display device or a video output terminal, and provides the image data to the eyelid extraction unit 6.

  The eyelid extraction unit 6 detects the eyelid edge 101 in the image data based on the image data provided from the imaging unit 5 and generates eyelid position information. Further, the blink determination unit 7 determines a blink mode (normal blink or irregular blink) based on the eyelid position information provided from the eyelid extraction unit 6. The eyelid extraction unit 6 and the blink determination unit 7 may be realized as an electric circuit, or may be realized as software inside a computer having a central processing unit and a memory.

  As shown in FIG. 2, the imaging unit 5 is an imaging device having an extremely high frame rate of 1 kHz or more. As such an imaging device, for example, an intelligent vision sensor (IVS) camera manufactured by Hamamatsu Photonics Co., Ltd. Is mentioned. The imaging unit 5 includes a light detection unit 51, an amplification unit 53, an A / D conversion unit 55, and a switch unit 57. Here, the light detection unit 51 is a so-called MOS type image pickup device, and includes a plurality of pixels 51 a arranged in a two-dimensional shape (m rows × n columns). Each of the plurality of pixels 51a generates a charge Q corresponding to the amount of incident light.

  The amplifying unit 53 includes m amplifiers 53 a corresponding to the number of rows of the light detecting unit 51. The m amplifiers 53a are electrically connected to the corresponding rows of the pixels 51a of the light detection unit 51, respectively, and sequentially receive the charges Q from the n columns of pixels 51a. The amplifier 53a amplifies the charge Q and converts the charge Q into an image signal S1 that is a voltage signal. Each of the m amplifiers 53a is electrically connected to a control unit (not shown), and changes the amplification factor when converting the charge Q into the image signal S1 according to the amplification factor control signal S3 from the control unit. Can be made.

  The A / D converter 55 has m A / D converters 55 a corresponding to the number of rows of the light detector 51. The m A / D converters 55a are electrically connected to the corresponding m amplifiers 53a, respectively, and receive an image signal S1 that is a voltage signal (analog signal) from the amplifier 53a, and thus an image that is a digital signal. Convert to data S2. In the present embodiment, the image data S2 converted into a digital signal is used as the image data from the imaging unit 5, but the image signal S1 that is an analog signal may be used as the image data from the imaging unit 5.

  The switch unit 57 includes m switches 57 a corresponding to the number of rows of the light detection unit 51. The m switches 57 a are provided between the corresponding m A / D converters 55 a and the eyelid extraction unit 6, and connect / disconnect the A / D converter 55 a and the eyelid extraction unit 6. Control. When the switch 57a is connected, the image data S2 from the A / D converter 55a is provided to the eyelid extraction unit 6. Each of the m switches 57a is electrically connected to the eyelid extraction unit 6, and connection / disconnection is individually controlled.

  Here, the light detection unit 51 and its peripheral circuits will be described in more detail. As illustrated in FIG. 3, the light detection unit 51 includes a plurality of pixels 51 a configured by photoelectric conversion elements such as photodiodes. The light detection unit 51 includes a plurality of capacitors 51b and a plurality of readout switches 51c corresponding to the plurality of pixels 51a. The photoelectric conversion element of the pixel 51a and the capacitor 51b are connected in parallel to each other, and one end of a readout switch 51c is connected to one end of the photoelectric conversion element and the capacitor 51b. The other end of the read switch 51c is connected to the input end of the amplifier 53a together with the other end of the other read switch 51c included in the same row. Each of the read switches 51c is electrically connected to a control unit (not shown), and connection / disconnection is individually controlled according to a control signal S4 from the control unit. The output end of the amplifier 53a is electrically connected to the input end of the A / D converter 55a, and the output end of the A / D converter 55a is electrically connected to the input end of the eyelid extraction unit 6.

  The operation of the above-described light detection unit 51 and its peripheral circuits is as follows. When the light image L2 is incident on the light detection unit 51, a charge Q corresponding to the incident light amount of the light image L2 for each pixel 51a is accumulated in the capacitor 51b. When the read switches 51c are sequentially connected in each row in accordance with an instruction from the control unit, the charges Q accumulated in the capacitors 51b are sequentially sent to the amplifier 53a. The charge Q is converted into a voltage signal by the amplifier 53a and amplified to become an image signal S1. The image signal S1 is converted from an analog signal to a digital signal by the A / D converter 55a to become image data S2. The image data S2 is provided to the eyelid extraction unit 6.

  In order to perform the calculation in the eyelid extraction unit 6 at high speed, for example, the imaging unit 5 preferably further includes a parallel calculation circuit corresponding to each of the pixels 51a. Such a parallel arithmetic circuit is connected to, for example, a subsequent stage of the A / D converter 55a.

  Next, generation of eyelid position information by the eyelid extraction unit 6 will be described. FIG. 4A is a diagram illustrating an example of the image data S2. The image data S2 includes the image D1 corresponding to the eye 100 illustrated in FIG. First, the eyelid extraction unit 6 extracts the darkest dark region D2 in the image data S2 in FIG. The dark area D2 is an area corresponding to the cornea (or pupil) of the eye 100 in the image data S2. For the extraction of the dark part region D2, for example, a region A1 and a region A2 whose longitudinal direction is the eyelid opening / closing direction (that is, the vertical direction of the face) are set, and luminance histograms (luminance distributions) in the regions A1 and A2 are compared with each other. Based on

  FIG. 4B is a graph showing an example of a luminance histogram in each of the areas A1 and A2. In FIG. 4B, a graph G1 shows a luminance histogram in the region A1, and a graph G2 shows a luminance histogram in the region A2. The eyelid extraction unit 6 can determine the dark region D2 included in the graph G1 by comparing the graph G1 and the graph G2 in FIG.

  Subsequently, the eyelid extraction unit 6 corresponds to the edge portion 101 (see FIG. 1) of the eyelid based on the luminance histogram of the region that passes through the dark region D2 and whose opening / closing direction of the eyelid is the longitudinal direction (here, the region A1). Image portions D3 and D4 are detected. The eyelid extraction unit 6 detects such image portions D3 and D4 by extracting edge portions E1 and E2 whose luminance changes rapidly in the graph G1. Then, the eyelid extraction unit 6 provides the position of the image portions D3 and D4 in the eyelid opening and closing direction to the blink determination unit 7 as eyelid position information. The eyelid extraction unit may detect the position of the eyelid using image processing such as contour extraction, area division, color reduction processing, and template matching.

  Next, a description will be given of the type of blink (normal blink or irregular blink) by the blink determination unit 7. Here, natural blinks performed unconsciously by the person to be measured are set as discrimination targets. Further, since the momentum of the upper eyelid is larger than the momentum of the lower eyelid during blinking, the eyelid extraction unit 6 detects the position of the upper eyelid.

  First, as shown in FIG. 5A, the blink determination unit 7 closes and opens the upper eyelid in one blink based on the eyelid position information provided from the eyelid extraction unit 6. The eyelid position data indicating the time change of the position is generated and acquired. This eyelid position data is data from which noise has been removed by the smoothing process.

  Subsequently, the blink determination unit 7 obtains the time difference (differentiation) of the eyelid position data, and as shown in FIG. 5B, the upper eyelid that closes and opens in one blink. Generate and obtain eyelid velocity data indicating the temporal change in velocity. This eyelid velocity data is data from which noise has been removed by a smoothing process.

  Subsequently, when the absolute value of the upper eyelid speed in the closing direction (minus direction in FIG. 5B) exceeds the threshold value Vth for a predetermined time (for example, 10 ms), the blink determination unit 7 starts to exceed that. Let the time be the blink start time Ts. Further, the blink determination unit 7 determines that the absolute value of the upper eyelid speed in the opening direction (plus direction in FIG. 5B) exceeds the threshold Vth for a predetermined time (for example, 10 ms), and then sets the threshold Vth for the predetermined time. When the time falls below (for example, 10 ms), the time when the time starts to fall is set as the blink end time Te. The threshold value Vth is, for example, 10 mm / s corresponding to 5% of 200 mm / s, which is an average value of blink speed.

  Note that the blink determination unit 7 may set the blink start time Ts and the blink end time Te based on the eyelid position data. That is, the blink determination unit 7 determines that the value of the change in the position of the upper eyelid in the closing direction (the minus direction in FIG. 5A) is a predetermined threshold (for example, 6 mm which is a general value of the blink depth). The time when the absolute value of the position of the upper eyelid in the closing direction is less than the threshold value after the eye is closed once, is the time when the eye blink end time Te. It is good. In addition, after the natural blink measurement, the closing and opening may be instructed and the change value of the eyelid due to the blink may be obtained. The setting of the blink start time Ts and the blink end time Te may be performed by individually separating and integrating the blink waveform and the feature value while using blink analysis software or the like.

  Subsequently, the blink determination unit 7 obtains the time difference (differentiation) of the eyelid velocity data, and as shown in FIG. 5C, the upper eyelid that closes and opens in one blink. Generate and acquire eyelid acceleration data indicating the temporal change in acceleration. This eyelid acceleration data is data from which noise has been removed by the smoothing process.

  In addition, about the eyelid acceleration data, the further noise removal may be performed as follows. That is, the acceleration of the upper eyelid is obtained in a period other than the period in which the blink is performed (from the blink start time Ts to the blink end time Te), and the average absolute value of the acceleration is used as the fluctuation amount. Then, for example, about three times the fluctuation amount is set as a threshold value, and acceleration having an absolute value equal to or less than the threshold value is removed as noise.

  Subsequently, the blink determination unit 7 closes and opens the eyelid in one blink (that is, a period from the blink start time Ts to the blink end time Te) based on the acquired eyelid position data. The extreme values (maximum value and minimum value) appearing in the time change of the position of are extracted. Moreover, the blink determination part 7 extracts the extreme value which appears in the time change of the speed of the eyelid which closes and opens in one blink based on the acquired eyelid speed data. Further, the blink determination unit 7 extracts an extremum that appears in the time change of the eyelid acceleration that closes and opens in one blink based on the acquired eyelid acceleration data.

  Then, as shown in FIG. 5, the blink determination unit 7 has two or less extracted extreme values (black circles in FIG. 5) in any of the eyelid position data, the eyelid velocity data, and the eyelid acceleration data. In this case, the blink is determined to be a normal blink.

  On the other hand, as shown in FIGS. 6 to 8, the blink determination unit 7 sets an extreme value (black circle in FIGS. 6 to 8) to 3 in at least one of the eyelid position data, the eyelid velocity data, and the eyelid acceleration data. If more than one point is extracted, the blink is determined to be an irregular blink.

  In the example of FIG. 6, three or more extreme values are extracted from the eyelid position data (see FIG. 6 (a)), the eyelid velocity data (see FIG. 6 (b)), and the eyelid acceleration data (see FIG. 6 (c)). ing. In the example of FIG. 7, three or more extreme values are not extracted from the eyelid position data (see FIG. 7A), but the eyelid velocity data (see FIG. 7B) and the eyelid acceleration data (FIG. 7C). ))), Three or more extreme values are extracted. In the example of FIG. 8, three or more extreme values are not extracted from the eyelid position data (see FIG. 8A) and the eyelid velocity data (see FIG. 8B), but the eyelid acceleration data (FIG. 8C ))), Three or more extreme values are extracted.

  As described above, in the blink measurement apparatus 1, even if three or more extreme values are not extracted from the eyelid position data or the eyelid velocity data, the time for acceleration of eyelids that close and open in one blink Blinks in which three or more extreme values appear in the change are reliably determined as irregular blinks.

  Here, a first method and a second method for extracting extreme values from eyelid acceleration data will be described with reference to FIGS. The first method and the second method are methods for removing false extreme values caused by noise or the like in extraction of extreme values from eyelid acceleration data. It should be noted that the extreme value in each of the eyelid position data and the eyelid velocity data can be extracted by the same method.

First, in the first method, for each target point (black circle and white circle in FIGS. 5 to 8) extracted as an extreme value, the difference in acceleration from the previous and subsequent target points is calculated, and both of the absolute values are predetermined. If it is less than the value, it is determined as a false extreme value. The first target point uses the acceleration at the blink start time Ts instead of the previous target point, and the last target point uses the acceleration at the blink end time Te instead of the subsequent target point. The determination of the previous false extreme, if the polarity appear the same polarity values are continuous (i.e., the maximum value - maximum or minimum value - when continuously the minimum value), the larger the absolute value of the acceleration The extreme values are determined as true extreme values, and the remaining extreme values are determined as false extreme values. Thereby, false extreme values (white circles in FIGS. 5 to 8) are removed, and true extreme values (black circles in FIGS. 5 to 8) are extracted. Note that the true extreme value may be counted after the true extreme value is extracted, or may be simultaneously with the extraction of the true extreme value.

  Next, in the second method, each of the target points (black circles and white circles in FIGS. 5 to 8) extracted as extreme values is sequentially scanned. When the difference from the previous target point exceeds a predetermined threshold, the target point is temporarily adopted as an extreme value. On the other hand, when the difference from the previous target point falls below a predetermined threshold, the next target point is compared with the previous target point, and the previous target point is determined by the larger value. Update. In accordance with the movement of the scanning position, the previous target point temporarily adopted is used as an extreme value. Thereby, false extreme values (white circles in FIGS. 5 to 8) are removed, and true extreme values (black circles in FIGS. 5 to 8) are extracted. Note that the true extreme value may be counted after the true extreme value is extracted, or may be simultaneously with the extraction of the true extreme value.

  As described above, in the blink measurement apparatus 1, the imaging unit 5 captures the eye 100 of the measurement target person (imaging process), and the blink determination unit 7 applies the image of the eye 100 captured by the imaging unit 5. Based on the blinking eye, the blinking eye that causes three or more extreme values to appear in the time change of the acceleration of the eyelid that is closed and opened in a single blink by the measurement subject is determined as an irregular blink (blink determination step). . As a result, an irregular blink accompanied by intermittent movement of the eyelids (fine movement of the eyelids) when the eyes are closed or opened (that is, the eyelids stop or open during the closing of the eyes, or the eyelids stop or close during the opening of the eyes) It is possible to detect anomalous blinks in which wrinkles appear. Thus, according to the blink measurement method performed by the blink measurement device 1 and the blink measurement device 1, the biological state (eye strain (VDT fatigue), fatigue, concentration, arousal (sleepiness)). As an index related to lifestyle-related diseases, mental illness, neurological diseases, dementia, etc.) Quantification and screening for lifestyle-related diseases, mental illnesses, neurological diseases, and dementia.

  In addition, when at least one extreme value is extracted from at least one of eyelid position data, eyelid velocity data, and eyelid acceleration data, the blink determination unit 7 determines that the blink is an irregular blink. Thus, even when three or more extreme values are not extracted from the eyelid position data and the eyelid velocity data, it is possible to reliably detect an irregular blink associated with slight eyelid movement when the eye is closed or opened.

  When discriminating irregular blinks, the extreme values in the eyelid velocity data have the speed value, the difference between the extreme values before and after (polarity and absolute value), and the time interval between the extreme values before and after the feature value. Alternatively, an auxiliary feature amount for discrimination may be used.

  Also, when discriminating irregular blinks, the extremum in the eyelid acceleration data has the acceleration value, the difference between the front and back extreme values (polarity and absolute value), and the time interval between the front and rear extreme values as feature quantities. Alternatively, it may be used as an auxiliary feature amount for discrimination.

Further, the following feature amounts may be calculated. The following feature values may be used for comprehensive judgment after being determined as anomalous blinks after being separately divided into those at the time of closing and those at the time of opening.
1: Maximum value and its time and half value width in temporal change of blink speed 2: Minimum value and its time and half value width in temporal change of blink speed 3: Maximum value and its time and half in temporal change of blink speed Value range 4: Minimum value, time and half value width of blink change over time 5: Generation interval of velocity extreme value in the same direction as the traveling direction 6: Number of occurrences of velocity extreme value in the same direction as the traveling direction 7: Traveling direction The difference between the speed extreme values in the same direction as the ratio, the ratio 8: the generation interval of the acceleration extreme values in the same direction as the travel direction 9: The number of occurrences of the acceleration extreme values in the same direction as the travel direction 10: the acceleration extreme values in the same direction as the travel direction , Ratio 11: generation interval of continuously generated velocity extreme values 12: difference of velocity extreme values generated continuously, ratio 13: generation interval of continuously generated acceleration extreme values 14: generated continuously Difference of acceleration extreme values, ratio 15: period from switching to closing eyeblink 6: the number temporal change in acceleration exceeds the threshold value 17: period change with time of the acceleration exceeds the threshold value 18: moving distance of the period change with time of the acceleration exceeds the threshold value

  Further, as shown in FIG. 8C, when an extremum is extracted within a predetermined time (for example, 50 ms) before the blink start time in the eyelid acceleration data, the subsequent blink is an irregular blink. It may be distinguished from the eyes.

  Although one embodiment of the present invention has been described above, the present invention is not limited to the above embodiment. For example, in the above embodiment, an imaging device (IVS camera) having a frame rate of 1 kHz or more is used as the imaging means, but an area sensor or a line sensor may be used. When using a line sensor, in order to detect a change in the position of the eyelid accompanying blinking, the line sensor may be installed so that the longitudinal direction of the line sensor is along the opening / closing direction of the eyelid.

Further, the following feature amount may be calculated.
1: Time interval between maximum speed and acceleration, minimum value 2: Speed, maximum acceleration, difference in minimum speed 3: Speed, maximum acceleration, speed ratio of minimum value 4: Maximum acceleration, half of minimum value Value range 5: Average of all calculated speeds, acceleration maxima, minimum time interval 6: All calculated speeds, acceleration maxima, standard deviation of minimum time interval 7: All calculated speeds , Average of the ratio of the time interval between the local maximum of the acceleration and the local minimum 8: the standard deviation of the ratio of all the calculated speeds, the local maximum of the acceleration, the local interval of the local minimum 9: the local maximum and the minimum value of all the calculated accelerations Average ratio of half-value widths of 10: Maximum deviation of all calculated accelerations, standard deviation of ratio of half-value widths of local minimum values 11: All of speed, acceleration maximum and minimum values confirmed in the entire blink interval Centroid time 12 at the time of the time until the eye-opening Period of

Further, a threshold value may be set for the eyelid acceleration data, and the following feature amount may be calculated.
1: Number of times the change in acceleration is equal to or greater than the threshold 2: Period in which the change in acceleration continuously exceeds the threshold 3: Number of intersections between the change in acceleration and the threshold 4: Change in acceleration is continuously below the threshold Number of times

Further, for example, the following information may be calculated using a blink ideal curve as shown in FIG.
1: RMS value of extracted blink and blink ideal curve 2: RMS value of extracted blink blink speed change and blink ideal curve speed change 3: Extracted blink blink acceleration change and blink RMS value of acceleration change of ideal curve

  DESCRIPTION OF SYMBOLS 1 ... Blink measuring device, 5 ... Imaging part (imaging means), 7 ... Blink discrimination | determination part (blink discrimination | determination means).

Claims (4)

An imaging means for imaging the eye of the measurement subject;
Based on the eye image picked up by the image pickup means, three or more true extreme values appear in the time change of the eyelid acceleration that closes and opens in one blink by the measurement subject. Eyeblink discriminating means for discriminating the eyeblink from an irregular eyeblink,
The blink determination means obtains eyelid acceleration data indicating a temporal change in acceleration of the eyelid that closes and opens in one blink, and performs one blink on the basis of the eyelid acceleration data. The extreme values appearing in the time change of the acceleration of the eyelids that are closed and opened at the time of extraction, and for each of the target points extracted as the extreme values, a difference in acceleration from the preceding and subsequent target points is calculated, When the extreme value other than the false extreme value where both of the absolute values of the differences are equal to or less than a predetermined threshold value is determined as a true extreme value, and the true extreme value having the same polarity appears continuously by the determination, The true extremum is extracted by selecting one of the continuous true extremums that has the larger absolute value of acceleration, and when the three or more true extremums are extracted, the blink Blink measurement characterized by discriminating from the irregular blink Location.   The blink determination unit obtains eyelid position data indicating temporal changes in the position of the eyelid that closes and opens in one blink, and based on the eyelid position data, 2. The blink according to claim 1, wherein the blink is determined to be the irregular blink when three or more extreme values appearing in the time change of the position of the eyelid that is closed and opened are extracted. Measuring device.   The blink determination means obtains eyelid speed data indicating a temporal change in the speed of the eyelid that closes and opens in one blink, and based on the eyelid speed data, one blink The eyeblink is determined to be the irregular eyeblink when three or more extreme values appearing in the temporal change in the speed of the eyelid closing and opening are extracted. Blink measurement device. An imaging step in which the imaging means images the eye of the person to be measured;
Based on the image of the eye picked up in the image pickup step, the blink determination means has a true extreme value in the temporal change in the acceleration of the eyelid closed and opened by the measurement subject in one blink. A blink discriminating step for discriminating from the blink that appears 3 or more points as an irregular blink,
In the blink determination step, the blink determination means acquires eyelid acceleration data indicating temporal changes in the acceleration of the eyelid that closes and opens in one blink, and based on the eyelid acceleration data Extracting extreme values appearing in the temporal change in acceleration of the eyelids that close and open in one blink, and the difference in acceleration from the preceding and following target points for each of the target points extracted as the extreme values And determining the extreme value other than the false extreme value for which both absolute values of the difference in acceleration are equal to or less than a predetermined threshold as a true extreme value, and the true extreme value having the same polarity is determined by the determination. When appearing continuously, the true extremum is extracted by selecting the true extremum having the larger absolute value of the true extremums, and three or more true extremums are extracted. The blink may be identified as the irregular blink. Eyeblink measurement method according to claim.

Images