JP2017176302A - Eyeball movement measuring apparatus, eyeball movement measuring method, and eyeball movement measuring program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an eyeball movement measuring apparatus and measuring method capable of detecting an initial motion time of the eyeball movement even from an image photographed with a low-pixel camera or a rough image in which the camera does not take close-up of a black eye.SOLUTION: The eyeball movement measuring apparatus includes: index display means for displaying an index on a display; eyeball photographing means for photographing an eyeball of a subject until a prescribed time elapses from a time when at least the index is displayed, to acquire eyeball images; black-eye area detection means for detecting black-eye areas in respective image frames within a photographing time; eyeball movement termination calculating means for calculating an eyeball movement termination time by binary clustering differential values of movements of the black-eye area between the image frames; and an initial motion time calculation means for calculating a time difference between the index display time and the eyeball movement termination time, as an initial motion time of the eyeball movement. Further, the eyeball movement measuring apparatus includes reminder means, more specifically a buzzer sound output device, for stimulating a visual sense, an auditory sense, or a tactile sense for notifying a subject of timing of displaying the index.SELECTED DRAWING: Figure 9

Description

  The present invention relates to a technique for measuring an initial movement time of eye movement from a face image captured by a camera.

In recent years, an aging society has been progressing rapidly, and various diseases have become social problems. One of the diseases is dementia, the number of which is estimated to be 4.4 million in Japan in 2010, and is said to increase to nearly double in the next 10 years.
Dementia is a state in which intelligence once developed has been reduced due to an organic disorder of the brain, and has two symptoms: a core symptom that is said to be caused by brain damage, and a peripheral symptom that is said to be caused by anxiety and resentment due to the disorder. . Specific symptoms include memory impairments such as repeating the same thing over and over in core symptoms, disorientation such as getting lost in the way you should know, aphasia that forgets the names of things, There are disapprovals that forget what the object is, and apraxia that forgets how to use the object, and peripheral symptoms include hallucinations, delusions, excitement, and depression.
However, about 80% of patients diagnosed with dementia are reported to have overlooked dementia by their primary doctors, and it is difficult to say that early detection has been achieved.

The current test for dementia is said to be important to interview the person and family and to observe the patient at that time. Dementia tests such as HDS-R (Revised Hasegawa Simple Intelligence Evaluation Scale) and MMSE (Mini-Mental State Examination), MRI (Magnetic Resonance Imaging) and PET (Positron Emission Tomography) There are also objective diagnostic methods such as brain image diagnosis.
However, these methods have problems such as the need for specialists, time and labor. And the biggest problem is that one of the major features of the disease called dementia is the symptom that meta-memory is invading and the patient is not aware that he / she is ill. There is a problem that early detection is not possible.
Therefore, it is expected that dementia can be detected early if there is something that can be easily measured to measure body temperature and blood pressure, or something that can be measured naturally without being conscious.

As a method for easily screening for brain dysfunction such as dementia, research is being conducted to detect dementia by analyzing facial expressions, voice, and blood. Facial expressions and voices can be measured in daily life, which is a handy method, but is easily influenced by the subject becoming aware of the examination. In addition, blood is often collected at a medical checkup, so early detection can be expected, but it cannot be easily measured in daily life.
On the other hand, eye movements are suitable for examination because they are not easily affected by being aware that they are examinations, and can be easily measured in daily life. And it is known that patients with dementia have a prolonged initial movement time of eye movements compared to healthy individuals (see Non-Patent Document 1), and early detection of dementia is possible by measuring eye movements. It is believed that.
In this specification, the initial movement time of the eye movement is defined as the time from when the visual target appears until the eye catches the visual target and stops moving. In the following, the term “initial movement time” simply refers to the initial movement time of eye movement.

As a technique for measuring eye movement, for example, a method for measuring sliding eye movement using a goggle-type target presentation device is known (for example, see Non-Patent Document 1). This method is carried out on 19 dementia patients, 18 healthy elderly people, and 7 healthy young people, and knowledge such as extension of initial movement time in dementia patients has been obtained.
However, the method for measuring eye movement disclosed in Non-Patent Document 1 requires a large-scale device and is difficult to measure on a daily basis.

In a method for measuring eye movement, an eye tracking system is known as a technique using a small device (see, for example, Non-Patent Document 2). According to this, since the movement of the eyeball can be obtained with high accuracy and information such as where the user is looking is known, it is possible to check gaze information and the like.
However, the eye tracking system disclosed in Non-Patent Document 2 requires calibration in order to measure the line of sight, and there is a problem that a calibration error leads to a subsequent measurement error.

Further, an eye movement analysis system is known as an apparatus that grasps the relative relationship between the eyeball position and the target position, and provides auxiliary data for determining disease differentiation, severity, etc. (Patent Literature) 1). In the eye movement analysis system of Patent Document 1, as in Non-Patent Document 1, the movement of the eyeball following the target displayed on the display of the goggles type headset is displayed in front of it using the goggle type target presentation device. The eyeball image captured by the eyeball imaging device that is reflected by the arranged half mirror and captured by the imaging camera is subjected to image processing. Based on the eye movement data and the movement data of the target, the relationship is time on the horizontal axis and the vertical axis Is output as a tracking error display graph for displaying the deviation between the target position and the eyeball position.
The goggle-type headset used in the eye movement analysis system of Patent Document 1 is a spectacle-type compact device, and a screen that projects a visual target without fixing the head by mounting the goggle-type headset on the head. The eyeball can be imaged in a state where the head is stationary with respect to the imaging camera that images the eyeball with the head stationary relative to the display, etc., and an accurate target tracking function test can be performed. . That is, even when the head moves slightly, it is possible to image the eye movement following the target without causing image blur due to head movement.

  However, in this eye movement analysis system, since the disease is determined by calculating the deviation between the eye position and the target position, the eye position needs to be determined using calibration data that has been normalized in advance by the amount of movement of the target. There is. That is, as in Non-Patent Document 2, it is necessary to calibrate prior to measurement, and the standard eyeball from the center (reference position) of the display facing the visual axis of the standard eyeball facing the front. The visual axis is moved to a position where the visual axis is inclined at 30 ° to the left and right, and based on the eyeball images at the start and end points, the amount of eyeball movement on the image is normalized by the amount of movement of the target to calculate calibration data. There is a need.

  In recent years, home medical care systems in which doctors and patients are linked with camera images have been proposed. This is because the patient uses a video call to consult with a doctor while the patient is at home, but the camera used in such a system is usually low pixel. In many cases, it is difficult to calibrate a patient in advance or to wear a goggles headset on the head.

JP 2004-337348 A

Ichiro Fukumoto et al. Basic study of a simple diagnosis system for dementia using eye movement measurement. Nagaoka University of Technology research report, 2002. T Nakano et al. Atypical gaze patterns in children and adults with autism spectrum disorders dissociated from developmental changes in gaze behavior.Proceedings of the Royal Society B, 2010.

  Conventionally, when the initial movement time is measured by detecting the position of the black eye (circular black part in the center of the eyeball) for each frame with the camera, the time when the movement amount of the black eye position between frames appears, that is, the movement amount The time at which the eye movement increases and the time at which the movement amount of the position of the black eye between the frames disappears are obtained from the difference between the movement amounts, and the initial movement time of the eye movement is obtained. For this reason, in the case of a low pixel camera built in a personal computer (PC), or when the measured black eye is far from the camera position, the detection accuracy of the black eye position is rough, which may cause noise. It was buried and could not be measured correctly only by the difference in the amount of movement of the position of the black eye.

  In view of such a situation, the present invention provides an eye movement measurement device capable of detecting the initial movement time of an eye movement even from an image taken with a low-pixel camera or a rough image in which the camera does not take a close-up of the black eye. An object is to provide a measurement method.

In view of the above situation, the eye movement measurement apparatus of the present invention includes the following 1) to 5).
1) Index display means for displaying an index on the display 2) Eyeball imaging means for acquiring an eyeball image by photographing the eyeball of the subject until at least a predetermined time has elapsed from the time when the index is displayed. Black eye area detection means for detecting a black eye area in an image frame 4) Eye movement end calculation means for calculating an eye movement end time by binarizing and clustering the differential value of the movement amount of the black eye area between image frames 5) Index Initial movement time calculation means for calculating the time difference between the display time and the end of eye movement as the initial movement time of the eye movement

  According to the eye movement measurement device having the above configuration, even when the image taken by the camera is a rough image, the initial movement time of the eye movement can be calculated. Eye movement can be measured with a simple device such as a CCD (Charge Coupled Device) camera and a PC. Since the movement of the eyeball means the movement of the black eye, in order to detect the movement of the eyeball from the video, the movement of the black eye is measured. Here, the subject is not limited to a human and includes an animal that performs eye movement.

  The above 2) “until a predetermined time has passed since the time when the index is displayed” means that the subject recognizes the presence of the index from the timing when the index is displayed on the display, and the black eye moves to match the line of sight with the index However, the movement of the black eye ends when the line of sight matches the position of the index, but it means that a sufficient time has elapsed after the movement of the black eye is ended. Usually, the initial movement time of the human eye movement is within one second, so the predetermined time may be set to several seconds. At least the time when the index is displayed means that the eyeball of the subject may be photographed before the time when the index is displayed.

As a method of detecting the black eye region in each image frame within the imaging time by the black eye region detecting means in 3) above, a known method can be applied. For example, the method of Zhao et al. Can be preferably used. Zhao et al.'S method (see “Zijing Zhao et al. An accurate iris segmentation framework under relaxed imaging constraints using total variation model. In ICCV, 2015”) detects the eye model based on an elliptical fit. A model-based method is used, and black eyes can be detected with high accuracy. The main flow of Zhao et al.'S method is to input an image cropped around the eyes, perform preprocessing such as noise removal, perform smoothing to avoid the effects of eyelashes, wrinkles, etc., and then apply circular fitting Thus, the black eye is detected and post-processing such as wrinkle detection is performed.
Here, since the measurement time is not long and the subject is gazing at the target, assuming that there is no large facial movement such as a change in face orientation during the measurement period, the subject is affected by the large facial movement. The movement of the pure eyeball is not represented by changes in the coordinates of the black eye in the video. Since the black eye coordinates in the video can be obtained by cropping the video in the area around the eyes, the area around the eyes can be detected.

  The eye movement end calculation means 4) is a feature of the present invention, and calculates the eye movement end time by binarizing and clustering the differential value of the movement amount of the black eye region between image frames. The initial movement time of the eye movement is from the time the index appears until the eye stops catching the index (the line of sight matches the index), so if you find the time that the eye moves and stops from the time series change of the movement amount of the black eye area It will be good. What is necessary is to obtain a point where the position of the black eye becomes constant, but there is a blurring of the eyeball called fixation fine movement, and the position of the black eye includes noise and the like. Therefore, since the eye movement to be measured occurs in a very short time, it is mixed with noise and cannot be detected well. Therefore, paying attention to the fact that the eye movement within the measurement section changes to static, moving, and static, and the differential value of the movement amount of the black eye region is binarized and clustered, and when the eye movement becomes stationary, moving, and stationary Is calculated.

In the eye movement end calculation means of the eye movement measuring apparatus of the present invention, the absolute value of the differential value of the movement amount of the black eye region is large from the start of the black eye movement to the end of the black eye movement, before the start of the black eye movement and after the end of the black eye movement. Therefore, when calculating the threshold for dividing the two large and small classes into three in time series, the variance between the classes of the two large and small classes is calculated, the threshold is optimized so that the variance is maximized, and the black eye movement It is preferable to calculate the end point.
As described above, the differential value of the movement amount of the black eye region is binarized and clustered in order to calculate the time point when the eyeball movement is stationary, moving, and stationary. The absolute value of the differential value increases from the start time of the black eye operation to the end time of the black eye operation, and the absolute value of the differential value decreases before the black eye operation starts and after the black eye operation ends. Therefore, by calculating a threshold value that divides the two classes of the differential value having a large absolute value and the small value into three in time series, it is possible to calculate the time point when the eyeball movement is stationary, moving, and stationary. . Specifically, the variances between the two classes of the absolute value of the differential value are large and small, and the above threshold value is optimized so that the variance is maximized. As the threshold values, two threshold values are obtained: a threshold value for moving the eyeball from the stationary state and a threshold value for moving the eyeball from the stationary state. Obtaining the threshold value for stopping from the movement calculates the end point of the black eye movement.

In the eye movement measurement apparatus of the present invention, the movement amount of the black eye region is preferably the movement amount of the black eye centroid coordinates in the centroid coordinate system with the eye feature points as reference coordinates.
As described above, the movement of the eyeball means the movement of the black eye. Therefore, in order to detect the movement of the eyeball from the video, it is necessary to measure the movement of the black eye. The measurement time is short, and the subject gazes at the target and performs the measurement, so there is no large facial movement such as a change of face orientation during the measurement period, but there is actually a small facial movement Therefore, if the coordinates of the black eye in the video are simply obtained, the eye movement feature amount is erroneously detected due to the minute movement of the face. Therefore, instead of using the coordinates of the black eyes in the coordinates in the video as they are, the movement amount is measured by providing a reference point whose positional relationship with the black eyes does not change due to a minute facial movement.

In the eye movement measurement apparatus of the present invention, it is preferable to use only the movement amount of the black eye region in the horizontal direction as the movement amount of the black eye region.
When moving a human eyeball horizontally, the inner and outer rectus muscles are moved like a pulley. On the other hand, when moving the human eyeball in the vertical direction, the upper and lower rectus muscles are moved like a pulley. The upper centers of motor nerves that control these muscles are located in the frontal lobe in impulsive eye movements, in the occipital lobe in sliding eye movements, and in frontal and occipital lobes in dementia. In addition, in the case of elderly people, it is difficult to measure the eye movement in the vertical direction because the eyelid muscles weaken and the exposure of black eyes is extremely small even when the eyes are open. Therefore, only the movement amount of the black eye region in the horizontal direction is used.

The eye movement measurement device of the present invention preferably further includes a warning means for stimulating visual, auditory, or tactile sensation to inform the subject of the timing for displaying the index.
By providing the alerting means, the subject is alerted to the display of the index, and tries to align the line of sight with the index, thereby enabling more smooth eye movement measurement.

  The alerting means in the eye movement measuring device of the present invention is preferably a buzzer sound output device synchronized with the index display means. In the above alerting means, a buzzer sound is output simultaneously with the indicator display. For example, it is assumed that there is a buzzer sound output device connected to a PC via a USB (Universal Serial Bus), and a program for buzzer output at the same time that an index is displayed on the display of the PC is installed. Strictly speaking, there is a slight time difference between the index display time and the buzzer sound output time, but it is negligible in measurement.

Next, the eye movement measurement method of the present invention will be described.
The eye movement measurement method of the present invention includes the following a) to e).
a) An index display step for displaying an index on a display b) An eyeball imaging step for acquiring an eyeball image by capturing the eyeball of a subject until at least a predetermined time has elapsed from the time when the index is displayed c) Black eye area detection step for detecting a black eye area in an image frame d) Eye movement end calculation step for calculating an eye movement end time by binarizing and clustering the differential value of the movement amount of the black eye area between image frames e) Index Initial movement time calculation step for calculating the time difference between the display time and the end of eye movement as the initial movement time of eye movement

  In the eye movement measurement method of the present invention, in the eye movement end calculation step, the absolute value of the differential value of the movement amount of the black eye region is large from the start of the black eye movement to the end of the black eye movement, and before the black eye movement start and the black eye movement end. Since the latter is small, when calculating the threshold to divide the two large and small classes into three in time series, the variance between the classes of the large and small classes is calculated, the threshold is optimized so that the variance is maximized, It is preferable to calculate the operation end point.

In the eye movement measurement method of the present invention, the movement amount of the black eye region is preferably the movement amount of the black eye centroid coordinates in the centroid coordinate system using the eye feature points as reference coordinates.
Further, it is preferable to use only the movement amount of the black eye region in the horizontal direction as the movement amount of the black eye region.

  The eye movement measurement method of the present invention further includes a warning step for stimulating visual, auditory, or tactile sensation to inform the subject of the timing for displaying the index in synchronization with the index display step. preferable. Specifically, the alerting step outputs a buzzer sound in synchronization with the index display step.

The eye movement measurement program of the present invention is a program that causes a computer to execute the steps a) to e) of the above-described eye movement measurement method.
Moreover, the eye movement measurement apparatus of this invention is equipped with the computer and display which mounted said eye movement measurement program.

The brain dysfunction screening method of the present invention comprises the steps in the above-described eye movement measurement method and a determination step for determining brain dysfunction.
The brain dysfunction screening apparatus of the present invention includes the above-described eye movement measurement device and a determination unit that determines brain dysfunction.
Various studies have been conducted on the effects of dementia on gaze. Actually, patients with dementia have been reported to extend the initial movement time of eye movements compared to healthy individuals. From this, there is a possibility of early detection of dementia by measuring the initial movement time of eye movement, and the above-described eye movement measurement method can be used as a brain dysfunction screening method. Here, the determination step of determining the brain dysfunction is to determine that there is a possibility of cerebral dysfunction when, for example, the initial movement time of the eye movement exceeds a predetermined threshold time. Or delay in visual recognition time, extension of internal / external rectus muscle switching time, decrease in maximum speed during tracking, decrease in tracking gain, catch-up saccade (eye movement that occurs when chasing moving objects with eyes, delay, etc. Judgment of cerebral dysfunction by combining measurement results such as an increase in eye movement that reflexively jumps and catches up with moving objects when it cannot be tracked, an increase in the number of saccades during gaze, and a decrease in saccade amplitude You can also.

  According to the eye movement measurement method and measurement system of the present invention, it is possible to detect the initial movement time of the eye movement even from an image taken with a low pixel camera or a rough image in which the camera is not close-up. There is. Therefore, early detection of dementia can be expected using a video image in a state where a video call is made via a network.

Illustration of initial action time It is explanatory drawing of the muscular structure of an eye, (1) is a front view, (2) is a left view, (3) has shown the correlation diagram of the muscle which is related to a moving direction. Flow diagram of eye movement measurement method 1 Explanatory diagram of black eye center of gravity coordinates It is a calculation method of initial movement time, (1) shows the eyeball movement in the section, and (2) shows the differential value in the section. It is an experimental result of evaluation experiment 1, (1) has shown the error value of initial motion time, (2) has shown the measured value of initial motion time. It is an experimental result of the evaluation experiment 2, (1) shows the error value of the initial movement time, (2) shows the measured value of the initial movement time, and (3) shows the error value of the initial movement time excluding the outlier of the initial movement time. . It is an image figure of an optotype following experiment, (1) shows before an optotype display, and (2) shows after an optotype display. Functional block diagram of eye movement measurement device Flow diagram of eye movement measurement method 2

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The scope of the present invention is not limited to the following examples and illustrated examples, and many changes and modifications can be made.

First, an aspect of the eye movement measurement device will be described. FIG. 9 shows a functional block diagram of the eye movement measuring apparatus.
As shown in FIG. 9, the eye movement measurement device includes a display 1, an index display unit 6, an eyeball photographing unit 7, a black eye region detection unit 8, an eye movement end calculation unit 9, an initial movement time calculation unit 10, and an attention drawing unit 11. Composed. An index is displayed on the display 1 by the index display means 6. The alerting means 11 is specifically a buzzer sound output device 12 and outputs a buzzer sound to alert the subject. The eyeball photographing means 7 takes a video of the eyeball of the subject who sees the index displayed by the index display means 6. The black eye area detection means 8 detects a black eye area in each image frame within the imaging time. The eye movement end calculation means 9 calculates the eye movement end time by binarizing and clustering the differential value of the movement amount of the black eye region between image frames. The initial movement time calculation means 10 calculates a time difference between the index display time and the eye movement end time as the initial movement time of the eye movement.

Next, an eye movement measurement method will be described.
Eye movements can be broadly divided into impulse eye movements and sliding eye movements. The impulsive eye movement is a high-speed eye movement called saccade, which is seen when a gazing point is switched to a place where a person is willing to watch. Sliding eye movement is a movement that occurs when a moving object is chased by the eye, and is unique to primates with a developed visual field.

FIG. 1 shows an explanatory diagram of the initial movement time.
Here, since the image of the target appears and moves is captured, as shown in FIG. 1, in the section where the target appears in the moving image, the center of gravity coordinates of the black eye, which has been kept constant as shown in FIG. However, when the target appears at t ₀ , the movement starts with a slight delay and becomes constant when the target is captured. At this time, the time from the appearance of the visual target until the eye catches the visual target and stops moving is defined as the initial movement time of the eye movement.

  FIG. 8 is an image diagram of the target tracking experiment, where (1) shows before the target display and (2) shows after the target display. As shown in FIG. 8 (1), the target 2 does not appear on the display 1 before the target is displayed. As shown in FIG. 8 (2), the target 1 is displayed after the target is displayed. The target 2 appears above. The black-eye center-of-gravity coordinates start moving a little after the optotype 2 appears, and becomes constant when the optotype 2 is captured.

2A and 2B are explanatory diagrams of the muscular structure of the eye, in which (1) is a front view, (2) is a left side view, and (3) is a correlation diagram of the muscles involved in the moving direction.
As shown in FIG. 2, when moving the human eyeball in the horizontal direction, the inner and outer rectus muscles are moved like a pulley. On the other hand, when moving the human eyeball in the vertical direction, the upper and lower rectus muscles are moved like a pulley. The upper centers of motor nerves that control these muscles are located in the frontal lobe in impulsive eye movements, in the occipital lobe in sliding eye movements, and in frontal and occipital lobes in dementia. In addition, in the case of elderly people, it is difficult to measure the eye movement in the vertical direction because the eyelid muscles weaken and the exposure of black eyes is extremely small even when the eyes are open. From the above, in the measurement, the eye movement only in the horizontal direction is measured.

FIG. 3 shows a flowchart of the eye movement measurement method.
As shown in FIG. 3, the eye movement measurement method is roughly divided into three stages, and is a batch process in which a captured moving image is input. First, detection of the coordinates of the eye edge serving as a reference point and feature point detection (Process 1) for cropping the eye region in order to detect black eye coordinates in the video, and then the black eye in the video from the cropped eye image It consists of three parts: black eye detection (Process 2) for detecting coordinates, and eye movement function feature quantity measurement (Process 3) for measuring initial movement time from the obtained black eye barycentric coordinates.

  First, a moving image obtained by taking a video of the subject's face is input. First, in Process 1, face detection is performed from the captured frame image (S01), and feature points such as eyes and nose in the face are detected (S02). Then, an erroneous detection process is performed (S03), and a smoothing process is performed (S04). A crop image group of eyes cropped from each frame image is acquired (S05). On the other hand, the detected feature point coordinates of the eyes are output (S06). Next, in Process 2, black eye detection is performed from each image of the eye crop image group (S07), and black eye coordinates are detected (S08). Then, the initial movement time of eye movement is calculated in Process 3 from the black eye coordinates detected in Process 2 and the feature point coordinates obtained in Process 1. The calculated initial movement time of the eye movement is an eyeball function feature amount.

In Process 1, in order to detect feature points (S02), feature points were detected using a technique called ASM (Active Shape Model) that models and detects feature points such as eyes and noses in the face.
Since ASM cannot detect feature points well unless a rough position of the face is given, ASM is applied after performing face detection from the image with a face detector. Further, since the black-eye center-of-gravity coordinates are obtained by a relative position to the coordinates of the eye end obtained by ASM, there is a possibility that saccades and the like are erroneously detected due to ASM detection errors. Therefore, smoothing processing is performed by moving average using the coordinates of feature points of a certain number of frames before and after all frames in the moving image.
As described above, the detection of the coordinates of the edge of the eye serving as the reference point and the detection of the feature point for cropping the eye region are performed.

In Process 2, black eye detection is performed by the following method.
First, an image obtained by cropping the periphery of the eye is input, and after performing preprocessing such as noise removal, smoothing processing is performed to avoid the influence of eyelashes and wrinkles. Thereafter, black eyes are detected by circular fitting, and post-processing such as wrinkle detection is performed.

In Process 3, the initial motion time is calculated based on the time series change of the black eye centroid coordinates. Hereinafter, a method for obtaining the center of gravity coordinates of the black eye will be described.
Since the movement of the eyeball means the movement of the black eye, it is necessary to measure the movement of the black eye in order to detect the movement of the eyeball from the video. The measurement time is short, and the subject gazes at the target and performs the measurement, so there is no large facial movement such as a change of face orientation during the measurement period, but there is actually a small facial movement Therefore, if the coordinates of the black eye in the video are simply obtained, the eye movement feature amount is erroneously detected due to the minute movement of the face. Therefore, instead of using the coordinates of the black eye in the coordinates in the video as they are, a reference point is provided in which the positional relationship with the black eye does not change due to a minute facial movement.

  FIG. 10 shows a flowchart of the eye movement measurement method. First, an index is displayed on the display (index display step; S11). The eyeball image of the subject is photographed to obtain an eyeball image until a predetermined time has elapsed from the time when the index is displayed (eyeball photographing step; S12). Further, a black eye area in each image frame within the imaging time is detected (black eye area detection step; S13). Then, the differential value of the movement amount of the black eye region between the image frames is binarized and clustered to calculate the eyeball motion end time (eyeball motion end calculation step; S14). Finally, the time difference between the index display time and the eye movement end time is calculated as the initial movement time of the eye movement (initial movement time calculation step; S15).

  FIG. 4 is an explanatory diagram of the black-eye centroid coordinates. As shown in FIG. 4, the coordinates of the inner end 3b of the eye 3 are used as a reference point. The reason why the coordinates of the inner end 3b instead of the outer end 3a of the eye 3 are used as reference points is that errors are less likely to occur. Then, the distance between the center of the black eye 4 and the coordinate of the inner end 3b of the eye 3 is used as the black eye center-of-gravity coordinate 5 for measurement of the eye movement function feature amount.

The initial movement time is the time from when an object moves to when the eye catches the object and stops. Therefore, it can be said that the initial movement time may be calculated by calculating the time when the eye stops moving from the time-series change of the black eye centroid coordinates.
However, in the method of finding the point where the center of gravity of the black eye is constant, the coordinates of the center of gravity of the black eye include noise, etc., and the impulsive eye movement to be measured occurs in a very short time. I can't.

Therefore, the following initial movement time calculation method is used. FIG. 5 shows a calculation method of the initial movement time, where (1) represents eye movement within the section, the horizontal axis represents time, and the vertical axis represents black eye barycentric coordinates. Further, (2) represents a differential value in the section, the horizontal axis represents time, and the vertical axis represents the differential value of the black-eye centroid coordinates.
As shown in FIG. 5 (1), the eyeball movement within the measurement section changes between stationary, moving, and stationary. At this time, the absolute value of the differential value of the center of gravity coordinates of the black eye changes as small, large, and small. Therefore, as shown in FIG. 5 (2), the section is divided into two classes, a class having a small differential value and a class having a large differential value. Can do.

Here, as shown in FIG. 5 (1) and (2), the time _{t end} time _{t the begin} and iris began motion of the iris has finished the motion, as a threshold for separating the two classes into three, by discriminant analysis Like the binarization of the image, the variance between the two classes represented by the following formula 1 is obtained, and the two threshold values are optimized so that this is maximized. Time), and the initial movement time of the eye movement is obtained by calculating the difference from the time when the target appears (index display time). The interclass variance σ ² is x for the center of gravity coordinates of the black eye, ω _{low for} the number of frames of the class with a small differential value, m _{low for} the average of the differential value, ω _{high for} the number of frames of the class with a large differential value, and the average of the differential value. _Assuming m _high , each is represented by the following formula 1 (in formula 1, n is a constant).

(Evaluation Experiment 1)
The subjects were 13 healthy men in their 20s and 3 women. The experiment was conducted in a quiet closed room where there was nothing to draw attention other than the terminal used for the experiment.
In addition, among the subjects, there were six people who regularly used glasses, and measurements were performed with the glasses removed if possible.

  As an experiment content, a target tracking experiment was conducted. Specifically, first, ask the subject to sit in front of the tablet terminal and look at the screen. At this time, an image of a black circle is displayed on the screen, and this moves from end to end of the screen. The face of the subject at this time was photographed with the camera of the tablet terminal. In this experiment, it was only necessary to see a black circle, and in order to prevent measurement errors due to reflection of glasses, etc., during the experiment, measurements were taken with glasses removed if possible, and experiments were performed with glasses worn. There were two people.

  The face position was set to 210 mm from the screen. In addition, two types of target speeds were provided, and the experiment was performed with 2.6 pixels / frame as Slow and 5.2 pixels / frame as Fast. The screen drawing is 60 fps. Assuming that one target is from the disappearance of the target once to the end of the movement, one sequence is taken in Slow, and 14 sequences are taken in Fast to unify the moving time of the target.

FIG. 6 shows the experimental results of the evaluation experiment 1, where (1) shows the error value of the initial movement time, and (2) shows the measurement value of the initial movement time.
From FIG. 6B, the initial movement time of the eye movement is constant and is about 0.3 seconds regardless of the target speed. Further, as shown in FIG. 6A, the error is less than one frame (0.033 seconds). According to Non-Patent Document 1, the accuracy required to discriminate between a dementia patient and a healthy person may be 0.065 seconds or less, so it can be said that the accuracy of the experimental results of this evaluation experiment was obtained. .

(Evaluation Experiment 2)
The subjects were 13 healthy men in their 20s and 3 women. The experiment was conducted in a quiet closed room where there was nothing to draw attention other than the terminal used for the experiment.
In addition, among the test subjects, there were six people who regularly used glasses, and the measurement was performed while wearing glasses. Other conditions that are not particularly described are the same as those in the evaluation experiment 1.

  The experiment was conducted assuming that it was used daily. In an evaluation experiment that assumes daily use, it is assumed that you are using an application that displays received information such as e-mail and news in a pop-up on the screen, such as push notification of a news application on a smartphone. An experiment was conducted to actually read the news of interest. For the selected news, the news published on the experimental day was selected so that it could be read with a certain degree of interest in the same age as the test subject, avoiding those that differed greatly in interest by readers such as politics and faith.

Specifically, as in the evaluation experiment 1, the test subject asks the subject to sit in front of the tablet terminal and see the screen displayed on the display. At this time, the weather forecast of the website is displayed on the screen by the browser. After a certain period of time, another news title pops up on the edge of the screen, so you have to tap the pop-up. When you tap it, the news will be displayed on the screen, so I asked you to read the displayed news. The face of the subject at this time was photographed with the camera of the tablet terminal.
In addition, in this evaluation experiment, it is necessary to read the text and it is assumed that it will be used in daily life. Therefore, for the six subjects who usually wear spectacles, measurements were performed while wearing spectacles. . Only one sequence was taken for each subject.

FIG. 7 shows the experimental results of Evaluation Experiment 2, where (1) is the error value of the initial movement time, (2) is the measured value of the initial movement time, and (3) is the error value of the initial movement time excluding the outliers of the initial movement time. Is shown.
As shown in FIG. 7 (1), the initial movement time is measured with an accuracy of 0.19 seconds, and the accuracy is lower than that of the evaluation experiment 1. However, the experimental results showed a large variance and only some subjects tended to have large errors. Moreover, as shown in FIG. 7 (2), it can be seen that the initial movement time is remarkably increased in some subjects. From these facts, FIG. 7 (3) shows the error value obtained except for the case where the correct label of the initial motion time is longer than the median + 50% range. As shown in FIG. 7 (3), if an extremely large initial movement time is excluded, the error value can be obtained with 0.064 seconds and can be obtained with sufficient accuracy. In the evaluation experiment 1, the subject looks at the target on the screen, and immediately follows the target if the target moves. However, in the evaluation experiment 2, since the subject looks at the object freely at first, the reaction is delayed because of focusing on another object or moving the eyes.

  INDUSTRIAL APPLICABILITY The present invention can be used in screening apparatuses and methods for detecting dementia, reducing cognitive function ability due to depression, the presence of diseases of the eyeball and surrounding muscles, and brain function damage.

DESCRIPTION OF SYMBOLS 1 Display 2 Target 3 Eye 3a Outer edge part 3b Inner edge part 4 Black eye 5 Black eye center-of-gravity coordinate 6 Index display means 7 Eyeball imaging means 8 Black eye area detection means 9 Eyeball movement end calculation means 10 Initial motion time calculation means 11 Attention means 12 Buzzer sound output device

Claims (16)

An indicator display means for displaying the indicator on the display;
Eyeball photographing means for photographing an eyeball of a subject and acquiring an eyeball image until at least a predetermined time has elapsed from the time when the index is displayed; a black eye area detecting means for detecting a black eye area in each image frame within the imaging time; By performing binarization clustering on the differential value of the amount of movement of the black eye region between image frames, eye movement end calculation means for calculating the eye movement end time, and the time difference between the index display time and the eye movement end time An eye movement measuring device comprising: initial movement time calculating means for calculating the initial movement time of the first movement time. In the eye movement end calculation means,
Since the absolute value of the differential value of the movement amount of the black eye region is large from the start time of the black eye operation to the end time of the black eye operation, and small before the start of the black eye operation and after the end of the black eye operation, three large and small classes in time series When calculating the threshold value to be divided into two, the variance between classes of two large and small classes is calculated, the threshold value is optimized so as to maximize the variance, and the black eye movement end point is calculated. The eye movement measuring device described.   The eye movement measuring apparatus according to claim 1 or 2, wherein the movement amount of the black eye region is a movement amount of the black eye barycentric coordinate in the barycentric coordinate system having the eye feature point as a reference coordinate.   The eye movement measurement apparatus according to claim 1, wherein only the movement amount of the black eye region in the horizontal direction is used as the movement amount of the black eye region.   The eye movement measurement according to any one of claims 1 to 4, further comprising an alerting means for stimulating visual, auditory, or tactile sensation for informing a subject of timing for displaying the indicator. apparatus.   6. The eye movement measuring device according to claim 5, wherein the alerting means is a buzzer sound output device synchronized with the indicator display means. An indicator display step for displaying the indicator on the display;
An eyeball photographing step of capturing an eyeball image by capturing the eyeball of the subject until at least a predetermined time has elapsed from the time when the index is displayed; a black eye region detecting step of detecting a black eye region in each image frame within the imaging time; By performing binarization clustering on the differential value of the movement amount of the black eye region between image frames, the eye movement end calculation step for calculating the eye movement end time, and the time difference between the index display time and the eye movement end time An eye movement measurement method comprising: an initial movement time calculating step for calculating the initial movement time of the first movement time. In the eye movement end calculation step,
Since the absolute value of the differential value of the movement amount of the black eye region is large from the start time of the black eye operation to the end time of the black eye operation, and small before the start of the black eye operation and after the end of the black eye operation, three large and small classes in time series The threshold value to be divided into two types is calculated by calculating a variance between two classes of large and small classes, optimizing the threshold value so as to maximize the variance, and calculating a black eye movement end point. The eye movement measurement method described.   9. The eye movement measurement method according to claim 7, wherein the movement amount of the black eye region is a movement amount of the black eye barycentric coordinate in a barycentric coordinate system having the eye feature point as a reference coordinate.   The eye movement measurement method according to claim 7, wherein only the movement amount of the black eye region in the horizontal direction is used as the movement amount of the black eye region.   11. A reminding step for stimulating visual, auditory, or tactile sensation for notifying a subject of timing for displaying the indicator in synchronization with the indicator displaying step is further provided. The eye movement measurement method according to any one of the above.   The eye movement measuring method according to claim 11, wherein the alerting step outputs a buzzer sound in synchronization with the indicator displaying step.   The eye movement measurement program which makes a computer perform each step of the eye movement measurement method in any one of Claims 7-12.   An eye movement measuring apparatus comprising a computer and a display on which the eye movement measuring program according to claim 13 is mounted. Each step in the eye movement measurement method according to any one of claims 7 to 12,
A method for screening cerebral dysfunction, comprising a determination step for determining cerebral dysfunction. The eye movement measurement device according to any one of claims 1 to 6 and 14,
Determination means for determining brain dysfunction;
A cerebral dysfunction screening apparatus comprising: