JP2003319907A - Eye ball movement analysis system and eye ball imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To precisely and easily grasp a correlation between an eye-ball position and a target position by precisely inspecting an eye tracking function without fixing the head. <P>SOLUTION: The movement of eye balls following a target projected on the display (4) of a goggle-type head set (5) is reflected by a half mirror (6) arranged on this side of it and imaged by an imaging camera (7). Then the image of the eye balls is image-processed to read the time change of eye ball positions as eye ball movement data. Then, the eye ball movement data and movement data of the target projected on the display (4) are superposed synchronously and displayed graphically. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an eye movement analysis system for picking up the movement of an eyeball that follows a target displayed on a display and analyzing the movement of the eyeball, and an eyeball image pickup apparatus used therefor.

[0002]

2. Description of the Related Art Eye movement is closely related to the function of the cranial nerves. In particular, the examination of the index tracking function, which is a basic movement function of the eye, is performed in the nervous system such as Parkinson's disease, Alzheimer's disease, spinocerebellar degeneration and dizziness. Used in the diagnosis of disease. In this index tracking function test, a target (index) displayed on a display such as a screen is moved linearly at a constant speed, the movement of the eyeball that follows this target is imaged by a video camera, and the eyeball is based on the image data. It analyzes movement.

For example, a pulse representing the movement start time and movement end time of the target is obtained by imaging the movement of the eyeball when the target is moved from left to right at a constant speed, reading the temporal change of the eyeball position from the eyeball image. The graph is displayed together with the waveform.

According to this, to what extent the movement start time of the eyeball is delayed from the movement start time of the target, whether the eyeball moves linearly from the movement start time of the target to the movement end, or the target stops. After doing, it is possible to read from the graph how the eyeball is moving.

[0005]

However, since the general index tracking function inspection projects the target on the screen in front of the subject, it requires a relatively large-scale projection device and a dedicated inspection room, Regardless of the general hospital, it is a heavy burden for general practitioners to install such equipment.

Further, since a human instinctively changes the direction of the head when following a target projected on a screen in front, an accurate index tracking function test cannot be performed unless the head is fixed. . In this case, it is general to use by pressing the forehead and the chin, but since the head is still not restrained, it is impossible to eliminate the image blur caused by the head movement.

That is, when performing the index tracking function test, it is necessary that the head is stationary with respect to the screen on which the target is projected, and the head is stationary with respect to the imaging camera which images the eyeball. It is necessary to have

If only an eyeball image is to be taken, an eyeball image pickup apparatus of a type in which a goggle-type headset is provided with an image pickup camera facing the eyeball is used to make the head stand still with respect to the image pickup camera. However, in this case, the imaging camera interferes with the view of the screen in front. Moreover, even if the position of the imaging camera is devised so that the screen can be seen, the head cannot be stopped still with respect to the screen on which the target is displayed.

Further, the analysis result is that the temporal change of the eyeball position is graphically displayed together with the movement start time and movement end time of the target, and the cranial nerve disease is diagnosed by the eye movement pattern and the like. Here, in the case of a healthy person, the position of the eyeball does not shift from the position of the target, but in a patient with a cranial nervous system disease, the position shifts. However, in the conventional graph, it is difficult to accurately grasp the relative relationship between the eyeball position and the position of the target, and thus it is difficult to grasp the medical condition in more detail.

For example, even if the target starts to move, it does not follow immediately and after catching up with the target after a time delay occurs,
When the amount of movement of the eyeball overshoots or undershoots the amount of movement of the target, when the eyeball position catches up with the movement of the target, and how much the overshoot amount or undershoot amount is As described above, there is a problem that it is difficult to grasp the relative relationship between the target position and the eyeball position.

Therefore, according to the present invention, firstly, without fixing the head, the head is made to stand still with respect to a display such as a screen showing a target, and the head is also provided for an image pickup camera for picking up an eyeball. By capturing an image of the eyeball in a stationary state, it is possible to perform an accurate index tracking function test, and secondly, to make it possible to grasp the relative relationship between the eyeball position and the target position more accurately and easily. This is a technical issue.

[0012]

In order to solve this problem, the present invention relates to an eyeball image pickup device for picking up the movement of an eyeball that follows a target displayed on a display, and its movement based on the imaged eyeball image. An eye movement analysis system comprising an analyzer for performing analysis, wherein the eyeball imaging device is a goggle type headset provided with the display at a position facing at least one eyeball, and an image of the eyeball facing the display. And a half mirror for reflecting the reflected light of the eyeball by the half mirror, the analyzer, the eyeball image data obtained by the eyeball imaging device based on the eyeball image captured by the eyeball imaging device eye movement data Image reading means for reading the image data, the eye movement data read by the image processing means, and the display. The movement data of the target to be displayed on synchronously overlaid, characterized in that it comprises an analysis graph output unit for displaying the graph.

According to the present invention, the goggle type headset of the eyeball image pickup device is provided with the display for displaying the target and the image pickup camera for picking up an image of the eyeball.
When this goggle type headset is mounted on the head, the display and the imaging camera are fixed to the head. The image pickup camera is attached to, for example, an ear hook frame of a goggle type headset via a mounting bracket so that the position can be adjusted in the XYZ directions and the head can be swung, so that the optical axis can be accurately imaged. Is possible.

If this eyeball imaging device is worn on the head,
Since the head is relatively stationary with respect to the display and the imaging camera, even if the head moves a little, the eye movement that follows the target can be accurately performed without causing the image blur due to the head movement. It is possible to take an image, and it is possible to accurately check the index tracking function of the eyeball.

Here, if a controller for controlling the image displayed on the display is provided and the brightness of the display is adjusted to set the contrast of the image to be high, the target can be targeted regardless of the brightness of the surroundings or even in a patient with weak eyesight. It is possible to display it easily. Further, the controller may freely set the moving speed and direction of the target, and may control the turning on and off of the target. This allows horizontal and vertical index movements when the target is moved in the horizontal and vertical directions, saccades (impulsive eye movements) when the target is moved rapidly, and smooth pursuit (sliding property) when the target is moved slowly. Eye movement) can be observed.

Then, the eye movement data is read from the eyeball image picked up by the eyeball image pickup device by the image processing means, and a graph superposed in synchronization with the movement data of the target moved and displayed on the display is outputted as an analysis result. To be done.

As a result, since the graph lines showing the position-time change of the eyeball position and the target position are displayed in an overlapping manner, the relative relationship between the target position and the eyeball position is determined based on the overlap and the deviation of the two graph lines. It can be more accurately and easily grasped, and therefore, it is possible to provide supplementary data when a doctor comprehensively diagnoses in consideration of other clinical findings.

Further, the analyzer for performing the movement analysis extracts pattern data of the eye movement based on the movement data of the target and the eye movement data, and the pattern data extracted by the pattern data extracting means. If a disease determination unit that compares the eye movement pattern data with the eye movement pattern data registered in advance according to the disease is provided, auxiliary data regarding the presence or absence of the disease and the type of the disease can be obtained.

For example, the measured eye movement is examined, and the error between the target movement and the eye movement and the temporal analysis result of the error (including the velocity and acceleration of the eye movement) and the error between the target movement and the eye movement. Disease specificity of the pattern, the time delay of the movement start time of the eyeball, the eyeball is moving linearly from the movement start time of the target to the movement end, or the movement of the eyeball after the target has stopped, or By reading the nystagmus or the like, extracting the pattern data thereof, and comparing this with the previously registered pattern data of the eye movement, it is possible to trace auxiliary data useful for the discrimination and judgment of the disease.

[0020]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be specifically described below with reference to the drawings. FIG. 1 is an explanatory diagram showing an eye movement analysis system according to the present invention, FIG. 2 is an explanatory diagram showing an eyeball image pickup device, FIG. 3 is an explanatory diagram showing a screen at the time of image pickup, FIG. 4 is a flowchart showing a processing procedure of an analyzer, FIG. 5 is an analysis graph.

An eye movement analysis system 1 shown in FIG. 1 includes an eye image pickup device 2 for picking up an eye movement that follows a target, and a controller for controlling the movement of the target and performing a movement analysis based on the picked-up eye image. Equipped with 3.

The eyeball image pickup device 2 includes a goggle type headset 5 provided with a display 4 for displaying a target at a position facing at least one (right eye) eyeball.
A half mirror 6 that reflects the image of the eyeball facing the display 4 and an image pickup camera 7 that captures the eyeball reflected by the half mirror 6 are provided.

The goggle type headset 5 is a commercially available head mount display Ey manufactured by Olympus Corporation in which individual liquid crystal displays 4 and 4 facing both eyes are formed.
e-Trek (trade name) was used. Further, since the size of the display 4 is fixed, when the headset 5 is worn, the visual axis of the eyeball is observed when following a target moving left and right from the center of the display 4 facing the front-facing eyeball. The distance between the display 4 and the eyeball is set to 23 mm or less by the nose pad 8 so that the inclination angle of 30 ° can be secured in each direction by 30 °.

The image pickup camera 7 is an ultra-compact probe camera with a built-in CCD element, and a lens 9 for capturing an image.
The illumination light emitting section 10 is annularly arranged around the, and the image pickup optical axis Cx is attached to the ear hanging frame 11 of the goggle type headset 5 so as to intersect obliquely with the visual axis Ex of the eyeball facing the front. It is mounted via a mounting bracket 12 so as to be positionally adjustable and swingable in the XYZ directions. Note that the illumination light emitting section 10 may be an optical fiber that guides light such as an external halogen light source even when a small light emitting body such as an LED is provided, and the installation location is not limited as long as the eyeball can be imaged.

The half mirror 6 is arranged so as to cover the entire surface of the display 4 on one side, and the image pickup optical axis Cx of the image pickup camera 7 attached to the ear hanging frame 11.
Is provided so as to be inclined toward the eyeball so that the illumination light emitted from the illumination light emitting section 10 is reflected by the half mirror 6 to illuminate the eyeball.

The display 4 and the image pickup camera 7 of the eyeball image pickup device 2 are connected to the controller 3 formed of a personal computer, and from the measurement start time Ts to the measurement end time Te,
The target is displayed on the display 4 by the video signal output from the controller 3, and the imaging camera 7
The eyeball image captured in step 3 is captured by the controller 3.

In the controller 3, the time change of the target position from the measurement start time Ts to the measurement end time Te is preset as target movement data according to the moving direction and moving speed of the target set by keyboard input or mouse input. The target movement data storage area 14 is stored. Then, when a predetermined start key is pressed, a video signal generated by the video signal generating means 15 based on the target movement data is output to the display 4 to display the target in an arbitrary color, and the start key is pressed. Is the measurement start time T
recorded as s.

The target displayed on the display 4 stands still from the measurement start time Ts to the movement start time Ta, is moved in a predetermined movement direction and movement speed from the movement start time Ta to the stop time Tb, and then from the stop time Tb. The movement data is set so as to be stationary until the measurement end time (see TGs (a) to (h) in FIG. 5). The moving speed can be freely set (0.1 to 20 seconds), but in this example, the time from the moving start time Ta to the time Tb is set to 2 to 3 seconds.

FIG. 3 shows the display 16 of the controller 3 during image pickup, and the window W ₁ for displaying the eyeball image captured by the image pickup camera 7 and the eyeball image pickup device 2 are shown.
The window W ₂ for displaying the image of the target displayed on the display 4 is overlapped and displayed.

On the other hand, the eyeball image captured by the image pickup camera 7 is output to the controller 3 and analyzed according to a preset program as the analyzer 17. The analyzer 17 displays on the display 4 the image processing means for reading the temporal change of the eyeball position as the eye movement data based on the eyeball image captured by the eyeball imaging device 2, the eye movement data read by the image processing means, and the display 4. It is provided with an analysis graph output means for synchronously superimposing and displaying the movement data of the projected target. In addition, pattern data extraction means for extracting pattern data of eye movement based on the movement data of the target and eye movement data displayed in a superimposed graph, and the pattern data extracted by the pattern data extraction means according to the disease in advance. A disease determination means for identifying the presence or absence of a disease and the type of the disease by comparing with the registered eye movement pattern data is provided.

FIG. 4 is a flow chart showing the processing procedure of the analyzer 17. In step STP1, the measurement start time Ts is read out, and in step STP2, the moving image input from the imaging camera 7 is read at a predetermined time interval (eg, from the time Ts). Capture as a still image every 0.05 seconds.

Next, in step STP3, the position of the eyeball center is read based on each still image read from the measurement start time Ts to the measurement end time Te, and the eyeball position and its time with respect to the moving direction of the target are taken as eyeball movement data. The data is stored in a preset storage area.

The eyeball position in this case is determined by using the calibration data that is normalized in advance by the moving amount of the target. The calibration is performed prior to the measurement, and targets from the center (reference position) of the display 4 facing the visual axis of the standard eyeball facing forward to the position at which the standard visual axis of the eyeball tilts 30 ° to the left and right. Is moved, and the eye movement amount on the image is normalized by the movement amount of the target based on the eyeball images at the start point and the end point, and the calibration data is calculated. As a result, if the eyeball accurately follows the target, the eyeball position read from the eyeball image and the target position of the target movement data exactly match.

When it is determined in step STP4 that all still images have been processed, the process proceeds to STP5, and the eye movement data from the measurement start time Ts to the measurement end time Te is read out, and the time-eyeball position is determined. Is displayed on the display 16 of the controller 3 as an eye movement graph line having X-Y coordinates.

Next, in step STP6, the target movement data set by the controller 3 is read out, and the eye movement graph in which the target movement graph line having the time-target position as the XY coordinates is displayed in step STP5. To be displayed in synchronization with.

Next, in step STP7, the eye movement pattern data is extracted based on the target movement data and the eye movement data. In step STP8, the extracted eye movement pattern data is converted into a preregistered disease. The presence / absence of the corresponding disease and the type of the disease are identified by comparing with the corresponding eye movement pattern data and the statistical value, and the analysis result is output in step STP9, and the process ends.

Here, the eye movement pattern data may be anything as long as it represents the tendency of the eye movement pattern observed in a patient having a specific disease. For example, the error or variance of the eye movement data with respect to the target movement data, etc. The statistical value of is considered, step STP7 and step ST
The data calculated by the same statistical processing in P8 is used.

The above steps STP1 to STP8
In the processing of step STP1, processing of steps STP1 to STP4 is image processing means, processing of steps STP5 to STP6 is analysis graph output means, processing of step STP7 is pattern data extraction means, and processing of steps STP8 to STP9 is disease determination means. Is.

FIGS. 5 (a) to 5 (h) are graph diagrams EG showing eye movement data when the target is slowly moved from the front to 30 ° horizontally rightward over 5 seconds.
The graph diagram TG showing the target movement data is overlapped and outputted on the display 16 or printed out.

FIG. 5A shows that when the target is followed, the eye movement data graph EG is always above the target movement data graph TG, and if the motion target position is exceeded, the target position will be exceeded. OKN-like nystagmus (opto-kinetic-nystagmus: optokinetic nystagmus) is seen. In FIG. 5B, when the target is followed, the graph diagram EG of the eye movement data is always lower than the graph diagram TG of the target movement data,
Gaze directional nystagmus that catches up with the target position later is seen. In FIG. 5 (c), gaze nystagmus caused by a central abnormality in which nystagmus occurs when the target is followed up to a certain angle is seen. In FIG. 5D, nystagmus caused by a peripheral abnormality in which the eyeball position is not always stable when the eyeball is in the front position when following the target is seen. FIG. 5 (e) shows nystagmus peculiar to spinocerebellar degeneration, in which a rectangular wave-like nystagmus movement is observed in which the target is stopped before the target position and then returned after the target position and stopped. . FIG. 5F also shows eye movements peculiar to spinocerebellar degeneration, and the graph diagram EG of the eye movement data is the graph diagram T of the target movement data.
There is an overshoot that is always above G, and when following the target, it goes too far ahead of the target position and does not return to the target position. Figure 5
(G) is the eye movement peculiar to Parkinson's disease, the graph EG of the eye movement data is always lower than the graph TG of the target movement data, and when the target is followed, the target is caught until the target stops. There is no undershoot. FIG. 5 (h) shows nystagmus seen in ocular muscle abnormalities, strabismus, etc., where portions that accurately follow the target and portions that move away from the target appear alternately.

When the analysis graph is displayed in this way,
Since the relationship between the target position and the eye movement is clear at a glance, the eye movement can be accurately grasped at the time of diagnosis, and there is no risk of misreading the graph.

Further, by comparing the pattern data obtained by statistically processing these graphs with the pattern data registered in advance, it is possible to determine the presence and type of disease. In this case, first, the error and the variance of the measured eye movement data with respect to the target movement data are obtained, and if the error and the variance are close to 0, it is determined that there is no disease, and if the error and the variance are small, the error and variance are determined as shown in FIG. ~
It corresponds to any of (d), and when it is large, it is shown in FIG.
It can be judged that it corresponds to any of (h).

When it is judged that the condition corresponds to any of FIGS. 5A to 5D, the frequency of the eye movement data is measured,
When the frequency is low and the error is positive, it corresponds to FIG. 5A, and when the frequency is low and the error is negative, FIG.
It can be determined that the case corresponds to (b), the case where the frequency is initially low and increases from the middle, corresponds to FIG. 5C, and the case where the frequency is high from the beginning corresponds to FIG. 5D.

Further, when it is judged to correspond to any of FIGS. 5 (e) to 5 (h), the velocity change of the eye movement is calculated from the analysis graph, and the portion where the movement of the eye and the target match is determined. Read if there is.

If there is an eyeball stop, FIG. 5 (e)
If the error is positive, there is no eyeball stop, and the movements of the eyeball and the target do not match, the case corresponds to FIG.
If the error is minus, there is no eyeball stop, and the movements of the eyeball and the target do not match, it corresponds to FIG. 5 (g). If there is no eyeball stop and the movements of the eyeball and the target match, then FIG.
It can be judged that it corresponds to (h).

Then, in the case corresponding to FIG. 5A, it is OK.
N-like nystagmus, suspicion of gaze directional nystagmus in the case of FIG. 5 (b), suspicion of gaze nystagmus caused by central abnormality in the case of FIG. 5 (c), FIG. ) Corresponds to suspected nystagmus due to abnormalities in the peripheral region such as the semicircular canal, and corresponds to Figure 5 (e) or (f) for spinocerebellar degeneration, and corresponds to Figure 5 (g) In the case of Parkinson's disease, it is judged that there is a suspicion of eye muscle abnormality in the case of Fig. 5 (h), and in the case of Parkinson's disease or spinocerebellar degeneration, how much the eye position deviates from the target It is possible to judge the severity, the therapeutic effect, and the pharmacological effect.

It should be noted that the head is rotated with the eyeball imaging device 2 attached to measure the vestibular oculomotor reflex caused by head movement and perform eyeball movement in dizziness when the target is projected and when the target is not projected, It is possible to compare both, and it is expected to be a clinically useful device.

Further, the case where the eyeball image pickup apparatus 1 picks up only the right eyeball has been described, but the present invention is not limited to this, and the image pickup cameras 7 may be provided on the left and right sides so that both eyes can be picked up at the same time. .

[0049]

As described above, according to the present invention, there is provided an image pickup camera for keeping the head stationary with respect to a display such as a screen displaying a target without fixing the head, and for taking an image of an eyeball. On the other hand, by taking an image of the eyeball with the head still, it has a very excellent effect that an accurate index tracking function test can be performed.
Furthermore, since the eye movement graph and the target movement graph are synchronously overlapped and displayed, the relative relationship between them can be more accurately and easily grasped.

Further, since it is sufficient to attach the goggle type headset to the patient's head, it can be easily used on the bedside regardless of the place of use, and by increasing the brightness of the display, it can be used in a completely dark place. It has a very excellent effect that it can be used without it.

[Brief description of drawings]

FIG. 1 is an explanatory diagram showing an eye movement analysis system according to the present invention.

FIG. 2 is an explanatory diagram showing an eyeball imaging device.

FIG. 3 is an explanatory diagram showing a screen during image capturing.

FIG. 4 is a flowchart showing a processing procedure of the analyzer.

FIG. 5 is a graph showing analysis results.

[Explanation of symbols]

1 ... Eye movement analysis system 2 ... Eyeball imaging device 3 ... Controller 4 ………… Display 5 ... Goggles type headset 6 ... Half mirror 7 ... Imaging camera 10 ......... Illumination light emitting section Cx ………… Imaging optical axis Ex ......... view axis 11 ………… Ear-hanging frame 17 ……… Analyzer

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Haruyuki Nantani             5-14-8 Hiyoshihoncho, Kohoku Ward, Yokohama City, Kanagawa Prefecture

Claims (9)

[Claims] 1. An eye movement analysis system comprising: an eyeball image pickup device for picking up a movement of an eyeball that follows a target displayed on a display; and an analyzer for performing a movement analysis based on the picked up eyeball image. The eyeball imaging device, a goggle type headset provided with the display at a position facing at least one eyeball, a half mirror for reflecting the image of the eyeball facing the display, and a reflection image of the eyeball by the half mirror. An image capturing means for capturing an image, wherein the analyzer is an image processing means for reading out a temporal change of the eye position as eye movement data based on the eye image captured by the eye image capturing device, and the image processing means. The eye movement data and the target movement data displayed on the display are synchronously superimposed. Eye movement analysis system, characterized in that it comprises an analysis graph output means for displaying the graph Te. 2. An eye movement analysis system comprising an eyeball image pickup device for picking up a movement of an eyeball that follows a target displayed on a display, and an analyzer for performing a movement analysis based on the picked up eyeball image, The analyzer is an image processing unit that reads out a temporal change of the eyeball position as eye movement data based on an eyeball image captured by the eyeball imaging device, and the eye movement data read by the image processing unit and moves to the display. An eye movement analysis system comprising: an analysis graph output means for synchronously superposing and displaying the displayed movement data of the target. 3. Based on an eyeball image when a target is moved by a predetermined amount from a reference position, calibration data is calculated by normalizing the eyeball movement amount on the image with the movement amount of the target, and based on the calibration data. The eye movement analysis system according to claim 1 or 2, wherein eye movement data is read by the image processing means. 4. The pattern data extracting means for extracting pattern data of the eye movement based on the movement data of the target and the eye movement data, and the pattern data extracted by the pattern data extracting means as a disease. The eye movement analysis system according to any one of claims 1 to 3, further comprising: disease determination means for identifying the presence or absence of a disease and the type of the disease by comparing the eye movement pattern data registered in advance. 5. The eye movement analysis system according to claim 4, wherein the pattern data is a statistical value obtained by statistically processing the movement data of the target and the eye movement data. 6. An eyeball image pickup device for picking up an eyeball movement that follows a target displayed on a display,
A goggle type headset having the display at a position facing at least one eyeball, a half mirror for reflecting an image of the eyeball facing the display, and an imaging camera for capturing a reflected image of the eyeball by the half mirror. An eyeball imaging device characterized by the above. 7. The display so that the visual axes of the eyeballs that follow the target displayed on the display can be tilted by 30 ° in the left-right direction when the visual axis when facing the front is 0 °. The eyeball imaging device according to claim 6, wherein the position and size of the eyeball are set. 8. An image pickup optical axis of the image pickup camera is attached to a goggle type headset so that the image pickup optical axis intersects obliquely with a visual axis of an eyeball facing front, and the half mirror is attached to the eyeball with the image pickup optical axis. The eyeball image pickup device according to claim 6 or 7, wherein the angle is set to reflect the light toward the eyeball. 9. The eyeball image pickup device according to claim 7, wherein an illumination light emitting portion for image pickup of an eyeball is integrally attached to an image pickup camera.