JP2008173188A - System for measurement of eye position in closed state of eyes - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a system 100 for measurement of eye position which accurately measures a visual line angle simultaneously with brain function measurement using fMRI while the eyes of a subject M are closed or the subject M is asleep. <P>SOLUTION: The system for measurement of eye position comprises a correlation data calculation section for calculating correlation data that shows correlation between a visual line angle and surface image related data, and a visual line angle calculation section 51 for calculating a visual line angle of a subject M when the eye is imaged on the basis of the correlation data and surface image related data acquired by imaging the eyelids of the subject M when the eye is closed by means of an optical imaging apparatus 2 separately from imaging at each point for acquiring the surface image related data used for calculating the correlation data. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an eye position measurement system that measures a line-of-sight angle when a subject is closed.

  Knowing which part of the brain is active during sleep is a very important factor for sleep studies, and therefore fMRI, which can measure brain function non-invasively, is often used. On the other hand, for example, there is a technique that classifies sleep into REM sleep and non-REM sleep depending on the presence or absence of rapid eye movement (REM: Rapid Eye Movement). Are known. For this reason, it is expected that sleep research will be further advanced if the relationship between brain function and eye movement is elucidated, and various proposals have been made regarding eye movement measurement techniques for that purpose.

  One of them is EOG (eltro-oculogram). EOG is a method of measuring the potential difference between the retina and the cornea with an electrode attached to the skin around the eyeball, and the eye movement can be measured by a change in the potential difference. Further, there is also known one that can detect the position of the eyeball when the eye is closed by using a contact lens with a coil and an alternating magnetic field.

  However, in the MRI apparatus that generates a strong magnetic field, artifacts are generated in the former, and accurate measurement is difficult. In the latter case, since it must be worn on the eyeball, the burden on the subject is great, and it is difficult to use the device in the MRI apparatus due to the influence of the magnetic field. That is, it is very difficult to measure eye movement simultaneously with brain function measurement using fMRI depending on these eye movement measurement techniques.

  Therefore, as shown in Non-Patent Document 1, there has been proposed an apparatus that can detect an eyeball position in an MRI apparatus by imaging an eyelid surface during sleep using an infrared video camera and performing image processing. ing. However, there is a problem that the accuracy is low even if a certain amount of eye movement can be detected because the shape of the eyelid surface varies depending on the subject.

On the other hand, if fMRI is used in the first place, functional images in various parts of the body can be obtained. Therefore, it is not necessary to use another detector for measuring eye movement. There may be doubts that it may be necessary to measure the function and eye movement at the same time, but at present, images with fMRI cannot be obtained with spatial resolution as high as MRI. As described above, there are many cavities, and if an image is taken under the same conditions as the brain, a large distortion occurs in the image, and it is impossible to capture eye movement from the image. Furthermore, since the speed of eye movement reaches several hundred degrees / second in terms of viewing angle, it is impossible to accurately measure eye movement from functional images captured every 2 to 3 seconds.
Proceedings of the 1st Composite Medical Engineering Symposium p152-p155 “Eye Position Estimation Using Infrared Video for Brain Function Evaluation During Sleep”

  The present invention has been made in view of such a problem, and learns a conversion rule from a surface optical image to a line-of-sight angle for each subject using the line-of-sight angle obtained from the MRI structure image as a teacher signal. Is used to accurately measure the gaze angle of a subject who is closed or sleeping while simultaneously measuring brain function with fMRI, thereby contributing to further progress in sleep research, etc. is there.

That is, the eye position measurement system according to the present invention comprises an eye position learning mechanism and an eye position measurement mechanism,
The eye position learning mechanism changes the line of sight of the subject in the closed eye state, and obtains the structural image related data and the optical imaging apparatus, which are data related to the structural image of the eyeball obtained by the MRI apparatus at a plurality of points where the line of sight is changed. An image-related data acquisition unit that acquires a pair of surface image-related data that is data related to the surface image of the eyelid, and a line-of-sight angle measurement unit that measures the line-of-sight angle at each point based on the structural image-related data And a correlation data calculation unit that compares the respective line-of-sight angles with the surface image-related data corresponding to the respective line-of-sight angles, and calculates correlation data indicating a correlation between the line-of-sight angle and the surface image-related data, and
In addition to imaging at each point, the eye position measurement mechanism performs imaging based on surface image-related data obtained by imaging the eyelid of the subject in the closed eye state with an optical imaging device and the correlation data. A gaze angle calculation unit that estimates and calculates the gaze angle of the subject at the time.

  In addition, the eye position learning mechanism according to the present invention causes a closed eye subject to change his / her line of sight, and structural image related data and optical data that are data related to the structure image of the eyeball obtained by the MRI apparatus at a plurality of points where the line of sight is changed. An image-related data acquisition unit that acquires a pair of surface image-related data that is data related to the surface image of the eyelid obtained by the imaging device, and the line-of-sight angle at each point is measured based on the structure image-related data A line-of-sight angle measurement unit that compares the surface image-related data corresponding to each line-of-sight angle, and a correlation data calculation unit that calculates correlation data indicating a correlation between the line-of-sight angle and the surface image-related data. It is characterized by having.

  In order to facilitate the calculation of correlation data, the surface image-related data contains the brightness value at each coordinate in a line or two-dimensional region virtually set on the upper eyelid side on the original image. It is desirable.

  In order to facilitate gaze angle measurement for learning, it is preferable that the structural image-related data indicates an eyeball extracted image and a lens extracted image extracted from the original image.

  As a specific aspect of the visual angle measurement unit, the visual angle measurement unit measures the visual angle based on the position of the crystalline lens with respect to the positional relationship of each eyeball obtained from the structural image related data. it can.

  In order to obtain a highly accurate correlation, the correlation data calculation unit forms a neural network as the correlation data, and the line-of-sight angle actually measured at each point as the input teaching signal at the time of forming the neural network and the above-mentioned It is preferable to provide at least the coordinates and to give the luminance value at the coordinates as the output teaching signal.

  The eye position measurement mechanism according to the present invention uses correlation data obtained by the eye position learning mechanism, and relates to a surface image obtained by imaging an eyelid in a closed eye state by an optical imaging device. A line-of-sight angle calculation unit that estimates and calculates the line-of-sight angle at the time of imaging based on the data and the correlation data is provided.

  Further, the eye position learning program according to the present invention causes a closed eye subject to change his / her line of sight, and structural image related data which is data on a structural image of an eyeball obtained by an MRI apparatus at a plurality of points where the line of sight is changed, and An image-related data acquisition unit that acquires a pair of surface image-related data that is data related to the surface image of the eyelids obtained by the optical imaging device, and a line-of-sight angle at each point based on the structure image-related data A line-of-sight angle measurement unit for actual measurement, a correlation data calculation unit that compares the surface image-related data corresponding to each line-of-sight angle and calculates correlation data indicating a correlation between the line-of-sight angle and the surface image-related data; It is characterized by causing the computer to exhibit the functions as described above.

  The eye position measurement program according to the present invention uses the correlation data obtained by the eye position learning mechanism, and is a surface image obtained by imaging an eyelid in a closed eye state by an optical imaging device. Based on the related data and the correlation data, the computer functions as a line-of-sight angle calculation unit that estimates and calculates the line-of-sight angle at the time of imaging.

According to the present invention configured as described above, a conversion rule from a surface optical image to a line-of-sight angle is learned for each subject using the line-of-sight angle obtained from the MRI structure image as a teacher signal, and the learning result is used. Since the gaze angle when the subject is closed is estimated and calculated from the surface optical image, the accuracy and reliability of gaze angle calculation (hereinafter also referred to as eye position measurement) when the eye is closed can be greatly improved.
In addition, since measurement can be performed without being affected by the noise even when used in combination with an MRI apparatus, for example, the subject is allowed to sleep in the gantry of the MRI apparatus, and the eye position measurement system is operated during the sleep. If the fMRI image is acquired while measuring the eye movement, the relationship between the brain function during sleep and the eye movement can be grasped more accurately. And this is thought to contribute to new advances in sleep research.
Furthermore, as long as learning is completed, it is not always necessary to use MRI for eye position measurement, and it is only necessary to have an optical real image or video image around the eyelid. Can be measured by configuration.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a schematic diagram showing an entire eye position measurement system 100 according to the present embodiment.

  The eye position measurement system 100 measures the line-of-sight angle (or eye position) of the subject M in the closed eye state. In terms of physical configuration, as shown in FIG. The optical imaging apparatus 2 and the information processing apparatus 3 that is a general-purpose or dedicated computer are provided. From a functional aspect, the apparatus includes an eye position learning mechanism 4 and an eye position measurement mechanism 5 as shown in FIG. Has been.

  As is well known, the MRI apparatus 1 includes a gantry 11 that generates a strong magnetic field and radio waves, and resonates hydrogen nuclei that are constituent atoms of water and fat to resonate the internal cross-sectional structure of the subject M inside the gantry 11. Can be imaged. The MRI apparatus 1 can output not only the structural image but also a functional image based on a temporal change in signal intensity for each voxel depending on the degree of blood oxidation. The optical imaging device 2 is installed outside the gantry 11 and images the periphery of the eyelid of the subject M through a mirror 21 disposed in the gantry 11. Here, an infrared detection type device is used. ing. This is because infrared light from the LED 22 that cannot be perceived by the subject M and has little influence on sleep is used for illumination of the subject M. The infrared light from the LED 22 is supplied into the gantry 11 using the optical fiber 23. Further, when visible light is used for illumination, the optical imaging device 2 may use a normal one accordingly. The information processing apparatus 3 is configured mainly by a general-purpose or dedicated so-called computer including a CPU, a memory, an input / output unit, an I / O interface, and the like, and additionally includes an analog electric circuit or the like. There is also.

  The eye position learning mechanism 4 and the eye position measurement mechanism 5 are configured to perform their functions by the cooperation of the MRI apparatus 1, the optical imaging apparatus 2, and the information processing apparatus 3, In particular, the information processing apparatus 3 operates as a CPU or the like according to a predetermined program installed therein, and as shown in FIG. 2, the image-related data acquisition unit 41, the gaze angle measurement unit 42, the correlation data calculation unit 43, the gaze angle It functions as the calculation unit 51 and the like.

  Briefly describing each unit, the image-related data acquisition unit 41 asks the subject M who is lying in the gantry 11 of the MRI apparatus 1 to change his / her line of sight over a plurality of points. At each point, data relating to the structural image of the eyeball obtained by the MRI apparatus 1 (hereinafter referred to as structural image-related data) and data relating to the surface image of the eyelid obtained by the optical imaging apparatus 2 (hereinafter referred to as surface image-related) Data). The line-of-sight angle measurement unit 42 measures the line-of-sight angle at each point based on the structural image related data. The correlation data calculation unit 43 compares the respective line-of-sight angles with the surface image-related data corresponding to the respective line-of-sight angles, and calculates correlation data indicating the correlation between the line-of-sight angle and the surface image-related data. The line-of-sight angle calculation unit 51 images the eyelid of the subject M in the closed eye state with the optical imaging device 2 separately from the imaging at each point, and based on the obtained surface image related data and the correlation data, The line-of-sight angle at the time of imaging is estimated and calculated.

  The operation of the eye position measuring system 100 having such a configuration will be described below by dividing the operation into the eye position learning mechanism 4 and the eye position measuring mechanism 5.

  First, the operation of the eye position learning mechanism 4 will be described.

  An operator instructs the subject M who is lying in the gantry 11 of the MRI apparatus 1 in an eye-closed state (but awake) to change his / her line of sight over a plurality of points. The line-of-sight setting points here are, for example, left, middle, and right, but of course, as long as the subject M is possible, the points may be increased.

  Then, at each point, the subject M fixes his / her line of sight, and in that state, the cross-sectional structure image viewed from the head including the left and right eyeball portions is imaged by the MRI apparatus 1, and both the left and right by the optical imaging apparatus 2. A surface optical image of the eyelid is taken almost simultaneously.

  Each original image data obtained by this imaging (a structural image is shown on the left in FIG. 3 and left and right optical images are shown in FIG. 4) is transmitted to the information processing device 3, where it is processed and converted into image-related data. The

  The image processing is performed by the image related data acquisition unit 41. As for the structural image related data, as schematically shown in FIG. 5, the eyeball is extracted from the original image by binarization labeling or the like to generate an eyeball extracted image (see FIG. 6), and the original image and the eyeball extracted image. And the like, and a lens extraction image (see FIG. 6) is generated. Data including the eyeball extracted image and the lens extracted image is structural image related data. Regarding surface image-related data, first, eyelids are detected from the original image by binarization processing, etc., and, for example, the end points of the eyelid row are connected on the original image, or the upper eyelid side by a predetermined pixel from it. In addition, left and right lines (shown in FIG. 7) are set, and a luminance value string is generated for each unit area obtained by dividing the luminance on the left and right lines by dividing a predetermined number of pixels (one or more) into one unit area. This luminance value sequence is the surface image related data. For example, the order of each unit from the left is set as coordinates indicating the position (for example, natural numbers from 1 to m). The image-related data acquisition unit 41 sets the image-related data set in a predetermined area of the memory by pairing each image-related data obtained in this way for each point and linking it to the identifier of the subject M. Store in the storage unit D1. In some cases, no image processing is performed, and the image-related data becomes the original image data itself.

Next, the line-of-sight angle measurement unit 42 measures the line-of-sight angle at each point based on the structural image related data. Specifically, as shown in FIG. 3 right and FIG. 8, for example, the line-of-sight angle calculation unit 42 first establishes a virtual straight line A connecting the centers of the respective eyeballs from the eyeball extracted image and is perpendicular to the virtual straight line A. in vertical virtual line B _R extending from the center of each _eyeball, enact B _L. Next, virtual straight lines C _R and C _L from the center of each eyeball to the center of the crystalline lens are established. And B an angle between _R and C _R the viewing angle of the right eye, the angle between B _L and C _L and viewing angle of the left eye.

  Finally, the correlation data calculation unit 43 compares the respective line-of-sight angles with the corresponding surface image-related data, and calculates correlation data indicating the correlation between the line-of-sight angle and the surface image-related data. Specifically, as shown in FIG. 9, in addition to the line-of-sight angle obtained for each point as an input teacher signal, the coordinates (indicated as x in FIG. 9) and their related values (here, coordinates x) 2 to the power of n, where n is a natural number greater than or equal to 2) and a bias, and the luminance value of the unit area indicated by the coordinates is used as an output teacher signal, for example, a three-layer neural network structure between them To learn by back-propagation. This is performed for all unit areas and for each point. The neural network formed in this way is the correlation data in this embodiment, and learning ends when the neural network is formed. In this embodiment, this neural network (correlation data) is paired with the subject identifier and stored in the correlation data storage D3 section.

  By using such a neural network (that is, correlation data), the luminance corresponding to the coordinate value can be calculated by inputting an arbitrary line-of-sight angle and coordinate value. Thus, for example, in this embodiment, the estimated surface image related data generation unit 7 is provided, and the estimated surface image related data generation unit 7 calculates pixel luminance values of all coordinates for all line-of-sight angles for each predetermined unit. Then, the brightness value sequence calculated for each line-of-sight angle, that is, the estimated surface image-related data is stored in the estimated surface image-related data storage unit D2 set in a predetermined area of the memory.

  Next, the operation of the eye position measurement mechanism 5 will be described.

  Here, as described above, the estimated surface image related data obtained using the neural network as the correlation data, and the surface image related data obtained by imaging with the optical imaging device 2 separately from the imaging at the time of learning ( Hereinafter, the line-of-sight angle calculation unit 51 compares the measured surface image-related data with the estimated surface image-related data. In the comparison, the luminance values of corresponding coordinates in the estimated surface image-related data and the measured surface image-related data are compared, and the Euclidean distance between the luminance values is calculated. Then, the line-of-sight angle calculation unit 51 extracts, from all the estimated surface image-related data, the closest Euclidean distance to the measured surface image-related data. Finally, the line-of-sight angle corresponding to the extracted estimated surface image-related data is estimated as the line-of-sight angle of the subject M at the time of imaging. Incidentally, the error between the gaze angle actually measured by a subject M and the estimated gaze angle is 1.15 ± 3.46 [degrees] for the right eye and 1.28 ± 4.05 [degrees] for the left eye. This shows that the accuracy is very high.

  Then, by performing imaging with the optical imaging apparatus 2 in a time series in a predetermined cycle, the eye movement of the subject M when the eye is closed can be obtained as a moving image or time series data (see FIG. 10).

  Therefore, according to the eye position measurement system 100 according to the present embodiment configured as described above, the conversion rule from the surface optical image to the line-of-sight angle is determined for each subject M using the line-of-sight angle obtained from the MRI structure image as a teacher signal. Since learning is performed and the line-of-sight angle is estimated and calculated from the surface optical image using the learning result, as described above, the accuracy and reliability of the calculation result can be greatly improved.

  In addition, since measurement can be performed without being affected by noise in the MRI apparatus 1, for example, the subject M is caused to sleep in the gantry 11 of the MRI apparatus 1, and the eye position measurement system 100 is operated during the sleep. If the fMRI image is acquired while measuring the eye movement, the relationship between the brain function during sleep and the eye movement can be grasped more accurately. And this is thought to contribute to new advances in sleep research.

  Furthermore, if learning is completed, that is, if there is correlation data related to the subject, it is not always necessary to use MRI for subsequent eye position measurement. For example, it can be used in combination with other brain function measuring devices such as NIRS. It doesn't matter. Further, if it is used only for measuring the eye movement of the subject, it is sufficient to have an optical real image or video image around the eyelid, and the eye position can be measured with a very simple configuration. .

The present invention is not limited to the above embodiment.
For example, it is not always necessary to use a neural network for the correlation data. In addition, as the surface image-related data, the luminance value of each coordinate in the two-dimensional region as well as the line may be used. The structural image related data may be in other formats, and when the line-of-sight angle is actually measured from the structural image related data, the position of the nose may be used as a reference. In addition, the present invention is not limited to the illustrated example, and various modifications can be made without departing from the spirit of the present invention.

1 is a schematic overall view showing an eye position measurement system according to an embodiment of the present invention. The functional block diagram which shows the function of the information processing apparatus in the embodiment. The original image figure which shows the structure original image etc. of the eyeball in the embodiment. The original image figure which shows the surface original image of the eyelid in the embodiment. The flowchart which shows the production | generation flow of the structure related image in the embodiment. The image figure which shows the structure original image in the same embodiment, and the eyeball extraction image and lens extraction image which were extracted from there. Explanatory drawing for demonstrating the left-right line in the embodiment. Explanatory drawing for demonstrating the measurement process of the gaze angle in the embodiment. The schematic diagram for demonstrating the neural network in the embodiment. Experimental data when eye movement is actually measured in the same embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Eye position measurement system 1 ... MRI apparatus 2 ... Optical imaging device 4 ... Eye position learning mechanism 41 ... Image related data acquisition part 42 ... Gaze angle measurement part 43 ... Correlation data calculation unit 5 ... eye position measurement mechanism 51 ... gaze angle calculation unit M ... subject

Claims (9)

Regarding the image of the eyelid obtained by the optical imaging device and the structural image related data, which is data regarding the structural image of the eyeball obtained by the MRI apparatus at a plurality of points where the gaze of the subject in the closed eye state is changed. An image-related data acquisition unit that acquires a pair of surface image-related data that is data;
A line-of-sight angle measurement unit that measures the line-of-sight angle at each point based on the structural image-related data;
A correlation data calculation unit that compares the respective line-of-sight angles with the surface image-related data corresponding to the respective line-of-sight angles, and calculates correlation data indicating a correlation between the line-of-sight angles and the surface image-related data;
Separately from the imaging at each point, based on the surface image related data obtained by imaging the eyelid of the subject in the closed eye state with an optical imaging device and the correlation data, the gaze angle of the subject at the time of imaging is determined. An eye position measurement system comprising: a gaze angle calculation unit that performs estimation calculation. Structured image related data that is data related to the structure image of the eyeball obtained by the MRI apparatus at a plurality of points where the line of sight is changed by the subject in the closed eye state and data related to the surface image of the eyelid obtained by the optical imaging apparatus An image-related data acquisition unit that acquires a pair of surface image-related data,
A line-of-sight angle measurement unit that measures the line-of-sight angle at each point based on the structural image-related data;
An eye position learning mechanism comprising: a correlation data calculation unit that compares the line-of-sight angle with the surface image-related data corresponding to the line-of-sight angle and calculates correlation data indicating a correlation between the line-of-sight angle and the surface image-related data .   The eye position learning mechanism according to claim 2, wherein the surface image related data includes a luminance value at each coordinate in a line or two-dimensional region virtually set on the upper eyelid side on the original image.   The eye position learning mechanism according to claim 2 or 3, wherein the structure image related data indicates an eyeball extracted image and a lens extracted image extracted from an original image.   The eye position learning mechanism according to any one of claims 2 to 4, wherein the line-of-sight angle measurement unit measures the line-of-sight angle based on a position of a crystalline lens with respect to a positional relationship of each eyeball obtained from the structural image related data.   The correlation data calculation unit forms a neural network as the correlation data, and provides at least the line-of-sight angle actually measured at each point and the coordinates as an input teaching signal when the neural network is formed, and an output teaching signal The eye position learning mechanism according to claim 3, wherein a luminance value at the coordinates is given. Using the correlation data obtained by the eye position learning mechanism according to claim 2,
An eye position measurement mechanism comprising a line-of-sight angle calculation unit that estimates and calculates a line-of-sight angle at the time of imaging based on surface image-related data obtained by imaging an eyelid in a closed eye state by the optical imaging device and the correlation data . Structured image related data that is data related to the structural image of the eyeball obtained by the MRI apparatus at a plurality of points where the line of sight is changed by the subject in the closed eye state and data related to the surface image of the eyelid obtained by the optical imaging device An image-related data acquisition unit that acquires a pair of surface image-related data,
A line-of-sight angle measurement unit that measures the line-of-sight angle at each point based on the structural image-related data;
The computer functions as a correlation data calculation unit that compares the line-of-sight angles with the surface image-related data corresponding to the line-of-sight angles and calculates correlation data indicating the correlation between the line-of-sight angles and the surface image-related data. Eye position learning program. Using the correlation data obtained by the eye position learning mechanism according to claim 2,
Based on the surface image-related data obtained by imaging the eyelid in a closed eye state with the optical imaging device and the correlation data, the computer functions as a gaze angle calculation unit that estimates and calculates the gaze angle at the time of imaging. Eye position measurement program.