CN113040701A - Three-dimensional eye movement tracking system and tracking method thereof - Google Patents
Three-dimensional eye movement tracking system and tracking method thereof Download PDFInfo
- Publication number
- CN113040701A CN113040701A CN202110264441.2A CN202110264441A CN113040701A CN 113040701 A CN113040701 A CN 113040701A CN 202110264441 A CN202110264441 A CN 202110264441A CN 113040701 A CN113040701 A CN 113040701A
- Authority
- CN
- China
- Prior art keywords
- eye
- image
- eye movement
- oct
- delta
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
- A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
- A61B3/102—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for optical coherence tomography [OCT]
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
- A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
- A61B3/113—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for determining or recording eye movement
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
- A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
- A61B3/14—Arrangements specially adapted for eye photography
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Animal Behavior & Ethology (AREA)
- Biophysics (AREA)
- Ophthalmology & Optometry (AREA)
- Veterinary Medicine (AREA)
- Public Health (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Human Computer Interaction (AREA)
- Eye Examination Apparatus (AREA)
Abstract
The invention discloses a three-dimensional eye movement tracking system, which comprises a pupil camera imaging module, an image recognition analysis module, a motor control module and an OCT system module which are sequentially connected; the tracking method of the system comprises the following steps: firstly, continuously imaging an object to be detected in real time through a pupil camera module; when the object to be measured, namely the eyes, has eye movement along the eye axis or the transverse direction, namely the horizontal direction or the longitudinal direction, namely the vertical direction, the sizes of the eye movement along the eye axis, the horizontal direction and the vertical direction can be respectively calculated through the image recognition and analysis module, and then the displacement amounts delta Z, delta X and delta Y of the corresponding motors in the three directions are calculated; adjusting a corresponding motor to move according to the calculated motor displacement so that the object to be measured always keeps an accurate working distance and is aligned with the center of the ocular lens; and performing three-dimensional imaging on the anterior segment or the posterior segment of the eye of the object to be detected by the OCT system module. The invention can track the three-dimensional eye movement in real time and calculate the size and the direction of the eye movement, and compensate the eye movement in diagnosis in real time through the motor control module, thereby realizing high-quality system imaging, reducing the complexity of a diagnosis system and improving the success rate and the efficiency of diagnosis.
Description
Technical Field
The invention belongs to the technical field of imaging devices, and particularly relates to a three-dimensional eye movement tracking system and a tracking method thereof.
Background
The Optical Coherence Tomography (OCT) technique is a three-dimensional imaging technique, has the advantages of high sensitivity, high imaging speed, high resolution, and the like, and is widely applied to the fields of ophthalmology (including anterior segment and posterior segment of eye), cardiovascular department, dermatology department, and the like. In the last two decades, an important progress of OCT technology is that the scanning rate is faster and faster, and the imaging speed of OCT is greatly increased from the conventional time domain OCT (OCT), to the frequency domain-based OCT (SD-OCT), and to the swept source-based OCT (SS-OCT). The imaging speed is high, and the unique clinical application advantages are achieved: can complete two-dimensional and/or three-dimensional scanning in a shorter time, thereby reducing the sensitivity of eye movement of a person to imaging quality during imaging, greatly improving the efficiency of clinical diagnosis and treatment and being popular with clinicians.
The influence of eye movement on OCT imaging is mainly reflected in the influence on the imaging precision and repeatability, and the diagnosis and treatment efficiency is reduced clinically. Further reducing the effect of eye movement on OCT imaging quality, on the one hand the OCT sweep speed can be further increased, for example SS-OCT VG200 axial scan rate which has been used clinically for visual imaging has been up to 20 ten thousand times per second (ref: Yang J, Chen Y.Vitreeting transaction with OCT. Ophthalmol Retina.2019 Nov; 3 (11): 961. doi: 10.1016/j.oret.2019.07.003. PMID: 31699311), in which system eye movement tracking technology is still required; there are megahertz grades of OCT available on the laboratory platform, such as the documents Klein T, Wieser W, Eigenwillig CM, Biedermann BR, Huber R.Megahertz OCT for ultra-wide-field reflecting with a 1050 nm Fourier domain mode-locked laser Opt express.2011; 19: 3044-62. Available:http：//www.ncbi.nlm.nih.gov/pubmed/ 21369128doi: 10.1364/OE.19.003044 PMID: 21369128, the OCT axial scan rate of the disclosed Fourier domain mode-locked laser is 1.37 million per second, but such a system is currently relatively complexIt is difficult to apply clinically. On the other hand, focus may be on continuing improvements in eye tracking technology. For example, the Scanning Laser Ophthalmoscope (SLO) -based eye movement tracking OCT technology disclosed in Kari v.violola, Boy Braaf, Christy k.shehy, Qiang Yang, Pavan tireveedhula, David w.arthorn, Johannes f.de Boer, and Austin Roorda, "Real-time eye movement compensation for OCT imaging with tracking SLO," biomed.op.express 3, 2950-fold 2963(2012) can reduce the influence caused by eye movement well. However, this technique is only suitable for posterior segment OCT, not for anterior segment OCT, and still sensitive to eye movement for large field OCT systems due to the insufficient high frame rate of SLO.
The eye movement tracking technology based on the pupil camera is widely applied to a three-dimensional eye movement tracking system due to the advantage of low cost. For example, the documents Oscar Carrasco-Zevallos, Derek Nankivil, Brenton Keller, Christian Viehland, Brandon J.Lujan, and Joseph A.IZatt, Pupil tracking optical coherence tomography for precision control of Pupil entry position, biomed.Opt.Express 6, 3405 + 3419 (2015); the two-dimensional eye movement tracking based on the pupil camera can effectively track eye movement in the horizontal and/or vertical direction, and further compensate for posterior segment OCT (optical coherence tomography); the documents Carrasco-Zevallos OM, Nankivil D, Viehland C, Keller B, IZatt JA.Pupil Tracking for Real-Time Motion Corrected inorganic Segment Optical Coherence Tomography.PLoS one.2016; 11(8): e0162015.published 2016 Aug 30. doi: 10.1371/joural. bone.0162015 discloses a pupil camera based two-dimensional eye tracking that tracks horizontal and/or vertical eye movements and compensates for anterior segment OCT. However, these methods compensate the eye movement to the driving waveform of the OCT scanning galvanometer, are limited by the range size and frame rate of the compensation, cannot adjust the center alignment of the eye and the ocular lens in real time based on the eye movement, and do not track the eye movement in the direction of the eye axis in real time.
Disclosure of Invention
The purpose of the invention is as follows: aiming at the defects, the invention provides a three-dimensional eye movement tracking system and a tracking method thereof, which can effectively reduce the influence of eye movement on the imaging quality of anterior segment OCT and posterior segment OCT.
The technical scheme is as follows: the invention provides a three-dimensional eye movement tracking system and a tracking method thereof, wherein the three-dimensional eye movement tracking system comprises a pupil camera imaging module, an image recognition analysis module, a motor control module and an OCT system module which are sequentially connected;
the pupil camera imaging module acquires images of pupils in real time through a pupil camera;
the image recognition and analysis module obtains the variation delta R of the radius or the diameter of the annular light mapping point in the real-time pupil camera image relative to the radius or the diameter of the annular light mapping point in the initial pupil camera image through a threshold method, further obtains the movement amount of the human eye movement along the direction of the eye axis, then converts the movement amount into the accurate working distance, and obtains the displacement delta Z which the motor should move; in parallel, the deviation of the center of the annular reflection point in the real-time pupil camera image from the optical axis of the pupil camera is obtained through an image recognition and analysis module, and is compared with the deviation of the center of the annular reflection point in the initial pupil camera image from the optical axis of the pupil camera to obtain the deviation of human eyes in the horizontal and vertical directions, and accordingly, the displacement amounts delta X and delta Y of the compensation motor in the horizontal and vertical directions are converted;
the motor control module respectively adjusts corresponding motors according to the calculated delta X, delta Y and delta Z, so that the human eyes always keep accurate working distance under the condition of eye movement and are aligned with the centers of the ocular lenses;
the OCT system module is used for carrying out anterior segment OCT and posterior segment OCT imaging under the assistance of real-time eye movement tracking.
Furthermore, the pupil camera imaging module is composed of an annular pupil illumination LED, an ocular lens, an imaging lens and a camera, wherein the annular pupil illumination LED is composed of at least 3 LEDs which are arranged in an annular shape at equal intervals.
A tracking method of the three-dimensional eye movement tracking system as described above, comprising the steps of:
(1) carrying out imaging acquisition on an object to be detected so as to obtain at least two images;
(2) tracking the movement of the eye along the direction of the eye axis to obtain the variation delta R of the radius or the diameter of the annular light mapping point in the second image relative to the radius or the diameter of the annular light mapping point in the initial image; converting the variable quantity delta R into a displacement quantity delta Z of the movement of the motor;
(3) tracking the movement of eyes along the horizontal direction and the vertical direction, obtaining the offset of eyes in the horizontal direction and the vertical direction, and converting the offset into the horizontal direction displacement delta X and the vertical direction displacement delta Y of a motor for adjusting the center alignment of the ocular lens and the eyes;
(4) adjusting the motor to move according to the calculated delta X, delta Y and delta Z;
(5) and finally imaging through an OCT system module.
Further, the specific steps of tracking the movement of the eye along the direction of the eye axis in the step (2) are as follows:
(2.1) calculating the radius or the diameter of the annular light mapping point in each image through fitting according to the annular light mapping points in the images;
(2.2) when there is eye movement, the radius or diameter of the annular light mapping point in the latter image has a variation Δ R relative to the diameter of the annular light mapping point in the former image, and when the variation Δ R exceeds a threshold, a command is sent to the motor control module to make the motor move by a corresponding displacement Δ Z.
Further, the specific steps of calculating the variation Δ R and the displacement Δ Z in the step (2.2) are as follows: under paraxial approximation conditions, the diameter of the annular pupil illumination LED is:
wherein m isLIs the magnification of the optical system, RcThe radius of the cornea of the human eye, w is the distance between the eye and the ocular lens, and d is the distance between the center of the annular pupil illumination LED and the top point of the ocular lens;
the variation of the diameter of the annular reflection point along with the working distance can be obtained by calculating the derivative according to the formula:
wherein the amount of change
The motor moves by the corresponding displacement amount Δ Z ═ dw.
Further, the method for determining the threshold in step (2.2) is as follows: the threshold value of the size of the change of the radius or the diameter of the annular light mapping point is not less than the test repeatability of the radius or the diameter of the annular light mapping point.
Further, the specific steps of tracking the movement of the eye in the horizontal and vertical directions in the step (3) are as follows:
(3.1) taking one image as a first image, and calculating the deviation of the center of an annular light reflecting point in the first image from the optical axis of a pupil camera;
(3.2) selecting another image as a second image, and calculating the deviation of the center of the annular light reflecting point in the second image from the optical axis of the pupil camera;
(3.3) when there is eye movement, the deviation of the center of the annular light reflex point in the second image from the optical axis of the pupil camera has a variation relative to the deviation of the center of the annular light reflex point in the initial image, namely the center of the annular light reflex point in the first image from the optical axis of the pupil camera, when the variation exceeds a threshold value, the horizontal position and the vertical position are adjusted, so that the eye is always coincided with the center of the ocular lens, wherein the horizontal displacement of the movement is delta X, and the vertical displacement is delta Y.
Further, the specific steps of imaging by the OCT system module in step (5) are as follows: when the anterior segment of the eye is subjected to OCT imaging, the anterior segment of the eye can be imaged by moving an OCT anterior-posterior segment switching module into an OCT optical path from the inside of the system; when the OCT imaging of the posterior segment of the eye is carried out, the OCT light path is shifted out from the OCT light path by the anterior-posterior segment switching module of the OCT imaging system from the inside of the system, and then the imaging of the posterior segment of the eye can be carried out.
By adopting the technical scheme, the invention has the following beneficial effects:
the three-dimensional eye movement tracking system can track and compensate three-dimensional eye movements in real time, overcomes the defect that the two-dimensional eye movement tracking system cannot track the eye movements in the direction of the eye axis, increases the size range of eye movement tracking in the horizontal direction and the vertical direction, and is better applied to a clinical diagnosis system. The light path design of the three-dimensional eye movement tracking system based on the pupil camera can be decoupled with other imaging modules, and the three-dimensional eye movement tracking system is simultaneously applied to an anterior segment and a posterior segment OCT system, so that the defect that the system can only be applied to the posterior segment OCT system based on SLO is overcome. In addition, the pupil camera-based three-dimensional eye movement tracking system has higher frame rate and lower cost compared with the SLO imaging technology, thereby having the potential of wider application.
Drawings
FIG. 1 is a schematic diagram of the system of the present invention;
FIG. 2 is a schematic diagram of the optical path of an annular illumination LED imaged by a lens after being reflected by the cornea of an eye in an embodiment;
FIG. 3 is an image of a 360 degree annular LED formed by the annular illumination LED after reflection from the cornea of an eye in an exemplary embodiment;
FIG. 4 is a diagram of an optical system for posterior segment OCT imaging and anterior segment OCT imaging used in an exemplary embodiment;
FIG. 5 is a diagram of pupil camera imaging optics used in an embodiment;
FIG. 6 is an image of OCT posterior segment of the eye caused by horizontal eye movement without eye movement tracking in an embodiment:
FIG. 7 is an image of the posterior segment of an OCT eye caused by vertical eye movement without eye tracking in an exemplary embodiment;
FIG. 8 is an image of an OCT posterior segment of an eye made by eye movement along the axial direction of the eye without eye tracking in an exemplary embodiment;
FIG. 9 is an image of an OCT posterior segment of an eye with eye tracking in an embodiment;
FIG. 10 is an image of an OCT anterior ocular segment resulting from horizontal eye movement without eye tracking in an exemplary embodiment;
FIG. 11 is an image of an OCT anterior ocular segment resulting from vertical eye movement without eye tracking in an exemplary embodiment;
FIG. 12 is an image of an OCT anterior ocular segment resulting from eye movement in the direction of the axis of the eye in the absence of eye movement tracking in an exemplary embodiment;
FIG. 13 is an image of an OCT anterior ocular segment with eye tracking in an embodiment.
Detailed Description
The present invention is further illustrated by the following examples, which are intended to be purely exemplary and are not intended to limit the scope of the invention, as various equivalent modifications of the invention will occur to those skilled in the art upon reading the present disclosure and fall within the scope of the appended claims.
As shown in fig. 1, the three-dimensional eye tracking system of the present invention includes a pupil camera imaging module, an image recognition and analysis module, a motor control module, and an OCT system imaging module;
the pupil camera imaging module is used for acquiring pupil images in real time, the frame rate of the embodiment reaches 60 frames per second, and the higher frame rate is limited by the exposure time of the camera and the number of pixels in the horizontal/vertical direction;
the image recognition and analysis module is used for recognizing the annular light mapping points in the pupil camera image, further calculating the radius or the diameter of the annular light mapping points and the offset of the centers of the annular light mapping points relative to the optical axis of the pupil camera through fitting, the image analysis module can also continuously read at least two images in a short time, take the first image as a reference image, calculate the radius or the diameter of the annular light mapping points in the second image and the offset of the centers of the annular light mapping points from the optical axis of the pupil camera, compare the first image with the reference image, calculate the direction and the size of three-dimensional eye movement according to the calculation, and further convert the three-dimensional eye movement direction and the displacement into corresponding motor compensation;
the motor control module is used for controlling the motor to move according to the motor compensation direction and displacement calculated by the image recognition and analysis module so as to track the movement of human eyes;
the OCT imaging system is used for three-dimensional imaging of the anterior segment and/or the posterior segment of the eye. Fig. 4 is an optical path system diagram of the posterior segment OCT and the anterior segment OCT. According to the invention, OCT imaging can be respectively carried out through the OCT front-back section switching module in the figure 4 without influencing other optical paths. When the anterior segment of the eye is imaged, the anterior segment of the eye can be imaged by moving the anterior segment switching module and the posterior segment switching module of the OCT into an OCT optical path from the inside of the system. On the contrary, when the OCT imaging of the posterior segment of the eye is carried out, the OCT light path can be moved out of the OCT light path by the anterior-posterior segment switching module of the OCT in the system, and then the imaging of the posterior segment of the eye can be carried out. The operation is convenient, and the field resolution of the anterior segment OCT can be improved. In addition, the front-back section switching does not affect the optical path of the pupil camera imaging module and the optical paths of other imaging modules in the system.
The optical system of the pupil camera imaging module is shown in fig. 5, the ocular lens is a double aspheric lens, and the pupil camera imaging module and the OCT imaging module share the ocular lens and the dichroic mirror 1. In pupil camera imaging, scattered light emitted by a pupil illumination LED is incident on a cornea of a human eye, so that the brightness and contrast of the pupil are enhanced, and diffuse reflected light on the cornea is finally incident on a pupil camera imaging module through an ocular lens, the dichroic mirror 1, the dichroic mirror 2 and other lenses in a system, is finally collected by a pupil camera, and is displayed on a computer screen after being acquired by software. The pupil illumination LED is also acquired by the pupil camera imaging module under the reflection of the cornea of the human eye, and is displayed as a reflection point on the pupil camera image, and the image is shown in figure 3. When the cornea is positioned at the accurate working distance position of the ocular lens, the pupil lighting LED has clear imaging and high contrast; when away from the exact working distance, the imaging becomes blurred.
Regarding eye movement tracking in the horizontal direction, the vertical direction, and in the direction of the eye axis in the present embodiment: wherein the eye movement tracking in the horizontal direction, the vertical direction and the direction along the eye axis are completely independent of each other, can be respectively and independently applied to a one-dimensional eye movement tracking system, and can also be combined and applied to a two-dimensional and three-dimensional eye movement tracking system; where the eye axis is horizontal/vertical when combined for two-dimensional and three-dimensional eye tracking systems, parallel tracking and compensation is possible. Eye tracking in the horizontal and vertical directions differs from eye tracking in the direction of the eye axis in that the algorithms in the image recognition and analysis module differ and the compensation motors in the motor control module differ.
In this embodiment, the image recognition and analysis module calculates the size and direction of eye movement in the horizontal and vertical directions and the size and direction of eye movement in the direction of the eye axis, respectively.
The method for calculating the size and direction of the eye movement in the horizontal and vertical directions comprises the following steps: the image recognition and analysis module is used for recognizing the annular light reflecting points in the pupil camera image, and further calculating the centers of the annular light reflecting points through fitting, and further calculating the deviation of the centers of the annular light reflecting points relative to the optical axis of the pupil camera. The image analysis module continuously reads at least two images in a short time, takes the first image as a reference image, calculates the size and the direction of eye movement along the horizontal direction and the vertical direction by comparing the variation of the deviation of the centers of the annular light reflecting points of the second image and the reference image relative to the optical axis of the pupil camera, and then converts the variation into the size and the moving direction of corresponding motor compensation displacement.
The method for calculating the size and direction of the eye movement along the direction of the axis of the eye is as follows: the image recognition and analysis module is used for recognizing the annular light mapping points in the pupil camera images and further calculating the radius or the diameter of the annular light mapping points through fitting, and the image analysis module can also continuously read at least two images in a short time, take the first image as a reference image, calculate the offset of the radius or the diameter of the annular light mapping points in the second image relative to the reference image, calculate the size and the direction of eye movement along the direction of the eye axis, and further convert the size and the moving direction into the corresponding displacement size and the moving direction compensated by the motor.
First, specifically, for the size and direction of eye movement in the horizontal and vertical directions in the present example: the field of view size of pupil camera imaging module is 16mm 12mm, and single pixel size sets up to 18 um. The selected eye movement thresholds in the horizontal direction and the vertical direction are as follows: eye movement in the horizontal direction: not less than 0.5 pixels (i.e. 9um), not more than 4mm (16mm x 25%); vertical eye movement: not less than 0.5 pixels (i.e. 9um), not more than 3mm (12mm x 25%).
The center of the pupil camera image is used as the origin of coordinates, the right side of the eye movement in the horizontal direction and the upper side deviation in the vertical direction are used as positive directions, and the left side and the lower side are used as negative directions. The mathematical expression of the eye movement threshold value in the horizontal direction is intervals of [ -4mm, -9um ] and [9um, 4mm ]; the mathematical expression of the vertical eye movement size threshold is the intervals [ -3mm, -9um ] and [9um, 3mm ].
By further example, when the eye movement in the horizontal direction or the vertical direction is 1mm, the image recognition and analysis module acquires that the amount of change of the offset of the center of the annular light mapping point relative to the optical axis of the pupil camera in the current image is 1mm, and the direction is along the right side of the horizontal direction or the upper side of the vertical direction. The amount of displacement of the horizontal or vertical compensation motor is thus 1mm, the direction being the left side in the horizontal direction and/or the lower side in the vertical direction. When the eye movement in the horizontal direction and/or the vertical direction is-1 mm, the image recognition and analysis module acquires that the variation of the offset of the center of the annular light mapping point relative to the optical axis of the pupil camera in the current image is 1mm, and the direction is along the left side of the horizontal direction and/or the lower side of the vertical direction. The displacement of the horizontal or vertical compensation motor is thus 1mm, the direction being the right side in the horizontal direction and/or the upper side in the vertical direction. The imaging effect of the OCT imaging system can be seen with reference to fig. 6, 7, 9, 10, 11, and 13: as shown in fig. 6, when there is no eye movement tracking, the optical axis of the OCT optical path shifts in the horizontal direction when imaging the posterior segment of the eye caused by eye movement in the horizontal direction, causing degradation of the image quality, which is illustrated as a dark area on the right side; as shown in fig. 7, when there is no eye movement tracking, the optical axis of the OCT optical path is shifted in the vertical direction when the posterior segment of the eye is imaged due to eye movement in the vertical direction, which causes the quality of the image to be degraded, and the image is illustrated as a case where the signal intensity of the whole OCT image is significantly attenuated; as shown in fig. 9, when there is eye movement tracking, the posterior segment image is clear and is not disturbed by eye movement horizontally, vertically and along the axis of the eye. As shown in fig. 10, when there is no eye movement tracking, the anterior segment image caused by the eye movement in the horizontal direction is shifted, causing the image to be shifted to one side as a whole, and the figure is exemplified in that the eye movement is in the left direction, causing the OCT anterior segment image to be shifted to the left side; as shown in fig. 11, when there is no eye movement tracking, the anterior segment image caused by eye movement in the vertical direction is shifted, so that the left and right images of the image are not displayed completely, and the complete anterior segment tissue structure cannot be seen clearly; as shown in fig. 13, when eye movement is tracked, the anterior segment image is clear and is not disturbed by eye movement horizontally, vertically, and in the direction of the eye axis.
Then, specifically, for the size and direction of the eye movement in the direction of the axis of the eye in the present embodiment: the annular illuminating lamp consists of 12 LEDs which are arranged at equal intervals. Based on the optical diagram of fig. 2, under paraxial approximation, the relationship between the diameter of the annular reflection spot and the distance from the eye to the eyepiece can be described by the following formula:
wherein m isLIn this embodiment, fig. 2 is a diagram of the magnification of the optical system depending on the eyepiece lens and the imaging lens. RcIs the radius of the cornea of the human eye, phiringThe diameter of the annular LED, w is the distance from eyes to the ocular lens, and d is the distance from the center of the annular illuminating lamp to the vertex of the ocular lens.
In this embodiment, w is much larger than d and RcTherefore, the above formula can be further simplified as:
the change of the diameter of the annular reflection point along with the working distance can be obtained by the following formula:
note the negative sign on the right of the equation, indicating that in this embodiment, as the working distance becomes larger, the diameter of the image (i.e., the spot) formed by the ring illuminator becomes smaller; conversely, as the working distance becomes smaller, the diameter of the image (i.e., the reflection point) formed by the ring illumination lamp becomes larger. Wherein the variation Δ R ═ d ΦringThe motor moves by the corresponding displacement Δ Z — dw.
In this example, mLIs 2, phiringIs 52mm, w is 30mm, considering a normal healthy human eye RcIs 7.8 mm. When eye movements cause the eye to move 1mm away from the eyepiece (dw 1), the measured annular spot diameter changesMeasured as
The compensation displacement of the corresponding motor is-1 mm, and the negative sign indicates that the motor moving direction is the direction of decreasing working distance. When eye movement causes the human eye to approach the ocular lens by 1mm (dw ═ 1), the variation in the diameter of the annular reflection spot is measured as
The compensation displacement of the corresponding motor is Δ Z1 mm, and the positive sign indicates that the motor moving direction is a direction of increasing the working distance.
The imaging effect of the OCT imaging system can be seen with reference to fig. 8, 9, 12, and 13, respectively: as shown in fig. 8, when there is no eye movement tracking, the working distance is inaccurate when imaging the posterior segment of the eye caused by eye movement along the direction of the eye axis, which causes the OCT image to become dark on both sides, resulting in that the tissue structure of the edge cannot be seen clearly; as shown in fig. 9, when there is eye movement tracking, the posterior segment image is clear and is not disturbed by eye movement horizontally, vertically and along the axis of the eye. As shown in fig. 12, when there is no eye movement tracking, the anterior segment image caused by eye movement in the direction of the eye axis is shifted, which is exemplified by the case where the eye movement is shifted by 6mm, resulting in a higher image position, and thus the tissue structure at the cornea is not clearly seen; as shown in fig. 13, when eye movement is tracked, the anterior segment image is clear and is not disturbed by eye movement horizontally, vertically, and in the direction of the eye axis.
In this example, the results of the repeatability test of the annular spot size are shown in table 1. The diameter measurement value of the annular reflection point is obtained by respectively calculating 30-50 images continuously selected by the model eye and the human eye, so that the repeatability of the diameter of the annular reflection point is obtained. Repeatability is defined herein as one standard deviation of multiple measurements. The measurements in table 1 show that the repeatability of the model eye and the human eye can reach 1.98um and 2.16um respectively, and the accuracy is enough to track the tiny eye movement along the direction of the eye axis.
According to the repeatability tests in table 1, the threshold of the radius or diameter variation of the annular light mapping point in this embodiment can be set to 2.5um, and for eye movements greater than 2.5um along the direction of the eye axis, the solution proposed by the present invention can perform real-time eye movement tracking and compensation.
| Sample (I) | Number of images | Diameter (mm) | Repeatability (um) | Repeatability (percentage) |
| Model eye | 30 | 5.46 | 1.98 | 0.04％ |
| Human eye | 50 | 4.68 | 2.16 | 0.05％ |
Table 1: repeatability of annular light mapping spot diameter
Finally, the OCT imaging system is used to image the anterior and/or posterior segments in three dimensions: in this embodiment, OCT imaging can be performed separately without affecting other optical paths by the OCT front-back section switching module in fig. 4. When the anterior segment of the eye is imaged, the anterior segment of the eye can be imaged by moving the anterior segment switching module and the posterior segment switching module of the OCT into an OCT optical path from the inside of the system. On the contrary, when the OCT imaging of the posterior segment of the eye is carried out, the OCT light path can be moved out of the OCT light path by the anterior-posterior segment switching module of the OCT in the system, and then the imaging of the posterior segment of the eye can be carried out. The operation is convenient, and the field resolution of the anterior segment OCT can be improved. In addition, the front-back section switching does not affect the optical path of the pupil camera imaging module and the optical paths of other imaging modules in the system. As can be seen in fig. 9 and 13: as shown in fig. 9, when there is eye movement tracking, the posterior segment image is clear and not disturbed by eye movement horizontally, vertically and along the direction of the eye axis; as shown in fig. 13, when eye movement is tracked, the anterior segment image is clear and is not disturbed by eye movement horizontally, vertically, and in the direction of the eye axis.
Claims (8)
1. A three-dimensional eye movement tracking system is characterized by comprising a pupil camera imaging module, an image recognition analysis module, a motor control module and an OCT system module which are sequentially connected;
the pupil camera imaging module acquires images of pupils in real time through a pupil camera;
the image recognition and analysis module obtains the variation delta R of the radius or the diameter of the annular light mapping point in the real-time pupil camera image relative to the radius or the diameter of the annular light mapping point in the initial pupil camera image through a threshold method, further obtains the movement amount of the human eye movement along the direction of the eye axis, then converts the movement amount into the accurate working distance, and obtains the displacement delta Z which the motor should move; then obtaining the offset of the center of the annular reflection point in the real-time pupil camera image from the optical axis of the pupil camera through an image recognition and analysis module, comparing the offset with the offset of the center of the annular reflection point in the initial pupil camera image from the optical axis of the pupil camera, obtaining the offset of human eyes in the horizontal direction and the vertical direction, and converting the offset into the horizontal direction displacement delta X and the vertical direction displacement delta Y of a motor for adjusting the center alignment of the ocular lens and the human eyes;
the motor control module respectively adjusts corresponding motors according to the calculated delta X, delta Y and delta Z, so that the human eyes always keep accurate working distance under the condition of eye movement and are aligned with the centers of the ocular lenses;
the OCT system module is used for carrying out anterior segment OCT and posterior segment OCT imaging under the assistance of real-time eye movement tracking.
2. The system of claim 1, wherein the pupil camera imaging module comprises an annular pupil illumination LED, an eyepiece, an imaging lens and a camera, wherein the annular pupil illumination LED comprises at least 3 LEDs arranged at equal intervals and arranged in an annular shape.
3. A tracking method of a three-dimensional eye movement tracking system according to claim 1 or 2, comprising the steps of:
(1) carrying out imaging acquisition on an object to be detected so as to obtain at least two images;
(2) tracking the movement of the eye along the direction of the eye axis to obtain the variation delta R of the radius or the diameter of the annular light mapping point in the second image relative to the radius or the diameter of the annular light mapping point in the initial image; converting the variable quantity delta R into a displacement quantity delta Z of the movement of the motor;
(3) tracking the movement of eyes along the horizontal direction and the vertical direction, obtaining the offset of eyes in the horizontal direction and the vertical direction, and converting the offset into the horizontal direction displacement delta X and the vertical direction displacement delta Y of a motor for adjusting the center alignment of the ocular lens and the eyes;
(4) adjusting the motor to move according to the calculated delta X, delta Y and delta Z;
(5) and finally imaging through an OCT system module.
4. The tracking method of the three-dimensional eye movement tracking system according to claim 3, wherein the step (2) of tracking the movement of the eye along the direction of the eye axis comprises the following steps:
(2.1) calculating the radius or the diameter of the annular light mapping point in each image through fitting according to the annular light mapping points in the images;
(2.2) when there is eye movement, the radius or diameter of the annular light mapping point in the latter image has a variation Δ R relative to the diameter of the annular light mapping point in the former image, and when the variation Δ R exceeds a threshold, a command is sent to the motor control module to make the motor move by a corresponding displacement Δ Z.
5. The tracking method of the three-dimensional eye movement tracking system according to claim 4, wherein the step (2.2) of calculating the variation Δ R and the displacement Δ Z comprises the following steps: under paraxial approximation conditions, the diameter of the annular pupil illumination LED is:
wherein m isLIs the magnification of the optical system, RcThe radius of the cornea of the human eye, w is the distance between the eye and the ocular lens, and d is the distance between the center of the annular pupil illumination LED and the top point of the ocular lens;
the variation of the diameter of the annular reflection point along with the working distance can be obtained by calculating the derivative according to the formula:
wherein the amount of change
The motor moves by the corresponding displacement amount Δ Z ═ dw.
6. The tracking method of the three-dimensional eye movement tracking system according to claim 4, wherein the threshold value in the step (2.2) is determined by the following method: the threshold value of the size of the change of the radius or the diameter of the annular light mapping point is not less than the test repeatability of the radius or the diameter of the annular light mapping point.
7. The tracking method of the three-dimensional eye movement tracking system according to claim 3, wherein the specific steps of tracking the eye movement in the horizontal and vertical directions in the step (3) are as follows:
(3.1) taking one image as a first image, and calculating the deviation of the center of an annular light reflecting point in the first image from the optical axis of a pupil camera;
(3.2) selecting another image as a second image, and calculating the deviation of the center of the annular light reflecting point in the second image from the optical axis of the pupil camera;
(3.3) when there is eye movement, the deviation of the center of the annular light reflex point in the second image from the optical axis of the pupil camera has a variation relative to the deviation of the center of the annular light reflex point in the initial image, namely the center of the annular light reflex point in the first image from the optical axis of the pupil camera, when the variation exceeds a threshold value, the horizontal position and the vertical position are adjusted, so that the eye is always coincided with the center of the ocular lens, wherein the horizontal displacement of the movement is delta X, and the vertical displacement is delta Y.
8. The tracking method of the three-dimensional eye movement tracking system according to claim 3, wherein the specific steps of imaging by the OCT system module in the step (5) are as follows: when the anterior segment of the eye is subjected to OCT imaging, the anterior segment of the eye can be imaged by moving an OCT anterior-posterior segment switching module into an OCT optical path from the inside of the system; when the OCT imaging of the posterior segment of the eye is carried out, the OCT light path is shifted out from the OCT light path by the anterior-posterior segment switching module of the OCT imaging system from the inside of the system, and then the imaging of the posterior segment of the eye can be carried out.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110264441.2A CN113040701A (en) | 2021-03-11 | 2021-03-11 | Three-dimensional eye movement tracking system and tracking method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110264441.2A CN113040701A (en) | 2021-03-11 | 2021-03-11 | Three-dimensional eye movement tracking system and tracking method thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113040701A true CN113040701A (en) | 2021-06-29 |
Family
ID=76511388
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110264441.2A Pending CN113040701A (en) | 2021-03-11 | 2021-03-11 | Three-dimensional eye movement tracking system and tracking method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113040701A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114098632A (en) * | 2022-01-27 | 2022-03-01 | 北京鹰瞳科技发展股份有限公司 | Method for controlling a motor in a fundus camera and related product |
| CN114359254A (en) * | 2022-01-13 | 2022-04-15 | 视微影像(河南)科技有限公司 | Anterior segment OCT image light beam detection and positioning tracking method and system |
| CN114373216A (en) * | 2021-12-07 | 2022-04-19 | 图湃(北京)医疗科技有限公司 | Eye movement tracking method, device, equipment and storage medium for anterior segment OCTA |
Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102058391A (en) * | 2009-11-17 | 2011-05-18 | 佳能株式会社 | Apparatus and method for imaging optical coherence tomographic image |
| US20120274783A1 (en) * | 2011-04-29 | 2012-11-01 | Optovue, Inc. | Imaging with real-time tracking using optical coherence tomography |
| CN103025229A (en) * | 2010-04-29 | 2013-04-03 | 麻省理工学院 | Method and apparatus for motion correction and image enhancement for optical coherence tomography |
| CN103096785A (en) * | 2010-03-12 | 2013-05-08 | 佳能株式会社 | Ophthalmologic apparatus and control method for the same |
| CN103961058A (en) * | 2013-01-31 | 2014-08-06 | 佳能株式会社 | Optical coherence tomographic imaging apparatus and method for controlling the same |
| CN104958061A (en) * | 2015-07-28 | 2015-10-07 | 北京信息科技大学 | Fundus OCT imaging method utilizing three-dimensional imaging of binocular stereo vision and system thereof |
| US20150294147A1 (en) * | 2013-05-29 | 2015-10-15 | Wavelight Gmbh | Apparatus for optical coherence tomography of an eye and method for optical coherence tomography of an eye |
| CN105608675A (en) * | 2015-12-18 | 2016-05-25 | 天津迈达医学科技股份有限公司 | Fundus tissue OCT image motion artifact correction method |
| US20160338589A1 (en) * | 2014-01-30 | 2016-11-24 | Oscar M. Carrasco-Zevallos | Systems and methods for eye tracking for motion corrected ophthalmic optical coherenece tomography |
| CN107609516A (en) * | 2017-09-13 | 2018-01-19 | 重庆爱威视科技有限公司 | Adaptive eye moves method for tracing |
| CN108992035A (en) * | 2018-06-08 | 2018-12-14 | 云南大学 | The compensation method of blinkpunkt positional shift in a kind of tracking of eye movement |
| CN109938691A (en) * | 2017-12-21 | 2019-06-28 | 清弘生医股份有限公司 | The continuity measurement method of eyeball and the continuity measuring device of eyeball |
| CN110604543A (en) * | 2018-06-15 | 2019-12-24 | 株式会社拓普康 | Ophthalmic device |
| CN110680273A (en) * | 2019-11-06 | 2020-01-14 | 深圳市斯尔顿科技有限公司 | Ophthalmic detection system and method |
| CN112155511A (en) * | 2020-09-30 | 2021-01-01 | 广东唯仁医疗科技有限公司 | Method for compensating human eye shake in OCT (optical coherence tomography) acquisition process based on deep learning |
| CN112168132A (en) * | 2020-11-09 | 2021-01-05 | 苏州大学 | Method for performing fundus refraction compensation judgment and imaging optimization by using OCT (optical coherence tomography) signals |
| CN112244756A (en) * | 2020-10-19 | 2021-01-22 | 深圳市斯尔顿科技有限公司 | Multifunctional full-automatic ophthalmic measuring method and system |
-
2021
- 2021-03-11 CN CN202110264441.2A patent/CN113040701A/en active Pending
Patent Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102058391A (en) * | 2009-11-17 | 2011-05-18 | 佳能株式会社 | Apparatus and method for imaging optical coherence tomographic image |
| CN103096785A (en) * | 2010-03-12 | 2013-05-08 | 佳能株式会社 | Ophthalmologic apparatus and control method for the same |
| CN103025229A (en) * | 2010-04-29 | 2013-04-03 | 麻省理工学院 | Method and apparatus for motion correction and image enhancement for optical coherence tomography |
| US20120274783A1 (en) * | 2011-04-29 | 2012-11-01 | Optovue, Inc. | Imaging with real-time tracking using optical coherence tomography |
| CN103961058A (en) * | 2013-01-31 | 2014-08-06 | 佳能株式会社 | Optical coherence tomographic imaging apparatus and method for controlling the same |
| US20150294147A1 (en) * | 2013-05-29 | 2015-10-15 | Wavelight Gmbh | Apparatus for optical coherence tomography of an eye and method for optical coherence tomography of an eye |
| US20160338589A1 (en) * | 2014-01-30 | 2016-11-24 | Oscar M. Carrasco-Zevallos | Systems and methods for eye tracking for motion corrected ophthalmic optical coherenece tomography |
| CN104958061A (en) * | 2015-07-28 | 2015-10-07 | 北京信息科技大学 | Fundus OCT imaging method utilizing three-dimensional imaging of binocular stereo vision and system thereof |
| CN105608675A (en) * | 2015-12-18 | 2016-05-25 | 天津迈达医学科技股份有限公司 | Fundus tissue OCT image motion artifact correction method |
| CN107609516A (en) * | 2017-09-13 | 2018-01-19 | 重庆爱威视科技有限公司 | Adaptive eye moves method for tracing |
| CN109938691A (en) * | 2017-12-21 | 2019-06-28 | 清弘生医股份有限公司 | The continuity measurement method of eyeball and the continuity measuring device of eyeball |
| CN108992035A (en) * | 2018-06-08 | 2018-12-14 | 云南大学 | The compensation method of blinkpunkt positional shift in a kind of tracking of eye movement |
| CN110604543A (en) * | 2018-06-15 | 2019-12-24 | 株式会社拓普康 | Ophthalmic device |
| CN110680273A (en) * | 2019-11-06 | 2020-01-14 | 深圳市斯尔顿科技有限公司 | Ophthalmic detection system and method |
| CN112155511A (en) * | 2020-09-30 | 2021-01-01 | 广东唯仁医疗科技有限公司 | Method for compensating human eye shake in OCT (optical coherence tomography) acquisition process based on deep learning |
| CN112244756A (en) * | 2020-10-19 | 2021-01-22 | 深圳市斯尔顿科技有限公司 | Multifunctional full-automatic ophthalmic measuring method and system |
| CN112168132A (en) * | 2020-11-09 | 2021-01-05 | 苏州大学 | Method for performing fundus refraction compensation judgment and imaging optimization by using OCT (optical coherence tomography) signals |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114373216A (en) * | 2021-12-07 | 2022-04-19 | 图湃(北京)医疗科技有限公司 | Eye movement tracking method, device, equipment and storage medium for anterior segment OCTA |
| CN114359254A (en) * | 2022-01-13 | 2022-04-15 | 视微影像(河南)科技有限公司 | Anterior segment OCT image light beam detection and positioning tracking method and system |
| CN114359254B (en) * | 2022-01-13 | 2023-12-19 | 视微影像(河南)科技有限公司 | Method and system for detecting, positioning and tracking OCT (optical coherence tomography) image light column of anterior segment of eye |
| CN114098632A (en) * | 2022-01-27 | 2022-03-01 | 北京鹰瞳科技发展股份有限公司 | Method for controlling a motor in a fundus camera and related product |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10660519B2 (en) | Systems and methods for eye tracking for motion corrected ophthalmic optical coherence tomography | |
| Savini et al. | Comparison of anterior segment measurements by 3 Scheimpflug tomographers and 1 Placido corneal topographer | |
| CN113040701A (en) | Three-dimensional eye movement tracking system and tracking method thereof | |
| JP5743380B2 (en) | Optical tomographic imaging apparatus and optical tomographic imaging method | |
| JP5728302B2 (en) | Ophthalmic apparatus, ophthalmic system, ophthalmic apparatus control method, and program for the control method | |
| JP6966737B2 (en) | Small removable microscope-mounted keratometer for cataract surgery | |
| US9808152B2 (en) | Anterior ocular segment optical coherence tomographic imaging device and anterior ocular segment optical coherence tomographic imaging method | |
| US20110234975A1 (en) | Optical tomographic imaging apparatus and imaging method for optical tomographic image | |
| US10893799B2 (en) | Method for determining the topography of the cornea of an eye | |
| US10123699B2 (en) | Ophthalmologic apparatus and imaging method | |
| JP2022075772A (en) | Ophthalmologic apparatus | |
| CN110755032A (en) | Ophthalmic detection system and method | |
| CN107730545B (en) | Optical imaging method and system for dynamically eliminating ghost image | |
| WO2021153087A1 (en) | Ophthalmic device, control method thereof, and storage medium | |
| CN112043233B (en) | Human eye scattering objective measuring instrument capable of eliminating human eye aberration influence | |
| JP2020039430A (en) | Image processing device, image processing method and program | |
| CN105411523B (en) | Cornea image processing method | |
| CN110680273A (en) | Ophthalmic detection system and method | |
| US10674902B2 (en) | Information processing apparatus, operation method thereof, and computer program | |
| CN113040700A (en) | Eye movement tracking system and tracking method thereof | |
| JP6793416B2 (en) | Anterior segment optical interference tomography device and anterior segment optical interference tomography method | |
| JP6664960B2 (en) | Method for implementing OCT and other imaging of the eye | |
| JP7297302B2 (en) | ophthalmic equipment | |
| WO2024004455A1 (en) | Opthalmic information processing device, opthalmic device, opthalmic information processing method, and program | |
| US20210145274A1 (en) | Method and Apparatus for Parallel Optical Coherence Tomographic Funduscope |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |