CN113040701B - Three-dimensional eye movement tracking system and tracking method thereof - Google Patents
Three-dimensional eye movement tracking system and tracking method thereof Download PDFInfo
- Publication number
- CN113040701B CN113040701B CN202110264441.2A CN202110264441A CN113040701B CN 113040701 B CN113040701 B CN 113040701B CN 202110264441 A CN202110264441 A CN 202110264441A CN 113040701 B CN113040701 B CN 113040701B
- Authority
- CN
- China
- Prior art keywords
- eye
- image
- annular
- eye movement
- oct
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000004424 eye movement Effects 0.000 title claims abstract description 111
- 238000000034 method Methods 0.000 title claims abstract description 21
- 210000001747 pupil Anatomy 0.000 claims abstract description 74
- 238000003384 imaging method Methods 0.000 claims abstract description 73
- 238000006073 displacement reaction Methods 0.000 claims abstract description 32
- 238000004458 analytical method Methods 0.000 claims abstract description 17
- 230000003287 optical effect Effects 0.000 claims description 41
- 238000005286 illumination Methods 0.000 claims description 14
- 238000013507 mapping Methods 0.000 claims description 12
- 210000004087 cornea Anatomy 0.000 claims description 10
- CLOMYZFHNHFSIQ-UHFFFAOYSA-N clonixin Chemical compound CC1=C(Cl)C=CC=C1NC1=NC=CC=C1C(O)=O CLOMYZFHNHFSIQ-UHFFFAOYSA-N 0.000 claims description 4
- 238000012360 testing method Methods 0.000 claims description 4
- 238000003745 diagnosis Methods 0.000 abstract description 4
- 230000001105 regulatory effect Effects 0.000 abstract 1
- 238000012014 optical coherence tomography Methods 0.000 description 77
- 238000005516 engineering process Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000005259 measurement Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000003759 clinical diagnosis Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000010191 image analysis Methods 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 241000501754 Astronotus ocellatus Species 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000002526 effect on cardiovascular system Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000001525 retina Anatomy 0.000 description 1
- 230000002207 retinal effect Effects 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
- A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
- A61B3/102—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for optical coherence tomography [OCT]
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
- A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
- A61B3/113—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for determining or recording eye movement
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
- A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
- A61B3/14—Arrangements specially adapted for eye photography
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Animal Behavior & Ethology (AREA)
- Biophysics (AREA)
- Ophthalmology & Optometry (AREA)
- Veterinary Medicine (AREA)
- Public Health (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Human Computer Interaction (AREA)
- Eye Examination Apparatus (AREA)
Abstract
The invention discloses a three-dimensional eye movement tracking system, which comprises a pupil camera imaging module, an image recognition analysis module, a motor control module and an OCT system module which are connected in sequence; the tracking method of the system comprises the following steps: firstly, continuously imaging an object to be detected in real time through a pupil camera module; when an object to be detected, namely an eye, moves along the eye axis or transversely, namely horizontally or longitudinally, namely vertically, the eye movement size along the eye axis, horizontally and vertically can be calculated through the image recognition analysis module, and then the displacement amounts delta Z, delta X and delta Y of corresponding motors in the three directions are calculated; then, the corresponding motor is regulated to move according to the calculated motor displacement, so that the object to be measured always keeps an accurate working distance and is aligned with the center of the ocular lens; and performing three-dimensional imaging on the anterior ocular segment or the posterior ocular segment of the object to be detected through the OCT system module. The invention can track three-dimensional eye movement in real time and calculate the size and direction of the eye movement, and real-time compensation is carried out on the eye movement in diagnosis through the motor control module, thereby realizing high-quality system imaging, reducing the complexity of a diagnosis system and improving the success rate and efficiency of diagnosis.
Description
Technical Field
The invention belongs to the technical field of imaging devices, and particularly relates to a three-dimensional eye movement tracking system and a tracking method thereof.
Background
The Optical Coherence Tomography (OCT) technology is a three-dimensional imaging technology, has the advantages of high sensitivity, high imaging speed, high resolution and the like, and is widely applied to the fields of ophthalmology (including anterior ocular segment and posterior ocular segment), cardiovascular department, dermatological department and the like. An important advance in OCT technology has been the faster and faster scan rates over the last two decades, ranging from traditional time domain OCT to frequency domain based OCT (SD-OCT) to swept source technology based OCT (SS-OCT), with the speed of OCT imaging being greatly evolving. The imaging speed is high, and the imaging method has the unique clinical application advantages: the two-dimensional and/or three-dimensional scanning can be completed in a shorter time, so that the sensitivity of eye movement of a person to imaging quality in imaging is reduced, the clinical diagnosis and treatment efficiency is greatly improved, and the imaging device is popular with clinicians.
The influence of eye movement on OCT imaging is mainly reflected in the influence on the imaging precision and repeatability, and the diagnosis and treatment efficiency is clinically reduced. Further reducing the impact of eye movement on OCT imaging quality, on the one hand, can further increase the sweep rate of OCT, for example, SS-OCT VG200 axial scan rate for which microimaging has been used clinically has reached 20 tens of thousands of times per second (reference ：Yang J,Chen Y.Vitreoretinal Traction with Vitreoschisis Using OCT.Ophthalmol Retina.2019Nov;3(11):961.doi:10.1016/j.oret.2019.07.003.PMID:31699311), in this system, which still requires eye movement tracking techniques; laboratory platforms have OCT of megahertz scale, for example, OCT axial scan rate based on fourier domain mode-locked laser as disclosed in reference Klein T,Wieser W,Eigenwillig CM,Biedermann BR,Huber R.Megahertz OCT for ultrawide-field retinal imaging with a 1050nm Fourier domain mode-locked laser.Opt Express.2011;19:3044–62.Available:http://www.ncbi.nlm.nih.gov/pubmed/21369128doi:10.1364/OE.19.003044PMID:21369128 has reached 1.37 mega per second, but such systems are currently relatively complex and difficult to apply to clinic.
The eye movement tracking technology based on the pupil camera is widely applied to a three-dimensional eye movement tracking system with the advantage of low cost. For example, literature Oscar Carrasco-Zevallos,Derek Nankivil,Brenton Keller,Christian Viehland,Brandon J.Lujan,and Joseph A.Izatt,Pupil tracking optical coherence tomography for precise control of pupil entry position,Biomed.Opt.Express 6,3405-3419(2015); discloses a two-dimensional eye movement tracking based on a pupil camera, which can effectively track eye movements in horizontal and/or vertical directions, and further compensate for posterior segment OCT; the pupil camera-based two-dimensional eye movement tracking disclosed in document Carrasco-Zevallos OM,Nankivil D,Viehland C,Keller B,Izatt JA.Pupil Tracking for Real-Time Motion Corrected Anterior Segment Optical Coherence Tomography.PLoS One.2016;11(8):e0162015.Published 2016Aug 30.doi:10.1371/journal.pone.0162015 tracks eye movement in the horizontal and/or vertical directions, and compensates for anterior segment OCT. However, these methods are all to compensate the magnitude of eye movement to the driving waveform of the OCT scanning galvanometer, are limited by the compensated range size and frame rate, cannot adjust the center alignment of the eye and the eyepiece in real time based on the eye movement, and do not track the eye movement in the direction of the eye axis in real time.
Disclosure of Invention
The invention aims to: in view of the above drawbacks, the present invention provides a three-dimensional eye movement tracking system and a tracking method thereof, which can effectively reduce the influence of eye movement on the imaging quality of OCT of anterior segment and OCT of posterior segment.
The technical scheme is as follows: the invention provides a three-dimensional eye movement tracking system and a tracking method thereof, wherein the three-dimensional eye movement tracking system comprises a pupil camera imaging module, an image recognition analysis module, a motor control module and an OCT system module which are connected in sequence;
The pupil camera imaging module acquires images of pupils in real time through a pupil camera;
The image recognition analysis module obtains the change delta R of the radius or the diameter of the annular reflecting point in the real-time pupil camera image relative to the radius or the diameter of the annular reflecting point in the initial pupil camera image through a threshold method, further obtains the movement amount of the eye movement of a person along the direction of the eye axis, converts the movement amount into an accurate working distance, and obtains the displacement delta Z of the motor which should move; in parallel, the offset of the center of the annular reflecting point in the real-time pupil camera image from the pupil camera optical axis is obtained through the image recognition analysis module, and compared with the offset of the center of the annular reflecting point in the initial pupil camera image from the pupil camera optical axis, the offset of human eyes in the horizontal and vertical directions is obtained, and the displacement deltaX and deltaY of the horizontal and vertical direction compensation motor are converted according to the offset;
The motor control module respectively adjusts corresponding motors according to the calculated delta X, delta Y and delta Z, so that the eyes always keep an accurate working distance and are aligned with the center of the ocular lens under the condition of eye movement;
The OCT system module performs OCT imaging of anterior ocular segment and OCT imaging of posterior ocular segment with the assistance of real-time eye movement tracking.
Further, the pupil camera imaging module is composed of annular pupil illumination LEDs, an ocular, an imaging lens and a camera, wherein the annular pupil illumination LEDs are composed of at least 3 LEDs which are arranged at equal intervals and are arranged in an annular shape.
A tracking method of the three-dimensional eye movement tracking system as described above, comprising the steps of:
(1) Imaging and collecting the object to be detected so as to obtain at least two images;
(2) Tracking the movement of the eyes along the eye axis direction to obtain the variation delta R of the radius or the diameter of the annular reflecting point in the second image relative to the radius or the diameter of the annular reflecting point in the initial image; converting the variable delta R into a displacement delta Z of motor movement;
(3) Tracking the movement of eyes along the horizontal and vertical directions to obtain the offset of the eyes in the horizontal and vertical directions, and converting the offset into the displacement delta X of the motor for adjusting the alignment of the centers of the ocular lens and the eyes and the displacement delta Y of the motor in the vertical direction;
(4) Adjusting the motor to move according to the calculated delta X, delta Y and delta Z;
(5) Finally imaging is performed by the OCT system module.
Further, the specific steps of tracking the movement of the eye along the axial direction in the step (2) are as follows:
(2.1) calculating the radius or diameter of the annular light reflecting points in each image by fitting according to the annular light reflecting points in the images;
(2.2) when there is eye movement, the radius or diameter of the annular reflecting spot in the latter image will have a change amount DeltaR relative to the diameter of the annular reflecting spot in the former image, when the change amount DeltaR exceeds the threshold value, a command is sent to the motor control module to make the motor move by a corresponding displacement DeltaZ.
Further, the specific steps of calculating the variation Δr and the displacement Δz in the step (2.2) are as follows: under paraxial approximation, the relationship between annular spot diameter and eye-to-eyepiece distance can be described by the following formula:
Wherein m L is the magnification of the optical system, R c is the cornea radius of the human eye, phi ring is the diameter of the annular LED, w is the distance between the eye and the ocular lens, and d is the distance between the center of the annular pupil illumination LED and the vertex of the ocular lens;
the change of the annular light mapping spot diameter along with the working distance can be obtained by the derivative of the above formula:
Wherein the amount of change The motor moves by a corresponding displacement Δz= -dw.
Further, the method for determining the threshold in the step (2.2) is as follows: the radius or diameter variation size threshold of the annular light reflecting spot is not less than the test repeatability of the radius or diameter of the annular light reflecting spot.
Further, the specific steps of tracking the movement of the eye in the horizontal and vertical directions in the step (3) are as follows:
(3.1) taking one of the images as a first image, and calculating the offset of the center of the annular reflecting point in the first image from the optical axis of the pupil camera;
(3.2) selecting another image as a second image, and calculating the offset of the center of the annular reflecting point in the second image from the optical axis of the pupil camera;
And (3.3) when the eyes move, the offset of the center of the annular light reflecting spot in the second image from the optical axis of the pupil camera is changed relative to the offset of the center of the annular light reflecting spot in the initial image, namely the first image from the optical axis of the pupil camera, when the change exceeds a threshold value, the horizontal and vertical positions are adjusted so that the eyes always coincide with the center of the ocular lens, wherein the horizontal displacement of the movement is delta X, and the vertical displacement of the movement is delta Y.
Further, the specific steps of imaging by the OCT system module in the step (5) are as follows: when OCT imaging is carried out on the anterior segment of the eye, the OCT anterior-posterior segment switching module is moved into an OCT light path from the inside of the system, so that the anterior segment of the eye can be imaged; when OCT imaging of the posterior segment of the eye is carried out, the OCT anterior-posterior segment switching module is moved out of the OCT optical path from the system, so that the posterior segment of the eye can be imaged.
The invention adopts the technical scheme and has the following beneficial effects:
The three-dimensional eye movement tracking system can track and compensate three-dimensional eye movement in real time, overcomes the defect that the two-dimensional eye movement tracking system cannot track eye movement in the eye axis direction, increases the size range of eye movement tracking in the horizontal and vertical directions, and is better applied to clinical diagnosis systems. The three-dimensional eye movement tracking system based on the pupil camera has the advantages that the light path design can be decoupled from other imaging modules, and the three-dimensional eye movement tracking system is simultaneously applied to the OCT system of the anterior segment and the posterior segment of the eye, so that the defect that the three-dimensional eye movement tracking system based on the SLO can only be applied to the OCT system of the posterior segment of the eye is overcome. In addition, compared with SLO imaging technology, the three-dimensional eye tracking system based on the pupil camera has the advantages of higher frame rate and lower cost, and therefore has the potential of wider application.
Drawings
FIG. 1 is a schematic diagram of a system architecture of the present invention;
FIG. 2 is a schematic diagram of the optical path of an annular illumination LED imaged by a lens after reflection from the cornea of an eye in an exemplary embodiment;
FIG. 3 is a 360 degree annular LED image of an embodiment of the annular illumination LED after reflection from the cornea of the eye;
FIG. 4 is a diagram of an optical system for posterior segment OCT imaging and anterior segment OCT imaging as used in the exemplary embodiment;
FIG. 5 is a diagram of pupil camera imaging optics used in an embodiment;
fig. 6 is an image of an OCT posterior segment of an eye with horizontal eye movement without eye movement tracking according to an embodiment:
FIG. 7 is an image of the OCT posterior segment of an eye resulting from vertical eye movement without eye movement tracking in an embodiment;
FIG. 8 is an image of the posterior segment of an OCT eye resulting from eye movement along the axis of the eye without eye movement tracking in an embodiment;
FIG. 9 is an image of an OCT posterior segment with eye tracking in an embodiment;
FIG. 10 is an image of an OCT anterior ocular segment imaged by horizontal eye movement without eye movement tracking in an embodiment;
FIG. 11 is an image of an OCT anterior ocular segment resulting from vertical eye movement without eye movement tracking in an embodiment;
FIG. 12 is an image of an OCT anterior ocular segment resulting from eye movement along the ocular axis without eye tracking in an embodiment;
Fig. 13 is an image of an OCT anterior ocular segment with eye tracking in an embodiment.
Detailed Description
The present application is further illustrated below in conjunction with specific embodiments, it being understood that these embodiments are meant to be illustrative of the application and not limiting the scope of the application, and that modifications of the application, which are equivalent to those skilled in the art to which the application pertains, fall within the scope of the application defined in the appended claims after reading the application.
As shown in fig. 1, the three-dimensional eye tracking system of the present invention includes a pupil camera imaging module, an image recognition and analysis module, a motor control module, and an OCT system imaging module;
The pupil camera imaging module is used for acquiring pupil images in real time, the frame rate of the embodiment reaches 60 frames per second, and the higher frame rate is limited by the exposure time of the camera and the number of pixels in the horizontal/vertical direction;
The image recognition analysis module is used for recognizing annular light mapping points in the pupil camera image, calculating the radius or the diameter of the annular light mapping points and the offset of the center of the annular light mapping points relative to the pupil camera optical axis through fitting, continuously reading at least two images in a short time, taking a first image as a reference image, calculating the radius or the diameter of the annular light mapping points and the offset of the center of the annular light mapping points from the pupil camera optical axis in a second image, comparing the radius or the diameter of the annular light mapping points with the reference image, calculating the direction and the size of three-dimensional eye movement according to the comparison result, and converting the calculation result into the corresponding motor-compensated movement direction and displacement;
The motor control module is used for controlling the motor to move according to the motor compensation direction and the displacement calculated by the image recognition analysis module so as to track the movement of the human eyes;
The OCT imaging system is used to three-dimensionally image the anterior and/or posterior segments of the eye. Wherein fig. 4 is a diagram of the optical path system of posterior segment OCT and anterior segment OCT. The invention can respectively perform OCT imaging through the OCT front-back section switching module in fig. 4 without affecting other light paths. When OCT imaging of the anterior ocular segment is carried out, the OCT anterior-posterior segment switching module is moved into the OCT optical path from the inside of the system, so that the anterior ocular segment can be imaged. On the contrary, when OCT imaging of the posterior segment of the eye is carried out, the OCT anterior-posterior segment switching module is moved out of the OCT optical path from the inside of the system, so that the posterior segment of the eye can be imaged. The operation is convenient, and the visual field resolution of the anterior ocular segment OCT can be improved. In addition, the front-to-back section switching does not affect the pupil camera imaging module optical path and the optical paths of other imaging modules in the system.
The optical system of the pupil camera imaging module is shown in fig. 5, the ocular lens is a double aspheric lens, and the pupil camera imaging module and the OCT imaging module share the ocular lens and the dichroic mirror 1. In pupil camera imaging, scattered light emitted by a pupil illumination LED is incident on the cornea of a human eye, so that the brightness and contrast of the pupil are enhanced, and diffuse reflection light on the cornea passes through an ocular lens, a dichroic mirror 1, a dichroic mirror 2 and other lenses in the system, finally enters a pupil camera imaging module, is collected by a pupil camera, is acquired by software, and is displayed on a computer screen. The pupil illumination LED is also acquired by the pupil camera imaging module under the reflection of the cornea of the human eye, and is displayed as a reflecting point on the pupil camera image, and the image is shown in fig. 3. When the cornea is positioned at the accurate working distance position of the ocular lens, the pupil illumination LED images clearly, and the contrast ratio is high; when away from the exact working distance, the imaging blurs.
In this embodiment, the following is performed with respect to the eye movement in the horizontal direction, the vertical direction, and the eye axis direction: the eye movement tracking in the horizontal direction, the vertical direction and the eye axis direction are completely independent of each other, and can be respectively and independently applied to a one-dimensional eye movement tracking system and can be combined to a two-dimensional and three-dimensional eye movement tracking system; wherein the eye axis and horizontal/vertical can be tracked and compensated in parallel when the combination is applied to two-dimensional and three-dimensional eye movement tracking systems. Eye tracking in the horizontal and vertical directions differs from eye tracking in the eye axis direction in that the algorithms in the image recognition analysis module differ and in that the compensation motors in the motor control module differ.
In this embodiment, the image recognition analysis module calculates the magnitude and direction of eye movement in the horizontal and vertical directions and the magnitude and direction of eye movement in the eye axis direction, respectively.
The method for calculating the size and direction of eye movement in the horizontal and vertical directions is as follows: the image recognition analysis module is used for recognizing annular light reflecting points in the pupil camera image, and further calculating the center of the annular light reflecting points through fitting, and further calculating the offset of the center of the annular light reflecting points relative to the optical axis of the pupil camera. The image analysis module continuously reads at least two images in a short time, takes a first image as a reference image, calculates the size and direction of eye movement along the horizontal and vertical directions by comparing the variation of the offset of the center of the annular reflecting point of the second image and the reference image relative to the optical axis of the pupil camera, and converts the size and the direction of eye movement into the corresponding motor-compensated displacement size and movement direction.
The method for calculating the magnitude and direction of eye movement along the eye axis direction is as follows: the image recognition analysis module is used for recognizing annular light mapping points in the pupil camera image, further calculating the radius or the diameter of the annular light mapping points through fitting, the image analysis module can also continuously read at least two images in a short time, the first image is taken as a reference image, the offset of the radius or the diameter of the annular light mapping points in the second image relative to the reference image is calculated, the size and the direction of eye movement along the eye axis direction are calculated, and further the displacement size and the movement direction of corresponding motor compensation are converted.
First, in particular, in this example, for the magnitude and direction of eye movement in the horizontal and vertical directions: the field of view of the pupil camera imaging module is 16mm x 12mm, and the single pixel size is set to 18um x 18um. The eye movement threshold values in the selected horizontal direction and the vertical direction are as follows: eye movement size in horizontal direction: no less than 0.5 pixels (i.e., 9 um), no more than 4mm (16 mm x 25%); vertical eye movement size: not less than 0.5 pixels (i.e., 9 um) and not more than 3mm (12 mm x 25%).
The center of the pupil camera image is used as the origin of coordinates, the right side of the eye movement in the horizontal direction and the upper side in the vertical direction are offset to be positive directions, and the left side and the lower side are negative directions. The mathematical expression of the horizontal eye movement magnitude threshold is the interval [ -4mm, -9um ] and [9um,4mm ]; the mathematical expression of the vertical eye movement magnitude threshold is the interval [ -3mm, -9um ] and [9um,3mm ].
Further by way of example, when the eye movement in the horizontal direction or the vertical direction is 1mm, the image recognition analysis module acquires that the amount of change in the offset of the annular luminous spot center from the pupil camera optical axis in the current image is 1mm, the direction being on the right side in the horizontal direction or on the upper side in the vertical direction. The displacement of the horizontal or vertical direction compensation motor is 1mm, and the direction is left side along the horizontal direction and/or lower side along the vertical direction. When the eye movement in the horizontal direction and/or the vertical direction is-1 mm, the image recognition analysis module acquires that the variation of the deviation of the center of the annular reflecting point in the current image relative to the optical axis of the pupil camera is 1mm, and the direction is along the left side in the horizontal direction and/or the lower side in the vertical direction. The displacement amount of the movement of the horizontal or vertical direction compensation motor is 1mm, and the direction is the right side along the horizontal direction and/or the upper side along the vertical direction. The imaging effect by OCT imaging systems can be seen with reference to fig. 6, 7,9, 10, 11, 13, respectively: as shown in fig. 6, when there is no eye movement tracking, the optical axis of the OCT optical path is shifted in the horizontal direction when the posterior segment of the eye is imaged due to the eye movement in the horizontal direction, resulting in degradation of the image quality, which is illustrated as a dark area on the right side; as shown in fig. 7, when there is no eye movement tracking, the optical axis of the OCT optical path shifts in the vertical direction when the posterior segment of the eye is imaged due to the eye movement in the vertical direction, resulting in a decrease in the quality of the image, which is exemplified by a significant attenuation in the signal intensity of the OCT whole image; as shown in fig. 9, when there is eye movement tracking, the posterior segment image is clear and is not disturbed by eye movement in the horizontal, vertical and eye axis directions. As shown in fig. 10, when there is no eye movement tracking, the anterior ocular segment image due to the horizontal eye movement is shifted, causing the image to be shifted to one side as a whole, and the image is exemplified as eye movement in the left direction, causing the OCT anterior ocular segment image to be shifted to the left side; as shown in fig. 11, when there is no eye movement tracking, the anterior ocular segment image caused by the eye movement in the vertical direction is shifted, so that the left and right images of the image are not displayed fully, and the complete anterior ocular segment tissue structure cannot be seen clearly; as shown in fig. 13, when there is eye movement tracking, the anterior ocular segment image is clear and is not disturbed by eye movement in the horizontal, vertical, and axial directions.
Then, specifically, regarding the magnitude and direction of eye movement along the eye axis direction in the present embodiment: the annular lighting lamp consists of 12 LEDs which are arranged at equal intervals. Based on the optical schematic of fig. 2, the relationship between the annular spot diameter and the eye-to-eyepiece distance under paraxial approximation can be described by the following formula:
Where m L is the magnification of the optical system, in this embodiment fig. 2, depending on the ocular lens and imaging lens. R c is the cornea radius of human eye, phi ring is the diameter of the annular LED, w is the distance from the eye to the ocular, and d is the distance from the center of the annular illumination lamp to the apex of the ocular.
In this embodiment, w is much larger than d and R c, so the above formula can be further simplified to:
the change of the annular mapping spot diameter along with the working distance can be obtained by the following steps:
Note that the negative sign on the right of the equation indicates that in this embodiment, as the working distance becomes larger, the diameter of the image (i.e., the luminous spot) formed by the ring-shaped illumination lamp becomes smaller; conversely, as the working distance becomes smaller, the diameter of the image (i.e., the luminous spot) formed by the annular illumination lamp becomes larger. Wherein the amount of change The motor moves by a corresponding displacement Δz= -dw.
In this example, m L is 2, Φ ring is 52mm, w is 30mm, and R c is 7.8mm for a normal healthy human eye. When eye movement causes the human eye to be away from the ocular lens by 1mm (dw=1), the measured annular luminous spot diameter variation isThe compensation displacement of the corresponding motor is deltaz= -1mm, and the negative sign indicates that the motor movement direction is the direction of decreasing working distance. When eye movement causes the human eye to approach the eyepiece by 1mm (dw= -1), the measured annular spot diameter variation is/>The compensation displacement of the corresponding motor is Δz=1 mm, and the positive sign indicates that the motor movement direction is the direction of increasing the working distance.
The imaging effect by OCT imaging systems can be seen with reference to fig. 8, 9, 12, 13, respectively: as shown in fig. 8, when there is no eye tracking, the working distance is inaccurate when the posterior segment of the eye is imaged due to eye tracking along the direction of the eye axis, so that the OCT image appears dark at both sides, and the tissue structure of the edge cannot be seen clearly; as shown in fig. 9, when there is eye movement tracking, the posterior segment image is clear and is not disturbed by eye movement in the horizontal, vertical and eye axis directions. As shown in fig. 12, when there is no eye movement tracking, the anterior segment image is deviated due to eye movement along the eye axis direction, the image is deviated by 6mm, and the image position is higher, so that the tissue structure at the cornea cannot be seen clearly; as shown in fig. 13, when there is eye movement tracking, the anterior ocular segment image is clear and is not disturbed by eye movement in the horizontal, vertical, and axial directions.
In this example, the repeatability test results of the annular spot diameters are shown in Table 1. 30-50 images are selected continuously for model eyes and human eyes respectively, and annular light reflecting spot diameter measurement values are obtained through calculation respectively, so that the repeatability of the annular light reflecting spot diameter is obtained. Repeatability is defined herein as one standard deviation of the multiple measurements. The measurements of table 1 show that the repeatability of the model eye and the human eye can be up to 1.98um and 2.16um, respectively, with sufficient accuracy to track micro-eye movements along the eye axis.
According to the repeatability test of table 1, the threshold of the change of the radius or diameter of the annular light reflecting spot in this embodiment can be set to 2.5um, and for eye movements greater than 2.5um along the eye axis, the scheme provided by the invention can perform real-time eye movement tracking and compensation.
| Sample of | Number of images | Diameter (mm) | Repeatability (um) | Repeatability (percent) |
| Model eye | 30 | 5.46 | 1.98 | 0.04％ |
| Human eyes | 50 | 4.68 | 2.16 | 0.05％ |
Table 1: repeatability of annular photosite diameter
Finally, the OCT imaging system is used to three-dimensionally image the anterior and/or posterior segments of the eye: in this embodiment, by the OCT front-back section switching module in fig. 4, OCT imaging can be performed separately without affecting other optical paths. When OCT imaging of the anterior ocular segment is carried out, the OCT anterior-posterior segment switching module is moved into the OCT optical path from the inside of the system, so that the anterior ocular segment can be imaged. On the contrary, when OCT imaging of the posterior segment of the eye is carried out, the OCT anterior-posterior segment switching module is moved out of the OCT optical path from the inside of the system, so that the posterior segment of the eye can be imaged. The operation is convenient, and the visual field resolution of the anterior ocular segment OCT can be improved. In addition, the front-to-back section switching does not affect the pupil camera imaging module optical path and the optical paths of other imaging modules in the system. As can be seen in fig. 9 and 13: as shown in fig. 9, when there is eye movement tracking, the posterior segment image is clear and is not disturbed by eye movement in the horizontal, vertical and eye axis directions; as shown in fig. 13, when there is eye movement tracking, the anterior ocular segment image is clear and is not disturbed by eye movement in the horizontal, vertical, and axial directions.
Claims (7)
1. The three-dimensional eye movement tracking system is characterized by comprising a pupil camera imaging module, an image recognition analysis module, a motor control module and an OCT system module which are connected in sequence;
The pupil camera imaging module acquires images of pupils in real time through a pupil camera;
The image recognition analysis module obtains the change delta R of the radius or the diameter of the annular reflecting point in the real-time pupil image relative to the radius or the diameter of the annular reflecting point in the initial pupil image through a threshold method, further obtains the movement amount of the eye movement of a person along the direction of the eye axis, converts the movement amount into an accurate working distance, and obtains the displacement delta Z of the motor which should move; the offset of the center of the annular reflecting point in the real-time pupil image from the optical axis of the pupil camera is obtained through the image recognition analysis module, and compared with the offset of the center of the annular reflecting point in the initial pupil image from the optical axis of the pupil camera, the offset of human eyes in the horizontal and vertical directions is obtained, and the offset is converted into the horizontal displacement delta X and the vertical displacement delta Y of a motor for adjusting the alignment of the centers of the ocular lens and the human eyes;
The motor control module respectively adjusts corresponding motors according to the calculated delta X, delta Y and delta Z, so that the eyes always keep an accurate working distance and are aligned with the center of the ocular lens under the condition of eye movement;
the OCT system module performs OCT imaging of anterior ocular segment and OCT imaging of posterior ocular segment with the assistance of real-time eye movement tracking;
the tracking method of the three-dimensional eye movement tracking system comprises the following steps:
(1) Imaging and collecting the object to be detected so as to obtain at least two images;
(2) Tracking the movement of the eyes along the eye axis direction to obtain the variation delta R of the radius or the diameter of the annular reflecting point in the second image relative to the radius or the diameter of the annular reflecting point in the initial image; converting the variable delta R into a displacement delta Z of motor movement;
(3) Tracking the movement of eyes along the horizontal and vertical directions to obtain the offset of the eyes in the horizontal and vertical directions, and converting the offset into the horizontal displacement delta X and the vertical displacement delta Y of a motor for adjusting the alignment of the centers of the ocular lens and the eyes;
(4) Adjusting the motor to move according to the calculated delta X, delta Y and delta Z;
(5) Finally imaging is performed by the OCT system module.
2. The three-dimensional eye movement tracking system of claim 1, wherein the pupil camera imaging module is comprised of annular pupil illumination LEDs, an ocular, an imaging lens, and a camera, the annular pupil illumination LEDs being comprised of at least 3 equally spaced LEDs arranged in an annular configuration.
3. A three-dimensional eye movement tracking system according to claim 1, wherein the specific steps of tracking the movement of the eye in the direction of the eye axis in step (2) of the tracking method in the system are as follows:
(2.1) calculating the radius or diameter of the annular light reflecting points in each image by fitting according to the annular light reflecting points in the images;
(2.2) when there is eye movement, the radius or diameter of the annular reflecting spot in the latter image will have a change amount DeltaR relative to the diameter of the annular reflecting spot in the former image, when the change amount DeltaR exceeds the threshold value, a command is sent to the motor control module to make the motor move by a corresponding displacement DeltaZ.
4. A three-dimensional eye movement tracking system according to claim 3, characterized in that the specific steps of calculating the amount of change Δr and the amount of displacement Δz in step (2.2) of the tracking method in the system are as follows: under paraxial approximation, the relationship between annular spot diameter and eye-to-eyepiece distance is described by the following formula:
Wherein m L is the magnification of the optical system, R c is the cornea radius of the human eye, phi ring is the diameter of the annular LED, w is the distance between the eye and the ocular lens, and d is the distance between the center of the annular pupil illumination LED and the vertex of the ocular lens;
the change of the annular light mapping spot diameter along with the working distance can be obtained by the derivative of the above formula:
Wherein the amount of change The motor moves by a corresponding displacement Δz= -dw.
5. A three-dimensional eye movement tracking system according to claim 3, wherein the threshold value determination method in step (2.2) of the tracking method in the system is as follows: the radius or diameter variation size threshold of the annular light reflecting spot is not less than the test repeatability of the radius or diameter of the annular light reflecting spot.
6. A three-dimensional eye movement tracking system according to claim 1, wherein the specific steps of tracking the movement of the eye in the horizontal and vertical directions in step (3) of the tracking method in the system are as follows:
(3.1) taking one of the images as a first image, and calculating the offset of the center of the annular reflecting point in the first image from the optical axis of the pupil camera;
(3.2) selecting another image as a second image, and calculating the offset of the center of the annular reflecting point in the second image from the optical axis of the pupil camera;
And (3.3) when the eyes move, the offset of the center of the annular light reflecting spot in the second image from the optical axis of the pupil camera is changed relative to the offset of the center of the annular light reflecting spot in the initial image, namely the first image from the optical axis of the pupil camera, when the change exceeds a threshold value, the horizontal and vertical positions are adjusted so that the eyes always coincide with the center of the ocular lens, wherein the horizontal displacement of the movement is delta X, and the vertical displacement of the movement is delta Y.
7. A three-dimensional eye movement tracking system according to claim 1, wherein the specific steps of imaging by the OCT system module in step (5) of the tracking method in the system are as follows: when OCT imaging is carried out on the anterior segment of the eye, the OCT anterior-posterior segment switching module is moved into an OCT light path from the inside of the system, so that the anterior segment of the eye can be imaged; when OCT imaging of the posterior segment of the eye is carried out, the OCT anterior-posterior segment switching module is moved out of the OCT optical path from the system, so that the posterior segment of the eye can be imaged.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110264441.2A CN113040701B (en) | 2021-03-11 | 2021-03-11 | Three-dimensional eye movement tracking system and tracking method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110264441.2A CN113040701B (en) | 2021-03-11 | 2021-03-11 | Three-dimensional eye movement tracking system and tracking method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113040701A CN113040701A (en) | 2021-06-29 |
| CN113040701B true CN113040701B (en) | 2024-05-28 |
Family
ID=76511388
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110264441.2A Active CN113040701B (en) | 2021-03-11 | 2021-03-11 | Three-dimensional eye movement tracking system and tracking method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113040701B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114373216A (en) * | 2021-12-07 | 2022-04-19 | 图湃(北京)医疗科技有限公司 | Eye movement tracking method, device, equipment and storage medium for anterior segment OCTA |
| CN114359254B (en) * | 2022-01-13 | 2023-12-19 | 视微影像(河南)科技有限公司 | Method and system for detecting, positioning and tracking OCT (optical coherence tomography) image light column of anterior segment of eye |
| CN114098632B (en) * | 2022-01-27 | 2022-11-29 | 北京鹰瞳科技发展股份有限公司 | Method for controlling a motor in a fundus camera and related product |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102058391A (en) * | 2009-11-17 | 2011-05-18 | 佳能株式会社 | Apparatus and method for imaging optical coherence tomographic image |
| CN103025229A (en) * | 2010-04-29 | 2013-04-03 | 麻省理工学院 | Method and apparatus for motion correction and image enhancement for optical coherence tomography |
| CN103096785A (en) * | 2010-03-12 | 2013-05-08 | 佳能株式会社 | Ophthalmologic apparatus and control method for the same |
| CN103961058A (en) * | 2013-01-31 | 2014-08-06 | 佳能株式会社 | Optical coherence tomographic imaging apparatus and method for controlling the same |
| CN104958061A (en) * | 2015-07-28 | 2015-10-07 | 北京信息科技大学 | Fundus OCT imaging method utilizing three-dimensional imaging of binocular stereo vision and system thereof |
| CN105608675A (en) * | 2015-12-18 | 2016-05-25 | 天津迈达医学科技股份有限公司 | Fundus tissue OCT image motion artifact correction method |
| CN107609516A (en) * | 2017-09-13 | 2018-01-19 | 重庆爱威视科技有限公司 | Adaptive eye moves method for tracing |
| CN108992035A (en) * | 2018-06-08 | 2018-12-14 | 云南大学 | The compensation method of blinkpunkt positional shift in a kind of tracking of eye movement |
| CN109938691A (en) * | 2017-12-21 | 2019-06-28 | 清弘生医股份有限公司 | The continuity measurement method of eyeball and the continuity measuring device of eyeball |
| CN110604543A (en) * | 2018-06-15 | 2019-12-24 | 株式会社拓普康 | Ophthalmic device |
| CN110680273A (en) * | 2019-11-06 | 2020-01-14 | 深圳市斯尔顿科技有限公司 | Ophthalmic detection system and method |
| CN112155511A (en) * | 2020-09-30 | 2021-01-01 | 广东唯仁医疗科技有限公司 | Method for compensating human eye shake in OCT (optical coherence tomography) acquisition process based on deep learning |
| CN112168132A (en) * | 2020-11-09 | 2021-01-05 | 苏州大学 | Method for performing fundus refraction compensation judgment and imaging optimization by using OCT (optical coherence tomography) signals |
| CN112244756A (en) * | 2020-10-19 | 2021-01-22 | 深圳市斯尔顿科技有限公司 | Multifunctional full-automatic ophthalmic measuring method and system |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2702351A4 (en) * | 2011-04-29 | 2014-11-05 | Optovue Inc | Improved imaging with real-time tracking using optical coherence tomography |
| BR112015020170A2 (en) * | 2013-05-29 | 2017-07-18 | Wavelight Gmbh | apparatus and method for optical coherence tomography |
| US10660519B2 (en) * | 2014-01-30 | 2020-05-26 | Duke University | Systems and methods for eye tracking for motion corrected ophthalmic optical coherence tomography |
-
2021
- 2021-03-11 CN CN202110264441.2A patent/CN113040701B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102058391A (en) * | 2009-11-17 | 2011-05-18 | 佳能株式会社 | Apparatus and method for imaging optical coherence tomographic image |
| CN103096785A (en) * | 2010-03-12 | 2013-05-08 | 佳能株式会社 | Ophthalmologic apparatus and control method for the same |
| CN103025229A (en) * | 2010-04-29 | 2013-04-03 | 麻省理工学院 | Method and apparatus for motion correction and image enhancement for optical coherence tomography |
| CN103961058A (en) * | 2013-01-31 | 2014-08-06 | 佳能株式会社 | Optical coherence tomographic imaging apparatus and method for controlling the same |
| CN104958061A (en) * | 2015-07-28 | 2015-10-07 | 北京信息科技大学 | Fundus OCT imaging method utilizing three-dimensional imaging of binocular stereo vision and system thereof |
| CN105608675A (en) * | 2015-12-18 | 2016-05-25 | 天津迈达医学科技股份有限公司 | Fundus tissue OCT image motion artifact correction method |
| CN107609516A (en) * | 2017-09-13 | 2018-01-19 | 重庆爱威视科技有限公司 | Adaptive eye moves method for tracing |
| CN109938691A (en) * | 2017-12-21 | 2019-06-28 | 清弘生医股份有限公司 | The continuity measurement method of eyeball and the continuity measuring device of eyeball |
| CN108992035A (en) * | 2018-06-08 | 2018-12-14 | 云南大学 | The compensation method of blinkpunkt positional shift in a kind of tracking of eye movement |
| CN110604543A (en) * | 2018-06-15 | 2019-12-24 | 株式会社拓普康 | Ophthalmic device |
| CN110680273A (en) * | 2019-11-06 | 2020-01-14 | 深圳市斯尔顿科技有限公司 | Ophthalmic detection system and method |
| CN112155511A (en) * | 2020-09-30 | 2021-01-01 | 广东唯仁医疗科技有限公司 | Method for compensating human eye shake in OCT (optical coherence tomography) acquisition process based on deep learning |
| CN112244756A (en) * | 2020-10-19 | 2021-01-22 | 深圳市斯尔顿科技有限公司 | Multifunctional full-automatic ophthalmic measuring method and system |
| CN112168132A (en) * | 2020-11-09 | 2021-01-05 | 苏州大学 | Method for performing fundus refraction compensation judgment and imaging optimization by using OCT (optical coherence tomography) signals |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113040701A (en) | 2021-06-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113040701B (en) | Three-dimensional eye movement tracking system and tracking method thereof | |
| US10660519B2 (en) | Systems and methods for eye tracking for motion corrected ophthalmic optical coherence tomography | |
| US9918634B2 (en) | Systems and methods for improved ophthalmic imaging | |
| US8770753B2 (en) | Scanning and processing using optical coherence tomography | |
| JP5728302B2 (en) | Ophthalmic apparatus, ophthalmic system, ophthalmic apparatus control method, and program for the control method | |
| US20120140171A1 (en) | Ophthalmologic apparatus and blood flow velocity calculation method | |
| JP2010201102A (en) | Optical tomographic imaging apparatus | |
| JP6889581B2 (en) | Methods for Robust Eye Tracking and Ophthalmic Devices for It | |
| US10123699B2 (en) | Ophthalmologic apparatus and imaging method | |
| CN103815867A (en) | Retina dark field optical coherence tomographic imager for continuous adjustable ring lighting | |
| WO2021153087A1 (en) | Ophthalmic device, control method thereof, and storage medium | |
| CN105411523B (en) | Cornea image processing method | |
| WO2020158282A1 (en) | Ophthalmic information processing device, ophthalmic device, ophthalmic information processing method, and program | |
| CN113040700B (en) | Eye movement tracking system and tracking method thereof | |
| US10674902B2 (en) | Information processing apparatus, operation method thereof, and computer program | |
| CN113876301B (en) | Scanning-free three-dimensional AO-OCT imaging system and method for aberration chromatography detection and numerical correction | |
| CN109793492B (en) | Method and device for measuring small Liang Wangmai movement of human eyes | |
| WO2022097621A1 (en) | Ophthalmic information processing device, ophthalmic device, ophthalmic information processing method, and program | |
| WO2021256132A1 (en) | Ophthalmic device, method for controlling ophthalmic device, and program | |
| WO2021014744A1 (en) | Ophthalmological information processing device, ophthalmological device, ophthalmological information processing method, and program | |
| EP4209170A1 (en) | Ophthalmic apparatus | |
| CN114098627A (en) | Anterior segment OCT imaging system for ophthalmology examination | |
| CN117064326A (en) | OCT main body-based binocular synchronous anterior ocular segment imaging system with double probes | |
| JP2024003558A (en) | Ophthalmologic information processing device, ophthalmologic device, ophthalmologic information processing method, and program | |
| JP2023114533A (en) | Ophthalmologic apparatus, control method of ophthalmologic apparatus, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |