CN110215184B - Closed-loop control system and method of fundus camera - Google Patents

Closed-loop control system and method of fundus camera Download PDF

Info

Publication number
CN110215184B
CN110215184B CN201910580318.4A CN201910580318A CN110215184B CN 110215184 B CN110215184 B CN 110215184B CN 201910580318 A CN201910580318 A CN 201910580318A CN 110215184 B CN110215184 B CN 110215184B
Authority
CN
China
Prior art keywords
fundus
imaging system
control
auxiliary
mirror
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910580318.4A
Other languages
Chinese (zh)
Other versions
CN110215184A (en
Inventor
张�杰
张金莲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing Boshi Medical Technology Co ltd
Original Assignee
Nanjing Boshi Medical Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing Boshi Medical Technology Co ltd filed Critical Nanjing Boshi Medical Technology Co ltd
Priority to CN201910580318.4A priority Critical patent/CN110215184B/en
Priority to PCT/CN2019/095200 priority patent/WO2020258375A1/en
Publication of CN110215184A publication Critical patent/CN110215184A/en
Application granted granted Critical
Publication of CN110215184B publication Critical patent/CN110215184B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/0016—Operational features thereof
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • A61B3/12—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for looking at the eye fundus, e.g. ophthalmoscopes
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • A61B3/12—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for looking at the eye fundus, e.g. ophthalmoscopes
    • A61B3/1225—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for looking at the eye fundus, e.g. ophthalmoscopes using coherent radiation
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • A61B3/14—Arrangements specially adapted for eye photography
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
    • A61F9/00—Methods or devices for treatment of the eyes; Devices for putting-in contact lenses; Devices to correct squinting; Apparatus to guide the blind; Protective devices for the eyes, carried on the body or in the hand
    • A61F9/007—Methods or devices for eye surgery
    • A61F9/008—Methods or devices for eye surgery using laser
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50—Constructional details
    • H04N23/55—Optical parts specially adapted for electronic image sensors; Mounting thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Ophthalmology & Optometry (AREA)
  • Public Health (AREA)
  • Animal Behavior & Ethology (AREA)
  • Veterinary Medicine (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Physics & Mathematics (AREA)
  • Surgery (AREA)
  • Molecular Biology (AREA)
  • Medical Informatics (AREA)
  • Biophysics (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Optics & Photonics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Vascular Medicine (AREA)
  • Eye Examination Apparatus (AREA)

Abstract

The invention discloses a closed-loop control system and a method of an eye fundus camera, wherein the closed-loop control system comprises an optical system for controlling the eye fundus camera in a real-time closed-loop manner, namely a main imaging system; the system also comprises one or more auxiliary imaging systems, and internal optical tracking is carried out by the auxiliary imaging systems through converting fundus motion signals obtained by the main imaging system into the auxiliary imaging systems. By adopting the invention, the auxiliary system also has the function of optically tracking the position and the target of the eyeground in real time by utilizing the closed-loop control eyeground tracking signal of the main eyeground camera system (namely the main system) to drive one or more auxiliary systems.

Description

Closed-loop control system and method of fundus camera
Technical Field
The invention relates to the fundus target tracking and imaging technology in the medical field, in particular to a closed-loop control system and a method of a fundus camera.
Background
A Fundus Camera (Fundus Camera), also called Fundus Camera, is an ophthalmoscopic tool commonly used for observing the distribution of retina, optic disc, Fundus capillaries, etc. The medical application of the fundus camera can screen whether the optic nerve, retina, choroid and refraction medium of the fundus have pathological changes, and can also assist in diagnosing and judging the disease condition of other diseases through the fundus camera, for example, screening retinal images to detect cerebral infarction, cerebral hemorrhage, cerebral arteriosclerosis, brain tumor, diabetes, nephropathy, hypertension, retinopathy of prematurity, glaucoma, macular degeneration and the like. The earlier these diseases are detected, the more beneficial the clinical treatment, so the fundus camera is an indispensable medical equipment for clinical screening of fundus diseases.
However, the existing fundus camera usually does not have a real-time closed-loop control function, and cannot well support real-time optical tracking of the fundus position. Therefore, when the fundus camera is used for acquiring the fundus position information to control another device to operate on the same fundus position, for example, to control Optical Coherence Tomography (OCT) to perform tomography on the same fundus position or to control the laser hitting position of laser surgery, it is difficult to accurately control the spatial position of the light of the device on the fundus in real time due to the movement of the eyeball and the fundus (retina).
Disclosure of Invention
In view of the above, the present invention provides a closed-loop control system of a fundus camera and a method thereof, which uses the closed-loop control fundus tracking signal of a primary fundus camera imaging system (i.e. a primary imaging system) to drive one or more auxiliary systems, so that the auxiliary systems also have the function of optically tracking the position and target of the fundus in real time.
In order to achieve the purpose, the technical scheme of the invention is as follows:
a closed-loop control system of a fundus camera comprises an optical system for real-time closed-loop control of the fundus camera, namely a main imaging system; the system also comprises one or more auxiliary imaging systems, and internal optical tracking is carried out by the auxiliary imaging systems through converting fundus motion signals obtained by the main imaging system into the auxiliary imaging systems; the main imaging system comprises a light path structure consisting of an eye fundus camera, a light source, a plurality of lenses and a spectroscope, and an orthogonal inclined lens group; the orthogonal tilting mirror group includes: a first tilting mirror SM1 moving in the horizontal direction and a second tilting mirror SM2 moving in the vertical direction; the optical system converts fundus motion information acquired from the fundus camera image into residual motion information compensated by the SM1 and SM2, and respectively controls the SM1 and the SM2 to compensate translation motion or/and control a fundus camera to compensate fundus rotation in real time through a translation control command or/and a fundus rotation control command by utilizing the relationship between control parameters.
The relationship between the control parameters is expressed by equation (1):
(xt+1,yt+1,θt+1)＝(xt,yt,θt)+g(Δxt,Δyt,Δθt) (1)
wherein: (x)t,yt) Theta is a translational control command accumulated on the first tilting mirror SM1 and the second tilting mirror SM2 at the present timetAccumulating fundus/retina rotation control commands for the current time; (Δ x)t,Δyt) Is composed ofResidual fundus translation, Δ θ, obtained from images of the fundus cameratIs the residual fundus rotation obtained from the image; (x)t+1,yt+1) For the next sample time, the panning control commands, θ, that need to be updated for SM1 and SM2t+1A fundus/retina rotation control instruction which needs to be updated at the next sampling moment; the corner mark t represents a time series; g is the gain of the closed loop control system.
Control instructions to control the SM1 and SM2 are configured to be issued from a personal computer or a dedicated processor connected to the fundus camera of the optical system.
Setting fundus motion signals obtained by the main imaging system as (x, y, theta); the spatial transformation relation f (x, y, theta; x ', y ', theta ') of the first tilting mirror SM1 and the second tilting mirror SM2 of the main imaging system and the scanning mirror of the auxiliary imaging system can be realized by system calibration; at any sampling instant, the control signal delivered to the tracking mirror of the secondary imaging system according to equation (1) is such that the following relationship exists:
(x't+1,y't+1,θ't+1)＝f(x,y,θ；x',y',θ')(xt+1,yt+1,θt+1) (2)
wherein the corner mark t represents a time series, (x)t+1,yt+1,θt+1) Is the fundus motion signal of the primary imaging system at the next sampling timing, (x't+1,y't+1,θ't+1) Assisting a control signal of the imaging system for the next sampling moment; (x)t+1,yt+1) The next sample time requires updated pan control commands, θ, for the SM1 and SM2 of the main imaging systemt+1Fundus/retina rotation control instructions that need to be updated for the main imaging system for the next sampling time; (x't+1,y't+1) Control instruction of scanning mirror or tracking mirror translation position updated for auxiliary imaging system is needed for next sampling time theta't+1The control instruction of the rotation position of the auxiliary imaging system is needed for the next sampling moment.
The tracking mirrors of the primary imaging system, i.e. the first tilting mirror SM1 and the second tilting mirror SM2, can be shared by the secondary imaging system.
A closed-loop control method of a fundus camera, comprising the steps of:
A. setting a main imaging system, and additionally arranging an orthogonal tilting mirror group in an optical path system of the main imaging system, wherein the orthogonal tilting mirror group comprises: a first tilting mirror SM1 moving in the horizontal direction and a second tilting mirror SM2 moving in the vertical direction;
B. converting fundus motion information acquired from the fundus camera image into residual motion information compensated by the SM1 and SM2 using an optical system of the main imaging system;
C. respectively operating the SM1 and the SM2 to compensate translation motion or/and controlling the fundus camera to compensate fundus rotation in real time through a translation control command or/and a fundus rotation control command by utilizing the relationship among the control parameters; the relationship between the control parameters is expressed by equation (1):
(xt+1,yt+1,θt+1)＝(xt,yt,θt)+g(Δxt,Δyt,Δθt) (1)
wherein: (x)t,yt) Theta is a translational control command accumulated on the first tilting mirror SM1 and the second tilting mirror SM2 at the present timetAccumulating fundus/retina rotation control commands for the current time; (Δ x)t,Δyt) For residual fundus translation obtained from images of the fundus camera, Δ θtIs the residual fundus rotation obtained from the image; (x)t+1,yt+1) For the next sample time, the panning control commands, θ, that need to be updated for SM1 and SM2t+1A fundus/retina rotation control instruction which needs to be updated at the next sampling moment; the corner mark t represents a time series; g is the gain of the closed-loop control system;
D. one or more auxiliary imaging systems are additionally arranged, fundus motion signals obtained by the main imaging system are converted into the auxiliary imaging systems, and the auxiliary imaging systems are used for internal optical tracking.
Wherein: setting the fundus motion signal obtained by the main imaging system in the step D as (x, y, theta); the spatial transformation relation f (x, y, theta; x ', y ', theta ') of the first tilting mirror SM1 and the second tilting mirror SM2 of the main imaging system and the scanning mirror of the auxiliary imaging system can be realized by system calibration; at any sampling instant, the control signal delivered to the tracking mirror of the secondary imaging system according to equation (1) is such that the following relationship exists:
(x't+1,y't+1,θ't+1)＝f(x,y,θ；x',y',θ')(xt+1,yt+1,θt+1) (2)
wherein the corner mark t represents a time series, (x)t+1,yt+1,θt+1) Is the fundus motion signal of the primary imaging system at the next sampling timing, (x't+1,y't+1,θ't+1) Assisting a control signal of the imaging system for the next sampling moment; (x)t+1,yt+1) The next sample time requires updated pan control commands, θ, for the SM1 and SM2 of the main imaging systemt+1Fundus/retina rotation control instructions that need to be updated for the main imaging system for the next sampling time; (x't+1,y't+1) Control instruction of scanning mirror or tracking mirror translation position updated for auxiliary imaging system is needed for next sampling time theta't+1The control instruction of the rotation position of the auxiliary imaging system is needed for the next sampling moment.
The optical system for controlling the fundus camera in a real-time closed-loop manner and the implementation method thereof have the following beneficial effects:
the invention uses the fundus camera to carry out closed-loop optical fundus tracking, firstly obtains fundus motion (compensation) signals with high precision, high stability and strong anti-interference capability, then converts the fundus motion signals into fundus motion (compensation) signals of a second optical system through a proper spatial conversion relation, controls the precise position of light of the second optical system on the fundus/retina by using the fundus motion (compensation) signals, and the fundus motion (compensation) signals after spatial conversion also have the closed-loop signal characteristics of high precision, high stability and strong anti-interference capability. The second optical system can be an OCT imaging system, and the position of OCT scanning eyeground is a spatial subset of the imaging position of the eyeground camera; the second optical system may also be a fundus laser treatment system. In embodiments of the invention, the location at which the laser strikes the fundus is a spatial subset of the fundus camera imaging location. The second optical system may also be other optical systems for different clinical applications.
Drawings
FIG. 1 is a schematic view of a conventional fundus camera optical system;
FIG. 2 is an example of a fundus image taken by a prior art fundus camera;
FIG. 3 is a schematic view of an optical system of a conventional main fundus camera (i.e., a main system) integrated with a second system (i.e., an auxiliary system);
FIG. 4 is a schematic view of an optical system of a conventional main fundus camera (i.e., a main system) integrating a plurality of auxiliary systems;
FIG. 5 is a schematic diagram of an optical system for real-time closed-loop control of a fundus camera according to an embodiment of the present invention;
FIG. 6 is a schematic structural diagram of an optical system of a real-time closed-loop control fundus camera according to a second embodiment of the present invention;
FIG. 7 is a schematic view of a rotational imaging camera of the present invention used to compensate for rotational movement of the fundus;
FIG. 8 is a schematic diagram of an optical system for implementing closed-loop control of an auxiliary system by using a main fundus camera imaging system according to an embodiment of the present invention;
FIG. 9 is a schematic diagram of an optical system for implementing closed-loop control of an auxiliary system by using a main fundus camera imaging system according to a second embodiment of the present invention;
FIG. 10 is a diagram illustrating a mapping relationship between the primary and secondary system imaging spaces.
Detailed Description
The technical solution of the present invention will be described in further detail with reference to the accompanying drawings and embodiments of the present invention.
Fig. 1 is a schematic view of an optical system of a conventional fundus camera. Incident Light emitted from the Light Source (LS) 1 is collimated by the lens L1 to reach the beam splitter (DM) 1. Part of the incident light is reflected by DM1 into lens L2 to the fundus (Eye), i.e., retina. The light reflected from the fundus is transmitted through the lens L2, the beam splitter DM1, and the focusing lens L3, and is finally received by the fundus camera.
In the embodiment of the present invention, the fundus camera may operate alone, or may operate under control of a PC or a dedicated Processor (Processor). An image of the fundus can be obtained using the fundus camera shown in fig. 1, as shown in fig. 2. The image may be in color or black and white.
Fig. 3 is a schematic view of an optical system of a conventional primary fundus camera (i.e., a primary system or a primary system) integrated with a secondary system (i.e., a secondary system).
In clinical applications, a second optical system is usually integrated in the dominant fundus camera. In the embodiment of the invention, an eyeground camera with a function of closed-loop control and real-time optical tracking of the position of an eyeground is defined as a main system; meanwhile, other optical systems integrated in the main system, a common path (non-common path) or a common path (common path), are defined as sub-systems.
As shown in fig. 3, light emitted from the light source LS2 of the secondary system passes through the optical components of the secondary system, such as a lens L2 and a scanning mirror (scanning mirrors), and then reaches the beam splitter DM 2. Typically, the DM2 transmits all light passing through the primary system, but reflects all secondary system light. The optics inside the secondary system, such as the scanning mirrors (scanning mirrors), may be controlled by the primary system or may be operated separately.
Fig. 4 is a schematic view of an optical system in which a conventional main fundus camera (i.e., a main system) integrates a plurality of auxiliary systems.
As shown in fig. 4, an optical system is provided in which a plurality of sub systems are integrated on the basis of fig. 3. Each subsystem may have its own particular functionality, such as one for fundus Coherence Tomography (OCT) and another for projecting a focused beam of light onto the fundus for therapeutic purposes. The optics inside each auxiliary system, such as Scanning Mirrors (Mirrors) and Optical components (Optical components), can be controlled by the main system or can be operated independently.
FIG. 5 is a schematic structural diagram of an optical system of a real-time closed-loop control fundus camera according to an embodiment of the present invention.
In the embodiment of the invention, the improved optical system structure of the fundus camera is improved, so that the improved optical system of the fundus camera has the functions of real-time closed-loop control and optical tracking of the fundus position/retinal target.
As shown in fig. 5, the optical system of the real-time closed-loop control fundus camera is additionally provided with an orthogonal tilting Mirror group, and the tilting Mirror group includes two orthogonally moving one-dimensional tilting mirrors (SM), specifically: the first tilting mirror SM1 moves in the horizontal direction, and the second tilting mirror SM2 moves in the vertical direction. Obviously, the tilting mirrors SM1 and SM2 can move in any direction and angle in 360 degree space as long as the movement axes of the tilting mirrors SM1 and SM2 satisfy the orthogonal relationship.
In an embodiment of the invention, the tilting mirror device used is a 6210H biaxial scanning mirror from CTI (Cambridge Technology Inc).
In another embodiment of the present invention, the tilting mirrors SM1 and SM2 may be replaced by a two-dimensional tilting mirror with two orthogonal axes of motion, and one device that can be implemented is the S334-2SL two-dimensional tilting mirror of pi (physics instrument).
As shown in fig. 5, the control instructions/signals (Controlled by PC) for controlling the tilting mirrors SM1 and SM2 can be obtained from a PC or a dedicated processor.
A simple implementation is to acquire the fundus location signal (x, y, θ) from fundus image signals acquired by a fundus camera. Where (x, y) is the amount of translation of the eyeball/retina obtained from the fundus image, and θ is the amount of rotation of the eyeball/retina obtained from the fundus image. The method adopted by the embodiment of the invention is to calculate (x, y, theta) by using a Cross Correlation (Cross Correlation) algorithm. In a time sequence, a fundus image obtained earlier as a reference image is defined as, for example, R, and a fundus image obtained from a fundus camera at any time later is defined as TkHere, subscript k (═ 1, 2, 3, … …) is in time order and occurs after the reference picture. Running the Cross-correlation Algorithm xcorr (T)kR) then T is obtainedkAnd the spatial relative relationship (x, y, θ) between R. Running the Cross-correlation Algorithm xcorr (T)kR) can be implemented either by a conventional Fast Fourier Transform (FFT) or by other meansThe method is realized.
The three parameters described above, i.e., (x, y, θ), generally describe the movement of the eyeball/fundus target more completely.
In the embodiment shown in fig. 5, a closed loop control scheme is disclosed. By closed loop control is meant that the tilting mirrors SM1 and SM2 are placed before the fundus camera as signal detector, and that the image drift caused by eye movements has been optically compensated in real time by the tilting mirrors SM1 and SM2 before the fundus image signal enters the fundus camera, so that image drift and amount of rotation, in effect residual motion information, is obtained by the fundus camera at any one time. As mentioned above, the residual motion information is still obtained from the image by the cross-correlation algorithm. Thus, in the time-space domain, the relationship of the control parameters can be expressed in the form of equation (1):
(xt+1,yt+1,θt+1)＝(xt,yt,θt)+g(Δxt,Δyt,Δθt) (1)
wherein: (x)t,yt) Are the pan control commands, θ, accumulated on the tilting mirrors SM1 and SM2 at the current timetIs the control command accumulated in the fundus/retina rotation at the current moment (in special cases, for example, only translation is performed but not rotation, the theta istIs 0); (Δ x)t,Δyt) Is the residual fundus translation, Δ θ, obtained from the image of the fundus cameratIs the residual fundus rotation amount (angle) obtained from the image; (x)t+1,yt+1) Is the next sample time the translation control command, θ, that needs to be updated for the tilting mirrors SM1 and SM2t+1Is the fundus/retina rotation control command that needs to be updated (by controlling the fundus camera) at the next sampling time; g is the gain of the closed loop control system. The corner mark t represents a time series.
In equation (1) above, the eyeball/retina rotation signal can be compensated digitally or by an opto-mechanical method as shown in FIG. 6 below.
FIG. 6 is a schematic structural diagram of an optical system of a real-time closed-loop control fundus camera according to a second embodiment of the present invention.
In optomechanical compensation, it is one practice to mount the fundus camera on an eye Rotation signal compensation device, such as a Rotation Stage, so that the fundus camera can be rotated along the optical axis to optically compensate for the amount of Rotation of the fundus in real time. In this case, θtIs a rotation control command accumulated on the rotary table at the current time, Delta thetatIs the residual fundus rotation amount, θ, obtained from the image of the fundus camerat+1Is the rotation control command that needs to be updated for the turntable at the next sampling time. In this embodiment, the rest of the optical path structure is the same as that shown in fig. 5.
As shown in fig. 6, the optical system of the real-time closed-loop control fundus camera further provides another eyeball rotation signal compensation device, such as a mechanical device for compensating the eyeball rotation signal in real time. In this embodiment, a two-dimensional camera for imaging is mounted on a rotary stage (see a dotted rectangular frame) whose rotation axis coincides with the optical axis of the optical system.
As a method of acquiring fundus location information from a camera image, a Cross Correlation method (Cross Correlation) is generally used as shown in equation (1). The method comprises the following steps: selecting a reference image, importing an external file or extracting fundus position information from a real-time video; in the next time sequence, the offset of any future frame of the current image and the reference image is calculated, including the translation amount and the rotation amount, as shown in (1) (. DELTA.x)t,Δyt,Δθt)。
FIG. 7 is a schematic view of a rotational imaging camera used to compensate for rotational movement of the fundus. Fundus rotation can be caused by a number of factors, including eye rotation, head rotation, or other causes, but for fundus imaging systems the end result is rotation of the fundus image. Therefore, this is collectively called fundus rotation.
As shown in fig. 7, the lower part of fig. 7A is the reference image, and the upper part is the position of the imaging camera. The lower part of fig. 7B is that the image obtained by the fundus camera also appears rotated due to the fundus rotation, such as "rotation by θ degrees" of the schematic clock.
The rotation angle θ is obtained from equation (1) in a closed loop manner from the above images of fig. 7A and 7B by a cross-correlation algorithm, and then the angle θ can be sent to a rotation stage that controls the fundus (imaging) camera, and the light sensing surface of the imaging camera is rotated by θ degrees as well to compensate for the rotation amount θ of the fundus. The equivalent result of the rotation compensation is to restore the image obtained by the camera to the position of the lower image in fig. 7A.
Fig. 8 is a schematic diagram of an optical system for implementing closed-loop control of an auxiliary system by using a main fundus camera imaging system according to an embodiment of the present invention.
As shown in fig. 8, the fundus tracking signal is controlled by the closed loop of the primary system to drive one or more opto-mechanical like devices of the secondary system, so that the secondary system can also track the fundus position.
As shown in fig. 8, the closed loop control of the dominant fundus camera imaging system still employs the relationship of equation (1). In order to achieve the purpose that the auxiliary system can also track the eye movement, the eye movement signal (x, y, θ) obtained by the main imaging system needs to be converted into the scanning mirror (scanning mirrors) or the tracking mirror of the auxiliary system. The tracking mirrors of the main imaging system, i.e. the tilting mirrors SM1 and SM2, are used for internal tracking of the main imaging system, and the scanning mirrors or tracking mirrors of the secondary system are used for internal optical tracking of the secondary system.
The spatial transformation relationship f (x, y, theta; x ', y ', theta ') of the tracking mirrors SM1 and SM2 of the primary imaging system and the scan mirror of the secondary system is achieved by system calibration (calibration). Then, at any sampling instant, the control signal delivered to the secondary system tracking mirror according to equation (1) has the following relationship:
(x't+1,y't+1,θ't+1)＝f(x,y,θ；x',y',θ')(xt+1,yt+1,θt+1) (2)
and (3) using the result of the formula (2) to adjust the position of the scanning mirror of the auxiliary system to realize the real-time tracking of the auxiliary system on the target. But this set of signals does not include the functions specific to the sub-system scanning mirror for the sub-system itself, such as OCT scanning of the fundus.
FIG. 9 is a schematic diagram of an optical system for implementing closed-loop control of an auxiliary system by using a main fundus camera imaging system according to a second embodiment of the present invention.
FIG. 9 illustrates another embodiment of the present invention for driving tracking of one or more secondary imaging systems with closed-loop control of optical tracking of a primary imaging system.
As shown in fig. 9, the tracking mirrors of the primary imaging system, i.e., the tilting mirrors SM1 and SM2, are shared by all the secondary systems. That is, the arrival of the eye movement signal to the secondary system has been compensated by the real-time compensation of SM1 and SM 2. If the secondary system is an imaging system, such as OCT, then the real-time image of OCT is already stabilized by SM1 and SM 2.
In FIG. 9, the secondary system's scanning mirror is used for its own purpose only, such as the B-scan and C-scan of OCT, or for navigating the spatial location at which the focused laser beam is projected onto the fundus. However, there are some special reasons involved in the design of the optical system, and there are different optical magnifications, spatial offsets, etc. between the main system and the auxiliary system, then, in order to make the auxiliary system accurately track the fundus location, it is still necessary to convert the eyeball motion information of the main system to the scanning mirror of the auxiliary system to realize the optical tracking of the auxiliary system through the spatial transformation relation similar to equation (2).
FIG. 10 is a diagram illustrating a mapping relationship between the primary and secondary system imaging spaces.
As shown in fig. 10, it is assumed that the size of the imaging plane of the main optical system is exactly twice the size of the imaging plane of the secondary system, and the imaging centers of the two systems are identical, which is very common in clinical practice, for example, by "digging out" a local area from the main system by the secondary system for optical/digital amplification, or other imaging. Then, when the spatial transformation relation of equation (2) is used, it is easy to obtain:
x't+1＝xt+1/2 (3)
y't+1＝yt+1/2 (4)
θ't+1＝θt+1 (5)
obviously, the spatial mapping relationship between the primary system and the secondary system in the above equations (3), (4), and (5) may be other than that shown in fig. 10, and the results (3) to (5) obtained from equation (2) are also different. This particular relationship, once the design of the optical system is determined, is usually obtained by a one-time calibration measurement and calculation.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention.

Claims (3)

1. A closed-loop control system of a fundus camera comprises an optical system for real-time closed-loop control of the fundus camera, namely a main imaging system; the system is characterized by further comprising one or more auxiliary imaging systems, wherein the auxiliary imaging systems perform internal optical tracking by converting fundus motion signals obtained by the main imaging system into the auxiliary imaging systems; the main imaging system comprises a light path structure consisting of an eye fundus camera, a light source, a plurality of lenses and a spectroscope, and an orthogonal inclined lens group; the orthogonal tilting mirror group includes: a first tilting mirror SM1 moving in the horizontal direction and a second tilting mirror SM2 moving in the vertical direction; the optical system converts fundus motion information acquired from the fundus camera image into residual motion information compensated by the SM1 and SM2, and respectively controls the SM1 and the SM2 to compensate translation motion or/and control a fundus camera to compensate fundus rotation in real time through a translation control command or/and a fundus rotation control command by utilizing the relationship between control parameters;
the relationship between the control parameters is expressed by equation (1):
(xt+1,yt+1,θt+1)＝(xt,yt,θt)+g(Δxt,Δyt,Δθt) (1)
wherein: (x)t,yt) Theta is a translational control command accumulated on the first tilting mirror SM1 and the second tilting mirror SM2 at the present timetAccumulating fundus/retina rotation control commands for the current time; (Δ x)t,Δyt) For residual fundus translation obtained from images of the fundus camera, Δ θtIs the residual fundus rotation obtained from the image; (x)t+1,yt+1) Needs to be updated for SM1 and SM2 for the next sampling instantTranslation control command, θt+1A fundus/retina rotation control instruction which needs to be updated at the next sampling moment; the corner mark t represents a time series; g is the gain of the closed-loop control system; and wherein the orthogonal tilting mirror groups SM1 and SM2 of the primary imaging system are shared by the secondary imaging system, the arrival of fundus motion signals at the secondary imaging system being compensated by the shared orthogonal tilting mirror groups;
when the main imaging system and the auxiliary imaging system have different optical magnification or space offset, the fundus motion information of the main imaging system needs to be converted into a scanning mirror of the auxiliary imaging system to realize the optical tracking of the auxiliary imaging system; wherein, the fundus motion signal obtained by the main imaging system is (x, y, theta); the spatial transformation relation f (x, y, theta; x ', y ', theta ') of the first tilting mirror SM1 and the second tilting mirror SM2 of the main imaging system and the scanning mirror of the auxiliary imaging system can be realized by system calibration; at any sampling instant, the control signal delivered to the tracking mirror of the secondary imaging system according to equation (1) is such that the following relationship exists:
(x't+1,y't+1,θ't+1)＝f(x,y,θ；x',y',θ')(xt+1,yt+1,θt+1) (2)
wherein the corner mark t represents a time series, (x)t+1,yt+1,θt+1) Is the fundus motion signal of the primary imaging system at the next sampling timing, (x't+1,y't+1,θ't+1) Assisting a control signal of the imaging system for the next sampling moment; (x)t+1,y t+1) The next sample time requires updated pan control commands, θ, for the SM1 and SM2 of the main imaging systemt+1Fundus/retina rotation control instructions that need to be updated for the main imaging system for the next sampling time; (x't+1,y't+1) Control instruction of scanning mirror or tracking mirror translation position updated for auxiliary imaging system is needed for next sampling time theta't+1The control instruction of the rotation position of the auxiliary imaging system is needed for the next sampling moment.
2. A closed loop control system for a fundus camera according to claim 1 wherein the control instructions controlling the SM1 and SM2 are configured to be issued from a personal computer or dedicated processor connected to the fundus camera of the optical system.
3. A closed-loop control method based on the closed-loop control system of the fundus camera according to any one of claims 1 to 2, comprising the steps of:
A. setting a main imaging system, and additionally arranging an orthogonal tilting mirror group in an optical path system of the main imaging system, wherein the orthogonal tilting mirror group comprises: a first tilting mirror SM1 moving in the horizontal direction and a second tilting mirror SM2 moving in the vertical direction;
B. converting fundus motion information acquired from the fundus camera image into residual motion information compensated by the SM1 and SM2 using an optical system of the main imaging system;
C. respectively operating the SM1 and the SM2 to compensate translation motion or/and controlling the fundus camera to compensate fundus rotation in real time through a translation control command or/and a fundus rotation control command by utilizing the relationship among the control parameters; the relationship between the control parameters is expressed by equation (1):
(xt+1,yt+1,θt+1)＝(xt,yt,θt)+g(Δxt,Δyt,Δθt) (1)
wherein: (x)t,yt) Theta is a translational control command accumulated on the first tilting mirror SM1 and the second tilting mirror SM2 at the present timetAccumulating fundus/retina rotation control commands for the current time; (Δ x)t,Δyt) For residual fundus translation obtained from images of the fundus camera, Δ θtIs the residual fundus rotation obtained from the image; (x)t+1,yt+1) For the next sample time, the panning control commands, θ, that need to be updated for SM1 and SM2t+1A fundus/retina rotation control instruction which needs to be updated at the next sampling moment; the corner mark t represents a time series; g is the gain of the closed-loop control system;
D. one or more auxiliary imaging systems are additionally arranged, fundus motion signals obtained by the main imaging system are converted into the auxiliary imaging systems, and the auxiliary imaging systems perform internal optical tracking; and wherein the orthogonal tilting mirror groups SM1 and SM2 of the primary imaging system are shared by the secondary imaging system, the arrival of fundus motion signals at the secondary imaging system being compensated by the shared orthogonal tilting mirror groups;
when the main imaging system and the auxiliary imaging system have different optical magnification or space offset, the fundus motion information of the main imaging system needs to be converted into a scanning mirror of the auxiliary imaging system to realize the optical tracking of the auxiliary imaging system; setting fundus motion signals obtained by the main imaging system in the step D as (x, y, theta); the spatial transformation relation f (x, y, theta; x ', y ', theta ') of the first tilting mirror SM1 and the second tilting mirror SM2 of the main imaging system and the scanning mirror of the auxiliary imaging system can be realized by system calibration; at any sampling instant, the control signal delivered to the tracking mirror of the secondary imaging system according to equation (1) is such that the following relationship exists:
(x't+1,y't+1,θ't+1)＝f(x,y,θ；x',y',θ')(xt+1,yt+1,θt+1) (2)
wherein the corner mark t represents a time series, (x)t+1,yt+1,θt+1) Is the fundus motion signal of the primary imaging system at the next sampling timing, (x't+1,y't+1,θ't+1) Assisting a control signal of the imaging system for the next sampling moment; (x)t+1,yt+1) The next sample time requires updated pan control commands, θ, for the SM1 and SM2 of the main imaging systemt+1Fundus/retina rotation control instructions that need to be updated for the main imaging system for the next sampling time; (x't+1,y't+1) Control instruction of scanning mirror or tracking mirror translation position updated for auxiliary imaging system is needed for next sampling time theta't+1The control instruction of the rotation position of the auxiliary imaging system is needed for the next sampling moment.
CN201910580318.4A 2019-06-28 2019-06-28 Closed-loop control system and method of fundus camera Active CN110215184B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201910580318.4A CN110215184B (en) 2019-06-28 2019-06-28 Closed-loop control system and method of fundus camera
PCT/CN2019/095200 WO2020258375A1 (en) 2019-06-28 2019-07-09 Closed-loop control system and method for fundus camera

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910580318.4A CN110215184B (en) 2019-06-28 2019-06-28 Closed-loop control system and method of fundus camera

Publications (2)

Publication Number Publication Date
CN110215184A CN110215184A (en) 2019-09-10
CN110215184B true CN110215184B (en) 2020-11-20

Family

ID=67815356

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910580318.4A Active CN110215184B (en) 2019-06-28 2019-06-28 Closed-loop control system and method of fundus camera

Country Status (2)

Country Link
CN (1) CN110215184B (en)
WO (1) WO2020258375A1 (en)

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6460618B2 (en) * 2013-01-31 2019-01-30 キヤノン株式会社 Optical coherence tomography apparatus and control method thereof
US9468373B2 (en) * 2013-09-24 2016-10-18 Sony Interactive Entertainment Inc. Gaze tracking variations using dynamic lighting position
CN104116495B (en) * 2014-07-11 2016-02-10 北京理工大学 A kind of Technology of Retina Optical Coherence Tomography detection-display system
US10317672B2 (en) * 2014-12-11 2019-06-11 AdHawk Microsystems Eye-tracking system and method therefor
US9775515B2 (en) * 2015-05-28 2017-10-03 University Of Rochester System and method for multi-scale closed-loop eye tracking with real-time image montaging
CN109924942B (en) * 2019-04-25 2024-04-05 南京博视医疗科技有限公司 Optical image stabilizing method and system based on line scanning imaging system

Also Published As

Publication number Publication date
CN110215184A (en) 2019-09-10
WO2020258375A1 (en) 2020-12-30

Similar Documents

Publication Publication Date Title
KR101900907B1 (en) Electronically controlled fixation light for ophthalmic imaging systems
Hammer et al. Adaptive optics scanning laser ophthalmoscope for stabilized retinal imaging
CN103687532B (en) The misalignment controlled for the image processor of ophthalmic system reduces
US7896496B2 (en) Stabilized retinal imaging with adaptive optics
CN110200585B (en) Laser beam control system and method based on fundus imaging technology
US10743759B2 (en) Optical coherence tomography method, system and computer program product therefor
US10939817B2 (en) Fundus imaging apparatus and imaging method
CN110301886B (en) Optical system for real-time closed-loop control of fundus camera and implementation method thereof
JP6442960B2 (en) Fundus imaging device with wavefront compensation
US9775515B2 (en) System and method for multi-scale closed-loop eye tracking with real-time image montaging
CN109924942A (en) A kind of photorefractive crystals method and system based on Line-scanning Image Acquisition System
CN110200582B (en) Laser beam control system and method based on fundus imaging technology
CN110215184B (en) Closed-loop control system and method of fundus camera
EP2312298B1 (en) Optical coherence tomography in ophthalmology
US10674902B2 (en) Information processing apparatus, operation method thereof, and computer program
CN210228108U (en) Optical image stabilization system based on line scanning imaging system
EP3804607A1 (en) Ophthalmic scanning system and method
CN115120177A (en) Miniaturized retina imaging device and imaging method
CN110051320B (en) Method for calculating fundus target movement amount of line scanning imaging system
CN110200584B (en) Target tracking control system and method based on fundus imaging technology
JP2023511723A (en) Method and apparatus for orbit determination and imaging
Ferguson et al. Tracking adaptive optics scanning laser ophthalmoscope
WO2020232309A1 (en) Method and apparatus to track binocular eye motion
CN109924943A (en) A kind of digital image stabilization method and system based on improved Line-scanning Image Acquisition System
JPH0314448B2 (en)

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant