Classifications

• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
  • A61F9/00—Methods or devices for treatment of the eyes; Devices for putting in contact-lenses; Devices to correct squinting; Apparatus to guide the blind; Protective devices for the eyes, carried on the body or in the hand
  • A61F9/007—Methods or devices for eye surgery
  • A61F9/008—Methods or devices for eye surgery using laser
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
  • A61B3/0016—Operational features thereof
  • A61B3/0025—Operational features thereof characterised by electronic signal processing, e.g. eye models
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
  • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
  • A61B3/1015—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for wavefront analysis
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
  • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
  • A61B3/113—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for determining or recording eye movement
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
  • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
  • A61B3/14—Arrangements specially adapted for eye photography
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
  • A61F9/00—Methods or devices for treatment of the eyes; Devices for putting in contact-lenses; Devices to correct squinting; Apparatus to guide the blind; Protective devices for the eyes, carried on the body or in the hand
  • A61F9/007—Methods or devices for eye surgery
  • A61F9/008—Methods or devices for eye surgery using laser
  • A61F9/00802—Methods or devices for eye surgery using laser for photoablation
  • A61F9/00804—Refractive treatments
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
  • A61F9/00—Methods or devices for treatment of the eyes; Devices for putting in contact-lenses; Devices to correct squinting; Apparatus to guide the blind; Protective devices for the eyes, carried on the body or in the hand
  • A61F9/007—Methods or devices for eye surgery
  • A61F9/008—Methods or devices for eye surgery using laser
  • A61F9/00802—Methods or devices for eye surgery using laser for photoablation
  • A61F9/00804—Refractive treatments
  • A61F9/00806—Correction of higher orders
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
  • A61F9/00—Methods or devices for treatment of the eyes; Devices for putting in contact-lenses; Devices to correct squinting; Apparatus to guide the blind; Protective devices for the eyes, carried on the body or in the hand
  • A61F9/007—Methods or devices for eye surgery
  • A61F9/008—Methods or devices for eye surgery using laser
  • A61F9/00825—Methods or devices for eye surgery using laser for photodisruption
  • A61F9/00827—Refractive correction, e.g. lenticle
  • A61F9/00829—Correction of higher orders
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
  • G06F3/013—Eye tracking input arrangements
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/0002—Inspection of images, e.g. flaw detection
  • G06T7/0012—Biomedical image inspection
  • G06T7/0014—Biomedical image inspection using an image reference approach
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
  • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
  • G06T7/337—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving reference images or patches
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
  • A61F9/00—Methods or devices for treatment of the eyes; Devices for putting in contact-lenses; Devices to correct squinting; Apparatus to guide the blind; Protective devices for the eyes, carried on the body or in the hand
  • A61F9/007—Methods or devices for eye surgery
  • A61F9/008—Methods or devices for eye surgery using laser
  • A61F2009/00844—Feedback systems
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
  • A61F9/00—Methods or devices for treatment of the eyes; Devices for putting in contact-lenses; Devices to correct squinting; Apparatus to guide the blind; Protective devices for the eyes, carried on the body or in the hand
  • A61F9/007—Methods or devices for eye surgery
  • A61F9/008—Methods or devices for eye surgery using laser
  • A61F2009/00844—Feedback systems
  • A61F2009/00846—Eyetracking
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
  • A61F9/00—Methods or devices for treatment of the eyes; Devices for putting in contact-lenses; Devices to correct squinting; Apparatus to guide the blind; Protective devices for the eyes, carried on the body or in the hand
  • A61F9/007—Methods or devices for eye surgery
  • A61F9/008—Methods or devices for eye surgery using laser
  • A61F2009/00844—Feedback systems
  • A61F2009/00848—Feedback systems based on wavefront
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
  • A61F9/00—Methods or devices for treatment of the eyes; Devices for putting in contact-lenses; Devices to correct squinting; Apparatus to guide the blind; Protective devices for the eyes, carried on the body or in the hand
  • A61F9/007—Methods or devices for eye surgery
  • A61F9/008—Methods or devices for eye surgery using laser
  • A61F2009/00861—Methods or devices for eye surgery using laser adapted for treatment at a particular location
  • A61F2009/00872—Cornea
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
  • A61F9/00—Methods or devices for treatment of the eyes; Devices for putting in contact-lenses; Devices to correct squinting; Apparatus to guide the blind; Protective devices for the eyes, carried on the body or in the hand
  • A61F9/007—Methods or devices for eye surgery
  • A61F9/008—Methods or devices for eye surgery using laser
  • A61F2009/00878—Planning
  • A61F2009/0088—Planning based on wavefront
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
  • A61F9/00—Methods or devices for treatment of the eyes; Devices for putting in contact-lenses; Devices to correct squinting; Apparatus to guide the blind; Protective devices for the eyes, carried on the body or in the hand
  • A61F9/007—Methods or devices for eye surgery
  • A61F9/008—Methods or devices for eye surgery using laser
  • A61F2009/00878—Planning
  • A61F2009/00882—Planning based on topography
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
  • A61F9/00—Methods or devices for treatment of the eyes; Devices for putting in contact-lenses; Devices to correct squinting; Apparatus to guide the blind; Protective devices for the eyes, carried on the body or in the hand
  • A61F9/007—Methods or devices for eye surgery
  • A61F9/008—Methods or devices for eye surgery using laser
  • A61F2009/00897—Scanning mechanisms or algorithms
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
  • A61F9/00—Methods or devices for treatment of the eyes; Devices for putting in contact-lenses; Devices to correct squinting; Apparatus to guide the blind; Protective devices for the eyes, carried on the body or in the hand
  • A61F9/007—Methods or devices for eye surgery
  • A61F9/008—Methods or devices for eye surgery using laser
  • A61F9/00825—Methods or devices for eye surgery using laser for photodisruption
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/30—Subject of image; Context of image processing
  • G06T2207/30004—Biomedical image processing
  • G06T2207/30041—Eye; Retina; Ophthalmic

Definitions

• the present invention relates generally to laser eye surgery methods and systems. More specifically, the present invention relates to registering a first image of a patient's eye with a second image of a patients eye and to tracking a position and a torsional orientation of the patient's eye during laser eye surgery so as to register a customized ablation profile with the patient's eye.
• Known laser eye procedures generally employ an ultraviolet or infrared laser to remove a microscopic layer of stromal tissue from the cornea of the eye to alter the refractive characteristics of the eye.
• the laser removes a selected shape of the corneal tissue, often to correct refractive errors of the eye.
• Ultraviolet laser ablation results in photo-decomposition of the corneal tissue, but generally does not cause significant thermal damage to adjacent and underlying tissues of the eye.
• the irradiated molecules are broken into smaller volatile fragments photochemically, directly breaking the intermolecular bonds.
• Laser ablation procedures can remove the targeted stroma of the cornea to change the cornea's contour for varying purposes, such as for correcting myopia, hyperopia, astigmatism, and the like.
• Control over the distribution of ablation energy across the cornea may be provided by a variety of systems and methods, including the use of ablatable masks, fixed and moveable apertures, controlled scanning systems, eye movement tracking mechanisms, and the like.
• the laser beam often comprises a series of discrete pulses of laser light energy, with the total shape and amount of tissue removed being determined by the shape, size, location, and/or number of a pattern of laser energy pulses impinging on the cornea.
• a variety of algorithms may be used to calculate the pattern of laser pulses used to reshape the cornea so as to correct a refractive error of the eye.
• Known systems make use of a variety of forms of lasers and/or laser energy to effect the correction, including infrared lasers, ultraviolet lasers, femtosecond lasers, wavelength multiplied solid-state lasers, and the like.
• Alternative vision correction techniques make use of radial incisions in the cornea, intraocular lenses, removable corneal support structures, thermal shaping, and the like.
• the calculation of the ablation profile, the transfer of information from the measurement system to the ablation system, and the operation of the ablation system all provide opportunities for the introduction of errors, so that the actual visual accuities provided by real-world wavefront-based correction systems may not be as good as might be theoretically possible.
• wavefront measurements aligning the customized laser ablation pattern with the patient's eye.
• the wavefront measurement and the eye should share a common coordinate system. For example, when the wavefront measurement is taken, the patient will generally be in a seated position. However, when the laser eye surgery is being performed, the patient will generally be in a supine position, which may not position the patient's eye in the same position or torsional orientation as the eye when the wavefront measurement was taken.
• the present invention provides methods and systems which can improve laser eye surgery.
• the methods and software of the present invention can register a first image of the patient's eye with a second image of the patient's eye.
• the methods can determine a torsional offset ⁇ Q between the eye in the first image and the eye in the second image.
• a method comprises selecting at least one marker on the iris of the eye in the first image. A corresponding marker is located on the iris in the second image. The first image of the eye and the second image of the eye are registered by substantially matching a common reference point in the first and second images and matching the marker on the iris of the image of the first eye and the marker on the iris of the image of the second eye.
• a laser treatment can be centered and torsionally aligned with the second image of the eye.
• the second image of the eye can be obtained while the patient's eye is aligned with a laser beam that is to deliver the laser treatment.
• the common reference point is a pupil center. In other embodiments, the common reference point can be determined through a function of a pupil center and an iris center.
• the first image of the eye is obtained during the measurement of a wavefront (which reflects the lower and higher order optical aberrations in the optical system of the patient's eye) and the second image of the eye is obtained when the patient is positioned in the optical axis of the therapeutic laser.
• the patient's eye in the first image can be registered with the patient's eye when it is positioned in an optical axis of the therapeutic laser so that the laser treatment is delivered in a torsionally correct orientation.
• the present invention can track the torsional movement of the eye over time 0(f).
• Tracking of the torsional orientation of the patient's eye allows a computer processor to adjust a delivery of the customized ablation treatment to account for the changes in the position and orientation of the patient's eye.
• the present invention provides for torsional tracking of the eye.
• a tracking algorithm can establish the exact amount of rotation of the eye with respect to the wavefront image taken during the wavefront measurement. This torsional rotation of the eye can be compensated for by making corresponding adjustment of the laser beam delivery.
• a reference point (such as a pupil center) is located in a first image of the eye. At least one marker is identified in the first image of the eye. The reference point is also located in a second image of the eye. A corresponding marker is identified in the second image of the eye.
• a cyclotorsional rotation of the eye is estimated between the first image and second image by comparing the orientation of the at least one markers relative to the pupil center in the first image and the second image.
• the present invention provides a method of performing laser eye surgery.
• the method comprises measuring a wavefront measurement of the patient's eye.
• An image of the patient's eye is obtained during the measuring of the wavefront measurement.
• a laser treatment of the patient's eye is generated based on the wavefront measurement.
• the position of the patient's eye is registered with the image of the patient's eye obtained during the measuring of the wavefront measurement so that the customized laser treatment can accurately delivered to the patient's eye.
• the laser treatment is delivered to the patient's eye while the torsional orientation of the patient's eye is monitored.
• the delivery of the laser treatment is adjusted based on the monitored torsional orientation of the patient's eye.
• the present invention provide a laser surgery system.
• the laser surgery system provides a computer processor configured to receive a first image of an eye and at least one of a wavefront measurement and an ablation pattern for the eye.
• An eye tracker can be coupled to the computer processor to track a position of the eye under an optical axis of a laser beam.
• a torsional tracker is coupled to the computer processor to track a torsional orientation of the eye.
• the computer processor can be configured to adjust a delivery of the ablation pattern based on a change of position and/or torsional orientation of the eye.
• the present invention provides a laser surgery system comprising a system for registering a first image of an eye with a second image of an eye.
• Thee system includes a computer processor that is configured to receive a first image of an eye.
• An imaging device can be coupled to the computer processor.
• the imaging device can obtain a second image of the eye.
• the computer processor can be configured to locate a reference point, such as a pupil center, in the first and second image of the eye and locate at least one marker in the first image and find a corresponding marker in the second image.
• the computer processor can register the first and second image by substantially matching the reference points (e.g., pupil centers) and markers of the first and second image.
• Figure 1 schematically illustrates a simplified system of the present invention
• FIG. 1 schematically illustrates one laser surgery system of the present invention
• Figure 3 illustrates one exemplary wavefront measurement device of the present invention
• Figure 3A illustrates an alternative wavefront measurement device of the present invention
• FIG. 4 schematically illustrates an exemplary system of the present invention
• Figure 5 schematically illustrates a method of the registering a first image with a second image
• Figure 6 A illustrates a reference image of an eye
• Figure 6B illustrates a rotated image that corresponds to the reference image of Figure 6 A
• Figures 6C and 6D illustrate a center of a pupil and center of an iris
• Figure 6E illustrate an inner and outer radii of a range of the iris radii
• Figure 7A illustrates an unwrapped iris that is segmented into 24 sectors, with each sector having a numbered marker;
• Figure 7B illustrates a corresponding unwrapped iris in which the markers are torsionally rotated from their original positions;
• Figure 7C illustrates two iris images and texture blocks when the iris ring is not unwrapped
• Figure 7D illustrates two iris images and texture blocks when the iris ring is unwrapped
• Figure 8A illustrates an unwrapped iris
• Figure 8B illustrates an unwrapped iris with LED reflections
• Figure 9 is a graph that illustrates an angular rotation of the 24 markers
• Figure 10 is a simplified method of tracking a torsional rotation of a patient's eye
• Figure 11 is a frame image of a patient's eye and two markers on the iris that are used for tracking a torsional rotation of the patient's eye;
• Figure 12 illustrates six reference blocks/markers of the patient's iris that are used to track the torsional rotation of the patient's eye;
• Figure 13 illustrates the relative positions of the reference markers relative to the center of the patient's pupil
• Figure 14 illustrates torsional angle estimates for an eye having a dark- colored iris
• Figure 15 illustrates torsional angle estimates for an eye having a light colored iris
• Figures 16A and 16B are charts summarizing results for a data set processed by one alignment algorithm of the present invention.
• Figure 17A is an image of an eye that has too much shadow to discern markers;
• Figure 17B is a chart illustrating an eye having an RMS that is above
• Figure 18A is an original frame image of an eye
• Figure 18B is a final frame in which the image of the eye is rotated
• Figure 19A is a reference frame
• Figure 19B is a zeroth frame having two pixel blocks marked for tracking
• Figure 20 is a chart of a pupil position over time
• Figure 21 is a chart of the pupil radius from frame 0 to frame 500;
• Figure 22 is a chart that illustrates errors per frame/block
• Figure 23 is a chart that illustrates a measured torsional angle of the eye;
• Figure 24 depicts the tracking results for a 30-frame sequence starting with the 345 th frame;
• Figure 25 is a chart that shows the torsional data extracted from the slower acquired sequence
• Figures 26A and 26B show alignment results using a sine-method between the wavefront measurement position of the iris and the first image of the video sequence
• Figures 27 A and 27B show measurements of the torsional eye movements with respect to the reference image
• Figure 28 shows a difference between two torsional angle estimates
• Figure 29A illustrates two torsion estimates
• Figure 29B illustrates the error between the two estimates of Figure 29A.
• the present invention is particularly useful for enhancing the accuracy and efficacy of laser eye surgical procedures such as photorefractive keratectomy (PRK), phototherapeutic keratectomy (PTK), laser in situ keratomileusis (LASIK), and the like.
• the efficacy of the laser eye surgical procedures can be enhanced by tracking the torsional orientation of the patient's eye so that a laser ablation pattern is more accurately aligned with the real-time orientation of the patient's eye.
• FIG. 1 schematically illustrates a simplified system of one embodiment of the present invention.
• the illustrated system of the present invention can include a laser system 15 coupled to a wavefront measurement device 10 that measures aberrations and other optical characteristics of an entire optical tissue system.
• the data from such a wavefront measurement device may be used to generate an optical surface from an array of optical gradients.
• the optical surface need not precisely match an actual tissue surface, as the gradients will show the effects of aberrations which are actually located throughout the ocular tissue system. Nonetheless, corrections imposed on an optical tissue surface so as to correct the aberrations derived from the gradients should correct the optical tissue system.
• an optical tissue surface may encompass a theoretical tissue surface (derived, for example, from wavefront sensor data), an actual tissue surface, and/or a tissue surface formed for purposes of treatment (for example, by incising corneal tissues so as to allow a flap of the corneal epithelium to be displaced and expose the underlying stroma during a LASIK procedure).
• Laser eye surgery system 15 includes a laser 12 that produces a laser beam 14.
• Laser 12 is optically coupled to laser delivery optics 16, which directs laser beam 14 to an eye of patient P.
• a delivery optics support structure (not shown here for clarity) extends from a frame 18 supporting laser 12.
• a microscope 20 is mounted on the delivery optics support structure, the microscope often being used to image a cornea of eye E.
• Laser 12 generally comprises an excimer laser, typically comprising an argon- fluorine laser producing pulses of laser light having a wavelength of approximately 193 nm.
• Laser 12 will preferably be designed to provide a feedback stabilized fluence at the patient's eye, delivered via delivery optics 16.
• the present invention may also be useful with alternative sources of ultraviolet or infrared radiation, particularly those adapted to controllably ablate the corneal tissue without causing significant damage to adjacent and/or underlying tissues of the eye.
• sources include, but are not limited to, solid state lasers and other devices which can generate energy in the ultraviolet wavelength between about 185 and 205 nm and/or those which utilize frequency-multiplying techniques.
• Laser 12 and delivery optics 16 will generally direct laser beam 14 to the eye of patient P under the direction of a computer processor 22.
• Processor 22 will generally selectively adjust laser beam 14 to expose portions of the cornea to the pulses of laser energy so as to effect a predetermined sculpting of the cornea and alter the refractive characteristics of the eye.
• both laser 14 and the laser delivery optical system 16 will be under computer control of processor 22 to effect the desired laser sculpting process so as to deliver the customized ablation profile, with the processor ideally altering the ablation procedure in response to inputs from the optical feedback system.
• the feedback will preferably be input into processor 22 from an automated image analysis system, or may be manually input into the processor by a system operator using an input device in response to a visual inspection of analysis images provided by the optical feedback system.
• Processor 22 will often continue and/or terminate a sculpting treatment in response to the feedback, and may optionally also modify the planned sculpting based at least in part on the feedback.
• Laser beam 14 may be adjusted to produce the desired sculpting using a variety of alternative mechanisms.
• the laser beam 14 may be selectively limited using one or more variable apertures.
• An exemplary variable aperture system having a variable iris and a variable width slit is described in U.S. Patent No. 5,713,892, the full disclosure of which is inco ⁇ orated herein by reference.
• the laser beam may also be tailored by varying the size and offset of the laser spot from an axis of the eye, as described in U.S. Patent No. 5,683,379, and as also described in co-pending U.S. Patent Application Serial Nos. 08/968,380, filed November 12, 1997; and 09/274,999 filed March 22, 1999, the full disclosures of which are inco ⁇ orated herein by reference.
• Still further alternatives are possible, including scanning of the laser beam over the surface of the eye and controlling the number of pulses and/or dwell time at each location, as described, for example, by U.S. Patent Nos. 4,665,913 (the full disclosure of which is inco ⁇ orated herein by reference) and as demonstrated by other scanning laser systems such as the LSX laser by LaserSight, LadarVision by Alcon/ Autonomous, and the 217C by Technolas; using masks in the optical path of laser beam 14 which ablate to vary the profile of the beam incident on the cornea, as described in U.S. Patent Application Serial No.
• laser system 15 will generally include a computer system or programmable processor 22.
• Processor 22 may comprise (or interface with) a conventional PC system including the standard user interface devices such as a keyboard, a display monitor, and the like.
• Processor 22 will typically include an input device such as a magnetic or optical disk drive, a CD drive, an internet connection, or the like.
• Such input devices will often be used to download a computer executable code from a computer network or a tangible storage media 29 embodying steps or programming instructions for any of the methods of the present invention.
• Tangible storage media 29 includes, but is not limited to a CD-R, a CD-RW, DVD, a floppy disk, an optical disk, a data tape, a non-volatile memory, or the like, and the processor 22 will include the memory boards and other standard components of modern computer systems for storing and executing this code.
• Wavefront measurement device 10 typically includes a wavefront measurement assembly 11 and an imaging assembly 13.
• Wavefront measurement assembly 11 can be used to measure and obtain a wavefront elevation surface of at least one of the patient's eyes and imaging assembly 13 can obtain still or moving images of the patient's eye during the wavefront measurement.
• imaging assembly 13 is a CCD camera that can obtain a still image of the patient's eye.
• the image(s) obtained by imaging assembly 13 can thereafter be used to register the wavefront measurement and/or a customized ablation pattern (based on the wavefront measurement) with the patient's eye during the laser surgical procedure.
• the wavefront measurement assembly 11 and imaging assembly 13 can be coupled to or integral with a computer system 17 that can generate and store the wavefront measurements and images of the patient's eye. Thereafter, the patient's wavefront data can be stored on a computer readable medium, such as a CD-R, CD-RW, DVD-R, floppy disk, optical disk, a hard drive, or other computer readable medium.
• the computer system of the wavefront measurement device can generate and save an ablation profile based on the wavefront data.
• the wavefront data and/or the customized ablation profile can be loaded into a laser surgical system 15 through reading of the computer readable medium or through delivery into a memory of surgical system 15 over a local or wide-area network (LAN or WAN).
• Laser eye surgery system 15 can include a computer controller system 22 that is in communication with an imaging assembly 20 and a laser assembly 12.
• Computer system 22 can have software stored in a memory and hardware that can be used to control the delivery of the ablative energy to the patient's eye, the tracking of the position (translations in the x, y, and z directions and torsional rotations) of the patient's eye relative to an optical axis of laser beam 14, and the like.
• computer system 22 can be programmed to calculate a customized ablation profile based on the wavefront data, register the image(s) taken with imaging assembly 11 with the image(s) taken by imaging assembly 20, and measure the torsional offset, ⁇ 0 , between the patient's eye in the two images. Additionally, computer system 22 can be programmed to measure, in real-time, the movement (x(t), y(t), z(t), and rotational orientation ⁇ (t)) of the patient's eye relative to the optical axis of the laser beam so as to allow the computer system to modify the delivery of the customized ablation profile based on the real-time position of the patient's eye.
• FIG. 3 one embodiment of a wavefront measurement device 10 of the present invention is schematically illustrated.
• the illustrated wavefront measurement device 10 is merely an example of one wavefront measurement device that can be used with the embodiments of the present invention and other conventional or proprietary wavefront measurement devices can be used.
• wavefront measurement device 10 includes an imaging assembly 13 that can image the patient's eye E during the wavefront measurement.
• Wavefront measurement assembly 13 includes an image source 32 which projects a source image through optical tissues 34 of eye E and so as to form an image 44 upon a surface of retina R.
• the image from retina R is transmitted by the optical system of the eye (specifically, optical tissues 34) and imaged onto a wavefront sensor 36 by system optics 38.
• the imaging assembly 11 can be in communication with a computer system 22 to deliver the image(s) of the patient's eye to a memory in the computer.
• Wavefront sensor 36 can also communicate signals to computer 17 for determination of a corneal ablation treatment program.
• Computer 17 may be the same computer which is used to direct operation of the laser surgery system 15, or at least some or all of the computer components of the wavefront measurement device 10 and laser surgery system may be separate. Data from wavefront sensor 36 may be transmitted to laser system computer 22 via tangible media 29, via an I/O port, via an networking connection such as an intranet, the Internet, or the like.
• Wavefront sensor 36 generally comprises a lenslet array 38 and an image sensor 40.
• the lenslet array separates the transmitted image into an array of beamlets 42, and (in combination with other optical components of the system) images the separated beamlets on the surface of sensor 40.
• Sensor 40 typically comprises a charged couple device or CCD, and senses the characteristics of these individual beamlets, which can be used to determine the characteristics of an associated region of optical tissues 34.
• image 44 comprises a point or small spot of light
• a location of the transmitted spot as imaged by a beamlet can directly indicate a local gradient of the associated region of optical tissue.
• Eye E generally defines an anterior orientation ANT and a posterior orientation POS.
• Image source 32 generally projects an image in a posterior orientation through optical tissues 34 onto retina R.
• Optical tissues 34 again transmit image 44 from the retina anteriorly toward wavefront sensor 36.
• Image 44 actually formed on retina R may be distorted by any imperfections in the eye's optical system when the image source is originally transmitted by optical tissues 34.
• image source projection optics 46 may be configured or adapted to decrease any distortion of image 44.
• image source optics may decrease lower order optical errors by compensating for spherical and/or cylindrical errors of optical tissues 34. Higher order optical errors of the optical tissues may also be compensated through the use of an adaptive optic element, such as a deformable mirror.
• Use of an image source 32 selected to define a point or small spot at image 44 upon retina R may facilitate the analysis of the data provided by wavefront sensor 36. Distortion of image 44 may be limited by transmitting a source image through a central region 48 of optical tissues 34 which is smaller than a pupil 50, as the central portion of the pupil may be less prone to optical errors than the peripheral portion. Regardless of the particular image source structure, it will be generally be beneficial to have well-defined and accurately formed image 44 on retina R.
• a series of wavefront sensor data readings may be taken.
• a time series of wavefront data readings may help to provide a more accurate overall determination of the ocular tissue aberrations.
• a plurality of temporally separated wavefront sensor measurements can avoid relying on a single snapshot of the optical characteristics as the basis for a refractive correcting procedure.
• Still further alternatives are also available, including taking wavefront sensor data of the eye with the eye in differing configurations, positions, and or orientations.
• a patient will often help maintain alignment of the eye with wavefront device 13 by focusing on a fixation target, as described in U.S. Patent No. 6,004,313, the full disclosure of which is inco ⁇ orated herein by reference.
• a fixation target as described in U.S. Patent No. 6,004,313, the full disclosure of which is inco ⁇ orated herein by reference.
• optical characteristics of the eye may be determined while the eye accommodate or adapts to image a field of view at a varying distance.
• Further alternatives include rotating of the eye by providing alternative and/or moving fixation targets within wavefront device 11.
• the location of the optical axis of the eye may be verified by reference to the data provided from an imaging assembly or pupil camera 13 that images the eye concurrently during the wavefront measurements.
• a pupil camera 13 images pupil 50 and/or the iris so as to allow subsequent determination of a position and torsional orientation of the pupil and/or iris for registration of the wavefront sensor data relative to the optical tissues, as will also be described hereinbelow.
• Figure 3 A An alternative embodiment of a wavefront sensor system is illustrated in Figure 3 A.
• the major components of the system of Figure 3 A are similar to those of Figure 3.
• Figure 3 A includes an adaptive optical element 52 in the form of a deformable mirror.
• the source image is reflected from deformable mirror 52 during transmission to retina R, and the deformable mirror is also along the optical path used to form the transmitted image between retina R and imaging sensor 40.
• Deformable mirror 52 can be controllably deformed to limit distortion of the image formed on the retina, and may enhance the accuracy of the wavefront data.
• the structure and use of the system of Figure 3 A are more fully described in U.S. Patent No. 6,095,651, the full disclosure of which his inco ⁇ orated herein by reference.
• the components of one embodiment of a wavefront system for measuring the eye and ablations comprise elements of a VISX WaveScanTM, available from VISX, Inc. of Santa Clara, California.
• a preferred embodiment includes a WaveScan with a deformable mirror as described above.
• An alternate embodiment of a wavefront measuring device is described in U. S. Patent No. 6,271,915, the full disclosure of which is inco ⁇ orated herein by reference.
• a treatment program map may be calculated from the wavefront elevation map so as to remove the regular (spherical and/or cylindrical) and irregular errors of the optical tissues.
• a table of ablation pulse locations, sizes, shapes, and/or numbers can be developed.
• An exemplary method and system for preparing such an ablation table is described in co-pending U.S. Patent Application No. 09/805,737 filed on March 13, 2001 and entitled “Generating Scanning Spot Locations for Laser Eye Surgery," the full disclosure of which is inco ⁇ orated herein by reference.
• Ablation table may optionally be optimized by sorting of the individual pulses so as to avoid localized heating, minimize irregular ablations if the treatment program is interrupted, and the like.
• a corneal ablation pattern may be calculated by processor 17 or 22 (or by another separate processor) for ablating the eye with laser ablation system 15 so as to correct the optical errors of the eye.
• Such calculations will often be based on both the measured optical properties of the eye and on the characteristics of the corneal tissue targeted for ablation (such as the ablation rate, the refractive index, the propensity of the tissue to form "central islands” or decreased central ablation depths within a uniform energy beam, and the like).
• the results of the calculation will often comprise an ablation pattern in the form of an ablation table listing ablation locations, numbers of pulses, ablation sizes, and or ablation shapes to effect the desired refractive correction.
• Wavefront measurement assembly 13 can use wavefront sensors 36, such as Hartmann-Shack sensors, for obtaining a wavefront elevation surface 54 of the patient's eye.
• Wavefront elevation surface 54 can be run through a treatment algorithm 58 to generate a treatment table or ablation profile 60 that is customized to correspond to the patient's wavefront elevation surface 54.
• ablation profile 60 can be calculated by a processor of wavefront device 10, laser system 15, or by a separate processor and stored in a memory of computer 17, 22.
• imaging assembly 11 can concurrently obtain an image 56 of the patient's eye, e.g., pupil and iris.
• the image of the patient's eye 56 can be analyzed by an algorithm 62 that locates the center of the pupil and or iris, calculates the radius of the pupil and or iris, and locates markers 64 in the patient's iris for subsequent registration and tracking.
• ablation profile 60 In order to register the ablation profile 60 and the patient's eye during the laser treatment, the ablation pattern and the patient's eye should share a common coordinate system. Thus, ablation profile 60 should be positionally and torsionally aligned with the patient's eye when the patient's eye is positioned in the path of the laser beam. Additionally, the translational and torsional orientation of the patient's eye should be tracked during the surgical procedure to ensure an accurate delivery of the ablation profile.
• pupil camera 20 is a video device that can obtain streaming video of the patient's eye.
• One frame 66 of the streaming video typically the first frame of the streaming video, can be analyzed by the computer processor to locate the pupil center, iris center, and/or markers 64 that were originally located in the reference image 56.
• a torsionally offset, ⁇ o, between reference image 56 and video frame image 66 of the patient's eye is calculated.
• the computer can track the translational position (x(t), y(t), and z(t)) of the patient's eye E with a high speed eye tracker (HSET) 68 and the torsional orientation ( ⁇ (t)) of the eye with a torsional tracker 70. Because the position of the center of the pupil is tracked with the HSET 68, the torsional tracker 70 generally has to estimate the position of the markers 64 with respect to the pupil center.
• the computer can correct the delivery of the customized ablation pattern by adjusting the patient's customized treatment table 60 by adding in the translation and torsional measurements into the table.
• the treatment table can be adjusted such that at time t, if the overall rotation angle of the eye is ⁇ (t), and the next pulse of the laser is supposed to be delivered at location (x,y) on the cornea, the new location of the delivery of the pulse can be defined by:
• torsional tracker 70 can use the markers 64 identified above, other high-contrast iris patches, or if the patient's iris contains too little texture, the surgeon will have an option of drawing artificial landmarks 72 on the eye for tracking.
• the algorithm it is possible for the algorithm to decide if artificial markers are required.
• the translational position and torsional orientation ofthe patient's eye can be tracked and analyzed by a computer processor in real-time so that the x(t), y(t), z(t) and ⁇ (t) information 74 can be used to adjust the customized treatment table 60 so that laser 12 delivers the appropriate ablation pattern 76 to the patient's eye.
• a first step of the present invention entails registering a reference image ofthe eye taken during the calculation ofthe wavefront elevation map with a second image ofthe eye taken just prior to the delivery ofthe ablation energy.
• Figures 5 to 9 illustrate aspects of one embodiment of a method of the present invention.
• Figure 5 schematically illustrates the data flow through an alignment algorithm that can torsionally register a reference image with a second image ofthe eye to determine the torsional displacement between the two images ofthe eye.
• An initial step in the method is to obtain the first, reference image.
• the images were 768x576 pixels and have 256 gray levels.
• the image contains the pupil and the iris. In some images, part ofthe iris may be occluded by one or both of the eyelids or cropped by the camera's field of view.
• the present invention can use a variety of imaging devices to produce different images and can be illuminated under various types of illumination.
• the smallest distance between the edge ofthe pupil and the obstructing elements, such as eyelids, eyelashes, strong shadows or highlights should be sufficiently large to leave a portion ofthe iris completely exposed for the entire 360-degree range.
• the largest possible portion ofthe iris is in sha ⁇ focus so as to expose its texture.
• a pupil finding algorithm can be used to locate the pupil, calculate the radius ofthe pupil and find the center ofthe pupil (Step 82).
• the pupil is located by thresholding the image by analyzing a pixel value histogram and choosing the position of a first "dip" in the histogram after at least 2000 pixels are below the cutoff threshold. All pixels below the threshold are labeled with "1" and pixels above the threshold are labeled with "0". Pixels labeled with "1" would generally correspond to the pupil, eyelashes, and possibly other regions ofthe image. It should be appreciated however, that the number of pixels employed will be related to the area ofthe pupil and will vary with applications ofthe invention.
• the two distinguishing features about the pupil region, compared to other non-pupil regions is its large size and central location.
• regions intersecting with a 5 -pixel wide inner frame ofthe image can be discarded and the largest remaining region can be selected as the pupil.
• the selected pupil region can be filled to remove any holes created by reflections, or the like.
• the remaining region of the image may also be analyzed for convexity. If the ratio ofthe area ofthe region to the area of its convex hull was less then 0.97, a circle completion procedure can be applied to the convex points on the region's boundary.
• One way of performing such an analysis is through a Matlab function "imfeature(...,'CovexHull')".
• a radius and center ofthe pupil can be estimated by a standard weighted least-square estimation procedure. If the convexity quotient was above 0.97, the radius and centroid can obtained using conventional methods, such as Matlab's "imfeature(..., 'Centroid', ⁇ quivDiameter')" function.
• an iris finding algorithm can be used to locate the iris, calculate the radius ofthe iris, and/or locate the iris center. Since the images ofthe eye from both imaging assembly 11 and the camera 20 both contain the pupil and iris, in some embodiments it may be more accurate to register the images by calculating the center ofthe pupil and the center ofthe iris and expressing the position ofthe pupil center with respect to the center of the iris.
• the center of the iris may be described as a center of a circle corresponding to the outer boundary ofthe iris.
• the position ofthe center ofthe iris can be used to calculate a pupil offset from the iris center.
• FIG. 6C and 6D schematically illustrate simplified images ofthe eye taken with image assembly 11 and camera 20, respectively that can be analyzed to find the pupil center and iris center.
• Marker 200 marks the iris center in both images, marker 204 corresponds to the pupil center in image 56 and marker 206 corresponds to the pupil center in the laser image 66.
• the pupil has changed in size (as shown by the gray outline) and the center ofthe pupil has moved relative to the center ofthe iris 200.
• the measured wavefront measurement and corresponding ablation pattern can be centered over center position C that is calculated by the above equation.
• the boundary of the iris may be soft in terms of contrast and may also degraded by shadows and light reflections, there may be difficulties associated with detecting the outer iris boundary in infrared images ofthe eye.
• One method for detection of both iris and the pupil in the image I(x,y) is to minimize the following integral over all possible values of iris radius and center:
• the pupil center has already been found (as described above), that the iris has a limited range of possible values and the iris center is usually not very far from the pupil center.
• Figure 6E since the center ofthe pupil and the center ofthe iris are not far from each other, it is possible to estimate the radial derivative ofthe image intensity with respect to the iris center by the radial derivative with respect to the pupil center.
• the limited range of iris radius values occurring in nature, allows restriction of a range of possible search to a ring centered at pupil center and having inner and outer radii such that the iris edge should always be located somewhere within the range.
• the numerical search range can be between approximately 10.5 mm and 14 mm. In other embodiments, the range may be larger or smaller, if desired. See Burns et al., IOVS, July 2002.
• circles 208, 210 illustrate a potential range for the iris radius.
• the values ofthe radial derivative that exceed certain threshold can be passed to the weighted least square estimator for the best circle fit through the set of points, as is described herein.
• the initial weights ofthe points are proportional to their intensity. After enough iterations (e.g., two iterations) are performed to converge to a stable solution, the algorithm converges to the answer represented by the red circle.
• the iris finding algorithm shows tolerance to other edges detected by the derivative operator, but corresponding to other structures in the image (e.g., LASIK flap).
• the original images can be smoothed with a Gaussian kernel and sub-sampled by a factor of four prior to a derivative computation.
• the boundary ofthe iris can be localized with sub-pixel accuracy, but it might be slightly displaced from its true location if the shadows in the image soften the boundary edge. However, the errors are fairly well balanced in all directions from the center, so that the final result is very close to the actual center.
• the first image e.g., wavefront image
• the first image e.g., wavefront image
• An error of one pixel in the boundary estimation on one side ofthe iris would result in about 10 ⁇ m error in the estimate ofthe iris center.
• the errors of a few pixels in the iris boundary would still be within the acceptable accuracy for the ablation centering.
• a width of the iris ring can be extracted from the images.
• the iris can be treated as an elastic sheet stretched between pupil and the outer rim ofthe iris.
• the width ofthe iris band can be set to 76 pixels for images of dark- colored eyes, and 104 pixels for the light-colored eyes. It should be appreciated, however, that other width estimations can be used.
• the radius ofthe iris in the reference images of Figures 6A and 6B were estimated to be 320 pixels and assumed to be roughly constant for all people.
• the iris ring can then be unwrapped and divided into a fixed number of sectors, by converting the Cartesian iris coordinates into polar coordinates, centered at the pupil. (Step 86).
• Applicant has found that unwrapping and scaling the iris ring allows better matching of texture blocks between different images ofthe eye by means of pure translation.
• the software may have trouble matching of texture blocks that have rotated ( Figure 7C), whereas if the iris ring is unwrapped, the texture blocks have the same relative shape ( Figure 7D).
• the iris ring can be sampled at one-pixel steps in the radial direction for the reference image.
• the dynamic range of pixel values in the iris may be adjusted to remove outliers due to reflections from the illumination LED lights.
• the pixel value histogram can be thresholded so that all the pixels with values above the threshold are assigned the value ofthe threshold.
• some band-pass filtering may be applied to the iris bands prior to region selection to remove lighting variation artifacts.
• the iris region is segmented into twenty four sectors of fifteen degrees. It should be appreciated, however, that in other embodiments, the iris region can be segmented into more than twenty four sectors or less than twenty four sectors.
• the markers in the reference image can be stored and later located in the second image ofthe eye so as to estimate the torsional displacement ofthe eye between the two images.
• One embodiment of a method of locating the markers is described more fully in Groen, E., "Chapter 1 on Video-oculography,” PhD Thesis, University of Utrecht (1997), the complete disclosure of which is inco ⁇ orated herein by reference.
• the markers should be sufficiently distinct and have high contrast. There are several possible ways to select such points.
• a square mask of size MxM (for example, 21x21 for dark-colored eyes and 31x31 for light-colored eyes) is defined.
• the mask can be scanned over each ofthe twenty four sectors, and for each pixel in each sector a value is computed from the region inside the mask centered at that pixel.
• the value assigned to the pixel is determined as the sum of amplitudes of all spatial frequencies present in the region.
• the sum ofthe amplitudes can be computed by a Fourier transform ofthe region. If desired, the central 5x5 portion ofthe Fourier spectrum can be nulled to remove a DC component.
• the maximum value can then be located in each sector, such that the boundary of its corresponding mask is at least 5 pixels away from the iris image boundary in order to avoid getting close to the pupil margin and other boundary artifacts, such as the eyelid and eyelashes.
• the "winning" positions and the corresponding blocks are stored for later comparison.
• the second image ofthe eye can also be obtained.
• the second image is obtained with a laser surgical system's microscope camera prior to delivering the ablative energy to the patient.
• the laser camera has a resolution of 680x460 pixels using 256 grayscale levels.
• the magnification ofthe laser camera in relation to the reference camera from the CCD camera was estimated to be 0.885.
• the eye can be illuminated by a set of infrared LED lights having a wavelength of 880 nm. It should be appreciated, however, that many other imaging devices can be used to obtain different image types, including images that do not require a magnification, images of different resolution, and images that are illuminated by other light wavelengths.
• the sectors in the second image are located and the salient regions that correspond to the salient regions in the reference image are located. (Step 94; Figure 7B). For each sector in the second image, a best matching region is located.
• the search is constrained to the matching sector and the two adjacent sectors in the second image, thus limiting possible matches to within 15 degrees, which is a reasonable biological limit for ocular cyclo-rotation. It should be appreciated however, in other embodiments, the range of limiting the possible match may be larger or smaller than 15 degrees.
• an angular displacement for each marker is calculated to estimate a total torsional angle ofthe eye between the first, reference image and the second image. (Step 96; Figure 9).
• the displacement of each marker would be identical and equal to the torsional angle.
• the center ofthe pupil may not be estimated correctly. This introduces a sinusoidal distribution of displacement angles around the true torsional angle. The amplitude ofthe sinusoid is usually quite small.
• the actual shape ofthe pupil is often elliptical and not round.
• the estimated angles can be fitted with a number of different functions using an iterative weighted estimation as follows:
• TAs are the estimates ofthe true torsional angle and ⁇ is the angular coordinate ofthe markers.
• Application ofthe functions to the torsional angle data can thereafter provide an estimate for the torsional angle ⁇ 0 between the reference image and the second image.
• the initial torsional angle, ⁇ 0> computed by the alignment algorithm can be added to every subsequent frame for tracking ofthe torsional orientation ofthe patient's eye.
• the total torsional orientation ⁇ tota - (t) ofthe patient's eye in the laser image can be described as follows:
• ⁇ (t) is the measured torsional angle between the eye in the initial frame ofthe video stream and the eye in the n th frame at time t.
• the alignment algorithm that calculates ⁇ 0 does not have to produce results in real time
• a tracking algorithm that tracks the torsional rotation ⁇ (t) ofthe eye should work at frame rate, which demands quick, efficient and accurate computations.
• the high speed eye tracker (HSET) ofthe laser surgical system can be used to keep track ofthe translation ofthe pupil the x, y, and z directions. Having the position of the pupil readily available requires only that the torsional tracker estimate the positions ofthe iris landmarks with respect to the center ofthe pupil.
• the iris can undergo rigid translations (e.g., movement in the x, y, and z directions), rotations, as well as some non-rigid affine transformations of scaling and shearing. While the torsional angle is not affected by the non-rigid transformations, it is preferable that the non-rigid transformations be taken into account in order to ensure accurate feature matching from frame to frame.
• A 1 + D
• D a deformation matrix
• d the translation ofthe feature window.
• computer vision literature such as Lucas B.D. and Kanade, T. "An Iterative Image Registration Technique and Application to Stereo Vision” ILCAI (1981), Shi, J. and Tomasi, C. “Good Features to Track,” IEEE Conference on Computer Vision and Pattern Recognition 1994, and Hager, G.D. and Toyama, K. "X- Vision: A portable Substrate for Real-Time Vision Applications,” Computer Vision and Image Understanding 1996, the complete disclosures of which are inco ⁇ orated herein by reference. Parameters of deformation and translation are determined by Newton-Raphson minimization procedure which can produce accurate results.
• FIG. 10 schematically illustrates a simplified method of tracking the torsional rotation ofthe patient's eye during the surgical procedure.
• the pupil and iris are located in both the first frame and n th frame ofthe video stream.
• Reference points can be located in the first frame and the corresponding reference points can be located in the n th frame ofthe video stream.
• the angular offset between the reference points in the two images can then be calculated to estimate the torsional rotation ofthe eye.
• Step 104 The steps can be repeated for each of frames ofthe video stream until the ablation procedure is completed.
• Step 105 ).
• Figure 11 is an example of a first frame 106 from the video stream of the eye taken prior to the laser ablation.
• a pupil 108 has been located (as noted by circular outline 110 image around the circumference ofthe pupil), and two reference loci or points 112, 114 are selected for torsional fracking.
• reference points 112, 114 are a subset ofthe points chosen for registration (described above).
• the points 112, 114 can be chosen automatically by the software ofthe present invention based on its texture strength, and positioning relative to the pupil (e.g., 8 o'clock position and 2 o'clock position). In alternative embodiments, however, it may be possible to independently select points 112, 114 separate from the original markers using the same technique described above or to manually select or draw the reference points 112, 114 on the patient's iris.
• the process of selecting points for fracking can be automatic or surgeon-assisted.
• the automatic process can select one point on the right ofthe pupil and one on the left based on which reference block in the corresponding neighborhood has best block-match score and also included in the estimate ofthe alignment angle, i.e. not an outlier.
• One exemplary selection algorithm selects a subset of blocks that are not outliers. From this subset, blocks are removed that are in the positional domain of possible reflections.
• the texture ofthe remaining blocks from the laser image may be quantified by the second largest eigenvector ⁇ 2 .
• Two blocks, roughly on the opposite sides ofthe pupil are chosen, such that they have the largest ⁇ 2 in the group.
• the "left block” is selected from the valid blocks centered around the 8-o' clock position, and the "right block” is selected among the valid blocks centered at the 2-o'clock position. The coordinates ofthe centers of these blocks can be used to initialize tracking.
• the blocks/loci 112, 114 have been selected in the first frame, for each consecutive frame ofthe video feed, the blocks are located within a region ofthe iris that has the same position with respect to the pupil ofthe eye.
• the region is generally limited to approximately 15 degrees, since the eye will generally not rotate more than such a range, and within such a time between each consecutive frame ofthe video stream, the torsional rotation will likely be much less than the 15 degrees.
• the range of analysis can be limited to a smaller or larger range, if desired.
• the spatially corresponding regions ofthe first frame and the n th frame can be compared for affine displacement, giving preference to rigid transformations. In one embodiment, only horizontal and vertical displacements are reported by the tracking algorithm.
• Figure 12 illustrates six images of selected blocks 112, 114. Images 116, 118 are images of blocks 112, 114 in reference image 66. Blocks 120, 122 are the corresponding blocks from the new, real-time frame. Block images 124, 126 are the best fransformed block from the first frame that match the target block. From the change in the positional coordinates ofthe blocks 112, 114, a torsional angle between the first frame and the second frame can be computed. ( Figure 13).
• one part ofthe described embodiment ofthe tracking algorithm is to estimate the motion parameters of a given block or marker. If I is the block in the original frame and J is the spatially corresponding block in a subsequent frame, let x be the pixel coordinates in these blocks. To estimate an affine transformation matrix A and translation vector D, the following equation can be minimized:
• Matrix A can be decomposed into a rotation component and a scale/shear component as follows:
• a linear system for computing rigid motion parameters is:
• Gy(x) —I(x) * w(x) dy
• Gr(x) ⁇ y- -I(x) - x- -I(x) ⁇ w(x) ox dy
• magnification factor ofthe laser's camera was adjusted to match that ofthe imaging device ofthe wavefront measurement device, thus eliminating scaling issues. Also, as the resolution ofthe laser camera increased due to larger magnification factor, more details became visible on the light-colored irises.
• a majority of the iris should be visible so that a minimum width ofthe iris ring is more then 80 pixels.
• the focus ofthe camera should be adjusted so that most of the iris is in focus providing the highest possible texture resolution ofthe iris ring. Several images can be taken to ensure good quality.
• images with strong shadows and reflections on the iris should be rejected in order to avoid strong false markers.
• images should be saved into a file of type BMP or TIF.
• image names should contain unique name ofthe subject, left or right indicator for the eye and the ID ofthe device from which they come (e.g., laser image or wavefront image). [155] While capturing the laser image the same reference points should be considered.
• the illumination when obtaining the wavefront image should be the same when obtaining the image with the laser camera.
• Applicants have found that dark- colored eyes have more rich texture under the infrared illumination and light -colored eyes have more rich texture under visible light.
• the striated trabecular meshwork of elastic pectinate ligament creates a predominant texture under visible light.
• deeper slower modulated stromal features dominate the iris pattern. See for example Daugman, J. "High confidence visual recognition of persons by a test of statistical independence," IEEE Transactions of Pattern Analysis and Machine Intelligence, vol. 15(11), pp 1148-1161 (1993).
• Image quality may also be degraded by LED reflections. However, because illumination is required, it may be unavoidable to have several LED reflections on the iris. These features can be handled by the algorithm as described above. These reflections, however, can greatly degrade the image quality. As shown in Figure 17 A, the shadow makes it impossible to discern any texture ofthe right side ofthe iris. As a result, as shown in the Figure 17B, the alignment data obtained from the image in Figure 17A was rejected due to the large RMS factor (i.e., above 1). Therefore, the alignment algorithm of the present invention can have an internal quality of fit check that automatically rejects bad data.
• a first step was to mark the expected position ofthe LASIK flap as an invalid region, preventing the algorithm from selecting reference blocks in that area ofthe iris.
• a second step is to apply band-pass filtering to the unwrapped iris images.
• the convolution kernel was set to be the difference of 2-D Gaussian distributions with standard deviations equal to 3 and 12 pixels.
• a third step was the introduction of bi-directional alignment, when the blocks were selected and matched from the wavefront device to the laser and from the laser to the wavefront device. This essentially doubled the number of data points used for sinusoidal fitting.
• the second set of results comes from a 500-frame sequence capturing 25 seconds of real video of an eye.
• Several variables were tracked during the video processing: pupil center position, pupil radius, torsional angle, and error estimates for the two blocks tracked for each frame. The sequence was also visually inspected to verify the black match and the overall eye torsion.
• the zero 1 frame ( Figure 19 A) was used as a reference with two 31 x 31 pixel blocks marked for tracking. The last frame shows the same blocks at the appropriate locations.
• Figure 19B Figures 20-23 show the data extracted from the video sequence.
• Figure 20 shows the pupil position over time.
• Figure 21 shows the change ofthe pupil radius from frame 0 to 500.
• Figure 22 illustrates errors per frame/block.
• Figures 23 shows the torsional angle ofthe markers (relative to the first frame ofthe video).
• #totai( ) Tracking[(reference image, video](t) + Alignment[Wavefront reference image]
• Figures 29A and 29B show two different torsional angle estimates that include the alignment with the wavefront measurement image.
• the reference frames for the two estimates were 0.41 degrees clockwise 134 (Figure 29A) and 1.17 degrees clockwise 136 (Figure 29 A).
• the errors between the estimates are shown in Figure 29B as a function ofthe frame number. As in previous tests, the errors do not exceed 1 degree for any frame.

Landscapes

• Health & Medical Sciences (AREA)
• Engineering & Computer Science (AREA)
• Life Sciences & Earth Sciences (AREA)
• Ophthalmology & Optometry (AREA)
• Physics & Mathematics (AREA)
• General Health & Medical Sciences (AREA)
• Veterinary Medicine (AREA)
• Surgery (AREA)
• Animal Behavior & Ethology (AREA)
• Biomedical Technology (AREA)
• Public Health (AREA)
• Heart & Thoracic Surgery (AREA)
• Medical Informatics (AREA)
• Molecular Biology (AREA)
• Biophysics (AREA)
• Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
• Vascular Medicine (AREA)
• Optics & Photonics (AREA)
• Theoretical Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Human Computer Interaction (AREA)
• General Engineering & Computer Science (AREA)
• Signal Processing (AREA)
• Radiology & Medical Imaging (AREA)
• Quality & Reliability (AREA)
• Laser Surgery Devices (AREA)
• Eye Examination Apparatus (AREA)
• Image Processing (AREA)
• Length Measuring Devices By Optical Means (AREA)

Priority Applications (5)

Applications Claiming Priority (2)

Publications (1)

Family

ID=29712073

Family Applications (1)

Country Status (8)

Cited By (15)

Families Citing this family (216)

Citations (4)

Family Cites Families (116)

• 2002
  • 2002-11-19 EP EP02784501.5A patent/EP1516156B1/en not_active Expired - Lifetime
  • 2002-11-19 AU AU2002346438A patent/AU2002346438A1/en not_active Abandoned
  • 2002-11-19 US US10/300,714 patent/US7044602B2/en not_active Expired - Lifetime
  • 2002-11-19 CA CA2487411A patent/CA2487411C/en not_active Expired - Fee Related
  • 2002-11-19 WO PCT/US2002/037051 patent/WO2003102498A1/en not_active Ceased
  • 2002-11-19 CN CNB028293150A patent/CN100442006C/zh not_active Expired - Fee Related
  • 2002-11-19 JP JP2004509342A patent/JP4256342B2/ja not_active Expired - Fee Related
  • 2002-11-19 MX MXPA04011893A patent/MXPA04011893A/es active IP Right Grant
• 2006
  • 2006-03-28 US US11/277,743 patent/US7261415B2/en not_active Expired - Lifetime
• 2007
  • 2007-07-10 US US11/775,840 patent/US7431457B2/en not_active Expired - Lifetime
• 2008
  • 2008-09-15 US US12/210,933 patent/US8740385B2/en active Active
• 2014
  • 2014-04-22 US US14/258,854 patent/US9596983B2/en not_active Expired - Fee Related
• 2017
  • 2017-02-10 US US15/430,087 patent/US10251783B2/en not_active Expired - Lifetime

Patent Citations (4)

Non-Patent Citations (3)

Also Published As

Similar Documents

Legal Events