Classifications

• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
  • A61F9/00—Methods or devices for treatment of the eyes; Devices for putting in contact-lenses; Devices to correct squinting; Apparatus to guide the blind; Protective devices for the eyes, carried on the body or in the hand
  • A61F9/08—Devices or methods enabling eye-patients to replace direct visual perception by another kind of perception
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B26/00—Optical devices or arrangements for the control of light using movable or deformable optical elements
  • G02B26/08—Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
  • G02B26/0816—Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light by means of one or more reflecting elements
  • G02B26/0833—Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light by means of one or more reflecting elements the reflecting element being a micromechanical device, e.g. a MEMS mirror, DMD
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B26/00—Optical devices or arrangements for the control of light using movable or deformable optical elements
  • G02B26/08—Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
  • G02B26/10—Scanning systems
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B26/00—Optical devices or arrangements for the control of light using movable or deformable optical elements
  • G02B26/08—Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
  • G02B26/10—Scanning systems
  • G02B26/101—Scanning systems with both horizontal and vertical deflecting means, e.g. raster or XY scanners
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
  • G02B27/0093—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
  • G02B27/01—Head-up displays
  • G02B27/0101—Head-up displays characterised by optical features
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
  • G02B27/01—Head-up displays
  • G02B27/017—Head mounted
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
  • G02B27/10—Beam splitting or combining systems
  • G02B27/1006—Beam splitting or combining systems for splitting or combining different wavelengths
  • G02B27/102—Beam splitting or combining systems for splitting or combining different wavelengths for generating a colour image from monochromatic image signal sources
  • G02B27/104—Beam splitting or combining systems for splitting or combining different wavelengths for generating a colour image from monochromatic image signal sources for use with scanning systems
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N3/00—Scanning details of television systems; Combination thereof with generation of supply voltages
  • H04N3/02—Scanning details of television systems; Combination thereof with generation of supply voltages by optical-mechanical means only
  • H04N3/08—Scanning details of television systems; Combination thereof with generation of supply voltages by optical-mechanical means only having a moving reflector
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N9/00—Details of colour television systems
  • H04N9/12—Picture reproducers
  • H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
  • H04N9/3179—Video signal processing therefor
  • H04N9/3188—Scale or resolution adjustment
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
  • G02B27/01—Head-up displays
  • G02B27/0101—Head-up displays characterised by optical features
  • G02B2027/0118—Head-up displays characterised by optical features comprising devices for improving the contrast of the display / brillance control visibility
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
  • G02B27/01—Head-up displays
  • G02B27/0101—Head-up displays characterised by optical features
  • G02B2027/0138—Head-up displays characterised by optical features comprising image capture systems, e.g. camera
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
  • G02B27/01—Head-up displays
  • G02B27/0101—Head-up displays characterised by optical features
  • G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
  • G02B27/01—Head-up displays
  • G02B27/0101—Head-up displays characterised by optical features
  • G02B2027/0141—Head-up displays characterised by optical features characterised by the informative content of the display
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
  • G02B27/01—Head-up displays
  • G02B27/017—Head mounted
  • G02B2027/0178—Eyeglass type

Definitions

• the present invention relates to an apparatus, a system and a method for aiding the vision of visually impaired individuals, and in particular, to such an apparatus, system and method which assists visually impaired individuals having a retina with reduced functionality.
• Visually impaired individuals frequently suffer from reduced vision due to reduced functionality of their retina. Such reduced functionality may be due to damage, and/or to various disease processes which gradually reduce the function of the retina and hence lead to impaired vision over time.
• One example of such a disease process relates to AMD, or age- related macular degeneration, the leading cause of blindness among adults in the US.
• the macula is the part of the retina which provides sharply focused images and hence is particularly required for such activities as reading and driving.
• AMD poses another challenge, which is that since its effects on the retina are uneven, mainly affecting the macula, such that assistive or corrective devices need to be able to relate to uneven retinal function, which may further vary between individuals.
• assistive or corrective devices need to be able to relate to uneven retinal function, which may further vary between individuals.
• the most effective devices have involved implants to the eye, but these devices are highly experimental, very invasive and clearly undesirable (from a therapeutic standpoint) in many ways.
• Some taught devices do not require any implantable element, as described for example in US Patent No. 7023621 to Dietrich; however this device assumes an ideal retina, with entirely homogenous functioning over the entire retina. For a normal, healthy retina, such an assumption, while incorrect, may still yield a reasonably effective device; however, for a retina with reduced function, or otherwise uneven function, such an assumption is not correct and greatly reduces the effectiveness of the device.
• the background art does not teach or suggest an apparatus, system or method for aiding the vision of visually impaired individuals having a retina with reduced functionality.
• reduced functionality it is meant that the retina functions at a lower or lesser level than for a normal healthy retina, and/or that the retina has uneven function at different locations, so that for example and without limitation, one or more parts of the retina at different locations may function at a higher level, while one or more other parts of the retina, at other locations, may function at a lower level.
• the present invention is of an apparatus, system or method for aiding the vision of visually impaired individuals having a retina with reduced functionality, which overcomes the drawbacks of the background art by overcoming such reduced and/or uneven retinal function.
• the system of the present invention comprises a mapping unit, for mapping the retinal function at various locations of the retina; and a projection unit for projecting light onto the retina as a video projector, in which the location and/or intensity of the light at various parts of the retina is determined at least in part by the mapped retinal function.
• both units may be implemented in a single apparatus; the mapping software and analysis functions may also optionally be implemented in that apparatus or alternatively may be implemented at a remote location. Communication between such a remote location and the apparatus may optionally be wireless or wired, as necessary.
• the level of retinal function at each of a plurality of portions of the retina determines whether that portion of the retina is stimulated through the video projection process or alternatively whether that portion of the retina is avoided during the video projection process; for the latter situation, the video data is projected onto the "remapped" retina, in which excessively low function portions of the retina are not stimulated but instead are avoided, as described in greater detail below.
• this division between “stimulation” and “avoidance” may be implemented flexibly, according to the overall light level available, the type of vision desired (reading, driving, night vision) and so forth.
• the apparatus in order to compensate for movement of the eye, further comprises an eye tracker.
• Figure 1 shows a virtual retinal display (V D) according to at least some embodiments of the present invention that projects a direct video image onto the user's retina;
• the VRD 100 comprises light source(s) 120 such as for example a laser, or more particularly a laser diode;
• Figure 3 shows an exemplary, illustrative, optional embodiment of the projection optics 140 as a schematic block diagram
• FIG. 4 shows operation of video processor 170
• Figure 5 is an exemplary segmentation of retinal FOV where each angular segment is represented with a cell in the RST;
• Figure 6 is an example of the first 10 rows by 10 columns of an illustrative RRST, for a retina which demonstrates reduced sensitivity within the center of this region;
• Figure 7 relates to an exemplary, illustrative method for receiving an incoming image and for projecting that image appropriately onto the retina of the subject through the retinal projection apparatus;
• Figure 8 shows exemplary analog video signal derived from the camera of a raster scan; the middle plot shows the retinal position of the light signal during the scan; the lower plot shows the video signal along the retinal position;
• Figure 9 is a video signal resulting from the camera, here shown as a digital signal for clarity; the middle plot indicates the relative sensitivity scores of the retina according the position of the retina;
• Figure 10 in the upper diagram (10a), shows an image of letters ABC represented with the retinal projector as raster scan of row 3 and 4; the second diagram of Figure 10 (10b) represents how the projected image appears in field of vision with stocoma at the central portion; the third diagram of Figure 10 (10c) shows the letters ABC when the raster image was projected at rows 5 to 6;
• Figure 11 relates to an exemplary, illustrative method according to at least some embodiments of the present invention for receiving an incoming image and for projecting that image appropriately onto the retina of the subject through the retinal projection apparatus, for example as described herein, in which one or more portions of the retina are essentially nonfunctional;
• Fig 12 shows the schematics of an exemplary, illustrative scan over the field of vision where the indicated path, in the shape of an "S-curve" with a plurality of inflection points, indicates the trajectory of the scan.
• Photons from a light source are modulated with video information by an intensity modulator.
• the modulated light is scanned in a first direction and in a second direction generally perpendicular to the first direction by a scanners to create a raster of photons that is projected directly onto the retina of the user by projection optics to produce the perception an image without any intermediate image outside of the eye that is viewed or perceived by the user.
• the light source can be a monochromatic light or beams of red, green and yellow or blue light may be RGB video signals to scan colored photons directly onto the user's eye.
• V D virtual retinal display
• the VRD 100 utilizes a light beam 110 emitted by a light source(s) 120, an optical scanner 130 that scans the light beam 110 in a raster mode horizontally and vertically , and is further relayed by projection optics 140 through the pupil 151 of user's eye 155 directly on the retina 152.
• a video processor 170 accepts an external video signal 171 from an external video source 175 and processes signal 171 in accordance with aspects of mapping as described in greater detail below, and optionally and preferably also according to the instantaneous position of pupil 151 as optionally and more preferably detected by an eye tracking system 160.
• the video processor 170 then generates a synchronization signal 172 to the scanner 130, and a modulation signal 173 according which light beam 110 is intensity modulated.
• the intensity modulated beam of light 110 as emitted from scanner 130 through optics 140 raster scans the retina 152 and so produces the perception of an image without any intermediate image outside of the eye 155.
• VRD 100 Since the image is formed directly on the retina, without any intermediate displays or projection screens, the device can be reduced in size and weight, which in turn makes VRD 100 suitable for mounting onto head 150 of the user. However in any case it should be noted that VRD 100 as described herein is not intended for implantation to the eye 155 or attachment to the eye 155 and instead remains outside of the eye 155; only the intensity modulated beam of light enters the eye 155. As seen in Fig 1 VRD 100 preferably features a head-mounted part 101 which includes the light source(s) 120, the scanner 130, projection optics 140 and optionally and preferably eye tracking system 160.
• a power source such as battery (not shown) may optionally be located in a remote module 102 which can also be portable but need not be located in proximity to the eye 155.
• a power source such as battery (not shown)
• the power source of remote module 102 is connected to head-mounted part 101 with a wire 103 or other power connector or cable.
• the VRD 100 comprises light source(s) 120 such as for example a laser, or more particularly a laser diode.
• the VRD 100 may contain one or several light sources 120.
• light sources 120 preferably feature at least three lasers which emit at red, green and blue (designated 201, 202, and 203
• the light sources 120 do not need to be coherent or at a narrow spectral width; light emitting diodes (LED) may also optionally be employed.
• LED light emitting diodes
• the individual light sources 120 are preferably further provided with collimation optics 205 as shown.
• the light beam 110 emitted has low divergence.
• Light emitted from a laser source can become collimated to a beam with a very small divergence angle.
• collimation of incoherent light as LED is more difficult.
• the LED can be collimated by using an aspheric lens with numerical aperture greater than or equal to numerical aperture of the beam, which could for example optionally be incorporated into collimation optics 205 as shown.
• the beam will have residual divergence.
• VRD 100 comprises further optical elements to project the light into the pupil 151, such as projection optics 140 for example, a LED source can still provide sufficient intensity and low divergence to be utilized as light source 120.
• the light sources 120 are optionally and preferably further provided with a light attenuator 206 such that the light emitted from light source 120 is well within the eye-safe intensity level.
• Light attenuator 206 may optionally comprise silica glass or ceramics for example.
• each light source 120 is optionally and preferably provided with a separate light attenuator 206, as shown in the exemplary configuration of Figure 2.
• Projection of color images provides a particular challenge, because the intensity perception of the eye 155 is different at each color (wavelength); therefore each color needs to be attenuated separately and to a differing degree to comply with eye-safety requirements at the specific spectral range.
• the required safety levels are typically given in various regulations and directives e.g., the US standard ANSI Z136 or the European standard EN 207.
• the beams from individual light sources are optionally and preferably combined by a beam combiner 207 based on dichroic filters, or spectrally dispersive components, to form light beam 110.
• a beam combiner 207 based on dichroic filters, or spectrally dispersive components, to form light beam 110.
• the beam of red light 221 emitted from the red light source 201 is first bent by mirror 224 and subsequently combined with beam of green light 222 by dichroic filter 225.
• the dichroic filter 225 transmits the red light 221 and reflects the green light 222.
• dichroic filter 226 reflects the blue light 223, while the red and the green beams 221, 222 are transmitted.
• a beam of light 110 that contains all RGB components is emitted from the beam combiner 207.
• an additional light attenuator 211 is situated to receive light beam 110.
• the light passes a polarization control unit 212 that can contain polarizers, waveplates such as ⁇ /2 or ⁇ /4, or other optical elements for controlling the polarization.
• polarization control unit 212 can contain polarizers, waveplates such as ⁇ /2 or ⁇ /4, or other optical elements for controlling the polarization.
• additional attenuation can be provided by the polarization elements of polarization control unit 212.
• a non-limiting example of a device that could be used to implement polarization control unit 212 is a variable attenuator based on polarization control, which may for example and without limitation be purchased from Newport Inc. (Irvine, CA, USA), (http://assets.newport.com/webDocuments- EN/images/Variable Attenuators.pdf).
• the beam 110 can be further deflected, here shown as being bent by mirror 227.
• the light beam 110 passes through an intensity monitor unit ("light tap") 215.
• a small fraction of light 213 is split from the beam 110 by a beam splitter 214, typically 1% to 5% of the main beam 110.
• Light fraction 213 is then fed to intensity monitor unit 215 and used to monitor the light intensity.
• the signal from intensity monitor 215 is used as a feedback signal 216 to control the light intensity and to disable (shut off) the light sources 120, which provides additional level of safety to the operation of VRD 100.
• feedback signal 216 is optionally and preferably communicated to control logic electronics 217
• each light source 120 could have its own control logic electronics 217).
• Control logic electronics 217 compare the value of the power of light source 120, such as a laser for example, to a predetermined value (which corresponds to eye-safe power value). If the monitored light power is exceeds the predetermined value, then control logic electronics 217 would shut down the respective light source 120 or light sources 120.
• the light source 120 can be disabled in a variety of ways, for example by interrupting the supply of power to light source
• control logic 217 is preferably implemented in the driver wherein light emission is disabled by turning off the current to the laser.
• Control logic electronics 217 is an example of a modulator, albeit in the form of a cut-off device.
• light beam 110 is also optionally intensity modulated as follows.
• the video processor 170 generates a modulation signal 173 according which the beam of light 110 is intensity modulated.
• the intensity modulation is achieved by modulating the current driving the light source 120, which may for example be a laser diode.
• the light source 120 is directly modulated the emitted beam of light 110 emerges from the source
• the modulation signal 173 is fed as a drive signal to a transducer of the acousto-optic modulator.
• the modulation signal is preferably provided to all sources 120, according to the RGB video information for RGB light sources.
• the signals are communicated to the external modulator drivers and each light source 120 is provided with a modulator (not shown).
• the VRD 100 may include a safety feature where the laser is disabled by the intensity monitor signal 216.
• modulators can be utilized.
• the dynamic range of the modulation is defined by the original image data requirements and it is limited by the actual modulation ability of the chosen light modulating device.
• the light beam 110 incident on the scanner 130 is intensity modulated. With color projection each individual beam is intensity modulated according the RGB video information before being combined to light beam 110.
• the beam scanner 130 is used to steer or deflect the light beam 110 horizontally and vertically in a two dimensional raster pattern.
• the deflected beam 111 is further focused on the retina 152 by the projection optics 140, and optical system of the eye 155.
• a particular point on the retina 152 where the light focuses (such a point may also be referred to herein as a "pixel") corresponds to a particular horizontal and vertical angle spanned by the scanner 130.
• the focused beam 112 scans the retina 152 as the scanner 130 causes focused beam to scan horizontally and vertically.
• the intensity of the focused beam corresponds to the light intensity of the modulated beam 110. In such a manner a raster display is drawn directly onto the retina 152 which the user perceives as an image.
• beam scanner 130 features a fast steering mirror for creation of the row raster, described herein as horizontal scanner 231, and a slow addressable axis spanning the rows, described herein as vertical scanner 232.
• MEMS micro-electro ⁇ mechanical systems
• Scanner requirements for horizontal scanner 231 and vertical scanner 232 are defined by at least the following factors: frame rate, resolution, pixel count, and optical system limitations.
• Fast steering axis ("horizontal scanner") 231 frequency is defined as [M of rows in a frame] x [N of frames], in which each frame is an image projected onto the retina as the previously described raster display which is composed of a number of rows M.
• Slow steering axis (“vertical scanner”) 232 movement bandwidth is determined by the required frame rate. It is preferable to use an addressable scanner with position feedback for this purpose in order to be able to compensate for various aberrations of the surfaces of the optical path, including with regard to any aberrations of the various lenses and mirrors in the optical path.
• Each of horizontal scanner 231 and vertical scanner 232 is preferably driven by video processor 170 according to synchronization signal 172 for the scanning voltage, but may also include a slow varying voltage that depends on the feedback signal 263 received from the eye tracking system 160 as will be explained further.
• vector scan where the light is steered between specified points, which may also be implemented according to various embodiments of the present invention.
• the mirrors that is, horizontal scanner 231 and vertical scanner 232
• Vector scan mode requires addressable scanners that operate in a close loop with feedback on their position and velocity for both axes.
• the projection optics 140 acts in conjugation with the refractive optical system of the eye 155 to convey the light beam through the pupil 151 and focus on the retina 152.
• the projection optics 140 is designed such that the beam of light 111 deflected by the scanner 130 at angles spanning the entire field of view (FOV) essentially crosses the optical axis at the center of pupil 151 while avoiding vignetting.
• FOV field of view
• Vignetting is a visual artifact which occurs when the peripheral parts of an image appear darker, due to a reduction of the image's brightness or saturation at the periphery compared to the image center.
• Fig 3 shows an exemplary, illustrative, optional embodiment of the projection optics 140 as a schematic block diagram, in which some components have been omitted for the sake of clarity.
• the specific embodiment shown features an arrangement sometimes referred to as a "4-f scanner design" where the scan mirror 305 is conjugated with the pupil 151.
• Lens 301 is positioned essentially such that its back focal plane coincides with the pupil 151. Focusing of the beam 112 on the retina 152 is achieved by the refractive mechanism of the ophthalmic system of eye 155, particularly the cornea 306 and the lens 307.
• the front focal plane of lens 301 coincides with back focal plane of lens 302.
• a third lens 303 may be required so as to provide the fine tuning divergence of the collimated beam 110.
• projecting optics for projections optics 140 are optionally possible including but not limited to more optical elements depending on details of the optical components such as light sources and scanner, and to account for refraction errors of the ophthalmic system (e.g., astigmatisms; not shown).
• an additional lens (not shown) is provided to overcome for ophthalmic conditions such as myopia, or hyperopia, hence a certain beam divergence, or convergence, need to be introduced, while on the other hand a perfectly collimated beam is preferably adapted for individuals that require no refracting correction.
• the projection optics 140 can be designed such that no light other than projected light beam 112 passes to the user's eye 155, or alternatively the projection optics 140 can be designed such that the user views the real world separately from the projected image.
• the virtual image projected by the VRD 100 is preferably superimposed on the image of the real-world.
• the VRD 100 optionally and preferably further comprises an eye tracking system 160 that may contain an IR illumination 261 and IR sensitive camera 262 which captures IR reflections from the outer surface of eye 155 and in this way detects the position of the pupil 151 as it moves.
• the IR light path 264 can be combined with the video signal light by a beam combiner 265.
• the instantaneous position of the pupil 151 provides a feedback signal to offset the scanner 130 deflection in the direction of the pupil 151, and is further processed in the video processor 170 to derive the video information to be projected onto retina 152.
• the operation of video processor 170 will now be explained also in conjunction with Figure 4.
• the video processor 170 optionally and preferably receives two main external input signals: an external video signal 171 from an external video source 175, namely an electronic device which generates video data such as a camera, a computer display, a mobile device or any other device that can generate video data (not shown).
• a second input to the video processor 170 is preferably a signal communicated by the eye tracking system 160 of the instantaneous position of the pupil 151.
• Other inputs to the video processor 170 not shown for clarity may optionally include feedback from the scanner and the instantaneous intensity communicated by intensity monitor unit 215.
• the video processor 170 preferably has two main output signals: it provides the signal of modulation of light beam intensity (modulation signal 173), and provides signal 172 to the scanner 130 synchronized with the modulation signal 173.
• the signal 172 to the scanner 130 is composed of total of four signals, two for each scanning axes.
• the signal to each scanning direction is composed of high frequency signal that triggers the raster scan, and a slow varying voltage which is an offset to the scanning mirror 130 to track the instantaneous position of the pupil 151.
• the video processor 170 In a light tracking system light, typically infrared, is reflected from the eye 155 and sensed by a video camera or some other optical sensor 262. The information is then analyzed and the position of the pupil 151 is determined. In response to the detected position of the pupil 151, the scanning mirror 130 is moved, both vertically and horizontally, such that the exit pupil of the VRD 100 is aligned with the pupil 151. Accordingly, the video processor 170 generates a high frequency signal 411 for triggering the raster scan in a horizontal direction, and slow varying signal 412 for horizontal offset which are fed through an amplifier stage 415 to horizontal scan 231 of the scanning mirror 130. Similarly, the video processor generates a high frequency signal 413 for triggering the raster scan in a vertical direction, and slow varying signal 414 for vertical offset which are fed through an amplifier stage 416 to vertical scan 232 of the scanning mirror 130.
• a high frequency signal 411 for triggering the raster scan in a horizontal direction
• the input of the pupil position also serves to process the projected video frame from the external video frame as follows.
• the entire retinal field of view (FOV) is decided by the largest angle of light rays that enter the pupil and reach the retina. Rays projected at higher angles will not reach the retina and hence will not be perceived.
• the virtual image projected by the VRD can be tuned to match exactly the frame size of the incoming external video, or alternatively the VRD can be tuned to project only part of the external video frames.
• the external video signal that enters the video processor fills the entire retinal FOV the virtual retinal image is said to be at unity magnification (xl).
• the VRD projects only part of the external video frame across the entire FOV of the retina the VRD operates at larger magnifications.
• magnification in this example refers to the portion of the external video frame that is projected onto the retina, such that such a frame can be said to be magnified when only part of the frame is projected over the retinal FOV. It should also be noted that the retinal FOV does not necessarily refer to the entire retina, as described in greater detail below, since part of the retina may be damaged or inoperative.
• the situation is similar to an ordinary process of visual perception when an extended (panoramic) object is viewed.
• the user moves her pupil across the object in such a manner that different parts of the object are perceived at a given position of the pupil.
• the instantaneous position of the pupil therefore defines a partial frame of the object that the user perceives.
• the VRD operates at magnification higher than unity the instantaneous position of the pupil defines the part of the external video frame that can be projected to the retina.
• the video processor stores the external video frames in a buffer, and according the instantaneous position of the pupil, processes a sub-frame to be projected by the VRD to accommodate available FOV.
• the central part of the frame corresponds to the center of the pupil where the pupil aperture defines the "window" frame.
• VRD at large magnification
• AMD age related macular degeneracy
• the parts of the image that fall on those parts are not perceived.
• the image is magnified details of the magnified image are projected over the undamaged parts in the retina which can compensate for the damaged ones.
• the shortcoming in magnification is that peripheral parts of the external video frame are not imaged.
• the VRD of the present invention is provided with tunable magnification controller 420 that can be adjusted by the user to provide input to video processor 170 regarding the extent of desired magnification.
• tunable magnification controller 420 can be adjusted by the user to provide input to video processor 170 regarding the extent of desired magnification.
• magnification controller 420 can be adjusted by the user to provide input to video processor 170 regarding the extent of desired magnification.
• magnification controller 420 can be adjusted by the user to provide input to video processor 170 regarding the extent of desired magnification.
• the frame processed from the external incoming video is further processed in the video processor with respect to retinal mapping of each individual as explained below.
• a retinal sensitivity map, or map of levels of retinal function at different locations (pixels), is preferably provided for the best operation of the below described method according to at least some embodiments of the present invention.
• Art known devices failed to include such a map, which means that they would not be effective for individuals having damaged retinas or retinas with reduced and/or uneven functioning.
• the map is preferably obtained by a mapping unit, which may optionally and preferably be implemented with the device of Figures 1 and 2 according to at least some embodiments of the present invention.
• mapping unit may optionally be implemented with the projection unit (shown as the apparatus of Figures 2 above in a non-limiting example), including the mapping software and analysis functions; alternatively these latter functions may optionally be performed separately and then the results fed to the processing unit of the projection apparatus. It may optionally be performed offline by a separate computer or alternatively through a combined device.
• Electroretinogram (ERG), pattern ERG (PERG), visual evoked potential (VEP) and other optometric and ophthalmic tests described in the art specify the visual field of an individual wherein these test provide information of an loss of vision or a reduction in sensitivity (threshold) of selected areas of the human retina.
• ERG Electroretinogram
• PERG pattern ERG
• VEP visual evoked potential
• other optometric and ophthalmic tests described in the art specify the visual field of an individual wherein these test provide information of an loss of vision or a reduction in sensitivity (threshold) of selected areas of the human retina.
• These tests are well known in the art and may optionally be implemented with various types of apparatus, also as known in the art.
• US Patent No. 5233373 describes an apparatus and a method of use thereof for implementing a PERG test on a retina of a patient, which is hereby incorporated by reference as if fully set forth herein, which includes a method of testing different locations of the retina in
• the retinal sensitivity map obtained in ophthalmic evaluation is decoded into a retinal sensitivity table (RST).
• RST retinal sensitivity table
• a typical adult human retina is about 72% of a sphere of 22 mm in diameter.
• the entire retinal field of view (FOV) is decided by the largest angle of light rays that enter the pupil and reach the retina.
• Fig 5 Shown in Fig 5 is an exemplary segmentation of retinal FOV where each angular segment is represented with a cell in the RST; the row number and column number corresponds to an angular sector of the entire retinal field of view area element, and the value in the cell represents the sensitivity of the area as follows: the highest level of sensitivity has the score 1, the second level of sensitivity has the score 2, and so forth, where complete loss of vision is represented with the highest number.
• the sensitivity scale is decided by the resolving capability of the applied diagnostic test; optionally also the number and location of the cells are determined according to the capabilities of the applied diagnostic test.
• the retinal map is transformed into the RST according to one or more conversion factors; for example, if the resolution of the diagnostic test is lower than expected for conversion to the RST, then the conversion process needs to accommodate this lowered resolution in order for all cells of the RST to be populated with data.
• the placement of the eye, and specifically of the pupil, during the diagnostic test from which the retinal map is obtained also needs to be considered with regard to the transformation to the RST, so that any movements of the pupil away from the placement during the test may be suitably compensated.
• the diagnostic test is applied more than once, so that the retinal map is obtained a plurality of times with the pupil in a plurality of different locations, in order to assist with the transformation process and also with calibration of the RST according to pupil movements.
• the light signal is scanned in horizontal and vertical directions and projected onto the user eye the light signal is scanned over the user retina, horizontally and vertically. Since the RST represents a map of the retina where each cell corresponds to a retinal sector or pixel, it is important that the RST is arranged according to the light scan trajectory.
• RST represents the absolute sensitivity of different sectors of the retina, which is non-uniform (non-homogenous) even for a normal retina, i.e., of a healthy individual with unimpaired vision. Certain portions of the retina are more sensitive to the light than other portions even in such an individual. For individuals with diseased or damaged retinas, the lack of uniformity may be much more pronounced, even to the point of having non-functioning parts of the retina.
• a relative RST (RRST) is constructed, where sensitivity values or the RST obtained for an individual are normalized relative to standard RST, obtained for individuals with normal vision. Such a standard RST may optionally be obtained for a particular population or alternatively is obtained for a number of individuals with healthy, normal vision, and is then used for calibration of RSTs obtained from individuals that are to receive the retinal projector apparatus as described herein.
• Fig 6 is an example of the first 10 rows by 10 columns of an illustrative RRST, for a retina which demonstrates reduced sensitivity within the center of this region, such that sectors corresponding to row 6, column 5, and row 6 column 6 (written as (6,5) and (6,6) for short) correspond to lower functionality and hence greater loss of vision, while the peripheral pixels of this region still have normal sensitivity as compared to healthy individuals.
• Figure 7 relates to an exemplary, illustrative method for receiving an incoming image and for projecting that image appropriately onto the retina of the subject through the retinal projection apparatus, for example as described herein.
• the method may be generalized for a plurality of images by repeating the process described herein for one image.
• the external digital image is converted to respective video signals for a plurality of pixels.
• these video signals for the plurality of pixels are adjusted at least in intensity according to the RRST, to form transformed video signals.
• the method is adjusted if there are some parts of the retina that are so damaged or reduced in function that they are practically inoperative, at least in terms of receiving projected light, as described in greater detail below with regard to Figures 10.
• all (or nearly all) of the pixels of the retina receive projected light.
• RRST RRST
• the upper plot in Figure 8 shows exemplary analog video signal derived from the camera of a raster scan; the middle plot shows the retinal position of the light signal during the scan.
• the video signal corresponds to the value Vi, at the same time the retinal position is Pi.
• the video signal has the value V 2 , while the retinal position at that instance is P 2 .
• the lower plot shows the video signal along the retinal position.
• the upper plot is a video signal resulting from the camera, here shown as a digital signal for clarity.
• the middle plot indicates the relative sensitivity scores of the retina according the position of the retina.
• the light source (a laser, LED or other light source) is modulated with signal which is the multiplication of video data (signals) from the camera with the value of the RRST given and corresponding to the video data.
• the function of the RRST can be also understood as follows.
• the user creates the image of the observed scene on the retina.
• To every point in the observed scene (object) corresponds a small area element on the retina.
• an image is acquired by a digital camera. Every pixel in the camera image corresponds to particular point in observed object. Therefore every pixel in the camera image is associated with an area element in the user retina.
• the camera image is intensified in those pixels associated with the retinal elements with a reduced sensitivity.
• the RRST contains the information where the image should be intensified, and the amplification factor. Mathematically, if the image of the camera is represented by a matrix M, the RRST is represented by the matrix R, then the intensity corrected matrix A results in
• the video processor before conveying the signal to intensity modulator preferably processes the projected video frames by multiplication with the RRST as explained.
• the RSST is determined according to the relative position of the user pupil.
• the VRD operates at magnification larger than xl only a portion is available to the user at a specific location of the pupil. The user moves his pupil to observe the entire panoramic view. Since the RSST is affixed to the pupil the mapping of RSST needs to be processed only on the projected image which is smaller than the buffered image external video image.
• stage 3 the position of the eye, and hence the relative location and angle of the pupil is determined by the eye tracking mechanism.
• stage 4 the video signals are adjusted according to the position of the eye: more specifically, the eye tracker provides the feedback to offset to the scanners mirrors so that the exit pupil of the Retinal Display (i.e. - the light projector) is approximately aligned with the entrance pupil of the eye.
• the instantaneous position of the pupil is also feedback to the video processor, such that only the video signal associated to the visible window as determined by the instantaneous position of the pupil is transferred to logic unit and multiplied according to the RSST and its associated sync signals are fed to the scanner mirrors.
• stage 5 light is projected onto the retina according to the transformed video signals.
• Stages 3-5 are optionally and preferably repeated for each row or column of pixels for the transformed video signals, as necessary in order to project the entire image onto the retina.
• an exemplary, illustrative method for projecting light onto a retina in which at least a portion of the retina is effectively non-functional.
• effectively non-functional it is meant that the specific portion of the retina cannot respond with sufficient functionality to be able to effectively receive projected light.
• Such effectively non-functional areas may include for example scotomas.
• FIG. 10 shows an image of letters ABC represented with the retinal projector as raster scan of row 3 and 4.
• the second diagram of Figure 10 (10b) represents how the projected image appears in field of vision with stocoma at the central portion.
• the third diagram of Figure 10 (10c) shows the letters ABC when the raster image was projected at rows 5 to 6. The degree of severity of stocoma and its position within the retina is deduced in the RRST. If the region where significant part of field of vision is degenerated to the extent of complete vision loss then according to the present invention the respective video information is translated to the raster scanned rows corresponding to healthy sectors.
• the circumvention of degenerated sector in the retina can be in non-continuous signals such that the projected image has certain "black" rows corresponding to rows where light is not projected.
• the projected image can appear as in the last graph (lOd) of Fig 10 in which the image was broken and re-projected at rows ahead.
• Figure 11 relates to an exemplary, illustrative method according to at least some embodiments of the present invention for receiving an incoming image and for projecting that image appropriately onto the retina of the subject through the retinal projection apparatus, for example as described herein, in which one or more portions of the retina are essentially non-functional.
• the method may be generalized for a plurality of images by repeating the process described herein for one image.
• stage 1 may optionally be performed as for Figure 7.
• stage 2 due to one or more essentially non-functioning parts of the retina, at least a portion of these video signals for the plurality of pixels are adjusted both in intensity and also in location according to the RRST, to form transformed video signals. At least some video signals may optionally be adjusted only in intensity, but for the sake of clarity, the description herein focusses on those signals which need to be adjusted both in intensity and location.
• stage 3 the position of the eye, and hence the relative location and angle of the pupil according to a previously determined position during the previously described diagnostic test, is determined.
• stage 4 the transformed video signals are adjusted according to the position of the eye and more specifically, any differences between the current location and angle of the pupil, and the previously determined location and angle of the pupil during diagnostic tests. For example, depending upon movement of the pupil a "high" signal at a pixel may in fact be transformed to a "low” signal to be projected at a pixel, effectively shifting and transforming the map according to the current pupil position, but may also determine whether a signal is sent at all, as described in greater detail below.
• adjusting the location of the video signals is performed as follows. Translating a portion of the image from degenerated to healthy portion of the retina is accomplished by introducing a controlled delay of the video signal relative to the raster scanner.
• Fig 12 shows the schematics of an exemplary, illustrative scan over the field of vision where the indicated path, in the shape of an "S-curve" with a plurality of inflection points, indicates the trajectory of the scan. The frame starts at time indicated by t 0 . With a bi-directional raster scan t indicates the start time of row number 1, t 2 indicates the start time of the second row, and similarly for subsequent lines, where r represent indicate the begin time of n th -row scan.
• the laser is intensity modulated with respect to the video information of the camera.
• the video information associated with row# 1 starts at ti and ends at ti+T.
• the time t 2 - (ti+T) to beginning of raw # 2 is "dead" time required for the scanner to re-position vertically to assume the raster scan of subsequent row.
• the plots on the left hand side in FIG 12 show exemplary video signal according which the laser is intensity modulated. For each raw there corresponds a video signal.
• a typical image may include hundreds of raster rows, in the example only the first four rows are shown.
• Plots on the right hand side of Fig 12 illustrate an example when the video signal corresponding originally to rows #3 and rows #4 is delayed and instead appears shifted to row #5, while during the scan of rows #3 and #4 the laser is turned off.
• the original video signal that was corresponding to subsequent rows has also has been shifted by one row.
• the video information of the last row will be omitted from the video frame.
• the last row, or even a number of rows if the shift is more than one row correspond to the peripheral of the field of vision and very often are less critical for vision related activities such as for example reading.
• stage 5 Evaluating the best strategy for re-projecting and more precisely which rows are skipped depends largely on the severity of retinal degeneration at its central regions information which is stored in the RSST.
• stage 5 light is projected onto the retina according to the transformed video signals.
• Stages 3-5 are optionally and preferably repeated for each row or column of pixels for the transformed video signals, as necessary in order to project the entire image onto the retina.

Landscapes

• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Optics & Photonics (AREA)
• Health & Medical Sciences (AREA)
• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Biomedical Technology (AREA)
• Ophthalmology & Optometry (AREA)
• Heart & Thoracic Surgery (AREA)
• Vascular Medicine (AREA)
• Life Sciences & Earth Sciences (AREA)
• Animal Behavior & Ethology (AREA)
• General Health & Medical Sciences (AREA)
• Public Health (AREA)
• Veterinary Medicine (AREA)
• Eye Examination Apparatus (AREA)
• Chemical And Physical Treatments For Wood And The Like (AREA)
• Illuminated Signs And Luminous Advertising (AREA)

Priority Applications (11)

Applications Claiming Priority (2)

Publications (2)

Family

ID=50149514

Family Applications (1)

Country Status (11)

Cited By (9)

Families Citing this family (34)

Citations (3)

Family Cites Families (18)

• 2013
  • 2013-08-19 BR BR112015003899A patent/BR112015003899A2/pt not_active IP Right Cessation
  • 2013-08-19 JP JP2015528004A patent/JP2015526187A/ja active Pending
  • 2013-08-19 KR KR1020157007083A patent/KR20150048789A/ko not_active Withdrawn
  • 2013-08-19 CN CN201380055324.9A patent/CN104812342B/zh not_active Expired - Fee Related
  • 2013-08-19 EP EP13830523.0A patent/EP2887913A4/en not_active Withdrawn
  • 2013-08-19 AU AU2013307201A patent/AU2013307201B2/en not_active Ceased
  • 2013-08-19 RU RU2015108858A patent/RU2015108858A/ru not_active Application Discontinuation
  • 2013-08-19 WO PCT/IL2013/050702 patent/WO2014030158A1/en active Application Filing
  • 2013-08-19 US US14/423,668 patent/US10058454B2/en not_active Expired - Fee Related
  • 2013-08-19 CA CA2882747A patent/CA2882747A1/en not_active Abandoned
• 2015
  • 2015-03-07 IN IN1864DEN2015 patent/IN2015DN01864A/en unknown

Patent Citations (3)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events