US20200133007A1 - Integrated MicroOptic Imager, Processor, and Display - Google Patents
Integrated MicroOptic Imager, Processor, and Display Download PDFInfo
- Publication number
- US20200133007A1 US20200133007A1 US16/726,516 US201916726516A US2020133007A1 US 20200133007 A1 US20200133007 A1 US 20200133007A1 US 201916726516 A US201916726516 A US 201916726516A US 2020133007 A1 US2020133007 A1 US 2020133007A1
- Authority
- US
- United States
- Prior art keywords
- display
- optical
- processor
- optical system
- detector
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 230000003287 optical effect Effects 0.000 claims abstract description 90
- 238000000034 method Methods 0.000 claims abstract description 10
- 230000008569 process Effects 0.000 claims abstract description 8
- 239000000463 material Substances 0.000 claims description 4
- 239000013078 crystal Substances 0.000 claims description 2
- 230000000007 visual effect Effects 0.000 claims 11
- 239000011521 glass Substances 0.000 description 10
- 239000012212 insulator Substances 0.000 description 9
- 239000000758 substrate Substances 0.000 description 7
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 3
- 238000002474 experimental method Methods 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 3
- 239000004065 semiconductor Substances 0.000 description 3
- 229910052710 silicon Inorganic materials 0.000 description 3
- 239000010703 silicon Substances 0.000 description 3
- KRHYYFGTRYWZRS-UHFFFAOYSA-N Fluorane Chemical compound F KRHYYFGTRYWZRS-UHFFFAOYSA-N 0.000 description 2
- 238000003491 array Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 150000001875 compounds Chemical class 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 229920000515 polycarbonate Polymers 0.000 description 2
- 239000004417 polycarbonate Substances 0.000 description 2
- 210000001525 retina Anatomy 0.000 description 2
- 229910000679 solder Inorganic materials 0.000 description 2
- 229920002972 Acrylic fiber Polymers 0.000 description 1
- MDNWOSOZYLHTCG-UHFFFAOYSA-N Dichlorophen Chemical compound OC1=CC=C(Cl)C=C1CC1=CC(Cl)=CC=C1O MDNWOSOZYLHTCG-UHFFFAOYSA-N 0.000 description 1
- 229910000530 Gallium indium arsenide Inorganic materials 0.000 description 1
- 101000916532 Rattus norvegicus Zinc finger and BTB domain-containing protein 38 Proteins 0.000 description 1
- VYPSYNLAJGMNEJ-UHFFFAOYSA-N Silicium dioxide Chemical compound O=[Si]=O VYPSYNLAJGMNEJ-UHFFFAOYSA-N 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 229910021419 crystalline silicon Inorganic materials 0.000 description 1
- 229910003460 diamond Inorganic materials 0.000 description 1
- 239000010432 diamond Substances 0.000 description 1
- 239000003989 dielectric material Substances 0.000 description 1
- 230000004907 flux Effects 0.000 description 1
- 239000005350 fused silica glass Substances 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- 238000001465 metallisation Methods 0.000 description 1
- 239000004033 plastic Substances 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B27/0172—Head mounted characterised by optical features
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/332—Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
- H04N13/344—Displays for viewing with the aid of special glasses or head-mounted displays [HMD] with head-mounted left-right displays
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B3/00—Simple or compound lenses
- G02B3/0006—Arrays
- G02B3/0037—Arrays characterized by the distribution or form of lenses
- G02B3/005—Arrays characterized by the distribution or form of lenses arranged along a single direction only, e.g. lenticular sheets
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B3/00—Simple or compound lenses
- G02B3/0006—Arrays
- G02B3/0037—Arrays characterized by the distribution or form of lenses
- G02B3/0056—Arrays characterized by the distribution or form of lenses arranged along two different directions in a plane, e.g. honeycomb arrangement of lenses
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B3/00—Simple or compound lenses
- G02B3/0006—Arrays
- G02B3/0037—Arrays characterized by the distribution or form of lenses
- G02B3/0062—Stacked lens arrays, i.e. refractive surfaces arranged in at least two planes, without structurally separate optical elements in-between
- G02B3/0068—Stacked lens arrays, i.e. refractive surfaces arranged in at least two planes, without structurally separate optical elements in-between arranged in a single integral body or plate, e.g. laminates or hybrid structures with other optical elements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/60—Editing figures and text; Combining figures or text
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/204—Image signal generators using stereoscopic image cameras
- H04N13/207—Image signal generators using stereoscopic image cameras using a single 2D image sensor
- H04N13/232—Image signal generators using stereoscopic image cameras using a single 2D image sensor using fly-eye lenses, e.g. arrangements of circular lenses
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/302—Image reproducers for viewing without the aid of special glasses, i.e. using autostereoscopic displays
- H04N13/307—Image reproducers for viewing without the aid of special glasses, i.e. using autostereoscopic displays using fly-eye lenses, e.g. arrangements of circular lenses
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
- H04N23/11—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths for generating image signals from visible and infrared light wavelengths
-
- H04N5/332—
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B27/0172—Head mounted characterised by optical features
- G02B2027/0174—Head mounted characterised by optical features holographic
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B2027/0178—Eyeglass type
Abstract
An optical system for displaying light from a scene includes an active optical component that includes a first plurality of light directing apertures, an optical detector, a processor, a display, and a second plurality of light directing apertures. The first plurality of light directing apertures is positioned to provide an optical input to the optical detector. The optical detector is positioned to receive the optical input and convert the optical input to an electrical signal corresponding to intensity and location data. The processor is connected to receive the data from the optical detector and process the data for the display. The second plurality of light directing apertures is positioned to provide an optical output from the display.
Description
- This application is a continuation of U.S. patent application Ser. No. 16/269,447 filed on Feb. 6, 2019 and titled “Integrated MicroOptic Imager, Processor, and Display”, which is a continuation of U.S. patent application Ser. No. 16/119,833 filed on Aug. 31, 2018, and titled “Integrated Microoptic Imager, Processor, and Display”, which is a continuation of U.S. patent application Ser. No. 15/185,437 filed on Jun. 17, 2016, and titled “Integrated Microoptic Imager, Processor, and Display”, which is a continuation of PCT Application No. PCT/US2014/070991 filed on Dec. 17, 2014 and titled “Integrated Microoptic Imager, Processor, and Display”, which claims priority to U.S. Provisional Application No. 61/963,928, filed Dec. 17, 2013, “Integrated MicroOptic Imager, Processor, and Display,” each of which are incorporated herein by reference in their entireties.
- This patent application generally relates to a structure for imaging a scene. More particularly, it relates to a stack structure. Even more particularly it relates to a compact stack structure.
- Imaging devices have required significant space either for optical input or for optical display or both. Applicants recognized that better schemes than those available are needed and such solutions are provided by the following description.
- One aspect of the present patent application is an optical system for displaying light from a scene. The optical system includes an active optical component that includes a first plurality of light directing apertures, an optical detector, a processor, a display, and a second plurality of light directing apertures. The first plurality of light directing apertures is positioned to provide an optical input to the optical detector. The optical detector is positioned to receive the optical input and convert the optical input to an electrical signal corresponding to intensity and location data. The processor is connected to receive the data from the optical detector and process the data for the display. The second plurality of light directing apertures is positioned to provide an optical output from the display.
- The foregoing will be apparent from the following detailed description, as illustrated in the accompanying drawings, in which:
-
FIG. 1 is an exploded three dimensional view of one embodiment of the active optical component of the present patent application; -
FIG. 2 is a block diagram of the embodiment of the active optical component ofFIG. 1 ; -
FIG. 3a is a cross sectional view of another embodiment of the active optical component of the present patent application in which detector, processor, and display connect on a common surface; -
FIG. 3b is a top view of the detector, processor, and display ofFIG. 3 a; -
FIG. 4 is a cross sectional view of the embodiment of the active optical component ofFIG. 1 in which detector, processor, and display are all in separate layers; -
FIG. 5a is a cross sectional view showing the input light directing apertures as a micro-lens array; -
FIG. 5b is a cross sectional view showing the input light directing apertures as an array of pinholes; -
FIG. 5c is a cross sectional view showing the input light directing apertures as an array of diffraction gratings; -
FIG. 6 is a cross sectional view showing the output light directing apertures directing light to form a single image on the retina of a nearby eye; -
FIGS. 7a and 7b are three dimensional views showing a curved active optical component of the present patent application included in a pair of glasses; -
FIG. 7c is a three dimensional view showing a planar active optical component of the present patent application included in a pair of glasses; -
FIG. 7d is a three dimensional views showing a curved active optical component of the present patent application included in a pair of glasses; and -
FIGS. 8a, 8b, 8c, 8d, 8e, 8f, 8g, and 8h are process steps to fabricate a curved active optical component of the present patent application. - In one embodiment, the system uses light directing apertures, such as
micro-lens arrays stacked component 31 includingoptical detector 32,processor 34, anddisplay 36 located between the twolight directing apertures optical component 40, as shown inFIG. 1 and in the block diagram inFIG. 2 . - Light directing apertures arc fabricated of a material such as molded glass, fused silica, acrylic plastic, polycarbonate, Uvex, CR39, and Trivex.
-
Optical detector 32 includes an array of receptors that receive photons from the scene outside throughlight directing apertures 30 a and converts the photons to electrical signals corresponding to intensity and location in the scene outside.Optical detector 32 can include a charge coupled device, a complementary metal-oxide semiconductor sensor chip, and such low light detectors as a microchannel amplifier imaging chip combination and an electron bombarded integrated circuit (EBIC), and for short wave infrared at low light level, an InGaAs focal plane array. - In one embodiment
optical detector 32 has serial electrical connections for storing image data inmemory 42 ofprocessor 34. In another embodiment,optical detector 32 has multipleparallel connections 58 for storing this image data inmemory 42 ofprocessor 34. -
Processor 34 also includesinput assembler 44,arithmetic logic units 46 withdata caches 48,execution manager 50,central processing unit 52, andlocal cache 54 that digitally process the image data fromdetector 32 and formats the data fordisplay 36, providing an output through either a wire connector ormultiple connectors 56 to display 36. The images provided ondisplay 36 are seen by the eye of the viewer through optical outputlight directing apertures 30 b. - In one embodiment,
optical detector 32,processor 34, and display 36 share a common interconnect surface, which is backsurface 60 ofdisplay 36, as shown inFIGS. 3a, 3b .Detector 32 andprocessor 34 are interconnected with each other and withdisplay 36 throughconnectors back surface 60. - In another alternative,
detector 32,processor 34 anddisplay 36 are on separate layers, as shown inFIG. 4 . In thisembodiment detector 32 andprocessor 34 have throughchip connections layer interconnectors processor 34 has a first side and a second side, and the first side is electrically connected tooptical detector 32 and the second side is electrically connected to display 36. Alternatively, standard cable connectors (not shown) are used for the connections fromdetector 32 toprocessor 34 and fromprocessor 34 to display 36. - In one experiment an assembly of input side optics was built and tested with
light directing apertures 30 a that were micro-lenses that each had a focal length f=9.3 mm and with 3.2 mm apertures in a 3×3 array. The field of view was 20°, the display resolution was 2048 pixels×2048 pixels, and each pixel was 5.5×5.5 microns on a side with an optical resolution of 55 line pairs per degree (1p/0) Each lens of the micro-lens array was a compound lens. Total thickness of the input optics micro lens array was 8.5 mm and the spacing todetector 32 was 1 mm. The lens array was custom diamond turned in Zeonex plastic. - In one experiment an assembly of output side optics was purchased and tested. The resolution was 2 line pairs/degree. The field of view was 17 degrees. The focal length was 3.3 mm. The aperture was Imm diameter. Each lens was 3 mm thick molded polycarbonate. The micro lenses were purchased from Fresnel Technologies, Fort Worth, Tex. and were part number 630. The display was a 15×11 mm Sony OLED micro-display, part number ECX322A.
- As in the experiment,
light directing apertures 30 a can have different dimensions than light directingapertures 30 b. - While
light directing apertures FIGS. 1, 2, and 5 a, light directing apertures can be pinholes, as shown inFIG. 5b , and diffraction gratings, as shown inFIG. 5c . Zone plates, holograms, gradient index material, and photonics crystals can also be used. Each lens of a micro-lens array can be compound lens, as shown inFIG. 5 a. - In one embodiment, adjacent ones of the light directing apertures are configured to provide redundant scene elements on
detector 32.Processor 34 includes a program to superimpose data from redundant scene elements, such as data derived from adjacent ones of the plurality of light directing optical apertures, to create a single image with such changes as higher resolution, better signal to noise ratio, and higher contrast, as described in a paper, “Thin observation module by bound optics (TOMBO): concept and experimental verification,” by Jun Tanida et al, Applied Optics, Vol. 40, No. 11, 10 Apr. 2001 (“the Tanida paper”), in a paper, “PiCam: An Ultra-Thin High Performance Monolithic Camera Array,” by Venkatarama et al, ACM Transactions on Graphics, Proceedings of ACM SIGGRATH Asia, 32 (5) 2013, both of which are incorporated herein by reference and as described in U.S. Pat. Nos. 5,754,348 and 8,013,914, both of which are incorporated herein by reference.Processor 34 can also include a program to provide a higher magnification. - Detail of
display 36 andoutput portion 30 b located close to a user's eye is shown inFIG. 6 .Display 36 provides a two dimensional array of similar images of the scene presented to input optics. The user seesdisplay 36 throughoutput apertures 30 b. Wherever the user's eye is located theoutput apertures 30 b each direct a portion of their respective subimage so that a single image is formed on the retina from a combination of contributions from each lens, as described in the US20140168783 patent application, incorporated herein by reference and in a paper, “Near-Eye Light Field Displays,” by Douglas Lanman and David Luebke, ACM Transactions on Graphics, Proceedings of ACM SIGGRATH Asia, 32 (6) November 2013, article number 220. This scheme allowsdisplay 36 andoutput portion 30 b to be located close to the user's eye, such as on a pair of glasses. - In one embodiment, an external electronic device is connected to
processor 34 throughconnector 56 for providing information ondisplay 36. The external electronic device may be a communications system, a wife, a GPS, a remote camera, another wearable optical system, a microphone, a digital compass, an accelerometer, a vehicle instrument, and an external computer. In one embodiment, the external information is provided on the display to overlay information from the scene, as described in U.S. Pat. No. 7,250,983, incorporated herein by reference. The system can thus augment what the viewer is seeing with overlaid information, for example, information about the subject or object being viewed. The overlaid information can be data that was previously stored. - In one embodiment, the system augments a user's vision by displaying images captured in wavelength bands including visible (0.4 to 0.7 microns), near infrared (0.7 to 1.0 microns), and short wave infrared (1.0 to 2.5 microns). With appropriate detectors, the system can also display images showing combinations, such as visible and near infrared, visible and short wave infrared, near infrared and short wave infrared, and visible, near infrared and short wave infrared. With appropriate detectors the system can also display images from objects providing light in other bands, including ultraviolet (0.2 to 0.4 micron), mid-wave infrared (2.5 to 6 micron), and long-wave infrared (6 to 14 micron). The system can thus augment user's vision by displaying images of the subject or object in a non-visible wavelength band. Well known detectors in the various wavelength bands can be used, as described in “Infrared Detectors: an overview,” by Antoni Rogalski, in Infrared Physics & Technology 43 (2002) 187-210.
- The present applicants found that with the multi-aperture array there is no change in the solid angle subtended as compared with using a single input lens. Nor is there a change in the flux of light collected by each pixel of the detector as compared with using a single input lens. They found that noise reduction was accomplished and resolution improved by using weighted averages of surrounding pixels as described in the Tanida paper.
- The thickness of active
optical component 40 is sufficiently reduced in this embodiment compared to previously existing devices, while the outputlight directing aperture 30 b allows the system to be located near the user's eye, so a pair of activeoptical components 40 can be mounted to replace the ordinary lenses in a pair ofglasses 74, as shown inFIGS. 7a-7d . In one alternative, glasses with the activeoptical components 40 can be worn over an ordinary pair of glasses. In another alternative, the glasses may have only one of the ordinary lenses so replaced, allowing normal vision with one eye.Light directing apertures component 31 may be planar as shown inFIG. 7c or they may be curved as shown inFIGS. 7a-7b and 7 d. - Curved semiconductor components are described in U.S. Pat. Nos. 6,027,958, 6,953,735, and 8,764,255 and US patent application 21040004644, all of which are incorporated herein by reference. Curved stacked components may include thinned crystalline silicon for detector and processor. Thinned silicon will roll up. It is sufficiently flexible that it can have different curvature in each of two dimensions. Other semiconductors are similarly flexible when thinned. Thinning is also advantageous for through silicon contacts.
Display 36 is fabricated on a flexible substrate. Arrays oflight directing apertures - A process to fabricate curved
stacked component 31′ is shown inFIGS. 8a- 8f Detector 32 is grown epitaxially on the sacrificial oxide insulator surface of silicon-oninsulator substrate 80, as shown inFIG. 8a , withactive side 82 ofdetector 32 facingoxide insulator 84 and itsback surface 86 with electrical contacts exposed. As grown,detector 32 is in the range from 2 to 20 microns thick. -
Processor 34 is grown epitaxially on the sacrificial oxide insulator surface of silicon-on-insulator substrate 90, as shown inFIG. 8b , withactive surface 92 and its electrical connections todetector 32 exposed and with itselectrical contacts 94 for contact to the display facingoxide insulator 96. As grown,processor 34 is in the range from 2 to 20 microns thick. -
Display 36 is grown epitaxially on the sacrificial oxide insulator surface of silicon-on-insulator substrate 100, as shown inFIG. 8c , with its electrical connections to theprocessor 102 exposed and with its display elements facingoxide insulator 106. As grown,display 36 is in the range from 10 to 30 microns thick. In one embodiment the display base material layer is silicon. Display elements may include metalization, deposited light emitting diodes, mirrors, and dielectric materials. - In the next step electrical contacts between
detector wafer 32 andprocessor wafer 34 are aligned, as shown inFIG. 8d , anddetector wafer 32 is bonded toprocessor wafer 34 using a standard contact to contact bonding method such as solder bonding or compression bonding. - In the next step detector-
processor stack 110 is released fromprocessor substrate wafer 90 using a process such as hydrofluoric acid or zenon difluoride, as shown inFIG. 8 e. - In the next step the now exposed electrical connections of
processor 34 are aligned and bonded to display 36 electrical contacts using a process such as solder bonding or compression bonding, as shown inFIG. 8 f. - In the next step detector-processor-
display stack 120 is released from bothdisplay substrate wafer 100 and fromdetector substrate wafer 80, as shown inFIG. 8g . The detector-processor-display stack is now flexible and has its electrical contacts aligned for electrical communication between layers. In addition, an electrical lead brought out to the edge ofstack 120 or on the outside surface of eitherdetector 32 ordisplay 36 is used for connection to bring in power from a battery and signal from an external electronic device. Through connectors allow power brought in to one to be distributed to all three layers. The battery can be mounted elsewhere, such as in the glasses frame. - In the next step the detector-processor-display stack is aligned with and connected with rigid input
curved lens array 130 a and outputcurved lens array 130 b fabricated as molded optics, conforming to their curvature, as shown inFIG. 8h , to providecurved stack 31′. - While several embodiments, together with modifications thereof, have been described in detail herein and illustrated in the accompanying drawings, it will be evident that various further modifications are possible without departing from the scope of the invention as defined in the appended claims. Nothing in the above specification is intended to limit the invention more narrowly than the appended claims. The examples given are intended only to be illustrative rather than exclusive.
Claims (20)
1. An optical system for displaying light from a scene, comprising:
a see-through optical system including:
an optical detector,
a processor; and
a display;
wherein the optical detector, the processor, and the display are arranged such that the processor is in-between the optical detector and the display,
wherein each of the optical detector, the processor, and the display are electronically coupled to each other, and
wherein a total stacked height of the see-through optical system is less than 70 microns.
2. An optical system according to claim 1 , wherein the optical detector is configured to convert the optical input into an electrical signal corresponding to intensity and location data of the light.
3. An optical system according to claim 2 , wherein the processor receives and processes the electrical signal from the optical detector so as to produce an image for the display.
4. An optical system according to claim 3 , further including a first microlens array, wherein the first microlens array is positioned to provide an optical output derived from the image on the display.
5. An optical system according to claim 4 , wherein the first microlens array includes one or more of a lens, a pinhole, a diffraction grating, a zone plate, a hologram, a gradient index material, and/or a photonics crystals.
6. An optical system according to claim 4 , wherein adjacent members of the first microlens array provide redundant scene elements, and wherein the processor uses the redundant scene elements to adjust at least one from the group consisting of: a signal to noise ratio, a contrast, and a resolution.
7. An optical system according to claim 1 , further comprising an eyeglasses frame having left and right lenses, wherein the see-through optical system is sized and configured to replace a portion of one of the left or right lenses.
8. An optical system according to claim 1 , wherein see-through optical system is curved.
9. An optical system according to claim 1 , further comprising an external electronic device, wherein the external electronic device provides information to the processor, and wherein the processor overlays the information on the display as an image derived from the information.
10. An optical system according to claim 1 , wherein the optical detector, the processor, and the display interconnect on a common surface.
11. An optical system according to claim 1 , wherein the see-through optical system is sized and configured to be a visual prosthetic device suitable for wearing on a user's eye.
12. A visual assistance device suitable for wearing on a user's eye, the visual assistance comprising:
a processor, a display, and an optical detector, wherein the processor, optical detector, and the display are electronically coupled to each other on a common side, and
wherein a total stacked height of the processor, the optical detector, and the display, is less than 70 microns.
13. A visual assistance device according to claim 12 , further including at least one active optical component and wherein the processor, optical detector, display, and at least one active optical component allows the user at least a partial view through a stacked combination of the processor, optical detector, display, and at least one active optical component.
14. A visual assistance device according to claim 13 , wherein the at least one active optical component is positioned to provide an optical output derived from the image on the display.
15. A visual assistance device according to claim 13 , wherein the at least one active optical component is a two-dimensional array of light directing apertures.
16. A visual assistance device according to claim 12 , wherein the optical detector is positioned to receive an optical input from the first light directing device.
17. A visual assistance device according to claim 16 , wherein the optical detector is configured to convert the optical input into an electrical signal corresponding to intensity and location data of the light.
18. A visual assistance device according to claim 17 , wherein the processor receives and processes the electrical signal from the optical detector so as to transmit an image to the display.
19. A visual assistance device according to claim 12 , wherein the processor is located between the optical detector and the display.
20. A visual assistance device according to claim 19 , wherein the optical detector, the processor, and the display interconnect on a common side surface.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/726,516 US20200133007A1 (en) | 2013-12-17 | 2019-12-24 | Integrated MicroOptic Imager, Processor, and Display |
US17/484,128 US11460706B2 (en) | 2013-12-17 | 2021-09-24 | Integrated microoptic imager, processor, and display |
Applications Claiming Priority (6)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201361963928P | 2013-12-17 | 2013-12-17 | |
PCT/US2014/070991 WO2015095417A1 (en) | 2013-12-17 | 2014-12-17 | Integrated microptic imager, processor, and display |
US15/185,437 US10067348B2 (en) | 2013-12-17 | 2016-06-17 | Integrated microoptic imager, processor, and display |
US16/119,833 US10215988B2 (en) | 2013-12-17 | 2018-08-31 | Integrated MicroOptic imager, processor, and display |
US16/269,447 US10520736B2 (en) | 2013-12-17 | 2019-02-06 | Integrated Microoptic imager, processor, and display |
US16/726,516 US20200133007A1 (en) | 2013-12-17 | 2019-12-24 | Integrated MicroOptic Imager, Processor, and Display |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/269,447 Continuation US10520736B2 (en) | 2013-12-17 | 2019-02-06 | Integrated Microoptic imager, processor, and display |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/484,128 Continuation US11460706B2 (en) | 2013-12-17 | 2021-09-24 | Integrated microoptic imager, processor, and display |
Publications (1)
Publication Number | Publication Date |
---|---|
US20200133007A1 true US20200133007A1 (en) | 2020-04-30 |
Family
ID=52355214
Family Applications (5)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/185,437 Active 2035-02-07 US10067348B2 (en) | 2013-12-17 | 2016-06-17 | Integrated microoptic imager, processor, and display |
US16/119,833 Active US10215988B2 (en) | 2013-12-17 | 2018-08-31 | Integrated MicroOptic imager, processor, and display |
US16/269,447 Active US10520736B2 (en) | 2013-12-17 | 2019-02-06 | Integrated Microoptic imager, processor, and display |
US16/726,516 Abandoned US20200133007A1 (en) | 2013-12-17 | 2019-12-24 | Integrated MicroOptic Imager, Processor, and Display |
US17/484,128 Active US11460706B2 (en) | 2013-12-17 | 2021-09-24 | Integrated microoptic imager, processor, and display |
Family Applications Before (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/185,437 Active 2035-02-07 US10067348B2 (en) | 2013-12-17 | 2016-06-17 | Integrated microoptic imager, processor, and display |
US16/119,833 Active US10215988B2 (en) | 2013-12-17 | 2018-08-31 | Integrated MicroOptic imager, processor, and display |
US16/269,447 Active US10520736B2 (en) | 2013-12-17 | 2019-02-06 | Integrated Microoptic imager, processor, and display |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/484,128 Active US11460706B2 (en) | 2013-12-17 | 2021-09-24 | Integrated microoptic imager, processor, and display |
Country Status (6)
Country | Link |
---|---|
US (5) | US10067348B2 (en) |
EP (1) | EP3085082B1 (en) |
JP (1) | JP6654774B2 (en) |
KR (1) | KR102283111B1 (en) |
CA (1) | CA2934528C (en) |
WO (1) | WO2015095417A1 (en) |
Families Citing this family (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107003523B (en) * | 2014-10-24 | 2019-05-14 | 埃马金公司 | Immersion based on micro-display wears view device |
CN106291953B (en) | 2015-06-05 | 2019-01-08 | 北京智谷睿拓技术服务有限公司 | display control method and device |
CN106297611B (en) | 2015-06-05 | 2021-08-10 | 北京智谷睿拓技术服务有限公司 | Display control method and device |
CN106297610B (en) * | 2015-06-05 | 2020-03-17 | 北京智谷睿拓技术服务有限公司 | Display control method and device |
US10110883B2 (en) | 2015-09-30 | 2018-10-23 | Daqri, Llc | Bidirectional holographic lens |
US20170115489A1 (en) * | 2015-10-26 | 2017-04-27 | Xinda Hu | Head mounted display device with multiple segment display and optics |
DE102016221123A1 (en) * | 2016-10-26 | 2018-04-26 | Bayerische Motoren Werke Aktiengesellschaft | Method and device for operating a display system with data glasses |
US10616470B2 (en) * | 2017-08-31 | 2020-04-07 | Snap Inc. | Wearable electronic device with hardware secured camera |
US10930709B2 (en) | 2017-10-03 | 2021-02-23 | Lockheed Martin Corporation | Stacked transparent pixel structures for image sensors |
US10690910B2 (en) | 2018-02-07 | 2020-06-23 | Lockheed Martin Corporation | Plenoptic cellular vision correction |
US10838250B2 (en) | 2018-02-07 | 2020-11-17 | Lockheed Martin Corporation | Display assemblies with electronically emulated transparency |
US10951883B2 (en) | 2018-02-07 | 2021-03-16 | Lockheed Martin Corporation | Distributed multi-screen array for high density display |
US10979699B2 (en) | 2018-02-07 | 2021-04-13 | Lockheed Martin Corporation | Plenoptic cellular imaging system |
US11616941B2 (en) | 2018-02-07 | 2023-03-28 | Lockheed Martin Corporation | Direct camera-to-display system |
WO2020101917A1 (en) * | 2018-11-13 | 2020-05-22 | Lockheed Martin Corporation | Plenoptic cellular vision correction |
KR20190101324A (en) * | 2019-08-12 | 2019-08-30 | 엘지전자 주식회사 | Electronic device |
US20220398759A1 (en) | 2019-11-27 | 2022-12-15 | Trinamix Gmbh | Depth measurement through display |
EP4244659A1 (en) | 2020-11-13 | 2023-09-20 | trinamiX GmbH | Depth measurement through display |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7995078B2 (en) * | 2004-09-29 | 2011-08-09 | Noregin Assets, N.V., L.L.C. | Compound lenses for multi-source data presentation |
US20130021226A1 (en) * | 2011-07-21 | 2013-01-24 | Jonathan Arnold Bell | Wearable display devices |
US20150049390A1 (en) * | 2013-08-14 | 2015-02-19 | Nvidia Corporation | Hybrid optics for near-eye displays |
US10325951B2 (en) * | 2008-10-07 | 2019-06-18 | Mc10, Inc. | Methods and applications of non-planar imaging arrays |
Family Cites Families (31)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE524189C (en) | 1929-06-28 | 1938-03-15 | I G Farbenindustrie Akt Ges | Process for the production of polymerization products from vinyl ethers |
JPH07168120A (en) * | 1993-12-13 | 1995-07-04 | Sony Corp | Heat mount type picture display device |
US5754348A (en) | 1996-05-14 | 1998-05-19 | Planetweb, Inc. | Method for context-preserving magnification of digital image regions |
US6027958A (en) | 1996-07-11 | 2000-02-22 | Kopin Corporation | Transferred flexible integrated circuit |
US7242388B2 (en) * | 2001-01-08 | 2007-07-10 | Vkb Inc. | Data input device |
JP4243034B2 (en) * | 2001-03-29 | 2009-03-25 | 日本放送協会 | Encoder |
US7084886B2 (en) * | 2002-07-16 | 2006-08-01 | Idelix Software Inc. | Using detail-in-context lenses for accurate digital image cropping and measurement |
JP4083521B2 (en) * | 2001-10-29 | 2008-04-30 | オリンパス株式会社 | Imaging device |
US6953735B2 (en) | 2001-12-28 | 2005-10-11 | Semiconductor Energy Laboratory Co., Ltd. | Method for fabricating a semiconductor device by transferring a layer to a support with curvature |
US20060192869A1 (en) * | 2005-02-28 | 2006-08-31 | Kazutora Yoshino | Multi-dimensional input, transfer, optional memory, and output method |
US8217381B2 (en) * | 2004-06-04 | 2012-07-10 | The Board Of Trustees Of The University Of Illinois | Controlled buckling structures in semiconductor interconnects and nanomembranes for stretchable electronics |
US7250983B2 (en) | 2004-08-04 | 2007-07-31 | Trident Technologies, Inc. | System and method for overlaying images from multiple video sources on a display device |
US7969462B2 (en) * | 2005-03-30 | 2011-06-28 | L-3 Communications Corporation | Digitally enhanced night vision device |
JP2007086500A (en) * | 2005-09-22 | 2007-04-05 | Sony Corp | Display device |
KR100730406B1 (en) * | 2005-11-16 | 2007-06-19 | 광운대학교 산학협력단 | Three-dimensional display apparatus using intermediate elemental images |
CN2911724Y (en) * | 2006-04-24 | 2007-06-13 | 胡超 | Glasses type integrated 3-D video multimedia player |
JP4930315B2 (en) * | 2007-01-19 | 2012-05-16 | 株式会社デンソー | In-vehicle information display device and light irradiation device used therefor |
JP4686496B2 (en) | 2007-03-30 | 2011-05-25 | 株式会社東芝 | Imaging device |
JP3137491U (en) * | 2007-09-14 | 2007-11-22 | 祐司 野口 | Portable tv camera |
JP4538766B2 (en) * | 2008-08-21 | 2010-09-08 | ソニー株式会社 | Imaging device, display device, and image processing device |
JP2010058631A (en) * | 2008-09-03 | 2010-03-18 | Mazda Motor Corp | Vehicular information display device |
US10244181B2 (en) * | 2009-02-17 | 2019-03-26 | Trilumina Corp. | Compact multi-zone infrared laser illuminator |
US20120212406A1 (en) * | 2010-02-28 | 2012-08-23 | Osterhout Group, Inc. | Ar glasses with event and sensor triggered ar eyepiece command and control facility of the ar eyepiece |
US8582209B1 (en) * | 2010-11-03 | 2013-11-12 | Google Inc. | Curved near-to-eye display |
CN103018904A (en) * | 2011-09-23 | 2013-04-03 | 奇想创造事业股份有限公司 | Head-mounted image acquisition analysis display system and method thereof |
GB201118248D0 (en) * | 2011-10-23 | 2011-12-07 | Thomson James W | Vehicle side marker and downward light |
FR2989518A1 (en) | 2012-04-13 | 2013-10-18 | St Microelectronics Crolles 2 | Method for manufacturing integrated image sensor, involves forming pixels at end of columns, and deforming structure such that another end of each of columns is brought closer or separated to form surface in shape of polyhedral cap |
US9377900B1 (en) * | 2012-06-18 | 2016-06-28 | Amazon Technologies, Inc. | Optical touch sensor |
US9841537B2 (en) | 2012-07-02 | 2017-12-12 | Nvidia Corporation | Near-eye microlens array displays |
US8764255B2 (en) | 2012-10-10 | 2014-07-01 | Universal Display Corporation | Semi-rigid electronic device with a flexible display |
US9399596B1 (en) * | 2013-12-13 | 2016-07-26 | Google Inc. | Methods and systems for bonding multiple wafers |
-
2014
- 2014-12-17 JP JP2016560633A patent/JP6654774B2/en active Active
- 2014-12-17 WO PCT/US2014/070991 patent/WO2015095417A1/en active Application Filing
- 2014-12-17 CA CA2934528A patent/CA2934528C/en active Active
- 2014-12-17 KR KR1020167019356A patent/KR102283111B1/en active IP Right Grant
- 2014-12-17 EP EP14827938.3A patent/EP3085082B1/en active Active
-
2016
- 2016-06-17 US US15/185,437 patent/US10067348B2/en active Active
-
2018
- 2018-08-31 US US16/119,833 patent/US10215988B2/en active Active
-
2019
- 2019-02-06 US US16/269,447 patent/US10520736B2/en active Active
- 2019-12-24 US US16/726,516 patent/US20200133007A1/en not_active Abandoned
-
2021
- 2021-09-24 US US17/484,128 patent/US11460706B2/en active Active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7995078B2 (en) * | 2004-09-29 | 2011-08-09 | Noregin Assets, N.V., L.L.C. | Compound lenses for multi-source data presentation |
US10325951B2 (en) * | 2008-10-07 | 2019-06-18 | Mc10, Inc. | Methods and applications of non-planar imaging arrays |
US20130021226A1 (en) * | 2011-07-21 | 2013-01-24 | Jonathan Arnold Bell | Wearable display devices |
US20150049390A1 (en) * | 2013-08-14 | 2015-02-19 | Nvidia Corporation | Hybrid optics for near-eye displays |
Also Published As
Publication number | Publication date |
---|---|
CA2934528A1 (en) | 2015-06-25 |
US20190033601A1 (en) | 2019-01-31 |
KR102283111B1 (en) | 2021-07-29 |
KR20160101087A (en) | 2016-08-24 |
US20190243139A1 (en) | 2019-08-08 |
WO2015095417A1 (en) | 2015-06-25 |
EP3085082B1 (en) | 2020-04-29 |
US20220011582A1 (en) | 2022-01-13 |
WO2015095417A8 (en) | 2016-10-13 |
JP6654774B2 (en) | 2020-02-26 |
EP3085082A1 (en) | 2016-10-26 |
US11460706B2 (en) | 2022-10-04 |
US20160370588A1 (en) | 2016-12-22 |
CA2934528C (en) | 2022-06-28 |
US10215988B2 (en) | 2019-02-26 |
US10067348B2 (en) | 2018-09-04 |
JP2017510212A (en) | 2017-04-06 |
US10520736B2 (en) | 2019-12-31 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11460706B2 (en) | Integrated microoptic imager, processor, and display | |
US9977248B1 (en) | Augmented reality display system | |
US20190057957A1 (en) | Augmented reality display system | |
TWI781195B (en) | Laminated lens structure, solid-state imaging element, and electronic apparatus | |
US11437423B2 (en) | Image sensor, manufacturing method, and electronic device | |
US9300885B2 (en) | Imaging device, portable information terminal, and imaging system | |
CN108428716A (en) | Augmented reality display system | |
US20140118516A1 (en) | Solid state imaging module, solid state imaging device, and information processing device | |
US20120019624A1 (en) | Image sensor for generating stereoscopic images | |
EP2782136A2 (en) | Solid state imaging device and portable information terminal | |
US9462166B2 (en) | Imaging device, portable information terminal, and display device | |
JP2019066610A (en) | Imaging device, manufacturing method of imaging device, and electronic apparatus | |
US11496680B2 (en) | Imaging unit | |
US10725274B1 (en) | Immersed dichroic optical relay | |
JP5248971B2 (en) | Imaging unit manufacturing method and imaging unit | |
JP2005031460A (en) | Compound eye optical system | |
US20240038798A1 (en) | Color filter, image sensor, and electronic apparatus having the image sensor | |
JP5452800B2 (en) | Stereoscopic imaging device | |
WO2014119923A1 (en) | Mems stereoscopic image capture apparatus |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
AS | Assignment |
Owner name: MARSUPIAL HOLDINGS INC., VERMONT Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:PARKER, WILLIAM P.;STRAUSS, MICHAEL A.;ROUSSEAU, IAN M.;AND OTHERS;SIGNING DATES FROM 20160616 TO 20161010;REEL/FRAME:053637/0169 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |