Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
  • H04N23/13—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths with multiple sensors
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
  • H04N23/13—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths with multiple sensors
  • H04N23/16—Optical arrangements associated therewith, e.g. for beam-splitting or for colour correction
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B1/00—Optical elements characterised by the material of which they are made; Optical coatings for optical elements
  • G02B1/002—Optical elements characterised by the material of which they are made; Optical coatings for optical elements made of materials engineered to provide properties not available in nature, e.g. metamaterials
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B1/00—Optical elements characterised by the material of which they are made; Optical coatings for optical elements
  • G02B1/10—Optical coatings produced by application to, or surface treatment of, optical elements
  • G02B1/11—Anti-reflection coatings
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B13/00—Optical objectives specially designed for the purposes specified below
  • G02B13/001—Miniaturised objectives for electronic devices, e.g. portable telephones, webcams, PDAs, small digital cameras
  • G02B13/0015—Miniaturised objectives for electronic devices, e.g. portable telephones, webcams, PDAs, small digital cameras characterised by the lens design
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B5/00—Optical elements other than lenses
  • G02B5/20—Filters
  • G02B5/208—Filters for use with infrared or ultraviolet radiation, e.g. for separating visible light from infrared and/or ultraviolet radiation
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B7/00—Mountings, adjusting means, or light-tight connections, for optical elements
  • G02B7/02—Mountings, adjusting means, or light-tight connections, for optical elements for lenses
• • G—PHYSICS
  • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
  • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
  • G03B11/00—Filters or other obturators specially adapted for photographic purposes
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T5/00—Image enhancement or restoration
  • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T5/00—Image enhancement or restoration
  • G06T5/80—Geometric correction
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/50—Constructional details
  • H04N23/55—Optical parts specially adapted for electronic image sensors; Mounting thereof
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/80—Camera processing pipelines; Components thereof
  • H04N23/81—Camera processing pipelines; Components thereof for suppressing or minimising disturbance in the image signal generation
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/90—Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10024—Color image
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/20—Special algorithmic details
  • G06T2207/20212—Image combination
  • G06T2207/20221—Image fusion; Image merging

Definitions

• a meta-optical computation camera can perform color splitting to generate at least one high-quality image.
• Meta-optics can be one potential solution to miniaturizing optics because the lens is flat (rather than being curved). Meta-optics, however, cannot always correct all aberrations optically (e.g., for highly constrained systems). Thus, a need exists for an improved smaller camera.
• a system comprises a first camera including a first filter having a spectral response, a first meta-optic lens, and a first sensor.
• the system also comprises a second camera including a second filter having a spectral response different than the spectral response of the first filter, a second meta-optic lens, and a second sensor.
• the system also comprises a processor configured to receive a representation of a first aberrated image of a scene captured by the first camera, receive a representation of a second aberrated image of the scene captured by the second camera, and generate a representation of a superposed image of the scene based on the representation of the first image and the representation of the second image.
• a system comprises a plurality of cameras configured to generate a representation of a plurality of images of a scene, each camera from the plurality of cameras including a filter with a spectral response different from a spectral response of a filter for each remaining camera, a meta-optic lens, and a sensor.
• the system also comprises a processor configured to receive the representation of the plurality of images of the scene, and generate a representation of a superposed image of the scene based on the representation of the plurality of images.
• a method comprises receiving a representation of a plurality of images of a scene captured by a plurality of cameras, each camera from the plurality of cameras including a filter, a meta-optic lens, and a sensor. For each camera from the plurality of cameras, a spectral response of the filter for that camera is different from a spectral response of the filter for each remaining camera from the plurality of cameras. A representation of a superposed corrected image of the scene is generated based on the representation of the plurality of images.
• Figure 1 shows a system for generating a non-aberrated image based on a representation(s) of multiple aberrated images each captured using an individual meta-optic lens, according to an embodiment.
• Figure 2 shows a camera system with a baffle and multiple meta-optic lenses, according to an embodiment.
• Figure 3 shows a flowchart of a method for generating an image based on a representation(s) of multiple images each captured using a meta-optic lens, according to an embodiment.
• the techniques discussed herein relate to using a plurality of meta-optic lenses, as well as circuitry and/or software, to reduce camera system size (e.g., reduce track length of optics) while also maintaining and/or enhancing imaging performance. Said similarly, the techniques discussed herein use meta-optics, which can reduce camera system size, but don’t rely on the meta-optics to directly correct all aberrations optically. Rather, meta-optics can operate alongside circuitry and/or software to produce high-quality (e.g., non-aberrated) images. [0011] A plurality of meta-optics can be used, where each meta-optic is tailored to subranges (e.g., colors) of the full spectral range.
• Each meta-optic lens can, for example, impart a high- quality blur kernel that is nearly invariant over the field of view for its respective color channel (i.e., spectral subrange).
• spectral subrange i.e., spectral subrange
• Figure 1 shows a diagram of a camera system 100 that can generate an image 190 (e.g., formed image or representation of an image) based on representations of images 150A, 150B, 150C captured by cameras 110A, HOB, 110C, according to an embodiment.
• placing cameras 110A, HOB, 110C adjacent to one another e.g., in a side- by-side arrangement, as shown in Figure 1 can decrease a thickness and/or track length of camera system 100.
• the total track length of the camera system 100 is, for example, less than 1 millimeter.
• the cameras 110A, 110B, 110C can be slightly diagonal to one another (e.g., in a triangular formation).
• camera system 100 can be and/or be included in a camera, webcam, laptop, tablet, phone, television, smart-glasses, smart-watch, vehicle, or any other image capturing device (not shown).
• Each camera 110A, 110B, 110C can include a single meta-optic lens 130A, 130B, 130C.
• the filters 120A, 120B, 120C, meta-optic lenses 130A, 130B, 130C, and sensors 140A, 140B, 140C can be included in a housing 102.
• the circuitry 160 is included in the housing 102.
• the circuitry 160 is not included in the housing 102, in which case electronic representations of the images 150A, 150B, 150C generated by sensors 140A, 140B, 140C in the housing 102 can be sent to the circuitry 160 over, for example, a trace(s) connecting sensor 104A to processor 170 and/or memory 180, a trace(s) connecting sensor 140B to processor 170 and/or memory 180, and a trace(s) connecting sensor 140C to processor 170 and/or memory 180.
• housing 102 has a length that is less than approximately (e.g., within 10% of) at least one of: 6 mm, 5 mm, 4.5 mm, 4 mm, 3.5 mm, or 3 mm. In some implementations, housing 102 has a width that is less than approximately (e.g., within 10% of) at least one of: 2 mm, 1.5 mm, 1.4 mm, 1.3 mm, 1.2 mm, 1.1 mm, 1 mm, 0.9 mm, 0.8 mm, 0.7 mm, 0.6 mm, or 0.5 mm.
• housing 102 has a height that is less than approximately (e.g., within 10% of) at least one of: 1.5 mm, 1.1 mm, 1 mm. 0.9 mm, 0.8 mm, 0.7 mm. 0.6 mm. or 0.5 mm. In some implementations, the dimensions (length x width x height) of the housing 102 is approximately (e.g., within 10% of) 1 mm x 5 mm x 1.4 mm.
• the filters 120A, 120B, 120C can each have a substantially similar size relative to one another, have a substantially different size relative to one another (e.g., based on differing predetermined sampling rates), or a combination thereof (e.g., similar size to one filter but different size to relative to another filer).
• the meta-optic lenses 130A, 130B, 130C can each have a substantially similar size relative to one another, have a substantially different size relative to one another (e.g., based on differing predetermined sampling rates), or a combination thereof (e.g., similar size to one filter but different size to relative to another filer).
• the sensors 140A, 140B, 140C can each have a substantially similar overall size, pixel count, and/or pixel size relative to one another, have a substantially different overall size, pixel count, and/or pixel size relative to one another (e.g., based on differing predetermined sampling rates), or a combination thereof (e.g., similar overall size, pixel count, and/or pixel size to one filter but different overall size, pixel count, and/or pixel size to relative to another filer).
• Figure 1 shows images 150A, 150B, 150C as formed images.
• images 150A, 150B, 150C are electronic representations (e.g., not in a human-viewable form) provided by the sensors 140A, 140B, 140C respectively to the circuity 160, and do not ever need to be transformed into formed images and/or can be transformed into formed images (e.g., in a human-viewable form) at the circuitry 160.
• the images 150A, 150B, 150C are shown for the sake of providing a sense of what the representations captured by the sensors 140 A, 140B, 140C can look like in some implementations.
• the image 190 can be generated by the circuitry 160 as a representation of the image 190 and/or a formed version of the image 190.
• the first camera 110A includes a filter 120 A, meta-optic lens 130A, and sensor 140 A (also referred to as a “detector”).
• Light e.g., white light
• a scene e.g., a human and their background
• a representation of e.g., an electronic signal representing
• the filter 120A can be a color filter.
• the filter 120A can be of any color, such as red, blue, yellow, green, cyan, magenta, or a combination thereof.
• the filter 120 A can be one of red, green, or blue.
• the filter 120A can be one of red, green, blue, or yellow.
• Using a color filter can allow the meta-optic lens 130A to contend with a wavelength range that is narrower than the full visible spectrum, which in turn can allow the meta-optic lens 130A to more easily provide a field of vision (FoV)-invariant blur kernel.
• FoV field of vision
• the meta-optic lens 130A is substantially flat, transparent at visible and infrared wavelengths (i.e., the meta-optic lens 130A has a substantially uniform spectral response at visible and infrared wavelengths), and/or compatible with single-stage lithography processes.
• the meta-optic lens 130A can include, for example, a patterned dielectric layer having a range of different widths; in some implementations, the dielectric can be gallium nitride, silicon nitride, and/or titanium dioxide.
• a shape of the meta-optic lens 130A is not substantially convex nor substantially concave. The term “substantially” is used to account for variability (e.g., manufacturing variability, environmental variability, etc.) that may occur.
• the sensor 140 A can convert an optical image into an electronic signal representing an image (e.g., image 150A).
• the sensor 140A is an image sensor (e.g., charge-coupled device, active-pixel sensor, etc.). After light has travelled through the filter 120 A and meta-optic lens 130A, the light can contact the sensor 140A. Thereafter, the sensor 140A generate a representation of the image 150A. In some implementations, the sensor 140A is not a Bayer-filtered sensor.
• the representation of the image 150A (e.g., if transformed into a formed image) has the same hue as the filter 120A. For example, if the filter 120A was red, the image 150A also has a red hue. As another example, if the filter 120A was green, the image 150A also has a green hue. As another example, if the filter 120A was blue, the image 150A also has a blue hue. Furthermore, the representation of image 150A (e.g., if transformed into a formed image) can include at least one aberration. In some implementations, the representation of the image 150A is blurry (e.g., if transformed into a formed image) (i.e., the scene captured by the first camera 110A is not clearly or distinctly visible in the image 150A).
• the representation of the image 150A is blurry (e.g., if transformed into a formed image) (i.e., the scene captured by the first camera 110A is not clearly or distinctly visible in the image 150A).
• the second camera HOB and third camera 110C can each be similar and/or identical to the camera 110A, with at least one difference being that the filter 120 A of the first camera 110A, the filter 120B of the second camera HOB, the filter 120C of the third camera 110C each differs from the others.
• each filter 120A, 120B and 120C can have a spectral subrange mutually exclusive from the spectral subrange of the other filters 120 A, 120B and 120C.
• each filter 120A, 120B and 120C can have a spectral subrange different from, but possibly partially overlapping with, the spectral subrange of the other filters 120A, 120B and 120C.
• filters 120A, 120B, 120C are each associated with a different color.
• filter 120A can be a red color filter
• filter 120B can be a green color filter
• filter 120C can be a blue color filter.
• the representations of the images 150B, 150C can also each include and/or be associated with at least one aberration, where, for example, the at least one aberration for each representation of image 150A, 150B, and 150C can be of the same type (e.g., blurry, noisy, etc.), but to differing degrees (e.g., due to spectral response differences).
• the representations of images 150B, 150C are blurry (i.e., the scene captured by the second camera HOB and/or third camera 110C is not clearly or distinctly visible in the images 150B, 150C).
• the representations of images 150A, 150B, 150C can be sent to and received by circuitry 160, which includes a processor 170 and memory 180 operatively coupled to the processor 170 (e.g., via a system bus).
• the circuitry 160 can be used to transform the representations of images 150A, 150B, 150C into formed images.
• the circuitry 160 can be used to generate the image 190 (e.g., as a formed image and/or as an electronic representation), which is less aberrated than images 150A, 150B, 150C and/or substantially non-aberrated (e.g., not blurry or not as blurry).
• the processor 170 can be, for example, a hardware based integrated circuit (IC) or any other suitable processing device configured to run and/or execute a set of instructions or code.
• the processor 170 can be a general-purpose processor, a central processing unit (CPU), an accelerated processing unit (APU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a programmable logic array (PLA), a complex programmable logic device (CPLD), a programmable logic controller (PLC) and/or the like.
• the processor 170 can be configured to run any of the methods and/or portions of methods discussed herein.
• the memory 180 can be, for example, a random-access memory (RAM), a memory buffer, a hard drive, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), and/or the like.
• the memory 180 can be configured to store any data used by the processor 170 to perform the techniques discussed herein.
• the memory 180 can store, for example, one or more software programs and/or code that can include instructions to cause the processor 170 to perform one or more processes, functions, and/or the like.
• the memory 180 can include extendible storage units that can be added and used incrementally.
• the memory 180 can be a portable memory (for example, a flash drive, a portable hard disk, and/or the like) that can be operatively coupled to the processor 170. In some instances, the memory 180 can be remotely operatively coupled with a compute device (not shown in Figure 1).
• a portable memory for example, a flash drive, a portable hard disk, and/or the like
• the memory 180 can be remotely operatively coupled with a compute device (not shown in Figure 1).
• the memory 180 can store a software model(s).
• the processor 170 can use the software model(s) to transform representations of the images 150A, 150B, 150C generated by the sensors 140A, 140B, 140C into formed images 150A, 150B, 150C.
• the software model(s) performs parallax correction, deconvolution, distortion correction, and/or up-sampling on each of the images 150A, 150B, 150C (or transformations generated based on the images 150A, 150B, 150C) such that the images 150A, 150B, 150C are less aberrated and/or no longer aberrated (e.g., less or no blur).
• the representations of the images 150A, 150B, 150C are first separately deconvolved to generate representations of deconvolved images. Additional details related to deconvolving the representations of images (such as images 150A, 150B, 150C) to generate the representations of deconvolved images can be found in “Neural nano-optics for high-quality thin lens imaging” (Tseng, E., Colburn, S., Whitehead, J. et al. Neural nanooptics for high-quality thin lens imaging. AW Commun 12, 6493 (2021)), the contents of which are incorporated by reference in its entirety herein.
• each of these representations of deconvolved images are aligned (i.e., parallax corrected) to a single camera in the camera system 100, as well as multiple (e.g., all) cameras in the camera system 100. Additional details related to parallax correction to a single camera can be found in “Disparity Estimation for Image Fusion in a Multi-aperture Camera” (Mustaniemi J., Kannala J ., Heikkila J. (2015) Disparity Estimation for Image Fusion in a Multi-aperture Camera. In: Azzopardi G., Petkov N. (eds) Computer Analysis of Images and Patterns. CAIP 2015. Lecture Notes in Computer Science, vol 9257.
• the software model(s) can superpose the representation of each of the nonaberrated versions of the images 150A, 150B, 150C over one another to generate the image 190.
• the image 190 is an RGB (i.e., (red, green, blue)) image.
• demosaicing can be performed by the software model(s) to generate the image 190 after performing the parallax correction and deconvolution discussed above. Additional details related to the demosaicing that can be performed to generate the image 190 can be found at “HIGH-QUALITY LINEAR INTERPOLATION FOR DEMOSAICING OF BAYER-PATTERNED COLOR IMAGES” (H. S.
• a representation of the image 190 is stored in the memory 180.
• a representation of the image 190 is sent to a compute device (not shown in Figure 1) communi cably coupled to the camera system 100 (e.g., via a wired or wireless connection or network).
• a representation of the image 190 is transformed into a formed image and displayed via a display (not shown in Figure 1); the image 190 can be displayed, for example, to a user associated with (e.g., using or having access to) the camera system 100.
• each of the cameras can include a meta-optic lens, sensor, and unique color filter.
• Each of the cameras can also be operatively coupled to circuitry (e.g., circuitry 160) configured to generate representations of aberrated images (e.g., image 150A, 150B, 150C), a representation of a substantially non-aberrated image (e.g., image 190) from the aberrated images and/or an image with less aberration (e.g., image 190) from and relative to the aberrated images.
• the first camera can be associated with a first spectral subrange (e.g., red, orange, and yellow), and the second camera can be associated with a second spectral subrange (e.g., green, blue, indigo, and violet).
• the camera system 100 is a combined visible/near-infrared (VIS/NIR) camera with two channels (i.e., a VIS channel and a NIR channel); in such a scenario, a first channel (e.g., camera) can include a VIS filter and the second channel (e.g., camera) can include a NIR filter.
• the first camera can include a red color filter
• the second camera can include a green color filter
• the third camera can include a blue color filter
• the fourth camera can include a yellow camera filter.
• two cameras can be associated with substantially (e.g., at least 95% similar, at least 99% similar, etc.) the same spectral range.
• filter 120A of camera 110A can have substantially the same spectral range as the filter of a camera not shown in FIG. 1 (e.g., both filters are green, both filters are red, both filters are blue, both filters are yellow, and/or the like).
• Two cameras having the same spectral response but being slightly (e.g., one to two millimeters) offset from each other (e.g., due to the natural side-by- side positioning of the cameras on the device) can be used to correct for parallax error because differences between the two substantially similar images are not due to spectral differences but due to the physical offset of the two cameras.
• the two images can be used to improve the signal -to-noise ratio (SNR) when generating image 190.
• SNR signal -to-noise ratio
• the two nearly identical images can allow for the noise to be reduced by averaging two “noisy” images, where the average will have less noise (e.g., a noisy signal value at one pixel in one image will be reduced by the less-noisy signal values at that pixel in the other image - the noisy values and the less-noisy values get closer to the true value).
• the cameras 110A, HOB, and 110C in Figure 1 each included a single metaoptic lens 130A, 130B, and 130C, respectively, in other implementations, at least one of the cameras 110A, HOB, or 110C can include multiple meta-optic lenses.
• a camera e.g., camera 110A
• a filter e.g., filter 120A
• each camera 110A, 110B, and 110C can include at least one aperture (e.g., to block stray light and/or substantially off-angle light), where that at least one aperture can be located in front of and/or behind the meta-optic lens for that camera to manage the collection angle, chief ray angle, and/or the like.
• each camera 110A, 110B, and 110C can include two apertures, where one aperture is located in front of the meta-optic lens for that camera and the other aperture is located behind the meta-optic lens.
• the aperture located in front of the meta-optic lens(es) for a given camera can be smaller (i.e., a smaller opening) than the aperture located behind that meta-optic lens(es) for that camera.
• the apertures for the cameras 110A, HOB, 1 IOC can have the same size, different sizes, or a combination thereof.
• the apertures for the cameras 110A, HOB, 1 IOC can be located on a single common plane.
• the apertures for the cameras 110A, 110B, 1 IOC can be slightly diagonal to one another (e.g., in a triangular formation for implementations where the sensors are in a triangular formation).
• each camera 110A, 110B, and 1 IOC can include at least one refractive element (i.e., refractive lens) to correct and/or reduce aberrations, where that at least one refractive element can be located in front of and/or behind the meta-optic lens for that camera, and/or the filter for that camera.
• refractive element i.e., refractive lens
• incident light can travel through an aperture of a camera (e.g., camera 110A), pass through a refractive element thereafter, pass through a filter (e.g., filter 120A) thereafter, pass through a meta-optic lens (e.g., meta-optic lens 130A) thereafter, and be incident on a sensor (e.g., sensor 140A) thereafter to generate a representation of image 150A.
• a camera e.g., camera 110A
• a filter e.g., filter 120A
• meta-optic lens e.g., meta-optic lens 130A
• an infrared (IR) filter can be used.
• at least one of the cameras 110A, 110B, or 110C can include an IR filter.
• the IR filter can be located in front of the sensor for that camera (e.g., directly on top of the sensor, directly on top of the meta-optic lens, directly on top of the color filter, etc.).
• a fourth camera (not shown in Figure 1) can be included in camera system 100.
• the fourth camera can have a nearly identical structure to cameras 110A, 110B, 110C, but use an IR filter instead of a color filter.
• light from the same scene can travel through the IR filter, travel through a meta-optic lens included in the fourth camera, and be incident on a sensor included in the fourth camera to generate a representation of an image.
• the representation of the image generated by the fourth camera can be used, in addition to the representation of images 150A, 150B, 150C, by the circuitry 160 to generate a representation of image 190.
• Figure 1 shows three sensors 140A, MOB, 140C
• a single sensor (not shown in Figure 1) can be used.
• the camera system 100 can include multiple meta-optic lenses and multiple filters, but only a single sensor.
• a single sensor can receive light associated with (e.g., travelling through) multiple different meta-optic lenses and filters. Light passing through each meta-optic lens from the multiple meta-optic lenses can be incident on different sections of the sensor (with or without some overlap between some sections).
• meta-optic lens 130A can be incident on a left portion of the single sensor
• light passing through meta-optic lens 13 OB can be incident on a center portion of the single sensor
• light passing through meta-optic lens 130C can be incident on a right portion of the single sensor; in some implementations, part of the center portion partially overlaps with the left portion, and a different part of the center portion partially overlaps with the right portion.
• the filters 120A, 120B, and 120C shown in Figure 1 are in front of (e.g., in contact with) the meta-optic lens 130A, 13 OB, and 130C, respectively, the location of the filters 120A, 120B, 120C can vary in other implementations.
• a filter e.g., filter 120A, or an IR filter
• a sensor e.g., sensor 140A
• a filter e.g., filter 120A, or an IR filter
• AR anti-reflective
• IR infrared
• AR coating and/or IR coating can be disposed on and/or applied to a meta- optic lens (e.g., the entire lens, one side of the lens, etc.). In some implementations, AR coating and/or IR coating can be applied to and/or disposed on a filter (e.g., the entire filter, one side of the filter, etc.).
• first object being located “in front of’ a second object can refer to the first object being closer to incident light entering the camera 110A, HOB, 110C relative to the second object.
• a first object being located “behind” a second object can refer to the second object being closer to incident light entering the camera 110A, HOB, 110C relative to the first object.
• the first object and/or second object can be, for example, a meta-optic lens, color filter, IR filter, refractive element, filter, or sensor.
• a camera system comprises a first camera (e.g., camera 110A) including a first filter (e.g., filter 120A) having a spectral response (e.g., one of red, green or blue), a first meta-optic lens (e.g., meta-optic BOA), and a first sensor (e.g., sensor 140A).
• a first camera e.g., camera 110A
• a first filter e.g., filter 120A
• a first meta-optic lens e.g., meta-optic BOA
• a first sensor e.g., sensor 140A
• the camera system also comprises a second camera (e.g., camera 110B) including a second filter (e.g., filter 120B) having a spectral response (e.g., one of red, green, or blue) different than the spectral response of the first filter, a second meta-optic lens (e.g., meta-optic lens BOB), and a second sensor (e.g., sensor 140B).
• a second camera e.g., camera 110B
• a second filter e.g., filter 120B
• a spectral response e.g., one of red, green, or blue
• a second meta-optic lens e.g., meta-optic lens BOB
• a second sensor e.g., sensor 140B
• the camera system also comprises a processor (e.g., processor 170) configured to: receive a representation of a first image (e.g., image 150A) of a scene captured by the first camera, receive a representation of a second image (e.g., image 150B) of the scene captured by the second camera, and generate a representation of a superposed image (e.g., image 190) of the scene based on the representation of the first image and the representation of the second image, the representation of the superposed image having an aberration lesser than an aberration of the representation of the first image and the representation of the second image.
• the representation of the superposed image is transformed into a formed image and displayed via a display.
• the camera system further comprises a third camera (e.g., camera 110C) including a third filter having a third spectral response (e.g., one of red, green or blue) different than the spectral response of the first filter and the spectral response of the second filter, a third meta-optic lens (e.g., meta-optic lens 130C), and a third sensor (e.g., sensor 140C);
• the processor can be further configured to receive a representation of a third image (e.g., image 150C) of the scene captured by the third camera, where generating the representation of the superposed image of the scene can be further based on the representation of the third image.
• the camera system further comprises a third camera (e.g., camera 110C) including a third filter having a third spectral response (e.g., one of red, green or blue) substantially similar to the spectral response of the first filter but not the spectral response of the second filter, a third meta-optic lens (e.g., meta-optic lens 130C), and a third sensor (e.g., sensor 140C);
• the processor can be further configured to receive a representation of a third image (e.g., image 150C) of the scene captured by the third camera, where the representation of the first image and the representation of the third image are used for parallax correction and noise reduction during generating the representation of the superposed image.
• the spectral response of the first filter includes red
• the spectral response of the second filter includes green
• the spectral response of the third filter includes blue.
• the first camera, the second camera, and the third camera are disposed on a common plane (e.g., side-by-side), the second camera being disposed between the first camera and the second camera.
• an anti-reflective coating is disposed on only the first meta- optic lens, only the second meta-optic lens, or both the first meta-optic lens and the second meta-optic lens.
• an infrared coating is disposed on only the first meta-optic lens, only the second meta-optic lens, or both the first meta-optic lens and the second meta-optic lens.
• the camera system further includes at least one aperture stop located between (1) the first meta-optic lens and the first filter, and (2) the second meta- optic lens and the second filter. In some implementations, the camera system further includes at least one aperture stop located between (1) the first meta-optic lens and the first sensor, and (2) the second meta-optic lens and the second sensor.
• the camera system further comprises a fourth camera (not shown in Figure 1) including a fourth filter having a fourth spectral response (e.g., yellow color filter) different than the spectral response of the first filter, the spectral response of the second filter, and the spectral response of the third filter, a fourth meta-optic lens, and a fourth sensor;
• the processor can be further configured to receive a representation of a fourth image of the scene captured by the fourth camera, where generating the representation of the superposed image of the scene can be further based on the representation of the fourth image.
• a camera system comprises a plurality (e.g., two, three, four, five, etc.) of cameras (e.g., at least two of camera 110A, HOB, or 110C) configured to generate a representation of a plurality of images (e.g., at least two of image 150A, 150B, or 150C) of a scene, each camera from the plurality of cameras including a filter (e.g., filter 120A, 120B, or 120C) with a spectral response different from a spectral response of a filter for each remaining camera, a meta-optic lens (e.g., meta-optic lens 130A, 130B, or 130C), and a sensor (e.g., sensor 140 A, 140B, or 140C).
• a filter e.g., filter 120A, 120B, or 120C
• a meta-optic lens e.g., meta-optic lens 130A, 130B, or 130C
• sensor e.g., sensor 140 A, 140
• the camera system further comprises a processor (e.g., processor 170) configured to receive the representation of the plurality of images of the scene, and generate a representation of a superposed image (e.g., image 190) of the scene based on the representation of the plurality of images.
• a processor e.g., processor 170
• a superposed image e.g., image 190
• generating the representation of the superposed image can include performing parallax correction, deconvolution, distortion correction, and/or up- sampling to the representation of the plurality of images to generate a representation of a plurality of superposed images, and superposing the representation of the plurality of superposed images to form the representation of the superposed image.
• the camera system includes only two cameras. In such a case, the plurality of images can include at least a first image and a second image (and in some implementations, no additional images). In some implementations, the camera system includes only three cameras. In such a case, the plurality of images can include at least a first image, a second image, and a third image (and in some implementations, no additional images). In some implementations, the camera system includes only four cameras. In such a case, the plurality of images can include at least a first image, a second image, a third image, and a fourth image (and in some implementations, no additional images).
• camera system 100 includes a baffle.
• the baffle can be located above meta-optic lenses 130A, 13 OB, 130C such that light passes through the baffle before passing through meta-optic lenses 130A, 130B, 130C.
• the baffle is wider at the end further from the meta-optic lens, and gets narrower closer to the meta-optic lens to eliminate high angle rays.
• the baffle is attached to an aperture stop or also acts as an aperture stop.
• the baffle protrudes vertically along the length of the camera system 100.
• the baffle can eliminate and/or reduce cross talk by allowing light to be received by a given camera 110A, HOB, 110C while blocking that light from going into other cameras 110A, HOB, 110C (that light can be considered high-angle light from the perspective of the other cameras 110A, 110B, HOC).
• FIG. 2 shows a camera system with a baffle and multiple meta-optic lenses, according to an embodiment.
• Filters 206 includes four filters - red filter 206A, green filter 206B, green filter 206C, and blue filter 206D.
• Meta-optic lenses 204 includes four meta-optic lenes - meta optic lens 204A, meta-optic lens 204B, meta-optic lens 204C, and meta-optic lens 204D.
• Baffle 202 includes four openings - opening 202A, opening 202B, opening 202C, and opening 202D. Light can pass through opening 202A, through meta-optic lens 204A, and through filter 206A to an image sensor.
• Light can also pass through opening 202B, through meta-optic lens 204B, and through filter 206B to an image sensor.
• Light can also pass through opening 202C, through meta-optic lens 204C, and through filter 206C to an image sensor.
• Light can also pass through opening 202D, through meta-optic lens 204D, and through filter 206D to an image sensor.
• FIG. 2 shows the camera system with baffle 202 and without a separate aperture stop
• the baffle 202 can be configured to act as an aperture stop.
• the outer perimeter of each upper portion of the openings 202A, 202B, 202C, 202D can act as an aperture stop depending on the size of that outer perimeter relative to other dimensions of the camera system.
• the outer perimeter of each lower portion of the openings 202A, 202B, 202C, 202D can act as an aperture stop depending on the size of that outer perimeter relative to other dimensions of the camera system.
• a separate aperture stop can be included.
• Figure 3 shows a flowchart of a method for generating a representation of an image based on a representation of a plurality of images each captured using a meta-optic lens and unique color filter, according to an embodiment.
• the method discussed with respect to Figure 3 can be performed by/at the processor 170.
• At 302 receive a representation of a plurality (e.g., two, three, four, five) of images (e.g., image 150A, 150B, 150C) of a scene captured by a plurality of cameras (e.g., camera 110A, HOB, 110C), each camera from the plurality of cameras including a filter (e.g., filter 120A, 120B, or 120C), a meta-optic lens (e.g., meta-optic lens 130A, 130B, or 130C), and a sensor (e.g., sensor 140 A, 140B, or 140C).
• a filter e.g., filter 120A, 120B, or 120C
• a meta-optic lens e.g., meta-optic lens 130A, 130B, or 130C
• sensor 140 A, 140B, or 140C e.g., sensor 140 A, 140B, or 140C
• a spectral response of the filter for the camera differs from a spectral response of the filter for each remaining camera from the plurality of camera.
• generate a representation of a superposed corrected image (e.g., image 190) of the scene based on the representation of the plurality of images.
• the representation of the plurality of images are transformed into a representation of plurality of superposed images (e.g., via parallax correction, deconvolution, distortion correction, and/or up-sampling software), and the representation of the plurality of superposed images are superposed to generate the representation of the superposed corrected image at 304.
• the representation of the superposed corrected image is transformed into a formed image and displayed (e.g., automatically and without requiring further human input) at a display in response to generating the representation of the superposed corrected image at 304.
• the plurality of cameras includes only two cameras. In some implementations, the plurality of cameras includes only three cameras. In some implementations, the plurality of cameras includes only four cameras. In some implementations, the plurality of cameras includes more than four cameras.
• the spectral response of the filter for a first camera includes red
• the spectral response of the filter for a second camera includes green
• the spectral response of the filter for a third camera includes blue
• the spectral response of the filter for a fourth camera includes a color different than red, green, and blue (e.g., yellow, cyan, magenta, etc.).
• the spectral response of the filter for a first camera includes red
• the spectral response of the filter for a second camera includes green
• the spectral response of the filter for a third camera includes blue
• the filter for a fourth camera is an infrared filter.
• the plurality of cameras includes any number of cameras (e.g., at least 5 cameras, at least 10 cameras, at least 20 camera, etc.).
• each camera from the plurality of cameras can be associated with a different spectral response.
• two or more cameras from the plurality of cameras can have the same spectral response.
• two or more cameras from the plurality of cameras have the same spectral response, and the remaining cameras from the plurality of cameras are associated with spectral response different than all other cameras from the remaining cameras; for example, a first camera is associated with a spectral response that includes green, a second camera is associated with a spectral response that includes green, a third camera is associated with a spectral response that includes red, a fourth camera is associated with a spectral response that includes blue, and a fifth camera is associated with a spectral response that includes yellow.
• Various concepts may be embodied as one or more methods, of which at least one example has been provided.
• the acts performed as part of the method may be ordered in any suitable way. Accordingly, embodiments may be constructed in which acts are performed in an order different than illustrated, which may include performing some acts simultaneously, even though shown as sequential acts in illustrative embodiments.
• features may not necessarily be limited to a particular order of execution, but rather, any number of threads, processes, services, servers, and/or the like that may execute serially, asynchronously, concurrently, in parallel, simultaneously, synchronously, and/or the like in a manner consistent with the disclosure. As such, some of these features may be mutually contradictory, in that they cannot be simultaneously present in a single embodiment. Similarly, some features are applicable to one aspect of the innovations, and inapplicable to others.
• the disclosure may include other innovations not presently described. Applicant reserves all rights in such innovations, including the right to embodiment such innovations, file additional applications, continuations, continuations-in-part, divisionals, and/or the like thereof. As such, it should be understood that advantages, embodiments, examples, functional, features, logical, operational, organizational, structural, topological, and/or other aspects of the disclosure are not to be considered limitations on the disclosure as defined by the embodiments or limitations on equivalents to the embodiments.
• the terms “about” or “approximately” when preceding a numerical value indicates the value plus or minus a range of 10%.
• a range of values is provided, it is understood that each intervening value, to the tenth of the unit of the lower limit unless the context clearly dictates otherwise, between the upper and lower limit of that range and any other stated or intervening value in that stated range is encompassed within the disclosure. That the upper and lower limits of these smaller ranges can independently be included in the smaller ranges is also encompassed within the disclosure, subject to any specifically excluded limit in the stated range. Where the stated range includes one or both of the limits, ranges excluding either or both of those included limits are also included in the disclosure.
• a reference to “A and/or B”, when used in conjunction with open-ended language such as “comprising” can refer, in one embodiment, to A only (optionally including elements other than B); in another embodiment, to B only (optionally including elements other than A); in yet another embodiment, to both A and B (optionally including other elements); etc.
• the phrase “at least one,” in reference to a list of one or more elements, should be understood to mean at least one element selected from any one or more of the elements in the list of elements, but not necessarily including at least one of each and every element specifically listed within the list of elements and not excluding any combinations of elements in the list of elements.
• This definition also allows that elements may optionally be present other than the elements specifically identified within the list of elements to which the phrase “at least one” refers, whether related or unrelated to those elements specifically identified.
• “at least one of A and B” can refer, in one embodiment, to at least one, optionally including more than one, A, with no B present (and optionally including elements other than B); in another embodiment, to at least one, optionally including more than one, B, with no A present (and optionally including elements other than A); in yet another embodiment, to at least one, optionally including more than one, A, and at least one, optionally including more than one, B (and optionally including other elements); etc.
• Hardware modules may include, for example, a processor, a field programmable gate array (FPGA), and/or an application specific integrated circuit (ASIC).
• Software modules (executed on hardware) can include instructions stored in a memory that is operably coupled to a processor, and can be expressed in a variety of software languages (e.g., computer code), including C, C++, JavaTM, Ruby, Visual BasicTM, and/or other object-oriented, procedural, or other programming language and development tools.
• Examples of computer code include, but are not limited to, micro-code or micro-instructions, machine instructions, such as produced by a compiler, code used to produce a web service, and files containing higher-level instructions that are executed by a computer using an interpreter.
• embodiments may be implemented using imperative programming languages (e.g., C, Fortran, etc.), functional programming languages (Haskell, Erlang, etc.), logical programming languages (e.g., Prolog), object-oriented programming languages (e.g., Java, C++, etc.) or other suitable programming languages and/or development tools.
• Additional examples of computer code include, but are not limited to, control signals, encrypted code, and compressed code.
• processor should be interpreted broadly to encompass a general purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine and so forth.
• a “processor” may refer to an application specific integrated circuit (ASIC), a programmable logic device (PLD), a field programmable gate array (FPGA), etc.
• ASIC application specific integrated circuit
• PLD programmable logic device
• FPGA field programmable gate array
• processor may refer to a combination of processing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core or any other such configuration.
• memory should be interpreted broadly to encompass any electronic component capable of storing electronic information.
• the term memory may refer to various types of processor-readable media such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, magnetic or optical data storage, registers, etc.
• RAM random access memory
• ROM read-only memory
• NVRAM non-volatile random access memory
• PROM programmable read-only memory
• EPROM erasable programmable read only memory
• EEPROM electrically erasable PROM
• flash memory magnetic or optical data storage, registers, etc.
• instructions and “code” should be interpreted broadly to include any type of computer-readable statement s).
• the terms “instructions” and “code” may refer to one or more programs, routines, sub-routines, functions, procedures, etc.
• “Instructions” and “code” may comprise a single computer-readable statement or many computer-readable statements.

Landscapes

• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Engineering & Computer Science (AREA)
• Optics & Photonics (AREA)
• Signal Processing (AREA)
• Multimedia (AREA)
• Toxicology (AREA)
• Health & Medical Sciences (AREA)
• Theoretical Computer Science (AREA)
• Color Television Image Signal Generators (AREA)
• Geometry (AREA)
• Cameras In General (AREA)
• Blocking Light For Cameras (AREA)
• Studio Devices (AREA)

Priority Applications (2)

Applications Claiming Priority (2)

Publications (1)

Family

ID=85601548

Family Applications (1)

Country Status (5)

Families Citing this family (3)

Citations (4)

Family Cites Families (9)

• 2023
  • 2023-02-14 WO PCT/US2023/062549 patent/WO2023154946A1/en not_active Ceased
  • 2023-02-14 US US18/168,632 patent/US12177551B2/en active Active
  • 2023-02-14 JP JP2024570590A patent/JP2025506306A/ja active Pending
  • 2023-02-14 TW TW112105204A patent/TW202346911A/zh unknown
  • 2023-02-14 KR KR1020247030471A patent/KR20240151196A/ko active Pending

Patent Citations (4)

Non-Patent Citations (4)

Also Published As

Similar Documents

Legal Events