US20170053381A1 - Interlaced bi-sensor super-resolution enhancement - Google Patents
Interlaced bi-sensor super-resolution enhancement Download PDFInfo
- Publication number
- US20170053381A1 US20170053381A1 US15/341,547 US201615341547A US2017053381A1 US 20170053381 A1 US20170053381 A1 US 20170053381A1 US 201615341547 A US201615341547 A US 201615341547A US 2017053381 A1 US2017053381 A1 US 2017053381A1
- Authority
- US
- United States
- Prior art keywords
- array
- pixels
- pixel
- interlaced
- sensor
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 238000000034 method Methods 0.000 claims abstract description 33
- 238000013461 design Methods 0.000 claims abstract description 25
- 238000005070 sampling Methods 0.000 claims description 5
- 239000011295 pitch Substances 0.000 description 31
- 238000003491 array Methods 0.000 description 23
- 230000009977 dual effect Effects 0.000 description 15
- 238000005516 engineering process Methods 0.000 description 7
- 238000004590 computer program Methods 0.000 description 4
- 238000013459 approach Methods 0.000 description 3
- 238000012938 design process Methods 0.000 description 3
- 230000002708 enhancing effect Effects 0.000 description 3
- 239000010408 film Substances 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 239000000463 material Substances 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 235000012431 wafers Nutrition 0.000 description 2
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 1
- 230000003750 conditioning effect Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000007123 defense Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000008021 deposition Effects 0.000 description 1
- 238000002059 diagnostic imaging Methods 0.000 description 1
- 238000005530 etching Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 229910052710 silicon Inorganic materials 0.000 description 1
- 239000010703 silicon Substances 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
- 239000010409 thin film Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/30—Circuit design
- G06F30/39—Circuit design at the physical level
- G06F30/392—Floor-planning or layout, e.g. partitioning or placement
-
- H—ELECTRICITY
- H01—ELECTRIC ELEMENTS
- H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
- H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
- H01L27/14—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
- H01L27/144—Devices controlled by radiation
- H01L27/146—Imager structures
- H01L27/14601—Structural or functional details thereof
- H01L27/14609—Pixel-elements with integrated switching, control, storage or amplification elements
- H01L27/1461—Pixel-elements with integrated switching, control, storage or amplification elements characterised by the photosensitive area
-
- G06F17/5072—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- H—ELECTRICITY
- H01—ELECTRIC ELEMENTS
- H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
- H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
- H01L27/14—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
- H01L27/144—Devices controlled by radiation
- H01L27/146—Imager structures
- H01L27/14601—Structural or functional details thereof
- H01L27/14603—Special geometry or disposition of pixel-elements, address-lines or gate-electrodes
- H01L27/14605—Structural or functional details relating to the position of the pixel elements, e.g. smaller pixel elements in the center of the imager compared to pixel elements at the periphery
-
- H—ELECTRICITY
- H01—ELECTRIC ELEMENTS
- H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
- H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
- H01L27/14—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
- H01L27/144—Devices controlled by radiation
- H01L27/146—Imager structures
- H01L27/14601—Structural or functional details thereof
- H01L27/14634—Assemblies, i.e. Hybrid structures
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/45—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/40—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled
- H04N25/46—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled by combining or binning pixels
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/70—SSIS architectures; Circuits associated therewith
- H04N25/702—SSIS architectures characterised by non-identical, non-equidistant or non-planar pixel layout
-
- H04N5/2258—
-
- H04N5/347—
-
- H04N5/3696—
-
- H—ELECTRICITY
- H01—ELECTRIC ELEMENTS
- H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
- H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
- H01L27/14—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
- H01L27/144—Devices controlled by radiation
- H01L27/146—Imager structures
- H01L27/14643—Photodiode arrays; MOS imagers
Definitions
- the invention relates to enhancement of images and, more particularly, to interlaced bi-sensor super-resolution enhancement techniques and a resultant scalable pixel array suitable for a mega-pixel design.
- Enhancing the resolution of an image of a camera can be important for many different applications. For example, enhancing resolution of an image can find application in surveillance, military and defense, aerospace, traffic control, medical imaging, and commercial applications such as smart phones that do not have the luxury of large sensor size.
- One approach of enhancing a resolution of an image is through software algorithms.
- a common way to enhance a resolution is through super-resolution techniques depending on software post-processing algorithms.
- These software post-processing algorithms can be applicable to multi-framed or single-frame, where the effectiveness depends on the scene.
- Another approach may be to use two sensors (bi-sensor super resolution); however, there is an increased cost associated with this method.
- the increased cost is not only due to the need of two sensors, but also due to needing two full cameras, which in turn, requires two lenses (large, bulky, expensive), two processors, two batteries, two separate output streams, etc.
- a bi-sensor approach also requires ensuring identical optical paths for both sensors, which requires the use of a beamsplitter and a calibrated optical setup. This significantly increases complexity.
- a method comprises interlacing a first array of pixels of a first size with a second array of pixels of a second size.
- the interlacing of the first array of pixels with the second array of pixels avoids crossing two or more photosensitive areas of the first array of pixels and the second array of pixels.
- a method comprises: selecting a first array of pixels comprising pixels of a first pitch; selecting a second array of pixels comprising pixels of a second pitch which is different than the first pitch; and interlacing the first array of pixels and the second array of pixels to form a repeatable modular block for an interlaced bi-sensor super-resolution pixel array while avoiding crossing of photosensitive areas of the first array of pixels and the second array of pixels.
- a repeatable modular block of interlaced bi-sensor super-resolution pixel array comprises a first array of pixels comprising pixels of a first pitch interlaced with a second array of pixels comprising pixels of a second pitch which is different than the first pitch such that photosensitive areas of the first array of pixels and the second array of pixels are minimized.
- a computer system for interlacing pixel arrays comprises: a CPU, a computer readable memory and a computer readable storage media; program instructions to select a first array of pixels comprising pixels of a first pitch; program instructions to select a second array of pixels comprising pixels of a second pitch which is different than the first pitch; and program instructions to interlace the first array of pixels and the second array of pixels to form a repeatable modular block for an interlaced bi-sensor super-resolution pixel array while avoiding crossing of photosensitive areas of the first array of pixels and the second array of pixels; wherein the program instructions are stored on the computer readable storage media for execution by the CPU via the computer readable memory.
- FIG. 1 shows two pixel arrays which can be combined into a dual array single sensor
- FIG. 2 represents an interlacing of the two pixel arrays shown in FIG. 1 ;
- FIG. 3 shows two pixel arrays which can be combined into a dual array single sensor
- FIG. 4 represents an interlacing of the two pixel arrays shown in FIG. 3 ;
- FIG. 5 shows two pixel arrays which can be combined into a dual array single sensor
- FIG. 6 represents an interlacing of the two pixel arrays shown in FIG. 5 ;
- FIG. 7 shows two pixel arrays which can be combined into a dual array single sensor
- FIG. 8 represents an interlacing of the two pixel arrays shown in FIG. 7 ;
- FIG. 9 shows a computing infrastructure to implement aspects of the present invention.
- the invention relates to enhancement of images and, more particularly, to interlaced bi-sensor super-resolution enhancement techniques and a resultant scalable sensor array suitable for a mega-pixel design.
- the present invention provides a method to form building blocks of pixel arrays for copy and paste operations to form a pixel sensor array suitable for a mega-pixel design. In this way, the invention can be implemented in an interlaced bi-sensor super-resolution camera.
- the present invention provides a dual-array, single image sensor with different pitch interlaced pixels, especially used for super resolution (SR) of images.
- SR super resolution
- the method of the present invention satisfies the sampling rate diversity requirement in only one single sensor; compared to the need for two sensors to satisfy its different sampling rate requirement of conventional methods and systems.
- the sensor design interlaces two image sensor arrays within one sensor.
- the two interlaced arrays have a different pitch that will allow this dual array single sensor to function as a bi-sensor super-resolution camera.
- the array has pixels that are laid out in a modular (repeatable) block that will allow for typical image sensor array layout and which provides the capabilities of copy and paste operations to build a scalable pixel array sensor suitable for mega-pixel design.
- the present invention advantageously results in a very simplified sensor that cost the same as a single camera single image sensor setup and has no major sacrifice to silicon area.
- the sensor design of the present invention can be manufactured in a number of ways using a number of different tools.
- the methodologies and tools are used to form structures with dimensions in the micrometer and nanometer scale.
- the methodologies, i.e., technologies, employed to manufacture the sensor design of the present invention have been adopted from integrated circuit (IC) technology.
- the structures of the present invention are built on wafers and are realized in films of material patterned by photolithographic processes on the top of a wafer.
- the fabrication of the sensor design of the present invention uses three basic building blocks: (i) deposition of thin films of material on a substrate, (ii) applying a patterned mask on top of the films by photolithographic imaging, and (iii) etching the films selectively to the mask.
- the layout of the present invention is applicable to different technologies including, for example, CMOS 180 nm technology from Global Foundries as well as LF 150 nm technology, by KACST.
- FIG. 1 shows two arrays, Array A and Array B which can be combined into a dual array single sensor to function as bi-sensor super-resolution camera or building blocks as shown in FIG. 2 .
- Array B is shown in a block of 4 ⁇ 4, while Array A is shown in a block of 5 ⁇ 5; although other block sizes are contemplated by the present invention.
- the pitch of Array A is labeled P 1 and the pitch of Array B is labeled P 2 .
- Array A has pixels (Pixel A) of a smaller area (smaller pitch in both dimensions horizontal and vertical) with a specific ratio when compared to Array B (that has Pixel B).
- Pixel B has a pitch that is 1.25 times that of Pixel A, therefore, in the example of FIG. 1 , Pixel A has a pitch of 8 ⁇ m and Pixel B has a pitch of 10 ⁇ m, which is a ratio of 1.25. This case is only specific to the example ratio of 1.25, where other ratios may require different block sizes. It should be understood by those of skill in the art, that other pitches or different block sizes are thus contemplated by the present invention.
- the pixels contain a photodiode that it is usually smaller than the size of the pixel to leave room for the electronics required for pixel conditioning and control (e.g., such as the 3-transistor active pixel sensor design).
- the room for electronics is necessary in almost all CMOS image sensor design, except for the case of back-side illumination technology, where the electronics are integrated in a different layer than the photodiode.
- the present invention contemplates the need for different pitches for the different arrays in order to prevent blockage (e.g., crossover) of photosensitive areas as described herein when interlaced as shown in FIG. 2 .
- the pixels and photodiodes are assumed to be squares; although other shapes and sizes are contemplated by the present invention.
- FIG. 2 represents both of the arrays (Array A and Array B) overlapped in an interlaced manner without crossing photosensitive areas in accordance with aspects of the present invention, e.g., dual array single sensor.
- the dual array single sensor is more specifically representative of a repeatable, scalable modular block which can be used for a dual array single sensor suitable for a mega-pixel design.
- a cross in two or more photosensitive areas may result in a variation of the pixel pitch.
- FIG. 2 for example, at pixel (0,0), the first pixel on the top left corner, both photodiodes overlap fully, meaning that only one photodiode laid out in that location can serve for both pixels.
- some of the photodiodes may come in close proximity of each other; although, this is not an issue in FIG. 2 .
- it may be an issue in some layouts as there is a minimum well spacing between the photodiodes to avoid design rule check (DRC) violation. Accordingly, in implementing the present invention, a determination is made as to the minimum well spacing between the photodiodes to avoid design rule check (DRC) violation.
- a represents the size of the photodiode and a 2 is thus representative of the area of the photodiode.
- the present invention is not limited to a scaling factor of 1.25, and that other scaling factors can be implemented with the present invention, based on size of the pixels or arrays and other considerations described herein, e.g., avoidance of crossing any of the photosensitive areas (blockage of photosensitive areas when arrays are overlapped).
- the scaling factor can be based on (i+1)/i, where i is representative of an array, e.g., a 4 ⁇ 4 array will result in (4+1)/4 which equals 1.25 scaling factor.
- an upscaling factor (e.g., “s”) can be based on a 1 megabyte pixel in, e.g., a 6 ⁇ 6 array (36 megabyte array), where the scaling factor can be, e.g., (6+1)/6 which equals 1.17.
- the minimum well spacing rule is technology dependent and, as such, the minimum well spacing rule of 0.1 is provided as an illustrative example.
- the fill-factor (FF) which is a ratio of photosensitive part of the pixel to entire pixel area, can be estimated depending on the number of photosensitive parts of the pixel, which varies between 2 to 4, as follows:
- the interlaced array 100 of FIG. 2 can be used as a building block for a dual array single sensor.
- the building block of FIG. 2 can be used in copy and paste design processes known to those of skill in the art to complete a scalable array suitable for a mega-pixel design.
- the present invention can be provided in design processes which preferably employ and incorporate hardware and/or software modules for synthesizing, translating, or otherwise processing a design/simulation functional equivalent of the components, circuits, devices, or logic structures shown in FIG. 2 to generate a netlist which may contain equivalent design structures of the array of FIG. 2 .
- Netlist may comprise, for example, compiled or otherwise processed data structures representing a list of wires, discrete components, logic gates, control circuits, I/O devices, models, etc. that describes the connections to other elements and circuits in an integrated circuit design of FIGS. 1 and 2 .
- Resulting design structures, representative of the arrays of FIGS. 1 and 2 can reside on a storage medium or programmable gate array in a data format used for the exchange of data of mechanical devices and structures (e.g. IGES, DXF, Parasolid XT, JT, DRG, GDSII (GDS2), GL1, OASIS, etc.).
- the design structure when processed by an ECAD system generates a logically or otherwise functionally equivalent form of one or more of the embodiments of the invention shown in FIG. 2 (and FIGS. 4, 6 and 8 ).
- FIGS. 3 and 4 provide alternative processes to provide an interlaced array in accordance with aspects of the present invention.
- super resolution can be performed using only one dimensional dual pitch, instead of two. That is, FIG. 3 shows a scaling in one dimension, e.g., horizontal (but not both horizontal and vertical). In this case, the FF can be increased significantly. Also in this case, the repeatable modular block will be smaller, as shown in FIG. 4 (compared to FIG. 2 ).
- FIG. 3 shows two arrays, Array A and Array B which can be combined into a dual array single sensor to function as a bi-sensor super-resolution camera as shown in FIG. 4 .
- the interlaced array 100 ′ of FIG. 4 is representative of a repeatable, scalable modular block which can be used to for a dual array single sensor suitable for a mega-pixel design.
- Array B is shown in a block of 4 ⁇ 4, while Array A is shown in a block of 5 ⁇ 5.
- the pitch of Array A is P 1 (horizontal) and P 2 (vertical), whereas, the pitch of Array B is P 2 for both horizontal and vertical.
- Array A has rectangular pixels and Array B has square pitches.
- Array A has pixels (Pixel A) of a smaller area (smaller pitch in the horizontal) with a specific ratio when compared to Array B (that has Pixel B).
- the modular, repeatable building block of FIG. 4 can be used for a dual array single sensor suitable for a mega-pixel design.
- the building block of FIG. 4 will provide enhanced resolution.
- an upscale factor need only be used in a single dimension in this implementation which, accordingly, will only increase the bit size in one dimension.
- the building blocks of FIG. 4 can be used as repeatable, scalable modular blocks 100 ′ suitable for a mega-pixel design.
- the building block of FIG. 2 can be used in copy and paste design processes known to those of skill in the art to complete a dual array single sensor.
- Array B has a pitch P 2 (10 units) that is larger than Array A by a factor of 1.25.
- the photodiodes in this case are larger than the condition derived in equation 6, which results in some of the photodiodes in the 4 ⁇ 4 block overlapping.
- FIG. 6 some of the photodiodes of the Array A are removed in order to avoid DRC errors or to avoid modifying the pitch of Array B.
- the result of the interlaced layout 100 ′′ is shown in FIG.
- a variable sampling rate can also be implemented by removing pixels from secondary array (Array B), as shown in FIG. 8 .
- Array B samples are 50% less than the original case, also only 20% less than both arrays together. This was also tested and shown not to have any significant impact on the enhanced images. It should be understood by those of skill in the art that any combination of pixels can be removed from the primary array (Array A) or the secondary array (Array B) in order to avoid crossover of the pixels, as described herein.
- the present invention may be embodied as a system, method or computer program product.
- the present invention may take the form of a hardware embodiment, a software embodiment or a combination of software and hardware.
- the present invention may take the form of a computer program product (program instructions) embodied in any tangible storage medium of expression having computer-usable program code embodied in the medium, which implements the methods, techniques and processes herein.
- the computer readable storage media may be any medium that can contain, store, or communicate the program for use by or in connection with the instruction execution system, apparatus, or device.
- the computer readable storage media is not a signal per se, or transitory.
- the computer-usable or computer-readable medium may be, for example, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium.
- FIG. 9 shows an illustrative environment 10 for managing the processes in accordance with the invention.
- the environment 10 includes a server or other computing system 12 that can perform the processes described herein.
- the server 12 includes a computing device 14 which can be resident on a network infrastructure or computing device.
- the computing device 14 includes a processor (CPU) 20 , memory 22 A, an I/O interface 24 , and a bus 26 .
- the computing device includes random access memory (RAM), a read-only memory (ROM), and an operating system (O/S).
- RAM random access memory
- ROM read-only memory
- O/S operating system
- the computing device 14 is in communication with the external I/O device/resource 28 and the storage system 22 B.
- the I/O device 28 can comprise any device that enables an individual to interact with the computing device 14 (e.g., user interface) or any device that enables the computing device 14 to communicate with one or more other computing devices using any type of communications link.
- the processor 20 executes computer program code (e.g., program control or program instructions 44 ), which can be stored in the memory 22 A and/or storage system 22 B. While executing the computer program code, the processor 20 can read and/or write data to/from memory 22 A, storage system 22 B, and/or I/O interface 24 .
- the program code executes the processes of the invention.
- the present invention comprises the CPU, computer readable memory and computer readable storage media.
- program instructions are configured to select a first array of pixels comprising pixels of a first pitch.
- Program instructions are configured to select a second array of pixels comprising pixels of a second pitch which is different than the first pitch.
- Program instructions are configured to interlace the first array of pixels and the second array of pixels to form a repeatable modular block for an interlaced bi-sensor super-resolution pixel array while avoiding crossing of photosensitive areas of the first array of pixels and the second array of pixels.
- Program instructions are configured to provide the other methods, steps and techniques already described herein.
- the program instructions are stored on the computer readable storage media for execution by the CPU via the computer readable memory.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Power Engineering (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- Condensed Matter Physics & Semiconductors (AREA)
- Electromagnetism (AREA)
- Microelectronics & Electronic Packaging (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Architecture (AREA)
- Evolutionary Computation (AREA)
- Geometry (AREA)
- General Engineering & Computer Science (AREA)
- Transforming Light Signals Into Electric Signals (AREA)
- Solid State Image Pick-Up Elements (AREA)
Abstract
Interlaced bi-sensor super-resolution enhancement techniques and a resultant scalable pixel array suitable for a mega-pixel design are disclosed. The method includes interlacing a first array of pixels of a first size with a second array of pixels of a second size. The interlacing of the first array of pixels with the second array of pixels avoids crossing two or more photosensitive areas of the first array of pixels and the second array of pixels.
Description
- The invention relates to enhancement of images and, more particularly, to interlaced bi-sensor super-resolution enhancement techniques and a resultant scalable pixel array suitable for a mega-pixel design.
- Enhancing the resolution of an image of a camera can be important for many different applications. For example, enhancing resolution of an image can find application in surveillance, military and defense, aerospace, traffic control, medical imaging, and commercial applications such as smart phones that do not have the luxury of large sensor size.
- One approach of enhancing a resolution of an image is through software algorithms. For example, a common way to enhance a resolution is through super-resolution techniques depending on software post-processing algorithms. These software post-processing algorithms can be applicable to multi-framed or single-frame, where the effectiveness depends on the scene.
- Another approach may be to use two sensors (bi-sensor super resolution); however, there is an increased cost associated with this method. The increased cost is not only due to the need of two sensors, but also due to needing two full cameras, which in turn, requires two lenses (large, bulky, expensive), two processors, two batteries, two separate output streams, etc. Additionally, a bi-sensor approach also requires ensuring identical optical paths for both sensors, which requires the use of a beamsplitter and a calibrated optical setup. This significantly increases complexity.
- In an aspect of the invention, a method comprises interlacing a first array of pixels of a first size with a second array of pixels of a second size. The interlacing of the first array of pixels with the second array of pixels avoids crossing two or more photosensitive areas of the first array of pixels and the second array of pixels.
- In an aspect of the invention, a method comprises: selecting a first array of pixels comprising pixels of a first pitch; selecting a second array of pixels comprising pixels of a second pitch which is different than the first pitch; and interlacing the first array of pixels and the second array of pixels to form a repeatable modular block for an interlaced bi-sensor super-resolution pixel array while avoiding crossing of photosensitive areas of the first array of pixels and the second array of pixels.
- In an aspect of the invention, a repeatable modular block of interlaced bi-sensor super-resolution pixel array comprises a first array of pixels comprising pixels of a first pitch interlaced with a second array of pixels comprising pixels of a second pitch which is different than the first pitch such that photosensitive areas of the first array of pixels and the second array of pixels are minimized.
- In an aspect of the invention, a computer system for interlacing pixel arrays, comprises: a CPU, a computer readable memory and a computer readable storage media; program instructions to select a first array of pixels comprising pixels of a first pitch; program instructions to select a second array of pixels comprising pixels of a second pitch which is different than the first pitch; and program instructions to interlace the first array of pixels and the second array of pixels to form a repeatable modular block for an interlaced bi-sensor super-resolution pixel array while avoiding crossing of photosensitive areas of the first array of pixels and the second array of pixels; wherein the program instructions are stored on the computer readable storage media for execution by the CPU via the computer readable memory.
- The present invention is described in the detailed description which follows, in reference to the noted plurality of drawings by way of non-limiting examples of exemplary embodiments of the present invention.
-
FIG. 1 shows two pixel arrays which can be combined into a dual array single sensor; -
FIG. 2 represents an interlacing of the two pixel arrays shown inFIG. 1 ; -
FIG. 3 shows two pixel arrays which can be combined into a dual array single sensor; -
FIG. 4 represents an interlacing of the two pixel arrays shown inFIG. 3 ; -
FIG. 5 shows two pixel arrays which can be combined into a dual array single sensor; -
FIG. 6 represents an interlacing of the two pixel arrays shown inFIG. 5 ; -
FIG. 7 shows two pixel arrays which can be combined into a dual array single sensor; -
FIG. 8 represents an interlacing of the two pixel arrays shown inFIG. 7 ; and -
FIG. 9 shows a computing infrastructure to implement aspects of the present invention. - The invention relates to enhancement of images and, more particularly, to interlaced bi-sensor super-resolution enhancement techniques and a resultant scalable sensor array suitable for a mega-pixel design. For example, the present invention provides a method to form building blocks of pixel arrays for copy and paste operations to form a pixel sensor array suitable for a mega-pixel design. In this way, the invention can be implemented in an interlaced bi-sensor super-resolution camera.
- More specifically, the present invention provides a dual-array, single image sensor with different pitch interlaced pixels, especially used for super resolution (SR) of images. Advantageously, the method of the present invention satisfies the sampling rate diversity requirement in only one single sensor; compared to the need for two sensors to satisfy its different sampling rate requirement of conventional methods and systems.
- In embodiments, the sensor design interlaces two image sensor arrays within one sensor. The two interlaced arrays have a different pitch that will allow this dual array single sensor to function as a bi-sensor super-resolution camera. The array has pixels that are laid out in a modular (repeatable) block that will allow for typical image sensor array layout and which provides the capabilities of copy and paste operations to build a scalable pixel array sensor suitable for mega-pixel design. In this way, the present invention advantageously results in a very simplified sensor that cost the same as a single camera single image sensor setup and has no major sacrifice to silicon area.
- The sensor design of the present invention can be manufactured in a number of ways using a number of different tools. In general, though, the methodologies and tools are used to form structures with dimensions in the micrometer and nanometer scale. The methodologies, i.e., technologies, employed to manufacture the sensor design of the present invention have been adopted from integrated circuit (IC) technology. For example, the structures of the present invention are built on wafers and are realized in films of material patterned by photolithographic processes on the top of a wafer. In particular, the fabrication of the sensor design of the present invention uses three basic building blocks: (i) deposition of thin films of material on a substrate, (ii) applying a patterned mask on top of the films by photolithographic imaging, and (iii) etching the films selectively to the mask. The layout of the present invention is applicable to different technologies including, for example, CMOS 180 nm technology from Global Foundries as well as LF 150 nm technology, by KACST.
-
FIG. 1 shows two arrays, Array A and Array B which can be combined into a dual array single sensor to function as bi-sensor super-resolution camera or building blocks as shown inFIG. 2 . In the non-limiting illustrative example ofFIG. 1 , Array B is shown in a block of 4×4, while Array A is shown in a block of 5×5; although other block sizes are contemplated by the present invention. The pitch of Array A is labeled P1 and the pitch of Array B is labeled P2. - In embodiments, Array A has pixels (Pixel A) of a smaller area (smaller pitch in both dimensions horizontal and vertical) with a specific ratio when compared to Array B (that has Pixel B). In this example, Pixel B has a pitch that is 1.25 times that of Pixel A, therefore, in the example of
FIG. 1 , Pixel A has a pitch of 8 μm and Pixel B has a pitch of 10 μm, which is a ratio of 1.25. This case is only specific to the example ratio of 1.25, where other ratios may require different block sizes. It should be understood by those of skill in the art, that other pitches or different block sizes are thus contemplated by the present invention. - As further noted herein, it should be understood by those of skill in the art that the pixels contain a photodiode that it is usually smaller than the size of the pixel to leave room for the electronics required for pixel conditioning and control (e.g., such as the 3-transistor active pixel sensor design). The room for electronics is necessary in almost all CMOS image sensor design, except for the case of back-side illumination technology, where the electronics are integrated in a different layer than the photodiode. For these reasons, the present invention contemplates the need for different pitches for the different arrays in order to prevent blockage (e.g., crossover) of photosensitive areas as described herein when interlaced as shown in
FIG. 2 . - Also, in the example of
FIG. 1 , the pixels and photodiodes are assumed to be squares; although other shapes and sizes are contemplated by the present invention. Finally, the areas of the photodiodes are assumed to be equal (a1×a1=a2×a2) to maintain similar dynamic range, sensitivity and responsively characteristics between the two arrays. -
FIG. 2 represents both of the arrays (Array A and Array B) overlapped in an interlaced manner without crossing photosensitive areas in accordance with aspects of the present invention, e.g., dual array single sensor. The dual array single sensor is more specifically representative of a repeatable, scalable modular block which can be used for a dual array single sensor suitable for a mega-pixel design. - As should be understood, a cross in two or more photosensitive areas may result in a variation of the pixel pitch. In
FIG. 2 , for example, at pixel (0,0), the first pixel on the top left corner, both photodiodes overlap fully, meaning that only one photodiode laid out in that location can serve for both pixels. Also, some of the photodiodes may come in close proximity of each other; although, this is not an issue inFIG. 2 . However, it may be an issue in some layouts as there is a minimum well spacing between the photodiodes to avoid design rule check (DRC) violation. Accordingly, in implementing the present invention, a determination is made as to the minimum well spacing between the photodiodes to avoid design rule check (DRC) violation. - Using the infrastructure of
FIG. 6 , the condition to avoid overlap between square or rectangular photodiodes to result in the interlaced array ofFIG. 2 can be derived as follows: -
P2>P1 (1) -
P 2 =S×P 1 (2) -
s=scaling factor (in this case 1.25) (3) -
a=P 2 −P 1−min_spacing_rule (4). - It should be understood that “a” represents the size of the photodiode and a2 is thus representative of the area of the photodiode. It should be further understood that the present invention is not limited to a scaling factor of 1.25, and that other scaling factors can be implemented with the present invention, based on size of the pixels or arrays and other considerations described herein, e.g., avoidance of crossing any of the photosensitive areas (blockage of photosensitive areas when arrays are overlapped). For example, the scaling factor can be based on (i+1)/i, where i is representative of an array, e.g., a 4×4 array will result in (4+1)/4 which equals 1.25 scaling factor. In embodiments, an upscaling factor (e.g., “s”) can be based on a 1 megabyte pixel in, e.g., a 6×6 array (36 megabyte array), where the scaling factor can be, e.g., (6+1)/6 which equals 1.17.
- For the case shown in the 5×5 block of
FIG. 1 , with pixel pitch of 8 μm and 10 μm, and a minimum well spacing rule of 0.1 (with all units in micrometers), as an example: -
P 2=1.25×P 1 (5) -
a=1.9 (6) - The minimum well spacing rule is technology dependent and, as such, the minimum well spacing rule of 0.1 is provided as an illustrative example. The fill-factor (FF), which is a ratio of photosensitive part of the pixel to entire pixel area, can be estimated depending on the number of photosensitive parts of the pixel, which varies between 2 to 4, as follows:
-
2a2/P2 2<FF<4a2/P2 2 (7). - It should be understood that the interlaced
array 100 ofFIG. 2 can be used as a building block for a dual array single sensor. For example, the building block ofFIG. 2 can be used in copy and paste design processes known to those of skill in the art to complete a scalable array suitable for a mega-pixel design. For example, the present invention can be provided in design processes which preferably employ and incorporate hardware and/or software modules for synthesizing, translating, or otherwise processing a design/simulation functional equivalent of the components, circuits, devices, or logic structures shown inFIG. 2 to generate a netlist which may contain equivalent design structures of the array ofFIG. 2 . Netlist may comprise, for example, compiled or otherwise processed data structures representing a list of wires, discrete components, logic gates, control circuits, I/O devices, models, etc. that describes the connections to other elements and circuits in an integrated circuit design ofFIGS. 1 and 2 . Resulting design structures, representative of the arrays ofFIGS. 1 and 2 , can reside on a storage medium or programmable gate array in a data format used for the exchange of data of mechanical devices and structures (e.g. IGES, DXF, Parasolid XT, JT, DRG, GDSII (GDS2), GL1, OASIS, etc.). The design structure when processed by an ECAD system generates a logically or otherwise functionally equivalent form of one or more of the embodiments of the invention shown inFIG. 2 (andFIGS. 4, 6 and 8 ). -
FIGS. 3 and 4 provide alternative processes to provide an interlaced array in accordance with aspects of the present invention. InFIGS. 3 and 4 , though, super resolution can be performed using only one dimensional dual pitch, instead of two. That is,FIG. 3 shows a scaling in one dimension, e.g., horizontal (but not both horizontal and vertical). In this case, the FF can be increased significantly. Also in this case, the repeatable modular block will be smaller, as shown inFIG. 4 (compared toFIG. 2 ). - More specifically,
FIG. 3 shows two arrays, Array A and Array B which can be combined into a dual array single sensor to function as a bi-sensor super-resolution camera as shown inFIG. 4 . More specifically, the interlacedarray 100′ ofFIG. 4 is representative of a repeatable, scalable modular block which can be used to for a dual array single sensor suitable for a mega-pixel design. - In the non-limiting illustrative example of
FIG. 3 , Array B is shown in a block of 4×4, while Array A is shown in a block of 5×5. The pitch of Array A is P1 (horizontal) and P2 (vertical), whereas, the pitch of Array B is P2 for both horizontal and vertical. Hence, Array A has rectangular pixels and Array B has square pitches. In embodiments, Array A has pixels (Pixel A) of a smaller area (smaller pitch in the horizontal) with a specific ratio when compared to Array B (that has Pixel B). Finally, the areas of the photodiodes are assumed to be equal (a1×a1=a2×a2) to maintain similar dynamic range, sensitivity and responsively characteristics between the two arrays. - As shown in
FIG. 4 , using the equations above it is now possible to obtain a modular, repeatable, scalablemodular block 100′ of 5x2 primary pixels, as a minimum building block. As in the representation ofFIG. 2 , the modular, repeatable building block ofFIG. 4 can be used for a dual array single sensor suitable for a mega-pixel design. Although not as effective as the building block shown inFIG. 2 , the building block ofFIG. 4 will provide enhanced resolution. Also, it should be understood by those of skill in the art that an upscale factor need only be used in a single dimension in this implementation which, accordingly, will only increase the bit size in one dimension. Similar to that explained with regard toFIG. 2 , the building blocks ofFIG. 4 can be used as repeatable, scalablemodular blocks 100′ suitable for a mega-pixel design. For example, the building block ofFIG. 2 can be used in copy and paste design processes known to those of skill in the art to complete a dual array single sensor. - As shown in
FIGS. 5 and 6 , using a variable sampling rate for the primary data set (Array A) can help increase the FF. For example, inFIG. 5 , Array B has a pitch P2 (10 units) that is larger than Array A by a factor of 1.25. The photodiodes in this case are larger than the condition derived in equation 6, which results in some of the photodiodes in the 4×4 block overlapping. However, as shown inFIG. 6 , some of the photodiodes of the Array A are removed in order to avoid DRC errors or to avoid modifying the pitch of Array B. The result of the interlacedlayout 100″ is shown inFIG. 6 , where Array A samples are 36% less than the original case for Array A or 20% of both arrays together. This was tested and shown not to have any significant impact on the enhanced images, in that known software algorithms can extrapolate for the missing photodiodes, while still providing enhanced imagery. - As shown in
FIG. 7 , a variable sampling rate can also be implemented by removing pixels from secondary array (Array B), as shown inFIG. 8 . A shown in the interlacedlayout 100′″ ofFIG. 8 , Array B samples are 50% less than the original case, also only 20% less than both arrays together. This was also tested and shown not to have any significant impact on the enhanced images. It should be understood by those of skill in the art that any combination of pixels can be removed from the primary array (Array A) or the secondary array (Array B) in order to avoid crossover of the pixels, as described herein. - The present invention may be embodied as a system, method or computer program product. The present invention may take the form of a hardware embodiment, a software embodiment or a combination of software and hardware. Furthermore, the present invention may take the form of a computer program product (program instructions) embodied in any tangible storage medium of expression having computer-usable program code embodied in the medium, which implements the methods, techniques and processes herein. The computer readable storage media may be any medium that can contain, store, or communicate the program for use by or in connection with the instruction execution system, apparatus, or device. The computer readable storage media is not a signal per se, or transitory. The computer-usable or computer-readable medium may be, for example, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium.
-
FIG. 9 shows anillustrative environment 10 for managing the processes in accordance with the invention. Theenvironment 10 includes a server orother computing system 12 that can perform the processes described herein. Theserver 12 includes acomputing device 14 which can be resident on a network infrastructure or computing device. Thecomputing device 14 includes a processor (CPU) 20,memory 22A, an I/O interface 24, and abus 26. In addition, the computing device includes random access memory (RAM), a read-only memory (ROM), and an operating system (O/S). Thecomputing device 14 is in communication with the external I/O device/resource 28 and thestorage system 22B. The I/O device 28 can comprise any device that enables an individual to interact with the computing device 14 (e.g., user interface) or any device that enables thecomputing device 14 to communicate with one or more other computing devices using any type of communications link. Theprocessor 20 executes computer program code (e.g., program control or program instructions 44), which can be stored in thememory 22A and/orstorage system 22B. While executing the computer program code, theprocessor 20 can read and/or write data to/frommemory 22A,storage system 22B, and/or I/O interface 24. The program code executes the processes of the invention. - In embodiments, the present invention comprises the CPU, computer readable memory and computer readable storage media. In particular embodiments, program instructions are configured to select a first array of pixels comprising pixels of a first pitch. Program instructions are configured to select a second array of pixels comprising pixels of a second pitch which is different than the first pitch. Program instructions are configured to interlace the first array of pixels and the second array of pixels to form a repeatable modular block for an interlaced bi-sensor super-resolution pixel array while avoiding crossing of photosensitive areas of the first array of pixels and the second array of pixels. Program instructions are configured to provide the other methods, steps and techniques already described herein. The program instructions are stored on the computer readable storage media for execution by the CPU via the computer readable memory.
- The foregoing examples have been provided for the purpose of explanation and should not be construed as limiting the present invention. While the present invention has been described with reference to an exemplary embodiment, Changes may be made, within the purview of the appended claims, without departing from the scope and spirit of the present invention in its aspects. Also, although the present invention has been described herein with reference to particular materials and embodiments, the present invention is not intended to be limited to the particulars disclosed herein; rather, the present invention extends to all functionally equivalent structures, methods and uses, such as are within the scope of the appended claims.
Claims (9)
1. A method comprising: interlacing a first array of pixels of a first size with a second array of pixels of a second size, wherein a plurality of photodiodes of each of the first array of pixels and the second array of pixels cross over when interlaced, and wherein pixels of the second array of pixels are removed to implement a variable sampling rate.
2. The method of claim 1 , wherein the interlacing of the first array of pixels with the second array of pixels forms a repeatable modular block for an interlaced bi-sensor super-resolution pixel array.
3. The method of claim 2 , wherein the repeatable modular block is copied and pasted to form a scalable array for a mega-pixel design.
4. The method of claim 3 , wherein the first array of pixels and the second array of pixels have pixels of a different pitch.
5. The method of claim 1 , wherein the removing pixels of the second array of pixels avoids design rule check (DRC) errors.
6. The method of claim 5 , wherein the removing pixels of the second array of pixels avoids modifying the pitch of the second array of pixels.
7. The method of claim 1 , wherein the removing of the pixels of the second array of pixels is 50% less than an original case.
8. The method of claim 7 , wherein the removing of the pixels of the second array of pixels is 20% less than an original case of both the pixels of the first and second array of pixels.
9. The method of claim 7 , wherein any combination of pixels can be removed from the first array or the second array to avoid crossover of the pixels.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/341,547 US20170053381A1 (en) | 2014-07-17 | 2016-11-02 | Interlaced bi-sensor super-resolution enhancement |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/334,249 US9817935B2 (en) | 2014-07-17 | 2014-07-17 | Interlaced bi-sensor super-resolution enhancement |
US15/341,547 US20170053381A1 (en) | 2014-07-17 | 2016-11-02 | Interlaced bi-sensor super-resolution enhancement |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/334,249 Division US9817935B2 (en) | 2014-07-17 | 2014-07-17 | Interlaced bi-sensor super-resolution enhancement |
Publications (1)
Publication Number | Publication Date |
---|---|
US20170053381A1 true US20170053381A1 (en) | 2017-02-23 |
Family
ID=55074781
Family Applications (4)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/334,249 Active 2035-09-06 US9817935B2 (en) | 2014-07-17 | 2014-07-17 | Interlaced bi-sensor super-resolution enhancement |
US15/341,522 Active 2035-03-04 US10437955B2 (en) | 2014-07-17 | 2016-11-02 | Interlaced bi-sensor super-resolution enhancement |
US15/341,536 Active 2035-02-07 US10445454B2 (en) | 2014-07-17 | 2016-11-02 | Interlaced bi-sensor super-resolution enhancement |
US15/341,547 Abandoned US20170053381A1 (en) | 2014-07-17 | 2016-11-02 | Interlaced bi-sensor super-resolution enhancement |
Family Applications Before (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/334,249 Active 2035-09-06 US9817935B2 (en) | 2014-07-17 | 2014-07-17 | Interlaced bi-sensor super-resolution enhancement |
US15/341,522 Active 2035-03-04 US10437955B2 (en) | 2014-07-17 | 2016-11-02 | Interlaced bi-sensor super-resolution enhancement |
US15/341,536 Active 2035-02-07 US10445454B2 (en) | 2014-07-17 | 2016-11-02 | Interlaced bi-sensor super-resolution enhancement |
Country Status (1)
Country | Link |
---|---|
US (4) | US9817935B2 (en) |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105450924B (en) * | 2014-09-30 | 2019-04-12 | 北京智谷技术服务有限公司 | The acquisition methods and device of super-resolution image |
US9672594B2 (en) * | 2014-10-21 | 2017-06-06 | The Boeing Company | Multiple pixel pitch super resolution |
CN109033545B (en) * | 2018-07-03 | 2020-04-03 | 北京华大九天软件有限公司 | Method for quickly generating pixel layout according to definition of pixel array |
US11275967B2 (en) * | 2019-07-16 | 2022-03-15 | CICADA Imaging Inc. | Image upsampling by learning pairs of low-resolution dictionaries using a structured subspace model |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20150199792A1 (en) * | 2014-01-13 | 2015-07-16 | Arecont Vision, Llc. | System and method for obtaining super image resolution through optical image translation |
US20150312492A1 (en) * | 2014-04-24 | 2015-10-29 | Samsung Electronics Co., Ltd. | Image data processing device having image sensor with skewed pixel structure |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8054355B2 (en) * | 2008-10-16 | 2011-11-08 | Omnivision Technologies, Inc. | Image sensor having multiple sensing layers |
US8665342B2 (en) | 2011-03-03 | 2014-03-04 | King Abddulaziz City For Science And Technology | Model-independent generation of an enhanced resolution image from a number of low resolution images |
US8970706B2 (en) * | 2011-09-22 | 2015-03-03 | Basil Henry Scott | Dual pixel pitch imaging array with extended dynamic range |
-
2014
- 2014-07-17 US US14/334,249 patent/US9817935B2/en active Active
-
2016
- 2016-11-02 US US15/341,522 patent/US10437955B2/en active Active
- 2016-11-02 US US15/341,536 patent/US10445454B2/en active Active
- 2016-11-02 US US15/341,547 patent/US20170053381A1/en not_active Abandoned
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20150199792A1 (en) * | 2014-01-13 | 2015-07-16 | Arecont Vision, Llc. | System and method for obtaining super image resolution through optical image translation |
US20150312492A1 (en) * | 2014-04-24 | 2015-10-29 | Samsung Electronics Co., Ltd. | Image data processing device having image sensor with skewed pixel structure |
Also Published As
Publication number | Publication date |
---|---|
US20170068766A1 (en) | 2017-03-09 |
US20170069058A1 (en) | 2017-03-09 |
US10445454B2 (en) | 2019-10-15 |
US9817935B2 (en) | 2017-11-14 |
US20160019329A1 (en) | 2016-01-21 |
US10437955B2 (en) | 2019-10-08 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10437955B2 (en) | Interlaced bi-sensor super-resolution enhancement | |
US8861893B2 (en) | Enhancing video using super-resolution | |
CN108881750A (en) | Optical sensor and electronic equipment including light blocking pattern | |
US9673145B2 (en) | Semiconductor integrated circuit layout structure | |
CN110275606B (en) | Sensing element | |
US10241317B2 (en) | Image processing apparatus and imaging apparatus | |
JP2012506206A (en) | Image sensor having a plurality of sensor layers, and operation and manufacturing method thereof | |
US11024010B2 (en) | Super-resolution image sensor and producing method thereof | |
US20160282730A1 (en) | Semiconductor device, method of manufacturing the same, and pattern overlay inspection method | |
US11031432B2 (en) | Vertical microbolometer contact systems and methods | |
JP2018514969A (en) | Light field metadata | |
US9285510B2 (en) | Optical element array and solid-state imaging device including the array | |
CN115049548A (en) | Method and apparatus for restoring image obtained from array camera | |
Kobayashi et al. | High-definition and high-sensitivity CMOS image sensor with all-pixel image plane phase-difference detection autofocus | |
CN116487398A (en) | Image sensor and electronic device including the same | |
US11714214B2 (en) | Image sensor including color separating lens array and electronic apparatus including the image sensor | |
KR20230040182A (en) | Image sensor and electronic apparatus including the image sensor | |
CN114430467A (en) | Image sensor including color separation lens array and electronic device including the same | |
CN112822442A (en) | Heat map generation method and device and electronic equipment | |
US8661372B1 (en) | Optical proximity correction method | |
EP4345902A1 (en) | Image sensor and electronic apparatus including the same | |
US20160123803A1 (en) | Vector light sensor and array thereof | |
EP4120347A1 (en) | Optical sensor including planar nano-photonic microlens array and electronic apparatus including the same | |
EP4163978A1 (en) | Image sensor including color separating lens array and electronic apparatus including the image sensor | |
US20230176391A1 (en) | Image sensor including color separating lens array and electronic apparatus including the image sensor |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: KING ABDULAZIZ CITY FOR SCIENCE AND TECHNOLOGY, SA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ELDESOUKI, MUNIR;AL-SALEM, FAISAL MUHAMMED;REEL/FRAME:040200/0969 Effective date: 20140717 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |