US20200120315A1 - Combined visible and infrared image sensor incorporating selective infrared optical filter - Google Patents
Combined visible and infrared image sensor incorporating selective infrared optical filter Download PDFInfo
- Publication number
- US20200120315A1 US20200120315A1 US16/106,373 US201816106373A US2020120315A1 US 20200120315 A1 US20200120315 A1 US 20200120315A1 US 201816106373 A US201816106373 A US 201816106373A US 2020120315 A1 US2020120315 A1 US 2020120315A1
- Authority
- US
- United States
- Prior art keywords
- filter
- transmission
- vis
- optical filter
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000003287 optical effect Effects 0.000 title claims abstract description 133
- 230000005540 biological transmission Effects 0.000 claims abstract description 112
- 230000004438 eyesight Effects 0.000 description 24
- 238000003384 imaging method Methods 0.000 description 19
- 230000003595 spectral effect Effects 0.000 description 16
- 238000001228 spectrum Methods 0.000 description 11
- 238000010521 absorption reaction Methods 0.000 description 9
- 230000006872 improvement Effects 0.000 description 9
- 230000001066 destructive effect Effects 0.000 description 8
- 238000000034 method Methods 0.000 description 7
- 238000013461 design Methods 0.000 description 6
- 230000009977 dual effect Effects 0.000 description 6
- 238000002329 infrared spectrum Methods 0.000 description 5
- 239000000463 material Substances 0.000 description 5
- 239000011368 organic material Substances 0.000 description 5
- 101150026756 sir1 gene Proteins 0.000 description 5
- 238000001429 visible spectrum Methods 0.000 description 5
- 101000654471 Mus musculus NAD-dependent protein deacetylase sirtuin-1 Proteins 0.000 description 4
- 229910010272 inorganic material Inorganic materials 0.000 description 4
- 239000011147 inorganic material Substances 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 150000002894 organic compounds Chemical class 0.000 description 3
- 239000000758 substrate Substances 0.000 description 3
- 239000006059 cover glass Substances 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 238000001035 drying Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000004297 night vision Effects 0.000 description 2
- 238000000206 photolithography Methods 0.000 description 2
- 238000004528 spin coating Methods 0.000 description 2
- 239000012780 transparent material Substances 0.000 description 2
- VYPSYNLAJGMNEJ-UHFFFAOYSA-N Silicium dioxide Chemical compound O=[Si]=O VYPSYNLAJGMNEJ-UHFFFAOYSA-N 0.000 description 1
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 1
- GWEVSGVZZGPLCZ-UHFFFAOYSA-N Titan oxide Chemical compound O=[Ti]=O GWEVSGVZZGPLCZ-UHFFFAOYSA-N 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000009429 electrical wiring Methods 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000005693 optoelectronics Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 239000011347 resin Substances 0.000 description 1
- 229920005989 resin Polymers 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 229910052710 silicon Inorganic materials 0.000 description 1
- 239000010703 silicon Substances 0.000 description 1
- 229910052814 silicon oxide Inorganic materials 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- OGIDPMRJRNCKJF-UHFFFAOYSA-N titanium oxide Inorganic materials [Ti]=O OGIDPMRJRNCKJF-UHFFFAOYSA-N 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000003245 working effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
- H04N23/11—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths for generating image signals from visible and infrared light wavelengths
-
- H04N9/04553—
-
- H—ELECTRICITY
- H01—ELECTRIC ELEMENTS
- H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
- H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
- H01L27/14—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
- H01L27/144—Devices controlled by radiation
- H01L27/146—Imager structures
- H01L27/14601—Structural or functional details thereof
- H01L27/1462—Coatings
- H01L27/14621—Colour filter arrangements
-
- H—ELECTRICITY
- H01—ELECTRIC ELEMENTS
- H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
- H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
- H01L27/14—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
- H01L27/144—Devices controlled by radiation
- H01L27/146—Imager structures
- H01L27/14601—Structural or functional details thereof
- H01L27/14625—Optical elements or arrangements associated with the device
-
- H—ELECTRICITY
- H01—ELECTRIC ELEMENTS
- H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
- H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
- H01L27/14—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
- H01L27/144—Devices controlled by radiation
- H01L27/146—Imager structures
- H01L27/14601—Structural or functional details thereof
- H01L27/14625—Optical elements or arrangements associated with the device
- H01L27/14627—Microlenses
-
- H—ELECTRICITY
- H01—ELECTRIC ELEMENTS
- H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
- H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
- H01L27/14—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
- H01L27/144—Devices controlled by radiation
- H01L27/146—Imager structures
- H01L27/14643—Photodiode arrays; MOS imagers
- H01L27/14645—Colour imagers
-
- H—ELECTRICITY
- H01—ELECTRIC ELEMENTS
- H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
- H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
- H01L27/14—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
- H01L27/144—Devices controlled by radiation
- H01L27/146—Imager structures
- H01L27/14643—Photodiode arrays; MOS imagers
- H01L27/14649—Infrared imagers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
- H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics
- H04N25/13—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
- H04N25/131—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements including elements passing infrared wavelengths
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
- H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics
- H04N25/13—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
- H04N25/134—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements based on three different wavelength filter elements
-
- H04N9/04557—
Definitions
- This disclosure relates generally to image sensor systems that includes a hybrid image sensor pixel array of both visible (VIS) spectrum pixels and infrared (IR) spectrum pixels. More specifically, the image sensor system uses its VIS image sensing capability to produce a VIS color image, and uses its IR sensing capability to perform IR imaging functions.
- VIS visible
- IR infrared
- image sensors that feature security and machine vision applications, including gesture sensing, depth analysis, iris detection, eye tracking, night or low light vision, etc.
- these image sensors use traditional, VIS spectrum image sensing pixels to produce VIS images.
- these sensors also use additional, IR spectrum pixels to produce IR images. Basically, these sensors serve a dual purpose of producing both VIS and IR images.
- such a dual purpose image sensor employs a direct combination design wherein an IR sub-sensor is physically juxtaposed next to a VIS sub-sensor.
- the VIS sub-sensor only includes VIS imaging pixels, and the IR sub-sensor only includes IR imaging pixels.
- This design offers simplicity, but the VIS and IR sub-sensors inevitably have different vantage points with regard to an imaging target.
- the resulting VIS and IR images not only have different optical spectra (VIS versus IR), but are also not able to be completely superimposed onto each other. This creates difficulties for subsequent image processing.
- FIG. 1A is schematic drawing showing an exemplary embodiment of a hybrid VIS-IR image sensor system.
- FIG. 1B is top view of an exemplary VIS-IR image sensor pixel array, showing a minimal repeating unit containing individual pixels.
- FIG. 2A is a wavelength versus transmission plot that shows a characteristic optical transmission curve of a VIS-IR bandpass filter.
- FIG. 2B is a wavelength versus transmission plot that shows a set of characteristic optical transmission curves of the pixels within a VIS-IR pixel array.
- FIG. 2C is another wavelength versus transmission plot that shows another set of characteristic optical transmission curves of the pixels within another VIS-IR pixel array.
- FIG. 3A is a side-view drawing of part of an improved VIS-IR pixel array.
- FIG. 3B is another side-view drawing of part of another improved VIS-IR pixel array.
- FIG. 4A is a wavelength versus transmission plot that shows a characteristic optical transmission curve of a selective IR filter.
- FIG. 4B is another wavelength versus transmission plot that shows two characteristic optical transmission curves of a selective IR filter before and after improvement.
- FIG. 5 is schematic drawing showing an exemplary embodiment of a hybrid VIS-IR image sensor system that includes a selective IR filter.
- FIG. 6 shows an exemplary method that includes a number of steps to perform a VIS vision imaging mode and an IR vision imaging mode.
- FIG. 7A shows an embodiment of a VIS-IR pixel array.
- FIG. 7B shows an alternative embodiment of the VIS-IR pixel array of FIG. 7A .
- FIG. 8A shows another embodiment of a VIS-IR pixel array.
- FIG. 8B shows an alternative embodiment of the VIS-IR pixel array of FIG. 8A .
- FIG. 9 is a wavelength versus transmission plot that shows two characteristic optical transmission curves of selected IR filters or suppressed IR (SIR) filters.
- example or “embodiment” means that a particular feature, structure, or characteristic described in connection with the example is included in at least one example of the present invention.
- appearances of “example” or “embodiment” in various places throughout this specification are not necessarily all referring to the same example.
- particular features, structures, or characteristics may be combined in any suitable manner in one or more examples.
- FIG. 1A is a schematic drawing of an embodiment of a hybrid VIS-IR image sensor system 100 , which includes a main lens 110 , a VIS-IR bandpass filter module 120 , a VIS-IR image sensor pixel array 130 , a controller module 140 , a function logic module 150 , and an IR light source module 160 . Features and functions of each element are disclosed further herein.
- the main lens 110 focuses incoming light to pass through the VIS-IR bandpass filter module 120 , onto the underlying VIS-IR image sensor pixel array 130 .
- the main lens 110 is optically transparent, and allows all VIS-IR spectrum light to pass through. In other words, the main lens 110 does not have an IR attenuation (IR cut) function.
- the VIS-IR bandpass filter module 120 includes a VIS-IR bandpass filter (or filters) 121 that is attached to either the front side and/or the backside of a cover glass substrate 122 .
- the cover glass substrate 122 provides a mechanical support of the VIS-IR bandpass filter 121 , and is preferred to be optically transparent to allow the passage of the full spectrum of both VIS and IR light.
- the VIS-IR bandpass filter 121 is preferably a multi-layer structure consisting of multiple, alternating layers of transparent inorganic materials (e.g., silicon oxide and titanium oxide), and relies on the principle of destructive interference to stop certain spectral bands of incoming light, while allowing other spectral bands to pass through.
- the VIS-IR filter 121 as illustrated in FIG. 1A allows a VIS band and an IR band to pass through, while stopping the other wavelengths. This is shown in FIG. 2A , and is further elaborated below.
- the transmission curve 250 includes a VIS passband 260 of approximately 400-650 nanometer (nm) wavelength in the VIS range.
- the shape of the VIS passband 260 has the general form of a rectangle (or close to a trapezoid with two steep sides, i.e., the two base angles of the trapezoid are slightly under 90 degrees), with a height of almost 100% transmission, and it has almost no leakage outside the passband 260 .
- the VIS passband 260 is close to the ideal passband for the situation of passing a wavelength range.
- This near ideal passband characteristic is achieved through the use of destructive interference produced by the multi-layer transparent inorganic materials. It is appreciated that such a near-rectangular, no-leak passband characteristic is different from a bell-curve-shaped and leaky passband characteristic that results from an absorption by organic compounds, as shown in FIG. 2B , a point that will be further elaborated later in this disclosure.
- the wavelength-transmission curve 250 also includes an IR passband 270 of approximately 800-900 nm wavelength in the IR range.
- the shape of the IR passband 270 has the form of a steep triangle (whose two base angles are slightly under 90 degrees), which centers around 850 nm, and it has almost no leakage outside the IR passband 270 , i.e., the IR passband 270 is close to the ideal passband for the situation of passing a singular wavelength (at around 850 nm).
- This near ideal passband characteristic is again achieved through the use of destructive interference produced by multi-layer transparent inorganic materials.
- the characteristic curve 250 in FIG. 2A shows two passbands—a VIS passband 260 and an IR passband 270 .
- the VIS-IR filter 121 may be referred to as a dual bandpass filter, which functions to pass incoming light at a broad VIS range (400-650 nm) and a narrow IR range (850 ⁇ 50 nm, or 800-900 nm).
- the IR range may be even narrower, e.g., 850 ⁇ 30 nm, 850 ⁇ 10 nm, and so on. Other wavelengths are stopped by this VIS-IR filter 121 .
- the aforementioned passband values that are achieved by destructive interference produced by multi-layer transparent inorganic materials is dependent on the chief ray angle (CRA) of the incident light. This is due to the nature of destructive interference, which involves light reflecting off the interfaces of the multi-layer transparent materials.
- CRA chief ray angle
- the narrow IR passband is about 850 ⁇ 50 nm, i.e., 800-900 nm, as previously disclosed.
- the VIS-IR image sensor pixel array 130 is situated below the VIS-IR bandpass filter module 120 .
- the pixel array 130 comprises a multitude of VIS and IR pixels, wherein a group of VIS and IR pixels form a multiple repeating unit 135 , which is shown in FIG. 1B as part of a top view of the pixel array 130 .
- a minimal repeating unit is defined as the smallest group of pixels that make up an entire pixel array. According to FIG.
- the minimal repeating unit 135 of pixel array 130 consists of a green (G) pixel 131 , a red (R) pixel 132 , a blue (B) pixel 133 , and an IR pixel 134 , arranged in a square-shaped, 2 ⁇ 2 pattern.
- FIG. 1A shows a cross-sectional side view of (a representative portion of) the pixel array 130 , which comprises the four individual pixels 131 through 134 .
- the pixel array 130 includes many more individual pixels that may have different arrangements. All four pixels 131 , 132 , 133 , and 134 share similar features, and are disclosed herein by describing the green (G) pixel 131 as a representative pixel.
- the G pixel 131 includes a microlens 131 a, a G optical filter 131 b, and an image sensing member 131 c.
- the microlens 131 a may be made of a resin material, and primarily functions to concentrate incoming light into the pixel 131 itself
- the G optical filter 131 b contains organic material, and relies on chemically based optical absorption principle to pass a particular wavelength band of light while stopping other wavelengths. In this example, the G optical filter 131 b passes a green wavelength band of about 450-650 nm.
- FIG. 2B shows a wavelength-versus-transmission set of curves 200 that includes a G bandpass characteristic curve 220 , which corresponds to the G optical filter 131 b.
- the G curve 220 includes a main color G bandpass range of about 450-650 nm, but also has significant leakage ranges outside the 450-650 nm main passband, for example, in the 300-450 nm range, and also in the near-IR and IR range of 700-1100 nm.
- the leakage range is due to the nature of the chemically based optical absorption.
- Within the 450-650 nm main color G passband the appearance of the curve is similar to a partially skewed and non-smooth bell-curve shape.
- This non-ideal passband shape is also due to the chemically based optical absorption. These characteristics are different from the near ideal characteristics of the VIS-IR bandpass filter 121 as disclosed in FIG. 2A and its related description. It is appreciated that whereas a multi-layer optical filter that relies on destructive interference is applicable for large size (millimeter scale and above) optical filters such as filter 121 , this type of optical filter design is generally not feasible to make the optical filter for a small image sensor pixel (micrometer scale or less), such as pixel 131 .
- an organic compound filter that relies on chemically based optical absorption is more suitable for small size optical filters such as filter 131 b, because the requisite manufacturing process (e.g., photolithography, spin coating, and drying) can be readily applied to make small, micrometer (or sub-micron) size filters.
- the G pixel 131 includes the image sensing member 131 c directly below the G optical filter 131 b.
- the image sensing member 131 c may be of a CMOS (Complementary Metal Oxide Semiconductor) or CCD (Charge Coupled Device, similar to CMOS, but with minor variations) configuration.
- CMOS Complementary Metal Oxide Semiconductor
- CCD Charge Coupled Device, similar to CMOS, but with minor variations
- the sensing member 131 c is made with a silicon substrate (not labeled), into which a photodiode (not shown) is formed.
- Other parts such as a photodiode pinning layer (not shown), a well structure (not shown) around the photodiode, and isolation trenches (not shown) may also be present.
- pixels such as the R pixel 132 , the blue pixel 133 , and the IR pixel 134 each include their requisite microlenses ( 132 a, 133 a, and 134 a ), optical filters ( 132 b, 133 b, and 134 b ), and image sensing members ( 132 c, 133 c, and 134 c ).
- the optical filters 131 b, 132 b, 133 b, and 134 b each has its own specific characteristic transmission curve, i.e., a G transmission curve 220 , an R transmission curve 230 , a B transmission curve 210 , and an IR transmission 240 , respectively, as shown in FIG. 2B .
- These optical filters rely on chemically based optical absorption, and their characteristic curves have leakage ranges beyond their main spectral passbands, as well as having skewed and non-smooth bell-curve shapes within their main spectral passbands.
- the primary passband for the G transmission curve 220 is around 450-650 nm (with about 80% peak transmission); the primary passband for the R transmission curve 230 is around 650-850 nm (with about 50% peak transmission); the primary passband for the B transmission curve 210 is around 380-450 nm (with about 70% peak transmission); and the primary passband for the IR transmission curve 240 is around 800-1000 nm (with about 20% peak transmission), wherein the primary IR passband overlaps with the primary passband of the R transmission curve 230 at around 800-850 nm.
- each of the three VIS transmission curves contains one or several leaky passbands outside its respective main spectral passband, e.g., a leaky band being situated around the near-IR and IR range of 700-1100 nm, with a peak transmission value of around 20%.
- each VIS pixel R, G, and B is capable of detecting IR light to some degree.
- FIG. 2C shows an alternative wavelength-versus-transmission set of curves 250 .
- This set of curves 250 are similar to the set of curves 200 in FIG. 2B , except for the IR curve 260 that has a much higher transmission level than the previous IR curve 240 found in FIG. 2B .
- Both IR curves 240 and 260 have their main spectral passband situated around 800-1000 nm, but curve 260 has a peak transmission level of around 50 %, which is much higher than the 20% of curve 240 .
- Controller module 140 may be physically and electrically coupled with the VIS-IR image sensor pixel array 130 , as shown in FIG. 1A , and operates the hybrid VIS-IR image sensor system to select among a number of operating modes.
- these modes include (1) a VIS vision mode, which is generally appropriate for human vision; and (2) an IR vision mode, which is generally appropriate for security vision, machine vision and night vision.
- the controller module 140 includes a VIS vision mode control sub-module 141 that controls the VIS vision mode, and an IR vision mode control sub-module 142 that controls the IR vision mode.
- the VIS control sub-module 141 selectively controls the VIS pixels (e.g., pixels 131 , 132 and 133 ) of the pixel array 130 , as shown by the dotted lines connecting pixels 131 , 132 , and 133 with the VIS control sub-module 141 .
- the IR control sub-module 142 selectively controls the IR pixels (e.g., pixel 134 ) of the pixel array 130 , as shown by a dotted line connecting pixel 134 with the IR control sub-module 142 . It is appreciated that the aforementioned control of the pixel array 130 by the controller module 140 is optional.
- Function logic module 150 receives image signals (not shown) that are read out from the pixel array 130 , and processes these signals into images, e.g., VIS and IR images.
- VIS pixel signals are used to produce VIS images
- IR signals are used to produce IR images.
- Image processing is conventionally performed by an image signal processor (ISP, not shown) as part of the function logic module 150 .
- ISP image signal processor
- the IR light source module 160 operates to illuminate an imaging object with IR spectral light.
- the IR light source module 160 is an IR light emitting diode (LED) that emits a relative narrow spectral range of IR light at around 850 nm, with a tolerance of around plus and minus 50 nm, 30 nm, 10 nm, or even less.
- the emitted IR light of 850 ⁇ 50 nm may be continuous, or in rapid, intense pulses (e.g., MHz pulsing frequency, with each pulse period at nanosecond scale).
- the IR control sub-module 142 controls the IR light source module 160 to emit pulses of IR light at about 850 nm.
- the IR control sub-module 142 is engaged to control the IR light source module 160 to emit rapid pulses of intense IR light of about 850 ⁇ 50 nm (tolerance may be less than 50 nm, e.g., 30 nm, 10 nm, etc.) to illuminate an imaging subject.
- This illuminating IR light has sufficient power to overwhelm naturally occurring ambient light.
- This operation scheme is effective in both day and night, but is especially effective during the night (or in a foggy environment) when there is low ambient natural light.
- the light reflected off the imaging subject contains a high percentage of IR spectral component of about 850 ⁇ 50 nm (tolerance may be less than 50 nm), and other less significant spectral components (such as VIS spectra) are regarded as noise.
- this reflected light is essentially an IR light of about 850 ⁇ 50 nm (tolerance may be less than 50 nm).
- the aforementioned reflected IR light passes through the main lens 110 without any significant optical filtration.
- the narrow 850 ⁇ 50 nm tolerance may be less than 50 nm
- IR passband 270 shown in FIG.
- the VIS-IR bandpass filter 120 has a narrow IR passband whose center coincides with the center of the spectral range of the IR light that is emitted by the light source 160 , e.g., a 850 nm center. It is also appreciated that the IR passband 270 as shown in FIG. 2A is wide enough (i.e., has enough tolerance around its center) to permit most of the IR light reflecting off the imaging subject to pass through. It is further appreciated that the IR passband 270 is not exceedingly wide to permit too much noise to pass through.
- the light source 160 emits IR light with a spectral range of about 850 ⁇ 40 nm, and the VIS-IR bandpass filter 120 has a corresponding IR passband of about 850 ⁇ 50 nm.
- the pixel array 130 In the IR vision mode, the pixel array 130 essentially functions as an IR sensor array that relies on its IR pixels 134 to produce IR signals.
- the VIS pixels 131 , 132 and 133 are not engaged to operate in this IR vision mode, and only the IR pixels 134 are controlled by the IR control sub-module 142 to engage in operation. See FIG. 1A for control lines connecting the control module 140 to the pixel array 130 .
- the VIS control sub-module 141 disengages the VIS pixels 131 , 132 , and 133 .
- both VIS and IR pixels are engaged to operate, but due to the high percentage of IR component of the light that falls on the pixel array 130 , the signal output is primarily IR signals.
- the pixel array 130 functions essentially as an IR sensor. IR machine vision mode may be useful in applications such as distance measurement, movement tracking, etc.
- the control module 140 may be engaged to suppress the IR light source module 160 so that it does not emit IR light.
- the VIS-IR image sensor system 100 now depends on natural lighting to illuminate the imaging subject.
- Incoming natural light (not shown in FIG. 1A ) generally contains all VIS spectrum light, as well as some naturally occurring IR spectrum light.
- the incoming natural light passes through the main lens 110 without any significant optical filtration.
- the broad 400-650 nm VIS passband 260 shown in FIG.
- the VIS light in the main spectral range 400-650 nm to pass through while the narrow 850 ⁇ 50 nm IR passband 270 (shown in FIG. 2B ) allows only a small amount of naturally occurring IR light that is in the narrow spectral range of 850 ⁇ 50 nm to pass through.
- the pixel array 130 functions as a regular RGB sensor array to produce VIS image signals (R, G, and B signals).
- the VIS control sub-module 141 engages the VIS pixels 131 , 132 , and 133 to operate to produce VIS imaging signals, while the IR control sub-module 142 disengages the IR pixels 134 from operating.
- all VIS and IR pixels 131 , 132 , 133 , and 134 are allowed to operate, but the output from the IR pixels 134 is not directly used to form a final color image.
- the VIS pixels 131 , 132 , and 133 produce RGB signals, wherein the functional logic module 150 uses these RGB signals to produce a final color image after regular image processing steps such as interpolation.
- the IR component within the filtered incoming light does affect the RGB signals to some degree.
- the R transmission curve 230 of the R color filter 132 b, the G transmission curve 220 of the G color filter 131 b, and the B transmission curve 210 of the B color filter 133 b each contain a leaky passband in the IR range of about 700-1100 nm, which is wider than the 850 ⁇ 50 nm passband of the overlying VIS-IR dual passband filter module. Therefore, the RGB signals that are produced by the VIS pixels 131 , 132 , and 133 each contain some IR components (at 850 ⁇ 50 nm) in them. These are regarded as IR noise.
- Advanced signal processing may be used to remove these IR components, but this removal may be effective only in some circumstances.
- the IR pixel 134 may be set to operate at the same time as the VIS pixels 131 , 132 , and 133 .
- the IR signal produced by the IR pixel 134 may then be used as a corrective baseline to subtract the IR components (at 850 ⁇ 50 nm) from the VIS signals produced by the VIS pixels 131 , 132 , and 133 , such than the VIS signals may be accurately corrected to remove IR noise.
- IR noise is not possible.
- the optical filters 131 b, 132 b, 133 b, and 134 b have the transmission characteristics as shown in FIG. 2C , wherein the IR transmission curve 260 is much more pronounced than (or otherwise different from) the IR curve 240 in FIG. 2B , the aforementioned IR correction scheme will become inaccurate and unreliable.
- the IR signal produced by the IR pixel 134 is much different from (e.g., much higher than) the IR components within the RGB signals that are produced by VIS pixels 131 , 132 , and 133 .
- the IR noise adversely affects the B pixel 133 and the G pixel 131 more than it affects the R pixel 132 .
- the B and G spectra are further away from IR spectrum than the R spectrum; whereas there is a natural overlap between the R and IR spectra.
- the culprit IR noise refers to the IR component that passes through the IR passband 270 of the VIS-IR bandpass filter 120 , as shown in FIGS. 1A and 2A .
- An exemplary IR noise occurs at 850 ⁇ 50 nm (at zero degrees CRA).
- Another exemplary IR noise occurs at 825 ⁇ 45 nm (at 30 degree CRA).
- FIG. 3A shows the design of such an improvement.
- An improved VIS-IR image sensor pixel array 300 is an improved version over the original VIS-IR pixel array 130 as shown in FIG. 1A .
- the improved pixel array 300 has the same components as the original pixel array 130 , including the microlenses 131 a through 134 a, the optical filters 131 b through 134 b, and the image sensing members 131 c through 134 c.
- the improved pixel array 300 includes a selective IR cut filter 310 , which is superimposed with, i.e., either overlying above or underlying below (e.g., overlying above) the VIS optical filters 131 b, 132 b, and 133 b.
- the selective IR filter 310 may be a single unit of filter that overlies above a number of VIS optical filters (e.g., some combination of 131 b, 132 b, and 133 b ), or in the alternative as shown in FIG. 3B , it may be a single filter that overlies above a single VIS filter.
- each VIS optical filter has its own corresponding selective IR filter 310 , either overlying above or underlying below it.
- the improved pixel array 300 includes an optional IR pass filter 320 , which is superimposed with, i.e., either overlying above or underlying below (e.g., overlying above) the IR optical filter 134 b.
- the features of the selective IR cut filter 310 and the optional IR pass filter 320 are disclosed herein.
- the selective IR filter 310 is a notched filter, also known as a bandstop filter, which stops light transmission within a certain stopband (e.g., 850 ⁇ 50 nm), while allowing the rest of the spectrum to pass through it.
- the selective IR filter 310 contains organic material, which relies on chemically based optical absorption principle to stop a particular wavelength band of light while passing other wavelengths.
- the term “selective” indicates that the selective IR filter 310 selectively stops IR with a relatively narrow stop band, e.g., around 850 ⁇ 50 nm.
- FIG. 4A shows a characteristic light transmission curve 400 of this selective IR filter 310 .
- the shape of the stopband 405 is not ideal, and has an appearance that is similar to a partially skewed and non-smooth inverted bell-curve shape. It is appreciated that whereas a multi-layer optical filter based on destructive interference is applicable for large size (millimeter and above) optical filters such as filter 121 (shown in FIG. 1A ), such a filter design is generally not feasible to make the optical filter for small image sensor pixels (micrometer scale or below), such as the pixels of the pixel array 300 in FIG. 3A .
- an organic compound filter that relies on chemically based optical absorption is more suitable for such a situation, because the requisite manufacturing process (e.g., photolithography, spin coating, and drying) can be readily applied to make small, micrometer (or sub-micron) size optical filters.
- the requisite manufacturing process e.g., photolithography, spin coating, and drying
- the characteristic transmission curve 400 has a region of stopband 405 , which is located at about 550-950 nm. Outside this stopband 405 region, light transmission is relatively high (e.g., 90-100%). Within this stopband 405 , light transmission is significantly reduced. Further, at about 850 nm, light transmission is completely stopped. The overall effectiveness of the stopband 405 depends on its bandwidth.
- the stopband 405 starts at approximately 550 nm, and ends at approximately 950 nm.
- this characterization of stopband (“apparent stopband”) may not reflect the true effectiveness of the stopband 405 , because within the stopband 405 , the reduction of light transmission is not uniform.
- a half-power assessment appears to be more accurate in quantifying the effectiveness of the stopband 405 .
- W hp shown in FIG. 4A
- about half the incoming optical signal energy is stopped by an optical filter. This corresponds to approximately 30% transmission level of the transmission curve 400 . From FIG.
- the half-power bandwidth W hp may be regarded as the effective bandwidth of the stopband 405 , wherein W hp is in the range of 810-880 nm (about 70 nm in width), with 850 nm being the center point at which all light is stopped. Since the stopband 405 is skewed to the high wavelength side, a more accurate way to depict the effective bandwidth W hp of the stopband 405 is to use a three-number notation, 810-850-880 nm, wherein the three numbers each denotes the left end, the center, and the right end of the effective bandwidth region of this stopband.
- FIGS. 3A and 3B that show the selective IR filter 310 overlying directly above the VIS optical filters 131 b, 132 b, and 133 b
- an exemplary incoming natural light with VIS spectrum along with IR noise at 850 ⁇ 50 nm first reaches the selective IR filter 310 .
- the stopband 405 as depicted in FIG. 4A has an effective bandwidth of 810-850-880 nm, which essentially overlaps with the IR noise range of 800-850-900 nm (alternative notation of 850 ⁇ 50 nm, indicating left end, center, and right end). In particular, the two centers coincide at 850 nm. In other words, the most effective part of the stopband meets the highest IR noise point.
- the left (800 nm) and right (900 nm) ends of the IR noise range fall only slightly outside the effective bandwidth (810-880 nm) of the stopband 405 . Therefore, the selective IR filter 310 appears effective in essentially eliminating the IR noise that will cause a reddish hue in the final color image during the VIS human vision mode.
- the IR pass filter 320 in FIGS. 3A and 3B it is appreciated that this is an optional element. Since the selective IR filter 310 has been added to the VIS filters 131 b, 132 b, and 133 b, but not to the IR filter 134 b, some planarization is needed for the IR filter 134 b to increase overall thickness.
- the IR pass filter 320 provides such a planarization function.
- the IR pass filter 320 may be situated either directly above or below the IR filter 134 b.
- the IR pass filter contains organic material that is suitable for small image sensor pixels (micrometer scale or below).
- the IR pass filter 320 allows a wide range of IR to pass. Its IR passing range preferably covers the entire IR passing range of its adjoining IR filter 134 b, which is depicted to have the characteristic transmission curve 240 in FIG. 2B , or curve 260 in FIG. 2C .
- the IR passing function of filter 320 may be implemented in a number of ways. First, the IR pass filter 320 may be transparent enough to pass all spectra of VIS and IR (thus making filter 320 an “all-pass” filter, which is essentially a non-filter). Second, filter 320 may be a low-pass filter to only pass the longer wavelength spectrum in the IR range, e.g., approximately 700 nm and above.
- the existing IR filter 134 b may be simply made thicker (e.g., with additional thickness essentially equal to the thickness of the selective IR filter 310 ) to achieve planarization with the neighboring VIS filters 131 b, 132 b, and 133 b.
- Other effective options may exist. Different organic material may be appropriately chosen for each option above, as a person having ordinary skill in the art may recognize.
- FIG. 5 is a schematic drawing of an improved hybrid VIS-IR image sensor system 500 . It is largely similar to the VIS-IR system 100 as shown in FIG. 1A , but includes the selective IR filter 310 and the optional IR pass filter 320 within the improved pixel array 300 . Other components remain the same as the original system 100 .
- the presence of the selective IR filter 310 allows the improved hybrid VIS-IR system 500 to reduce IR noise in its final VIS images while engaging in the VIS vision mode. Its IR vision mode remains the same as in the original system 100 .
- the VIS-IR filter module 120 is made of multi-layer transparent materials, which uses destructive interference to produce its passbands. As a result, its passbands may shift, depending on the chief ray angle (CRA) of the incoming light.
- CRA chief ray angle
- the IR passband 270 as shown in FIG. 2A is about 850 ⁇ 50 nm, i.e., 800-850-900 nm.
- the IR passband 270 now becomes 825+45 nm, or 780-825-870 nm. This passband dependency on CRA allows IR noise to shift with CRA, thus rendering the selective IR filter less effective at some CRA values than other CRA values, as further explained below.
- the selective IR filter 310 has an effective stop bandwidth of 810-850-880 nm, which works well for a zero degree CRA incoming IR noise at 800-850-900 nm.
- the IR passband 270 of the VIS-IR filter 120 shifts to 825 ⁇ 45 nm, or 780-825-870 nm, hence the incoming IR noise now shifts to a range of 780-825-870 nm.
- the present selective IR filter will not be very effective to deal with this shifted IR noise.
- the selective IR filter stop bandwidth center of 850 nm no longer coincides with the shifted IR noise center of 825 nm.
- the left end (780 nm) of the shifted IR noise now falls well outside the left end (810 nm) of the selective IR filter stop bandwidth. This means that at a 30 degree CRA, the current selective IR filter 310 will lose some effectiveness to reduce IR noise.
- FIG. 4B illustrates an embodiment of a second improvement to the hybrid VIS-IR system (which is a first improvement to the selective IR filter 310 ).
- FIG. 4B shows the original characteristic transmission curve 400 of the selective IR filter 310 before improvement, which is the same as the transmission curve 400 shown in FIG. 4A .
- An improved characteristic transmission curve 410 is that of the selective IR filter 310 after improvement.
- Improved curve 410 solid curve
- the improved curve 410 has an effective stop bandwidth of 720-850-880 nm. This widening of effective bandwidth is achieved by modifying the organic material within the selective IR filter 310 .
- the improved stop bandwidth of 720-850-880 nm is more effective than the pre-improvement bandwidth of 810-850-880 nm to filter out incoming IR noise.
- the following table compares the two bandwidths quantitatively.
- the incoming IR light noise has a range of 800-850-900 nm.
- the improved selective IR filter stopband of 720-850-880 nm essentially catches all the IR noise range.
- the stopband center coincides with the noise range center at 850 nm.
- the left end (800 nm) of the noise range falls fully within the left end (720 nm) of the improved stopband.
- the incoming IR light noise has a shifted range of 780-825-870 nm, and the left end (780 nm) of this shifted noise range also falls fully within the left end (720 nm) of the improved stopband.
- This improved stopband of 720-850-880 nm is markedly more advantageous over the previous selective IR filter stopband of 810-850-880 nm.
- the pre-existing selective IR filter 310 is improved by widening its effective IR stopband W hp , e.g., from previously 810-850-880 nm to the wider range of 720-850-880 nm. This allows for an improved selective IR filter performance to better handle incident IR noise coming in at different directions, e.g., from zero to 30 degree CRA.
- the improved selective IR filter 310 is similarly implemented in the VIS-IR image sensor system 500 , as shown in FIG. 5 . Previous disclosure still applies, with the exception that the improved selective IR filter 310 now has a wider effective bandwidth (e.g., 720-850-880 nm), and this secondly improved system 500 can better handle incoming IR noise at various CRA angles (e.g., from zero to 30 degrees).
- a wider effective bandwidth e.g., 720-850-880 nm
- this secondly improved system 500 can better handle incoming IR noise at various CRA angles (e.g., from zero to 30 degrees).
- FIG. 6 shows an exemplary method 600 , which includes a number of steps. The order of some of the steps need not be chronologically arranged.
- a selection is made to engage either a VIS vision mode or an IR vision mode.
- Step 610 may be executed by the controller module 140 in FIG. 1A to select either the VIS mode 141 or the IR mode 142 . If VIS mode 141 is selected to be engaged, as shown in step 620 , then in a following step 621 , IR lighting of an imaging subject is suppressed. This may be accomplished by keeping the IR light source 160 in FIG. 1A from being switched on. Natural lighting is used to illuminate the imaging subject for this VIS mode.
- the natural light reflecting off the imaging subject may include both VIS and IR spectral components.
- the reflected natural light is filtered to allow specific VIS and IR bands to pass.
- the VIS-IR filter 120 allows the broad VIS band 260 and the narrow IR band 270 to pass through VIS and IR components respectively, as shown in FIGS. 1A and 2A .
- Step 623 is optional, wherein the passed-through IR band 270 is stopped from reaching the underlying pixel array 130 , particularly its VIS pixels 131 , 132 , and 133 .
- the selective IR filter 310 has a stopband 405 (see FIG. 4A ) that stops the IR noise that has passed through IR band 270 .
- the leftover VIS component of the reflected natural light is used to form a final VIS image that is relatively free of IR noise influence.
- IR lighting is used in step 631 to illuminate the imaging subject.
- the IR light source 160 in FIG. 1A may be switched on to emit rapid pulses of IR light towards the imaging subject. Due to the relatively high power of the IR pulses, the reflected light has a significant IR component, and the VIS spectrum may be regarded as low level noise.
- the IR mode 142 applies in both day and night, and works particularly well at low light conditions, including dark or foggy environments.
- the IR component is allowed to pass through to reach the underlying pixel array.
- the VIS-IR filter 120 see FIG. 1A
- Step 633 is optional, wherein the passed-through IR light is allowed to further propagate to reach the sensing region of the underlying pixel array.
- the IR pass filter 320 allows IR component to reach the imaging member 134 c of the IR pixel 134 (see FIG. 3A and FIG. 5 ).
- the IR component is used to perform IR vision functions, such as distance measurement, object tracking, low light imaging, etc.
- FIG. 7A shows an embodiment of the present invention similar to FIGS. 3A and 3B .
- a VIS-IR pixel array 700 comprises a green (G) pixel 731 , a red (R) pixel 732 , a blue (B) pixel 733 , and an IR pixel 734 .
- G pixel 731 , R pixel 732 , B pixel 733 , and IR pixel 734 are shown in a linear arrangement in FIG. 7A , they may be arranged in a square-shaped, 2 ⁇ 2 pattern, as shown in FIG. 1B .
- Pixel array 700 has the same components as the original pixel array 130 of FIG. 1A , including microlenses 731 a through 734 a (first through fourth microlens), optical filters 731 b through 734 b, and image sensing members 731 c through 734 c (first through fourth image sensing member).
- pixel array 700 includes special filters 731 d through 734 d.
- Special filters 731 d through 734 d include an IR filter similar to IR optical filter 134 b, a clear filter (C), and an SIR (suppressed IR) filter similar to selective IR filter 310 .
- a clear filter (C) passes both visible light and IR light.
- FIG. 7A shows optical filters 731 b - 734 b overlying above special filters 731 d - 734 d, optical filters 731 b - 734 d may be underlying below special filters 731 d - 734 d.
- optical filters 731 b (first G optical filter) and 734 b (second G optical filter) are similar to G optical filter 131 b, optical filter 732 b is R optical filter similar to R optical filter 132 b, and optical filters 733 b is B optical filter similar to B optical filter 133 b.
- Special filters 731 d is a first clear filter (C).
- Special filter 732 d (first special filter) is a second clear filter (C).
- First and second clear filter pass both visible light and IR light.
- Special filter 733 d (second special filter) is a SIR filter similar to selective IR filter 310 .
- Special filter 734 d is an IR optical filter similar to IR optical filter 134 b.
- the thickness of optical filters 731 b through 734 b may be 0.7 ⁇ m, and the thickness of special filters 731 d through 734 d may be 1.0 ⁇ m.
- optical filter 734 b may be IR optical filter similar to IR optical filter 134 b, and special filter 734 d may be a third clear filter as shown in FIG. 7B .
- Optical filters 731 b, 732 b, and 733 b have their own specific characteristic transmission curves, i.e., G transmission curve 220 , R transmission curve 230 , and B transmission curve 210 of FIGS. 2B or 2C , respectively.
- optical filter 734 b is the same as optical filter 731 b.
- optical filter 734 b is IR optical filter having IR transmission curve 240 of FIG. 2B or curve 260 of FIG. 2C .
- SIR filter 733 d is similar to selective IR filter 310 having a characteristic transmission curve 400 of FIGS. 4A or 4B .
- a clear filter (C) passes both visible light and IR light.
- While selective IR filter 310 is a notched filter, also known as a bandstop filter, which stops light transmission within a certain stopband (e.g., 850 ⁇ 50 nm), and allows the rest of the spectrum to pass through it.
- the characteristic transmission curve 400 has a region of stopband 405 , which is located at about 550-950 nm. Outside this stopband 405 region, light transmission is relatively high (e.g., 90-100%). Within this stopband 405 , light transmission is significantly reduced. Further, at about 850 nm, light transmission is completely or partially stopped. The overall effectiveness of the stopband 405 depends on its bandwidth.
- a VIS-IR pixel array 800 of the present invention comprises a green (G) pixel 831 , a red (R) pixel 832 , a blue (B) pixel 833 , and an IR pixel 834 .
- G pixel 831 , R pixel 832 , B pixel 833 , and IR pixel 834 are shown in a linear arrangement in FIG. 8A , they may be arranged in a square-shaped, 2 ⁇ 2 pattern, as shown in FIG. 1B .
- Optical filters 831 b (first G optical filter) and 834 b (second G optical filter) are similar to G optical filter 131 b, optical filter 832 b is R optical filter similar to R optical filter 132 b, and optical filters 833 b is B optical filter similar to B optical filter 113 b.
- Special filter 831 d is a first clear filter.
- Special filter 834 d is IR optical filter similar to IR optical filter 134 b.
- Special filter 832 d (first special filter) is a special filter SIR 1 filter similar to selective IR filter 310 , however having an exemplary characteristic transmission curve 902 with a first minimum transmission 904 at a stopband 905 (e.g., 850 ⁇ 50 nm) as shown in FIG. 9 .
- Special filter 833 d is a SIR 2 filter similar to selective IR cut filter 310 , however having an exemplary characteristic transmission curve 906 with a second minimum transmission 908 at stopband 905 (e.g., 850 ⁇ 50 nm) as shown in FIG. 9 .
- the thickness of optical filters 831 b through 834 b may be 0.7 ⁇ m
- the thickness of special filters 831 d through 834 d may be 1.0 ⁇ m.
- optical filter 834 b may be IR optical filter similar to IR optical filter 134 b
- special filter 834 d may be clear filter (third clear filter) as shown in FIG. 8B . Note there is no second clear filter in FIG. 8B .
- the minimum transmission 904 is different from the minimum transmission 908 .
- SIR 1 filter and SIR 2 filter may be made of different materials or different compositions of materials.
- transmission 904 at stopband 905 (e.g., 850 ⁇ 50 nm) of special filter 832 d (first special filter) for red pixel 832 is different from transmission 908 at stopband 905 (e.g., 850 ⁇ 50 nm) of special filter 833 d (second special filter) for blue pixel 833 .
- the ratio of transmission 904 at stopband 905 (e.g., 850 ⁇ 50 nm) of special filter 832 d (first special filter) to transmission in the passband 650-850 nm of optical filter 832 b (R) is different from the ratio of transmission 908 at stopband 905 (e.g., 850 ⁇ 50 nm) of special filter 833 d (second special filter) to transmission in the passband 380-450 nm of optical filter 833 b (B).
- SIR 1 /RSIR 2 /B where SIR 1 is the transmission of IR at stopband 905 (e.g., 850 ⁇ 50 nm) of the special filter at the red pixel, R is the transmission of the red optical filter at the red pixel in the passband 650-850 nm, SIR 2 is the transmission of IR at stopband 905 (e.g., 850 ⁇ 50 nm) of the special filter at the blue pixel, B is the transmission of the blue optical filter at the blue pixel in the passband 380-450 nm.
- the transmission of the red optical filter at the red pixel in the passband 650-850 nm may be 50% (curve 230 ), and the transmission of the blue optical filter at the blue pixel in the passband 380-450 nm may be 70% (curve 210 ).
- SIR 1 may be the same as SIR 2 .
- VIS-IR pixel arrays 700 and 800 may replace VIS-IR pixel array 130 in FIG. 1A or VIS-IR pixel array 300 in FIG. 5 forming a hybrid VIS-IR image sensor system 100 or 500 .
- Hybrid VIS-IR image sensor system 100 or 500 comprises a VIS-IR bandpass filter 121 having a transmission curve including a visible passband of 400-650 nm and an IR passband of 800-900 nm.
- VIS-IR pixel array 700 or 800 may be situated below VIS-IR bandpass filter 121 .
- Hybrid VIS-IR image sensor system works in either VIS mode or IR mode. The VIS mode produces visible images and the IR mode produces IR images.
- Hybrid VIS-IR image sensor system also comprises a light source 160 for emitting light in the IR passband of 800-900 nm
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Power Engineering (AREA)
- Electromagnetism (AREA)
- Condensed Matter Physics & Semiconductors (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- Microelectronics & Electronic Packaging (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Solid State Image Pick-Up Elements (AREA)
Abstract
Description
- This application is a continuation-in-part of U.S. patent application Ser. No. 15/620,757, filed Jun. 12, 2017.
- This disclosure relates generally to image sensor systems that includes a hybrid image sensor pixel array of both visible (VIS) spectrum pixels and infrared (IR) spectrum pixels. More specifically, the image sensor system uses its VIS image sensing capability to produce a VIS color image, and uses its IR sensing capability to perform IR imaging functions.
- Recent development in the image sensing field has resulted in image sensors that feature security and machine vision applications, including gesture sensing, depth analysis, iris detection, eye tracking, night or low light vision, etc. In one aspect, these image sensors use traditional, VIS spectrum image sensing pixels to produce VIS images. In another aspect, these sensors also use additional, IR spectrum pixels to produce IR images. Basically, these sensors serve a dual purpose of producing both VIS and IR images.
- Conventionally, such a dual purpose image sensor employs a direct combination design wherein an IR sub-sensor is physically juxtaposed next to a VIS sub-sensor. The VIS sub-sensor only includes VIS imaging pixels, and the IR sub-sensor only includes IR imaging pixels. This design offers simplicity, but the VIS and IR sub-sensors inevitably have different vantage points with regard to an imaging target. As a result, the resulting VIS and IR images not only have different optical spectra (VIS versus IR), but are also not able to be completely superimposed onto each other. This creates difficulties for subsequent image processing.
- Non-limiting and non-exhaustive examples of the invention are described with reference to the following figures, wherein like reference numerals refer to like parts throughout the various views unless otherwise specified.
-
FIG. 1A is schematic drawing showing an exemplary embodiment of a hybrid VIS-IR image sensor system. -
FIG. 1B is top view of an exemplary VIS-IR image sensor pixel array, showing a minimal repeating unit containing individual pixels. -
FIG. 2A is a wavelength versus transmission plot that shows a characteristic optical transmission curve of a VIS-IR bandpass filter. -
FIG. 2B is a wavelength versus transmission plot that shows a set of characteristic optical transmission curves of the pixels within a VIS-IR pixel array. -
FIG. 2C is another wavelength versus transmission plot that shows another set of characteristic optical transmission curves of the pixels within another VIS-IR pixel array. -
FIG. 3A is a side-view drawing of part of an improved VIS-IR pixel array. -
FIG. 3B is another side-view drawing of part of another improved VIS-IR pixel array. -
FIG. 4A is a wavelength versus transmission plot that shows a characteristic optical transmission curve of a selective IR filter. -
FIG. 4B is another wavelength versus transmission plot that shows two characteristic optical transmission curves of a selective IR filter before and after improvement. -
FIG. 5 is schematic drawing showing an exemplary embodiment of a hybrid VIS-IR image sensor system that includes a selective IR filter. -
FIG. 6 shows an exemplary method that includes a number of steps to perform a VIS vision imaging mode and an IR vision imaging mode. -
FIG. 7A shows an embodiment of a VIS-IR pixel array. -
FIG. 7B shows an alternative embodiment of the VIS-IR pixel array ofFIG. 7A . -
FIG. 8A shows another embodiment of a VIS-IR pixel array. -
FIG. 8B shows an alternative embodiment of the VIS-IR pixel array ofFIG. 8A . -
FIG. 9 is a wavelength versus transmission plot that shows two characteristic optical transmission curves of selected IR filters or suppressed IR (SIR) filters. - Corresponding reference characters indicate corresponding components throughout the several views of the drawings. Skilled artisans will appreciate that elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. For example, the dimensions of some of the elements in the figures may be exaggerated relative to other elements to help to improve understanding of various embodiments of the present invention. Also, common but well-understood elements that are useful or necessary in a commercially feasible embodiment are often not depicted in order to facilitate a less obstructed view of these various embodiments of the present invention.
- In the following description, numerous specific details are set forth to provide a thorough understanding of the examples. One skilled in the relevant art will recognize; however, that the techniques described herein can be practiced without one or more of the specific details, or with other methods, components, materials, etc. In other instances, well-known structures, materials, or operations are not shown or described in detail to avoid obscuring certain aspects.
- Reference throughout this specification to “example” or “embodiment” means that a particular feature, structure, or characteristic described in connection with the example is included in at least one example of the present invention. Thus, the appearances of “example” or “embodiment” in various places throughout this specification are not necessarily all referring to the same example. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more examples.
- Throughout this specification, several terms of art are used. These terms are to take on their ordinary meaning in the art from which they come, unless specifically defined herein or the context of their use would clearly suggest otherwise.
-
FIG. 1A is a schematic drawing of an embodiment of a hybrid VIS-IRimage sensor system 100, which includes amain lens 110, a VIS-IRbandpass filter module 120, a VIS-IR imagesensor pixel array 130, acontroller module 140, afunction logic module 150, and an IRlight source module 160. Features and functions of each element are disclosed further herein. - The
main lens 110 focuses incoming light to pass through the VIS-IRbandpass filter module 120, onto the underlying VIS-IR imagesensor pixel array 130. As an example, themain lens 110 is optically transparent, and allows all VIS-IR spectrum light to pass through. In other words, themain lens 110 does not have an IR attenuation (IR cut) function. - The VIS-IR
bandpass filter module 120 includes a VIS-IR bandpass filter (or filters) 121 that is attached to either the front side and/or the backside of acover glass substrate 122. Thecover glass substrate 122 provides a mechanical support of the VIS-IR bandpass filter 121, and is preferred to be optically transparent to allow the passage of the full spectrum of both VIS and IR light. The VIS-IR bandpass filter 121 is preferably a multi-layer structure consisting of multiple, alternating layers of transparent inorganic materials (e.g., silicon oxide and titanium oxide), and relies on the principle of destructive interference to stop certain spectral bands of incoming light, while allowing other spectral bands to pass through. In the current exemplary embodiment, the VIS-IR filter 121 as illustrated inFIG. 1A allows a VIS band and an IR band to pass through, while stopping the other wavelengths. This is shown inFIG. 2A , and is further elaborated below. - Referring to
FIG. 2A as an example, it shows a characteristic wavelength-versus-transmission curve 250 of the VIS-IR bandpass filter 121. Thetransmission curve 250 includes a VISpassband 260 of approximately 400-650 nanometer (nm) wavelength in the VIS range. Note that the shape of the VISpassband 260 has the general form of a rectangle (or close to a trapezoid with two steep sides, i.e., the two base angles of the trapezoid are slightly under 90 degrees), with a height of almost 100% transmission, and it has almost no leakage outside thepassband 260. In other words, the VISpassband 260 is close to the ideal passband for the situation of passing a wavelength range. This near ideal passband characteristic is achieved through the use of destructive interference produced by the multi-layer transparent inorganic materials. It is appreciated that such a near-rectangular, no-leak passband characteristic is different from a bell-curve-shaped and leaky passband characteristic that results from an absorption by organic compounds, as shown inFIG. 2B , a point that will be further elaborated later in this disclosure. - The wavelength-
transmission curve 250 also includes anIR passband 270 of approximately 800-900 nm wavelength in the IR range. Note again that the shape of theIR passband 270 has the form of a steep triangle (whose two base angles are slightly under 90 degrees), which centers around 850 nm, and it has almost no leakage outside theIR passband 270, i.e., theIR passband 270 is close to the ideal passband for the situation of passing a singular wavelength (at around 850 nm). This near ideal passband characteristic is again achieved through the use of destructive interference produced by multi-layer transparent inorganic materials. - It is appreciated that the
characteristic curve 250 inFIG. 2A shows two passbands—a VISpassband 260 and anIR passband 270. As a result, the VIS-IR filter 121 may be referred to as a dual bandpass filter, which functions to pass incoming light at a broad VIS range (400-650 nm) and a narrow IR range (850±50 nm, or 800-900 nm). In other examples, the IR range may be even narrower, e.g., 850±30 nm, 850±10 nm, and so on. Other wavelengths are stopped by this VIS-IR filter 121. - It is appreciated that the aforementioned passband values that are achieved by destructive interference produced by multi-layer transparent inorganic materials is dependent on the chief ray angle (CRA) of the incident light. This is due to the nature of destructive interference, which involves light reflecting off the interfaces of the multi-layer transparent materials. As an example, at a CRA of zero degrees (incoming light vertical to the incident plane, and is contained in the normal plane), the narrow IR passband is about 850±50 nm, i.e., 800-900 nm, as previously disclosed. In contrast, at a CRA of 30 degrees (light coming in obliquely at an angle, and is off the normal plane by 30 degrees), the IR passband shifts to a new range of 825±45 nm, or 780-870 nm. This passband dependency on CRA will have ramifications on the image sensor system design, and will be brought up again later in this disclosure.
- Referring now to
FIG. 1A , the VIS-IR imagesensor pixel array 130 is situated below the VIS-IRbandpass filter module 120. Thepixel array 130 comprises a multitude of VIS and IR pixels, wherein a group of VIS and IR pixels form a multiple repeatingunit 135, which is shown inFIG. 1B as part of a top view of thepixel array 130. A minimal repeating unit is defined as the smallest group of pixels that make up an entire pixel array. According to FIG. 1B, the minimal repeatingunit 135 ofpixel array 130 consists of a green (G)pixel 131, a red (R)pixel 132, a blue (B)pixel 133, and anIR pixel 134, arranged in a square-shaped, 2×2 pattern. -
FIG. 1A shows a cross-sectional side view of (a representative portion of) thepixel array 130, which comprises the fourindividual pixels 131 through 134. For illustrative purposes, only four pixels 131 (G), 132 (R), 133 (B), and 134 (IR) are shown in thepixel array 130. In reality, thepixel array 130 includes many more individual pixels that may have different arrangements. All fourpixels pixel 131 as a representative pixel. - The
G pixel 131 includes amicrolens 131 a, a Goptical filter 131 b, and animage sensing member 131 c. Themicrolens 131 a may be made of a resin material, and primarily functions to concentrate incoming light into thepixel 131 itself The Goptical filter 131 bcontains organic material, and relies on chemically based optical absorption principle to pass a particular wavelength band of light while stopping other wavelengths. In this example, the Goptical filter 131 b passes a green wavelength band of about 450-650 nm. -
FIG. 2B shows a wavelength-versus-transmission set ofcurves 200 that includes a G bandpasscharacteristic curve 220, which corresponds to the Goptical filter 131 b. TheG curve 220 includes a main color G bandpass range of about 450-650 nm, but also has significant leakage ranges outside the 450-650 nm main passband, for example, in the 300-450 nm range, and also in the near-IR and IR range of 700-1100 nm. The leakage range is due to the nature of the chemically based optical absorption. Within the 450-650 nm main color G passband, the appearance of the curve is similar to a partially skewed and non-smooth bell-curve shape. This non-ideal passband shape is also due to the chemically based optical absorption. These characteristics are different from the near ideal characteristics of the VIS-IR bandpass filter 121 as disclosed inFIG. 2A and its related description. It is appreciated that whereas a multi-layer optical filter that relies on destructive interference is applicable for large size (millimeter scale and above) optical filters such asfilter 121, this type of optical filter design is generally not feasible to make the optical filter for a small image sensor pixel (micrometer scale or less), such aspixel 131. Instead, an organic compound filter that relies on chemically based optical absorption is more suitable for small size optical filters such asfilter 131 b, because the requisite manufacturing process (e.g., photolithography, spin coating, and drying) can be readily applied to make small, micrometer (or sub-micron) size filters. - Returning to
FIG. 1A , theG pixel 131 includes theimage sensing member 131 c directly below the Goptical filter 131 b. Theimage sensing member 131 c may be of a CMOS (Complementary Metal Oxide Semiconductor) or CCD (Charge Coupled Device, similar to CMOS, but with minor variations) configuration. For example, in a CMOS configuration, the sensingmember 131 c is made with a silicon substrate (not labeled), into which a photodiode (not shown) is formed. Other parts such as a photodiode pinning layer (not shown), a well structure (not shown) around the photodiode, and isolation trenches (not shown) may also be present. These parts work to produce photoelectric signals based on the photoelectric effect. Various transistor gates (transfer gate, reset gate, source follower, and row select), floating nodes, and electrical wirings are useful to amplify and relay optoelectronic signals to be processed later, but are also not shown. - Similar to the above description of the
G pixel 131, other pixels such as theR pixel 132, theblue pixel 133, and theIR pixel 134 each include their requisite microlenses (132 a, 133 a, and 134 a), optical filters (132 b, 133 b, and 134 b), and image sensing members (132 c, 133 c, and 134 c). In particular, theoptical filters G transmission curve 220, anR transmission curve 230, aB transmission curve 210, and anIR transmission 240, respectively, as shown inFIG. 2B . These optical filters rely on chemically based optical absorption, and their characteristic curves have leakage ranges beyond their main spectral passbands, as well as having skewed and non-smooth bell-curve shapes within their main spectral passbands. - It is appreciated that in the above embodiment, the primary passband for the
G transmission curve 220 is around 450-650 nm (with about 80% peak transmission); the primary passband for theR transmission curve 230 is around 650-850 nm (with about 50% peak transmission); the primary passband for theB transmission curve 210 is around 380-450 nm (with about 70% peak transmission); and the primary passband for theIR transmission curve 240 is around 800-1000 nm (with about 20% peak transmission), wherein the primary IR passband overlaps with the primary passband of theR transmission curve 230 at around 800-850 nm. - In addition, each of the three VIS transmission curves (the
G curve 220, theR curve 230, and the B curve 210) contains one or several leaky passbands outside its respective main spectral passband, e.g., a leaky band being situated around the near-IR and IR range of 700-1100 nm, with a peak transmission value of around 20%. This means that each VIS pixel (R, G, and B) is capable of detecting IR light to some degree. -
FIG. 2C shows an alternative wavelength-versus-transmission set ofcurves 250. This set ofcurves 250 are similar to the set ofcurves 200 inFIG. 2B , except for theIR curve 260 that has a much higher transmission level than theprevious IR curve 240 found inFIG. 2B . Both IR curves 240 and 260 have their main spectral passband situated around 800-1000 nm, butcurve 260 has a peak transmission level of around 50%, which is much higher than the 20% ofcurve 240. -
Controller module 140 may be physically and electrically coupled with the VIS-IR imagesensor pixel array 130, as shown inFIG. 1A , and operates the hybrid VIS-IR image sensor system to select among a number of operating modes. In an embodiment, these modes include (1) a VIS vision mode, which is generally appropriate for human vision; and (2) an IR vision mode, which is generally appropriate for security vision, machine vision and night vision. In such an embodiment, thecontroller module 140 includes a VIS vision mode control sub-module 141 that controls the VIS vision mode, and an IR vision mode control sub-module 142 that controls the IR vision mode. As an example, the VIS control sub-module 141 selectively controls the VIS pixels (e.g.,pixels pixel array 130, as shown by the dottedlines connecting pixels control sub-module 141. The IR control sub-module 142 selectively controls the IR pixels (e.g., pixel 134) of thepixel array 130, as shown by a dottedline connecting pixel 134 with theIR control sub-module 142. It is appreciated that the aforementioned control of thepixel array 130 by thecontroller module 140 is optional. -
Function logic module 150 receives image signals (not shown) that are read out from thepixel array 130, and processes these signals into images, e.g., VIS and IR images. VIS pixel signals are used to produce VIS images, and IR signals are used to produce IR images. Image processing is conventionally performed by an image signal processor (ISP, not shown) as part of thefunction logic module 150. - IR
light source module 160 operates to illuminate an imaging object with IR spectral light. In an example, the IRlight source module 160 is an IR light emitting diode (LED) that emits a relative narrow spectral range of IR light at around 850 nm, with a tolerance of around plus and minus 50 nm, 30 nm, 10 nm, or even less. The emitted IR light of 850±50 nm may be continuous, or in rapid, intense pulses (e.g., MHz pulsing frequency, with each pulse period at nanosecond scale). In an embodiment as shown inFIG. 1A , during the IR vision mode, the IR control sub-module 142 controls the IRlight source module 160 to emit pulses of IR light at about 850 nm. - The workings of the VIS and IR vision modes are disclosed herein. When the IR mode is selected for machine vision or night vision application, the
IR control sub-module 142 is engaged to control the IRlight source module 160 to emit rapid pulses of intense IR light of about 850±50 nm (tolerance may be less than 50 nm, e.g., 30 nm, 10 nm, etc.) to illuminate an imaging subject. This illuminating IR light has sufficient power to overwhelm naturally occurring ambient light. This operation scheme is effective in both day and night, but is especially effective during the night (or in a foggy environment) when there is low ambient natural light. The light reflected off the imaging subject contains a high percentage of IR spectral component of about 850±50 nm (tolerance may be less than 50 nm), and other less significant spectral components (such as VIS spectra) are regarded as noise. Hence this reflected light is essentially an IR light of about 850±50 nm (tolerance may be less than 50 nm). The aforementioned reflected IR light passes through themain lens 110 without any significant optical filtration. As this IR light continues to pass through the VIS-IR dualbandpass filter module 120, the narrow 850±50 nm (tolerance may be less than 50 nm) IR passband 270 (shown inFIG. 2B ), which overlaps, or fully includes the reflected IR spectral range of 850±50 nm (tolerance may be less than 50 nm), allows most of this reflected IR light to pass through without significant loss, to continue on to reach theunderlying pixel array 130. - It is appreciated that the VIS-
IR bandpass filter 120 has a narrow IR passband whose center coincides with the center of the spectral range of the IR light that is emitted by thelight source 160, e.g., a 850 nm center. It is also appreciated that theIR passband 270 as shown inFIG. 2A is wide enough (i.e., has enough tolerance around its center) to permit most of the IR light reflecting off the imaging subject to pass through. It is further appreciated that theIR passband 270 is not exceedingly wide to permit too much noise to pass through. As an illustrative example, thelight source 160 emits IR light with a spectral range of about 850±40 nm, and the VIS-IR bandpass filter 120 has a corresponding IR passband of about 850±50 nm. - In the IR vision mode, the
pixel array 130 essentially functions as an IR sensor array that relies on itsIR pixels 134 to produce IR signals. In an embodiment, the VISpixels IR pixels 134 are controlled by the IR control sub-module 142 to engage in operation. SeeFIG. 1A for control lines connecting thecontrol module 140 to thepixel array 130. The VIScontrol sub-module 141 disengages the VISpixels sub-modules pixel array 130, the signal output is primarily IR signals. As a result, thepixel array 130 functions essentially as an IR sensor. IR machine vision mode may be useful in applications such as distance measurement, movement tracking, etc. - When VIS vision mode is selected for human vision, the
control module 140 may be engaged to suppress the IRlight source module 160 so that it does not emit IR light. The VIS-IRimage sensor system 100 now depends on natural lighting to illuminate the imaging subject. Incoming natural light (not shown inFIG. 1A ) generally contains all VIS spectrum light, as well as some naturally occurring IR spectrum light. The incoming natural light passes through themain lens 110 without any significant optical filtration. As the incoming light continues to pass through the VIS-IR dualbandpass filter module 120, the broad 400-650 nm VIS passband 260 (shown inFIG. 2B ) allows the VIS light in the main spectral range 400-650 nm to pass through, while the narrow 850±50 nm IR passband 270 (shown inFIG. 2B ) allows only a small amount of naturally occurring IR light that is in the narrow spectral range of 850±50 nm to pass through. Other spectra of incoming light, including most of the IR outside the 850±50 nm range, are filtered out. The unfiltered VIS and IR portions of the incoming light continue to reach theunderlying pixel array 130. - The
pixel array 130 functions as a regular RGB sensor array to produce VIS image signals (R, G, and B signals). In an embodiment, the VIScontrol sub-module 141 engages the VISpixels IR control sub-module 142 disengages theIR pixels 134 from operating. In an alternative where selection bysub-modules IR pixels IR pixels 134 is not directly used to form a final color image. The VISpixels functional logic module 150 uses these RGB signals to produce a final color image after regular image processing steps such as interpolation. - The IR component within the filtered incoming light does affect the RGB signals to some degree. For example, as shown in
FIG. 2B , theR transmission curve 230 of theR color filter 132 b, theG transmission curve 220 of theG color filter 131 b, and theB transmission curve 210 of theB color filter 133 b each contain a leaky passband in the IR range of about 700-1100 nm, which is wider than the 850±50 nm passband of the overlying VIS-IR dual passband filter module. Therefore, the RGB signals that are produced by the VISpixels - Advanced signal processing may be used to remove these IR components, but this removal may be effective only in some circumstances. As an example, when the
optical filters FIG. 2B , theIR pixel 134 may be set to operate at the same time as the VISpixels IR pixel 134 may then be used as a corrective baseline to subtract the IR components (at 850±50 nm) from the VIS signals produced by the VISpixels - However, in some situations, an accurate removal of IR noise is not possible. For example, when the
optical filters FIG. 2C , wherein theIR transmission curve 260 is much more pronounced than (or otherwise different from) theIR curve 240 inFIG. 2B , the aforementioned IR correction scheme will become inaccurate and unreliable. This is because the IR signal produced by theIR pixel 134 is much different from (e.g., much higher than) the IR components within the RGB signals that are produced by VISpixels IR pixel 134 and only engaging the VISpixels pixels B pixel 133 and theG pixel 131 more than it affects theR pixel 132. The B and G spectra are further away from IR spectrum than the R spectrum; whereas there is a natural overlap between the R and IR spectra. - It is appreciated that in the aforementioned IR noise problem, the culprit IR noise refers to the IR component that passes through the
IR passband 270 of the VIS-IR bandpass filter 120, as shown inFIGS. 1A and 2A . An exemplary IR noise occurs at 850±50 nm (at zero degrees CRA). Another exemplary IR noise occurs at 825±45 nm (at 30 degree CRA). - In order to solve the aforementioned IR noise problem relating to the VIS vision mode of operation, an exemplary first improvement is disclosed herein. The basic idea is use some means to stop the incoming IR noise that passes through the IR passband 270 (e.g., around 850±50 nm).
FIG. 3A shows the design of such an improvement. An improved VIS-IR imagesensor pixel array 300 is an improved version over the original VIS-IR pixel array 130 as shown inFIG. 1A . Theimproved pixel array 300 has the same components as theoriginal pixel array 130, including themicrolenses 131 a through 134 a, theoptical filters 131 b through 134 b, and theimage sensing members 131 c through 134 c. In addition, theimproved pixel array 300 includes a selectiveIR cut filter 310, which is superimposed with, i.e., either overlying above or underlying below (e.g., overlying above) the VISoptical filters selective IR filter 310 may be a single unit of filter that overlies above a number of VIS optical filters (e.g., some combination of 131 b, 132 b, and 133 b), or in the alternative as shown inFIG. 3B , it may be a single filter that overlies above a single VIS filter. For such an embodiment, each VIS optical filter has its own correspondingselective IR filter 310, either overlying above or underlying below it. Further, theimproved pixel array 300 includes an optionalIR pass filter 320, which is superimposed with, i.e., either overlying above or underlying below (e.g., overlying above) the IRoptical filter 134 b. The features of the selective IR cutfilter 310 and the optionalIR pass filter 320 are disclosed herein. - The
selective IR filter 310 is a notched filter, also known as a bandstop filter, which stops light transmission within a certain stopband (e.g., 850±50 nm), while allowing the rest of the spectrum to pass through it. Theselective IR filter 310 contains organic material, which relies on chemically based optical absorption principle to stop a particular wavelength band of light while passing other wavelengths. The term “selective” indicates that theselective IR filter 310 selectively stops IR with a relatively narrow stop band, e.g., around 850±50 nm. -
FIG. 4A shows a characteristiclight transmission curve 400 of thisselective IR filter 310. Since theselective IR filter 310 relies on chemically based optical absorption to stop a certain band of light to pass through, the shape of thestopband 405 is not ideal, and has an appearance that is similar to a partially skewed and non-smooth inverted bell-curve shape. It is appreciated that whereas a multi-layer optical filter based on destructive interference is applicable for large size (millimeter and above) optical filters such as filter 121 (shown inFIG. 1A ), such a filter design is generally not feasible to make the optical filter for small image sensor pixels (micrometer scale or below), such as the pixels of thepixel array 300 inFIG. 3A . Instead, an organic compound filter that relies on chemically based optical absorption is more suitable for such a situation, because the requisite manufacturing process (e.g., photolithography, spin coating, and drying) can be readily applied to make small, micrometer (or sub-micron) size optical filters. - According to
FIG. 4A , thecharacteristic transmission curve 400 has a region ofstopband 405, which is located at about 550-950 nm. Outside thisstopband 405 region, light transmission is relatively high (e.g., 90-100%). Within thisstopband 405, light transmission is significantly reduced. Further, at about 850 nm, light transmission is completely stopped. The overall effectiveness of thestopband 405 depends on its bandwidth. - There are a number of ways to quantitatively characterize the bandwidth for the
stopband 405. For example, as shown inFIG. 4A , thestopband 405 starts at approximately 550 nm, and ends at approximately 950 nm. However, this characterization of stopband (“apparent stopband”) may not reflect the true effectiveness of thestopband 405, because within thestopband 405, the reduction of light transmission is not uniform. In contrast, a half-power assessment appears to be more accurate in quantifying the effectiveness of thestopband 405. Generally speaking, within a half-power bandwidth Whp (shown inFIG. 4A ), about half the incoming optical signal energy is stopped by an optical filter. This corresponds to approximately 30% transmission level of thetransmission curve 400. FromFIG. 4A , the half-power bandwidth Whp, may be regarded as the effective bandwidth of thestopband 405, wherein Whp is in the range of 810-880 nm (about 70 nm in width), with 850 nm being the center point at which all light is stopped. Since thestopband 405 is skewed to the high wavelength side, a more accurate way to depict the effective bandwidth Whp of thestopband 405 is to use a three-number notation, 810-850-880 nm, wherein the three numbers each denotes the left end, the center, and the right end of the effective bandwidth region of this stopband. - Referring to
FIGS. 3A and 3B that show theselective IR filter 310 overlying directly above the VISoptical filters selective IR filter 310. Thestopband 405 as depicted inFIG. 4A has an effective bandwidth of 810-850-880 nm, which essentially overlaps with the IR noise range of 800-850-900 nm (alternative notation of 850±50 nm, indicating left end, center, and right end). In particular, the two centers coincide at 850 nm. In other words, the most effective part of the stopband meets the highest IR noise point. Additionally, the left (800 nm) and right (900 nm) ends of the IR noise range fall only slightly outside the effective bandwidth (810-880 nm) of thestopband 405. Therefore, theselective IR filter 310 appears effective in essentially eliminating the IR noise that will cause a reddish hue in the final color image during the VIS human vision mode. - Turning to the
IR pass filter 320 inFIGS. 3A and 3B , it is appreciated that this is an optional element. Since theselective IR filter 310 has been added to the VIS filters 131 b, 132 b, and 133 b, but not to theIR filter 134 b, some planarization is needed for theIR filter 134 b to increase overall thickness. TheIR pass filter 320 provides such a planarization function. In addition, theIR pass filter 320 may be situated either directly above or below theIR filter 134 b. The IR pass filter contains organic material that is suitable for small image sensor pixels (micrometer scale or below). - The
IR pass filter 320 allows a wide range of IR to pass. Its IR passing range preferably covers the entire IR passing range of its adjoiningIR filter 134 b, which is depicted to have thecharacteristic transmission curve 240 inFIG. 2B , orcurve 260 inFIG. 2C . The IR passing function offilter 320 may be implemented in a number of ways. First, theIR pass filter 320 may be transparent enough to pass all spectra of VIS and IR (thus makingfilter 320 an “all-pass” filter, which is essentially a non-filter). Second, filter 320 may be a low-pass filter to only pass the longer wavelength spectrum in the IR range, e.g., approximately 700 nm and above. Third, without implementing the optionalIR pass filter 320 at all, the existingIR filter 134 b may be simply made thicker (e.g., with additional thickness essentially equal to the thickness of the selective IR filter 310) to achieve planarization with the neighboring VIS filters 131 b, 132 b, and 133 b. Other effective options may exist. Different organic material may be appropriately chosen for each option above, as a person having ordinary skill in the art may recognize. -
FIG. 5 is a schematic drawing of an improved hybrid VIS-IRimage sensor system 500. It is largely similar to the VIS-IR system 100 as shown inFIG. 1A , but includes theselective IR filter 310 and the optionalIR pass filter 320 within theimproved pixel array 300. Other components remain the same as theoriginal system 100. The presence of theselective IR filter 310 allows the improved hybrid VIS-IR system 500 to reduce IR noise in its final VIS images while engaging in the VIS vision mode. Its IR vision mode remains the same as in theoriginal system 100. - As previously discussed, the VIS-
IR filter module 120 is made of multi-layer transparent materials, which uses destructive interference to produce its passbands. As a result, its passbands may shift, depending on the chief ray angle (CRA) of the incoming light. As a particular example, at a CRA of zero degrees, theIR passband 270 as shown inFIG. 2A is about 850±50 nm, i.e., 800-850-900 nm. In contrast, at a CRA of 30 degrees, theIR passband 270 now becomes 825+45 nm, or 780-825-870 nm. This passband dependency on CRA allows IR noise to shift with CRA, thus rendering the selective IR filter less effective at some CRA values than other CRA values, as further explained below. - As previously disclosed, the
selective IR filter 310 has an effective stop bandwidth of 810-850-880 nm, which works well for a zero degree CRA incoming IR noise at 800-850-900 nm. However, at a 30 degree CRA, theIR passband 270 of the VIS-IR filter 120 shifts to 825±45 nm, or 780-825-870 nm, hence the incoming IR noise now shifts to a range of 780-825-870 nm. The present selective IR filter will not be very effective to deal with this shifted IR noise. First, the selective IR filter stop bandwidth center of 850 nm no longer coincides with the shifted IR noise center of 825 nm. More significantly, the left end (780 nm) of the shifted IR noise now falls well outside the left end (810 nm) of the selective IR filter stop bandwidth. This means that at a 30 degree CRA, the currentselective IR filter 310 will lose some effectiveness to reduce IR noise. - To solve the aforementioned CRA dependent shifting of the IR noise, the
selective IR filter 310 is improved by widening the effective bandwidth Whp of its stopband.FIG. 4B illustrates an embodiment of a second improvement to the hybrid VIS-IR system (which is a first improvement to the selective IR filter 310).FIG. 4B shows the originalcharacteristic transmission curve 400 of theselective IR filter 310 before improvement, which is the same as thetransmission curve 400 shown inFIG. 4A . An improvedcharacteristic transmission curve 410 is that of theselective IR filter 310 after improvement. Improved curve 410 (solid curve) has a wider effective bandwidth than the original curve 400 (dotted curve). In this embodiment, theimproved curve 410 has an effective stop bandwidth of 720-850-880 nm. This widening of effective bandwidth is achieved by modifying the organic material within theselective IR filter 310. - The improved stop bandwidth of 720-850-880 nm is more effective than the pre-improvement bandwidth of 810-850-880 nm to filter out incoming IR noise. The following table compares the two bandwidths quantitatively.
-
TABLE 1 comparison of the pre and post improvement stop bandwidth of the selective IR filter Improved Selective IR Filter Selective IR Filter IR stopband 810-850-880 nm 720-850-880 nm 0 deg CRA noise range 800-850-900 nm 800-850-900 nm (well handled) (better handled) 30 deg CRA noise range 780-825-870 nm 780-825-870 nm (not well handled) (better handled) - At zero degree CRA, the incoming IR light noise has a range of 800-850-900 nm. The improved selective IR filter stopband of 720-850-880 nm essentially catches all the IR noise range. The stopband center coincides with the noise range center at 850 nm. The left end (800 nm) of the noise range falls fully within the left end (720 nm) of the improved stopband. More impressively, at 30 degree CRA, the incoming IR light noise has a shifted range of 780-825-870 nm, and the left end (780 nm) of this shifted noise range also falls fully within the left end (720 nm) of the improved stopband. This improved stopband of 720-850-880 nm is markedly more advantageous over the previous selective IR filter stopband of 810-850-880 nm.
- In summary, the pre-existing
selective IR filter 310 is improved by widening its effective IR stopband Whp, e.g., from previously 810-850-880 nm to the wider range of 720-850-880 nm. This allows for an improved selective IR filter performance to better handle incident IR noise coming in at different directions, e.g., from zero to 30 degree CRA. - The improved
selective IR filter 310 is similarly implemented in the VIS-IRimage sensor system 500, as shown inFIG. 5 . Previous disclosure still applies, with the exception that the improvedselective IR filter 310 now has a wider effective bandwidth (e.g., 720-850-880 nm), and this secondly improvedsystem 500 can better handle incoming IR noise at various CRA angles (e.g., from zero to 30 degrees). - The several embodiments disclosed above may utilize a method of operation described herein.
FIG. 6 shows anexemplary method 600, which includes a number of steps. The order of some of the steps need not be chronologically arranged. Atstep 610, a selection is made to engage either a VIS vision mode or an IR vision mode. Step 610 may be executed by thecontroller module 140 inFIG. 1A to select either the VISmode 141 or theIR mode 142. If VISmode 141 is selected to be engaged, as shown instep 620, then in a followingstep 621, IR lighting of an imaging subject is suppressed. This may be accomplished by keeping the IRlight source 160 inFIG. 1A from being switched on. Natural lighting is used to illuminate the imaging subject for this VIS mode. The natural light reflecting off the imaging subject may include both VIS and IR spectral components. Instep 622, the reflected natural light is filtered to allow specific VIS and IR bands to pass. For example, the VIS-IR filter 120 allows the broad VISband 260 and thenarrow IR band 270 to pass through VIS and IR components respectively, as shown inFIGS. 1A and 2A . Step 623 is optional, wherein the passed-throughIR band 270 is stopped from reaching theunderlying pixel array 130, particularly its VISpixels FIG. 3A andFIG. 5 ) has a stopband 405 (seeFIG. 4A ) that stops the IR noise that has passed throughIR band 270. Finally, the leftover VIS component of the reflected natural light is used to form a final VIS image that is relatively free of IR noise influence. - If the
IR mode 142 is selected to be engaged, as shown instep 630, then IR lighting is used instep 631 to illuminate the imaging subject. For example, the IRlight source 160 inFIG. 1A may be switched on to emit rapid pulses of IR light towards the imaging subject. Due to the relatively high power of the IR pulses, the reflected light has a significant IR component, and the VIS spectrum may be regarded as low level noise. TheIR mode 142 applies in both day and night, and works particularly well at low light conditions, including dark or foggy environments. Instep 632, the IR component is allowed to pass through to reach the underlying pixel array. For example, the VIS-IR filter 120 (seeFIG. 1A ) allows the IR band 270 (seeFIG. 2A ) to pass through the reflected light's IR component. Step 633 is optional, wherein the passed-through IR light is allowed to further propagate to reach the sensing region of the underlying pixel array. For example, theIR pass filter 320 allows IR component to reach theimaging member 134 c of the IR pixel 134 (seeFIG. 3A andFIG. 5 ). Finally, the IR component is used to perform IR vision functions, such as distance measurement, object tracking, low light imaging, etc. - Fifth embodiment: combinations of optical filters and special filters
-
FIG. 7A shows an embodiment of the present invention similar toFIGS. 3A and 3B . A VIS-IR pixel array 700 comprises a green (G)pixel 731, a red (R)pixel 732, a blue (B)pixel 733, and anIR pixel 734. AlthoughG pixel 731,R pixel 732,B pixel 733, andIR pixel 734 are shown in a linear arrangement inFIG. 7A , they may be arranged in a square-shaped, 2×2 pattern, as shown inFIG. 1B . -
Pixel array 700 has the same components as theoriginal pixel array 130 ofFIG. 1A , includingmicrolenses 731 a through 734 a (first through fourth microlens),optical filters 731 b through 734 b, andimage sensing members 731 c through 734 c (first through fourth image sensing member). In addition,pixel array 700 includes special filters 731 d through 734 d. Special filters 731 d through 734 d include an IR filter similar to IRoptical filter 134 b, a clear filter (C), and an SIR (suppressed IR) filter similar toselective IR filter 310. A clear filter (C) passes both visible light and IR light. AlthoughFIG. 7A showsoptical filters 731 b-734 b overlying above special filters 731 d-734 d,optical filters 731 b-734 d may be underlying below special filters 731 d-734 d. - In an embodiment shown in
FIG. 7A ,optical filters 731 b (first G optical filter) and 734 b (second G optical filter) are similar to Goptical filter 131 b,optical filter 732 b is R optical filter similar to Roptical filter 132 b, andoptical filters 733 b is B optical filter similar to Boptical filter 133 b. Special filters 731 d is a first clear filter (C). Special filter 732 d (first special filter) is a second clear filter (C). First and second clear filter pass both visible light and IR light. Special filter 733 d (second special filter) is a SIR filter similar toselective IR filter 310. Special filter 734 d is an IR optical filter similar to IRoptical filter 134 b. In an embodiment, the thickness ofoptical filters 731 b through 734 b may be 0.7 μm, and the thickness of special filters 731 d through 734 d may be 1.0 μm. Alternatively,optical filter 734 b may be IR optical filter similar to IRoptical filter 134 b, and special filter 734 d may be a third clear filter as shown inFIG. 7B . -
Optical filters G transmission curve 220,R transmission curve 230, andB transmission curve 210 ofFIGS. 2B or 2C , respectively. In the embodiment ofFIG. 7A ,optical filter 734 b is the same asoptical filter 731 b. In the embodiment ofFIG. 7B ,optical filter 734 b is IR optical filter havingIR transmission curve 240 ofFIG. 2B orcurve 260 ofFIG. 2C . SIR filter 733 d is similar toselective IR filter 310 having acharacteristic transmission curve 400 ofFIGS. 4A or 4B . A clear filter (C) passes both visible light and IR light. - While selective IR filter 310 (SIR filter 733 d) is a notched filter, also known as a bandstop filter, which stops light transmission within a certain stopband (e.g., 850±50 nm), and allows the rest of the spectrum to pass through it. According to
FIG. 4A , thecharacteristic transmission curve 400 has a region ofstopband 405, which is located at about 550-950 nm. Outside thisstopband 405 region, light transmission is relatively high (e.g., 90-100%). Within thisstopband 405, light transmission is significantly reduced. Further, at about 850 nm, light transmission is completely or partially stopped. The overall effectiveness of thestopband 405 depends on its bandwidth. - In another embodiment shown in
FIG. 8A , a VIS-IR pixel array 800 of the present invention comprises a green (G)pixel 831, a red (R)pixel 832, a blue (B)pixel 833, and anIR pixel 834. AlthoughG pixel 831,R pixel 832,B pixel 833, andIR pixel 834 are shown in a linear arrangement inFIG. 8A , they may be arranged in a square-shaped, 2×2 pattern, as shown inFIG. 1B . -
Optical filters 831 b (first G optical filter) and 834 b (second G optical filter) are similar to Goptical filter 131 b,optical filter 832 b is R optical filter similar to Roptical filter 132 b, andoptical filters 833 b is B optical filter similar to B optical filter 113 b.Special filter 831 d is a first clear filter.Special filter 834 d is IR optical filter similar to IRoptical filter 134 b.Special filter 832 d (first special filter) is a special filter SIR1 filter similar toselective IR filter 310, however having an exemplarycharacteristic transmission curve 902 with a firstminimum transmission 904 at a stopband 905 (e.g., 850±50 nm) as shown inFIG. 9 .Special filter 833 d (second special filter) is a SIR2 filter similar to selective IR cutfilter 310, however having an exemplarycharacteristic transmission curve 906 with a secondminimum transmission 908 at stopband 905 (e.g., 850±50 nm) as shown inFIG. 9 . In an embodiment, the thickness ofoptical filters 831 b through 834 b may be 0.7 μm, and the thickness ofspecial filters 831 d through 834 d may be 1.0 μm. Alternatively,optical filter 834 b may be IR optical filter similar to IRoptical filter 134 b, andspecial filter 834 d may be clear filter (third clear filter) as shown inFIG. 8B . Note there is no second clear filter inFIG. 8B . - The
minimum transmission 904 is different from theminimum transmission 908. SIR1 filter and SIR2 filter may be made of different materials or different compositions of materials. - In this manner,
transmission 904 at stopband 905 (e.g., 850±50 nm) ofspecial filter 832 d (first special filter) forred pixel 832 is different fromtransmission 908 at stopband 905 (e.g., 850±50 nm) ofspecial filter 833 d (second special filter) forblue pixel 833. Accordingly, the ratio oftransmission 904 at stopband 905 (e.g., 850±50 nm) ofspecial filter 832 d (first special filter) to transmission in the passband 650-850 nm ofoptical filter 832 b (R) is different from the ratio oftransmission 908 at stopband 905 (e.g., 850±50 nm) ofspecial filter 833 d (second special filter) to transmission in the passband 380-450 nm ofoptical filter 833 b (B). This would provide better color balance results, i.e., better balance among R, G, and B. - In other words, SIR1/RSIR2/B, where SIR1 is the transmission of IR at stopband 905 (e.g., 850±50 nm) of the special filter at the red pixel, R is the transmission of the red optical filter at the red pixel in the passband 650-850 nm, SIR2 is the transmission of IR at stopband 905 (e.g., 850±50 nm) of the special filter at the blue pixel, B is the transmission of the blue optical filter at the blue pixel in the passband 380-450 nm. For example, the transmission of the red optical filter at the red pixel in the passband 650-850 nm may be 50% (curve 230), and the transmission of the blue optical filter at the blue pixel in the passband 380-450 nm may be 70% (curve 210). In another embodiment, SIR1 may be the same as SIR2.
- It is appreciated that VIS-
IR pixel arrays IR pixel array 130 inFIG. 1A or VIS-IR pixel array 300 inFIG. 5 forming a hybrid VIS-IRimage sensor system image sensor system IR bandpass filter 121 having a transmission curve including a visible passband of 400-650 nm and an IR passband of 800-900 nm. VIS-IR pixel array IR bandpass filter 121. Hybrid VIS-IR image sensor system works in either VIS mode or IR mode. The VIS mode produces visible images and the IR mode produces IR images. Hybrid VIS-IR image sensor system also comprises alight source 160 for emitting light in the IR passband of 800-900 nm - The above description of illustrated examples of the invention, including what is described in the Abstract, is not intended to be exhaustive or to limit the invention to the precise forms disclosed. While specific examples of the invention are described herein for illustrative purposes, various modifications are possible within the scope of the invention, as those skilled in the relevant art will recognize.
- These modifications can be made to the invention in light of the above detailed description. The terms used in the following claims should not be construed to limit the invention to the specific examples disclosed in the specification. Rather, the scope of the invention is to be determined entirely by the following claims, which are to be construed in accordance with established doctrines of claim interpretation.
Claims (20)
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/106,373 US10609309B1 (en) | 2017-06-12 | 2018-08-21 | Combined visible and infrared image sensor incorporating selective infrared optical filter |
CN201910110212.8A CN110854142A (en) | 2018-08-21 | 2019-02-11 | Combined visible and infrared image sensor incorporating selective infrared filter |
TW108104707A TWI776018B (en) | 2018-08-21 | 2019-02-13 | Combined visible and infrared image sensor incorporating selective infrared optical filter and related pixel array |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/620,757 US10425597B2 (en) | 2017-06-12 | 2017-06-12 | Combined visible and infrared image sensor incorporating selective infrared optical filter |
US16/106,373 US10609309B1 (en) | 2017-06-12 | 2018-08-21 | Combined visible and infrared image sensor incorporating selective infrared optical filter |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/620,757 Continuation-In-Part US10425597B2 (en) | 2017-06-12 | 2017-06-12 | Combined visible and infrared image sensor incorporating selective infrared optical filter |
Publications (2)
Publication Number | Publication Date |
---|---|
US10609309B1 US10609309B1 (en) | 2020-03-31 |
US20200120315A1 true US20200120315A1 (en) | 2020-04-16 |
Family
ID=69951589
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/106,373 Active 2037-09-07 US10609309B1 (en) | 2017-06-12 | 2018-08-21 | Combined visible and infrared image sensor incorporating selective infrared optical filter |
Country Status (1)
Country | Link |
---|---|
US (1) | US10609309B1 (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20240015260A1 (en) * | 2022-07-07 | 2024-01-11 | Snap Inc. | Dynamically switching between rgb and ir capture |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB2373943A (en) | 2001-03-28 | 2002-10-02 | Hewlett Packard Co | Visible and infrared imaging camera |
US6824317B2 (en) | 2001-11-21 | 2004-11-30 | Thales Avionics, Inc. | Universal security camera |
US8408821B2 (en) | 2010-10-12 | 2013-04-02 | Omnivision Technologies, Inc. | Visible and infrared dual mode imaging system |
CN105122271B (en) * | 2012-12-10 | 2018-05-29 | 因维萨热技术公司 | The capture of scene and event in room and time |
KR101661088B1 (en) * | 2013-10-17 | 2016-09-28 | 제이에스알 가부시끼가이샤 | Optical filter, solid-state image pickup device, and camera module |
KR102286136B1 (en) | 2014-12-01 | 2021-08-06 | 에스케이하이닉스 주식회사 | Color filter array, image sensor having the same and infrared data acquisition method using the same |
DE102015011429B9 (en) * | 2015-09-01 | 2020-01-09 | Carl Zeiss Meditec Ag | Optical filter system and fluorescence observation system |
US20170140221A1 (en) | 2015-11-13 | 2017-05-18 | Intel Corporation | Dual function camera for infrared and visible light with electrically-controlled filters |
US10484623B2 (en) | 2016-12-20 | 2019-11-19 | Microsoft Technology Licensing, Llc | Sensor with alternating visible and infrared sensitive pixels |
-
2018
- 2018-08-21 US US16/106,373 patent/US10609309B1/en active Active
Also Published As
Publication number | Publication date |
---|---|
US10609309B1 (en) | 2020-03-31 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9666620B2 (en) | Stacked filter and image sensor containing the same | |
US11031423B2 (en) | Imaging element and camera system | |
KR102201627B1 (en) | Solid-state imaging device and imaging device | |
KR101244147B1 (en) | Physical information acquiring method, physical information acquiring device and semiconductor device | |
KR102523203B1 (en) | Solid state image sensor, method of manufacturing the same, and electronic device | |
EP3133646A2 (en) | Sensor assembly with selective infrared filter array | |
US7800684B2 (en) | Solid-state image pickup device and image pickup apparatus with a low reflective film on the substrate | |
US20160099280A1 (en) | Image sensors and methods of forming the same | |
JP5438374B2 (en) | Solid-state imaging device | |
JP2006238093A (en) | Imaging device | |
US20180295336A1 (en) | IMAGING SYSTEM FOR SENSING 3D image | |
JP2012075179A (en) | Physical information acquisition method and physical information acquisition apparatus | |
JP2011166477A (en) | Solid-state imaging element and image input device | |
WO2016111010A1 (en) | Solid-state imaging device | |
US10425597B2 (en) | Combined visible and infrared image sensor incorporating selective infrared optical filter | |
US10784300B1 (en) | Solid-state imaging devices | |
US10609309B1 (en) | Combined visible and infrared image sensor incorporating selective infrared optical filter | |
Takemoto et al. | Multi-storied photodiode CMOS image sensor for multiband imaging with 3D technology | |
TW201415613A (en) | Solid-state imaging device, method for manufacturing solid-state imaging device, and electronic device | |
TWI776018B (en) | Combined visible and infrared image sensor incorporating selective infrared optical filter and related pixel array | |
US20140285691A1 (en) | Solid state imaging device | |
TWI795897B (en) | Solid-state imaging device, method for manufacturing solid-state imaging device, and electronic machine | |
Takemoto et al. | Optical characteristics of multi-storied photodiode CMOS image sensor with 3D stacking technology | |
KR20150016872A (en) | Solid-state imaging device and method for manufacturing the solid-state imaging device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: OMNIVISION TECHNOLOGIES, INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:HO, TAWEI;LU, CHEN-WEI;LIU, CHENG-MING;SIGNING DATES FROM 20180813 TO 20180814;REEL/FRAME:046641/0724 |
|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 4 |