DE102013114450B4 - A method for improving image sensitivity and color information for an image captured by a camera device - Google Patents

Abstract

A method of enhancing image sensitivity and color information for an image captured by a camera device employing a split sub-pixel imaging chip containing a plurality of color-sensitive pixels each comprising a long exposure subpixel and a short exposure subpixel, the The method comprises: applying an IR transmission filter coating to selected subpixels, which comprises applying the IR transmission filter coating only to selected ones of the long exposure subpixels and the short exposure subpixels for only selected ones of the pixels corresponding to a selected color, the IR- Transmission filter coating allows near infrared light within a predetermined wavelength segment to be transmitted through the selected subpixels while blocking transmission of visible light with color information through the selected subpixels; andreconstructing color information within a visible spectral range for the long exposure subpixels and short exposure for the plurality of color sensitive pixels based on the applied IR transmission filter coating on the selected subpixels.

Description

TECHNICAL AREA

This disclosure relates to a method of improving the performance of vision systems in vehicles and, more particularly, to a method of improving image sensitivity and color information for an image captured by a camera device.

From the WO 2011/023224 A1 For example, there is essentially known a method of enhancing image sensitivity and color information for an image captured by a camera device utilizing an imaging chip containing a plurality of color sensitive pixels each comprising long exposure pixels and short exposure pixels . The method includes applying an IR transmission filter coating to selected pixels, the IR transmission filter coating allowing near infrared light within a predetermined wavelength segment to be transmitted through the selected pixels while blocking transmission of visible light with color information through the selected pixels and reconstructing color information within a visible spectral range for the long exposure and short exposure pixels for the plurality of color sensitive pixels based on the applied IR transmission filter coating on the selected pixels.

It is also from the US 2010/0 149 393 A1 known to use split subpixels, with one subpixel for long exposure and one subpixel for short exposure.

BACKGROUND

Vehicle systems often use vision systems in vehicles or vehicle-bound vision systems to record images which represent fields of vision to the rear, to the front and / or to the sides of a vehicle in question. It is known to use infrared (IR) light (e.g., IR components in ambient light or active IR light sources) that is received by vision systems in vehicles without an IR cut filter to increase visibility during night driving, or when Ambient light is otherwise weak. IR light, although not representing true color of an image, provides improved image sensitivity. Images recorded using IR light are often displayed as a colorless image or as an image close to a gray scale. As a result, cameras receiving IR light often lose color fidelity if the IR cut filter is not used due to the additional IR components captured in the final image.

SUMMARY

The method according to the invention for improving image sensitivity and color information for an image recorded by a camera device is characterized by the features of claim 1.

Figure list

• 1 eine beispielhafte Kameravorrichtung gemäß der vorliegenden Offenbarung veranschaulicht, die mehrere Linsenelemente und einen Bilderzeugungschip enthält, die koaxial entlang einer Längsachse angeordnet sind;
• 2 ein nicht beschränkendes Beispiel eines farbempfindlichen Bilderzeugungschips gemäß der vorliegenden Offenbarung veranschaulicht;
• 3 eine Darstellung von Quanteneffizienz-Antwortprofilen mehrerer Farben bezüglich einer Wellenlänge gemäß der vorliegenden Offenbarung veranschaulicht;
• 4 eine beispielhafte Ausführungsform eines Bilderzeugungschips mit geteilten Subpixeln gemäß der vorliegenden Offenbarung veranschaulicht;
• 5 eine beispielhafte Ausführungsform eines Bilderzeugungschips mit geteilten Subpixeln, der eine auf ausgewählten Subpixeln aufgebrachte IR-Durchlassfilterbeschichtung umfasst, gemäß der vorliegenden Offenbarung veranschaulicht;
• 6 eine Darstellung von Quanteneffizienz-Antwortprofilen mehrerer Farben bezüglich einer Wellenlänge für einen Bilderzeugungschip mit geteilten Subpixeln, der eine Applikation einer IR-Durchlassfilterbeschichtung auf ausgewählten Subpixeln umfasst, gemäß der vorliegenden Offenbarung veranschaulicht; und
• 7 ein beispielhaftes Flussdiagramm veranschaulicht, um eine Bildempfindlichkeit und Farbinformation für ein von der Kameravorrichtung von 1 unter Verwendung des beispielhaften Bilderzeugungschips mit geteilten Subpixeln von 5 aufgenommenes Bild gemäß der vorliegenden Offenbarung zu verbessern.

One or more embodiments will now be described, by way of example, with reference to the accompanying drawings, in which:

• 1 illustrates an exemplary camera device in accordance with the present disclosure including a plurality of lens elements and an imaging chip coaxially arranged along a longitudinal axis;
• 2 Figure 3 illustrates a non-limiting example of a color sensitive imaging chip in accordance with the present disclosure;
• 3 Figure 3 illustrates a plot of multiple colors quantum efficiency response profiles with respect to wavelength in accordance with the present disclosure;
• 4th illustrates an exemplary embodiment of a split sub-pixel imaging chip in accordance with the present disclosure;
• 5 FIG. 10 illustrates an exemplary embodiment of a split sub-pixel imaging chip including an IR transmission filter coating applied to selected sub-pixels, in accordance with the present disclosure;
• 6 Figure 3 illustrates a representation of multiple color quantum efficiency response profiles with respect to wavelength for a split sub-pixel imaging chip including application of an IR transmission filter coating on selected sub-pixels, in accordance with the present disclosure; and
• 7th an exemplary flowchart illustrates an image sensitivity and color information for one of the camera device of FIG 1 using the exemplary split sub-pixel imaging chip of 5 improve captured image according to the present disclosure.

DETAILED DESCRIPTION

Referring now to the drawings, wherein the showings are only used to illustrate certain exemplary embodiments 1 schematically a camera device 100 who have favourited a lens 20th , an infrared (IR) cut filter 40, at least one glare-reducing element 41 , a cover glass 42 , an imaging chip 44 and a camera board 46 contains. A lens case 30th surrounds the lens 20th who have favourited multiple lens elements 22nd , 24 , 25 , 26th and 28 includes. The multiple lens elements 22-28 , the IR cut filter 40 , the at least one glare-reducing element 41 , the cover glass 42 , the imaging chip 44 and the camera board 46 are with an optical axis 150 coaxial. It goes without saying that the IR cut filter 40 and the at least one glare-reducing element is shown for illustration only and is not applicable to the embodiments described herein. It is understood that although the optical axis 150 In the illustrated embodiment, since a camera coordinate system is defined, "longitudinal" is the optical axis 150 can include any angle from an origin based on which coordinate system is defined. For example, the optical axis 150 start from the rear of the vehicle and include an angle of inclination slightly downwards towards the ground, if a world coordinate system is defined. light 102 from a light source passes sequentially through each of the lens elements 22-28 , the IR cut filter 40 , the at least one glare-reducing element 41 and the cover glass 42 before it at the imaging chip 44 Will be received. In the illustrated embodiment, the cover glass is 42 on the imaging chip 44 placed. Although the term "glass" is used to refer to the cover glass 42 to describe, it goes without saying that polycarbonates or plastics can be used instead of glass.

In the embodiments described herein, the camera device is 100 configured to capture a field of view (FOV) behind a subject vehicle; however, it should also be understood that the camera device 100 can be expanded to include a FOV in front of the subject vehicle and to the sides of the subject vehicle. In one non-limiting exemplary embodiment, the camera device is 100 set up to record 180 ° FOV with a downward slope towards the ground surface on which a subject vehicle is traveling. The camera device 100 can receive light or other radiation and convert light energy into electrical signals in a pixel format using, for example, one of charge coupled device (CCD) sensors or complementary metal oxide semiconductor (CMOS) sensors. The camera device 100 may be mounted within or on any suitable structure that is part of the vehicle, such as bumpers, spoilers, trunk lids, dashboards, grilles, side mirrors, door panels, etc., as those skilled in the art will well understand and recognize. Image data from the camera device 100 can through a non-transitory processing device 48 in signal connection with the camera board 46 can be processed to generate an image that will appear on any display units of the vehicle, including a rearview mirror display device 50 can be displayed within the vehicle concerned. In one embodiment, the rearview mirror display device is 50 integrated within a human-machine interface (HMI) of the vehicle. In another embodiment, the rearview mirror display device is 50 integrated within a rearview mirror mounted on the inside of the vehicle concerned.

Control module, module, controller, controller, control unit, processor and similar expressions mean any or various combinations of one or more of the group: application-specific integrated circuit (s) (ASIC), electronic circuit (s) ), central processing unit (s) (preferably microprocessor (s)) and assigned working memory and storage device (s) (English: memory and storage) (read-only, programmable read-only, random access, hard disk, etc.), the execute one or more software or firmware programs or routines, circuit (s) with combinational circuit logic, input / output circuit (s) and devices, suitable signal conditioning and buffering circuits, and other components to provide the functionality described. Software, firmware, programs, instructions, routines, codes, algorithms and similar expressions mean any instruction set, including calibrations and look-up tables. The Control module has a set of control routines that are executed to provide the desired functions. Routines are executed, such as by a central processing unit, and are used to monitor inputs from sensing devices and other network-connected control modules and to execute control and diagnostic routines to control operation of actuators. Routines can be executed at regular intervals, for example every 3.125, 6.25, 12.5, 25 and 100 milliseconds during continuous engine and vehicle operation. Alternatively, routines can be executed in response to the occurrence of an event.

The imaging chip 44 may be a color sensitive imaging chip with an integrated circuit composed of an array of pixel sensors, each containing a photodetector and an active amplifier, or the imaging chip 44 can be a monochrome sensor. In one embodiment, the imaging chip is a complementary metal oxide semiconductor (CMOS) sensor. In another embodiment, the imaging chip is a charge coupled device (CCD) sensor and is mounted on a camera board of the camera device. Both CCD and CMOS sensors work by using light sensitive circuitry that responds to light and stores analog signals as digital data, i.e. the captured image. The IR cut filter 40 can be used to block infrared light in a range of wavelengths that exceed a predetermined wavelength. In one embodiment, the predetermined wavelength blocking infrared light may comprise 650 nm. The predetermined wavelength is not limited to any specific value, however, and in other contemplated embodiments may include wavelengths of 680 nm or 700 nm. Therefore, the IR cut filter 40 from the exemplary camera device 100 from 1 removed.

A CCD sensor is a photosensitive analog device that records light as a small electrical charge in each of its pixels or cells. A CCD is essentially a collection of CCD cells. The CCD circuit may contain multiple layers of capacitors (e.g., stages) to transport the analog signal, controlled by a clock signal, to an array of flip-flops for storing the data. Therefore, when light is received by a CCD sensor, the CCD sensor detects an electrical charge based on how much light has hit the particular CCD cell, each CCD cell being able to transfer its charge to its neighboring cell and then to an external circuit . An analog-to-digital converter can be used to read the charge as an integer in a range.

In a CMOS sensor, each pixel has adjacent transistors that perform the analog-to-digital conversion locally. In one embodiment, each pixel can be an active pixel sensor (APS). Imaging logic is built into the CMOS sensor itself, eliminating the need for additional circuitry required by CCD sensors to perform the analog-to-digital conversion. As a result, the power consumption for driving the CMOS sensor can be reduced. Although the design of CMOS sensors can be more complex or more expensive than CCD sensors because of the integrated circuit, CMOS sensors can be manufactured inexpensively on a standard silicon production line. A disadvantage of CMOS sensors is that they are more susceptible to noise than CCD sensors due to the additional circuit integrated on the sensor.

A motion blur within recorded images can result from long exposure times of the camera device. Exposure times show the accumulation of from the camera device 100 received light. One approach to reducing motion blur of an image is to reduce image exposure time and improve image sensitivity. IR cut filters block or block IR light. If there is no IR cut filter (e.g. IR cut filter 40 from 1 ) is used, an image sensitivity is increased, but the color fidelity is reduced. If an IR cut filter is used, the image sensitivity is reduced, but the color fidelity is maintained. It is understood that increased image sensitivity from the transmission of IR light for taking pictures during the night or other dark ambient conditions is desired. Accordingly, it is desirable to increase the color fidelity for driving conditions in daylight and to increase or increase the image intensity / sensitivity for driving conditions at night.

2 Figure 11 illustrates a non-limiting example of a color sensitive imaging chip 600 according to the present disclosure. The color sensitive imaging chip 600 contains several color pixels 602 and an imaging chip 604 . Each of the pixels 602 can be set between a short exposure time and a long exposure time. In some embodiments, long exposure times are desirable during low-light or night-time environments to increase the amount of light received. The imaging chip 604 can be a CMOS sensor or a CCD sensor. CMOS and CCD sensors are sensitive to wavelengths in the near infrared or IR range, which extends to about 1000 nm, which results in unnatural images due to an increased image sensitivity but poor color information. The multiple color pixels 602 include green (Gr and Gb), blue (B), and red (R) pixels, each with a respective light (wavelength) response. The formation of the multiple color pixels enables increased color sensitivity; however, long exposure times during which additional IR light is captured can result in poor color sensitivity, which distorts the color fidelity of the image. Accordingly, an IR cut filter (for example, IR cut filter 40 ) can be used to effectively block IR light at wavelengths that exceed a predetermined wavelength so that color fidelity is increased. Blocking IR light, however, reduces image sensitivity, which may be desirable during night driving conditions, and increases motion blur of the image. Therefore, one envisions embodiments in which the IR cut filter 40 is not used.

3 Figure 3 illustrates a representation of quantum efficiency response profiles 702-704 multiple colors with respect to wavelength according to the present disclosure. As used herein, the term “quantum efficiency” (QE) refers to a measurement of an electrical sensitivity of an imaging sensor to light. In one embodiment, the imaging sensor comprises the imaging chip 600 from 2 . The QE can be measured as a percentage of electrons per photon. Alternatively, QE can be measured as amps per watt. In the illustrated embodiment, the horizontal x-axis denotes wavelength (nm) and the vertical y-axis denotes a QE response (%). Color information is indicated by a dashed area 710 included the respective QE response peaks from each of the blue (B), green (Gb), green (Gr), and red (R) QE response profiles 702 , 704 , 706 or. 708 includes. A dashed vertical line 715 denotes a predetermined wavelength, for example 650 nm, at which an IR cut filter is used to block the transmission of IR light at wavelengths exceeding the predetermined wavelength. Accordingly, the area to the right of the dashed vertical line represents an IR area which provides improved image sensitivity information, and the area to the left of the dashed line 715 represents a true color or visible area of color information. Without the IR cut filter at the dashed vertical line 715 consequently, imaging chips receive increased energy (e.g., light output) to increase image sensitivity at the expense of distorting image color fidelity. With the IR cut filter at the dashed vertical line 715 however, IR light will appear at the dashed vertical line 715 It effectively blocks the exceeding wavelengths, so that image sensitivity is reduced, but color fidelity is retained. It goes without saying that a reduced image sensitivity during the night or driving conditions in poor lighting conditions is undesirable and can also contribute to motion blurring within recorded images.

4th Figure 11 illustrates an exemplary embodiment of an imaging chip 800 with split subpixels. The imaging chip 800 with divided sub-pixels contains a color filter part 802 and an exposure part 804 . Although not distinct, the color filters are part 802 and the exposure part 804 Separated for illustration purposes only to describe example embodiments discussed herein. pixel 850 include green (Gr and Gb), blue (B), and red (R) pixels, each with a respective light response due to different color filters. For example, color information is passed through the color filter part 802 where only green light is allowed to pass through each of the green (Gr and Gb) pixels, only red is allowed to pass through the red (R) pixel, and only blue is allowed to pass through the blue (B) pixel to get. One is aware of the fact that the human eye is more sensitive to green light. In addition, a four-pixel color filter (Gr, Gb, B, and R) is easier to implement than a 3-pixel color filter. Therefore, two green pixels (e.g. Gr and Gb color filters) are used because the human eye is more sensitive to green light and the use of a color filter design with four pixels is desirable. Referring to the exposure part, each is pixel 850 a divided subpixel that is a subpixel (S) 851 for short exposure and one subpixel (L) 852 covers for long exposure. An exploded view 860 illustrates the pixel 850 that the subpixel (S) 851 for short exposure and the subpixel (L) 852 covers for long exposure. A transmission with a higher light intensity enters the subpixel (L) for long exposure than that of the subpixel (S) for short exposure. In other words, the subpixel (L) 852 For long exposure, use a slower shutter speed (or larger area) to increase the accumulation of light entering the pixel. In one non-limiting example, the long exposure and short exposure sub-pixels corresponding to a respective pixel are not the same size.

5 10 illustrates an exemplary embodiment of a split sub-pixel imaging chip including an IR transmission filter coating applied to selected sub-pixels, in accordance with the present disclosure. The imaging chip 900 with divided sub-pixels is essentially identical to the imaging chip 800 with shared subpixels of 4th , where the same numbers describe the same features. In the illustrated embodiment denote pixels 950 with a "1" green (Gr) pixel, denote pixels 950 with a “2” green (Gb) pixels, denote pixels 950 with a "3" red (R) pixel, and pixel 950 with a “4” denote blue (B) pixels. The exposure part 904 contains an IR transmission filter coating applied to selected subpixels 975 . For illustration purposes are the selected subpixels with the IR transmission filter coating applied to them 975 Shown shaded, in contrast to subpixels without the IR transmission filter coating applied to them, which are shown as dashed lines. As used herein, the term "IR transmission filter coating" refers to a predetermined portion of wavelengths at which IR light is allowed to pass through the selected subpixels while blocking visible light containing color information. It goes without saying that subpixels without an IR transmission filter still allow IR light to pass through as long as no IR blocking filter is used, but not visible light within a specified color spectrum, for example within the green (G) color spectrum lock. In the illustrated embodiment, the predetermined range of wavelengths comprises a predetermined range of wavelengths (e.g. 720-950 nm). However, embodiments are envisioned wherein the predetermined portion of wavelengths include any wavelengths that exceed a predetermined wavelength (e.g., 720 nm). Embodiments herein are on the imaging chip 41 the camera device 100 directed that the imaging chip 900 with shared subpixels of 5 contains.

Embodiments herein are directed to the IR transmission filter coating 975 directed that only to selected one of the subpixels 951 or 952 is applied for only selected ones of the pixels 950 according to a selected color. In the illustrated embodiment, the selected ones include pixels 950 the green (Gr and / or Gb) pixels according to the selected color 950 as denoted by the "1" and "2". However, this disclosure does not limit an application of the IR transmission filter coating 975 on selected ones of the red (R) pixels or selected ones of the blue (B) pixels. It can be seen that although the exposure part 904 in the illustrated embodiment of FIG 5 all subpixels 951 for short exposure for the green (Gb) pixels 950 , denoted by “2”, and all subpixels 952 for long exposure for the green (Gr) pixels 950 , denoted by “1”, as the IR transmission filter applied to it 975 are shown having only a portion of these subpixels 951 and 952 among the several pixels 950 will have the IR transmission filter applied to it. This part is referred to as the "selected" of the pixels corresponding to the "selected color".

Because you can see the IR light through the selected subpixels 951 or 952 within the predetermined range of wavelengths while blocking visible light containing color information therefrom, image sensitivity is effectively separated from true color information. With reference to flowchart 1100 from 7th Described further, this blocked color information can be reconstructed within a visible spectral range in order to enable both true color information and image sensitivity information for the selected one of the subpixels 951 or 952 the selected one of the green (Gr and / or Gb) pixels 950 can be preserved and separated. As above with reference to the non-limiting exemplary color sensitive imaging chip 600 from 2 described, the non-separate inclusion of both color information and image sensitivity information results in distorted color fidelity and increased image motion blur. Similarly, while use of an IR cut filter will preserve and maintain color fidelity, it does so at the expense of reducing image sensitivity, which is desirable during nighttime and low-light driving conditions. Accordingly, application of the IR transmission filter coating to selected subpixels enables true color information to be separated from image sensitivity information, so that the imaging chip 900 (for example imaging chip 44 from 1 ) maintains image sensitivity and improves color fidelity. In addition, an overall exposure time of the image can be reduced to reduce image motion blur. It is understood that when the camera device 100 the IR cut filter 40 from 1 does not use subpixels 951 and 952 the pixel 950 achieve greater sensitivity in low light during night or low light driving conditions than if the IR cut filter were used.

6 illustrates a non-limiting representation 1000 of multiple color QE response profiles with respect to wavelength for an exemplary split sub-pixel imaging chip that includes application of an IR transmission filter coating to selected sub-pixels, in accordance with the present disclosure. The horizontal x-axis denotes a wavelength (nm), and the vertical y-axis denotes a response, i.e., QR response (%). The QE response profiles for the multiple colors include response profiles 1002 , 1004 , 1006 , 1008 for blue (B), green (Gb), green (Gr) or red (R) pixels. A dashed vertical line 1015 denotes a predetermined wavelength, for example 720 nm, at which an IR transmission filter coating is applied to selected ones of divided sub-pixels for long exposure and short exposure in each of the Gr and Gb pixels. The IR transmission filter coating can make the entire visible Block or block light within the visible spectral range. For the purposes of this disclosure, the terms "visible light" and "color information" are used interchangeably. It is understood that the area to the right of the dashed vertical line 1015 denotes an IR area, for example near a gray scale image, with improved or increased image sensitivity information, and the area to the left of the dashed vertical line 1015 denotes a true color or visible area with improved color information. As in the non-limiting illustration 1000 As illustrated, the color information blocked by the IR transmission filter coating on the selected subpixels for the Gr and Gb pixels can be reconstructed within the visible area by using the QE response of the selected subpixels (i.e., a QE response with IR transmission filter) is subtracted from the QE response (i.e. a QE response without an IR transmission filter) of the remaining subpixels that have no IR transmission filter coating applied to the Gr and Gb pixels. As used herein, the phrase “QE response with IR pass filter” refers to an accumulated intensity of the selected subpixels with the IR pass filter coating applied thereon, and the phrase “QE response without IR pass filter” refers to accumulated intensity of the remaining subpixels that have no IR transmission filter coating applied to them. For example, an accumulated intensity of the selected subpixels (i.e. the QE response with the IR pass filter) is subtracted from the accumulated intensity of the remaining subpixels (i.e. the QE response without the IR pass filter) for the Gr and Gb pixels. The reconstruction of the color information for the Gr and Gb pixels effectively separates image sensitivity information and color information.

In addition, the QE response with IR transmission filter of the selected subpixels can be used to generate a QE response with IR transmission filter for each of the subpixels for the R and B pixels using the respective QE response profiles 1002-1008 to approximate that in the non-limiting illustration 1000 are applied. Specifically, the accumulated intensity of the selected subpixels with the IR transmission filter applied thereon is used to approximate the accumulated intensity for each of the subpixels for the R and B pixels without the IR transmission filter coating applied thereon. It will be understood that methods such as interpolation can be used to compensate for the differences between the QE response profiles so that accurate approximations can be obtained. Since selected subpixels with the IR transmission filter coating applied to them are very close to neighboring R, B, G subpixels, assumptions can be made, for example, that an accumulated intensity on these neighboring subpixels is very similar without the IR transmission filter coating applied to them. Consequently, an accumulated intensity from the selected subpixels with the IR transmission filter coating applied thereon can be used to approximate the accumulated intensity of the neighboring subpixels as if an IR transmission filter coating were applied, even if such an application does not exist. As used herein, the term “approximated QE response with IR pass filter” refers to a QE response associated with an accumulated intensity indicating that an IR pass filter is applied to corresponding subpixels, even if such an application Does not exist. As used herein, the term “corresponding subpixels” refers to subpixels of the same for long exposure and short exposure times. Thereafter, the approximated QE response with IR transmission filter for respective one of the R and B pixels can be used to reconstruct the color information in the visible spectral range for the corresponding pixels so that image sensitivity information and color information are effectively separated. Accordingly, both improved image sensitivity in low light conditions and correct color information can be obtained using the exemplary split subpixel imaging chip which has an application of the IR transmission filter coating on selected subpixels.

7th illustrates an example flow chart 1100 to improve image sensitivity and color information for an image captured by the camera device of 1 using the exemplary imaging chip 900 with shared subpixels of 5 according to the present disclosure. The exemplary flow chart 1100 is made with reference to the non-restrictive representation 1000 of QE response profiles 1002-1008 , illustrated in 6 described. Table 1 is used as a key too 6 with the numerically labeled blocks and corresponding functions set out as follows. Table 1 BLOCK BLOCK CONTENTS 1102 Applying an IR transmission filter coating to selected subpixels of the imaging chip 900 with split subpixels. 1104 Reconstructing color information within a visible spectral range for the subpixels for long exposure and short exposure for all pixels corresponding to a selected color based on the selected subpixels with the IR transmission filter coating applied thereon. 1106 Reconstructing color information within the visible spectral range for the subpixels for long exposure and short exposure for all pixels corresponding to a first color based on the selected subpixels with the IR transmission filter coating applied thereon. 1108 Reconstructing color information within the visible spectral range for the subpixels for long exposure and short exposure for all pixels corresponding to a second color based on the selected subpixels with the IR transmission filter coating applied thereon.

The camera device 100 from 1 contains a lens element 28 , the light to the imaging chip 900 with shared subpixels (e.g. imaging chip 44 ) lets through. At block 1102 becomes the IR transmission filter coating 975 on selected subpixels 951 and 952 of the imaging chip 900 applied with divided subpixels. Specifically, the IR transmission filter coating is applied only to selected ones of the subpixels for only selected ones of the pixels corresponding to a selected color. In the illustrated embodiment of 5 includes the selected color green (Gr and Gb pixels); however, embodiments may equally be used herein when the selected color includes red or blue. The IR transmission filter coating enables infrared light to be transmitted through the selected subpixels within a predetermined range of wavelengths while blocking transmission of visible light containing color information through the selected subpixels. In the illustrated, non-limiting representation 1000 from 6 the predetermined portion of wavelengths includes all wavelengths exceeding a predetermined wavelength, for example 720 nm. The predetermined portion of wavelengths may include all wavelengths within a predetermined range of wavelengths, for example 720-950 nm in other embodiments. It will be understood that infrared light is transmitted through subpixels that have no IR transmission filter coating applied to them 975 exhibit; However, color information is not blocked in these unselected sub-pixels. Also is the exemplary IR cut filter 40 in the camera device 100 from 1 not included, allowing IR light to go to the imaging chip 900 is transmitted with divided subpixels. In detail in the following blocks 1104-1108 can describe color information within the visible spectral range for the subpixels 952 , 951 for long exposure or short exposure for the multiple color-sensitive pixels 950 based on the applied IR transmission filter coating 975 can be reconstructed on the selected sub-pixels according to the selected color. You can see that blocks 1104-1108 from the exemplary non-transitory processing device 48 from 1 which can also access QE response profiles and associated accumulated intensities for multiple colors, such as the QE response profiles shown in the non-limiting illustration 1000 from 6 are illustrated.

$$C_{S\_selected}={\left(I_{nonIR}\right)}_{S\_selected}-{\left(I_{IR}\right)}_{S\_selected}$$

wobei C_{L_selected} und C_{S_selected} rekonstruierte Farbinformation für die Subpixel für lange Belichtung bzw. kurze Belichtung entsprechend der ausgewählten Farbe repräsentieren,
(I_nonIR)_{L_selected} und (I_nonIR)_{S_selected} eine akkumulierte Intensität von Subpixel für lange bzw. kurze Belichtung entsprechend der ausgewählten Farbe repräsentieren, die keine aufgebrachte IR-Durchlassfilterbeschichtung aufweisen, und (I_IR)_{L_selected} und (I_IR)_{S_selected} eine akkumulierte Intensität von Subpixel für lange bzw. kurze Belichtung entsprechend der ausgewählten Farbe mit der darauf aufgebrachten IR-Durchlassfilterbeschichtung repräsentieren.Referring to block 1104 becomes color information within the visible spectral range for the subpixels 952 , 951 for long exposure or short exposure for all pixels according to the selected color (for example green) based on the selected subpixels with the IR transmission filter coating applied to them 975 reconstructed. Reconstruction of color information for the long exposure and short exposure sub-pixels for all the pixels corresponding to the selected color can be expressed as follows. $${\mathrm{C.}}_{L\_selected}={\left({\mathrm{I.}}_{nOn\mathrm{I.}\mathrm{R.}}\right)}_{L\_selected}-{\left({\mathrm{I.}}_{\mathrm{I.}\mathrm{R.}}\right)}_{L\_selected}$$

$${\mathrm{C.}}_{\mathrm{S.}\_selected}={\left({\mathrm{I.}}_{nOn\mathrm{I.}\mathrm{R.}}\right)}_{\mathrm{S.}\_selected}-{\left({\mathrm{I.}}_{\mathrm{I.}\mathrm{R.}}\right)}_{\mathrm{S.}\_selected}$$

where C _{L_selected} and C _{S_selected represent} reconstructed color information for the subpixels for long exposure and short exposure, respectively, according to the selected color,
(I _nonIR ) _{L_selected} and (I _nonIR ) _{S_selected represent} an accumulated intensity of subpixels for long and short exposure, respectively, corresponding to the selected color, which have no applied IR transmission _{filter coating} , and (I _IR ) _{L_selected} and (I _IR ) _{S_selected} an accumulated Represent intensity of subpixels for long or short exposure according to the selected color with the IR transmission filter coating applied to it.

For the subpixels 952 for long exposure, the color information within the visible spectral range can be reconstructed for all pixels corresponding to the selected color (for example all green Gr and Gb pixels) using equation [1] by generating a QE response with an IR filter from a subtracting the selected long exposure subpixels with the IR transmission filter coating applied thereon from a QE response without an IR transmission filter from one of the long exposure subpixels having no IR transmission filter coating applied thereto and corresponding to the selected color. Specifically, an accumulated intensity of one of the selected subpixels for long exposure having the IR transmission filter coating applied thereon is subtracted from an accumulated intensity of one of the subpixels for long exposure which have no IR transmission filter coating applied thereon and corresponds to the selected color. block 1104 similarly reconstructs the color information within the visible spectral range for the subpixels 951 for short exposure according to the selected color using equation [2].

It will be understood that the QE response with IR transmission filter corresponding to the selected color contains only image sensitivity information within the predetermined range of wavelengths, while the QE response without IR transmission filter corresponding to the selected color contains image sensitivity information interwoven with color information. Thus, subtracting the QE response with an IR pass filter from the response without an IR pass filter enables image intensity / sensitivity information to be removed from the IR spectrum so that the real color information is obtained by reconstructing all subpixel for pixel according to the selected color can.

Referring to block 1106 becomes color information within the visible spectral range for the subpixels 952 , 951 for long exposure or short exposure for all pixels corresponding to a first color reconstructed based on the selected subpixels corresponding to the selected color with the IR transmission filter coating applied thereon 975 . In the illustrated embodiment of 5 if the selected color includes green, the first color may include red or blue. It can be seen that no sub-pixels for the pixels corresponding to the first color have an application of the IR transmission filter coating. However, approximations can accordingly be obtained indicating subpixels corresponding to the first color with the IR transmission filter coating applied thereon by referring to QE responses with IR transmission filter and associated accumulated intensities of selected subpixels with the IR transmission filter actually applied thereon. For example, an approximated IR transmission filter QE response for the long exposure sub-pixels corresponding to the first color indicating the IR transmission filter coating has been applied thereon can be obtained within the predetermined range of wavelengths using the QE response with IR transmission filter from one of the selected subpixels for long exposure according to the selected color with the IR transmission filter actually applied on it. Such approximations are possible because QE responses are within the IR range to the right of the dashed vertical line 1015 for each of the QE response profiles 1002-1008 as in the non-limiting illustration 1000 from 6 illustrated are similar. Simple interpolations and compensations can be made to approximate the QE response with IR pass filter for the first color (e.g. red) using the QE response with IR pass filter for the selected color (e.g. green) that actually is obtained. Similarly, an approximated QE response with IR pass filter for the short exposure subpixels corresponding to the first color can be obtained using the same method adapted for short exposure subpixels.

$$C_{S\_selected}={\left(I_{nonIR}\right)}_{S\_selected}-{\left(I_{IR\_approx}\right)}_{S\_selected}$$

wobei C_{L_first} und C_{S_first} eine rekonstruktierte Farbinformation für die Subpixel für lange Belichtung bzw. kurze Belichtung entsprechend der ersten Farbe repräsentieren, (I_nonIR)_{L_first} und (I_noIR)_{S_first} eine akkumulierte Intensität von Subpixel für lange bzw. kurze Belichtung entsprechend der ersten Farbe repräsentieren, die keine aufgetragene IR-Durchlassfilterbeschichtung aufweisen, und (I_{IR_approx})_{L_selected} und (I_{IR_approx})_{S_selected} eine approximierte akkumulierte Intensität von Subpixel für lange bzw. kurze Belichtung entsprechend der ersten Farbe repräsentieren, die anzeigt, dass die approximierte IR-Durchlassfilterbeschichtung aufgebracht ist.The approximate QE response with IR transmission filter, which is obtained for each of the subpixels for long exposure and short exposure corresponding to the first color, now contains image sensitivity information within the predetermined wavelength segment, which is separated from color information. Substantially similar to equations [1] and [2], reconstruction of color information for the sub-pixels for long exposure and short exposure for all pixels corresponding to the first color can be expressed as follows. $${\mathrm{C.}}_{L\_first}={\left({\mathrm{I.}}_{nOn\mathrm{I.}\mathrm{R.}}\right)}_{L\_first}-{\left({\mathrm{I.}}_{\mathrm{I.}\mathrm{R.}\_apprOx}\right)}_{L\_selected}$$

$${\mathrm{C.}}_{\mathrm{S.}\_selected}={\left({\mathrm{I.}}_{nOn\mathrm{I.}\mathrm{R.}}\right)}_{\mathrm{S.}\_selected}-{\left({\mathrm{I.}}_{\mathrm{I.}\mathrm{R.}\_apprOx}\right)}_{\mathrm{S.}\_selected}$$

where C _{L_first} and C _{S_first represent} reconstructed color information for the subpixels for long and short exposure corresponding to the first color, (I _nonIR ) _{L_first} and (I _noIR ) _{S_first} an accumulated intensity of subpixels for long and short exposure corresponding to the first Represent colors that have no applied IR transmission _{filter coating} , and (I _{IR_approx} ) _{L_selected} and (I _{IR_approx} ) _{S_selected represent} an approximated accumulated intensity of subpixels for long and short exposure, respectively, corresponding to the first color, which indicates that the approximated IR transmission _{filter coating} is upset.

For the long exposure subpixels corresponding to the first color, the color information within the visible spectral range can be reconstructed using equation [3] by taking the approximated QE response with IR pass filter from an actual response without IR pass filter of a corresponding one of the first Color corresponding subpixels for long exposure is subtracted. Specifically, an accumulated intensity of one of the long exposure subpixels associated with the approximated QE response with IR transmission filter is subtracted from an accumulated intensity of one of the long exposure subpixels that have no IR transmission filter coating applied thereon and the first Color match. block 1106 similarly reconstructs the color information within the visible spectral range for the subpixels 951 for short exposures corresponding to the first color using equation [4].

It goes without saying that the approximated QE response with IR transmission filter corresponding to the first color contains only image sensitivity information within the predetermined range, while the actual response without IR transmission filter corresponding to the first color contains image sensitivity information that is interwoven with the color information. Thus, subtracting the approximate QE response with an IR pass filter from the actual response without an IR pass filter enables image sensitivity information to be removed so that true color information can be obtained by reconstruction for all subpixels for pixels corresponding to the first color (e.g. red).

Referring to block 1108 becomes color information within the predetermined wavelength segment for the subpixels 952 , 951 for long exposure or short exposure for all pixels corresponding to a second color based on the selected subpixels corresponding to the selected color with the IR transmission filter coating applied thereon 975 reconstructed. In the illustrated embodiment of 5 when the selected color includes green, the first color may include red or blue and the second color may include the other of red or blue. block 1108 approximates QE responses with IR pass filter for the long exposure and short exposure subpixels corresponding to the second color using methods adapted for the second color that are substantially identical to those above with reference to that block 1106 have been described.

block 1108 further reconstructs color information of the visible spectral range for the subpixels for long exposure and short exposure corresponding to the second color using equations adapted for the second color that are substantially identical to those described above with reference to equations [3] and [4] from block 1106 have been described. Consequently, the blocks 1106 and 1108 run in parallel.

The exemplary flow chart 1100 from 7th shows that application of the IR transmission filter to selected subpixels enables image sensitivity information that is separate from the color information to be obtained directly in the selected subpixels corresponding to the selected color. Moreover, the true color information in the respective pixels can be reconstructed by subtracting the image sensitivity information from the interwoven information containing both color information and sensitivity information. In addition, the image sensitivity information obtained for the selected pixels can be used to approximate image sensitivity information for the other pixels corresponding to the first or second color. Finally, the true color information in the pixels can be reconstructed corresponding to the first or second color by subtracting the approximated image sensitivity information. Accordingly, improved low light image sensitivity and correct color information can be obtained using the exemplary imaging chip 900 with divided subpixels, which have an application of the IR filter coating on selected subpixels. An added IR night vision could also be used to increase visibility while driving at night, and increased sensitivity in low light can reduce exposure time, thereby reducing motion blur in a captured image. In an exemplary embodiment, this is from the camera device 100 captured image a high dynamic range (HDR) image.

Claims (9)

A method of enhancing image sensitivity and color information for an image captured by a camera device employing a split sub-pixel imaging chip containing a plurality of color-sensitive pixels each comprising a long exposure subpixel and a short exposure subpixel, the Procedure includes: Applying an IR transmission filter coating to selected subpixels, which includes applying the IR transmission filter coating only to selected ones of the long exposure subpixels and the short exposure subpixels for only selected ones of the pixels corresponding to a selected color, the IR transmission filter coating allowing that near infrared light within a predetermined wavelength segment is transmitted through the selected sub-pixels, while transmission of visible light with color information through the selected sub-pixels is blocked; and Reconstructing color information within a visible spectral range for the subpixels for long exposure and short exposure for the plurality of color sensitive pixels based on the applied IR transmission filter coating on the selected subpixels. Procedure according to Claim 1 , wherein reconstructing color information within the visible spectral range for the subpixels for long exposure and short exposure for the plurality of color sensitive pixels comprises: for the subpixels for long exposure for all pixels corresponding to the selected color, reconstructing the color information within the visible spectral range based on a Subtracting a response from one of the selected long exposure sub-pixels having the IR transmission filter coating applied thereon from a response from one of the long exposure sub-pixels having no IR transmission filter coating applied thereto and corresponding to the selected color; and for the short exposure subpixels for all pixels corresponding to the selected color, reconstructing the color information within the visible spectral range based on subtracting a response from one of the selected short exposure subpixels having the IR transmission filter coating applied thereon from a response from one of the subpixels for short exposures that have no IR filter coating applied to them and that correspond to the selected color. Procedure according to Claim 2 , further comprising the step of: reconstructing color information within the visible spectral range for the subpixels for long exposure and short exposure for all pixels corresponding to a first color based on the IR transmission filter applied only to the selected one of the subpixels for long exposure and short exposure is. Procedure according to Claim 3 , being a Reconstructing color information within the visible spectral range for the subpixels for long exposure and short exposure for all pixels corresponding to a first color comprises: for the subpixels for long exposure corresponding to the first color: approximating a response showing that the IR transmission filter is applied thereto using the response from one of the selected sub-pixels for long exposure with the IR transmission filter coating applied thereon; Reconstructing the color information within the predetermined wavelength segment based on subtracting the approximated response from an actual response of a corresponding one of the long exposure sub-pixels corresponding to the first color; for the short exposure sub-pixels corresponding to the first color: approximating a response showing that the IR transmission filter has been applied thereto using the response from one of the selected short exposure sub-pixels with the IR transmission filter coating applied thereon; and reconstructing the color information within the visible spectral range based on subtracting the approximated response from an actual response of a corresponding one of the short exposure sub-pixels corresponding to the first color. Procedure according to Claim 3 , further comprising the step of: reconstructing color information within the visible spectral range for the subpixels for long exposure and short exposure for all pixels corresponding to a second color based on the IR transmission filter applied only to the selected ones of the subpixels for long exposure and short exposure has been. Procedure according to Claim 5 comprising: for the long exposure sub-pixels responsive to the second color: approximating a response showing that the IR transmission filter has been applied thereto using the response from one of the selected long exposure sub-pixels having the IR applied thereto -Transmission filter coating; Reconstructing the color information within the visible spectral range based on subtracting the approximated response from an actual response of a corresponding one of the long exposure sub-pixels corresponding to the second color; for the short exposure sub-pixels corresponding to the second color: approximating a response showing that the IR transmission filter was applied thereto using the response from one of the selected short exposure sub-pixels with the IR transmission filter coating applied thereon; and reconstructing the color information within the visible spectral range based on subtracting the approximate response from an actual response of a corresponding one of the short exposure sub-pixels corresponding to the second color. Procedure according to Claim 5 wherein the selected color, the first color and the second color each include different ones of green, red and blue. Procedure according to Claim 1 , where the selected color comprises green. Procedure according to Claim 1 wherein the predetermined wavelength portion comprises one of the following: all wavelengths exceeding a predetermined wavelength; and all wavelengths within a predetermined wavelength range.

Images