WO2015008383A1

WO2015008383A1 - Imaging device

Info

Publication number: WO2015008383A1
Application number: PCT/JP2013/069652
Authority: WO
Inventors: 朋和石原; 西澤　明仁; 吉田　大輔
Original assignee: 日立マクセル株式会社
Priority date: 2013-07-19
Filing date: 2013-07-19
Publication date: 2015-01-22

Abstract

Provided is an imaging device that obtains a suitable video signal by using an imaging unit that has pixels which have sensitivity in the visible light region and the near-infrared region, and pixels which have sensitivity in the near-infrared region. This image device has: an imaging unit having the aforementioned two types of pixels; a visible light gain setting unit that changes the photoelectric conversion gain with respect to the amount of light incident to the aforementioned visible light region pixels; a near-infrared light gain setting unit that changes the photoelectric conversion gain with respect to the amount of light incident to the aforementioned near-infrared light region pixels; a control unit that controls the aforementioned two variable light gain units; and a signal generation unit that generates a video signal from the outputs of the two types of pixels. For example, the near-infrared light gain setting unit is an aperture mechanism having a filter that prevents the transmission of near-infrared light.

Description

Imaging device

The present invention relates to an imaging apparatus.

There is the following Patent Document 1 as background art in this technical field. In the summary of the publication, as a subject, “exposure exposure control device that controls the exposure amount when photographing with visible light and infrared light simultaneously is performed, and a preferable exposure for both visible light system and infrared light system is achieved with a simple configuration. As a means for solving the problem, “a visible light imaging unit that receives visible light separated from incident light that enters through a diaphragm mechanism, and infrared light that is separated from incident light are used. An exposure condition determination unit that determines an appropriate exposure condition for each infrared imaging unit that receives light, an aperture value setting unit that sets an aperture value of an aperture mechanism based on each appropriate exposure condition, and an appropriateness of a visible light imaging unit Based on the exposure condition and the set aperture value, based on the visible light system shutter speed setting unit for setting the exposure time in the visible light imaging unit, the appropriate exposure condition of the infrared light imaging unit and the set aperture value, Infrared light And a infrared optics shutter speed setting unit that sets the exposure time in the image section. Has been described as ".

JP 2010-183191 A

For example, in an in-vehicle imaging device, the visible light irradiated by the vehicle with a moderate brightness in the front and slightly downward is received by receiving the reflected light of the near-infrared light emitted by the car in front of the front with high brightness. There is an imaging device that receives the reflected light of the night and captures the situation in the vicinity of the night. Since high-luminance visible light is not irradiated in front of the front, there is a feature that the driver of the oncoming vehicle is not put in a dangerous state even when it is desired to clearly capture a distant situation. For this purpose, the image pickup device of the image pickup device has a total of four pixels, that is, a pixel of three primary color signals of R, G, and B and a pixel of an IR signal (near infrared signal; hereinafter also referred to as I signal), or a near infrared signal. The pixel group includes a total of four pixels including the three primary color signals R + IR, G + IR, and B + IR and the IR signal.

In the imaging device described above, the primary color signal or the IR signal at the output of the imaging device may be saturated depending on the intensity of the reflected light. For example, if any of the primary color signals is saturated, it is a matter of course that good colors cannot be reproduced. On the other hand, if the IR signal is saturated, there is a big problem that whiteout occurs in the luminance signal. With the configuration shown in Patent Document 1, there is a method of improving these problems by separately processing the IR signal and the three primary color signals that are visible light.

However, since Patent Document 1 has an imaging unit that receives visible light and an imaging unit that receives infrared light, there is room for improvement in terms of practical cost.

An object of the present invention is to provide an imaging device in which, for example, brightness signal whiteout is reduced in view of the above-described problems.

In order to achieve the above object, the configuration described in the claims is adopted.

According to the present invention, it is possible to provide an imaging device with reduced whiteness of the luminance signal, which can contribute to the improvement of the basic performance of the imaging device.

Other effects according to the embodiment will be described below.

1 is a configuration diagram of an imaging device in Embodiment 1. FIG. FIG. 4 is a diagram illustrating an example of an arrangement of pixels of an imaging unit in Embodiments 1 to 3. FIG. 4 is a diagram illustrating an example of spectral characteristics of pixels of an imaging unit in Embodiments 1 to 3. FIG. 4 is a diagram illustrating an example of spectral characteristics of pixels of an imaging unit in Embodiments 1 to 3. FIG. 4 is a diagram illustrating an example of an arrangement of pixels of an imaging unit in Embodiments 1 to 3. FIG. 4 is a diagram illustrating an example of spectral characteristics of pixels of an imaging unit in Embodiments 1 to 3. FIG. 4 is a diagram illustrating an example of spectral characteristics of pixels of an imaging unit in Embodiments 1 to 3. FIG. 6 is a configuration diagram of an imaging unit in Embodiment 2. FIG. 9 is a configuration diagram of an imaging unit in Embodiment 3.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a configuration diagram of an imaging apparatus 100 according to the first embodiment.
The imaging device 100 is an imaging device according to the present invention.

Reference numeral 101 denotes an imaging unit, which is a lens that forms an image of light coming from a subject, a diaphragm, and a pixel having sensitivity in both the visible region and the near-infrared region on the same element, and a red region + near-infrared region (hereinafter referred to as R + I). ) Pixel, green region + near infrared region (hereinafter referred to as G + I) pixel, blue region + near infrared region (hereinafter referred to as B + I) pixel, and I pixel that is a pixel having sensitivity in the near infrared region Have Each pixel of the imaging element can set the exposure time of the light with an electronic shutter that sets a charge accumulation time for photoelectric conversion. Each pixel output is subjected to photoelectric conversion and A / D conversion in accordance with the amount of light imaged by the lens 101, and is output as digital data. Details of the pixel configuration of the image sensor of the imaging unit 102 will be described later.

Reference numeral 102 denotes a color signal processing circuit, which performs demosaicing using signals obtained from the positions of (R + I) pixel (G + I) pixel, (B + I) pixel, and I pixel, and mixes them in accordance with the spectral characteristics of the image sensor 102. The red, green, and blue signals in each pixel are generated based on the ratio, and further, white balance processing is performed to generate R, G, and B signals that are final color signals.

A gamma processing circuit 103 performs gamma correction processing on the R, G, and B signals generated by the color signal processing circuit 104. As is well known, gamma correction is performed to correct response characteristics of an external display device.
A color difference signal processing circuit 104 generates an image color difference signal from the output of the gamma processing circuit 103.

A luminance signal processing circuit 105 performs demosaicing on the (R + I) pixel, (G + I) pixel, (B + I) pixel, and I pixel signals of the imaging unit 101 in order to generate a luminance signal. A luminance signal is generated with a mixing ratio that matches the spectral characteristics of the image sensor 102.
Reference numeral 111 denotes a visible light amount detection circuit that detects signal amounts of (R + I) pixels, (G + I) pixels, and (B + I) pixels of the imaging unit 101.

A near-infrared luminance signal processing circuit 106 generates a luminance signal obtained from the near-infrared signal from the signal of the I pixel of the imaging unit 101.
Reference numeral 112 denotes a near-infrared light amount detection circuit that detects the amount of near-infrared light from the output of the invisible light luminance signal processing circuit.

An image synthesis processing circuit 107 synthesizes the outputs of the luminance signal processing circuit 105 and the near infrared luminance signal processing circuit 106.
A luminance gamma processing circuit 108 performs gamma correction processing on the output of the image composition processing circuit 107.

An image output processing unit 109 outputs the luminance signal output of the luminance gamma processing circuit 108 and the color difference signal output of the color difference signal processing circuit 104 in a predetermined signal format (for example, HDMI: High Definition Multimedia Interface).
Reference numeral 110 denotes a control unit that controls the exposure amount of the imaging unit based on the detection output from each component of the imaging device 100.

The signal output of each pixel of visible light and near-infrared light received by the imaging unit 101 is converted into a color signal by the color signal processing circuit 102 and further subjected to gamma correction processing by the gamma processing circuit 103 to generate a color difference. The circuit 104 processes the color difference signal of the color video signal.

At the same time, the luminance signal processing circuit 105 generates a luminance signal of a visible light component based on the signal output of each pixel of visible light and near infrared light received by the imaging unit 101, and the near infrared luminance signal processing circuit 106. Thus, the luminance signal of the near infrared signal component is generated.

In the image composition processing unit 107, the luminance signal generated from the visible light component by the luminance signal processing circuit 105 and the luminance signal component generated from the near infrared signal component by the near infrared luminance signal processing circuit 106 are combined. . Therefore, in the imaging apparatus 100, unlike a camera having sensitivity only to normal visible light, in order to generate a luminance signal that also includes a near-infrared component, an object having a small amount of visible light but a near-infrared component is selected. It can be captured as a video. The output of the image composition processing circuit 107 is a video luminance signal that has been gamma corrected by the luminance gamma processing circuit 108.

The color difference signal of the color video signal processed by the color difference generation circuit 104 and the luminance signal processed by the luminance gamma processing circuit 108 are output in a predetermined video output signal form by the image output processing unit 109.

On the other hand, the signal amount of the luminance signal of the visible light component processed by the luminance signal processing circuit 105 is detected by the visible light amount detection circuit 111, and the light amount of the near infrared light processed by the near infrared luminance signal processing circuit 106 is near. Detection is performed by the infrared light amount detection circuit 112. The control unit 110 keeps the light amount of the visible light and the light amount of the near infrared light in the imaging unit 101 optimally according to the outputs of the visible light amount detection circuit 111 and the near infrared light amount detection circuit 112. At this time, the light amount of visible light and the light amount of near-infrared light are independently controlled in accordance with the signal amount photoelectrically converted by the image sensor of the imaging unit 101.

For this reason, the influence of the difference in the light amount distribution between the visible light and the near infrared light of the subject is reduced, and an image signal having a good balance between the visible light and the near infrared light can be obtained. As will be described later, the amount of light is controlled by an electronic shutter that controls the charge accumulation time in the image sensor, or by an optical diaphragm mechanism such as a diaphragm blade.

Next, the imaging unit 101 in the present embodiment will be described.
FIG. 2 is a diagram illustrating an example of an arrangement of pixels of the imaging unit in the first to third embodiments, and illustrates an example of a pixel arrangement of visible light + near infrared sensor.

A pixel 201 having a main sensitivity for R, a pixel 202 having a main sensitivity for G, a pixel 203 having a main sensitivity for B, and a pixel 204 having a main sensitivity for I are arranged on the same image sensor. These are arranged in a grid as an example. With the arrangement of the pixels 201 to 204, pixels are repeatedly formed on the image sensor. Further, the exposure time of each pixel can be controlled by an electronic shutter by setting from the outside (for example, the control unit in FIG. 1).

FIG. 3 is a diagram illustrating an example of the spectral characteristics of the pixels of the imaging unit according to the first to third embodiments. The sensitivity characteristics, that is, spectral characteristics, of the pixels 201 to 204 shown in FIG. Is. In the figure, 301 is the spectral characteristic of the pixel 203 (FIG. 2), 302 is the spectral characteristic of the pixel 202 (FIG. 2), 303 is the spectral characteristic of the pixel 201 (FIG. 2), and 304 is the pixel 204 (FIG. 2). FIG. 2) is a spectral characteristic. As shown by the spectral characteristics 301 to 303, each has sensitivity in the wavelength region of I in addition to the wavelength region of visible light of R, G, and B.

A normal camera with only visible light range is composed of pixels with these spectral characteristics. Usually, in order to image only the visible light region, that is, to eliminate the influence of the I portion, an optical filter that blocks the wavelength region of the near-infrared light region (I) is light between the image sensor and the lens. Inserted on the axis and used. The spectral characteristic 304 has sensitivity only in the wavelength region of I, and by having this pixel together with the pixel having sensitivity in the visible light region, the color component and luminance component of the visible light region (R, G, B), A luminance component by the near-infrared light region (I) is generated at the same time. The image pickup device having this pixel configuration has a feature that the sensitivity of near infrared light can be increased in addition to visible light sensitivity because each pixel has sensitivity to near infrared light.

FIG. 4 is a diagram illustrating an example of the spectral characteristics of the pixels of the imaging unit according to the first to third embodiments. FIG. 3 illustrates sensitivity characteristics, that is, spectral characteristics, of the pixels 201 to 204 shown in FIG. An example different from FIG. In the figure, 401 is a spectral characteristic of a different example of the pixel 203 (FIG. 2), 402 is a spectral characteristic of a different example of the pixel 202 (FIG. 2), and 403 is a spectral characteristic of a different example of the pixel 201 (FIG. 2). 404 is a spectral characteristic of a different example of the pixel 204 (FIG. 2). As indicated by the spectral characteristics 401 to 403, each has sensitivity only in the wavelength range that is visible light of R, G, and B.

The spectral characteristic 404 is sensitive only to I. By providing this pixel together with the pixel having the sensitivity of the visible light region, the color component and the luminance component of the visible light region (R, G, B), A luminance component by the near-infrared light region (I) is generated at the same time. In the image sensor with this pixel configuration, the output of the R, G, and B pixels is a visible light component. For this reason, there is a feature that the sensitivity of a color image can be increased using a near-infrared light component while using a signal processing method for processing a signal obtained by cutting off near-infrared light. Alternatively, a color image faithful to the color of the subject can be visually generated using a signal processing method that processes a signal obtained by cutting off near-infrared light.

FIG. 5 is a diagram illustrating an example of an arrangement of pixels of the imaging unit in the first to third embodiments, and illustrates an example of a pixel arrangement of visible light + near infrared sensor, which is different from FIG. On the same image sensor, a pixel 501 having a main sensitivity for R, a pixel 502 を持つ having a main sensitivity for G, a pixel 503 having a main sensitivity for B, and a pixel 504 having all the sensitivities R, G, B, and I Are arranged in a grid pattern. Repeated pixels are formed on the image sensor with the arrangement of the pixels 501 to 504.

FIG. 6 is a diagram illustrating an example of the spectral characteristics of the pixels of the imaging unit according to the first to third embodiments. The sensitivity characteristics, that is, the spectral characteristics, of the pixels 501 to 504 illustrated in FIG. 5 with respect to the wavelength of light are illustrated. Is. In the drawing, 601 is the spectral characteristic of the pixel 503 (FIG. 5), 602 is the spectral characteristic of the pixel 502 (FIG. 2), 603 is the spectral characteristic of the pixel 501 (FIG. 5), and 604 is the pixel 604 ( FIG. 2) is a spectral characteristic. The

spectral characteristics

601, 602, and 603 have sensitivity in the wavelength range of I in addition to the wavelength range of visible light of R, G, and B, respectively.

A normal camera with only a visible light region is composed of pixels having these spectral characteristics. Normally, in order to image only the visible light region, that is, to eliminate the influence of the I portion, an optical filter that blocks the I wavelength region is inserted and used on the optical axis between the image sensor and the lens. Yes. The spectral characteristic 604 has sensitivity to all of R, G, B, and I. By providing this pixel with the above-mentioned sensitivity of the visible light region, the visible light region (R, G, B) A color component, a luminance component, and a luminance component due to I are generated simultaneously. An image sensor with this pixel configuration is suitable for applications that place particular emphasis on visible light sensitivity during imaging with little visible light.

FIG. 7 is a diagram illustrating an example of spectral characteristics of the pixels of the imaging unit according to the first to third embodiments. FIG. 6 illustrates sensitivity characteristics, that is, spectral characteristics, of the pixels 501 to 504 illustrated in FIG. An example different from FIG. In the figure, reference numeral 701 denotes a different spectral characteristic of the pixel 503 (FIG. 5), reference numeral 702 denotes a different spectral characteristic of the pixel 502 (FIG. 5), and reference numeral 703 denotes a different spectral characteristic of the pixel 501 (FIG. 5). 704 is a spectral characteristic of a different example of the pixel 504 (FIG. 5). Spectral characteristics 701 to 703 have sensitivity only in the wavelength range of visible light of R, G, and B, respectively.

The spectral characteristic 704 has sensitivity to all of R, G, B, and I. By providing this pixel together with the pixel having sensitivity in the visible light region, the color component of the visible light region (R, G, B) and A luminance component and a luminance component due to the near-infrared light region (I) are generated simultaneously. In the image pickup device having this pixel configuration, the R, G, and B pixels have only visible light components and have pixels that are sensitive to all visible light and near infrared. Using a signal processing method for processing a signal obtained by increasing the sensitivity of infrared light or cutting off near-infrared light, it is suitable for use in generating a color image faithful to the color of a subject visually.

FIG. 8 is a configuration diagram of the imaging unit in the second embodiment, and shows details of the imaging unit 101 shown in FIG. In the figure, reference numeral 801 denotes a lens, and reference numeral 802 denotes a stop. The stop 802 is driven and controlled by the control unit 110 (FIG. 1) via a drive circuit 804. Reference numeral 803 denotes an image sensor having, for example, the configuration shown in FIGS. The control unit 110 controls the exposure time of the image sensor 803, that is, the shutter time set for the electronic shutter so that the shutter time of all the pixels is the same. Here, the diaphragm 802 is made of a material that transmits visible light and blocks near-infrared light. For example, a diaphragm blade made of the same material as an IR blocking filter for blocking near-infrared light is used. Have.

With this configuration, visible light is controlled by the electronic shutter of the image sensor 803, and near-infrared light is controlled individually by the control unit 110 (FIG. 1) in a well-balanced manner. Therefore, it is possible to obtain a good image without overexposure on a subject that is irradiated with near infrared light, for example, outdoors at night. In this configuration, the amount of light is controlled by electronic shutter control for visible light, and the amount of light for near infrared light is controlled by aperture control, which has better light-shielding performance than the electronic shutter. A more suitable image can be obtained in an imaging environment with a large amount of infrared light.

FIG. 9 is a configuration diagram of the image pickup unit in the third embodiment, and shows details of the image pickup unit 101 shown in FIG. In the figure, a lens 801, an image sensor 803, and a drive circuit 804 are the same as those shown in FIG.

Reference numeral 901 denotes a diaphragm, which is driven and controlled by the control unit 110 (FIG. 1) via the drive circuit 804. Here, the diaphragm 901 is made of a material that transmits near-infrared light and cuts visible light. For example, the diaphragm 901 has diaphragm blades made of the same material as a visible light blocking filter for blocking visible light. is doing.

With this configuration, near infrared light is controlled by the electronic shutter of the image sensor 803, and visible light is controlled individually by the control unit 110 (FIG. 1) in a balanced manner by the diaphragm 901. Therefore, it is possible to obtain a good image without overexposure on a subject that is irradiated with visible light, for example, outdoors at night. In this configuration, the near-infrared light is controlled by the electronic shutter, and the visible light is controlled by the aperture control that has a better light shielding performance than the electronic shutter. It is possible to obtain a more suitable image in an imaging environment in which imaging is performed by effectively using these components.

In Example 2 and Example 3, the electronic shutter of the image sensor and the lens aperture can be referred to as a gain setting unit for changing the photoelectric conversion gain with respect to the amount of light incident on the pixel.

In any of the embodiments, when a color image is captured using an imaging unit having a pixel having sensitivity in the visible light region and a pixel having sensitivity in the near infrared region, the visible light region and the near red region are used. It is possible to provide an imaging apparatus that individually and suitably controls the exposure amounts of both the outer regions. Although the most visually obvious effect is reduction of whiteout, it has an effect of obtaining a good image, for example, a color image faithful to the color of the subject can be visually generated. Furthermore, unlike the above-mentioned patent documents, there is no need for a separate imaging unit that receives visible light and an imaging unit that receives infrared light, so there are few practical problems related to cost.

8 and 9, only one lens 801 is shown, but as is well known, it generally has a plurality of lenses including a convex lens and a concave lens, and these lenses are positioned inside the lens barrel. ing. The

diaphragms

802 and 901 may be positioned so as to be sandwiched between the plurality of lenses. Further, in a conventional imaging apparatus, a mechanical shutter for reinforcing the function of the electronic shutter of the imaging apparatus may be provided in the lens barrel. The

diaphragms

802 and 901 in each embodiment may be provided in a lens gap different from the mechanical shutter in the gap between a plurality of lenses, or may be provided in the same gap.

The embodiments described above do not limit the present invention. For example, both the diaphragm 802 by the IR blocking filter in the second embodiment and the diaphragm 901 by the visible light blocking filter in the third embodiment are provided, and the light amounts of both near infrared light and visible light are individually controlled. Also good. As described above, embodiments in which various modifications are made to the embodiments disclosed so far can be considered, and all are within the scope of the present invention.

DESCRIPTION OF SYMBOLS 100: Imaging device, 101: Imaging part, 102: Color signal processing circuit, 103: Gamma processing circuit, 104: Color difference generation processing circuit, 105: Luminance signal processing circuit, 106: Near-infrared luminance signal processing circuit, 107: Image Synthesis processing circuit, 108: luminance gamma processing circuit, 109: image output processing unit, 110: control unit, 111: visible light amount detection circuit, 112: near infrared light amount detection circuit, 801: lens, 802: diaphragm, 803: imaging Element, 804: drive circuit, 901: aperture.

Claims

An imaging device that images the subject based on incident light incident from the subject and generates a video signal,
A visible light pixel that converts incident visible light and outputs a signal according to the amount of visible light and a near red that converts incident near infrared light and outputs a signal according to the amount of near infrared light An imaging unit including external light pixels;
A visible light gain setting unit for setting a photoelectric conversion gain with respect to the amount of incident visible light in the visible light pixel;
A near-infrared light gain setting unit for setting a photoelectric conversion gain for the amount of incident near-infrared light in the near-infrared light pixel;
The photoelectric conversion gain in the visible light gain setting unit is controlled based on a signal output from the visible light pixel, and the photoelectric conversion gain in the near infrared light gain setting unit is controlled based on a signal output from the near infrared light pixel. A control unit for controlling
An imaging apparatus comprising: a signal processing unit that processes the signal output from the visible light pixel and the signal output from the near-infrared light pixel to generate the video signal.
The imaging device according to claim 1,
The said visible light pixel outputs the signal according to the light quantity of the incident near infrared light with the signal according to the light quantity of the said visible light. The imaging device characterized by the above-mentioned.
The imaging device according to claim 1,
The said near-infrared light pixel outputs the signal according to the light quantity of the incident visible light with the signal according to the light quantity of the said near-infrared light. The imaging device characterized by the above-mentioned.
The imaging device according to claim 3.
The said visible light pixel outputs the signal according to the light quantity of the incident near infrared light with the signal according to the light quantity of the said visible light. The imaging device characterized by the above-mentioned.
The imaging device according to claim 1,
The visible light gain setting unit has an electronic shutter for setting a charge accumulation time for photoelectric conversion in the visible light pixel,
The near-infrared light gain setting unit has an optical stop for limiting the amount of near-infrared light incident on the near-infrared light pixel.
The imaging device according to claim 1,
The visible light gain setting unit has an optical aperture for limiting the amount of light incident on the visible light pixel,
The near-infrared light gain setting unit includes an electronic shutter that sets a charge accumulation time for photoelectric conversion in the near-infrared light pixel.
The imaging apparatus according to claim 5,
The visible light gain setting unit has an optical aperture for limiting the amount of light incident on the visible light pixel.