JP6348254B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus.

  As a background art in this technical field, there is JP 2008-92247 A (Patent Document 1). In this publication, “[Problem] To improve the white balance of a color image output from a solid-state image sensor. [Solution] A plurality of photoelectric conversion elements 20 arranged in a matrix and having three different transmissions. Any one of the color filters 22R, 22G, and 22B having the center wavelength and the near-infrared light filter 22IR having the transmission center frequency in the near-infrared light region is disposed on each light receiving surface of the photoelectric conversion element 20, and the photoelectric conversion element The above problem can be solved by a solid-state imaging device in which any one of the color filters 22R, 22G, and 22B and the near-infrared light filter 22IR are arranged in each column of the 20 matrixes. " Yes.

JP 2008-92247 A

  There are surveillance cameras and in-vehicle cameras as main applications of the imaging apparatus. In both cases, there is a strong demand for clear shooting of a target subject even when it gets dark. For this problem, improvement by utilizing not only the light in the visible light region but also the near infrared region is described. However, when either the light amount in the visible light region or the light amount in the non-visible light region is insufficient, there is a problem that noise is biased. The present invention solves the above problems and provides an imaging apparatus capable of suppressing S / N degradation even when the near infrared region is utilized.

Of the inventions disclosed in the present application, the outline of typical ones will be briefly described as follows.
(1) a first pixel group including a first filter having a predetermined transmission center wavelength, and a second pixel group including a second filter having a transmission center wavelength different from the first filter; , And an exposure control processing unit that controls an exposure time independently for each of the first pixel group and the second pixel group of the imaging unit. An imaging device.

  According to the present invention, it is possible to provide an imaging apparatus capable of suppressing S / N degradation.

1 is an overall configuration schematic diagram of a first embodiment of an imaging apparatus according to the present invention. It is a figure which shows the image pick-up element of a 1st Example. It is a figure which shows the structure for the exposure time individual adjustment in a 1st Example. It is a figure which shows the image pick-up element of a 2nd Example. It is a figure which shows the structure for the exposure time individual adjustment in a 2nd Example. It is a whole block schematic diagram of the 3rd example of an imaging device concerning the present invention. It is a figure which shows an example of the sensitivity characteristic of each pixel of R, G, B. It is a figure which shows an example of the wavelength distribution of a light source. It is a figure which shows an example of the output characteristic of each pixel of R, G, B. It is a figure which shows an example of the sensitivity characteristic of each pixel of R, G, B, and Ir. It is a figure which shows the other example of wavelength distribution of a light source. It is a figure which shows an example of the output characteristic of each pixel of R, G, B, and Ir. It is a whole block schematic diagram of the 2nd example of an imaging device concerning the present invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a schematic diagram of the overall configuration of a first embodiment of an imaging apparatus according to the present invention. The imaging apparatus according to the present embodiment is configured by appropriately using an imaging unit 101, an image processing unit 102, a determination unit 103, and an exposure control processing unit 104.

  For example, as shown in FIG. 2, the imaging unit 101 includes an “R” filter having sensitivity to a red wavelength and a “G” filter having sensitivity to a green wavelength (G1 and G2 in the drawing). However, in the following description, each signal will be defined as G), and “B” filters with sensitivity to blue are arranged in a checkered pattern, and each color signal is assigned by these color filters. Assume that the obtained single-plate color imaging device has the sensitivity characteristics of the R, G, and B pixels as shown in FIG. Here, the wavelength distribution of the light source irradiated to the subject varies depending on the type of the light source.For example, when a white subject is photographed under the light source having the distribution shown in FIG. 8, the output characteristics of the R, G, and B pixels are shown in FIG. 7 is multiplied by FIG. 8, resulting in FIG. In order to output a white subject photographed under these conditions as a white image, for example, the signal level of B is reduced, gain correction is applied to increase the signal level of R, and each signal level is set to R = G = It can be realized by processing so that it becomes B. However, a noise signal is superimposed on a signal before gain correction, and there is a problem that noise is emphasized particularly in a color signal subjected to gain correction so as to increase the signal level. Also, if any of R, G, B pixels is in a charge saturation state (when the video is whiteout), lowering the gain will lower the signal level of the saturated pixel, but will not improve the whiteout of the video There was also a problem.

  In order to solve the above-described problem, in this embodiment, the imaging unit 101 can individually adjust the exposure time for each of R, G, and B pixels, and the determination unit 103 is input from the imaging unit 101. The ratio of the signal level of each R, G, B signal shall be obtained, and the exposure control processing unit 104 performs imaging so that the ratio of the signal level of each R, G, B signal obtained from the determination unit 103 is constant. The exposure time for each of the R, G, and B pixels of the unit 101 is adjusted individually.

In the above configuration, the ratio of the signal level of each R, G, B signal is obtained, for example, within a predetermined image area as a signal representing white of the subject for each R, G, B signal from the imaging unit 101 The average value AveR, AveG, AveB is obtained, and the signal level ratio is constant, for example, so that AveR = AveG = AveB
R pixel exposure time = AveG / AveR × G exposure time
B pixel exposure time = AveG / AveB × G exposure time. For example, when the condition of AveB>AveG> AveR is satisfied in the photographed image, the wavelength distribution of the light source has the characteristics as shown in FIG. Thus, the signal level of the B pixel can be reduced, and the exposure time of the R pixel can be lengthened to increase the signal level of the R pixel. Further, since the exposure time is controlled, the amount of noise superimposed by the imaging unit 101 does not change. In addition, when the B pixel having a high signal level is in a charge saturation state, the exposure time can be shortened so that the charge is not saturated, and whiteout of an image can be suppressed. In this embodiment, the average value in the predetermined image area is obtained as a signal representing the white of the subject. However, a method of extracting only a predetermined signal level in the predetermined image area may be used. . Further, if a method of recognizing an object such as a sign that seems to be white of a subject and extracting a pixel of the object, it is possible to extract the white of the subject with high accuracy.

  Next, an example of a specific configuration for independently adjusting the exposure time for each of R, G, and B pixels in the above embodiment will be described with reference to FIG. The imaging unit 101 includes a reset processing unit 301 and a readout processing unit 302, and R determined by the exposure control processing unit 104 based on a ratio of signal levels of R, G, and B pixels from the determination unit 103. , G, B based on the exposure time information of each pixel, the reset processing unit 301 generates a charge reset signal and the read processing unit 302 generates a charge readout signal, and these signals are used to generate R, G, B Each pixel is controlled individually. As described above, the exposure time can be individually adjusted by transmitting and controlling the charge reset signal and the charge readout signal corresponding to each of the R, G, and B pixels. Here, although not shown in the figure, when the global shutter system is realized, for example, the same charge reset signal and the same charge readout signal are applied to all the R pixels arranged on the imaging unit 101. Send it. When the rolling shutter system is realized, for example, a charge reset signal and a charge readout signal that are shifted by a predetermined time may be sent to all the R pixels arranged on the imaging unit 101. The G pixel and B pixel are the same as the R pixel. If this method is adopted, the charge reset signal and charge read signal generation can be simplified, and the circuit scale can be reduced. In addition, the wiring can be simplified, and the problem that the wiring layer blocks the passage of light and reduces the aperture ratio of the pixel can be suppressed.

  In addition, although pixels having a transmission center wavelength for each of R, G, and B in the visible light region are defined as R, G, and B, each pixel may be sensitive to invisible light, For example, it may be an R + IR pixel, a G + IR pixel, or a B + IR pixel. When such an imaging element is used for the imaging unit 101, the determination unit 103 may obtain a signal level ratio after separating R, G, and B by matrix calculation.

  FIG. 13 is a schematic diagram of the overall configuration of a second embodiment of the imaging apparatus according to the present invention. As in the first embodiment, the imaging apparatus according to the present embodiment is configured by appropriately using the imaging unit 1301, the image processing unit 102, the determination unit 103, and the exposure control processing unit 104. The configuration of the unit 1301 and the processing in the determination unit 103 and the exposure control processing unit 104 are different from those in the first embodiment.

  For example, as illustrated in FIG. 4, the imaging unit 1301 has an “R” filter having sensitivity to a red wavelength, a “G” filter having sensitivity to a green wavelength, and blue having sensitivity to an upper portion of the photoelectric conversion element. The “B” filter and the “IR” filter having sensitivity in the infrared or near-infrared non-visible light region are arranged in a checkered pattern to obtain a single-plate color image pickup device that obtains four types of color signals. , G, B, and IR are assumed to have the sensitivity characteristics shown in FIG. 10, for example. Here, the wavelength distribution of the light source irradiated to the subject varies depending on the type of the light source. For example, when a white subject is photographed under the light source having the distribution shown in FIG. 11, the output characteristics of the R, G, B, and IR pixels. Is a product of FIG. 10 and FIG. 11, resulting in FIG. In order to sufficiently secure the image level of the visible light region and the signal level of the non-visible light region captured under these conditions, for example, the signal level of R, G, B pixels having sensitivity in the visible light region is reduced, This can be realized by applying gain correction to increase the signal level of the IR pixel having sensitivity in the non-visible light region and processing each signal level so that R = G = B = IR. However, a noise signal is superimposed on a signal before gain correction, and there is a problem that noise of an IR pixel to which gain is applied to increase the signal level is particularly emphasized. Also, if any of R, G, B pixels is in a charge saturation state (when the video is whiteout), lowering the gain will lower the signal level of the saturated pixel, but will not improve the whiteout of the video There was also a problem.

  In order to solve the above-described problem, in this embodiment, the imaging unit 1301 can individually adjust the exposure time for pixels having sensitivity in the visible light region and pixels having sensitivity in the invisible light region. The unit 103 obtains a signal level ratio between at least one of R, G, and B signals having sensitivity in the visible light region input from the imaging unit 1301 and an IR signal having sensitivity in the invisible light region. The exposure control processing unit 104 individually adjusts the exposure time for each of R, G, B pixels and IR pixels of the imaging unit 1301 so that the ratio of the signal levels obtained from the determination unit 103 is constant. To do.

In the above configuration, the signal level ratio is obtained by obtaining average values AveG and AveIR in a predetermined image area as a signal representing white of the subject using, for example, each G signal and IR signal from the imaging unit 1301. For example, AveG = AveIR so that the signal level ratio is constant.
IR pixel exposure time = AveG / AveIR x G exposure time
Adjustment may be made such that R exposure time = G exposure time = B exposure time. For example, when the condition of AveG> AveIR is satisfied in the photographed image, the wavelength distribution of the light source has the characteristics as shown in FIG. The signal level of the IR pixel can be increased by extending the exposure time. Further, since the exposure time is controlled, the amount of noise superimposed by the imaging unit 101 does not change. In addition, when R, G, B pixels with high signal levels are in a charge saturation state, if the exposure time for R, G, B is shortened, the charge saturation can be prevented, and whiteout of the image is suppressed. It becomes possible to do. In this embodiment, the average value in the predetermined image area is obtained. However, a method of extracting only a predetermined signal level in the predetermined image area may be used. In addition, if a method of recognizing a predetermined object such as a sign of a subject and extracting a pixel of the object is used, if the reflection coefficients of visible light and invisible light are known for each object, the object is visible with high accuracy. The distribution of light and invisible light can be extracted.

  Next, an example of a specific configuration for independently adjusting the exposure time for the pixel having sensitivity in the visible light region and the pixel having sensitivity in the non-visible light region in the above embodiment will be described with reference to FIG. Will be described. Similar to the first embodiment, the imaging unit 1301 includes a reset processing unit 301 and a readout processing unit 302, and visible light individually determined by the exposure control processing unit 104 based on the signal level ratio from the determination unit 103. Based on the information on the exposure time of each R, G, B pixel having sensitivity in the region and the exposure time of the IR pixel having sensitivity in the invisible light region, the reset processing unit 301 reads out the charge reset signal, and reads out the processing unit 302. A charge readout signal is generated at, and each pixel is individually controlled using these signals. In the first embodiment, R, G, and B are individually controlled, whereas in this embodiment, R, G, and B have the same exposure time, thereby simplifying the generation of the charge reset signal and the charge readout signal. The circuit scale can be reduced. In addition, the wiring can be simplified, and the problem that the wiring layer blocks the passage of light and reduces the aperture ratio of the pixel can be suppressed.

  Further, the sensitivity characteristics of the visible light pixel and the invisible light pixel are assumed to be at the same level as shown in FIG. 10, but in the case of using a general image sensor such as a CMOS sensor or a CCD sensor, the visible characteristic is visible. The sensitivity of the invisible light pixel may be low with respect to the light pixel. Even when such an image sensor is used, adjustment can be made so that the level of each sensitivity characteristic is made uniform by changing the exposure time of visible light pixels and non-visible light pixels as in this embodiment. Can do.

  In addition, although pixels having a transmission center wavelength for each of R, G, and B in the visible light region are defined as R, G, and B, each pixel may be sensitive to invisible light, For example, it may be an R + IR pixel, a G + IR pixel, or a B + IR pixel. In addition, although a pixel having a transmission center wavelength in the infrared or near-infrared invisible light region is defined as IR, it may be sensitive to visible light, for example, an R + G + B + IR pixel. It may be. When such an imaging element is used for the imaging unit 1301, the determination unit 103 may obtain a signal level ratio after separating R, G, B, and IR by matrix calculation.

  FIG. 6 is a schematic diagram of the overall configuration of the third embodiment of the imaging apparatus according to the present invention. The imaging apparatus according to the present embodiment includes an imaging unit 1301, an image processing unit 102, a non-visible light projection unit 601, a visible light projection unit 602, a table unit 603, and an exposure control processing unit 104. Configured as appropriate.

  For example, as in the second embodiment, the imaging unit 1301 has an “R” filter having sensitivity to a red wavelength, a “G” filter having sensitivity to a green wavelength, and a blue wavelength to the top of the photoelectric conversion element. A single-plate color image sensor that obtains four types of color signals by arranging a "B" filter with sensitivity and an "IR" filter with sensitivity in the invisible light region of infrared or near infrared in a checkered pattern. Assume that the sensitivity characteristics of each of R, G, B, and IR are as shown in FIG. The exposure time can be individually adjusted for each of the R, G, and B pixels and the IR pixel.

  In addition, the imaging apparatus according to the present embodiment includes a non-visible light projector 601 and a visible light projector 602 that irradiate non-visible light toward a subject, and irradiation of the invisible light projector 601 is turned on. , OFF or intensity of irradiation, ON / OFF of irradiation of the visible light projecting unit 602, or intensity of irradiation, and further, each condition of the light source A irradiated at the place where the imaging apparatus of the present embodiment is installed Since the wavelength distribution of the light that finally irradiates the subject changes every time, the R, G, B pixel and IR pixel exposure times that are optimal for each condition are stored separately in advance. The table portion 603 is provided.

  In addition, the exposure control processing unit 104 determines whether the irradiation ON / OFF information of the invisible light projector 601 or the intensity of irradiation and the ON / OFF information of the irradiation of the visible light projector 602 or the intensity of irradiation. Based on the information, the exposure time of each of the R, G, B pixels and IR pixels is determined with reference to the information in the table unit 603, and the exposure time is adjusted individually.

  For example, when the irradiation of the invisible light projector 601 is turned on and the irradiation of the visible light projector 602 is turned on, when the wavelength distribution characteristics of each light are as shown in FIG. , B, and IR, the output characteristics of each pixel are multiplied by FIG. 10 and FIG. 11 and are as shown in FIG. In this case, the exposure time of the IR pixel may be longer than the exposure time of the R, G, and B pixels, and each pixel may have an appropriate signal amount. Accordingly, the information on the exposure time is stored in the table unit 603 in advance, and irradiation ON / OFF information of the invisible light projector 601 or information on the intensity of irradiation and ON of the visible light projector 602 are turned on. Therefore, the exposure time is determined based on OFF information or irradiation intensity information.

  According to the above configuration, it is not necessary to obtain the signal level ratio from the image signal by calculation as described in the first or second embodiment, and the processing can be simplified. In addition, the exposure time can be determined with high accuracy regardless of the subject in the video. Other configurations and effects are the same as those described in the first embodiment or the second embodiment.

  In the present embodiment, the non-visible light projecting unit and the visible light projecting unit are provided. However, any one of the configurations may be used. In addition, the invisible light projecting unit does not necessarily have sensitivity only to invisible light, and may have sensitivity in the visible light region. The visible light projecting unit is not necessarily sensitive to only visible light, and may be sensitive to invisible light.

  In addition, this invention is not limited to an above-described Example, Various modifications are included. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described. Further, a part of the configuration of a certain embodiment can be replaced with the configuration of another embodiment and applied as appropriate, and the configuration of another embodiment can be added to or combined with the configuration of another embodiment as appropriate. Is also possible. Further, it is possible to add, delete, and replace other configurations for a part of the configuration of each embodiment.

In addition, each of the above-described configurations may be configured such that some or all of them are configured by hardware, or are implemented by executing a program by a processor. Further, the control lines and information lines indicate what is considered necessary for the explanation, and not all the control lines and information lines on the product are necessarily shown. Actually, it may be considered that almost all the components are connected to each other.
As described above, according to the configuration of each embodiment, it is possible to provide an imaging device that suppresses S / N degradation and realizes a color image with good visibility.

101 ... Imaging unit 102 ... Image processing unit 103 ... Determination unit 104 ... Exposure control processing unit 301 ... Reset processing unit 302 ... Read processing unit 601 ... Invisible light projection Unit 602 ... Visible light projector 603 ... Table unit

Claims (2)

A first pixel group including a first filter having a predetermined transmission center wavelength and a second pixel group including a second filter having a transmission center wavelength different from the first filter are arranged. An image pickup unit,
A determination unit that calculates a ratio between an average value of the signal level from the first pixel group output from the imaging unit and an average value of the signal level of the signal from the second pixel group;
An exposure control processing unit that determines the exposure time of the second pixel group using the ratio calculated by the determination unit and the exposure time of the first pixel group;
Equipped with a,
The imaging unit includes a visible light pixel having sensitivity in a visible wavelength region as the first pixel group, and invisible light having sensitivity in a wavelength region of infrared or near infrared light as the second pixel group. With pixels,
The exposure control processing unit, the visible light pixel and the non-visible light pixel imaging apparatus characterized that you independently controlled individually exposure time. The imaging apparatus according to claim 1 ,
The imaging unit includes a reset processing unit that discharges accumulated charges due to exposure for each pixel, and a read processing unit for reading accumulated charges due to exposure for each pixel,
The exposure control processing unit controls the exposure time of the first pixel group and the second pixel group by controlling the timing of the reset processing unit and the readout processing unit according to the determined exposure time. An imaging apparatus characterized by:

Images