JP2010161459A - Infrared radiation imaging device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an infrared radiation imaging device capable of adjusting a composite ratio of a visible light component and an infrared component when imaging is executed under low illuminance. <P>SOLUTION: In executing imaging under an infrared radiation imaging condition, an infrared combined imaging device processes white balance per color of an imaging signal generated under visible light imaging condition in which an infrared cut filter is inserted into an optical path, and generates a white balance imaging signal. The imaging device separates an imaging signal generated under the infrared radiation condition into a visible light component signal and an infrared component imaging signal, generates a visible light luminance signal and a chroma image signal from the visible light component imaging signal, and generates an infrared luminance signal from the infrared component imaging signal. The imaging device adjusts the composite ratio of the visible light luminance signal and the infrared luminance signal to generate an luminance image signal. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an infrared irradiation type imaging apparatus capable of imaging an object under low illuminance by irradiating infrared rays.

  Unlike human eyes, the spectral sensitivity characteristics of image sensors (image sensors) such as CCD and MOS sensors are sensitive not only to visible light components but also to infrared components, so digital still cameras, movies In an imaging apparatus such as a camera or a medical camera, an infrared cut filter is disposed in front of the imaging element so as to receive only visible light.

  In such an imaging apparatus, if the infrared cut filter is removed to receive the infrared component, the amount of light incident on the imaging device increases, and the subject can be brightly imaged even under low illuminance. However, since the infrared wavelength is in a wavelength region that cannot be detected by the human eye and the infrared does not have original color information, the white balance of the captured image including the infrared component is significantly deteriorated. Therefore, in a normal infrared irradiation type imaging device, an imaging signal including an infrared component is treated as a luminance signal, and a monochrome image is generated.

  In recent years, there has been a demand from the market to generate a color image close to an actual appearance, not a monochrome image, using such an imaging device.

In Patent Document 1, when an infrared cut filter is disposed at an external position (at night shot shooting), black body curve data LB considering an infrared component is read and black body curve control is performed. Alternatively, gray world control is performed so that the ratio of the video signals R, G, and B is 1. Accordingly, there is a description that optimal white balance control is performed according to the shooting situation.
JP 2005-130317 A

  However, the technique related to the imaging apparatus and method described in Patent Document 1 has a problem that the synthesis ratio of the visible light component and the infrared component of the captured image cannot be adjusted. This is because the black body curve control based on the black body curve data LB in consideration of the former infrared component and the latter gray world control are both white balance controls for the mixed light of the visible light component and the infrared component. Because.

  Therefore, the present invention has been made in view of the above-described problems, and in the case of imaging by irradiating an infrared ray on a subject under low illuminance, the composite ratio of the visible light component and the infrared component is set. An object of the present invention is to provide an infrared irradiation type imaging apparatus that can obtain a captured image closer to the image quality desired by the user by adjusting and / or selecting and imaging.

  In the infrared irradiation type imaging device, the infrared irradiation means for flashing and irradiating the subject with infrared rays, the optical means for forming an optical image by forming an image of the subject, and the structure capable of moving forward and backward with respect to the optical path of the optical means. And an infrared cut filter for generating an optical image composed of visible light by blocking infrared rays contained in the optical image, and a visible light imaging condition for inserting the infrared cut filter on the optical path without irradiating the subject with infrared rays. And infrared irradiation control means for controlling the infrared irradiation imaging condition for causing the subject to flash infrared rays and causing the infrared cut filter to leave the optical path, and photoelectrically converting the optical image to generate an imaging signal for each color. White balance that generates white balance imaging signals by processing white balance for each color of imaging signals generated under visible light imaging conditions and imaging means to generate The image processing signal and the imaging signal input for each imaging condition are analyzed, and the imaging signal generated under the infrared irradiation imaging condition based on the analysis result is separated into a visible light component imaging signal and an infrared component imaging signal Re-sampling the plane phase of each color of the visible light component imaging signal to the same phase and generating a luminance signal and a chroma signal by performing matrix calculation, and converting the chroma signal into a white balance imaging signal A visible light image processing means for performing color adjustment based on the generated color tone signal and performing image processing including amplification processing on the luminance signal and the color tone chroma signal to generate a visible light luminance signal and a chroma image signal; An infrared image processing means for generating an infrared luminance signal by performing image processing on an infrared component imaging signal, and a luminance for generating a luminance image signal by synthesizing the visible light luminance signal and the infrared luminance signal. Comprising an image combining unit, a synthesis ratio adjusting operating means for adjusting operation of the mixing ratio of the visible light luminance signal and the infrared luminance signal.

  According to the present invention, when a subject under low illuminance is imaged by irradiating an infrared ray, the user adjusts and / or selects a combination ratio of the visible light component and the infrared component to capture the image. Thus, there is an effect that a captured image closer to the image quality desired by the user can be obtained.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of an infrared irradiation type imaging apparatus according to the present invention.

  In FIG. 1, this infrared irradiation type imaging apparatus includes an infrared light emitting diode (infrared irradiation means) 1, a lens (optical means) 2, an infrared cut filter 3, and an infrared irradiation control unit (infrared irradiation control means) 4. An image sensor (imaging means) 5, a white balance processing section (white balance processing means) 6, an infrared component separation section (infrared component separation means) 7, and a visible light image processing section (visible light image processing means) 8. And an infrared image processing unit (infrared image processing means) 9, a luminance image synthesizer (luminance image synthesizing means) 10, and a composition ratio adjustment operation unit (composition ratio adjustment operation means) 11.

  The infrared light emitting diode 1 emits infrared rays to the subject 12 in a blinking manner. The blinking frequency (Hz) is preferably a blinking frequency of at least four cycles or more with respect to the exposure period of the image sensor 5.

  This is because if the blinking frequency (Hz) is a low frequency that is less than four cycles with respect to the exposure period of the image sensor 5, the blurring of the subject 12 between the visible light component and the infrared component, and This is because it is conspicuous that a sharp image is captured differently in one frame, resulting in an unnatural captured image. Instead of the infrared light emitting diode 1, an infrared radiation element that can blink at high speed and has a larger amount of infrared radiation may be used.

  The lens 2 forms an optical image by forming an image of the subject 12. The infrared cut filter 3 has a structure capable of moving forward and backward with respect to the optical path of the lens 2 and blocks an infrared ray contained in the optical image to generate an optical image made of visible light.

  The infrared irradiation control unit 4 does not irradiate the subject 12 with infrared rays, inserts the infrared cut filter 3 on the optical path, and causes the subject 12 to irradiate infrared rays with the infrared cut filter 3 on the optical path. The infrared irradiation imaging conditions for exiting are controlled respectively.

  The image sensor 5 photoelectrically converts an optical image to generate an imaging signal RGBxy or an imaging signal RGBAxy for each RGB color. The white balance processing unit 6 processes white balance for each of the RGB colors of the imaging signal RGBxy generated under the visible light imaging condition, and generates a white balance imaging signal wRGBxy.

  The infrared component separation unit 7 analyzes the imaging signal RGBxy and the imaging signal RGBAxy described above, and converts the imaging signal RGBAxy generated under the infrared irradiation imaging condition based on the analysis result into the visible light component imaging signal [RGBxy]. And the infrared component imaging signal [Axy].

  The visible light image processing unit 8 resamples the planar phase xy for each RGB color of the visible light component imaging signal [RGBxy] to the same phase xy, generates a luminance signal Bxy and a chroma signal Txy by performing matrix calculation, Color adjustment is performed on the chroma signal Txy based on the white balance imaging signal wRGBxy to generate a color tone chroma signal Cxy.

  Further, the visible light image processing unit 8 performs image processing including amplification (GAIN) processing on the luminance signal Bxy and the color tone chroma signal Cxy to generate the visible light luminance signal Yxy and the chroma image signal (Cb, Cr) xy. Is to be generated.

  The chroma image signal (Cb, Cr) xy may be replaced with a color difference signal (Pb, Pr) xy for HDTV.

Here, if a matrix arithmetic expression of a general luminance signal Y and chroma signals (Cb, Cr) is shown,
Y = 0.299 * R + 0.587 * G + 0.114 * B
Cb = 0.564 * (B−Y) = − 0.169 * R−0.331 * G + 0.500 * B
Cr = 0.713 * (R−Y) = 0.500 * R−0.419 * G−0.081 * B
It is.

Moreover, if it shows about the matrix arithmetic expression of the general luminance signal Y for HDTV, and a color difference signal (Pb, Pr),
Y = 0.2126 * R + 0.7152 * G + 0.0722 * B
Pb = −0.1146 * R−0.3854 * G + 0.5000 * B
Pr = 0.5000 * R-0.4542 * G-0.0458 * B
It is.

  Note that the matrix arithmetic expression in the present embodiment may be calculated simultaneously with resampling, and is not necessarily limited to the above-described arithmetic expression.

  The infrared image processing unit 9 performs image processing on the infrared component imaging signal [Axy] to generate an infrared luminance signal Axy. The luminance image synthesizer 10 synthesizes the visible light luminance signal Yxy and the infrared luminance signal Axy to generate a luminance image signal Y′xy.

  The combination ratio adjustment operation unit 11 adjusts the combination ratio Y: A of the visible light luminance signal Yxy and the infrared luminance signal Axy. The combination ratio adjustment operation unit 11 controls the infrared irradiation control unit 4 based on the combination ratio Y: A. On the other hand, the amount of infrared irradiation is set. Based on the setting of the amount of infrared irradiation, the infrared irradiation control unit 4 performs PWM control for blinking irradiation on the infrared light emitting diode 1.

  Further, the luminance image synthesizer 10 may be controlled to attenuate the infrared luminance signal Axy by calculation based on the synthesis ratio Y: A. Or you may make it form the synthetic | combination ratio Y: A combining the setting of the infrared irradiation amount mentioned above, and this attenuation.

  The composition ratio adjustment operation unit 11 sets a limit of the maximum amplification factor MAX-GAIN (dB) for the amplification process of the visible light image processing unit 8 based on the composition ratio Y: A. Based on the limit setting of the maximum amplification factor MAX-GAIN (dB), amplification processing is performed by the visible light image processing unit 8.

  Furthermore, the composition ratio adjustment operation unit 11 has a plurality of modes, and the composition ratio Y: A is selected by selecting an arbitrary mode. Further, after this mode selection is made, fine adjustment of the composition ratio may be possible.

  FIG. 2 is a graph showing an outline of the composition ratio adjustment operation unit 11 according to the present embodiment.

  In FIG. 2, it is shown that the synthesis ratio adjustment operation unit 11 has a brightness priority mode, a color reproduction priority mode, and an SN ratio priority mode.

  In FIG. 2, when the color reproduction priority mode is selected and operated, the infrared irradiation amount is set so as to lower the ratio A of the combination ratio Y: A, and PWM control is performed based on this setting. It is shown that the signal level of the infrared luminance signal A is adjusted to be small.

  As described above, when the color reproduction priority mode is selected, the signal level of the luminance image signal Y1 ′ in the color reproduction priority mode is lower than the luminance image signal Y ′ in the brightness priority mode. However, since the signal level of the infrared luminance signal A having no color information instead decreases as indicated by A1, the color reproducibility is improved by the decreased amount.

  Alternatively, in FIG. 2, when the S / N ratio priority mode is selected and operated, the maximum amplification factor MAX-GAIN (dB) is set so as to reduce the Y ratio of the synthesis ratio Y: A (downward correction). It is shown that amplification processing (GAIN) is performed based on this setting, and the signal level of the visible light luminance signal Y is adjusted to be small as indicated by Y2.

  Thus, when the S / N ratio priority mode is selected, the signal level of the luminance image signal Y2 ′ in the S / N ratio priority mode is lower than that in the luminance priority signal Y ′ in the brightness priority mode. However, since the noise level that is increased by the amplification process of the visible light luminance signal Y is lowered instead, the SN ratio is improved by the amount that the noise level is lowered.

  FIG. 3 is a graph showing an example of the imaging signal level at the standard gain when the white object 12 under low illuminance is imaged under the visible light imaging condition.

  In FIG. 3, this graph shows the imaging signal level in a standard gain (GAIN = 0 dB) state that has not been amplified yet in the visible light image processing unit 8. For this reason, it is shown that this imaging signal has not yet reached the signal level of the lowest subject illuminance.

  FIG. 4 is a graph showing an example in which amplification processing is performed on the imaging signal level under the visible light imaging condition shown in FIG.

  This graph shows an example in which the visible light image processing unit 8 performs amplification processing (GAIN = 12 dB) on the imaging signal under the visible light imaging conditions shown in FIG. Here, the general amplification process is usually performed after white balance is applied to the RGB signal and then converted from the RGB signal to the YC signal by matrix calculation. In order to make a specific description of each priority mode shown in FIG. 2, in FIGS. 4 to 10, the YC signal subjected to the amplification process is converted for each RGB color.

  As shown in FIG. 4, in this example, the signal level of the lowest subject illuminance has not yet been reached even though the imaging signal has been subjected to amplification processing (GAIN = 12 dB).

  FIG. 5 is a graph showing an example of an imaging signal level obtained by imaging the white subject 12 under low illuminance in the infrared irradiation imaging condition and brightness priority mode.

  In FIG. 5, this graph shows that the signal level of the lowest subject illuminance is satisfied by imaging under the infrared irradiation imaging condition even though the subject 12 under low illuminance is imaged. Yes. As shown in FIG. 5, the shaded portion is a visible light component, and the white frame portion is an infrared component.

  Since FIG. 5 is an image pickup in the brightness priority mode, as already shown in FIG. 2, the composition ratio is Y: A and the visible light component Y (in FIG. 5). , R ′, G, and B ′), amplification processing (GAIN = 12 dB) is performed, and for infrared component A, the maximum infrared irradiation amount is set by PWM control. .

  FIG. 6 is a graph showing an example in which the imaging signal shown in FIG. 5 is separated into a visible light component imaging signal and an infrared component imaging signal, and color adjustment is performed based on the white balance imaging signal.

  As shown in FIG. 6, the brightness priority mode is a mode in which priority is given to taking a bright image of the subject 12 under low illuminance.

  FIG. 7 is a graph showing an example of an imaging signal level obtained by imaging a white subject 12 under low illuminance in an infrared irradiation imaging condition and a color reproduction priority mode.

  In FIG. 7, this graph shows that the signal level of the lowest subject illuminance is satisfied by imaging under the infrared irradiation imaging condition even though the subject 12 under low illuminance is imaged. Yes. As shown in FIG. 7, the shaded portion is the visible light component, and the white frame portion is the infrared component.

  Since FIG. 7 is an image pickup in the color reproduction priority mode, as already shown in FIG. 2, the composition ratio is Y: A1, and the visible light component Y (in FIG. 7). , R ′, G, and B ′), amplification processing (GAIN = 12 dB) is performed, and infrared component A1 is set so that the infrared irradiation amount is reduced by PWM control. It is.

  FIG. 8 is a graph showing an example in which the imaging signal shown in FIG. 7 is separated into a visible light component imaging signal and an infrared component imaging signal, and color adjustment is performed based on the white balance imaging signal.

  As shown in FIG. 8, this color reproduction priority mode is a mode in which imaging is performed with an emphasis on color reproducibility by lowering the ratio of the infrared component A having no original color information. Here, if the graph shown in FIG. 6 is compared with the graph of FIG. 8, the signal level of the infrared component A is different and the visible light component Y (in FIGS. 6 and 8) is different. Is shown in terms of R ′, G, B ′).

  FIG. 9 is a graph showing an example of an imaging signal level obtained by imaging the white subject 12 under low illuminance in the infrared irradiation imaging condition and SN ratio priority mode.

  In FIG. 9, this graph shows that the signal level of the lowest subject illuminance is satisfied by imaging under the infrared irradiation imaging condition even though the subject 12 under low illuminance is imaged. Yes. As shown in FIG. 9, the shaded portion is a visible light component, and the white frame portion is an infrared component.

  Since FIG. 9 is an image pickup in the S / N ratio priority mode, the composite ratio is Y2: A and the visible light component Y2 (in FIG. 9) as already shown in FIG. Is shown in terms of R ′, G, and B ′), it is shown that the maximum amplification factor MAX-GAIN (dB) is set.

  Here, in FIG. 9, it is shown as an example that MAX-GAIN is limited to 6 (dB). For infrared component A, the maximum infrared irradiation amount is set by PWM control.

  FIG. 10 is a graph showing an example in which the imaging signal shown in FIG. 9 is separated into a visible light component imaging signal and an infrared component imaging signal, and color adjustment is performed based on the white balance imaging signal.

  As shown in FIG. 10, the S / N ratio priority mode lowers the maximum amplification factor of the visible light component Y (shown in terms of R ′, G, B ′ in FIG. 10). In this mode, the noise level amplified by the amplification process is suppressed.

  As described above, the infrared irradiation type imaging device according to the embodiment of the present invention irradiates the subject 12 under the low illuminance with infrared rays and picks up an image. The user can adjust and / or select the combination ratio Y: A to provide an infrared irradiation imaging apparatus that can obtain a captured image closer to the image quality desired by the user.

  The embodiment of the present invention has been described in detail above with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes design changes and the like without departing from the gist of the present invention.

It is a block diagram which shows the structure of the infrared irradiation type imaging device by this invention. It is a graph which shows the outline | summary of the synthetic | combination ratio adjustment operation part by this embodiment. It is a graph which shows an example of the imaging signal level in a standard gain which imaged the white photographic subject under low illumination on visible light imaging conditions. It is a graph which shows an example which performed the amplification process with respect to the imaging signal level at the time of the visible light imaging condition shown in FIG. It is a graph which shows an example of the imaging signal level which imaged the white photographic subject under low illumination intensity in infrared irradiation imaging conditions and brightness priority mode. 6 is a graph showing an example in which the imaging signal shown in FIG. 5 is separated into a visible light component imaging signal and an infrared component imaging signal, and color adjustment is performed based on the white balance imaging signal. It is a graph which shows an example of the imaging signal level which imaged the white photographic subject under low illumination intensity in infrared irradiation imaging conditions and color reproduction priority mode. 8 is a graph illustrating an example in which the imaging signal illustrated in FIG. 7 is separated into a visible light component imaging signal and an infrared component imaging signal, and color adjustment is performed based on the white balance imaging signal. It is a graph which shows an example of the imaging signal level which imaged the white photographic subject under low illumination intensity in infrared irradiation imaging conditions and SN ratio priority mode. 10 is a graph showing an example in which the imaging signal shown in FIG. 9 is separated into a visible light component imaging signal and an infrared component imaging signal, and color adjustment is performed based on the white balance imaging signal.

1 Infrared light emitting diode (infrared irradiation means)
2 Lens (optical means)
3 Infrared cut filter 4 Infrared irradiation control unit (infrared irradiation control means)
5 Image sensor (imaging means)
6 White balance processing unit (white balance processing means)
7 Infrared component separation unit (Infrared component separation means)
8 Visible light image processing unit (visible light image processing means)
9 Infrared image processing unit (infrared image processing means)
10 Luminance image synthesizer (luminance image synthesizer)
11 Composite ratio adjustment operation unit (Composite ratio operation means)
12 Subject

Claims (7)

Infrared irradiation means for flashing and irradiating the subject with infrared rays;
Optical means for forming an optical image by imaging the subject;
An infrared cut filter configured to be movable back and forth with respect to the optical path of the optical means, for blocking infrared rays included in the optical image and generating the optical image composed of visible light;
Visible light imaging conditions in which the infrared cut filter is inserted into the optical path without irradiating the subject with the infrared light; and Infrared irradiation control means for controlling each of the infrared irradiation imaging conditions to be withdrawn, and
Imaging means for photoelectrically converting the optical image to generate an imaging signal for each color;
White balance processing means for processing white balance for each of the colors of the imaging signal generated under the visible light imaging conditions and generating a white balance imaging signal;
The imaging signal input according to the imaging condition is analyzed, and the imaging signal generated under the infrared irradiation imaging condition based on the analysis result is separated into a visible light component imaging signal and an infrared component imaging signal Means for separating infrared components;
The planar phase for each color of the visible light component imaging signal is resampled to the same phase, a matrix operation is performed to generate a luminance signal and a chroma signal, and a color based on the white balance imaging signal is generated for the chroma signal Visible light image processing means for generating a color chroma signal by performing adjustment, and performing image processing including amplification processing on the luminance signal and the color chroma signal to generate a visible light luminance signal and a chroma image signal;
Infrared image processing means for generating an infrared luminance signal by performing image processing on the infrared component imaging signal;
A luminance image synthesis means for synthesizing the visible light luminance signal and the infrared luminance signal to generate a luminance image signal;
A composition ratio adjusting operation means for adjusting a composition ratio of the visible light luminance signal and the infrared luminance signal;
An infrared irradiation type imaging device comprising: The synthesis ratio adjustment operation means sets the infrared irradiation amount for the infrared irradiation control means,
The infrared irradiation type imaging apparatus according to claim 1, wherein the infrared irradiation control unit performs PWM control of the blinking irradiation on the infrared irradiation unit based on the setting of the infrared irradiation amount. The synthesis ratio adjustment operation means sets a limit on the maximum amplification factor of the amplification process for the visible light image processing means,
The infrared irradiation imaging apparatus according to claim 2, wherein the visible light image processing means is subjected to the amplification processing based on the setting of the restriction on the maximum amplification factor. The synthesis ratio adjustment operation means includes:
A plurality of modes including at least a brightness priority mode, a color reproduction priority mode, and an SN ratio priority mode;
One mode can be selected from the plurality of modes,
When the color reproduction priority mode is selected and operated,
The PWM control is performed based on at least the setting of the irradiation amount,
When the SN ratio priority mode is selected and operated,
The infrared irradiation imaging apparatus according to claim 3, wherein the amplification process is performed based on at least a restriction on the maximum amplification factor. 5. The infrared ray according to claim 4, wherein after the mode is selected, the composition ratio adjustment operation means can finely adjust the composition ratio by an operation different from the mode selection. Irradiation imaging device. The infrared irradiation imaging apparatus according to claim 1, wherein the luminance image synthesizing unit attenuates the infrared luminance signal based on the synthesis ratio. The infrared irradiation imaging apparatus according to claim 1, wherein the blinking frequency of the blinking irradiation is a frequency that blinks at least four periods or more with respect to an exposure time of the imaging unit.

Images