JP2010161457A - Infrared radiation imaging device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an infrared radiation imaging device having good reproducibility even when radiated with infrared for imaging. <P>SOLUTION: In imaging under an infrared radiation imaging condition, an infrared combined imaging device processes white balance per color of an imaging signal generated based on visible light imaging conditions in which an infrared cut filter is inserted into an optical path, and generates a white balance imaging signal. The imaging device separates an imaging signal generated under the infrared radiation imaging condition into a visible light imaging signal and an infrared imaging signal, calculates a mean value of the signal level of the infrared component imaging signal, and corrects hue and saturation if the deviation between the signal level of the infrared component imaging signal and the mean value is large. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an infrared irradiation type imaging apparatus capable of imaging an object under low illuminance by irradiating infrared rays.

  Unlike human eyes, the spectral sensitivity characteristics of image sensors (image sensors) such as CCD and MOS sensors are sensitive not only to visible light components but also to infrared components, so digital still cameras, movies In an imaging apparatus such as a camera or a medical camera, an infrared cut filter is disposed in front of the imaging element so as to receive only visible light.

  In such an imaging apparatus, if the infrared cut filter is removed to receive the infrared component, the amount of light incident on the imaging device increases, and the subject can be brightly imaged even under low illuminance. However, since the infrared wavelength is in a wavelength region that cannot be detected by the human eye and the infrared does not have original color information, the white balance of the captured image including the infrared component is significantly deteriorated. Therefore, in a normal infrared irradiation type imaging device, an imaging signal including an infrared component is treated as a luminance signal, and a monochrome image is generated.

  In recent years, there has been a demand from the market to generate a color image close to an actual appearance, not a monochrome image, using such an imaging device.

In Patent Document 1, when an infrared cut filter is disposed at an external position (at night shot shooting), black body curve data LB considering an infrared component is read and black body curve control is performed. Alternatively, gray world control is performed so that the ratio of the video signals R, G, and B is 1. Accordingly, there is a description that optimal white balance control is performed according to the shooting situation.
JP 2005-130317 A

  However, the black body curve control based on the black body curve data LB in consideration of the infrared component of the former does not take into consideration the relationship between the direct light and the infrared light irradiated to the subject and the backlight, so that the subject is in the subject. When both forward light and backlight are present, good white balance cannot be obtained.

  Further, since the latter gray world control is based on the premise that there is no bias in the color of the subject, white balance cannot be appropriately processed if the subject color is biased.

  The present invention has been made in view of these technical problems, and in an infrared irradiation type imaging apparatus that makes it possible to brightly image a subject under low illuminance by irradiating infrared rays, white balance is appropriately set. The purpose is to obtain a color image with high color reproducibility.

  In an infrared irradiation type imaging apparatus, an infrared irradiation unit that irradiates a subject with infrared rays, an optical unit that forms an image of the subject and generates an optical image, and a structure that can advance and retreat with respect to the optical path of the optical unit. An infrared cut filter for blocking an infrared ray contained in the image to generate an optical image composed of visible light, and a visible light imaging condition for inserting the infrared cut filter on the optical path without irradiating the subject with the infrared ray, Infrared irradiation control means for controlling infrared irradiation imaging conditions for irradiating an object with infrared rays and causing the infrared cut filter to exit the optical path, and imaging for photoelectrically converting an optical image and generating an imaging signal for each color And white balance processing means for processing a white balance for each color of an imaging signal generated under visible light imaging conditions and generating a white balance imaging signal Infrared component separation that analyzes imaging signals input according to imaging conditions and separates imaging signals generated under infrared irradiation imaging conditions based on analysis results into visible light component imaging signals and infrared component imaging signals And a white balance imaging signal integrated for each color, and a visible light color ratio, which is a ratio of integral values, is calculated and stored, and a color balance coefficient of the visible light component imaging signal based on the visible light color ratio A color balance matching unit that generates a color-matched imaging signal by matching the image, an image processing unit that generates a color image signal by performing image processing on the color-matched imaging signal and the infrared component imaging signal, and an infrared component imaging signal The average value of the signal level is calculated, the positive / negative deviation between the signal level of the infrared component imaging signal and the average value is calculated with respect to the plane phase, the hue with respect to the plane phase having a deviation of a predetermined level or more, and , Characterized by comprising a color saturation correction setting means for setting a correction degree, a.

  According to the present invention, there is an effect that good color reproducibility can be obtained even when imaging is performed by irradiating a subject under low illuminance with infrared rays.

  According to the present invention, in the case of imaging by irradiating infrared rays to a subject under low illuminance, the irradiation direction of visible light is non-ordering light with respect to the direction of infrared rays irradiated to the subject to be imaged. Even in some cases, there is an effect of obtaining good color reproducibility.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of an infrared irradiation type imaging apparatus according to the first embodiment of the present invention.

  In FIG. 1, this infrared irradiation type imaging apparatus includes an infrared light emitting diode (infrared irradiation means) 1, a lens (optical means) 2, an infrared cut filter 3, and an infrared irradiation control unit (infrared irradiation control means) 4. An image sensor (imaging means) 5, a white balance processing section (white balance processing means) 6, an infrared component separation section (infrared component separation means) 7, a color balance matching section (color balance matching means) 8, An image processing unit (image processing unit) 9 and a hue saturation correction setting unit (hue saturation correction setting unit) 10 are configured.

  The infrared light emitting diode 1 irradiates the subject 11 with infrared rays. The lens 2 forms an optical image by forming an image of the subject 11. The infrared cut filter 3 has a structure capable of moving forward and backward with respect to the optical path of the lens 2 and blocks an infrared ray contained in the optical image to generate an optical image made of visible light.

  The infrared irradiation control unit 4 does not irradiate the subject 11 with infrared rays, inserts the infrared cut filter 3 on the optical path, and causes the subject 11 to irradiate infrared rays with the infrared cut filter 3 on the optical path. The infrared irradiation imaging conditions for exiting are controlled respectively.

  The image sensor 5 photoelectrically converts an optical image to generate an imaging signal RGBxy or an imaging signal RGBAxy for each RGB color. The white balance processing unit 6 processes white balance for each of the RGB colors of the imaging signal RGBxy generated under the visible light imaging condition, and generates a white balance imaging signal wRGBxy.

  The infrared component separation unit 7 analyzes the imaging signal RGBxy and the imaging signal RGBAxy described above based on, for example, the ratio and / or difference thereof, and the infrared irradiation imaging condition based on the analysis result. The generated imaging signal RGBAxy is separated into a visible light component imaging signal [RGBxy] and an infrared component imaging signal [Axy].

The color balance matching unit 8 integrates the white balance image pickup signal wRGBxy for each RGB color, calculates and stores a ratio ΣR: ΣG: ΣB for each visible light color, which is a ratio of the integrated values, and compares each ratio for each visible light color. The color matching imaging signal c [RGBxy] is generated by matching the color balance of the visible light component imaging signal [RGBxy] based on ΣR: ΣG: ΣB. Details of the color balance matching unit 8 will be described later.

  The image processing unit 9 performs color processing on the color matching imaging signal c [RGBxy] and the infrared component imaging signal [Axy] to generate a color image signal.

The hue saturation correction setting unit 10 calculates an average value Ave. of the signal level of the infrared component imaging signal [Axy]. [A] is calculated, and the signal level of the infrared component imaging signal [Axy] and the average value Ave. The positive / negative deviation ± Δ [Axy] from [A] is calculated with respect to the plane phase xy, and the hue (hue) and saturation (with respect to the plane phase xy for which the deviation of a predetermined level or more is calculated) (saturation) correction.

  FIG. 2 is a block diagram showing a configuration of an infrared irradiation imaging apparatus according to the second embodiment of the present invention.

  2, this block diagram is different from the hue / saturation correction setting unit 10 shown in FIG. 1 in that the hue / saturation correction setting unit 20 is different from the hue / saturation correction setting unit 10 in FIG. An infrared irradiation type imaging device having the hue saturation correction setting unit 20 is shown.

  The hue / saturation correction setting unit 20 further sets a maximum signal level [RGBxy] MAX for each color of RGB with respect to the color-matched image signal c [RGBxy] based on the white balance image signal wRGBxy.

  The hue / saturation correction setting units 10 and 20 correct a deviation in hue and saturation due to the forward light and the infrared light of the visible light and infrared light that irradiate the subject 11, and in particular, the subject 11. This is effective color reproducibility correction when there is both forward light and backlight. This correction of hue saturation will be described later.

  First, details of the color balance matching unit 8 commonly shown in the first embodiment and the second embodiment will be described.

The color balance matching unit 8 may calculate the visible light color ratio ΣR: ΣG: ΣB at least once for each scene to be captured in the block diagrams shown in FIGS. The storage of the calculation result is updated for each scene to be imaged.

Here, as for the calculation method of ΣR, ΣG, and ΣB, the above-described white balance imaging signal wRGBxy may be subjected to xy plane integration for each RGB color.
ΣR = ∬ (wRxy) dxdy
ΣG = ∬ (wGxy) dxdy
ΣB = ∬ (wBxy) dxdy
It is.

If the ratio for each color of RGB is calculated for ΣR, ΣG, and ΣB obtained in this way, the ratio ΣR: ΣG: ΣB for each visible light color can be obtained. Further, this solution is a solution showing the color deviation of the subject 11 in each scene of the subject 11 obtained under the visible light imaging condition.

In FIG. 2, the visible light component imaging signal [RGBxy] separated from the infrared irradiation imaging condition is matched with the color balance at this visible light color ratio ΣR: ΣG: ΣB. It is possible to obtain the color matching image signal c [RGBxy] reflecting the color balance and the white balance under the imaging conditions.

Here, regarding the color balance of the visible light component imaging signal [RGBxy],
Σ [R] = ∬ [Rxy] dxdy
Σ [G] = ∬ [Gxy] dxdy
Σ [B] = ∬ [Bxy] dxdy
It is.

To achieve color balance matching with this visible light component imaging signal [RGBxy] at a visible light color ratio ΣR: ΣG: ΣB,
Kr * Σ [R]: Kg * Σ [G]: Kb * Σ [B]
= ΣR: ΣG: ΣB
It would be good if

Further, if the color balance coefficient (Kr, Kg, Kb) is multiplied by the visible light component imaging signal [RGBxy], the color matching imaging signal c [RGBxy] is obtained.
c [RGBxy]
= (C [Rxy], c [Gxy], c [Bxy])
= (Kr * [Rxy], Kg * [Gxy], Kb * [Bxy])
It is.

  Next, problems with the infrared irradiation type imaging apparatus shown in FIGS. 1 and 2 and a solution to the problem will be described.

  FIG. 3 is a conceptual diagram illustrating an example of the subject 11 imaged under the visible light imaging condition and the subject 11 imaged under the infrared irradiation imaging condition.

  FIG. 3A shows that the irradiation direction of visible light is backlight with respect to the imaging direction under the visible light imaging condition of this conceptual diagram. Further, it is shown that the weak visible light due to the indirect light of the visible light hits the front part 11a of the subject 11.

  Further, it is shown that the visible light irradiation direction is non-ordering light with respect to the infrared irradiation direction in the infrared imaging condition of the conceptual diagram of FIG. This is because the infrared light emitting diode 1 that irradiates infrared rays is provided in the imaging apparatus body, and the irradiation direction of infrared rays is not arbitrary.

  FIG. 3B shows that the front part (dark part) 11a of the subject 11 irradiated with indirect visible light is lit up and brightened by the infrared irradiation.

  As described above, for the subject 11 in which the irradiation direction of the visible light is non-forward light with respect to the irradiation direction of the infrared ray, the ratio of the visible light component and the infrared component of the subject 11 is large depending on the part of the subject 11. It is different. For example, when imaging the subject 11 as shown in FIG. 3, the visible light component imaging signal [RGBxy] of the imaging signal RGBAxy by the infrared component separation unit 7 and the infrared component imaging signal [Axy] There is a concern that the separation accuracy for separation (reproducibility with respect to an actual subject) may deteriorate.

  FIG. 4 is a graph showing an example of imaging signal levels for each of RGB colors when the white subject irradiated with the indirect light shown in FIG. 3 is imaged under visible light imaging conditions.

  Here, when the image sensor 5 is a single-plate color filter specification by Bayer arrangement, etc., the plane phase xy for each color of RGB is not uniform, so for example, resampling processing (not shown) is performed. The plane phase xy for each RGB color may be aligned in advance. This resampling process may be, for example, general bicubic interpolation or bilinear interpolation.

  FIG. 5 is a graph showing an example of imaging signal levels for each of the RGB colors when the portion of the white subject irradiated with the indirect light shown in FIG. 3 is imaged under infrared irradiation imaging conditions.

  4 and FIG. 5, the hatched portion indicates the level of the imaging signal photoelectrically converted by the visible light component, and in FIG. In the portion shown, the level of the imaging signal photoelectrically converted by the infrared component is shown. Further, as shown in FIG. 5, the signal level of the entire imaging signal is improved by imaging by irradiating infrared rays. This is done to brightly image the subject 11 under low illuminance.

  FIG. 6 is a graph showing an example in which the color balance is matched by separating the imaging signal shown in FIG. 5 into a visible light component imaging signal and an infrared component imaging signal.

  As shown in FIG. 6, the infrared component indicated by the white frame remains in the visible light component imaging signal (R, G, B). Thus, when the irradiation direction of visible light is non-forward light with respect to the irradiation direction of infrared rays, the separation accuracy is deteriorated.

  FIG. 7 is a graph showing an example of the imaging signal level for each of the RGB colors when the part of the subject 11 irradiated with the indirect light shown in FIG. 3 is imaged under infrared irradiation imaging conditions instead of a single red wavelength. is there.

  In FIG. 7, this graph shows a state in which photoelectric conversion by infrared components is performed on colors (G, B) other than the red single wavelength R which is the original color of the subject 11. . Further, it has been shown that such a residual infrared component is a cause of deterioration in color reproducibility.

  Regarding the residual amount of the infrared component that deteriorates the color reproducibility shown in FIG. 7, the visible light component imaging signal [RGBxy] of the imaging signal RGBAxy by the infrared component separation unit 7 and the infrared component imaging signal [Axy], It depends on the separation accuracy of separation.

  Regarding this problem, there is a concern when visible light irradiation is not uniform on the subject 11 or when the imaging environment is such that infrared irradiation is not uniform on the subject 11. This problem becomes even more pronounced when the irradiation direction and the infrared irradiation direction are non-forward light.

  For example, in the example shown in FIG. 7, it is shown that the subject 11 having a single red wavelength is lit up by being irradiated with infrared rays and is brightly improved so that it can be imaged. However, in exchange for this, the red hue is slightly shifted to the blue side, and the red saturation is also deteriorated.

  Here, since the planar phase xy of the subject 11 having a single red wavelength is also the portion most irradiated with infrared rays in FIG. 3, the average value Ave. This is a deviation + Δ [Axy] of a predetermined level or more with respect to [A].

  FIG. 8 is a graph showing an example in which hue saturation correction is performed on the imaging signal level shown in FIG.

In FIG. 8, A shown in this graph indicates the signal level of the infrared component imaging signal [Axy] on an arbitrary plane phase xy, and the average value Ave. About deviation [Axy] from [A]
Δ [Axy]
= [Axy] -Ave. [A]
≧ (Lv +)
It is shown that.

As described above, the plane phase xy to be corrected for hue saturation is the average value Ave. It is determined by the deviation Δ [Axy] from [A]. The setting of the plane phase xy is set by the hue / saturation correction setting units 10 and 20 shown in FIGS.

Here, a specific example of a correction method for hue saturation correction will be described. Deviation Δ [Axy]
The color matching imaging signal c [RGBxy] is within the color gamut range of the white balance imaging signal wRGBxy generated under the visible light imaging condition with respect to the planar phase xy to be corrected for hue saturation determined by. It is sufficient to correct it. For example, this color gamut range is provided as a color gamut map in the color balance matching unit 8 shown in FIG. 1, and the color balance matching unit 8 generates a visible light color ratio ΣR: ΣG: ΣB. Previous white balance image signal w
The color gamut is mapped based on RGBxy.

  In the second embodiment, the hue saturation correction setting unit 20 generates the color matching imaging signal c [RGBxy] for the planar phase xy to be corrected for hue saturation under the visible light imaging condition. The maximum signal level [RGBxy] MAX for each color of the white balance imaging signal wRGBxy thus set is further set as a correction value.

  When this [RGBxy] MAX is set as a correction value, the saturation of the color matching image signal c [RGBxy] should tend to be corrected to a lower saturation side than the saturation of the white balance image signal wRGBxy. It is.

However, for example, the signal level of the (G, B) signal shown in FIG. 8 is correctly corrected to OFFSET (not shown), and the color reproducibility when imaging a single red wavelength is improved. This is because, when a single red wavelength is imaged under visible light imaging conditions, the maximum signal level [RGBxy] MAX for each RGB color of the white balance imaging signal wRGBxy is shown as follows:
[RGBxy] MAX = [R00xy]
This is because the color matching imaging signal c [RGBxy] is corrected to the original red color that the red short wavelength has.

  As described above, the infrared irradiation type imaging device according to the embodiment of the present invention can obtain good color reproducibility when imaging by irradiating the subject 11 under low illuminance with infrared rays. It is possible to provide an infrared irradiation type imaging device.

  Further, according to the present invention, in the case of imaging by irradiating the subject 11 under low illuminance with infrared rays, the direction of infrared irradiation is relative to the direction of visible light irradiated to the subject 11 to be imaged. Even if it is a case where it is a non-forward light, the infrared irradiation type imaging device which can acquire favorable color reproducibility can be provided.

  The embodiment of the present invention has been described in detail above with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes design changes and the like without departing from the gist of the present invention.

  Further, the above-described embodiments include inventions at various stages, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements. For example, even if some constituent requirements are deleted from all the constituent requirements shown in the embodiment, the problem described in the column of the problem to be solved by the invention can be solved, and the effect described in the column of the effect of the invention Can be obtained as an invention.

1 is a block diagram illustrating a configuration of an infrared irradiation imaging device according to a first embodiment of the present invention. It is a block diagram which shows the structure of the infrared irradiation type imaging device by the 2nd Embodiment of this invention. It is a conceptual diagram which shows an example of the to-be-photographed object on visible light imaging conditions, and the to-be-photographed object on infrared irradiation imaging conditions. FIG. 4 is a graph showing an example of imaging signal levels for each of RGB colors when the white subject irradiated with the indirect light shown in FIG. 3 is imaged under visible light imaging conditions. It is a graph which shows an example of the imaging signal level for every RGB color at the time of imaging the site | part of the white subject irradiated with the indirect light shown in FIG. 3 on infrared irradiation imaging conditions. FIG. 6 is a graph illustrating an example in which the imaging signal shown in FIG. 5 is separated into a visible light component imaging signal and an infrared component imaging signal, and color balance is matched. It is a graph which shows an example of the image pick-up signal level for each RGB color at the time of imaging the part of the subject irradiated with the indirect light shown in FIG. It is a graph which shows an example which performed hue saturation correction | amendment with respect to the imaging signal level shown by FIG.

1 Infrared light emitting diode (infrared irradiation means)
2 Lens (optical means)
3 Infrared cut filter 4 Infrared irradiation control unit (infrared irradiation control means)
5 Image sensor (imaging means)
6 White balance processing unit (white balance processing means)
7 Infrared component separation unit (Infrared component separation means)
8 Color balance matching section (Color balance matching means)
9 Image processing unit (image processing means)
10 Hue / Saturation Correction Setting Unit (Hue / Saturation Correction Setting Unit)
11 Subject

Claims (2)

An infrared irradiation means for irradiating the subject with infrared;
Optical means for forming an optical image by imaging the subject;
An infrared cut filter configured to be movable back and forth with respect to the optical path of the optical means, for blocking infrared rays included in the optical image and generating the optical image composed of visible light;
Visible light imaging conditions for inserting the infrared cut filter on the optical path without irradiating the subject with the infrared ray, and irradiating the infrared ray on the subject to exit the infrared cut filter on the optical path. Infrared irradiation control means for controlling the infrared irradiation imaging conditions, respectively;
Imaging means for photoelectrically converting the optical image to generate an imaging signal for each color;
White balance processing means for processing white balance for each of the colors of the imaging signal generated under the visible light imaging conditions and generating a white balance imaging signal;
The imaging signal input according to the imaging condition is analyzed, and the imaging signal generated under the infrared irradiation imaging condition based on the analysis result is separated into a visible light component imaging signal and an infrared component imaging signal Means for separating infrared components;
The white balance imaging signal is integrated for each color, a ratio for each visible light color that is a ratio of the integrated values is calculated and stored, and the color of the visible light component imaging signal is calculated based on the ratio for each visible light color Color balance matching means for generating a color matching imaging signal by matching the balance coefficient;
Image processing means for performing color processing on the color matching imaging signal and the infrared component imaging signal to generate a color image signal;
The average value of the signal level of the infrared component imaging signal is calculated, and the positive / negative divergence between the signal level of the infrared component imaging signal and the average value is calculated with respect to the plane phase. Hue and saturation correction setting means for setting hue and saturation correction with respect to the planar phase;
An infrared irradiation type imaging device comprising: 2. The infrared irradiation type according to claim 1, wherein the hue saturation correction setting unit sets a maximum signal level for each of the colors with respect to the color matching image pickup signal based on the white balance image pickup signal. Imaging device.