JP2010161453A - Infrared radiation imaging device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an infrared radiation imaging device capable of applying superior white balance processing under a low illuminance. <P>SOLUTION: When imaging by infrared control, an infrared imaging signal is separated into a visible operation imaging signal and an infrared operation imaging signal per color. Light source determination is executed based on the visible light imaging signal, a white balance imaging signal is generated based on a result of light source determination, the white balance imaging signal is integrated per color, and the color balance of the visible light imaging signal is matched to generate a color matching imaging signal. A white balance correction imaging signal is generated for this color matching imaging signal based on a result of light source determination, and image processing is applied to the infrared operation imaging signal and the white balance correction imaging signal to generate a color image signal. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an infrared irradiation type imaging apparatus capable of imaging an object under low illuminance by irradiating infrared rays.

  Unlike human eyes, the spectral sensitivity characteristics of image sensors (image sensors) such as CCD and MOS sensors are sensitive not only to visible light components but also to infrared components, so digital still cameras, movies In an imaging apparatus such as a camera or a medical camera, an infrared cut filter is disposed in front of the imaging element so as to receive only visible light.

  In such an imaging apparatus, if the infrared cut filter is removed to receive the infrared component, the amount of light incident on the imaging device increases, and the subject can be brightly imaged even under low illuminance. However, since the infrared wavelength is in a wavelength region that cannot be detected by the human eye and the infrared does not have original color information, the white balance of the captured image including the infrared component is significantly deteriorated. Therefore, in a normal infrared irradiation type imaging device, an imaging signal including an infrared component is treated as a luminance signal, and a monochrome image is generated.

  In recent years, there has been a demand from the market to generate a color image close to an actual appearance, not a monochrome image, using such an imaging device.

In Patent Document 1, when an infrared cut filter is disposed at an external position (at night shot shooting), black body curve data LB considering an infrared component is read and black body curve control is performed. Alternatively, gray world control is performed so that the ratio of the video signals R, G, and B is 1. Accordingly, there is a description that optimal white balance control is performed according to the shooting situation.
JP 2005-130317 A

  However, the black body curve control based on the black body curve data LB in consideration of the former infrared component is performed by the arrangement of the infrared cut filter while the ratio and / or the difference between the visible light amount and the infrared light amount is unknown. Since the body curve data LA and LB are selected, the black body curve is not optimized.

  Further, since the latter gray world control is based on the premise that there is no bias in the color of the subject, white balance cannot be appropriately processed if the subject color is biased.

  The present invention has been made in view of these technical problems, and in an infrared irradiation type imaging apparatus that makes it possible to brightly image a subject under low illuminance by irradiating infrared rays, white balance is appropriately set. The purpose is to obtain a color image with high color reproducibility.

  An infrared irradiation type imaging apparatus capable of imaging an object under low illuminance by irradiating infrared light, and irradiating the object with infrared irradiating means, and imaging the subject to form an optical image An optical unit for generating, an infrared cut filter for generating an optical image composed of visible light by blocking infrared rays included in the optical image, and having a structure capable of moving forward and backward with respect to the optical path of the optical unit; Visible light imaging control that inserts an infrared cut filter into the optical path without irradiating the infrared ray, and irradiates the subject with infrared light, and causes the infrared cut filter to exit the optical path. Infrared imaging means including infrared imaging control, imaging means for photoelectrically converting an optical image to generate a visible light imaging signal or infrared imaging signal for each color, a visible light imaging signal and an infrared imaging signal And a visible light calculation / separation unit for each color that separates and outputs an infrared imaging signal into a visible light calculation imaging signal and an infrared calculation imaging signal for each color based on the ratio and / or the difference between It has a light source discrimination that discriminates whether or not the light source irradiated at the time of visible light imaging control is a black body trajectory for the imaging signal, and performs white balance based on the result of the light source discrimination, and white balance imaging First white balance processing means for generating a signal, and integrating the white balance imaging signal for each color, and matching the color balance of the visible light calculation imaging signal based on the ratio of each visible light color, which is the ratio of the integral values A color balance matching unit that generates a color-matched image pickup signal, and a second photon that generates a white-corrected image pickup signal by performing white balance correction on the color-matched image pickup signal based on a result of light source discrimination. And Ito balance processing means performs the white correction image signal, an infrared operation imaging signal, image processing, and includes image processing means for generating a color image signal.

  According to the present invention, when a subject under low illuminance is imaged by irradiating infrared rays, even if the subject has a color bias, an effect of obtaining a good white balance can be obtained. is there.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
FIG. 1 is a block diagram showing a configuration of an infrared irradiation type imaging apparatus according to the first embodiment of the present invention.

  In FIG. 1, this infrared irradiation type imaging apparatus includes an infrared light emitting diode (infrared irradiation means) 1, a lens (optical means) 2, an infrared cut filter 3, an infrared control unit (infrared control means) 4, An image sensor (imaging means) 5, a visible light calculation separation unit for each RGB (visible light calculation separation unit for each color) 6, a first white balance processing unit (first white balance processing means) 7, and color balance matching A unit (color balance matching unit) 8, a second white balance processing unit (second white balance processing unit) 9, and an image processing unit (image processing unit) 10.

  The infrared light emitting diode 1 irradiates the subject 11 with infrared rays. The lens 2 forms an optical image by forming an image of the subject 11. The infrared cut filter 3 has a structure capable of moving forward and backward with respect to the optical path of the lens 2 and blocks an infrared ray included in the optical image to generate an optical image composed of visible light.

  The infrared control unit 4 does not irradiate the subject 11 with infrared light, and controls the visible light imaging control for inserting the infrared cut filter 3 into the optical path, irradiates the subject 11 with infrared light, and And infrared imaging control for causing the infrared cut filter 3 to leave the optical path. The image sensor 5 photoelectrically converts an optical image to generate a visible light imaging signal RGBxy for each RGB color or an infrared imaging signal RGBIxy for each RGB color.

  The per-RGB visible light calculation / separation unit 6 converts the infrared imaging signal RGBIxy into a visible light arithmetic imaging signal [RGBxy] for each RGB color based on the ratio and / or difference between the visible light imaging signal RGBxy and the infrared imaging signal RGBIxy. And the infrared arithmetic imaging signal [Ixy].

  A specific example of this operation separation will be described.

First, for the visible light imaging signal RGBxy,
RGBxy = (R, G, B) xy
And RGB for each color and then integrating for each RGB color,
(ΣR, ΣG, ΣB)
= (∬ (Rxy) dxdy, ∬ (Gxy) dxdy, ∬ (Bxy) dxdy)
It is.

In addition, regarding the infrared imaging signal RGBIxy,
Ixy = (rI, gI, bI) xy
So
RGBIxy
= ((R + rI), (G + gI), (B + bI)) xy
And RGB for each color and then integrating for each RGB color,
(Σ (R + rI), Σ (G + gI), Σ (B + bI))
= (∬ ((R + rI) xy) dxdy, ∬ ((G + gI) xy) dxdy, ∬ ((B + bI) xy) dxdy)
It is.

Here, if a method of separating the R color of the infrared imaging signal RGBIxy, (R + rI) xy, into a visible light computed imaging signal [Rxy] and an infrared computed imaging signal [rIxy] by a ratio,
[Rxy] = (R + rI) xy * ΣR / Σ (R + rI)
[RIxy] = (R + rI) xy- [Rxy]
It is.

G color and B color are the same as R color,
[Gxy] = (G + gI) xy * ΣG / Σ (G + gI)
[GIxy] = (G + gI) xy- [Gxy]
[Bxy] = (B + bI) xy * ΣB / Σ (B + bI)
[BIxy] = (B + bI) xy- [Bxy]
It is.

In this way
RGBIxy
= [RGBxy] + [(rI, gI, bI) xy]
= [RGBxy] + [Ixy]
In this way, operation separation by ratio is performed.

Alternatively, if a method of separating the R color of the infrared imaging signal RGBIxy, (R + rI) xy, into a visible light arithmetic imaging signal [Rxy] and an infrared arithmetic imaging signal [rIxy] by a difference,
[Rxy] = (R + rI) xy-rIxy
[RIxy] = (R + rI) xy- [Rxy]
It is. Here, the rIxy signal is an xy signal extracted based on a difference between the visible light imaging signal RGBxy and the infrared imaging signal RGBIxy.

Similarly for G color and B color,
[Gxy] = (G + gI) xy-gIxy
[GIxy] = (G + gI) xy- [Gxy]
[Bxy] = (B + bI) xy-bIxy
[BIxy] = (B + rI) xy- [Bxy]
It is.

In this way, even if the operation is separated by difference,
RGBIxy
= [RGBxy] + [(rI, gI, bI) xy]
= [RGBxy] + [Ixy]
It is possible to calculate.

  The first white balance processing unit 7 determines whether or not the light source irradiated at the time of visible light imaging control is a locus of a black body (for example, a sun black body) with respect to the visible light imaging signal RGBxy. And white balance based on the result of the light source discrimination to generate a white balance image signal wRGBxy.

  Further, when it is determined by the light source determination by the first white balance processing unit 7 that the irradiation light source at the time of visible light imaging control is not a black body locus, this light source determination is performed by using the fluorescent light source as the irradiation light source. Determine if it is a thing.

The color balance matching unit 8 integrates the white balance imaging signal wRGBxy for each color of RGB, and based on a visible light color ratio (ΣR: ΣG: ΣB) that is a ratio of the integrated values, a visible light calculation imaging signal [RGBxy]. Are matched to generate a color matching image signal c [RGBxy].

  The second white balance processing unit 9 performs white balance correction on the color matching imaging signal c [RGBxy] based on the light source discrimination result (memory) described above, and generates a white corrected imaging signal w [RGBxy]. To do.

  The image processing unit 10 performs image processing on the white correction image pickup signal w [RGBxy] and the infrared arithmetic image pickup signal [Ixy] obtained in this way, and then combines them, or combines them. Image processing is performed to generate a color image signal.

  FIG. 2 is a graph showing an example of imaging signal levels (R, G, B) for each RGB color when a white subject 11 under low illuminance is imaged under visible light imaging conditions.

  FIG. 3 is a graph showing an example of imaging signal levels (R, G, B) for each of the RGB colors when the white subject 11 under low illuminance is imaged under infrared imaging conditions.

  In FIGS. 2 and 3, the hatched portion indicates the level of the imaging signal photoelectrically converted by the visible light component, and in FIG. 3, it is indicated by a white frame. In the portion shown, the level of the imaging signal photoelectrically converted by the infrared component is shown.

  FIG. 3 shows that the G signal level of the imaging signal is improved to the G signal level at the lowest subject illuminance by irradiating with infrared rays and imaging. FIG. 3 shows that the subject 11 under low illumination is brightly imaged.

  FIG. 4 shows an example in which the imaging signal levels (R, G, B) shown in FIG. 3 are separated into a visible light computed imaging signal level (R, G, B) and an infrared computed imaging signal level IR. It is a graph to show.

  In FIG. 4, the image signal RGBIxy is separated into a visible light calculation image pickup signal [RGBxy] and an infrared calculation image pickup signal [Ixy] by the visible light calculation separation unit 6 for each RGB, and one pixel in the xy plane signal It is an example about.

  FIG. 5 is a graph showing an example in which color balance matching is performed on the visible light calculation imaging signal levels (R, G, B) shown in FIG.

In FIG. 5, the visible light calculation imaging signal levels (R, G, B) are shown in a state in which the color balance is matched and the signal levels of RGB colors are substantially aligned. This is because the visible light calculation imaging signal [RGBxy] is color-balanced based on the ratio ΣR: ΣG: ΣB for each visible light color,
This is because the color matching imaging signal c [RGBxy] is generated.

  However, this color balance aligns the integrated values for each of the RGB colors in the frame, and is not a function to project the white object 11 in white. Therefore, for example, as shown in FIG. Although they are almost uniform, they are not necessarily exactly uniform.

  FIG. 6 shows an example in which the color matching imaging signal level (R, G, B) shown in FIG. 5 is represented by a vector scope.

  In FIG. 6, the color matching imaging signal level (R, G, B) is slightly deviated from the center point of the vector scope even though it is a white bright spot obtained by imaging the white subject 11. It has been shown that

  Such a shift in the color matching imaging signal level (R, G, B) means that the white balance accuracy of the color matching imaging signal c [RGBxy] is poor. Therefore, the color matching imaging signal c [RGBxy] is subjected to white balance correction by the second white balance processing unit 9 to correct the phase of the slightly bright white bright spot.

  The second white balance processing unit 9 extracts a signal having a relatively high signal level from the color matching image pickup signal c [RGBxy], and collates the extracted signal with a black body locus so that a white region is obtained. Estimation and white balance correction is performed on the estimated white region. In addition, if the extracted signal and the blackbody locus collate with the blackbody locus as a result of collation, the white area is estimated by collating the extracted signal with the chromaticity range of the fluorescent lamp. To do.

  This extracted signal will be supplementarily described.

  FIG. 7 shows an example of the reflectance (%) for each color under uniform illuminance.

  As shown in FIG. 7, the white (W) subject 11 has a higher reflectance (%) than the subject 11 of other colors (yellow Ye, cyan Cy, green G, magenta Mg, red R, blue B). There are statistically many subjects 11.

  Thus, extracting a signal having a relatively high signal level from the color matching imaging signal c [RGBxy] means extracting the xy plane coordinates of the subject 11 having a high reflectance (%). This extracted signal becomes the primary candidate for the temporary white area.

  FIG. 8 shows an example in which the extraction signal and the range of light source discrimination (black body locus, fluorescent lamp) are represented by a vector scope.

  As shown in FIG. 8, the second white balance processing unit 9 performs the hue and saturation of the extraction signal (provisional white region primary candidate), the locus of the black body, or the fluorescent light source. The white region (secondary) is estimated by collating with the chromaticity range. In other words, a primary candidate for the white region is extracted from the estimation of the reflectance (%) of the subject 11, and the light source discrimination obtained by the first white balance processing unit 7 is further applied to the primary extraction signal. The white region (second order) is estimated by comparing with the result of the above.

  FIG. 9 shows an example in which white balance correction is performed on the white region (secondary) using a vector scope.

  As described above, the white balance correction based on the light source discrimination result obtained by the first white balance processing unit 7 is performed on the white region estimated by the secondary extraction by the second white balance processing unit 9. It is given.

  FIG. 10 shows an example in which the phase of the bright spot in the white region is corrected to the center point of the vector scope.

  In FIG. 10, the above-described white balance correction is applied to the color matching image pickup signal levels (R, G, B). Here, when compared with the graph shown in FIG. It is shown that the white level is uniform.

  As described above, the infrared irradiation type imaging device according to the first embodiment of the present invention applies a color bias to the subject 11 when the subject 11 under low illuminance is irradiated with infrared rays. Even if there is a case, it is possible to perform a good white balance process.

(Second Embodiment)
FIG. 11 is a block diagram showing a configuration of an infrared irradiation type imaging apparatus according to the second embodiment of the present invention.

  In FIG. 11, this infrared irradiation type imaging device includes an infrared light emitting diode 1 (infrared irradiation means), a lens 2 (optical means), an infrared cut filter 3, an infrared control unit 4 (infrared control means), Image sensor 5 (imaging unit), RGB visible light calculation / separation unit 6 for each color (visible light calculation / separation unit for each color), and first / second white balance processing unit 20 (first / second white balance processing unit) ), A color balance matching unit 8 (color balance matching unit), and an image processing unit 10 (image processing unit).

  As shown in FIG. 11, the infrared irradiation type imaging apparatus according to the second embodiment is compared with the infrared irradiation type imaging apparatus according to the first embodiment. Instead of the second white balance processing unit 9, a first / second white balance processing unit 20 is provided.

  Since the first white balance processing unit 7 and the second white balance processing unit 9 already shown in FIG. 1 have a common arithmetic processing circuit, the first / second white balance shown in FIG. The processing unit 20 may be provided as a device in which both are integrated. In addition, the infrared control unit 4 shown in FIG. 11 performs switching control between execution of the first white balance and execution of the second white balance.

  In this way, by providing an integrated white balance processing unit, cost reduction and downsizing of the infrared irradiation imaging apparatus may be designed.

  In addition, since the operation | movement effect | action in each block of the infrared irradiation type imaging device by 2nd Embodiment is the same as the thing of 1st Embodiment, the description is abbreviate | omitted.

  As described above, in the infrared irradiation type imaging device according to the second embodiment of the present invention, when the subject 11 under low illuminance is irradiated with infrared rays to pick up an image, the subject 11 has a color bias. Even if there is a case, it is possible to perform a good white balance process.

  The infrared irradiation type imaging device according to the first and second embodiments of the present invention includes a moving image camera, a still image camera, a measuring instrument, an industrial camera, a medical camera, and a movie shooting camera. It can be applied to various products.

  The embodiment of the present invention has been described in detail above with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes design changes and the like without departing from the gist of the present invention.

  Further, the above-described embodiments include inventions at various stages, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements. For example, even if some constituent requirements are deleted from all the constituent requirements shown in the embodiment, the problem described in the column of the problem to be solved by the invention can be solved, and the effect described in the column of the effect of the invention Can be obtained as an invention.

1 is a block diagram illustrating a configuration of an infrared irradiation imaging device according to a first embodiment of the present invention. It is a graph which shows an example of the imaging signal level (R, G, B) for every RGB color at the time of imaging a white photographic subject under low illumination on visible light imaging conditions. It is a graph which shows an example of the imaging signal level (R, G, B) for every RGB color at the time of imaging the white photographic subject under low illumination on infrared imaging conditions. 4 is a graph showing an example in which the imaging signal levels (R, G, B) shown in FIG. 3 are separated into a visible light computed imaging signal level (R, G, B) and an infrared computed imaging signal level IR. . 5 is a graph illustrating an example in which color balance matching is performed on the visible light calculation imaging signal levels (R, G, B) illustrated in FIG. 4. 6 shows an example in which the color matching imaging signal levels (R, G, B) shown in FIG. 5 are represented by a vector scope. An example of reflectance (%) for each color under uniform illuminance is shown. An example in which an extraction signal and a range of light source discrimination (black body locus, fluorescent lamp) are represented by a vector scope is shown. An example in which a white balance correction is performed on a white region (secondary) is represented by a vector scope. An example is shown in which the phase of the bright spot in the white region is corrected to the center point of the vector scope. It is a block diagram which shows the structure of the infrared irradiation type imaging device by the 2nd Embodiment of this invention.

1 Infrared light emitting diode (infrared irradiation means)
2 Lens (optical means)
3 Infrared cut filter 4 Infrared control unit (infrared control means)
5 Image sensor (imaging means)
6 Visible light calculation / separation section for each RGB (visible light calculation / separation means for each color)
7 First white balance processing section (first white balance processing means)
8 Color balance matching section (Color balance matching means)
9 Second white balance processing section (second white balance processing means)
10 Image processing unit (image processing means)
11 Subject 20 First / second white balance processing section

Claims (4)

An infrared irradiation type imaging device capable of imaging by irradiating infrared rays to a subject under low illuminance,
Infrared irradiating means for irradiating the subject with the infrared;
Optical means for forming an optical image by imaging the subject;
An infrared cut filter configured to be movable back and forth with respect to the optical path of the optical means, for blocking infrared rays included in the optical image and generating the optical image composed of visible light;
Visible light imaging control for inserting the infrared cut filter into the optical path without irradiating the subject with the infrared ray, irradiating the subject with the infrared ray, and the infrared ray An infrared control means comprising: an infrared imaging control for causing the cut filter to exit the optical path;
Imaging means for photoelectrically converting the optical image to generate a visible light imaging signal or an infrared imaging signal for each color;
Based on a ratio and / or difference between the visible light imaging signal and the infrared imaging signal, the infrared imaging signal is separated into a visible light arithmetic imaging signal and an infrared arithmetic imaging signal for each color and output. A means for separating and calculating visible light for each color;
The visible light imaging signal has a light source determination for determining whether the light source irradiated during the visible light imaging control is a locus of a black body, and performs white balance based on the result of the light source determination. First white balance processing means for generating a white balance imaging signal;
The color that integrates the white balance imaging signal for each color and matches the color balance of the visible light calculation imaging signal based on the ratio of each visible light color that is the ratio of the integrated value to generate a color matching imaging signal Balancing means;
Second white balance processing means for performing white balance correction on the color matching imaging signal based on the result of the light source determination and generating a white corrected imaging signal;
Image processing means for performing color processing on the white correction imaging signal and the infrared arithmetic imaging signal to generate a color image signal;
An infrared irradiation type imaging apparatus comprising: When it is determined by the light source determination by the first white balance processing unit that the irradiation light source during the visible light imaging control is not the locus of the black body,
The infrared light irradiation type imaging apparatus according to claim 1, wherein the light source determination determines whether or not the irradiation light source is a fluorescent lamp. The second white balance processing means extracts a signal having a relatively high signal level from the color matching imaging signal, and compares the extracted signal with the locus of the black body or the fluorescent lamp to obtain a white color. The infrared irradiation imaging apparatus according to claim 2, wherein an area is estimated and white balance correction is performed on the white area. The first white balance processing means and the second white balance processing means have a common arithmetic processing circuit,
The infrared irradiation type imaging apparatus according to claim 1, wherein the infrared control unit performs switching control between the first white balance processing unit and the second white balance processing unit.

Images