JP5100615B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus such as a video camera in which an infrared cut filter can be inserted and removed.

  An imaging device such as a video camera is configured by performing various signal processing on an imaging optical system that forms a subject image with incident light, an imaging device that photoelectrically converts the subject image, and an output signal from the imaging device. And an image generation circuit for generating an image. As the image sensor, a CCD sensor, a CMOS sensor, or the like is used. In a single-plate type image pickup apparatus using only one image sensor, color filters corresponding to different colors are arranged for each pixel of the image sensor.

  To obtain red (R), green (G) and blue (B) color signals, RGB primary color filters are used as color filters, magenta (Mg), cyan (Cy), yellow (Ye), and G. May be used. All of these color filters are designed to transmit only R, G, B, Mg, Cy, or yellow light using dyes or pigments. On the other hand, it has a certain transmittance. In addition, since the photoelectric conversion unit of the image sensor is mainly composed of a semiconductor such as silicon (Si), the spectral sensitivity characteristic of the photoelectric conversion unit is sensitive to near infrared light having a long wavelength. For this reason, the signal obtained from the image sensor provided with the color filter also includes a photoelectric conversion component of light in the near infrared region.

  On the other hand, the color vision characteristic, which is a sensitivity characteristic for human colors, and the relative visibility characteristic, which is a sensitivity characteristic for brightness, are sensitivity characteristics from the visible range of 380 nm to 780 nm, and have almost sensitivity in the wavelength range longer than 700 nm. No. Therefore, in general imaging, in order to match color reproducibility with human color vision characteristics, an infrared cut filter for correcting visibility (hereinafter referred to as IRCF) that does not allow light in the near infrared region to pass through in front of the imaging device. ) Must be provided. On the other hand, when the sensitivity is more important than the color reproducibility, such as a surveillance camera, the near infrared light is used without using the IRCF in order to use near infrared light. Good.

  For this reason, when color reproducibility is important, IRCF is placed (inserted) in front of the image sensor to output a color image, and when priority is given to the sensitivity, IRCF is removed (extracted) from the front of the image sensor. An imaging device has been proposed. Normally, when the IRCF is removed from the front of the image sensor, the color balance of the image is lost, so a black and white image is output.

Patent Document 1 discloses an imaging apparatus that performs a matrix operation on RGB colors and outputs a color image without providing an IRCF.
JP 2005-295419 A

  However, as in the imaging device disclosed in Patent Document 1, it is difficult to perform matrix calculation for each color of RGB by software, and it is difficult to perform matrix calculation by hardware. Need to be added. Furthermore, since the comparison of each color is not performed before and after IRCF insertion / extraction, there is a possibility that the color of the same subject may change greatly due to IRCF insertion / extraction.

  According to the present invention, when a color image can be output even after the IRCF is extracted, the processing load by software is small, and the color image can be output without the need for additional hardware. An imaging device is provided.

  An image pickup apparatus according to one aspect of the present invention includes an image pickup device that photoelectrically converts a subject image formed by an image pickup optical system, an image generation unit that generates an image using an output signal from the image pickup device, and the generated image generation unit. Information acquisition means for acquiring information relating to the captured image, an infrared cut filter that is inserted into and removed from the optical path from the imaging optical system to the imaging element, and control means for switching the imaging mode between the color mode and the monochrome mode. . And a control means switches the imaging mode after an infrared cut filter is extracted according to the variation | change_quantity of the information regarding the image by insertion / extraction of an infrared cut filter.

  An imaging device according to another aspect of the present invention includes an imaging device that photoelectrically converts a subject image formed by an imaging optical system, and an image generation unit that generates an image using an output signal from the imaging device. A light source information generating means for generating information relating to a light source that illuminates a subject in the generated image, an infrared cut filter that is inserted into and extracted from an optical path from the imaging optical system to the imaging device, and an imaging mode as a color mode. Control means for switching between black and white mode. And a control means switches the imaging mode after an infrared cut filter is extracted according to the information regarding a light source, It is characterized by the above-mentioned.

  According to the present invention, when a color image can be output even after extraction of the infrared cut filter, the color image is output without a processing load by software and without the need for adding extra hardware. be able to. In addition, whether or not to output a color image after extraction of the infrared cut filter is determined according to the amount of change in information regarding the image due to insertion or extraction of the infrared cut filter or information about the light source. The change in the color of the same subject in the color image can be suppressed.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 shows the configuration of a video camera as an image pickup apparatus that is Embodiment 1 of the present invention. Reference numeral 1 denotes an imaging lens (imaging optical system), 2 denotes an IRCF (infrared cut filter), and 3 denotes an imaging element (hereinafter referred to as a CCD) constituted by a CCD sensor or a CMOS sensor. The light incident on the imaging lens 1 forms a subject image on the light receiving surface of the CCD 3 by the imaging action of the imaging lens 1.

  Reference numeral 4 denotes an AD converter that converts an analog signal from the CCD 3 into a digital signal. Reference numeral 5 denotes a video signal processing unit. When the color filter provided in the CCD 3 is a complementary color filter, a predetermined calculation is performed based on signals (output signals) indicating Cy, Mg, Ye and G components from the CCD 3. , R, G, and B imaging signals, that is, original images are generated. When the CCD 3 has a primary color filter, the video signal processing unit 5 generates R, G, and B imaging signals based on output signals indicating R, G, and B components from the CCD 3.

  Reference numeral 6 denotes a WB processing unit (white balance processing means), which amplifies R, G, and B imaging signals in the original image. Specifically, gains obtained by receiving R gain, G gain, and B gain (hereinafter also referred to as color gains) calculated by the RGB gain calculation unit 9 described later and multiplying each of the R, G, and B original signals, respectively. Is changed to perform white balance processing (hereinafter referred to as WB). The video signal processing unit 5 and the WB processing unit 6 constitute image generation means. The image after the white balance process is referred to as a WB image in the following description.

  Reference numeral 7 denotes a luminance / color information calculation unit serving as information acquisition means. Luminance information (luminance signal corresponding to the luminance of the subject) is an evaluation value for AE (automatic exposure control) and IRCF insertion / extraction determination from the WB image. ) Get Y. In addition, the luminance / color information calculation unit 7 acquires R, G, and B color signals that are evaluation values for viewing changes in color gain or color information for WB due to IRCF insertion / extraction. The color gain and color information correspond to information about the image.

  A γ processing unit 8 performs γ processing on the WB image and generates an output image. The WB gain calculation unit 9 detects a white region (specific region) of the subject from the R, G, and B color signals acquired by the luminance / color information calculation unit 7, and R, G, and B in the white region The R gain, the G gain, and the B gain are calculated so that the values of the B color signal are equal to each other. The calculated R gain, G gain, and B gain are transmitted to the WB processing unit 6.

  10 is a color information storage unit, and 11 is a color information comparison unit. The color information storage unit 10 stores R, G, and B color signals (that is, color information) every predetermined time, and transmits the color information to the color information comparison unit 11.

  The color information comparison unit 11 is the difference between the color information of a predetermined time before the current time transmitted from the color information storage unit 10 and the current color information transmitted from the luminance / color information calculation unit 7, that is, the color information. The amount of change is calculated.

  Reference numeral 12 denotes an IRCF insertion / extraction determination unit, which determines IRCF2 insertion / extraction based on the luminance information Y acquired by the luminance / color information calculation unit 7. When the luminance information Y is large, the IRCF insertion / extraction determination unit 12 determines to insert the IRCF 2 into the optical path from the imaging lens 1 (in the imaging optical system) to the imaging element 3. Thereby, imaging is performed without using light in the near infrared region (near infrared light) cut by the IRCF 2, and a color image is output from the video signal output unit 15 described later.

  On the other hand, when the luminance information Y is small, the IRCF insertion / extraction determination unit 12 determines to extract IRCF2 from the optical path. By extracting the IRCF 2 from the optical path, the image sensor 3 also photoelectrically converts near-infrared light, and the sensitivity of the image sensor 3 is improved. At this time, since the color balance is lost due to the influence of near-infrared light, it is common to output a black and white image from the video signal output unit 15 described later.

  An IRCF control circuit 13 performs IRCF2 insertion / extraction control in response to the determination result from the IRCF insertion / extraction determination unit 12. Specifically, the video camera of the present embodiment has an insertion / extraction mechanism that operates so that the IRCF 2 is inserted into / extracted from the optical path by the driving force of the actuator, and the IRCF control circuit 13 includes the IRCF control circuit 13. Controls actuator drive.

  Reference numeral 14 denotes a video selection unit as control means. In response to the determination result from the IRCF insertion / extraction determination unit 12, the video selection unit 14 selects either color image output or monochrome image output according to the amount of change in color information from the color information comparison unit 11 due to IRCF2 insertion / extraction. Select (set). In the following description, an imaging mode for outputting a color image is also referred to as a color mode, and an imaging mode for outputting a monochrome image is also referred to as a monochrome mode.

  The video signal output unit 15 receives the selection information from the video selection unit 14 and outputs a color image or a monochrome image. The output here corresponds to displaying an image on a display (not shown) or recording on a recording medium (semiconductor memory, magnetic tape, optical disk, etc.) (not shown).

  Next, selection of color image output and monochrome image output after extraction of IRCF2 in this embodiment will be described.

  In cameras that are used day and night, such as surveillance cameras, when the brightness is sufficient during the day, IRCF 2 is inserted into the optical path to output a color image. When the brightness becomes insufficient at night, sensitivity is more important than color reproducibility due to the effect of noise due to gain. In that case, since near-infrared light is used, the IRCF 2 is not inserted into the optical path, and the near-infrared light is photoelectrically converted by the CCD 3.

  2 and 3 show the spectral transmittance characteristics of the CCD. In a CCD having a primary color filter (hereinafter referred to as a primary color CCD), transmittance (that is, an imaging signal) for R, G, and B light as shown in FIG. 2 is obtained. Further, in a CCD having a complementary color filter (hereinafter referred to as a complementary color CCD), transmittance (imaging signal) for Mg, Gr, Cy and Y light as shown in FIG. 3 is obtained. “IRCF” in the figure is a spectral transmittance characteristic of IRCF 2 that removes light (near infrared light and infrared light) on the longer wavelength side than R.

  By extracting the IRCF 2 from the optical path, the CCD 3 becomes sensitive to light having a wavelength longer than about 650 nm that has been removed by the IRCF 2, so that the sensitivity of the CCD 3 can be increased. However, since the sensitivity to R light or Mg and Ye light is significantly increased due to the influence of near infrared light, the color balance is lost. If the color balance is lost, color reproduction cannot be performed correctly, so normally a black and white image is output.

  Here, FIGS. 4, 5, 6 and 7 show the spectral characteristics of various light sources. In the fluorescent lamp and the mercury lamp as shown in FIGS. 4 and 5, there is no wavelength component of 650 nm or more, and the amount of change in color information due to insertion / extraction of IRCF2 is not large. Therefore, it is possible to output a color image with substantially maintained color reproducibility. Since the color information is main information of the subject, it is preferable to output a color image.

  FIG. 6 shows the spectral characteristics of infrared illumination. In general, LEDs are often used for infrared illumination, and short wavelength light of 800 to 1000 nm is emitted. When the wavelength is longer than 800 nm, the transmittance of each color filter is almost the same for both the primary color CCD and the complementary color CCD, and the sensitivity of the CCD increases, but the color balance is not greatly lost. For this reason, a color image can be output. On the other hand, since the incandescent lamp shown in FIG. 7 contains many wavelength components of 650 nm or more, the image is reddish and the color balance is greatly lost. Therefore, it is better to extract IRCF2 out of the optical path and switch to monochrome image output.

  In the present embodiment, based on such a theory, the imaging mode is switched according to the amount of change in information (color information) related to the image due to IRCF2 insertion / extraction.

  Next, switching control between color image output and black-and-white image output corresponding to insertion / extraction of IRCF2 will be described using the flowchart of FIG. This control is executed mainly by the WB gain calculation unit 9, the luminance / color information calculation unit 7, the IRCF insertion / extraction determination unit 12, the IRCF control circuit 13, and the video selection unit 14 operating according to a computer program.

  When the luminance information Y is smaller than the threshold value Yth, the IRCF insertion / extraction determination unit 12 determines that IRCF2 should be extracted, and sends an IRCF2 extraction instruction to the IRCF control circuit 13. Thereby, the IRCF control circuit 13 extracts IRCF2 out of the optical path. At this time, the IRCF insertion / extraction determination unit 12 sends information indicating that the IRCF 2 has been extracted to the video selection unit 14 (S801).

  The luminance / color information calculation unit 7 divides the WB image into a plurality of regions and stores the R and B color information in the color information storage unit 10 for each predetermined time. The color information comparison unit 11 performs color information after extraction of the IRCF2 from the luminance / color information calculation unit 7 (in the extraction state) and before extraction of IRCF2 from the color information storage unit 10 (in the insertion state). ) The color information is compared with each other, and color information changes ΔR and ΔB are calculated (S802).

  Here, the white region (specific region) in the WB image is a region that reflects all wavelength components in the wavelength region of 400 to 650 nm. By using the white area calculated by the WB gain calculation unit 9 as the area for comparing the color information, it is possible to calculate the change amounts ΔR and ΔB of the color information without any deviation.

  If a region of interest in the WB image is determined, the region may be designated (set) in advance, and the amount of change in color information before and after IRCF2 extraction in that region may be obtained. In addition, when a subject to be noticed is determined in the WB image, the movement of the subject is detected by moving object detection processing, and based on the detection result, before and after IRCF2 extraction of an area including at least a part of the subject You may obtain | require the variation | change_quantity of color information. By providing a selection means for selecting an area for obtaining the amount of change in color information (information relating to an image), malfunctions caused by changes in color information in a region other than the region of interest or the region including the subject of interest are reduced. be able to. The attention area may be selected by the user via an operation switch (not shown). The same applies to other embodiments described later.

  In a scene where there is a lot of movement of the subject, the amount of change in color information in an area where the color change is small over a long time, or the amount of change in color information as an average value in the entire WB image may be obtained. Accordingly, it is possible to avoid erroneous calculation of the color information changes ΔR and ΔB due to the movement of the subject before and after the insertion and removal of the IRCF2.

  Next, the video selection unit 14 compares the color information changes ΔR and ΔB calculated in S802 with thresholds Rth and Bth as predetermined values, respectively (S803). The threshold values Rth and Bth are the maximum values of the change amount of the color information that allows the change (disintegration) of the color balance, and are parameters that depend on the number of pixels of the image, the noise feeling, and the like. The thresholds Rth and Bth may be arbitrarily set according to the usage environment of the video camera, the user's preference of the image, and the like.

Here, as a condition for determining that the change amount of the color information is larger than the threshold value,
ΔR> Rth or ΔB> Bth (1)
By setting, it becomes possible to react sensitively to changes in color information.

As a condition for determining that the amount of change in color information is greater than a threshold value,
ΔR> Rth and ΔB> Bth (2)
Is set, it can be determined that the amount of change in color information is large only when there is a certain change in color information, as compared to Equation (1).

  As described above, the combination of the threshold values Rth and Bth and the logic (or, and) prevents the erroneous determination with respect to the change in color information, and determines that there is a color change only with respect to the intended color change. Is possible.

  Also, when determining the amount of change in the color information, it is preferable to average the color information before and after extracting the IRCF 2 in terms of time. By using the color information averaged over time in this way, the amount of change in color information can be accurately calculated without being greatly affected by noise due to gain or the like.

  When the video selection unit 14 determines that the amount of change in color information before and after insertion / extraction of the IRCF 2 is larger than the threshold, that is, when an unacceptable collapse occurs in the color balance, it is assumed that color reproducibility is not secured. A monochrome image output is selected (S804). That is, the monochrome mode is selected (set) as the imaging mode after the IRCF 2 is extracted.

  On the other hand, when IRCF2 is extracted but it is determined that the amount of change in color information is smaller than the threshold value, that is, when the color balance is not largely lost and the color reproducibility is almost ensured, video selection is performed. The unit 14 selects color image output (S805). That is, the color mode is selected as the imaging mode after extracting the IRCF2.

  Thus, in this embodiment, a color image is output after IRCF2 extraction only when the amount of change in color information due to IRCF2 insertion is small, so the color of the same subject in the color image due to IRCF2 extraction is output. Change can be suppressed. Further, the processing load due to software when outputting a color image after extracting the IRCF 2 is small, and there is no need to add extra hardware.

  Next, the color image output continuation determination process after the IRCF 2 is extracted will be described with reference to the flowchart of FIG. This process is performed by the video selection unit 14.

  The video selection unit 14 calculates the amount of change in the color information every predetermined time T after the IRCF 2 is extracted (color image output state) (S901, S902). The predetermined time T here is determined according to the stability of the color information of R and B. The color information may fluctuate greatly due to the generation of noise components due to the influence of gain or the like. For this reason, the predetermined time T is set to a time during which noise components can be sufficiently absorbed by averaging the color information at the predetermined time T.

  In order to calculate the amount of change in color information, color information before extraction of IRCF2 is compared with color information calculated every predetermined time T after extraction of IRCF2. This is because when the subject is irradiated with illumination light from a light source different from that before IRCF2 extraction, the subject color information changes and correct color reproduction is not performed. This is for determination.

  Then, the video selection unit 14 determines the magnitude of the change amount of the color information, similarly to S803 (S903). When the change amount of the color information is large, the output is switched to monochrome image output (S904), and when the change amount is small, the color image output is continued (S905).

  In this way, by monitoring the change in color information even after the IRCF2 is extracted, it is possible to cope with the change in the light source after the IRCF2 is extracted and to continue outputting a color image with high color reproducibility. it can.

  FIG. 10 shows the configuration of a video camera that is Embodiment 2 of the present invention. In this embodiment, the same components as those shown in FIG. 1 are denoted by the same reference numerals as those in FIG. In this embodiment, a luminance information storage unit 16 is provided instead of the color information storage unit 10 shown in FIG. 1, and a luminance information comparison unit 17 is provided instead of the color information comparison unit 11 shown in FIG. It has been.

  The luminance information storage unit 16 stores luminance information Y as information related to an image every predetermined time, and transmits the luminance information Y to the luminance information comparison unit 17. The luminance information comparison unit 17 calculates the amount of change in luminance information from the luminance information transmitted from the luminance information storage unit 16 a predetermined time ago and the current luminance information calculated and transmitted by the luminance / color information calculation unit 7. Is calculated.

  As described in the first embodiment, when the luminance information Y is large, the IRCF insertion / extraction determination unit 12 determines to insert IRCF2 into the optical path. On the other hand, when the luminance information Y is small, it is determined to extract IRCF2 from the optical path.

  In the present embodiment, the video selection unit 14 selects either the color image output or the monochrome image output with the IRCF 2 extracted based on the determination result from the IRCF insertion / extraction determination unit 12 and the luminance information from the luminance information comparison unit 17. Select what to do.

  Next, switching control between color image output and black-and-white image output according to IRCF2 insertion / extraction will be described using the flowchart of FIG. This control is executed mainly by the WB gain calculation unit 9, the luminance / color information calculation unit 7, the IRCF insertion / extraction determination unit 12, the IRCF control circuit 13, and the video selection unit 14 operating according to a computer program.

  When the luminance information Y is smaller than the threshold value Yth, the IRCF insertion / extraction determination unit 12 determines that IRCF2 should be extracted, and sends an IRCF2 extraction instruction to the IRCF control circuit 13. Thereby, the IRCF control circuit 13 extracts IRCF2 out of the optical path. At this time, the IRCF insertion / extraction determination unit 12 sends information indicating that the IRCF 2 has been extracted to the video selection unit 14 (S1101).

  The luminance / color information calculation unit 7 divides the WB image into a plurality of regions, and stores the luminance information Y in the luminance information storage unit 16 for each predetermined time. The luminance information comparison unit 17 compares the luminance information after the IRCF2 extraction from the luminance / color information calculation unit 7 with the luminance information before the IRCF2 extraction from the luminance information storage unit 16, and the amount of change in the luminance information Y ΔY is calculated (S1102).

  In a scene where there is a lot of movement of the subject, the amount of change in luminance information in an area where the change in luminance is small for a long time or the amount of change in luminance information as an average value in the entire WB image may be obtained. Thereby, it is possible to avoid erroneous calculation of the luminance information change amount ΔY due to the movement of the subject before and after the insertion and removal of the IRCF2.

  Next, the video selection unit 14 compares the change amount ΔY of the luminance information Y calculated in S1102 with a threshold value Yth as a predetermined value (S1103). The threshold value Yth is a maximum value of the change amount of the color information that allows the change (disintegration) of the color balance, and is a parameter that depends on the number of pixels of the image, the sense of noise, and the like. The threshold value Yth may be arbitrarily set according to the use environment of the video camera, the user's preference of the image, and the like.

  Further, when determining the amount of change in the luminance information, it is preferable to average the luminance information before and after extracting the IRCF 2 in terms of time. By using luminance information averaged over time in this way, it is possible to accurately calculate the amount of change in luminance information without being greatly affected by noise due to gain or the like.

  When the video selection unit 14 determines that the amount of change ΔY in the luminance information Y before and after insertion / extraction of the IRCF 2 is greater than the threshold Yth, that is, when an unacceptable collapse occurs in the color balance, color reproducibility is not ensured. Then, monochrome image output is selected (S1104). That is, the monochrome mode is selected as the imaging mode after extracting the IRCF2.

  On the other hand, when IRCF2 is extracted but it is determined that the change amount ΔY of the luminance information Y is smaller than the threshold value Yth, that is, when the color balance is not largely lost and the color reproducibility is almost ensured. The video selection unit 14 selects a color image output (S1105). That is, the color mode is selected as the imaging mode after extracting the IRCF2.

  Thus, in this embodiment, a color image is output after IRCF2 extraction only when the amount of change in luminance information due to IRCF2 insertion is small, so the color of the same subject in the color image after IRCF2 extraction is the same. The change of can be suppressed. Further, the processing load imposed by software when outputting a color image is small, and there is no need to add extra hardware.

  Next, the color image output continuation determination processing after IRCF2 extraction will be described using the flowchart of FIG. This process is performed by the video selection unit 14.

  The video selection unit 14 calculates a change amount ΔY of the luminance information Y at every predetermined time T after the IRCF 2 is extracted (color image output state) (S1201, S1202). The predetermined time T here is determined according to the stability of the luminance information. Luminance information may fluctuate greatly due to the generation of noise components due to effects such as gain. For this reason, as the predetermined time T, a time during which noise components can be sufficiently absorbed by averaging the luminance information at the predetermined time T is set.

  In order to calculate the change amount ΔY of the luminance information Y, the luminance information before extraction of IRCF2 is compared with the luminance information calculated every predetermined time T after extraction of IRCF2. This is because when the subject is irradiated with illumination light from a light source different from that before the IRCF2 is extracted after the IRCF2 is extracted, a luminance change that changes the color information of the subject occurs, and correct color reproduction is achieved. This is to determine a state that is not performed.

  Then, the video selection unit 14 determines the magnitude of the change amount of the luminance information as in S1103 (S1203). When the change amount of the luminance information is larger than the predetermined value, the output is switched to monochrome image output (S1204), and when the change amount is smaller than the predetermined value, the color image output is continued (S1205).

  In this way, by monitoring the change in luminance information even after the IRCF2 is extracted, it is possible to cope with the change in the light source after the IRCF2 is extracted and to continue outputting a color image having high color reproducibility. it can.

  FIG. 13 shows the configuration of a video camera that is Embodiment 3 of the present invention. In this embodiment, the same components as those shown in FIG. 1 are denoted by the same reference numerals as those in FIG. In this embodiment, a light source calculation unit 18 and a light source setting unit 19 are provided in addition to the components shown in FIG.

  The light source calculation unit 18 as one of the light source information generation means calculates light source data from the R gain, G gain, and B gain calculated by the WB gain calculation unit 9. The light source data is information regarding the light source that illuminates the subject in the WB image.

  The light source setting unit 19 as other light source information generating means is used when the user manually sets light source data of the environment where the video camera is installed. The set light source data is sent to the WB gain calculation unit 9, and the WB gain calculation unit 9 calculates an R gain, a G gain, and a B gain based on the light source data. The light source data from the light source calculation unit 18 and the light source setting unit 19 is also transmitted to the video selection unit 14 and is used when selecting either color image output or monochrome image output according to the light source.

  Next, selection of color image output and monochrome image output after extraction of IRCF2 in this embodiment will be described.

  FIG. 14 shows the distribution of R gain and B gain by various light sources when WB is performed. The horizontal axis represents the R gain, and the vertical axis represents the B gain. The ellipses in the figure indicate the ranges where R gain and B gain exist under each light source. The range of R gain and B gain when WB is performed differs depending on the type of light source, and the type of light source can be roughly estimated from the range of R gain and B gain. If the type of the light source can be estimated, the ratio of the near infrared component of 650 nm or more to the wavelength component of 400 nm or more can be estimated, and as a result, the amount of change in color information due to IRCF2 insertion / extraction can also be estimated. Therefore, by estimating the type of light source from the R gain and B gain before extraction of IRCF2, it is possible to determine which one of color image output and monochrome image output should be selected after extraction of IRCF2. Thus, in this embodiment, the imaging mode is switched according to the light source data.

  In this embodiment, the amount of change in color information due to insertion / extraction of IRCF2 is not directly obtained, but the amount of change in color information due to insertion / extraction of IRCF2 results in correspondence between the light source data and the amount of change in color information. This corresponds to switching the imaging mode in accordance with.

  Further, in the present embodiment, the light source setting unit 19 that sets the light source by the user is provided on the assumption that the video camera is used under a specific light source. When the type of the light source is set by the light source setting unit 19, the color image output and the monochrome image output are performed based on the light source data set by the light source setting unit 19 instead of the light source estimated by the light source calculation unit 18. It may be determined which one should be selected.

  Furthermore, there may be a case where the imaging environment and the subject are scheduled, such as when the subject is illuminated with light from the light source at a specific time. In such a case, by selecting which of color image output and monochrome image output should be selected in combination with scheduling, it is possible to select an appropriate imaging mode even when the light source changes periodically. it can.

  Next, switching control between color image output and black-and-white image output in accordance with IRCF2 insertion / extraction will be described using the flowchart of FIG. This control is performed mainly by the WB gain calculation unit 9, the light source calculation unit 18, the IRCF insertion / extraction determination unit 12, the IRCF control circuit 13, and the video selection unit 14 operating according to a computer program.

  When the luminance information Y is smaller than the threshold value Yth, the IRCF insertion / extraction determination unit 12 determines that IRCF2 should be extracted, and sends an IRCF2 extraction instruction to the IRCF control circuit 13. Thereby, the IRCF control circuit 13 extracts IRCF2 out of the optical path. At this time, the IRCF insertion / extraction determination unit 12 sends information indicating that the IRCF 2 has been extracted to the video selection unit 14 (S1501).

  Next, the light source calculation unit 18 calculates light source data (S1502). Here, as described above, the light source data is generated from the distribution range of the R gain and the B gain. The light source data is transmitted to the video selection unit 14. Instead of these processes, the light source data set in the light source setting unit 19 may be transmitted to the video selection unit 14.

  Based on the light source data, the video selection unit 14 determines whether or not the light source is a light source whose color balance is largely lost due to the extraction of IRCF2. Specifically, when the near-infrared component of the light source is larger than a predetermined value (for example, when the ratio of the near-infrared component to the wavelength component of 400 nm or more is larger than the predetermined ratio), the color balance is greatly lost (Example) 1 is determined as a light source. On the other hand, when the near-infrared component is smaller than a predetermined value (for example, when the above-mentioned ratio of the near-infrared component is smaller than the predetermined ratio), the color balance does not collapse significantly (the change amount of color information is smaller than the threshold value). ) Determined as a light source.

  The video selection unit 14 selects black and white image output when the light source has a large color balance. That is, the monochrome mode is selected as the imaging mode after extracting the IRCF2. If the light source is not, color image output is selected. That is, the color mode is selected as the imaging mode after the IRCF2 is extracted (S1503, S1504, S1505).

  In this way, in this embodiment, when the light source is a light source with a small amount of change in color information due to insertion / extraction of IRCF2, a color image is output after extraction of IRCF2, so that the color change of the same subject in the color image is detected. Can be suppressed. Further, the processing load imposed by software when outputting a color image is small, and there is no need to add extra hardware.

  When the light source that illuminates the subject is a light source that hardly contains near-infrared light, the color of the subject is determined by the R gain and B gain existing in the hatched area A in FIG. 21 in the WB with the IRCF 2 inserted. It can be corrected to white. For this reason, when the color of the subject cannot be corrected to white by the R gain and B gain existing in the hatched range (hereinafter, referred to as white correction possible gain range) A, the light source includes a near infrared light source. It can be estimated that. That is, it can be estimated that near infrared light is incident on the CCD 3 after the IRCF 2 is extracted, and the color balance is lost.

  In this way, instead of the light source data, information on the light source is acquired as to whether or not the actual R gain and B gain exist in the white correction possible gain range A, not the color gain existing range for each type of light source. May be used. Only when the actual R gain and B gain are in the white correction possible gain range A, it is possible to output a color image after extracting the IRCF2.

  Next, the color image output continuation determination processing after IRCF2 extraction will be described using the flowchart of FIG. This process is performed by the light source calculation unit 18 and the video selection unit 14.

  The light source calculation unit 18 calculates light source data in the same manner as S1502 at every predetermined time T after the IRCF2 is extracted (color image output state) (S1601, S1602). The predetermined time T here is determined according to the stability of the color information of R and B. The color information may fluctuate greatly due to the generation of noise components due to the influence of gain or the like. For this reason, the predetermined time T is set to a time during which noise components can be sufficiently absorbed by averaging the color information at the predetermined time T. The light source data calculated every predetermined time T is transmitted to the video selection unit 14.

  The video selection unit 14 determines whether the light source is a light source capable of outputting a color image, based on the light source data transmitted every predetermined time T. When the light source is capable of outputting a color image, the color image output is maintained, and when the light source is not, switching to monochrome image output is performed (S1603, S1604, S1605).

  Thus, by monitoring the change in the light source data even after the IRCF 2 is extracted, it is possible to cope with the change in the light source after the IRCF 2 is extracted and to continue outputting a color image having high color reproducibility. it can.

  FIG. 17 shows the configuration of a video camera that is Embodiment 4 of the present invention. In this embodiment, the same components as those shown in FIG. 1 are denoted by the same reference numerals as those in FIG. In this embodiment, an IRCF extraction video setting unit 20 is provided in addition to the components shown in FIG.

  The IRCF extraction video setting unit 20 is provided to allow the user to select which one of color image output and monochrome image output is selected when IRCF2 is extracted.

  Next, selection of color image output and monochrome image output after extraction of IRCF2 in this embodiment will be described.

  The imaging mode in the IRCF2 extraction state, that is, whether to select color image output or monochrome image output, depends on the installation environment of the video camera and the user's preference. When the angle of view set for a certain field of view, the set light source, and the subject are determined, the user can set the imaging mode at the time of extraction of the IRCF 2 to prevent malfunctions contrary to the user's intention. Can do.

  In addition, when the imaging environment or subject is scheduled, such as when the angle of view or the subject is periodically changed, or when illumination light is irradiated on the subject at a specific time, The imaging mode may be set.

  FIG. 18 shows the configuration of a video camera that is Embodiment 5 of the present invention. In this embodiment, the same components as those shown in FIG. 1 are denoted by the same reference numerals as those in FIG. In this embodiment, in addition to the components shown in FIG. 1, a WB gain storage unit 21 and a WB gain comparison unit 22 are provided.

  The WB gain storage unit 21 stores the R gain, G gain, and B gain calculated by the WB gain calculation unit 9 every predetermined time. The WB gain comparison unit 22 includes the R gain, G gain, and B gain sent from the WB gain storage unit 21 a predetermined time ago, and the current R gain, G gain, and B gain calculated by the WB gain calculation unit 9. And the amount of change in each color gain as information about the image is calculated.

  Based on the result of determination by the IRCF insertion / removal determination unit 12 and the amount of change in each color gain calculated by the WB gain comparison unit 22, the video selection unit 14 performs either color image output or monochrome image output after IRCF2 extraction. Determine whether to select.

  Selection of color image output and monochrome image output after IRCF2 extraction in the present embodiment is the same as in the first embodiment.

  Next, the operation of whether to output color video or switch to monochrome video by IRCF insertion / extraction will be described using the flowchart of FIG. Switching control between color image output and monochrome image output corresponding to insertion / extraction of IRCF 2 will be described. This control is executed mainly by the WB gain calculation unit 9, the IRCF insertion / extraction determination unit 12, the WB gain comparison unit 22, the IRCF control circuit 13, and the video selection unit 14 operating according to a computer program.

  When the luminance information Y is smaller than the threshold value Yth, the IRCF insertion / extraction determination unit 12 determines that IRCF2 should be extracted, and sends an IRCF2 extraction instruction to the IRCF control circuit 13. Thereby, the IRCF control circuit 13 extracts IRCF2 out of the optical path. At this time, the IRCF insertion / extraction determination unit 12 sends information indicating that the IRCF 2 has been extracted to the video selection unit 14 (S1901).

  The WB gain comparison unit 22 calculates the R gain, G gain, and B gain after extraction of the IRCF 2 calculated by the WB gain calculation unit 9, and the R gain, G gain, and B of the IRCF 2 before extraction from the WB gain storage unit 21. Each B gain is compared. As a result, R gain, G gain, and B gain change amounts ΔRgain, ΔGgain, and ΔBgain due to extraction of IRCF2 are calculated (S1902).

  Next, the video selection unit 14 compares the change amounts ΔRgain, ΔGgain, and ΔBgain calculated in S1902 with threshold values Rgainth, Ggainth, and Bgainth as predetermined values, respectively (S1903). The thresholds Rgainth, Ggainth, and Bgainth are the maximum values of the amount of change in color gain (that is, color information) that is allowed to change (disintegrate) in color balance, and are parameters that depend on the number of pixels in the image, the sense of noise, and the like. . The thresholds Rgainth, Ggainth, and Bgainth may be arbitrarily set according to the usage environment of the video camera, the user's preference of the image, and the like.

As a condition for judging that the change amount of the color gain is larger than the threshold value,
ΔRgain> Rgainth or ΔGgain> Ggainth or ΔBgain> Bgainth (3)
By setting, it becomes possible to react sensitively to changes in color gain.

In addition, as a condition for determining that the change amount of the color gain is larger than the threshold value,
ΔRgain> Rgainth and ΔGgain> Ggainth and ΔBgain> Bgainth (4)
Is set, it can be determined that the amount of change in the color gain is large only when the change in the color gain is certain, as compared with the equation (3).

  As described above, the combination of the thresholds Rgainth, Ggainth, and Bgainth and logic (or, and) prevents erroneous determinations on changes in color gain, and there is a change in color only with respect to the intended change in color gain. It becomes possible to judge.

  Furthermore, when determining the magnitude of the change amount of the color gain, it is preferable to average the color gain before and after extracting the IRCF 2 in terms of time. By using the color gain averaged over time in this way, the amount of change in color gain can be accurately calculated without being greatly affected by noise due to AGC (auto gain control) or the like.

  When the video selection unit 14 determines that the change amount of the color gain before and after the insertion / extraction of the IRCF2 is larger than the threshold value, that is, when an unacceptable collapse occurs in the color balance due to the extraction of the IRCF2, the color reproducibility is secured. If not, monochrome image output is selected (S1904). That is, the monochrome mode is selected as the imaging mode.

  On the other hand, when IRCF2 is extracted, but it is determined that the change amount of the color gain is smaller than the threshold value, that is, the color balance is not largely lost by the extraction of IRCF2, and the color reproducibility is almost ensured. The video selection unit 14 selects a color image output (S1905). That is, the color mode is selected as the imaging mode.

  As described above, in this embodiment, a color image is output after IRCF2 extraction only when the amount of change in color gain due to IRCF2 insertion is small, so the color of the same subject in the color image due to IRCF2 extraction is output. Change can be suppressed. Further, the processing load due to software when outputting a color image after extracting the IRCF 2 is small, and there is no need to add extra hardware.

  Next, the color image output continuation determination processing after IRCF2 extraction will be described using the flowchart of FIG. This process is performed by the WB gain comparison unit 22 and the video selection unit 14.

  The WB gain comparison unit 22 calculates the change amounts of the R gain, the G gain, and the B gain at every predetermined time T after the IRCF 2 is extracted (color image output state) (S2001, S2002). The predetermined time T here is determined according to the stability of each color gain. The color gain may fluctuate greatly until WB becomes stable. For this reason, as the predetermined time T, a time during which the WB is sufficiently stabilized is set.

  Further, when calculating the amount of change in the R gain, G gain, and B gain, each color gain before extraction of IRCF2 is compared with each color gain calculated every predetermined time T after extraction of IRCF2. This is because when the subject is irradiated with illumination light from a light source different from that before IRCF2 extraction, the subject color information changes and correct color reproduction is not performed. This is for determination.

  Then, the video selection unit 14 determines the magnitude of the change amount of the color gain, similarly to S1903 (S2003). When the change amount of the color gain is larger than the threshold value, the output is switched to monochrome image output (S2004), and when the change amount is smaller than the threshold value, the color image output is continued (S905).

  In this way, by monitoring the change in the color gain even after the IRCF2 is extracted, it is possible to cope with the change in the light source after the IRCF2 is extracted and to continue outputting a color image having high color reproducibility. it can.

  Each embodiment described above is only a representative example, and various modifications and changes can be made to each embodiment in carrying out the present invention.

  For example, in each of the above-described embodiments, the imaging apparatus integrally including the imaging optical system has been described. However, the present invention can also be applied to an imaging apparatus in which the imaging optical system can be replaced.

1 is a block diagram showing a configuration of a video camera that is Embodiment 1 of the present invention. The figure which shows the spectral transmittance characteristic of primary color CCD. The figure which shows the spectral transmittance characteristic of complementary color CCD. The figure which shows the spectral characteristics of a fluorescent lamp. The figure which shows the spectral characteristics of a mercury lamp. The figure which shows the spectral characteristic of infrared illumination. The figure which shows the spectral characteristic of an incandescent lamp. 6 is a flowchart illustrating switching control between color image output and black-and-white image output according to IRCF insertion / extraction in the first embodiment. 6 is a flowchart illustrating a continuation determination process for color image output after IRCF extraction in the first embodiment. The block diagram which shows the structure of the video camera which is Example 2 of this invention. 9 is a flowchart showing switching control between color image output and monochrome image output in accordance with IRCF insertion and removal in the second embodiment. 10 is a flowchart showing a continuation determination process for color image output after IRCF extraction in the second embodiment. FIG. 6 is a block diagram illustrating a configuration of a video camera that is Embodiment 3 of the present invention. The figure which shows the existing range of R gain and B gain under each light source. 10 is a flowchart showing switching control between color image output and monochrome image output according to IRCF insertion and removal in the third embodiment. 10 is a flowchart illustrating color image output continuation determination processing after IRCF extraction in the third embodiment. FIG. 6 is a block diagram illustrating a configuration of a video camera that is Embodiment 3 of the present invention. FIG. 9 is a block diagram illustrating a configuration of a video camera that is Embodiment 4 of the present invention. 10 is a flowchart illustrating switching control between color image output and black-and-white image output according to IRCF insertion and removal in the fourth embodiment. 10 is a flowchart illustrating color image output continuation determination processing after IRCF extraction in the fourth embodiment. The figure which shows the white correction possible gain range.

Explanation of symbols

1 Imaging lens 2 Infrared cut filter (IRCF)
DESCRIPTION OF SYMBOLS 3 Image sensor 5 Video signal processing part 6 WB processing part 7 Brightness / color information calculation part 9 WB gain calculation part 11 Color information comparison part 12 IRCF insertion / extraction determination part 14 Video selection part 17 Luminance information comparison part 18 Light source calculation part 22 WB gain Comparison part

Claims (8)

An image sensor that photoelectrically converts a subject image formed by the imaging optical system;
Image generating means for generating an image using an output signal from the image sensor;
Information acquisition means for acquiring information relating to the generated image;
An infrared cut filter that is inserted into and removed from the optical path from the imaging optical system to the imaging device;
Control means for switching the imaging mode between a color mode and a monochrome mode;
The image pickup apparatus according to claim 1, wherein the control unit switches the image pickup mode after the infrared cut filter is extracted in accordance with a change amount of information regarding the image due to insertion and extraction of the infrared cut filter.   The imaging apparatus according to claim 1, wherein the information related to the image is color information of the image.   The imaging apparatus according to claim 1, wherein the information related to the image is luminance information of the image. White balance processing means for performing white balance processing on the generated image;
The imaging apparatus according to claim 1, wherein the information related to the image is a color gain in the white balance process.   The control means sets the imaging mode to the black and white mode when the amount of change in information about the image is greater than a predetermined value, and sets the imaging mode when the amount of change in information about the image is less than the predetermined value. The image pickup apparatus according to claim 1, wherein the image pickup apparatus is set to a color mode. An image sensor that photoelectrically converts a subject image formed by the imaging optical system;
Image generating means for generating an image using an output signal from the image sensor;
Light source information generating means for generating information on a light source that illuminates a subject in the generated image;
An infrared cut filter that is inserted into and removed from the optical path from the imaging optical system to the imaging device;
Control means for switching the imaging mode between a color mode and a monochrome mode;
The said control means switches the said imaging mode after the said infrared cut filter is extracted according to the information regarding the said light source, The imaging device characterized by the above-mentioned.   The control means sets the imaging mode to the monochrome mode when the near-infrared component of the light source is greater than a predetermined value, and sets the imaging mode to the color mode when the near-infrared component is less than the predetermined value. The imaging apparatus according to claim 6, wherein White balance processing means for performing white balance processing on the generated image;
The imaging apparatus according to claim 6, wherein the information on the light source is acquired based on a color gain existing range in the white balance processing.