JP6466809B2 - Imaging apparatus and imaging method - Google Patents

Description

  The present invention relates to an imaging apparatus and an imaging method.

  As a background art of this technical field, there is Patent Document 1. Japanese Patent Laid-Open No. 2004-133620 discloses that “with regard to an imaging device and a television door phone device, even when a color photographed image is obtained so that the light illuminated for the subject does not become dazzling in a dark place, the visibility of the subject is immediately The imaging device includes an imaging infrared light source unit that illuminates a subject, a visible light source unit that illuminates the subject, an imaging unit that shoots the subject, and a shooting red light source. A light emission control unit that controls light emission of the external light source unit and the visible light source unit for photographing, and the light emission control unit starts emitting infrared light when the imaging unit starts photographing, and after the light emission starts, The amount of visible light increases more gradually than infrared light as time passes. "

JP 2010-212941 A

  According to Patent Document 1, since the photographing infrared light source unit and the photographing visible light source unit that illuminate the subject are used, the color reproducibility of the image captured by the imaging device is improved. However, even if the imaging device of Patent Document 1 is used, there are cases where all of the desired subject in the image is not irradiated with infrared light and visible light from the light source, and a part of the image to be picked up is another part. Color reproducibility is worse than. Like a surveillance camera, depending on the application of an imaging device, it is desirable that color reproducibility is good over all desired subjects in an image.

The disclosed imaging device has sensitivity to invisible light and visible light, and captures an image of a subject to generate an image signal. The image signal is generated in an image area obtained by dividing the image into a plurality of areas. A non-visible light amount detecting unit that detects the amount of visible light, a light source unit that irradiates the subject with invisible light, and a subject in an image area in which the amount of invisible light detected by the invisible light amount detecting unit is less than a predetermined light amount And a control unit that controls the light source unit so as to irradiate with invisible light.

  According to the disclosed imaging device, color reproducibility of at least a desired region in an image is improved.

1 is a configuration example of an imaging apparatus according to a first embodiment. It is a block diagram which shows an example of an imaging part. It is an example of the color filter arrangement | sequence of an image sensor part. It is an example of the spectral characteristic of a color filter. It is an example of light quantity control of a light source part. It is an example of a photographing scene. It is the image image | photographed with the imaging device. This is a shooting scene in which invisible light is irradiated to the subject person A by the light source unit A. It is an image taken by illuminating the subject person A with invisible light by the light source unit A. 4 is a configuration example of an imaging apparatus according to a second embodiment. It is an example of the control method by the control part of a light source part and an invisible light adjustment part. It is an example of the image which image | photographed the to-be-photographed object with the imaging device. It is an example which shows the relationship between the visible light amount and infrared light amount of a to-be-photographed object. It is a control curve of the gain for every pixel set to a non-visible light adjustment part. 6 is a configuration example of an imaging apparatus according to Embodiment 4. It is an example of control of a light source part and a filter. 10 is a configuration example of an imaging apparatus according to a fifth embodiment. It is an example of a photographing scene. It is the image image | photographed with the imaging device. This is a shooting scene in which invisible light is irradiated to the subject person A by the light source unit A. It is an image taken by illuminating the subject person A with invisible light by the light source unit A. 10 is a configuration example of Example 6.

  Hereinafter, examples will be described with reference to the drawings. In addition, the same code | symbol is used for the same element and the overlapping description is abbreviate | omitted.

  Infrared light will be described as an example of invisible light (light in the first wavelength band), but in each embodiment, the color filter of the image sensor unit is changed to visible light (light in the second wavelength band) and infrared light. Instead, it may be configured to have sensitivity to ultraviolet light as visible light and invisible light (light in the first wavelength band). When the color filter has sensitivity to infrared light as invisible light, the object in which the image is dark is irradiated with light obtained by adding visible light to infrared light as a predetermined amount of invisible light. As a result, an image with good color reproducibility can be obtained as the entire image. When the color filter is sensitive to ultraviolet light as invisible light, a colored ultraviolet light image can be obtained by synthesizing an ultraviolet light region image and a visible light color image.

  Hereinafter, infrared light including a near-infrared wavelength band as invisible light (light in the first wavelength band), red signal, green signal, blue signal as visible light (light in the second wavelength band), A white signal that is visible light including blue to red will be described as an example.

  The imaging apparatus according to the present exemplary embodiment is configured to irradiate a subject on the basis of the amount of visible light and the ratio of the amount of invisible light to visible light for each image region obtained by dividing a captured image into a plurality of regions. Adjust the amount of visible light. An image area obtained by dividing an image into a plurality of areas will be described later with an example.

  FIG. 1 is a configuration example of the imaging apparatus of the present embodiment. The imaging device includes an imaging unit 101, a signal processing unit 103, a visible light signal generation unit 104, a non-visible light signal generation unit 105, a white adjustment unit 106, an image synthesis unit 107, a luminance signal generation unit 108, a color difference signal generation unit 109, A control unit 110, a visible light amount detection unit 112, an invisible light amount detection unit 113, a signal level detection unit 114, and a light source unit 116 are included. Hereinafter, each part will be described.

  The imaging unit 101 has sensitivity to invisible light and visible light, images a subject, generates an image signal, and outputs the image signal. The imaging unit 101 includes a lens group including an imaging lens, a zoom lens, and a focus lens, an aperture, a shutter, an image sensor such as a CCD or a CMOS, an amplifier, an AD converter, and the like, and visible light and invisible imaged by the lens group. The pixel signal corresponding to the predetermined wavelength component is output by spectrally photoelectrically converting the light to correspond to the predetermined wavelength component by the color filter of the image sensor. The predetermined wavelength component is a component obtained by adding invisible light and visible light, and includes, for example, red light + infrared light, green light + infrared light, blue light + infrared light, and blue to red light. There are four types of visible light + infrared light.

  The signal processing unit 103 performs, for each predetermined wavelength component, a demosaic process from the pixel signal output from the imaging unit 101 (each pixel signal is RAW data corresponding to one of the predetermined wavelength components). A pixel signal (one pixel signal is pixel data corresponding to each of the four wavelength components) is output over the entire area of the image. Hereinafter, in the operation of each unit, when there is no need for description, the operation corresponding to the pixel signal for each predetermined wavelength component will be described, and the operation is assumed to cover the entire area of the image.

  The visible light signal generation unit 104 performs a matrix calculation process on the pixel signal for each predetermined wavelength component output from the signal processing unit 103, so that each visible light component is obtained from the pixel signal of each predetermined wavelength component. The visible light signal, which is the signal of, is extracted and output. The visible light signal is obtained by subtracting the infrared light component from a predetermined wavelength component, for example, red light, green light, blue light, and visible light including blue to red, red signal, green signal, blue signal, And a white signal which is visible light including blue to red.

  The invisible light signal generation unit 105 performs a matrix calculation process on the pixel signal for each predetermined wavelength component output from the signal processing unit 103, thereby generating an invisible light signal that is a signal for the invisible light component. Extract and output. The invisible light signal is an infrared light signal.

  In order to make the visible light signal generation unit 104 and the non-visible light signal generation unit 105 easy to understand, the illustration and description are made separately. However, for example, the matrix calculation processing is performed as white signal = red signal + green signal + blue signal. Equivalent to solving simultaneous equations with the red, green, blue and infrared light signals as variables.

  The white adjustment unit 106 performs white balance correction by multiplying each visible light signal output from the visible light signal generation unit 104 by each gain (gain corresponding to each visible light signal) corresponding to the amount of white balance deviation. And output each visible light signal after white balance correction. That is, the white balance correction is to change the ratio of the red signal, the green signal, and the blue signal, which are signals related to color, between the visible light signals. Note that the white adjustment unit 106 also outputs a white signal obtained by adding the red signal, the green signal, and the blue signal that have undergone white balance correction.

  The image composition unit 107 outputs a composite signal for each predetermined wavelength component, which combines the invisible light signal output from the invisible light signal generation unit 105 and the visible light signal output from the white adjustment unit 106. To do. The composite signal of a predetermined wavelength component is a red composite signal corresponding to a red signal + infrared light signal, a green composite signal corresponding to a green signal + infrared light signal, and a blue composite signal corresponding to a blue signal + infrared light signal. , And a white composite signal corresponding to a white signal + infrared light signal.

  The luminance signal generation unit 108 outputs the luminance signal generated by the synthesis process of adding the respective synthesized signals output from the image synthesis unit 107 at a predetermined ratio. The color difference signal generation unit 109 selects a red composite signal, a green composite signal, and a blue composite signal, which are color-related composite signals, from the respective composite signals output from the image composition unit 107, and the color difference signal from the selected composite signal. A composition process using a conversion formula for converting to color is executed, and a color difference signal is output.

  The visible light amount detection unit 112 detects the magnitude of the visible light signal output from the visible light signal generation unit 104 for each image region (divides the captured image into a plurality of regions (image regions) and relates to each of the divided images). To output information on the magnitude of the visible light signal for each image area. The detection processing of the magnitude of the visible light signal is performed by adding the signal level of the visible light signal of each pixel for each image area and converting the addition result as it is or as the magnitude of the visible light signal for each image area. Output. As described later, since the ratio of the size of the visible light signal and the size of the invisible light signal is used as the light amount ratio for each image area, the same processing (conversion) can be performed. If so, the same conversion is applied to any image region.). In addition, using any one of a red signal + green signal + blue signal, a white signal, and a white signal + red signal + green signal + blue signal as a visible light signal, the magnitude of the visible light signal for each image area is the same processing. What is necessary is just to detect.

  The invisible light amount detection unit 113 performs processing for detecting the size of the invisible light signal output from the invisible light signal generation unit 105 for each image region similar to the visible light amount detection unit 112, and Output size information for each image area. The detection processing of the magnitude of the invisible light signal is performed by adding the signal level of the invisible light signal of each pixel for each image area similar to the visible light amount detection unit 112 that divides the image into a plurality of parts. As the magnitude of the invisible light signal, the addition result is output as it is or after being converted. The reason why it can be used as it is or converted is the same as in the case of the visible light amount detection unit 112.

  The signal level detection unit 114 integrates the signal level of each pixel signal output from the signal processing unit 103 for each image region, or detects the peak value for each image region, etc. Such information is output for each image area.

  The light source unit 116 is an invisible infrared light source that can adjust the amount of light that the imaging apparatus irradiates a subject to be photographed (hereinafter, “subject”) for each image region, and the amount of light for each image region set by the control unit 110 described later. By applying a voltage corresponding to the control parameter to a light emitting unit (light source) corresponding to each image region, the subject is irradiated with a controlled amount of invisible light. The irradiation direction and irradiation region of the invisible light by each light emitting unit of the light source unit 116 are easily controlled by using a recent projection mapping (also called video mapping or mapping projection) technique.

  The control unit 110 performs exposure control of the imaging unit 101 based on the brightness target value set in advance in the control unit 110 and information on the brightness output from the signal level detection unit 114. At this time, since the signal level detection unit 114 outputs information related to brightness for each image region, the signal level detection unit 114 converts the information into information related to brightness for the entire image. The exposure control of the image capturing unit 101 uses exposure control parameters set in the image capturing unit 101 by the control unit 110. The exposure control parameters are the lens aperture, shutter speed, and amplifier gain of the imaging unit 101, and are parameters that control the amount of light input to the imaging unit 101 and the signal level of the pixel signal output from the imaging unit 101. Here, by previously setting a relationship between the exposure control parameter, the output information of the signal level detection unit 114 (information related to brightness), and the amount of light of the subject in the control unit 110, the control unit 110 can be The light quantity of the entire image can be calculated from the exposure control parameter set to 101 and the output information of the signal level detection unit 114.

  The amount of visible light applied to the subject is referred to as the absolute amount of visible light, and the amount of invisible light applied to the subject is referred to as the absolute amount of invisible light. Information on the appropriate infrared light amount (appropriate infrared light amount) with respect to the absolute amount of visible light is set in the control unit 110 in advance. Infrared light is not necessary when the absolute amount of visible light is sufficient. When the absolute amount of visible light is small, it is effective to irradiate the subject with infrared light in order to generate an image with less noise in the imaging device. However, if the amount of infrared light is increased, the color of the subject is reproduced. It becomes difficult. In the imaging unit 101, an infrared light amount that provides an image with good visibility experimentally or empirically with a balance between the noise amount of the image to be generated and the color reproducibility, corresponding to the absolute amount of visible light, An appropriate infrared light amount is set in advance.

  The appropriate infrared light amount may be set in advance in the control unit 110, or a memory is provided in the imaging apparatus, and the control unit 110 reads the set value of the appropriate infrared light amount set in the memory, so that the appropriate red light amount is set. An external light quantity may be obtained. When the relationship between the absolute amount of visible light and the appropriate infrared light amount is expressed by a relational expression, the control unit 110 calculates the appropriate infrared light quantity from the relational expression. In addition, when the absolute amount of visible light is divided into several ranges (stages) and an appropriate amount of infrared light corresponding to the absolute amount of visible light in each divided range is set, the control unit 110 displays visible light. Select an appropriate amount of infrared light corresponding to the absolute amount of.

  Based on the size of the visible light signal for each image region output from the visible light amount detection unit 112 and the size of the invisible light signal for each image region output from the non-visible light amount detection unit 113, the control unit 110 displays visible light. And the ratio of the amount of infrared light (visible light ratio) is calculated for each image region. The control unit 110 calculates the absolute amount of light for each image region of the subject (absolute amount of visible light) obtained from the information regarding the brightness for each image region output from the visible light ratio and the signal level detection unit 114 output for each image region. And the absolute amount of infrared light), the absolute amount of visible light and the absolute amount of infrared light are calculated for each image region. The control unit 110 calculates or selects a target infrared light amount (described later) for each image region from an appropriate infrared light amount preset in the control unit 110 corresponding to the calculated absolute amount of visible light for each image region. The control unit 110 controls the light amount control parameter (the appropriate infrared light amount and the appropriate infrared light amount so that the calculated absolute amount of infrared light matches the appropriate infrared light amount corresponding to the calculated absolute amount of visible light in each of the image regions. The light source unit 116 is controlled by setting a target infrared light amount according to a difference from the absolute amount of infrared light. An image in which the absolute amount of infrared light is larger than the appropriate infrared light amount 501 so that the amount of infrared light emitted from the light source unit 116 is larger in an image region where the absolute amount of infrared light is smaller than the appropriate infrared light amount 501 The control unit 110 controls the light source unit 116 with the light amount control parameter for each image region so that the amount of infrared light emitted from the light source unit 116 is reduced in the region.

  An example of shooting using the described imaging apparatus will be described later. As described above, in the imaging apparatus according to the present exemplary embodiment, the light amount control parameter for each image region set in the imaging unit 101 and the image regions of the signal level detection unit 114, the visible light amount detection unit 112, and the invisible light amount detection unit 113. From each output, the light source 116 can irradiate the subject with an appropriate amount of infrared light for each image region, and even if the visible light amount for each image region increases or decreases, the color reproducibility is high and the visibility is high. A good image can be generated.

  Although the imaging apparatus of the present embodiment has been described above, the circuit configuration is not limited to the present embodiment. For example, the signal level detection unit 114 relates to the brightness of a captured image or the brightness of contrast or the like. Instead of acquiring information for each image region, the control unit 110 uses information obtained by combining the output information of the visible light amount detection unit 112 and the output information of the invisible light amount detection unit 113 as information relating to the brightness for each image region. It may be used for exposure control and calculation of the absolute amount of visible light and invisible light for each image area. Thereby, the circuit scale can be reduced. Further, the white balance detection unit 111 may detect each color signal by the visible light amount detection unit 112 and acquire the amount of white balance deviation from the difference between detection results of each color signal. Thereby, the circuit scale can be reduced.

  Hereinafter, an example of each unit constituting the imaging unit 101 will be described.

  FIG. 2 is a configuration diagram illustrating an example of the imaging unit 101. The imaging unit 101 includes an optical unit 200 including a lens unit 201 and a diaphragm 202, an image sensor unit 203, an amplification unit 204 that amplifies a signal level, and an AD conversion unit 205 that converts an analog signal into a digital signal. The lens unit 201 is a lens group including an imaging lens, a zoom lens, and a focus lens, and adjusts the amount of light output by the optical unit 200 by opening and closing the diaphragm 202 in accordance with the exposure control parameters from the control unit 110.

  Incident light entering the imaging unit 101 enters the image sensor unit 203 through the optical unit 200. The image sensor unit 203 photoelectrically converts incident light through a color filter, which will be described later, and outputs a signal corresponding to the amount of incident light to the image sensor unit 203. The amplifying unit 204 amplifies the signal output from the image sensor unit 203. The AD conversion unit 205 converts the analog signal amplified by the amplification unit 204 into a digital signal.

  The level (signal level) of the signal output from the imaging unit 101 is controlled by adjusting the aperture 202 of the optical unit 200 and the exposure time of the image sensor unit 203 and the gain of the amplification unit 204. If the subjects are the same in brightness and the diaphragm 202 of the optical unit 200, the exposure time of the image sensor unit 203, and the gain of the amplification unit 204 are set to the same value, the signal levels output from the imaging unit 101 are equal. Therefore, it is possible to obtain the amount of light of the entire subject image from the exposure time of the aperture 202 of the optical unit 200 and the image sensor unit 203, the gain of the amplification unit 204 and the signal level.

  In a moving image for use in a surveillance camera that captures a dark subject such as at night, the exposure time of the image sensor unit 203 is shorter than the time of one frame of the moving image (the reciprocal of the frame rate), so the aperture 202 of the optical unit 200 is opened and amplified. In many cases, the gain of the unit 204 is increased. Furthermore, the visibility of the output image of the imaging apparatus is improved even in a dark subject such as at night using infrared light which is invisible light.

  FIG. 3 is an example of a color filter array of the image sensor unit 203 using infrared light. RIR301 is a color filter sensitive to red light and infrared light, GIR302 is a color filter sensitive to green light and infrared light, BIR303 is a color filter BIR sensitive to blue light and infrared light, and WIR304 is blue. This color filter has sensitivity to visible light and infrared light up to red light. By giving each color filter sensitivity to infrared light, the signal level output from the imaging unit 101 can be increased even in a dark subject such as at night.

  FIG. 4 is an example of spectral characteristics of the color filter described in FIG. The RIR pixel in which the color filter RIR301 is arranged has sensitivity to each of the red wavelength of visible light and the wavelength of infrared light. The GIR pixel in which the color filter GIR302 is disposed has sensitivity to each of the green wavelength of visible light and the wavelength of infrared light. The BIR pixel in which the color filter BIR303 is arranged has sensitivity in the vicinity of visible light blue and in the wavelength of infrared light. The WIR pixel in which the color filter WIR304 is disposed has sensitivity to each of visible light red, green and blue wavelengths and infrared light wavelength.

  The signal processing unit 103 performs an AD conversion on the output pixel signals of the image sensor unit 203 arranged with different color filters (for RAW data corresponding to any one of the predetermined wavelength components). Then, demosaic processing is executed by interpolation processing or the like. Since the GIR pixel signal, the BIR pixel signal, and the WIR pixel signal do not exist at the pixel position of a certain RIR pixel signal, the signal processing unit 103 generates an interpolation signal of each color by interpolation processing from the surrounding pixel signals of each color, A pixel signal interpolated by the interpolation signal is output.

  FIG. 5 is an example of light amount control of the light source unit 116. The horizontal axis represents the visible light amount (the amount of visible light) applied to the subject, and the vertical axis represents the infrared light amount (the amount of invisible light) applied to the subject. The appropriate infrared light amount 501 is an infrared light amount that provides good color reproducibility of the output image with respect to the visible light amount and provides the best visibility. The ambient infrared light amount 502 is the infrared light amount irradiated to the subject when no light is emitted from the light source unit 116. The target light amount 503 is a target light amount of infrared light irradiated from the light source unit 116 to the subject, and is a light amount obtained by subtracting the environmental infrared light amount 502 from the appropriate infrared light amount 501.

  As described above, the control unit 110 calculates the absolute amount of the light amount for each image region of the subject obtained from the information regarding the visible light ratio calculated for each image region and the brightness for each image region output from the signal level detection unit 114 ( The absolute amount of visible light and the absolute amount of infrared light are calculated for each image region using the sum of the absolute amount of visible light and the absolute amount of infrared light. The control unit 110 controls the light source unit 116 so that the calculated absolute amount of infrared light matches the appropriate infrared light amount 501 corresponding to the calculated absolute amount of visible light in each image region. An image in which the absolute amount of infrared light is larger than the appropriate infrared light amount 501 so that the amount of infrared light emitted from the light source unit 116 is larger in an image region where the absolute amount of infrared light is smaller than the appropriate infrared light amount 501 The control unit 110 controls the light source unit 116 with the light amount control parameter for each image region so that the amount of infrared light emitted from the light source unit 116 is reduced in the region.

  In FIG. 5, the relationship between the visible light amount and the appropriate infrared light amount 501 is linear, but the relationship between the visible light amount and the appropriate infrared light amount 501 is not limited to linear, and the imaging element (image sensor unit 203). Depending on the characteristics, it may become non-linear.

  As described above, since the pixel signal output from the imaging unit 101 (each pixel signal is RAW data corresponding to one of the predetermined wavelength components) is a digital signal, the processing after the signal processing unit 103 is performed. Suitable for execution by a processor. Therefore, the imaging apparatus can include the imaging unit 101, the light source unit 116, the processor, and the memory. The processor executes the processes of the signal processing unit 103 to the signal level detection unit 114 in FIG. 1 as described above, and stores the exposure control parameters, the visible light amount shown in FIG. 5, the appropriate infrared light amount 501, the environment in the memory. The relationship between the infrared light quantity 502 and the target light quantity 503, the light quantity control parameter for each image area, and the like are stored. However, the processor outputs the light amount control parameter for each image region stored in the memory to the light source unit 116, and the light source unit 116 emits the voltage corresponding to the output light amount control parameter for each image region corresponding to each image region. Applied to the part.

  As described above, the imaging apparatus appropriately controls the amount of infrared light emitted from the light source unit 116 for each image region with respect to the amount of visible light applied to the subject, and as a whole image. An image with good color reproducibility and good visibility can be obtained. In order to deepen the understanding of the described imaging apparatus, an example of imaging using the imaging apparatus will be described below.

  FIG. 6 is an example of a shooting scene that is shot using the imaging device. The light source unit 116 includes a light source unit A that emits invisible light toward the left side in the photographing direction and a light source unit B that emits invisible light toward the right side in the photographing direction. The subject in the shooting direction is irradiated with ambient light. Since the person A is in the shadow, the ambient light is not applied, and only the person B is irradiated with the ambient light. In addition, if the control part 110 controls the irradiation direction of the invisible light by the light source which the light source part 116 has, the number of light sources can be decreased.

  FIG. 7 is an image photographed by the imaging device under the photographing conditions of FIG. A person A and a person B are photographed as subjects, and the image is divided into an image area A including the person A and an image area B including the person B. The image region A is an image region irradiated with invisible light from the light source unit A, and the image region B is an image region irradiated with invisible light from the light source unit B.

  FIG. 8 shows a shooting scene in which invisible light is irradiated to the subject person A by the light source unit A. If the amount of invisible light in the image area A is smaller than the target amount of light according to the imaging conditions in FIG. 6, the control unit 110 increases the amount of invisible light emitted by the light source A as shown in FIG.

  FIG. 9 is an image captured by the light source unit A irradiating the subject person A with invisible light. In this way, an image with an appropriate amount of invisible light applied to the subject in the image area A can be acquired.

  As can be seen from the above examples, an image region obtained by dividing an image into a plurality of regions is an independent invisible light irradiation region, such as the light source unit A and the light source unit B, to the subject by the light source unit 116. Correspond. However, the irradiation region and the image region do not have to coincide with each other, as exemplified by the circular irradiation region and the rectangular image region.

  In addition, the image area in the operation of each unit of the imaging apparatus is divided into image areas so as to correspond to the irradiation area by the light source unit 116 set in advance according to the brightness in the imaging direction, and the divided image areas are pixel positions. Is stored in the control unit 110 in advance, and when a certain pixel signal is processed, it is grasped according to which image region the pixel position of the pixel signal is. If the image area is rectangular, the coordinates of the four pixel positions are stored in the control unit 110 in advance.

  As described above, by controlling the amount of invisible light to be irradiated for each image region corresponding to each subject in the image, the imaging apparatus can be used even if the amount of visible light increases or decreases for each image region in the image. An image with high color reproducibility and good visibility is generated.

  FIG. 10 is a configuration example of the imaging apparatus of the present embodiment. The imaging apparatus according to the present exemplary embodiment further includes a non-visible light adjustment unit 701 in addition to the imaging apparatus according to the first exemplary embodiment, and the operations of the non-visible light signal generation unit 105 and the image synthesis unit 107 are different from the first exemplary embodiment. The non-visible light adjustment unit 701 is configured to multiply the non-visible light signal for each image region output from the non-visible light signal generation unit 105 by the gain for each image region specified by the control unit 110, thereby making the non-visible light for each image region. The optical signal is output to the image composition unit 107.

  The image combining unit 107 outputs a combined signal for each predetermined wavelength component, which combines the invisible light signal for each image region and the visible light signal output from the white adjustment unit 106. In the first embodiment, the image composition unit 107 uses the invisible light signal output from the invisible light signal generation unit 105 for image composition. In this embodiment, the image region output from the invisible light adjustment unit 701 is used. A combined signal for each predetermined wavelength component in which the signal level of the invisible light signal is controlled for each image region is output using each invisible light signal.

  As described above, the signal level of the invisible light signal is controlled for each image region, so that the absolute amount of the invisible light for each image region is controlled equivalently and is visible for each image region in the image. Even if the amount of light increases or decreases, the imaging device generates an image with high color reproducibility and good visibility.

  FIG. 11 is an example of a method for controlling the light source unit 116 and the invisible light adjusting unit 701 by the control unit 110. In FIG. 11 (1), the same components as those in FIG. The threshold value 1 is that the appropriate infrared light amount 501 that is an appropriate infrared light amount for generating an image with good visibility with respect to the visible light amount irradiated on the subject and the infrared light source of the light source unit 116 are not turned on. Is the visible light amount when the ambient infrared light amount 502 irradiated to the subject is equal, and if the visible light amount irradiated to the subject is greater than the threshold value 1, irradiation of infrared light from the light source unit 116 is unnecessary. In addition, it is shown that the color reproducibility is better when the ambient infrared light amount 502 is reduced in accordance with the increase in the visible light amount. The threshold value 2 is sufficient as a visible light amount for generating an image with good color reproducibility and good visibility, and indicates a visible light amount that does not require infrared light. In order to improve the color reproducibility and the visibility of the image output from the imaging apparatus, the signal level of the invisible light is lowered from the threshold value 1 and the signal level is controlled to be minimized at the threshold value 2.

  (2) in FIG. 11 illustrates an example of the gain 801 set by the control unit 110 in the invisible light adjustment unit 701. The control unit 110 increases the gain of the invisible light adjustment unit 701 by 1 in an image region with a small amount of visible light, and the image composition unit 107 outputs the invisible light in the image region output from the invisible light signal generation unit 105. Use signal level. In an image area where the gain of the non-visible light adjusting unit 701 is 1, by increasing or decreasing the amount of infrared light irradiated to the subject in the image area from the light source unit 116, the amount of invisible light irradiated to the subject in the image area is increased or decreased. The non-visible light signal level of the image region output from the non-visible light signal generation unit 105 is controlled.

  The control unit 110 reduces the gain output to the non-visible light adjustment unit 701 in accordance with the increase in the visible light amount in the image region where the visible light amount exceeds the threshold value 1, and sets the threshold value 2 to the minimum value (for example, 0.01). . In the control unit 110, the invisible light adjustment unit 701 controls the invisible light signal level for each image region, so that the visible light amount from the light source unit 116 to the visible light amount that does not require the infrared light amount is imaged. The ratio of the invisible light component for each image region to the visible light component input to the combining unit 107 can be controlled, and the color reproducibility of the image output from the imaging apparatus can be improved.

  By providing the non-visible light adjusting unit 701 with the imaging device described above, an image with good visibility from low illuminance that requires an infrared light source to high illuminance subject that does not require infrared light is provided for each image region. Can be obtained.

  Note that threshold values 1 and 2 shown in (2) of FIG. 11 are not fixed values, but differ depending on the shooting environment. Further, the relationship between the visible light amount and the infrared light gain is an example, and may vary depending on the characteristics of the imaging unit 101 and the imaging environment, and is not limited to the illustrated relationship.

  As described in the first exemplary embodiment, the imaging apparatus according to the present exemplary embodiment may execute processing of each unit of the signal processing unit 103 to the signal level detection unit 114 and the invisible light adjustment unit 701 in FIG. .

  As described above, in the image area where the invisible light amount irradiated to the subject is larger than the necessary invisible light amount, the invisible light adjustment unit 701 reduces the signal level of the infrared light amount to reduce the color reproduction. A good quality image can be obtained.

  Although the example in which the color difference signal generation unit 109 that generates the color difference signal is arranged after the image composition unit 107 has been described as in the first embodiment, the color difference signal generation unit 109 is invisible when the color difference signal is generated. Since the optical signal component is not subtracted, the same color difference signal can be obtained even if the output signal of the white adjustment unit 106 is used.

  The present embodiment is an example of light amount control when there is a difference in visible light amount and invisible light amount depending on an image area. In this embodiment, invisible light will be described as infrared light.

  Further, in order to make the explanation easy to understand, this embodiment will be described as an image instead of the image area of the first and second embodiments. However, by replacing the image with the image area, the present embodiment can divide the image into a plurality of parts. It will be understood that the same applies to the image area.

  FIG. 12 is an example of an image obtained by imaging a subject with an imaging device. An image 1000 shows an example in which a subject 1001 and a subject 1002 are captured, the subject 1001 is close to the imaging device, and the subject 1002 is far from the imaging device. When there is a visible light source near the subject 1001 and no visible light source near the subject 1002, the visible light amount of the subject 1001 is larger than the visible light amount of the subject 1002. Thus, since the visible light amount varies depending on the subject in the image 1000, the appropriate infrared light amount also varies.

  In addition, the amount of infrared light irradiated to the subject by the light source unit 116 is determined by the amount of infrared light irradiated by the light source unit 116 and the distance between the light source unit 116 and the subject. Even when infrared light having the same light amount is irradiated to the imaging range, the infrared light amount received by the subject 1001 and the infrared light amount received by the subject 1002 are different. Therefore, the gain of the invisible light adjustment unit 701 is controlled for each pixel in order to output each subject in the screen imaged by the imaging device as an image with good visibility. Hereinafter, the control method will be described.

  Compared with the imaging device of the second embodiment (FIG. 10), the imaging device of the present embodiment has a magnitude (signal level) of the visible light signal output from the visible light amount detection unit 112 and an invisible light amount detection unit 113. The control unit 110 that operates by inputting the magnitude (signal level) of the invisible light signal to be output is different from the second embodiment. Therefore, a configuration example of the imaging apparatus according to the present embodiment will be described with reference to FIG.

  The control unit 110 determines the amount of infrared light (the target light amount 503 described in the first embodiment) that the light source unit 116 irradiates toward the imaging range. The target light amount 503 is determined from the appropriate infrared light amount 501 as described in the first embodiment. The appropriate infrared light amount 501 is the absolute amount of the light amount of the image obtained from the information relating to the brightness output from the signal level detection unit 114, the magnitude of the visible light signal output from the visible light amount detection unit 112, and invisible. It is determined based on the visible light ratio obtained from the magnitude of the invisible light signal output by the light amount detector 113.

  In the first embodiment, the target light amount 503 is determined for each image region. In this embodiment, the target light amount 503 is divided into a plurality of small regions in order to correspond to regions where the visible light amount is locally small. The target light quantity 503 is determined every time. If the image area of the first embodiment is sufficient to discriminate an area where the amount of visible light is locally small, the small area may be set as the image area. However, the small area is generally set smaller than the image area. The small area is an area that is set not for controlling the light source unit 116 but for processing of each part in the imaging apparatus, and can be arbitrarily reduced.

  Therefore, in the present embodiment, the outputs of the visible light amount detection unit 112, the invisible light amount detection unit 113, and the signal level detection unit 114 are input to the control unit 110 for each small area. The control unit 110 obtains the absolute amount of visible light for each small region, determines the appropriate infrared light amount 501 and the target light amount 503 using the absolute amount of visible light in the small region with the smallest absolute amount of visible light, The light source unit 116 is controlled.

  The control unit 110 obtains an appropriate infrared light amount 501 for each pixel from the visible light signal for each pixel from the visible light signal generation unit 104 and the invisible light signal for each pixel from the non-visible light signal generation unit 105. Further, the gain for each pixel of the invisible light adjusting unit 701 is determined.

  An example of a method for determining the gain for each pixel of the invisible light adjusting unit 701 by the control unit 110 will be described. Assume that the subject 1002 in FIG. 12 is in a small region with the least amount of visible light. FIG. 13 is an example showing the relationship between the visible light amount and the infrared light amount of the subject 1002. The horizontal axis of FIG. 13 represents the visible light amount irradiated to the subject (the signal level of the visible light signal for each pixel from the visible light signal generation unit 104), and the vertical axis represents the infrared light amount (non-lighting) applied to the subject. The signal level of the invisible light signal for each pixel from the visible light signal generation unit 105), and the appropriate infrared light amount 501 with respect to the visible light amount. In FIG. 13, the visible light amount 1101 of the subject 1001, the visible light amount 1102 of the subject 1002, the infrared light amount 1111 of the subject 1001, the infrared light amount 1112 of the subject 1002, the appropriate infrared light amount 1113 with respect to the visible light amount of the subject 1001, Appropriate infrared light 1114 is shown for the amount of visible light of the subject 1002. The infrared light amount here is a light amount obtained by adding the light amounts of the infrared light emitted from the light source unit 116 and the ambient infrared light.

  In FIG. 13, the infrared light quantity 1112 with respect to the visible light quantity 1102 of the subject 1002 is the appropriate infrared light quantity 1114, but in the subject 1001, the infrared light quantity 1111 irradiated to the subject 1001 is larger than the appropriate infrared light quantity 1113. ing. Since the light source unit 116 is controlled on the assumption that the subject 1002 is in a small area with the smallest visible light amount, the subject 1002 is irradiated with the appropriate infrared light amount 1114. Therefore, in a small region other than the small region including the subject 1002, the irradiated infrared light amount is larger than the appropriate infrared light amount 1113.

  FIG. 14 is a gain control curve for each pixel set in the invisible light adjusting unit 701. In FIG. 14, the horizontal axis represents the difference between the irradiated infrared light amount and the appropriate infrared light amount 1113, the vertical axis represents the gain for each pixel set in the invisible light adjustment unit 701, and the value on the horizontal axis The control curve 1301 of the value of a vertical axis | shaft is represented. Here, the gain when the invisible light signal level is output without adjusting the invisible light signal level by the invisible light adjusting unit 701 is set to 1.

  The control unit 110 sets the gain to 1 when there is no difference between the infrared light amount irradiated to the subject and the appropriate infrared light amount, and the difference between the infrared light amount irradiated to the subject and the appropriate infrared light amount becomes large. Accordingly, the gain is curvilinearly lowered along the control curve 1301. In FIG. 14, in consideration of noise amplification, control is performed so that the gain becomes 1 when the appropriate infrared light amount is larger than the infrared light amount irradiated to the subject. The gain may be increased.

  The non-visible light adjustment unit 701 multiplies the gain for each pixel determined by the control unit 110 by the signal level of the non-visible light signal for each pixel from the non-visible light signal generation unit 105, and the non-visible light as a multiplication result. The signal level of the visible light signal is output to the image composition unit 107.

  As described above, the control unit 110 controls the light amount of the light source unit 116 so as to supplement the light amount of a small region where the amount of visible light is the smallest, and the appropriate infrared light amount corresponding to the signal level for each pixel of visible light. And the gain for each pixel according to the control curve based on the difference between the signal level for each pixel of invisible light. Further, the invisible light adjusting unit 701 controls the signal level of each pixel of invisible light output to the image composition unit 107 using the gain for each pixel. With the control as described above, even when there is a variation in the visible light amount within the screen of the image captured by the imaging device, the level of the invisible light signal input to the image composition unit 107 can be appropriately controlled, and color reproducibility is good. An image can be generated.

  In this embodiment, the control in units of pixels has been described. However, when there is a decrease in visibility such as a pseudo contour due to a sudden change in gain between pixels, for example, a low-pass filter is provided, The gain change may be smoothed.

  FIG. 15 is a configuration example of the imaging apparatus of the present embodiment. The imaging apparatus according to the present exemplary embodiment includes a filter 1401 on the front surface (irradiation direction) of the light source unit 116 having the configuration of the first exemplary embodiment (FIG. 1). The filter 1401 is configured by a liquid crystal panel or the like that can adjust the transmittance of light emitted from the light source unit 116 for each divided region obtained by dividing the image into a plurality of regions, and corresponds to each divided region by a change in the light transmittance. Controls the amount of light applied to the subject.

  The light source unit 116 is an infrared light source capable of at least on / off control as in the first embodiment. However, in this embodiment, since the transmittance of the filter 1401 is adjusted for each divided region, it is not necessary to provide a light source corresponding to the divided region as illustrated in the first embodiment.

  FIG. 16 is a control example of the light source unit 116 and the filter 1401. 16 (1) is the same as FIG.

  The horizontal axis of (2) in FIG. 16 represents the visible light amount irradiated to the subject corresponding to the divided area, the vertical axis represents the transmittance of the filter 1401, and the control is performed according to the visible light amount irradiated to the subject. The transmittance 1501 controlled by the unit 110 is represented. When the amount of visible light is small, the transmittance of the filter 1401 is increased and most of the infrared light from the light source unit 116 is transmitted. As the visible light amount increases and the required infrared light decreases, the transmittance of the filter 1401 is lowered, and the infrared light component for each divided region input to the image composition unit 107 is reduced.

  With the above control, the amount of infrared light applied to the subject from the light source can be adjusted for each divided region, and an image with good color reproducibility and good visibility can be obtained as in the first embodiment. Further, by providing the invisible light adjusting unit 701 as in the second and third embodiments, the same effect can be obtained.

  In the first embodiment, a light source having a physical size restriction corresponding to the divided area is provided. However, since the filter used in this embodiment is small and has a small physical restriction, the number of divided areas can be increased, and the details are fine. By the control, an image with good color reproducibility and good visibility can be obtained.

  FIG. 17 is a configuration example of the imaging apparatus of the present embodiment. The imaging apparatus according to the present exemplary embodiment includes a non-visible light adjusting unit 701 having the configuration of the first exemplary embodiment (FIG. 1) between the non-visible light generating unit 105 and the image combining unit 107, and the white adjusting unit 106 and the image combining unit. A visible light adjusting unit 702 is provided between the unit 107 and the unit 107. Compared with the second and third embodiments, a visible light adjusting unit 702 is added. The imaging apparatus controls the light source corresponding to the image area in the same manner as in the first embodiment, and newly adjusts the signal levels of the invisible light and the visible light for each image area by the invisible light adjustment unit 701 and the visible light adjustment unit 702. .

  As in the second embodiment, the invisible light adjustment unit 701 multiplies the invisible light signal for each image region output from the invisible light signal generation unit 105 by the gain for each image region specified by the control unit 110. The invisible light signal for each image region is output to the image composition unit 107. The visible light adjustment unit 702 also multiplies the visible light signal for each image region output from the visible light signal generation unit 104 by the gain for each image region specified by the control unit 110, for each image region. Is output to the image composition unit 107.

  As in the first embodiment, the control unit 116 determines the amount of invisible light (infrared light) emitted from the light source unit 116 for each image region based on the relationship between the appropriate amount of infrared light and the absolute amount of visible light in advance. Control. The control unit 116 according to the present embodiment further sets a gain used by each of the invisible light adjustment unit 701 and the visible light adjustment unit 702 based on the relationship between the invisible light amount and the target invisible light amount for each image region. . The operation of the non-visible light adjusting unit 701 is the same as that of the second embodiment, and the operation of the visible light adjusting unit 702 is the non-visible light adjusting unit 701 except that the visible light signal output from the white adjusting unit 106 is targeted. It is the same as the operation of.

  FIG. 18 is an example of a shooting scene that is shot using the imaging device. Similarly to the example of the shooting scene in the first embodiment (FIG. 6), the light source unit 116 irradiates the left side toward the left side in the shooting direction and the right side toward the shooting direction. It has the light source part B which irradiates non-visible light. The subject in the photographing direction is irradiated with ambient light. Since the person A is in the shadow, the ambient light is not applied, and the person B, the person C, and the person D are irradiated with the ambient light.

  FIG. 19 is an image photographed by the imaging device under the photographing conditions of FIG. Persons A to D are photographed as subjects, and the image is divided into image areas A to D including each person. Image regions A and C are image regions irradiated with invisible light from the light source unit A, and image regions B and D are image regions irradiated with invisible light from the light source unit B. Therefore, the image area of the present embodiment is an area obtained by further dividing the image area of the first embodiment.

  FIG. 20 is a photographic scene in which invisible light is irradiated to the subject person A by the light source unit A. If the amount of invisible light in the image area A is smaller than the target amount of light according to the imaging conditions in FIG. 18, the control unit 110 increases the amount of invisible light emitted by the light source A as shown in FIG.

  FIG. 21 is an image captured by the light source unit A irradiating the subject person A with invisible light. In this way, an image with an appropriate amount of invisible light applied to the subject in the image area A can be acquired.

  On the other hand, since the irradiation light amount of the light source A is increased, the invisible light amount in the image area C is larger than the target invisible light amount. In such a case, the control unit 110 lowers at least one of the gain of the non-visible light adjustment unit 701 and the visible light adjustment unit 702 corresponding to the image region C and reduces the gain of the image region C input to the image composition unit 107. Control is performed so that the signal level (the sum of the signal level of invisible light and the signal level of visible light) does not increase.

  Further, in the image of FIG. 19, if the invisible light amount of the image area A is less than the target invisible light amount, the control unit 110 increases the irradiation light amount of the light source A as shown in FIG. An exposure control parameter is set in the imaging unit 101 so that the exposure time is shortened. By this procedure, the gain of the non-visible light adjusting unit 701 and the gain of the visible light adjusting unit 702 corresponding to the image areas A, B, and D are further increased and amplified so that the signal level is not lowered.

  As described above, the amount of invisible light is appropriately controlled individually for each subject present in the captured image, and the gain for adjusting the signal level for each image area smaller than the image area corresponding to the light source control is controlled. By doing so, the imaging apparatus generates an image with high color reproducibility and good visibility even when the increase or decrease in the visible light amount occurs for each image region in the captured image.

  The gains set in the invisible light adjusting unit 701 and the visible light adjusting unit 702 are set for each image region. However, the gains are gradually set within a predetermined range between the image regions so that the gain difference between the image regions is reduced. You may comprise so that it may change. With this configuration, it is possible to reduce the gain difference between the image areas, so that it is possible to suppress the adverse effects such as the occurrence of an edge.

  The image pickup apparatus of the present embodiment controls the light source so that the subject existing in the image area where the light amount of the invisible light of the light source irradiated to the subject is less than the target light amount is irradiated, The signal level of invisible light and visible light is changed for each image region, and the noise removal intensity of the invisible light and visible light is controlled for each image region.

  FIG. 22 shows a configuration example of this embodiment. The imaging apparatus of the present embodiment is provided with a non-visible light noise removing unit 2701 between the non-visible light adjusting unit 701 and the image synthesizing unit 107 having the configuration of the fifth embodiment (FIG. 17), and the visible light adjusting unit 702. A visible light noise removing unit 2702 is provided between the image synthesizing unit 107.

  The invisible light noise removing unit 2701 changes the intensity of noise removal processing for the invisible light signal for each image region. That is, the non-visible light noise removing unit 2701 can change for each image region whether the noise removal for the non-visible light signal is strong or the resolution is maintained while the noise removal is weakened.

  The visible light noise removal unit 2702 changes the intensity of noise removal processing on the visible light signal for each image region. In other words, the visible light noise removing unit 2702 can change for each image region whether the noise removal for the visible light signal is strong or the resolution is maintained while the noise removal is weakened.

  The noise removal method of the visible light noise removal unit 2702 and the invisible light noise removal unit 2701 may be any method that removes noise superimposed on an image. For example, there are a method of applying a spatial filter such as a moving average filter and a median filter, or a method of applying an interframe filter based on integration of interframe images.

  Similar to the first embodiment, the control unit 110 determines the amount of invisible light (infrared light) emitted from the light source unit 116 for each image region based on the relationship between the appropriate amount of infrared light and the absolute amount of visible light in advance. Control. Further, as in the fifth embodiment, the control unit 110 controls the signal gain of the visible light signal and the invisible light signal for each image region.

  In the present embodiment, the control unit 110 further controls the noise removal intensity of the invisible light noise removal unit 2701 for each image region in accordance with the magnitude relationship between the invisible light amount and the target invisible light amount for each image region. The noise removal strength of the visible light noise removal unit 2702 is controlled for each image region.

  According to the above configuration, the amount of light emitted from the light source unit 116 is increased, and the invisible light amount is still the target even in the image region in which the signal is amplified by the invisible light adjustment unit 701 and the visible light adjustment unit 702. When the amount is less than the invisible light amount, it is possible to increase the noise removal strength as an extremely dark region (= an image region where noise is conspicuous). On the other hand, for other image regions, it is possible to suppress adverse effects such as a decrease in resolution by reducing the noise removal strength.

  Moreover, it can be set as the structure which removes noise separately about each of a visible light signal and an invisible light signal, and the change of the noise removal intensity | strength according to the ratio of visible light and invisible light is attained.

  The invisible light noise removing unit 2701 and the visible light noise removing unit 2702 set the noise removal strength for each image region. However, the difference between the noise removal strengths between the image regions is reduced. You may comprise so that it may change gradually in a predetermined range. With this configuration, it is possible to reduce the noise removal intensity difference between the image areas, and to suppress adverse effects such as the generation of false edges.

  According to the imaging apparatus of the described embodiment, the color reproducibility of at least a desired region in an image is improved.

  In addition, when shooting a subject that is partially illuminated, or when shooting in a situation where a subject that emits light partially exists in the shot image, the imaging apparatus can output an image with a suitable image quality. .

  Furthermore, by providing a noise removal unit in the imaging apparatus, the imaging apparatus can output an image with good color reproducibility of at least a desired region in the image and a large signal-to-noise ratio.

  DESCRIPTION OF SYMBOLS 101 ... Imaging part, 103 ... Signal processing part, 104 ... Visible light signal generation part, 105 ... Non-visible light signal generation part, 106 ... White adjustment part, 107 ... Image composition 108: Luminance signal generation unit 109 109 Color difference signal generation unit 110 ... Control unit 112 ... Visible light amount detection unit 113 ... Invisible light amount detection unit 114 Signal level detection unit 116 ... light source unit 201 ... lens unit 202 ... stopper 203 ... image sensor unit 204 ... amplification unit 205 ... AD conversion unit 701 .. Invisible light adjusting unit, 1401... Filter, 702 .. Visible light adjusting unit, 2701... Invisible light noise removing unit, 2702.

Claims (7)

An imaging unit that is sensitive to invisible light and visible light, and that captures an image of a subject and generates an image signal;
A non-visible light amount detection unit for detecting the light amount of the non-visible light of the image signal of the image region obtained by dividing the image into a plurality of image regions;
A light source unit that irradiates the subject with the invisible light;
A filter that is arranged in the irradiation direction of the invisible light of the light source unit and adjusts the transmittance of the invisible light, and a light amount of the invisible light detected by the invisible light amount detection unit is less than a predetermined light amount. have a control unit for controlling the light source unit to illuminate the non-visible light to said object in said image area,
The controller is
In accordance with the difference between the predetermined light amount of the image area and the light amount of the invisible light detected by the invisible light amount detection unit, the light amount of the invisible light emitted by the light source unit is controlled,
An image pickup apparatus that controls the amount of the invisible light emitted from the light source unit by adjusting the transmittance of the filter .   An imaging unit that is sensitive to invisible light and visible light, and that captures an image of a subject and generates an image signal;
  A non-visible light amount detection unit for detecting the light amount of the non-visible light of the image signal of the image region obtained by dividing the image into a plurality of image regions;
  A light source unit that irradiates the subject with the invisible light;
  A non-visible light signal generating unit that generates the non-visible light signal of the image region from the image signal from the imaging unit;
  A non-visible light adjusting unit that adjusts the non-visible light signal by multiplying the determined non-visible light signal generated by the non-visible light signal generating unit by the determined first gain; and
  A control unit that controls the light source unit to irradiate the subject in the image area where the light amount of the invisible light detected by the invisible light amount detection unit is less than a predetermined light amount; ,
  The control unit determines the first gain in accordance with a difference between the predetermined light amount and the light amount detected by the invisible light amount detection unit in the image area.   A visible light signal generation unit that generates the visible light signal of the image region from the image signal from the imaging unit; and
  A visible light adjustment unit that adjusts the visible light signal by multiplying the visible light signal generated by the visible light signal generation unit by the determined second gain;
  The control unit adjusts at least one of the first and second gains according to an increment from the predetermined light amount of the invisible light due to irradiation from the light source unit in the image region. The imaging apparatus according to claim 2, characterized in that:   A non-visible light noise removing unit that removes the non-visible light noise superimposed from the non-visible light signal of the image region adjusted by the non-visible light adjusting unit with a first noise removal strength;
  A visible light noise removing unit that removes visible light noise superimposed from the visible light signal of the image region adjusted by the visible light adjusting unit with a second noise removal intensity;
  The control unit determines the first and second noise removal intensities according to a difference between the predetermined light amount and the light amount detected by the invisible light amount detection unit in the image area. The imaging device according to claim 3.   An imaging unit that has sensitivity to invisible light and visible light and captures an image of a subject to generate an image signal, a light source unit that irradiates the subject with the invisible light, and the invisible light of the light source unit An imaging method using an imaging device having a filter that adjusts the transmittance of the invisible light in the irradiation direction of the imaging device,
  Detecting the light amount of the invisible light of the image signal of the image region obtained by dividing the image into a plurality of image regions;
  Controlling the light source unit so as to irradiate the invisible light to the subject in the image region where the detected amount of the invisible light is less than a predetermined amount of light;
  In accordance with the difference between the predetermined light amount of the image area and the detected light amount of the invisible light, and controlling the light amount of the invisible light irradiated by the light source unit,
  The imaging method characterized by controlling the light quantity of the said invisible light which the said light source part irradiates by adjusting the said transmittance | permeability of the said filter.   An imaging method using an imaging device having sensitivity to invisible light and visible light and having an imaging unit that captures an image of a subject to generate an image signal and a light source unit that irradiates the subject with the invisible light. The imaging device
  Detecting the light amount of the invisible light of the image signal of the image region obtained by dividing the image into a plurality of image regions;
  Controlling the light source unit so as to irradiate the invisible light to the subject in the image region where the detected amount of the invisible light is less than a predetermined amount of light;
  Determining a first gain according to a difference between the predetermined light amount and the detected light amount of the invisible light in the image area;
  From the image signal from the imaging unit, generating the invisible light signal of the image region,
  An imaging method comprising adjusting the invisible light signal by multiplying the generated invisible light signal by the first gain.   The imaging device
  From the image signal from the imaging unit, generate the visible light signal of the image region,
  Multiplying the generated visible light signal by the determined second gain to adjust the visible light signal;
  The at least one of the first gain and the second gain is adjusted according to an increment from the predetermined light amount of the invisible light due to irradiation from the light source unit in the image region. 6. The imaging method according to 6.

Images