JP2021129275A - Imaging apparatus and imaging method - Google Patents

Abstract

To provide an imaging apparatus and an imaging method capable of suppressing reduction in image quality of even image data captured under a light source not having infrared components.SOLUTION: An imaging apparatus 10 comprises: an imaging unit 23 which converts incident light from an imaging target, to a first pixel signal including infrared components and invisible light component; and a signal processing unit 25 which performs predetermined processing by using the first pixel signal in a case where a light source to irradiate the imaging target with light is a predetermined light source, but performs the predetermined processing by using a second pixel signal obtained by reducing the infrared components from the first pixel signal in a case where the light source is not the predetermined light source.SELECTED DRAWING: Figure 1

Description

The present disclosure relates to an imaging apparatus and an imaging method.

The image sensor is sensitive not only to visible light but also to infrared light. The infrared light component acquired by such an image sensor improves visibility at night. On the other hand, when shooting is performed using this image sensor under a light source having a large amount of infrared light, color reproduction is impaired due to the influence of infrared light. Therefore, for example, in the daytime body, a process for reducing the infrared light component may be performed.

However, if the captured image data captured under a light source having no infrared component is subjected to a process of reducing the infrared light component, the image quality may be reduced.

Japanese Unexamined Patent Publication No. 2006-237738 JP-A-2019-205018

One aspect of the present disclosure provides an imaging device and an imaging method capable of suppressing a reduction in image quality even for image data captured under a light source having no infrared component.

In order to solve the above problems, in the present disclosure, an imaging unit that converts incident light from an imaging target into a first pixel signal containing an infrared light component and a visible light component and outputs the signal, and an imaging unit.
When the light source that irradiates the image pickup target with light is a predetermined light source, a predetermined process is performed using the first pixel signal, and when the light source is not a predetermined light source, the red from the first pixel signal. An image pickup apparatus is provided that includes a color processing unit that performs predetermined processing using a second pixel signal with a reduced external light component.

Further, a determination processing unit for determining a light source that irradiates the image pickup target with light based on the information on the infrared light component is provided.
Based on the determination information of the determination processing unit, the color processing unit changes the processing when the light source that irradiates the imaging target with light is a predetermined light source and the processing when the light source is not a predetermined light source. You may.

When it is determined that the infrared light component is less than a predetermined value, the determination processing unit may determine that it is the predetermined light source.

When the light source is a predetermined light source, the color processing unit may convert the first pixel signal into a Bayer array.

When the light source is not a predetermined light source, the color processing unit may convert the second pixel signal into a Bayer array.

A separation processing unit that separates the first pixel signal into the infrared light component and the visible light component is further provided.
The determination processing unit may determine the light source by using the infrared light component separated by the separation processing unit.

The determination processing unit may determine a light source that irradiates the image pickup target with light by using the visible light component.

The determination processing unit may calculate the color temperature using the visible light component and determine the light source based on the color temperature.

The visible light component is a color component of each of red light, blue light, and green light, and the determination processing unit determines the light source based on the ratio of each color component of red light, blue light, and green light. You may.

The determination processing unit may determine the light source by using the information regarding the infrared light component and the color temperature.

The determination processing unit may determine that the light source is a predetermined light source when the color temperature is in a predetermined range and the infrared light component is less than a predetermined value.

In order to solve the above problems, in the present disclosure, an imaging unit that converts incident light from an imaging target into a first pixel signal containing an infrared light component and a visible light component and outputs the signal, and an imaging unit.
A determination processing unit that determines whether or not to perform color processing for reducing the infrared light component from the first pixel signal based on the visible light component.
A color processing unit that performs color processing for reducing the infrared light component from the first pixel signal when it is determined to perform color processing for reducing the infrared light component.
An imaging device comprising the above is provided.

The determination processing unit may perform the determination based on the color temperature calculated using the visible light component.

The visible light component is a color component of each of red light, blue light, and green light, and the determination processing unit performs the determination based on the ratio of each color component of red light, blue light, and green light. May be good.

The determination processing unit may make the determination using the information regarding the infrared light component and the color temperature.

The determination processing unit does not perform color processing for reducing the infrared light component from the first pixel signal when the color temperature is in a predetermined range and the infrared light component is less than a predetermined value. May be determined.

In order to solve the above-mentioned problems, in the present disclosure, an imaging unit that converts incident light from an optical imaging target into a first pixel signal including an infrared light component and a visible light component and outputs the light is used.
A color that reduces the infrared light component from the first pixel signal based on at least one of information about the sunrise time and the sunset time, the position information of the light source that irradiates the imaging target with light, and the information about the illuminance of the headlight. A judgment processing unit that determines whether or not to perform processing,
A color processing unit that performs color processing for reducing the infrared light component from the first pixel signal when it is determined to perform color processing for reducing the infrared light component.
An imaging device comprising the above is provided.

The judgment processing unit
When it is determined that the tunnel is inside the tunnel based on the position information, it is determined that the color processing for reducing the infrared light component from the first pixel signal is not performed.
Or
If it is determined that the tunnel is outside the tunnel based on the position information and the current time is after the sunrise time and before the sunset time, it is determined that the color processing for reducing the infrared light component is performed.
Or
When the illuminance of the headlight is equal to or higher than a predetermined threshold value, it may be determined that the color processing for reducing the infrared light component from the first pixel signal is performed.

In order to solve the above problems, in the present disclosure, an imaging step of converting incident light from an imaging target into a first pixel signal including an infrared light component and a visible light component and outputting the signal is described.
When the light source that irradiates the image pickup target with light is a predetermined light source, a predetermined process is performed using the first pixel signal, and when the light source is not a predetermined light source, the red from the first pixel signal. A color processing step in which a predetermined process is performed using a second pixel signal with a reduced external light component, and
An imaging method is provided.

The figure which shows an example of the structure of the image pickup apparatus which concerns on embodiment. The figure which shows the example of the light transmittance of each wavelength of the optical filter of FIG. The figure explaining the pixel arrangement of an image sensor. The block diagram which shows the structural example of the signal processing part. The figure which shows the pixel arrangement of the image after bayerization processing by a signal processing unit. The figure which shows the example of the separation processing of an infrared light component IR. The figure which shows the 2nd pixel signal in the right figure of FIG. 6 as an image in a * b * color space. The figure which shows the 2nd pixel signal in the lower left figure of FIG. 6 as an image in a * b * color space. The figure which shows the separation processing example of the spectral characteristic in the 1st pixel signal. The flowchart which shows the processing example which concerns on 1st Embodiment. The block diagram which shows the structural example of the signal processing part which concerns on 2nd Embodiment. The flowchart which shows the processing example which concerns on 2nd Embodiment. The block diagram which shows the structural example of the signal processing part which concerns on 3rd Embodiment. The flowchart which shows the processing example which concerns on 4th Embodiment. The block diagram which shows the structural example of the signal processing part which concerns on 4th Embodiment. The flowchart which shows the processing example which concerns on 4th Embodiment. The block diagram which shows the schematic configuration example of the vehicle control system which is an example of the mobile body control system to which the technique which concerns on this disclosure can be applied. The figure which shows the example of the installation position of the image pickup part.

Hereinafter, embodiments of an imaging apparatus and an imaging method will be described with reference to the drawings. In the following, the main components of the imaging device and the imaging method will be mainly described, but the imaging device and the imaging method may have components and functions not shown or described. The following description does not exclude components or functions not shown or described.

(First Embodiment)
The configuration of the image pickup apparatus according to the present embodiment will be described with reference to FIG. FIG. 1 is a diagram showing an example of the configuration of the image pickup apparatus according to the present embodiment. The image pickup device 10 is, for example, a device capable of capturing an image corresponding to infrared light and visible light from the incident light reflected by the image pickup target, and is a lens 21, an optical filter 22, an image sensor 23, a storage unit 24, and a signal. The processing unit 25 is provided. The imaging target includes, for example, a landscape, a car, a person, or an object that reflects light, an object that emits light, or the like. In the image pickup device 10 according to the present embodiment, the storage unit 24 is configured in the image pickup device 10, but the present invention is not limited to this. For example, the storage unit 24 may be configured outside the image pickup device 10.

Light from the subject is incident on the lens 21. The lens 21 collects the incident light on the image sensor 23 via the optical filter 22. The optical filter 22 transmits infrared light and visible light among the incident light.

FIG. 2 is a diagram showing an example of light transmittance of each wavelength of the optical filter 22 of FIG. In the graph of FIG. 2, the horizontal axis represents the wavelength (nm) of light passing through the optical filter 22, and the vertical axis represents the transmittance (%) of the light. As shown in FIG. 2, in the optical filter 22, for example, the transmittance of visible light having a wavelength of about 390 nm to 660 nm and the transmittance of infrared light having a wavelength of about 820 nm to 1080 nm are larger than 0%. Therefore, the optical filter 22 passes visible light having a wavelength of about 390 nm to 660 nm and infrared light having a wavelength of about 820 nm to 1080 nm. The wavelengths of 820 nm to 1080 nm are examples, and the wavelength is not limited to this. For example, the infrared light may include the entire band of the near infrared light, or may include the entire band of the far infrared light.

The image sensor 23 is a CMOS (Complementary Metal-Oxide Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and has pixels in an RGBW pixel array. The image sensor 23 according to this embodiment corresponds to the imaging unit.

FIG. 3 is a diagram illustrating a pixel arrangement of the image sensor 23. As shown in FIG. 3, the pixel array of the captured image output from the image sensor 23 is an RGBW pixel array. Further, the first pixel signal of each pixel includes an infrared light (IR) component. The image sensor 23 according to this embodiment corresponds to the imaging unit.

The image sensor 23 uses the incident infrared light and visible light for each pixel, and the colors corresponding to the pixels (red (R), green (G), blue (B), white (W)). By receiving the light of the above and generating electric signals corresponding to the amount of received light as first pixel signals W _{+ IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR} , imaging is performed on a frame-by-frame basis. Therefore, the first pixel signal of each pixel obtained as a result of imaging includes an infrared light component IR and a visible light component as invisible light components. That is, the visible light component is, for example, any one of a red light component (R), a green light component (G), and a blue light component (B). _{The image sensor 23 AD-converts the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR} of each pixel obtained as a result of imaging and stores them in the storage unit 24 as captured image data.

The storage unit 24 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory. The storage unit 24 stores captured image data.

The signal processing unit 25 performs color processing for reducing the infrared light component from _{the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR.} More specifically, when the light source that irradiates the image pickup target with light is a predetermined light source, the signal processing unit 25 uses the first pixel signals W _{+ IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR} to determine the predetermined light source. When processing is performed and the light source is not a predetermined light source, the second pixel signals R, G, and B obtained by reducing the infrared light component IR from _{the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR are used.} Perform a predetermined process.

Further, the signal processing unit 25 performs a Bayerization process (remosaic process) on the captured image of the non-Bayer array supplied from the image sensor 23 to generate an RGB image of the Bayer array (RGB pixel array).

Further, the signal processing unit 25 can perform signal processing such as white balance processing and YCbCr conversion processing as development processing using RGB images. For example, the signal processing unit 25 outputs a YCbCr image obtained as a result of performing white balance processing and YCbCr conversion processing on the RGB image.

FIG. 4 is a block diagram showing a configuration example of the signal processing unit 25. As shown in FIG. 4, the signal processing unit 25 includes a separation processing unit 100, a determination processing unit 102, and a color processing unit 104.

_{Based on the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR} stored in the storage unit 24, the separation processing unit 100 uses the following equation (1) to obtain the first pixel signals W _{+ IR} , R _{+ IR.} , G _{+ IR} , and B _{+ IR} , respectively, the infrared light component IR and the visible light component second pixel signals R, G, and B are separated. In this way, the separation processing unit 100 separates the first pixel signals W _{+ IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR} into the infrared light component IR and the second pixel signals R, G, and B.

As described above, IR is an infrared light component contained in each of _{the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR.} Further, the second pixel signals R, G, and B are visible light components of _{the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR as described above.}

The determination processing unit 102 determines the light source that irradiates the image pickup target with light based on the information regarding the infrared light component IR. That is, the determination processing unit 102 has an infrared light component acquisition unit 200 and a determination unit 202.

The infrared light component acquisition unit 200 acquires the infrared light component IR from the separation processing unit 100. The infrared light component acquisition unit 200 may acquire information on the infrared light component IR from an external device.

The determination unit 202 estimates the light source that irradiates the imaging target with light based on the information regarding the infrared light component. The determination unit 202 determines whether or not the infrared light component IR is smaller than the predetermined value, and if it is smaller than the predetermined value, determines that the light source is a predetermined light source. For example, when the maximum value of the infrared light component IR is 4028, for example, 0 to 200, which is a value of 0 to 5% of the maximum value, is set as a predetermined value.

_{The color processing unit 104 multiplies each of the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR} by the white balance gain when the determination unit 202 determines that the light source is a predetermined light source. .. As a result, the ratio (white balance) of the first pixel signals R + IR, G + IR, and B + IR is corrected. Subsequently, the color processing unit 104 performs bayerization processing. In this case, since the infrared light component IR is a value smaller than the predetermined value without reducing the infrared light component IR, the first pixel signals W _{+ IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR} are the second pixels. It has values similar to the signals R, G, and B. Note that the white balance gain multiplication process, bayerization process, and the like are examples of predetermined processes.

On the other hand, when the determination unit 202 determines that the light source is not a predetermined light source, the color processing unit 104 reduces the infrared light component IR from _{the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR.} The white balance gain is multiplied for each of the two pixel signals R, G, and B. As a result, the ratio (white balance) of the second pixel signals R, G, and B is corrected. Subsequently, the color processing unit 104 performs bayerization processing.

FIG. 5 is a diagram showing a pixel arrangement of an image after bayerization processing by the color processing unit 104. As shown in FIG. 5, the color processing unit 104 generates an image of an RGB pixel array as an image after the baying process by performing the baying process on the image before the baying process. ..

When the determination unit 202 determines that the light source is not a predetermined light source, the color processing unit 104 uses the following equation (2) to set the second pixel signals R, G, and B as the luminance signal Y as an example of the predetermined processing. And the color difference signals Cb and Cr. In this case, when the determination unit 202 determines that the light source is a predetermined light source, the color processing unit 104 replaces the second pixel signals R, G, and B with the first pixel signals W _{+ IR} , R _{+ IR} , and G. ₊ using _IR, and _{B + IR,} calculates the equation (2).

FIG. 6 is a diagram showing an example of separation processing of the infrared light component IR. The upper figure on the left side shows the characteristics of the high-pressure sodium lamp, and the middle figure shows the spectral characteristics of _{the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR imaged under the high-pressure sodium lamp light source.} The figure below is a diagram showing the spectral characteristics _{of the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR} when the separation process of the infrared light component IR is not performed. The figure on the right shows the second pixel signals R, G, and B in which the infrared light component IR is separated from _{the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR.} The horizontal axis of each figure indicates the wavelength, and the vertical axis indicates the signal strength in an arbitrary unit.

As shown in the middle figure on the left side _{, the infrared light component IR is included in the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR} when the infrared light component IR is not separated. No. Therefore, the infrared light component IR calculated by the separation processing unit 100 according to the equation (1) becomes almost 0. As a result, the determination unit 202 determines that the light source is a predetermined light source, for example, a high-voltage halogen lamp. Then, since the color processing unit 104 is determined to be a predetermined light source by the determination unit 202, the bayering process, which is an example of the predetermined processing, uses _{the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR.} I do.

FIG. 7A is a diagram showing the second pixel signals R, G, and B in the right figure of FIG. 6 as images in the a * b * color space. An ideal color reproduction example is also shown in FIG. 7A. As shown in FIG. 7A, the infrared light component IR separation process of the equation (1) _{is performed on the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR} whose infrared light component IR is equal to or less than a predetermined value. When this is done, the amount of deviation from the ideal color reproduction example increases.

_{FIG. 7B is a diagram showing the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR} in the lower left figure of FIG. 6 as images in the a * b * color space. An ideal color reproduction example is also shown in FIG. 6B. As shown in FIG. 7B, the infrared light component IR separation process of the equation (1) _{is performed on the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR} whose infrared light component IR is equal to or less than a predetermined value. If this is not done, the amount of deviation from the ideal color reproduction example will decrease.

FIG. 8 is a diagram showing an example of separation processing of spectral characteristics in _{the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR} imaged under a light source that shakes the infrared light component IR. The figure on the left shows the spectral characteristics of the first pixel signals W _{+ IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR} before the IR separation process, and the figure on the right shows the second pixel signals R, G, and B after the IR separation process. The spectral characteristics of are shown. The horizontal axis of each figure indicates the wavelength, and the vertical axis indicates the signal strength in an arbitrary unit. As shown in the figure on the right side, in the spectral characteristics of the second pixel signals R, G, and B after the separation process, the infrared light component IR is reduced to almost 0, and the influence of infrared light is eliminated.

FIG. 9 is a flowchart showing a processing example according to the first embodiment. Here, the processing after the first pixel signals W _{+ IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR} are imaged by the image sensor 23 will be described.

First, the separation processing unit 100 calculates the infrared light component IR according to the equation (1) based on _{the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR stored in the storage unit 24 (step).} S100).

Next, the determination unit 202 determines whether or not the light source is a predetermined light source based on the infrared light component IR (step S102). When it is determined that the light source is a predetermined light source (No in step S102), the IR separation process of the formula (1) is not performed (step S104).

On the other hand, when it is determined that the light source is not a predetermined light source (Yes in step S102), the IR separation process of the formula (1) is performed (step S106). As a result, when it is determined that there is a predetermined light source, the color processing unit 104 performs a bayerization process based on _{the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR, and is not a predetermined light source.} In the case of determining that, the color processing unit 104 performs the baying process based on the second pixel signals R, G, and B that have undergone the IR separation process of the equation (1) (step S108).

As described above, the determination unit 202 determines whether or not there is a predetermined light source based on the infrared light component IR, and when it is determined that the predetermined light source is, for example, a high-pressure sodium lamp, the color processing unit 104 determines. , The first pixel signal W _{+ IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR} are used to perform a bayering process, which is a predetermined process. _{As a result, for the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR,} which contain only the infrared light component IR smaller than the predetermined value, the image quality of the infrared light component IR may be reduced. Since the reduction process is not performed, the reduction in image quality is suppressed.

On the other hand, when it is determined that the light source is not a predetermined light source, the color processing unit 104 performs a predetermined process based on the second pixel signals R, G, and B in which the infrared light component IR is reduced by the equation (1). The bayerization process is performed. As a result, it is possible to prevent the color reproducibility from being impaired by the influence of infrared light.

(Second Embodiment)
In the image pickup apparatus 10 according to the first embodiment, it is determined whether or not the light source is a predetermined light source based on the infrared light component IR, but in the image pickup apparatus 10 according to the second embodiment, the information regarding the color temperature is used. Based on this, it differs in that it is determined whether or not color processing for reducing the infrared light component IR is performed. Hereinafter, the differences from the image pickup apparatus 10 according to the first embodiment will be described.

FIG. 10 is a block diagram showing a configuration example of the signal processing unit 25 according to the second embodiment. As shown in FIG. 10, the determination processing unit 102 of the signal processing unit 25 according to the second embodiment includes a determination unit 202 and a color temperature calculation unit 204.

As shown in the equations (3) and (4), the color temperature calculation unit 204 converts the second pixel signals R, G and B calculated by the separation processing unit 100 into the chromaticities x and y, and formulates the formula. Based on (5), the color temperature is converted to C. Here, a, b, and c are coefficients.

In this way, the color temperature calculation unit 204 calculates the color temperature C using the second pixel signals R, G, and B calculated by the separation processing unit 100. For example, the color temperature of a high-pressure sodium lamp is 2000K to 2100K. Also, the candle flame is 1800K, the incandescent light bulb is 3200K, the sunlight on a sunny day is 5000-6000K, the cloudy weather is 7000K, and the shade on a sunny day is 7500K.

The determination unit 202 determines whether or not to perform color processing for reducing the infrared light component from the first pixel signals R + IR, G + IR, and B + IR based on the color temperature C calculated by the color temperature calculation unit 204. Further, the determination unit 202 determines whether or not the light source is a predetermined light source based on the color temperature C calculated by the color temperature calculation unit 204.

When the color temperature is 2000K to 2100K, the determination unit 202 determines that the color processing for reducing the infrared light component from _{the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR is not performed.} That is, the determination unit 202 determines that the light source is a predetermined light source when the color temperature is 2000K to 2100K. In this way, the determination unit 202 can also determine whether or not the light source is a predetermined light source by using the second pixel signals R, G, and B separated by the separation processing unit 100.

More specifically, the determination unit 202 determines the light source based on the color temperature C calculated by the color temperature calculation unit 204. The determination unit 202 has, for example, information indicating the name of the light source for each color temperature as a table. As a result, the determination unit 202 determines that the high-pressure sodium lamp is a predetermined light source if the color temperature is, for example, 2000K to 2100K.

On the other hand, if the color temperature is 1800K, it is determined that it is a "candle flame" that is not a predetermined light source, and if the color temperature is 3200K, it is determined that it is a "incandescent light bulb" that is not a predetermined light source, and the color temperature is If it is 5000 to 6000 K, it is determined that it is "sunlight" that is not a predetermined light source.

_{As a result, the color processing unit 104 uses the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR} to form a Bayer array when the color temperature is 2000K to 2100K, that is, when it is a high-pressure sodium lamp. Convert. In this case, the first pixel signals R + IR, G + IR, and B + IR, which contain only the infrared light component IR smaller than the predetermined value, are not subjected to the infrared light component IR reduction processing that may reduce the image quality. , The reduction of image quality is suppressed.

On the other hand, when the color temperature is, for example, 1800K, 3200K, 5000-7500K, the determination unit 202 determines that the light source is not a predetermined light source. That is, the determination unit 202 determines that color processing for reducing the infrared light component from _{the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR is performed.} As a result, the color processing unit 104 converts, for example, into a Bayer array using the second pixel signals R, G, and B when the color temperatures are 1800K, 3200K, and 5000 to 7500K. In the case of such a color temperature, since the first pixel signals W _{+ IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR} contain a red light component IR of a predetermined value or more, the second pixel signal IR is separated. By using the pixel signals R, G and B, it is possible to suppress the deterioration of color reproducibility due to the influence of infrared light.

Even if the determination unit 202 determines the light source based on the ratio of the second pixel signals R, G, and B, which are the color components of the red light, the blue light, and the green light, which are the information corresponding to the color temperature. good. In this case, the determination unit 202 has, for example, information indicating the light source names for each ratio of the second pixel signals R, G, and B as a table. As a result, the determination unit 202 determines that the high-pressure sodium lamp is a predetermined light source if, for example, the ratio of the two pixel signals R, G, and B is a predetermined value.

FIG. 11 is a flowchart showing a processing example according to the second embodiment. Here, the processing after the first pixel signals W _{+ IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR} are imaged by the image sensor 23 will be described.

First, the color temperature calculation unit 204 calculates the color temperature C using the second pixel signals R, G, and B calculated by the separation processing unit 100 (step S200).
Next, the determination unit 202 determines the light source based on the color temperature C calculated by the color temperature calculation unit 204. (Step S202). When it is determined that the light source is a predetermined light source (Yes in step S204), the IR separation process of the formula (1) is not performed (step S104).

On the other hand, when it is determined that the light source is not a predetermined light source (No in step S204), the IR separation process of the formula (1) is performed (step S106).

As described above, the determination unit 202 determines the light source based on the information regarding the color temperature C. When the light source is a predetermined light source, the color processing unit 104 performs a bayerization process, which is a predetermined process, using _{the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR. As a result, for the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR,} which contain only the infrared light component IR smaller than the predetermined value, the image quality of the infrared light component IR may be reduced. Since the reduction process is not performed, the reduction in image quality is suppressed.

On the other hand, when it is determined that the light source is not a predetermined light source, the color processing unit 104 performs a predetermined process based on the second pixel signals R, G, and B in which the infrared light component IR is reduced by the equation (1). The bayerization process is performed. As a result, it is possible to prevent the color reproducibility from being impaired by the influence of infrared light.

(Third Embodiment)
The determination unit 202 of the image pickup apparatus 10 according to the third embodiment is different from the determination unit 202 according to the third embodiment in that the light source is estimated by using the infrared light component IR in addition to the information regarding the color temperature C. do. Hereinafter, the differences from the image pickup apparatus 10 according to the second embodiment will be described.

FIG. 12 is a block diagram showing a configuration example of the signal processing unit 25 according to the third embodiment. As shown in FIG. 12, the determination processing unit 102 of the signal processing unit 25 according to the third embodiment further includes an infrared light component acquisition unit 200.

The determination unit 202 estimates the light source by also using the infrared light component IR acquired by the infrared light component acquisition unit 200. That is, the determination unit 202 determines the light source using the infrared light component and the information regarding the color temperature. For example, the determination unit 202 determines that the light source is a predetermined light source when the color temperature C is in a predetermined range and the infrared light component IR is less than a predetermined value.

More specifically, the determination unit 202 determines whether or not the light source is a predetermined light source by using the infrared light component IR when the color temperature is a predetermined temperature. For example, the determination unit 202 determines whether or not the light source is a high-pressure sodium lamp by using the infrared light component IR when the color temperature is 2000K to 2100K. As a result, it is possible to prevent the sunset or sunrise from being mistaken for a high-pressure sodium lamp. For example, the setting sun and the morning sun are about 2000K, and there is a possibility that the setting sun and the morning sun cannot be distinguished from the high-pressure sodium lamp by the judgment based only on the color temperature. On the other hand, in the case of a setting sun or the morning sun, the infrared light component IR is equal to or higher than a predetermined value. Therefore, when the color temperature is 2000K to 2100K, if the infrared light component IR is equal to or higher than a predetermined value, the determination unit 202 determines that it is a sunset or the sunrise, and the infrared light component IR is less than the predetermined value. If there is, it is determined that the lamp is a high-pressure sodium lamp.

In this way, by using the information of the infrared light component IR, the determination accuracy of the light source is further improved. Further, when the color temperature is other than the predetermined color temperature, the information of the infrared light component IR is not used, so that the decrease in the processing speed of the determination unit 202 is suppressed.

FIG. 13 is a flowchart showing a processing example according to the fourth embodiment. Here, the processing after the first pixel signals W _{+ IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR} are imaged by the image sensor 23 will be described.

The determination unit 202 determines whether or not the color temperature C calculated by the color temperature calculation unit 204 is a predetermined temperature (step S300). When it is determined that the temperature is a predetermined temperature (Yes in step S300), the information of the infrared light component IR is acquired (step S100), and the light source is estimated (step S302). On the other hand, when it is determined that the temperature is not a predetermined temperature (No in step S300), the light source is estimated without acquiring the infrared light component IR information (step S302).

As described above, in the image pickup apparatus 10 according to the present embodiment, when the color temperature C is a predetermined temperature of 2000K to 2100K, the determination unit 202 also uses the infrared light component IR to determine whether the light source is a predetermined light source. Judge whether or not. As a result, when different light sources (high-pressure sodium lamp, sunset or sunrise) exist even if the color temperature is the same 2000K to 2100K, the type of light source is high-pressure sodium lamp or sunset depending on the value of the infrared light component IR. Or it becomes possible to estimate whether it is the Asahi. Further, when the color temperature is other than the predetermined color temperature, the information of the infrared light component IR is not used, so that the decrease in the processing speed of the determination unit 202 is suppressed.

(Fourth Embodiment)
Based on the information about the environment, the image pickup apparatus 10 according to the fourth embodiment determines whether or not to perform color processing for reducing the infrared light component IR from _{the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR.} It differs from the image pickup apparatus 10 according to the first embodiment in that it determines. Hereinafter, the differences from the image pickup apparatus 10 according to the first embodiment will be described.

FIG. 14 is a block diagram showing a configuration example of the signal processing unit 25 according to the fourth embodiment. As shown in FIG. 14, the signal processing unit 25 according to the fourth embodiment includes an information acquisition unit 106, a position determination unit 210, a sunshine time determination unit 212, and a headlight information determination unit 214.

The information acquisition unit 106 acquires position information from GPS (Global Positioning System). In addition, the information acquisition unit 106 acquires information on the sunrise time and sunset time at that position from weather information and the like. Furthermore, the information acquisition unit 106 acquires information on the amount of light from the headlights of the vehicle on which the image pickup device 10 is mounted.

The position determination unit 210 determines whether or not the image pickup apparatus 10 is at the position where the high-pressure sodium lamp is turned on, based on the position information acquired by the information acquisition unit 106. For example, the position determination unit 210 determines that the image pickup device 10 is at a position where the high-pressure sodium lamp is turned on if it is in a tunnel. In this case, when the position determination unit 210 determines that the high-pressure sodium lamp is lit, the determination unit 202 emits infrared light from _{the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR.} It is determined that the color treatment for reducing the component IR is not performed. In this way, in the tunnel, the high-pressure sodium lamp is normally lit, but since it is lit, it can be determined with higher accuracy that the high-pressure sodium lamp is lit. When it is determined that the color processing unit 104 is in the tunnel, the color processing unit 104 converts it into a Bayer array using _{the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR.}

The sunshine time determination unit 212 determines whether or not the position of the image pickup apparatus 10 is outside the tunnel and the current time is in the daytime time zone based on the information of the sunrise time and the sunset time acquired by the information acquisition unit 106. .. That is, the sunshine time determination unit 212 determines whether or not "sunrise time" <"current time" <"sunset time". Even if "sunrise time" <"current time" <"sunset time", it is determined that the time zone is midday.

When the determination unit 202 determines that it is in the daytime time zone, it determines that the color processing for reducing the infrared light component IR from _{the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR is performed.} On the other hand, when it is determined that the time zone is not in the daytime, it is determined that the color processing for reducing the infrared light component IR from _{the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR is not performed.} As a result, when it is determined that the time zone is in the daytime, the color processing unit 104 converts the second pixel signals R, G, and B into a Bayer array. On the other hand, when it is determined that the time zone is not in the daytime, the color processing unit 104 _{converts the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR} into a Bayer array.

The headlight information determination unit 214 determines whether the position of the imaging device 10 is outside the tunnel and the light amount of the headlight is equal to or more than a predetermined threshold value based on the position information and the information of the light amount of the headlight acquired by the information acquisition unit 106. Judge whether or not. In this case, when the headlight information determination unit 214 determines that the amount of light of the headlight is equal to or greater than a predetermined threshold value, the determination unit 202 obtains the first pixel signals W _{+ IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR.} It is determined that color processing for reducing the infrared light component IR is performed. The amount of light of the headlight is automatically adjusted according to the information of the external illuminance. From this, if the illuminance corresponds to the high-pressure sodium lamp, the amount of light of the headlight is automatically controlled to be less than a predetermined threshold value. On the other hand, if the illuminance does not correspond to the high-pressure sodium lamp and is equal to or less than a predetermined illuminance, the amount of light of the headlight is automatically controlled to be equal to or higher than a predetermined threshold value.

As a result, when it is determined that the amount of light of the headlight is equal to or greater than a predetermined threshold value, the color processing unit 104 converts the headlights into a Bayer array using the second pixel signals R, G, and B. As described above, if the amount of light of the headlight is less than a predetermined threshold value, it can be determined with higher accuracy that the image is illuminated by the high-pressure sodium lamp. Further, the headlight information determination unit 214 may change the threshold value based on the time information. This makes it possible to consider the effects of sunlight at dawn / dusk.

FIG. 15 is a flowchart showing a processing example according to the fourth embodiment. Here, the processing after the first pixel signals W _{+ IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR} are imaged by the image sensor 23 will be described.

The position determination unit 210 determines whether or not the image pickup apparatus 10 is in the tunnel based on the position information acquired by the information acquisition unit 106 (step S400). When determining that the tunnel is inside (Yes in step S300), the determination unit 202 determines that IR separation is not performed (step S104). On the other hand, when determining that the vehicle is outside the tunnel (No in step S400), the sunshine time determination unit 212 determines whether or not the current time is in the daytime zone based on the information of the sunrise time and the sunset time. (Step S402). When it is determined that the time zone is not midday (No in step S402), the determination unit 202 determines whether or not the illuminance corresponds to the high-pressure sodium lamp (step S404). When the determination unit 202 determines that the amount of light of the headlight is equal to or greater than a predetermined threshold value, the headlight information determination unit 214 determines that the illuminance does not correspond to the high-pressure sodium lamp (No in step S404).

On the other hand, the determination unit 202 determines that the illuminance corresponding to the high-pressure sodium lamp is illuminance when the headlight information determination unit 214 determines that the amount of light of the headlight is less than a predetermined threshold value (Yes in step S404). ). In this way, it is possible to determine whether or not the position is illuminated by the high-pressure sodium lamp even outside the tunnel. For example, the vicinity of an interchange on a highway may be illuminated by a high-pressure sodium lamp even though it is outside the tunnel.

On the other hand, when it is determined that the time zone is midday (Yes in step S402), the determination unit 202 determines that IR separation is performed (step S106).

As described above, in the image pickup apparatus 10 according to the present embodiment, the determination unit 202 determines the first pixel signal based on at least one of the information regarding the sunrise time and the sunset time, the position information, and the information regarding the illuminance of the headlight. It is determined whether or not color processing for reducing the infrared light component IR from _{W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR is performed.} Since the daytime time zone is determined more accurately based on the information about the sunrise time and the sunset time, the infrared light from _{the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR during the daytime time zone.} If color processing that reduces the component IR is performed, the determination can be made with higher accuracy. Further, based on the position information, it is determined whether or not the tunnel is inside. As a result, it can be determined that the high-pressure sodium lamp is lit in the tunnel, so that the infrared light component IR is reduced from _{the first pixel signals W + IR} , R _{+ IR} , G _{+ IR} , and B _{+ IR.} It can be determined with higher accuracy that the color processing is not performed. Furthermore, based on the information on the illuminance of the headlight, if the illuminance of the headlight is less than a predetermined threshold value, it is determined that the position is illuminated by the high-pressure sodium lamp outside the tunnel, and the first pixel signal W _{+ IR} , It can be determined with higher accuracy that the color processing for reducing the infrared light component IR from _{R + IR} , G _{+ IR} , and B _{+ IR is not performed.}

(Fifth Embodiment)
<Example of application to mobiles>
The technology according to the present disclosure (the present technology) can be applied to various products. For example, the technology according to the present disclosure is realized as a device mounted on a moving body of any kind such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobility, an airplane, a drone, a ship, and a robot. You may.

FIG. 16 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile control system to which the technique according to the present disclosure can be applied.

The vehicle control system 12000 includes a plurality of electronic control units connected via the communication network 12001. In the example shown in FIG. 16, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside information detection unit 12030, an in-vehicle information detection unit 12040, and an integrated control unit 12050. Further, as a functional configuration of the integrated control unit 12050, a microcomputer 12051, an audio image output unit 12052, and an in-vehicle network I / F (interface) 12053 are shown.

The drive system control unit 12010 controls the operation of the device related to the drive system of the vehicle according to various programs. For example, the drive system control unit 12010 provides a driving force generator for generating the driving force of the vehicle such as an internal combustion engine or a driving motor, a driving force transmission mechanism for transmitting the driving force to the wheels, and a steering angle of the vehicle. It functions as a control device such as a steering mechanism for adjusting and a braking device for generating a braking force of a vehicle.

The body system control unit 12020 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 12020 functions as a keyless entry system, a smart key system, a power window device, or a control device for various lamps such as headlamps, back lamps, brake lamps, blinkers or fog lamps. In this case, the body system control unit 12020 may be input with radio waves transmitted from a portable device that substitutes for the key or signals of various switches. The body system control unit 12020 receives inputs of these radio waves or signals and controls a vehicle door lock device, a power window device, a lamp, and the like.

The vehicle exterior information detection unit 12030 detects information outside the vehicle equipped with the vehicle control system 12000. For example, the image pickup unit 12031 is connected to the vehicle exterior information detection unit 12030. The vehicle outside information detection unit 12030 causes the image pickup unit 12031 to capture an image of the outside of the vehicle and receives the captured image. The vehicle exterior information detection unit 12030 may perform object detection processing or distance detection processing such as a person, a vehicle, an obstacle, a sign, or a character on the road surface based on the received image.

The imaging unit 12031 is an optical sensor that receives light and outputs an electric signal according to the amount of the light received. The image pickup unit 12031 can output an electric signal as an image or can output it as distance measurement information. Further, the light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

The vehicle interior information detection unit 12040 detects information in the vehicle. For example, a driver state detection unit 12041 that detects the driver's state is connected to the in-vehicle information detection unit 12040. The driver state detection unit 12041 includes, for example, a camera that images the driver, and the in-vehicle information detection unit 12040 determines the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041. It may be calculated, or it may be determined whether the driver is dozing.

The microcomputer 12051 calculates the control target value of the driving force generator, the steering mechanism, or the braking device based on the information inside and outside the vehicle acquired by the vehicle exterior information detection unit 12030 or the vehicle interior information detection unit 12040, and the drive system control unit. A control command can be output to 12010. For example, the microcomputer 12051 realizes ADAS (Advanced Driver Assistance System) functions including vehicle collision avoidance or impact mitigation, follow-up driving based on inter-vehicle distance, vehicle speed maintenance driving, vehicle collision warning, vehicle lane deviation warning, and the like. It is possible to perform cooperative control for the purpose of.

Further, the microcomputer 12051 controls the driving force generator, the steering mechanism, the braking device, and the like based on the information around the vehicle acquired by the vehicle exterior information detection unit 12030 or the vehicle interior information detection unit 12040. It is possible to perform coordinated control for the purpose of automatic driving, etc., which runs autonomously without depending on the operation.

Further, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information outside the vehicle acquired by the vehicle exterior information detection unit 12030. For example, the microcomputer 12051 controls the headlamps according to the position of the preceding vehicle or the oncoming vehicle detected by the external information detection unit 12030, and performs coordinated control for the purpose of anti-glare such as switching the high beam to the low beam. It can be carried out.

The audio-image output unit 12052 transmits an output signal of at least one of audio and an image to an output device capable of visually or audibly notifying the passenger of the vehicle or the outside of the vehicle. In the example of FIG. 16, an audio speaker 12061, a display unit 12062, and an instrument panel 12063 are exemplified as output devices. The display unit 12062 may include, for example, at least one of an onboard display and a heads-up display.

FIG. 17 is a diagram showing an example of the installation position of the imaging unit 12031.

In FIG. 17, the vehicle 12100 has imaging units 12101, 12102, 12103, 12104, and 12105 as imaging units 12031.

The imaging units 12101, 12102, 12103, 12104, and 12105 are provided at positions such as, for example, the front nose, side mirrors, rear bumpers, back doors, and the upper part of the windshield in the vehicle interior of the vehicle 12100. The image pickup unit 12101 provided on the front nose and the image pickup section 12105 provided on the upper part of the windshield in the vehicle interior mainly acquire an image in front of the vehicle 12100. The imaging units 12102 and 12103 provided in the side mirrors mainly acquire images of the side of the vehicle 12100. The imaging unit 12104 provided on the rear bumper or the back door mainly acquires an image of the rear of the vehicle 12100. The images in front acquired by the imaging units 12101 and 12105 are mainly used for detecting a preceding vehicle or a pedestrian, an obstacle, a traffic light, a traffic sign, a lane, or the like.

Note that FIG. 17 shows an example of the photographing range of the imaging units 12101 to 12104. The imaging range 12111 indicates the imaging range of the imaging unit 12101 provided on the front nose, the imaging ranges 12112 and 12113 indicate the imaging ranges of the imaging units 12102 and 12103 provided on the side mirrors, respectively, and the imaging range 12114 indicates the imaging range of the imaging units 12102 and 12103. The imaging range of the imaging unit 12104 provided on the rear bumper or the back door is shown. For example, by superimposing the image data captured by the imaging units 12101 to 12104, a bird's-eye view image of the vehicle 12100 as viewed from above can be obtained.

At least one of the imaging units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the image pickup units 12101 to 12104 may be a stereo camera composed of a plurality of image pickup elements, or may be an image pickup element having pixels for phase difference detection.

For example, the microcomputer 12051 has a distance to each three-dimensional object within the imaging range 12111 to 12114 based on the distance information obtained from the imaging units 12101 to 12104, and a temporal change of this distance (relative velocity with respect to the vehicle 12100). By obtaining can. Further, the microcomputer 12051 can set an inter-vehicle distance to be secured in front of the preceding vehicle in advance, and can perform automatic brake control (including follow-up stop control), automatic acceleration control (including follow-up start control), and the like. In this way, it is possible to perform coordinated control for the purpose of automatic driving or the like in which the vehicle travels autonomously without depending on the operation of the driver.

For example, the microcomputer 12051 converts three-dimensional object data related to a three-dimensional object into two-wheeled vehicles, ordinary vehicles, large vehicles, pedestrians, electric poles, and other three-dimensional objects based on the distance information obtained from the imaging units 12101 to 12104. It can be classified and extracted and used for automatic avoidance of obstacles. For example, the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into obstacles that can be seen by the driver of the vehicle 12100 and obstacles that are difficult to see. Then, the microcomputer 12051 determines the collision risk indicating the risk of collision with each obstacle, and when the collision risk is equal to or higher than the set value and there is a possibility of collision, the microphone 12061 or the display unit 12062 is used. By outputting an alarm to the driver and performing forced deceleration and avoidance steering via the drive system control unit 12010, driving support for collision avoidance can be provided.

At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether or not a pedestrian is present in the captured image of the imaging units 12101 to 12104. Such pedestrian recognition includes, for example, a procedure for extracting feature points in an image captured by an imaging unit 12101 to 12104 as an infrared camera, and pattern matching processing for a series of feature points indicating the outline of an object to determine whether or not the pedestrian is a pedestrian. It is done by the procedure to determine. When the microcomputer 12051 determines that a pedestrian is present in the captured images of the imaging units 12101 to 12104 and recognizes the pedestrian, the audio image output unit 12052 outputs a square contour line for emphasizing the recognized pedestrian. The display unit 12062 is controlled so as to superimpose and display. Further, the audio image output unit 12052 may control the display unit 12062 so as to display an icon or the like indicating a pedestrian at a desired position.

The example of the vehicle control system to which the technique according to the present disclosure can be applied has been described above. The technique according to the present disclosure can be applied to the imaging unit 12031 and the like among the configurations described above. Specifically, the imaging device 10 can be applied to the imaging unit 12031. Imaging Unit 12031 By applying the technique according to the present disclosure, it is possible to obtain a photographed image that is easier to see even when illuminated by a specific light source, so that driver fatigue can be reduced.

The present technology can have the following configurations.

(1) An imaging unit that converts incident light from an imaging target into a first pixel signal containing an infrared light component and a visible light component and outputs the signal.
When the light source that irradiates the image pickup target with light is a predetermined light source, a predetermined process is performed using the first pixel signal, and when the light source is not a predetermined light source, the red from the first pixel signal. An imaging device including a color processing unit that performs predetermined processing using a second pixel signal with a reduced external light component.

(2) A determination processing unit for determining a light source that irradiates the image pickup target with light based on the information on the infrared light component is further provided.
Based on the determination information of the determination processing unit, the color processing unit changes the processing when the light source that irradiates the image pickup target with light is a predetermined light source and the processing when the light source is not a predetermined light source. , (1).

(3) The imaging apparatus according to (2), wherein the determination processing unit determines that the infrared light component is a predetermined light source when it is determined that the infrared light component is less than a predetermined value.

(4) The imaging according to any one of (1) to (3), wherein the color processing unit converts the first pixel signal into a Bayer array when the light source is a predetermined light source. Device.

(5) The image pickup apparatus according to any one of (1) to (4), wherein the color processing unit converts the light source into a Bayer array using the second pixel signal when the light source is not a predetermined light source. ..

(6) Further provided with a separation processing unit that separates the first pixel signal into the infrared light component and the visible light component.
The imaging apparatus according to any one of (2) to (5), wherein the determination processing unit determines the light source by using the infrared light component separated by the separation processing unit.

(7) The imaging device according to (2), wherein the determination processing unit determines a light source that irradiates the imaging target with light using the visible light component.

(8) The image pickup apparatus according to (2), wherein the determination processing unit calculates a color temperature using the visible light component and determines the light source based on the color temperature.

(9) The visible light component is a color component of each of red light, blue light, and green light, and the determination processing unit is the light source based on the ratio of each color component of red light, blue light, and green light. The imaging device according to (7) or (8).

(10) The imaging device according to (8) or (9), wherein the determination processing unit determines a light source using information on the infrared light component and the color temperature.

(11) The imaging according to (10), wherein the determination processing unit determines that the light source is a predetermined light source when the color temperature is in a predetermined range and the infrared light component is less than a predetermined value. Device.

(12) The determination processing unit determines the light source based on at least one of information regarding the sunrise time and the sunset time, the position information of the light source that irradiates the imaging target with light, and the information regarding the illuminance of the headlight. The imaging device according to (2).

(13) An imaging unit that converts incident light from an imaging target into a first pixel signal containing an infrared light component and a visible light component and outputs the signal.
A determination processing unit that determines whether or not to perform color processing for reducing the infrared light component from the first pixel signal based on the visible light component.
A color processing unit that performs color processing for reducing the infrared light component from the first pixel signal when it is determined to perform color processing for reducing the infrared light component.
An imaging device comprising.

(14) The image pickup apparatus according to (13), wherein the determination processing unit makes the determination based on the color temperature calculated using the visible light component.

(15) The visible light component is a color component of each of red light, blue light, and green light, and the determination processing unit makes the determination based on the ratio of each color component of red light, blue light, and green light. The imaging apparatus according to (13) or (14).

(16) The image pickup apparatus according to (14) or (15), wherein the determination processing unit makes the determination using information on the infrared light component and the color temperature.

(17) The determination processing unit performs color processing for reducing the infrared light component from the first pixel signal when the color temperature is in a predetermined range and the infrared light component is less than a predetermined value. The imaging apparatus according to (16), wherein it is determined not to perform the above.

(18) An imaging unit that converts incident light from an imaging target into a first pixel signal containing an infrared light component and a visible light component and outputs the signal.
A color that reduces the infrared light component from the first pixel signal based on at least one of information about the sunrise time and the sunset time, the position information of the light source that irradiates the imaging target with light, and the information about the illuminance of the headlight. A judgment processing unit that determines whether or not to perform processing,
A color processing unit that performs color processing for reducing the infrared light component from the first pixel signal when it is determined to perform color processing for reducing the infrared light component.
An imaging device comprising.

(19) The judgment processing unit
When it is determined that the tunnel is inside the tunnel based on the position information, it is determined that the color processing for reducing the infrared light component from the first pixel signal is not performed.
Or
If it is determined that the tunnel is outside the tunnel based on the position information and the current time is after the sunrise time and before the sunset time, it is determined that the color processing for reducing the infrared light component is performed.
Or
The imaging apparatus according to (18), wherein when the illuminance of the headlight is equal to or higher than a predetermined threshold value, it is determined that color processing for reducing the infrared light component is performed from the first pixel signal.

(20) An imaging process in which incident light from an imaging target is converted into a first pixel signal containing an infrared light component and a visible light component and output.
When the light source that irradiates the image pickup target with light is a predetermined light source, a predetermined process is performed using the first pixel signal, and when the light source is not a predetermined light source, the red from the first pixel signal. A color processing step of performing a predetermined process using a second pixel signal with a reduced external light component, and
An imaging method.

The aspects of the present disclosure are not limited to the individual embodiments described above, but also include various modifications that can be conceived by those skilled in the art, and the effects of the present disclosure are not limited to the contents described above. That is, various additions, changes and partial deletions are possible without departing from the conceptual idea and purpose of the present disclosure derived from the contents defined in the claims and their equivalents.

10: Image pickup device, 23: Image sensor, 102: Judgment processing unit, 104: Color processing unit.

Claims (20)

An imaging unit that converts the incident light from the imaging target into a first pixel signal containing an infrared light component and a visible light component and outputs it.
When the light source that irradiates the image pickup target with light is a predetermined light source, a predetermined process is performed using the first pixel signal, and when the light source is not a predetermined light source, the red from the first pixel signal. A color processing unit that performs predetermined processing using a second pixel signal with a reduced external light component,
An imaging device. Further, a determination processing unit for determining a light source that irradiates the image pickup target with light based on the information on the infrared light component is provided.
Based on the determination information of the determination processing unit, the color processing unit changes the processing when the light source that irradiates the image pickup target with light is a predetermined light source and the processing when the light source is not a predetermined light source. , The imaging apparatus according to claim 1. The imaging device according to claim 2, wherein the determination processing unit determines that the infrared light component is a predetermined light source when it is determined that the infrared light component is less than a predetermined value. The imaging device according to claim 1, wherein the color processing unit converts the first pixel signal into a Bayer array when the light source is a predetermined light source. The imaging device according to claim 1, wherein the color processing unit converts the second pixel signal into a Bayer array when the light source is not a predetermined light source. A separation processing unit that separates the first pixel signal into the infrared light component and the visible light component is further provided.
The imaging apparatus according to claim 2, wherein the determination processing unit determines the light source by using the infrared light component separated by the separation processing unit. The imaging device according to claim 2, wherein the determination processing unit determines a light source that irradiates the imaging target with light using the visible light component. The imaging device according to claim 2, wherein the determination processing unit calculates a color temperature using the visible light component and determines the light source based on the color temperature. The visible light component is a color component of each of red light, blue light, and green light, and the determination processing unit determines the light source based on the ratio of each color component of red light, blue light, and green light. The imaging device according to claim 7. The imaging device according to claim 8, wherein the determination processing unit determines a light source using information on the infrared light component and the color temperature. The imaging device according to claim 10, wherein the determination processing unit determines that the light source is a predetermined light source when the color temperature is in a predetermined range and the infrared light component is less than a predetermined value. 2. The determination processing unit determines the light source based on at least one of information regarding the sunrise time and the sunset time, the position information of the light source that irradiates the imaging target with light, and the information regarding the illuminance of the headlight. The imaging apparatus according to. An imaging unit that converts the incident light from the imaging target into a first pixel signal containing an infrared light component and a visible light component and outputs it.
A determination processing unit that determines whether or not to perform color processing for reducing the infrared light component from the first pixel signal based on the visible light component.
A color processing unit that performs color processing for reducing the infrared light component from the first pixel signal when it is determined to perform color processing for reducing the infrared light component.
An imaging device comprising. The imaging device according to claim 13, wherein the determination processing unit makes the determination based on the color temperature calculated by using the visible light component. The visible light component is a color component of each of red light, blue light, and green light, and the determination processing unit makes the determination based on the ratio of each color component of red light, blue light, and green light. The imaging device according to claim 13. The imaging device according to claim 14, wherein the determination processing unit makes the determination using information on the infrared light component and the color temperature. The determination processing unit does not perform color processing for reducing the infrared light component from the first pixel signal when the color temperature is in a predetermined range and the infrared light component is less than a predetermined value. 16. The imaging device according to claim 16. An imaging unit that converts the incident light from the imaging target into a first pixel signal containing an infrared light component and a visible light component and outputs it.
A color that reduces the infrared light component from the first pixel signal based on at least one of information about the sunrise time and the sunset time, the position information of the light source that irradiates the imaging target with light, and the information about the illuminance of the headlight. A judgment processing unit that determines whether or not to perform processing,
A color processing unit that performs color processing for reducing the infrared light component from the first pixel signal when it is determined to perform color processing for reducing the infrared light component.
An imaging device comprising. The judgment processing unit
When it is determined that the tunnel is inside the tunnel based on the position information, it is determined that the color processing for reducing the infrared light component from the first pixel signal is not performed.
Or
If it is determined that the tunnel is outside the tunnel based on the position information and the current time is after the sunrise time and before the sunset time, it is determined that the color processing for reducing the infrared light component is performed.
Or
The imaging device according to claim 18, wherein when the illuminance of the headlight is equal to or higher than a predetermined threshold value, it is determined that color processing for reducing the infrared light component is performed from the first pixel signal. An imaging process in which incident light from an imaging target is converted into a first pixel signal containing an infrared light component and a visible light component and output.
When the light source that irradiates the image pickup target with light is a predetermined light source, a predetermined process is performed using the first pixel signal, and when the light source is not a predetermined light source, the red from the first pixel signal. A color processing step in which a predetermined process is performed using a second pixel signal with a reduced external light component, and
An imaging method.

Images