WO2013118337A1

WO2013118337A1 - Imaging device

Info

Publication number: WO2013118337A1
Application number: PCT/JP2012/071854
Authority: WO
Inventors: 西澤　明仁
Original assignee: 日立コンシューマエレクトロニクス株式会社
Priority date: 2012-02-06
Filing date: 2012-08-29
Publication date: 2013-08-15
Also published as: JP2013162339A

Abstract

There is a strong demand for imaging devices, including surveillance cameras and car-mounted cameras, that are capable of capturing images of target subjects clearly even after dark. In the present invention, an image with a good S/N ratio can be obtained when producing colors even under a condition wherein fewer signals of a visible-light area are obtained due to darkness. A luminance signal and a chrominance signal are generated from visible light pixel signals and near-infrared light pixel signals, a brightness level of a subject is determined, the proportion of the near-infrared pixel signals in the generated luminance signal is controlled according to the determination result, and a chrominance signal is generated by emphasizing or substituting a specific color or a shape and object color according to the luminance signal.

Description

Imaging device

The present invention relates to an imaging apparatus.

There is Patent Document 1 as background art in this technical field. In the summary of this publication, the problem is described as “achieving good color reproducibility without providing an infrared removing filter for color vision correction in the imaging device”, and “red, green, blue of the red, green, and blue” as a solution. First, second, and third color filters that extract light, and a fourth color filter that extracts only light in the near-infrared region, and a first corresponding to the spectral transmittance of each color filter. Color signal generation means (5) for outputting the first to fourth signals (R1b, Gb, Bb, R2b), and the red, green, and blue color signals (Re, Ge, Be) from the first to fourth signals. The signal processing means (4) includes spectral sensitivity correction means (8) that corrects response characteristics in the near-infrared region of the color signal generation means (5). . "

Moreover, there is Patent Document 2 as another background technology in this technical field. In the summary of the publication, as a problem, “conventionally, since a color image is generated based only on a visible light pixel signal, the sensitivity of the image sensor is insufficient at night, and a good color image can be obtained. As a solution, “a plurality of pixels are provided with an image sensor 2 having sensitivity in the wavelength region of visible light and near infrared light, and luminance is obtained from the obtained visible light pixel signal and near infrared light pixel signal”. An imaging device that generates a signal and a color signal, adds a visible light / near infrared light dominant determination unit 4c, determines the magnitude relationship between the visible light pixel signal and the near infrared light pixel signal, and the determination result The luminance signal generation method is switched by controlling the YC processing unit 4a according to the generation of the luminance signal and the color signal using the visible light pixel signal in the daytime, and when there is little visible light such as at night, the luminance signal is Generated using near infrared pixel signal In the darker case, a color signal is generated using a visible light pixel signal that has undergone pixel mixing or long exposure, and as a result, good color photography can be performed to improve the visibility of the user. . "

JP 2005-341470 A JP 2010-93472 A

There are surveillance cameras and in-vehicle cameras as main applications of the imaging apparatus. In both cases, there is a strong demand for clear shooting of a target subject even when it gets dark. For this problem, the patent document describes an improvement method by utilizing not only the light in the visible light region but also the near infrared region. Further, since the color reproducibility is impaired by utilizing the near infrared region, a method for correcting the responsiveness of the color signal in the near infrared region is also described. However, when a color signal is generated in a state where the signal in the visible light region is reduced due to darkness, it is necessary to amplify a signal having a small visible light region in principle, so that the image has a lot of noise and a poor S / N. It was not taken into account.
Accordingly, an object of the present invention is to provide an imaging apparatus that can suppress S / N degradation while utilizing the near infrared region.

In order to solve the above object, the configuration described in the claims is adopted.

According to the present invention, it is possible to provide an imaging device that suppresses S / N degradation and realizes a color image with good visibility even when the shooting environment becomes dark at dusk.

It is a 1st block diagram of the imaging device in one Example. It is a 2nd block diagram of the imaging device in one Example. It is a 3rd block diagram of the imaging device in one Example. It is a 4th block diagram of the imaging device in one Example. It is a figure which shows the spectral characteristic in one Example. It is a circuit diagram of the IR component removal circuit in one Example. It is the 1st circuit diagram of the substitution circuit in one example. It is a figure which shows the control characteristic which concerns on the brightness in one Example. It is a figure which shows the control characteristic based on the color extraction result in one Example. It is a 2nd circuit diagram of the replacement circuit in one Example. It is a figure which shows the control characteristic which concerns on the multiplication result in one Example. It is a 3rd circuit diagram of the replacement circuit in one Example.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a first block diagram of an imaging apparatus according to an embodiment. In this embodiment, the image sensor 10, the interpolation synchronization circuit 11, the luminance matrix circuit 121, the IR component adjustment circuit 122, and the gamma adjustment circuit 123, the luminance signal generation circuit 12, the comparison circuit 16, the specific color extraction circuit 15, the IR It is composed of a color signal generation circuit 13 comprising a component removal circuit 131, a WB adjustment circuit 132, a gamma adjustment circuit 133 and a matrix circuit 134, and a replacement circuit 18, and red and near-infrared above the photoelectric conversion element constituting the image sensor 10. "R + IR" filter that passes components, "G + IR" filter that passes green and near infrared components, "IR" filter that passes near infrared components, and blue and near infrared components pass The “B + IR” filter is arranged in a checkered pattern to obtain four types of color signals.

FIG. 5 is a diagram showing spectral characteristics in one embodiment. The four types of filters are “R + IR” filter, “G + IR” filter, “IR” filter, and “B + IR”, as described in (a) Spectral sensitivity of image sensor in the spectral characteristics of FIG. A filter that is sensitive to wavelengths such as 104, 103, 105, 102 for each of the filters. In this embodiment, the above filter configuration is used. However, any filter other than the above configuration may be used as long as it is a filter that can separate a visible light component and a near infrared component and can produce a color image. For example, instead of the near-infrared IR filter, a filter having sensitivity in the entire region from blue to near-infrared may be used, and an “R” filter, “G” filter, “IR” filter, and “B” filter may be used. .

In the present embodiment, the light separated into four colors by the color filter 101 is photoelectrically converted by the image sensor 10 and sent to the interpolation synchronization circuit 11, and the interpolation synchronization circuit 11 “R + IR”, “G + IR”, “IR”, and “B + IR” signals are generated by interpolation processing for all pixel positions. The signals “R + IR”, “G + IR”, and “B + IR” generated by the interpolation process are sent to the luminance matrix circuit 121, and the luminance matrix circuit 121 performs a predetermined ratio. The mixed signal is sent to the comparison circuit 16, the IR component adjustment circuit 122 and the specific color extraction circuit 15, and the “IR” signal generated by the interpolation synchronization circuit 11 is sent to the comparison circuit 16 and the IR component adjustment circuit 122. ing.
The IR component adjustment circuit 122 generates a luminance signal including the near-infrared component according to the comparison result of the luminance signal including the near-infrared component generated by the luminance matrix circuit 121 and the near-infrared signal by the comparison circuit 16. The near-infrared component is controlled and sent to the gamma adjustment circuit 123 to correct the gamma characteristic and output to the output terminal 124 as a luminance signal.

FIG. 6 is a circuit diagram of the IR component removal circuit 131 in one embodiment. The “R + IR”, “G + IR”, “IR”, and “B + IR” signals generated by the interpolation processing are sent to the IR component removal circuit 131 and subjected to the subtraction processing shown in FIG. The near-infrared component is removed from each of the signals “R + IR”, “G + IR”, and “B + IR”, and the

spectral sensitivities

137, 136 after IR component removal shown in FIG. “R signal”, “G signal”, and “B signal” having the characteristics described in 135 are generated. The “R signal”, “G signal”, and “B signal” generated by the IR component removal circuit 131 are sent to the WB (White Balance) adjustment circuit 132, and the “R signal”, “G signal” are selected according to the characteristics of the light source. The signal gains of “signal” and “B signal” are respectively controlled and sent to the specific color extraction circuit 15 and the gamma adjustment circuit 133. The gamma adjustment circuit 133 performs gamma correction and sends it to the matrix circuit 134 to generate color difference signals “RY” and “BY” from the gamma-corrected “R signal”, “G signal”, and “B signal”, and a replacement circuit. It is sent to 18. The replacement circuit 18 replaces or emphasizes a part of the input color difference signals “RY” and “BY” with another signal according to the outputs from the comparison circuit 16 and the specific color extraction circuit 15 and outputs it as a color signal. It outputs to 186.

In the present embodiment, the IR component adjustment circuit 122 controls the near-infrared component of the luminance signal including the near-infrared component according to the brightness. As it is generated and darkens, it gradually operates using signals in the near-infrared region to produce a luminance signal. For this reason, a luminance signal with a certain level of brightness can be created even when the shooting environment becomes dark. Further, the output of the color signal generation circuit 13, that is, the output 184 of the matrix circuit 134, is generated only by a visible light region signal that is meaningful as a color signal, with the IR component removal circuit 131 removing the near infrared region signal. Therefore, the color reproducibility is not deteriorated by the near-infrared signal. Next, an embodiment of the replacement circuit 18 will be described with reference to FIGS.

FIG. 7 is a first circuit diagram of the replacement circuit 18 in one embodiment. An adder circuit 188, a subtractor circuit 187, and

multiplier circuits

186 and 189 are configured. The output signal 181 of the comparison circuit 16 and the output signal 182 of the specific color extraction circuit 15 are multiplied by the multiplier circuit 189 and set in advance. The output signal 184 of the matrix circuit 134 is subtracted from the set value L2 which is a predetermined signal level by the subtraction circuit 187, and the multiplication result of the multiplication circuit 189 and the subtraction result of the subtraction circuit 187 are multiplied by the multiplication circuit 186. The output of the circuit 186 and the output signal 184 of the matrix circuit 134 are added by the adder circuit 188, and an output signal 185 is output.

FIG. 8 is a diagram illustrating control characteristics relating to brightness in one embodiment.
FIG. 9 is a diagram illustrating control characteristics according to the color extraction result in one embodiment.
FIGS. 8 and 9 show examples of characteristics of the output signal 181 of the comparison circuit 16 and the output signal 182 of the specific color extraction circuit 15. Although not shown, the L2 is given by a control means such as a microcomputer or a predetermined circuit.

8, the output value of the output signal 181 increases as it becomes darker, and the output value of the output signal 182 increases as the characteristic of FIG. 9 approaches the specified specific color. Since the output 185 of the replacement circuit 18 is an output obtained by mixing the output signal 184 of the matrix circuit 134 and the set value L2 at a predetermined ratio, the darker the color and the closer to the specified specific color, the more the output 185 is obtained from the signal 184. Also, the replacement circuit 18 operates so that the value of the set value L2 becomes dominant. For this reason, as the color becomes darker, the color of the target subject portion such as a sign, tail lamp, signal, etc. is gradually replaced with a preset signal having a good S / N. It is possible to add a color while suppressing deterioration, and to display a target subject portion with a color having a good S / N.

When the image is made darker and the visible light component is reduced and the near-infrared component is used to create a video signal, a luminance signal with a constant signal level can be obtained from the near-infrared component, but only from the visible light component. The color of the produced color is reduced and the overall color is light. Even in this case, since the S / N can be displayed well with a hue and color density similar to or very close to that of a target subject such as a sign, tail lamp, signal, etc. It is possible to provide an object to be noticed with color images in a conspicuous form. In addition, since the target portion other than the target portion to be noticed is very light colored or black and white, the subject portion to be noticed can be particularly emphasized and displayed.

FIG. 10 is a second circuit diagram of the replacement circuit 18 in one embodiment. The signal enhancement circuit 200 and the multiplication circuit 202 are configured, the output signal 181 of the comparison circuit 16 and the output signal 182 of the specific color extraction circuit 15 are multiplied by the multiplication circuit 202, and the matrix is determined according to the multiplication result 201 of the multiplication circuit 202. The signal enhancement circuit 200 operates so as to perform signal enhancement that increases the signal gain of the output signal 184 of the circuit 134. The characteristics of the output signal 181 of the comparison circuit 16 and the output signal 182 of the specific color extraction circuit 15 may be the same as in the embodiment of FIG.
FIG. 11 is a diagram illustrating the control characteristics according to the multiplication result 201 in one embodiment, and illustrates the control characteristics in the signal enhancement circuit 200. The signal enhancement circuit 200 operates so as to decrease the signal enhancement amount when the signal of the multiplication result 201 is small and large as in the control characteristic, and to increase the signal enhancement amount near the center of the signal of the multiplication result 201.

In the present embodiment, the enhancement of the color signal is suppressed by the signal enhancement circuit 200 when it is bright and very dark, and the gain of the color signal in the specific color portion is set according to the result of the specific color extraction circuit 15 only in the case of the darkness between them. Works to raise. For this reason, in the case of dim light, the gain of the color signal can be increased only in the target subject portion such as a sign, a tail lamp, or a signal to enhance the signal. When the bright and sufficient visible light component is sufficient and good color reproduction is obtained, and when the visible light component is hardly obtained because it is too dark, S / N deterioration can be suppressed to suppress enhancement of the color signal. In such a dim environment, only the target subject portion such as a sign, tail lamp, signal, etc., can display the S / N with the same or similar color and darkness as when bright, so the car is driven. It is possible to provide a subject that should be noticeable in a form that is conspicuous to a person in color video. It is very effective in preventing automobile accidents by preventing oversight of subjects to watch out for at dusk and nighttime busy streets where accidents are likely to occur.

FIG. 12 is a third circuit diagram of the replacement circuit 18 in one embodiment. The configuration is such that the set value L2 portion in FIG. 7 is replaced with a signal that has been subjected to enhancement processing that darkens all input signals in advance by the signal enhancement circuit 210 with respect to the output signal 184 of the matrix circuit 134. ing. In this embodiment, the gain for the difference signal between the output signal 184 of the matrix circuit 134 and the output signal of the signal enhancement circuit 210 is increased according to the results of the output 181 of the comparison circuit 16 and the output 182 of the specific color extraction circuit 15, The permutation circuit 18 operates so as to mix the signal having a higher density with the output signal 184 of the matrix circuit 134 at a predetermined ratio. Therefore, the darker and closer to the specified color, the higher the percentage of the signal that increases the gain in the output and the darker the color, and the gain of the color signal increases only for the target subject part such as a sign, tail lamp or signal. Emphasize the signal. For this reason, the S / N can be displayed well with a hue and color density that is similar to or close to that of a target subject such as a sign, tail lamp, signal, and the like while suppressing deterioration of the S / N. It is possible to provide a subject that should be noticed in a form that is conspicuous to the driving person in color video.

S / N can be shown well in shades and color depths that are similar to or very similar to those of bright subjects at dusk and nighttime busy streets where accidents are likely to occur. It is valid. The video display for the driver can be installed in a part of the driver's front field of view, for example, in the windshield or in the vicinity of the dashboard. .

FIG. 2 is a second block diagram of the imaging apparatus according to the embodiment. This embodiment is an example in which the specific color extraction circuit 15 is changed to the specific shape extraction circuit 17 in the embodiment of FIG. 1, and in the case of FIG. 1, a specific color portion is prepared in advance when dark. It is replaced with color data or operates so that the signal gain in the specific color portion is controlled. In this embodiment, when dark, the specific shape portion is replaced with color data prepared in advance. Alternatively, the signal gain in the part having the specific shape is controlled. For example, the specific shape is a sign, a signal, or the like, which is particularly important for a driver driving a car. In this way, in a busy downtown area at dusk or at night, an object with a specific shape can be projected well in color, so it is very effective in preventing automobile accidents. The video display for the driver can be installed in a part of the driver's front field of view, for example, in the windshield or in the vicinity of the dashboard. . Since this embodiment emphasizes the color of an object having a specific shape, it has an effect of preventing an accident due to oversight of a sign or the like.

FIG. 3 is a third block diagram of the imaging apparatus according to an embodiment. In this embodiment, the specific color extraction circuit 15, the specific color extraction circuit 15, the specific shape extraction circuit 17, and the outputs of the specific color extraction circuit 15 and the specific shape extraction circuit 17 in the embodiment of FIG. In this example, the specific subject extraction circuit 19 for determining a specific subject is used. In the case of FIG. 1, when it becomes dark, the specific color portion is replaced with color data prepared in advance, or the signal gain in the specific color portion is controlled, but in this embodiment, Compared to Example 1, a specific color portion and a specific shape portion are further extracted, a specific subject is extracted from the result, and when the specific subject portion becomes dark, it is replaced with color data prepared in advance, Alternatively, the signal gain of the portion of the specific subject is controlled. In this embodiment, substantially the same effect as that of the embodiment of FIGS. 1 and 2 can be obtained.

In particular, in this embodiment, the subject is determined from both the color and the shape and the color is replaced or the portion to be emphasized is determined. This has the effect of reducing false detections in which the emphasized part is mistaken. Note that the color data to be replaced may be the same color as the actual object, or may be a different color for emphasis. In this embodiment, since a specific subject is judged and emphasized, not only the signs and signals but also the color can be replaced or emphasized without overlooking a human or a specific moving object. It has the effect of preventing accidents.

FIG. 4 is a fourth block diagram of the imaging apparatus according to the embodiment. This embodiment is an example in which the comparison signal of the comparison circuit 16 is changed in the embodiment of FIG. In FIG. 1, FIG. 2 and FIG. 3, the brightness is judged by comparing a signal obtained by mixing “R + IR”, “G + IR”, and “B + IR” at a predetermined ratio with an “IR” signal. In the example, although not shown, the brightness is compared with a set value L1 set by a control means such as a microcomputer or a circuit and a signal obtained by mixing “R + IR”, “G + IR”, and “B + IR” at a predetermined ratio. This is an example of determining. A signal obtained by mixing “R”, “G”, and “B” not including the IR component at a predetermined ratio is compared with the set value L1, or “R”, “G”, and “B” not including the IR component are compared. The brightness may be judged by comparing a signal obtained by mixing "" at a predetermined ratio with an "IR" signal. This embodiment can obtain substantially the same effect as the embodiment of FIG. Further, the same effect can be obtained by combining with the embodiments of FIGS.

In addition, this invention is not limited to the above-mentioned Example, Various modifications are included. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described. In addition, a part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of one embodiment. In addition, it is possible to add, delete, or replace another configuration with respect to a part of the configuration of each embodiment.

In addition, each of the above-described configurations may be configured such that a part or all of the configuration is configured by hardware, or is realized by executing a program by a processor. Further, the control lines and information lines indicate what is considered necessary for the explanation, and not all the control lines and information lines on the product are necessarily shown. Actually, it may be considered that almost all the components are connected to each other.

10: Image sensor, 11: Interpolation synchronization circuit, 12: Luminance signal generation circuit, 13: Color signal generation circuit, 15: Specific color extraction circuit, 16: Comparison circuit, 17: Specific shape extraction circuit, 18: Replacement circuit , 19: specific subject extraction circuit, 121: luminance matrix circuit, 122: IR component adjustment circuit, 123: gamma adjustment circuit, 131: IR component removal circuit, 132: WB adjustment circuit, 133: gamma adjustment circuit, 134: matrix circuit .

Claims

An imaging device for imaging an object,
An imaging element having a plurality of photoelectric conversion elements capable of imaging the object in both visible and near-infrared wavelength regions;
A comparison / determination unit that determines brightness based on a visible light pixel signal supplied from the imaging element, or a visible light pixel signal and a near-infrared light pixel signal;
A luminance signal generation unit that generates and outputs a luminance signal from the visible light pixel signal and the near-infrared light pixel signal supplied from the imaging element based on a determination result in the comparison determination unit;
A color signal generation unit that generates a color signal from the visible light pixel signal and the near-infrared light pixel signal supplied from the imaging element;
A specific color extraction unit that extracts a specific color of the object from the color signal generated by the color signal generation unit;
The color signal generated by the color signal generation unit based on the determination result in the comparison determination unit and the detection result of the specific color in the specific color extraction unit is replaced with a value related to a predetermined value, or the color signal is An imaging apparatus comprising: a replacement / emphasis unit that emphasizes and outputs.
An imaging device for imaging an object,
An imaging element having a plurality of photoelectric conversion elements capable of imaging the object in both visible and near-infrared wavelength regions;
A comparison / determination unit that determines brightness based on a visible light pixel signal supplied from the imaging element, or a visible light pixel signal and a near-infrared light pixel signal;
A luminance signal generation unit that generates and outputs a luminance signal from the visible light pixel signal and the near-infrared light pixel signal supplied from the imaging element based on a determination result in the comparison determination unit;
A color signal generation unit that generates a color signal from the visible light pixel signal and the near-infrared light pixel signal supplied from the imaging element;
A specific shape extraction unit that extracts a specific shape of the object from the color signal generated by the color signal generation unit and / or the luminance signal generated by the luminance signal generation unit;
Based on the determination result in the comparison determination unit and the detection result of the specific shape in the specific shape extraction unit, the color signal generated in the color signal generation unit is replaced with a value related to a predetermined value, or the color signal is emphasized And a replacement / emphasis unit for outputting the image.
The imaging device according to claim 2,
A specific color extraction unit that extracts a specific color of the object from the color signal generated by the color signal generation unit;
The replacement / emphasis unit is the color signal generation unit based on the determination result in the comparison determination unit, the detection result of the specific color in the specific color extraction unit, and the detection result of the specific shape in the specific shape extraction unit. An imaging apparatus, wherein the generated color signal is replaced with a value related to a predetermined value, or the color signal is emphasized and output.
2. The imaging apparatus according to claim 1, wherein the comparison determination unit determines brightness based on a magnitude relationship between the visible light pixel signal and the near-infrared light pixel signal.
3. The imaging apparatus according to claim 2, wherein the comparison and determination unit determines brightness based on a magnitude relationship between the visible light pixel signal and the near-infrared light pixel signal.
2. The imaging apparatus according to claim 1, wherein the comparison determination unit determines brightness by comparing the magnitude of the visible light pixel signal with a predetermined value.
3. The imaging apparatus according to claim 2, wherein the comparison determination unit determines brightness by comparing the magnitude of the visible light pixel signal with a predetermined value.
2. The imaging apparatus according to claim 1, further comprising a projection unit that projects an image captured when the imaging apparatus is mounted on a vehicle onto a windshield of the vehicle.
3. The imaging apparatus according to claim 2, further comprising a projection unit that projects an image captured when the imaging apparatus is mounted on a vehicle onto a windshield of the vehicle.
The imaging apparatus according to claim 1, further comprising a display unit that displays an image captured when the imaging apparatus is mounted on a vehicle between a windshield of the vehicle and a driver. apparatus.
The imaging apparatus according to claim 2, further comprising a display unit that displays an image captured when the imaging apparatus is mounted on a vehicle between a windshield of the vehicle and a driver. apparatus.