JP4239738B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus, and more particularly, to provide infrared irradiation and ultraviolet irradiation to a subject without using an infrared cut filter and an ultraviolet cut filter in an optical path by using an image sensor having sensitivity in an infrared region and an ultraviolet region. The present invention relates to an imaging device that outputs a color image even in the dark.

Some imaging devices in the prior art shoot a subject by removing infrared cut filters and irradiating infrared rays when shooting in a night vision state. For example, it is disclosed in Japanese Patent Application Laid-Open No. 11-103464, and this is clear when an image is taken without an infrared cut filter disposed in front of the lens and an infrared light emitting system is added, even when it is dark. Focusing on the point that an image can be recorded, the subject is irradiated with infrared rays.
As shown in FIG. 10, this configuration converges with a lens system 11 that converges image light from a subject, an infrared cut filter 12 that is controlled by a microcomputer 23 and moves to a position off the optical path, and a lens system 11. A CCD 14 for converting an image into an electric signal, an S / H & AGC and A / D converter 15 for A / D conversion and sampling / gain control of the electric signal converted by the CCD 14, and black balance correction of the digital signal. A clamp circuit 16, a white balance correction circuit 17 that performs white balance correction, a gamma correction circuit 18 that forms color difference signals, a signal processing circuit 19, a color reproduction matrix 20 that is optimized based on matrix coefficients, and an A / A specific color extraction control function that detects the darkest point from the D-converted image signal and corrects the black balance. Control of micro-infrared radiation, moving controls the infrared cutoff filter 12 at a position off the optical path, and is largely constituted by a microcomputer 23 for controlling the like of the white balance.

  The operation of the imaging apparatus having such a configuration will be described below with reference to the flowchart shown in FIG.

First, when the mode is not the night vision mode, the normal shooting mode is processed and the control values are set (steps ST21, ST22, ST23).
In step ST21, when the night vision mode is selected, the infrared cut filter 12 is released and the subject is irradiated with infrared rays (steps ST24 and ST25).
Then, the subject irradiated with infrared rays is imaged, white balance control is set to a fixed value, and control values are set (steps ST26 and ST23).

  In such an apparatus, when a subject is photographed by irradiating infrared light in a night vision state, control is performed such that the white balance control value is fixed, for example, the entire screen becomes green.

Japanese Patent Laid-Open No. 2001-36916 discloses a method for outputting a color image when a subject is photographed in a night vision state. This is to reproduce colors from a look-up table by performing infrared irradiation and ultraviolet irradiation.
Japanese Patent Application Laid-Open No. 11-103464 (page 3, FIG. 1) Japanese Patent Laid-Open No. 2001-36916 (page 3, Fig. 1)

However, what was described in the prior art is heavy and complicated because the colors are reproduced from the lookup table in two images by switching between infrared irradiation and ultraviolet irradiation, and white balance control and color reproduction matrix control are also performed. In addition, since black balance control is not mentioned, there is a problem that color reproducibility is low.
Therefore, the present patent aims to solve these problems and provide an image pickup apparatus capable of outputting a color image with good color reproducibility even when shooting in a night vision state.

  In order to solve the above problems, an imaging apparatus according to the present invention is configured as follows.

(1) The imaging device
It has an image sensor with sensitivity in the infrared region and ultraviolet region,
The color is determined from the imaging signal obtained by photographing the subject by moving the infrared cut filter and ultraviolet cut filter located in the optical path to a position outside the optical path and simultaneously giving the subject infrared irradiation and ultraviolet irradiation. By doing so, a color image in the dark is output.

(2) An imaging device includes an irradiating unit that irradiates a subject with infrared rays and ultraviolet rays simultaneously, a lens system that collects light reflected from the subject irradiated with the infrared rays and ultraviolet rays, an infrared cut filter that cuts infrared rays, An ultraviolet cut filter that cuts ultraviolet rays, an image sensor that has sensitivity in the infrared region and the ultraviolet region collected by the lens system, and a color image signal that outputs a color image signal by determining a color from an image signal output from the image sensor A color determination unit, and the irradiation unit irradiates the subject with infrared rays and ultraviolet rays at the same time in the night vision color mode, and moves the infrared cut filter and the ultraviolet cut filter to a position out of the optical path. Is to control.
(3) The color determination means detects the darkest point of the image pickup signal obtained by the image pickup device, and corrects the detected darkest point in the achromatic axis direction in consideration of the black adaptation rate. The image pickup apparatus according to (2), further comprising a correcting unit.
(4) The color determination means detects the darkest point of the image pickup signal obtained by the image pickup device, and corrects the detected darkest point in the achromatic axis direction in consideration of the black adaptation rate. (2) The image pickup apparatus according to (2), further comprising a white balance correction unit that includes a correction unit and white balance correction is performed on the image pickup signal that has been black balance corrected by the black balance correction unit.
(5) The color determination means detects the darkest point of the image pickup signal obtained by the image pickup device, and corrects the detected darkest point in the achromatic axis direction in consideration of the black adaptation rate. A color that includes a correction unit, a white balance correction unit that corrects a white balance of the image signal that has been black balance corrected by the black balance correction unit, and that optimizes the video signal corrected by the white balance correction unit using a color reproduction matrix; (2) The imaging apparatus according to (2), further including a reproduction matrix correction unit.
(6) The color reproduction matrix correction means is provided with an optimum matrix coefficient suitable for the state from the arrangement information of the ultraviolet cut filter and the infrared cut filter and the irradiation information of the infrared ray and the ultraviolet ray, and based on the matrix coefficient The imaging apparatus according to (5), wherein color reproduction is optimized.

It has an image sensor that has sensitivity in the infrared region and the ultraviolet region, and the infrared ray cut filter and ultraviolet ray cut filter arranged in the optical path can be moved to a position off the optical path to enable photographing, and the subject is irradiated with infrared rays and ultraviolet rays. By giving irradiation, a color image can be output even in the dark. Further, color reproducibility can be further improved by performing black balance control, white balance control, and color reproduction matrix control.

  Embodiments of an imaging apparatus according to the present invention will be described below with reference to the drawings.

  As shown in FIG. 1, the imaging apparatus of the present invention includes a lens system 11 that converges image light from a subject, an infrared cut filter 12 that can be moved to a position off the optical path under the control of a microcomputer 23, and a micro An ultraviolet cut filter 13 that can move to a position off the optical path under the control of the computer 23, and a CCD (Charge Coupled Device) as a semiconductor imaging device that converts image light that has passed through the lens system 11 into an electrical signal by photoelectric conversion. Transfer element) 14, an S / H & AGC and A / D converter 15 that controls and holds the gain of the converted electric signal, a clamp circuit 16 that performs black balance correction, and a white balance correction circuit that performs white balance correction 17, a gamma correction circuit 18 that generates a color difference signal, and a signal A processing circuit 19, a color reproduction matrix 20 that performs optimization processing based on the matrix coefficient, an infrared irradiation unit 21 that irradiates the subject with infrared rays under the control of the microcomputer 23, and a subject under the control of the microcomputer 23 An ultraviolet irradiation unit 22 that irradiates ultraviolet rays, control of infrared irradiation, control of ultraviolet irradiation, a specific color extraction control function for detecting the darkest point from an A / D converted image signal and correcting black balance, and infrared irradiation And a microcomputer 23 which controls the control of the ultraviolet rays, the control of the irradiation of ultraviolet rays, the control of the white balance, the control of the color reproduction matrix, and the like. Among them, the clamp circuit 16, the white balance correction circuit 17, the gamma correction circuit 18, the signal processing circuit 19, and the color reproduction matrix 20 form a color determination unit.

  Control of the imaging apparatus having such a configuration will be described below with reference to the flowchart shown in FIG.

In step ST11, in the night vision color mode, when the night vision state is detected in the night vision state, the subject is automatically irradiated with infrared rays and ultraviolet rays simultaneously under the control of the microcomputer 23. The cut filter 12 and the ultraviolet cut filter 13 are removed from the optical path (step ST14).
The image light from the subject enters the CCD 14 through the lens system 11 that converges the image light. The image light incident on the CCD 14 is converted into an electric signal by photoelectric conversion. The converted electrical signal is input to the S / H & AGC and A / D converter 15, and the signal sampled and gain controlled by the S / H & AGC and A / D converter 15 is A / D converted into a digital signal. Converted.
Then, the digital signal is converted to black balance correction (step ST16).
This black balance correction is to correct the deviation of the darkest point of the subject and shift from the achromatic axis, and to control the clamp circuit 16 in consideration of human adaptation to black.
In particular, the gain is increased here to increase the sensitivity of R, G, and B, and the infrared cut filter 12 and the ultraviolet cut filter 13 are imaged out of the optical path , so that the darkest point deviates from the achromatic axis. It is highly possible that this is effective.
In order to perform this control, it is necessary to detect the darkest point of the subject. This is because the signal after A / D conversion is input to the microcomputer 23, the darkest point of the subject is detected, and the darkest point is the origin. The clamp circuit 16 is controlled so as to correct in the ((Y, BY, RY) = (0, 0, 0)) direction, that is, the achromatic axis direction.
For example, in a subject as shown in FIG. 3, the darkest point such as black hair (point A), a black portion of clothing (point B), or a dark portion (point C) where the light source does not hit is detected. Then, as shown in FIG. 4, the detected darkest point is the black of the human eye in the origin (Y, BY, RY) = (0, 0, 0) direction, that is, the achromatic axis direction. This is to be corrected in consideration of the black adaptation rate, which is the adaptation rate to.

This specific black balance correction process will be described in detail below.
First, the RGB signals after the A / D converter 14 are converted into XYZ values that are values that take human visual characteristics into consideration.
For example, an RGB value obtained by applying a power correction to an 8-bit RGB value using the sRGB conversion, which is an international standard for a color space defined by IEC (International Electrotechnical Commission), as shown in the following equation (1): Convert to Other G and B are also converted by the same formula.

  Next, as shown in the equation (2), the RGB value subjected to nonlinear conversion is converted into an XYZ value.

  Here, as this matrix, for example, a value such as the expression (3) is used in the sRGB conversion.

  Note that a look-up table may be used for conversion from these RGB values to XYZ values.

Next, the black adaptation rate is defined as Kadp, and black that takes into account black adaptation is defined as shown in equation (4). Here, XS, IN, MK YS, IN, MK ZS, IN, MK are XYZ values of the detected darkest point. Further, the XYZ values of the darkest point with the black adaptation rate taken into account are XS, IN, KYS, IN, KZS, IN, K.
Here, when Kadp = 1.0, it represents a black adaptation rate of 100%, and black balance correction is performed on the assumption that the black adaptation is completely at the darkest point of the detected subject.
When Kadp = 0.0, the black adaptation rate is 0%. At this time, black balance correction is not performed because there is no black adaptation.
This black adaptation rate represents the adaptation rate of the human eye to black, which is described in “K. / IS & T Electronic Imaging 2002: Proc. SPIE Vol. 4663, p. 217-228, the optimum value depends on the subject, but gives an approximate median of 0.4 to 0.6 I will do it.

Here, as shown in FIG. 5, Kadp may allow the user to freely input the black adaptation rate using a GUI.
Note that the 1/3 power is used for calculation in a space approximating human color perception space, and is not limited to this.

Next, black balance correction conversion as shown in equation (5) is performed based on the darkest point defined by equation (4) and considering black adaptation.
Here, XS, IN YS, IN ZS, IN represent XYZ values of the image data before correction, and XS, OUT YS, OUT ZS, OUT are XYZ values of the image data after black balance correction. .

  Note that γx, γy, and γz are defined by an equation such as equation (6), for example, a monotonically increasing function as shown in FIG.

  XS, OUT YS, OUT ZS, OUT after black adaptation obtained in this way are converted back to RGB values by performing inverse transformations of Equations (1), (2), and (3). .

Returning to FIG. 2, the signal after the black balance correction is input to the white balance correction circuit 17 (step ST17).
The white of the subject is detected by inputting the signal after A / D conversion to the microcomputer 23. Here, as a white detection method, for example, as shown in FIG. 7, information of medium luminance integration, high luminance integration, and total integration is output from the image of the subject, and for example, the integral value of the maximum area is selected. For example, white is detected. A white balance control value is calculated using the detected white signal, and a gain value is set in the white balance correction circuit 17.

Next, gamma correction is performed, and the signal processing circuit 19 converts it into a color difference signal. Thereafter, color reproduction matrix processing or the like is performed (step ST18).
For this color reproduction matrix processing, arrangement information of the infrared cut filter 12 and ultraviolet cut filter 13 and irradiation information of infrared rays and ultraviolet rays are given from the microcomputer 23, and optimum matrix coefficients suitable for the state are given.
For this matrix coefficient, as shown in FIG. 8, a matrix coefficient that optimizes color reproduction in a color chart such as a Macbeth chart or a color bar is previously obtained by adjustment processing, thereby correcting the color reproduction. The Specifically, in FIG. 8, matrix coefficients (black square marks) are red R coefficients and yellow Y coefficients when RY is the X axis and BY is the Y axis. Is located on the upper left surface, green G coefficient is located on the lower left surface, magenta Mg coefficient is located on the upper left surface, blue B coefficient and cyan Cy coefficient are located on the lower right surface. Then, the color information (white square mark) obtained this time is moved to a position near each color information (black square mark) as shown in FIG. 9 for optimization.
In addition, even when ambient light is added, there is a mechanism for calculating an optimum matrix by interpolation, and the correction process is performed.
After the above processing, the signal is converted into, for example, a luminance signal and a color difference signal forming a video signal and converted into an image such as JPEG (step ST13).

It has an image sensor that has sensitivity in the infrared region and the ultraviolet region, the infrared cut filter arranged in the optical path, and the ultraviolet cut filter are moved to a position off the optical path to enable shooting, and the subject is irradiated with infrared rays. By applying ultraviolet irradiation, a color image can be output even in the dark, and color reproducibility can be further improved by performing black balance control, white balance control, and color reproduction matrix control.

It is a block diagram which comprises the imaging device of this invention. It is a flowchart which shows operation | movement similarly. It is explanatory drawing which shows a mode that the darkest point was detected similarly. FIG. 4 is a schematic diagram of black balance correction. It is an input screen for the black adaptation rate. This is a specific example of the function γr. It is explanatory drawing which shows the integration range of white balance similarly. It is explanatory drawing which shows optimization of a color reproduction matrix similarly. It is explanatory drawing which shows optimization of a color reproduction matrix similarly. It is a book figure of the imaging device in a prior art. It is a flowchart which shows the operation | movement in a prior art.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11; Lens system, 12; Infrared cut filter, 13; Ultraviolet cut filter, 14; CCD, 15; S / H & AGC * A / D converter, 16; Clamp circuit, 17: White balance correction circuit, 18: Gamma correction circuit , 19; signal processing circuit, 20; color reproduction matrix, 21; infrared irradiation unit, 22; ultraviolet irradiation unit, 23;

Claims (6)

It has an image sensor with sensitivity in the infrared region and ultraviolet region,
The color is determined from the imaging signal obtained by photographing the subject by moving the infrared cut filter and ultraviolet cut filter located in the optical path to a position outside the optical path and simultaneously giving the subject infrared irradiation and ultraviolet irradiation. To output a color image in the dark. Irradiating means for simultaneously irradiating the subject with infrared rays and ultraviolet rays;
A lens system for collecting the light reflected from the subject irradiated with infrared rays and ultraviolet rays;
An infrared cut filter for cutting infrared rays;
An ultraviolet cut filter that cuts ultraviolet rays,
An imaging device having sensitivity in an infrared region and an ultraviolet region collected by the lens system;
Color determination means for determining a color from an image pickup signal output by the image pickup device and outputting a color image signal;
The irradiating means irradiates the subject with infrared rays and ultraviolet rays at the same time in the night vision color mode, and controls the infrared cut filter and the ultraviolet cut filter to move to a position out of the optical path. An imaging device.   The color determination means includes black balance correction means for detecting the darkest point of the image pickup signal obtained by the image sensor and correcting the detected darkest point in the achromatic axis direction in consideration of the black adaptation rate. The imaging apparatus according to claim 2, wherein the imaging apparatus is provided.   The color determination means includes black balance correction means for detecting the darkest point of the image pickup signal obtained by the image sensor and correcting the detected darkest point in the achromatic axis direction in consideration of the black adaptation rate. The imaging apparatus according to claim 2, further comprising a white balance correction unit that white balance corrects an imaging signal that has been black balance corrected by the black balance correction unit.   The color determination means includes black balance correction means for detecting the darkest point of the image pickup signal obtained by the image sensor and correcting the detected darkest point in the achromatic axis direction in consideration of the black adaptation rate. A color reproduction matrix correction for optimizing the video signal corrected by the white balance correction means with a color reproduction matrix, the white balance correction means for white balance correction of the imaging signal black-balance-corrected by the black balance correction means The imaging apparatus according to claim 2, further comprising means.   The color reproduction matrix correction means is provided with an optimum matrix coefficient suitable for the state from the arrangement information of the ultraviolet cut filter and the infrared cut filter and the irradiation information of the infrared ray and the ultraviolet ray, and performs color reproduction based on the matrix coefficient. The imaging apparatus according to claim 5, wherein the imaging apparatus is optimized.

Images