JP2005117535A - Image pickup device and image pickup method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem wherein occurrence of a color rolling phenomenon can not be completely suppressed since the amplitude of a chrominance signal is not almost changed even when the speed of an automatic white balance control is increased for suppressing the occurrence of the color rolling phenomenon. <P>SOLUTION: When the occurrence of the color rolling phenomenon is detected by a color rolling phenomenon detecting circuit 151, the difference between an integration value of B before the occurrence of the color rolling phenomenon held by an integration value before color rolling hold circuit 152 and an integration value of B in the occurrence of the color rolling phenomenon held in an integration value in color rolling hold circuit 153 is found by a differential circuit 154, and these differential data are added to the integration value of B by an adder 156. White balance control is then performed on the basis of integration values of primary color signals including this integration value of B. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an imaging apparatus and an imaging method, and more particularly to an imaging apparatus and an imaging method using a solid-state imaging element as an imaging device.

  In an imaging apparatus using a solid-state imaging device such as a CCD (Charge Coupled Device) image sensor as an imaging device, for example, in the case of the NTSC system where the field frequency (vertical scanning frequency) of the solid-state imaging device is 59.94 [Hz] When a subject is imaged under the illumination of a fluorescent lamp operating with a commercial power source having a frequency of 60 [Hz], a phenomenon in which luminance and color change with a long period (hereinafter, this phenomenon is referred to as a color rolling phenomenon) occurs. . This color rolling phenomenon is, for example, that the luminance changes in a short cycle (about 20 Hz) when a subject is imaged under illumination of a fluorescent lamp that operates with a power source of 50 [Hz] using a solid-state imaging device compliant with the NTSC system. This phenomenon is different from the flicker phenomenon. Hereinafter, the color rolling phenomenon will be described in more detail.

  The fluorescent lamp blinks (lights on / off) at a cycle according to the power frequency. The spectral characteristics of the fluorescent lamp change depending on the light emission timing within each blinking period, and the light of the fluorescent lamp is not always white. And since the field frequency 59.94 [Hz] of a solid-state image sensor and the power supply frequency 60 [Hz] of a fluorescent lamp differ slightly, the phase relationship between the field period of a solid-state image sensor and the blinking period of a fluorescent lamp is comparatively comparative. It changes periodically with a long period. In other words, due to the slight difference between the field frequency of the solid-state image sensor and the power supply frequency of the fluorescent lamp, the light emission timing of the fluorescent lamp captured by the imaging device gradually shifts, resulting in a color rolling phenomenon in which the imaging result is colored. To do. This color rolling phenomenon becomes more conspicuous especially as the electronic shutter speed of the solid-state imaging device becomes higher, in other words, as the exposure time becomes shorter.

  Even when a subject is imaged under the illumination of a fluorescent lamp that operates with a commercial power supply having a power supply frequency of 50 [Hz] using a solid-state image pickup device compliant with the PAL method, the field frequency of the solid-state image pickup device and the power supply frequency of the fluorescent lamp Is the same in terms of the standard, but actually there is a slight difference between the two, so that a color rolling phenomenon occurs as in the case described above. In order to suppress the occurrence of this color rolling phenomenon, conventionally, automatic white balance adjustment is performed to detect periodic changes in the illuminance of the light source and adjust the white balance of the image signal obtained by the solid-state imaging device according to the detection result. The adjustment speed was increased (for example, see Patent Document 1).

JP-A-11-285010

  However, even if the adjustment speed of the automatic white balance adjustment is increased, in principle, the occurrence of the color rolling phenomenon cannot be completely suppressed for the reason described below. FIG. 6 is a diagram when the video signal is measured by a measuring instrument called a vector scope. In FIG. 6, the vertical axis represents the color difference signal (R−Y) and the horizontal axis represents the color difference signal (B−Y). As shown in FIG. 6A, the color rolling phenomenon extends in the yellow (Ye) direction, but since it is not only pure yellow, there is a continuous color from the center. When the operation of automatic white balance adjustment is performed in this state, as shown in FIG. 6B, the saturation of yellow is reduced in accordance with the average of the color signals, but the saturation of blue is increased. Even if the automatic white balance adjustment speed is increased, the color signal amplitude hardly changes, and as a result, the occurrence of the color rolling phenomenon cannot be completely suppressed.

  Further, when the adjustment speed of the automatic white balance adjustment is increased, it is necessary to apply a large gain to the image signal obtained by the solid-state imaging device, so that the color noise increases as a whole. As described above, the color rolling phenomenon tends to greatly increase in the yellow (Ye) direction. Therefore, to suppress the color rolling phenomenon by automatic white balance adjustment, it is necessary to increase the gain for the blue image signal. For this reason, when the video signal is displayed on the display, blue noise increases on the entire screen.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide an imaging apparatus capable of reliably suppressing the occurrence of a color rolling phenomenon.

  In order to achieve the above object, in the first invention, an integral value of a primary color signal obtained by separating the image light of a subject from an output signal of a solid-state imaging device that converts it into an electrical signal is calculated, and the calculated primary color signal Based on the integrated value, the occurrence of a periodic color change of the subject image that occurs during imaging under fluorescent lamp illumination is detected. Then, a difference between the integral value before detection of the color change and the integral value at the time of detection of the color change is added to the integral value at the time of detection of the color change, and based on the integrated value after this addition Adjust the white balance.

  In the first aspect of the invention, when the color change, that is, the occurrence of the color rolling phenomenon is detected, the difference between the integrated value before the occurrence of the color rolling phenomenon and the integrated value at the time of occurrence of the color rolling phenomenon occurs. By adding to the integrated value at this time, the change in the integrated value due to the occurrence of the color rolling phenomenon can be interpolated, so that the change due to the color rolling phenomenon of the integrated value can be kept small. As a result, the amplifier gain at the time of performing white balance adjustment based on the integral value of the primary color signal does not change greatly, so that color noise on the output due to the occurrence of the color rolling phenomenon can be suppressed.

  In order to achieve the above object, in the second invention, an integral value of a primary color signal obtained by separating an image signal of a subject from an output signal of a solid-state imaging device that converts the image light into an electrical signal is calculated, and the calculated primary color signal Based on the integrated value, the occurrence of a periodic color change of the subject image that occurs during imaging under fluorescent lamp illumination is detected. Then, when the color change is detected, predetermined interpolation data is added to the primary color signal, and white balance is adjusted based on the added primary color signal.

  In the second invention, when the occurrence of the color rolling phenomenon is detected, predetermined interpolation data is added to the primary color signal of B, and white balance adjustment is performed based on the primary color signal after the addition, thereby performing color rolling. Since the change in the primary color signal due to the occurrence of the phenomenon can be interpolated, the color rolling phenomenon itself can be suppressed in the color signal processing unit.

  According to the present invention, when the color rolling phenomenon occurs, it is avoided that the white balance adjustment increases the gain and the color noise increases due to the color rolling phenomenon, or the color rolling phenomenon occurs. By interpolating the change of the primary color signal and suppressing the color rolling phenomenon itself in the color signal processing unit, the occurrence of the color rolling phenomenon can be surely suppressed, so that a good image quality can be obtained.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram illustrating a configuration of an imaging apparatus according to an embodiment of the present invention. As is clear from FIG. 1, the imaging apparatus according to the present embodiment includes, as an example, a lens 11, an imaging device 12, an analog signal processing unit 13, a digital signal processing unit 14, a system controller 15, and a timing generator (TG) 16. It is the composition which has.

  The lens 11 focuses image light from the subject on the imaging surface of the imaging device 12. The imaging device 12 is a solid-state imaging device such as a CCD image sensor or a CMOS image sensor. A color filter (not shown) is provided on the imaging surface, and is based on various drive signals supplied from the timing generator 15. 1 frame image signal obtained by converting the image light imaged on the imaging surface by the lens 11 into an electrical signal in pixel units, for example, in field units.

  The analog signal processing unit 13 includes a CDS (Correlated Double Sampling) circuit and an AGC (Automatic Gain Control) circuit, and performs an image signal output from the imaging device 12 by a CDS circuit. The fixed pattern noise included in the image signal is removed, and the AGC circuit stabilizes the signal level (gain adjustment).

  The digital signal processing unit 14 includes an A / D converter 21, a Y (luminance) / C (color) separation circuit 22, a luminance signal processing unit 23, a color signal processing unit 24, D / A converters 25 and 26, and an optical detector. (OPD) 27 and the like. The A / D converter 21 converts the output signal of the analog signal processing unit 13 into a digital signal. Note that the A / D converter 21 may be configured to be provided before the digital signal processing unit 14.

  The Y / C separation circuit 22 separates the image signal digitized by the A / D converter 21 into a luminance signal and a color signal. The luminance signal processing unit 23 performs signal processing for setting the signal level to an appropriate level as automatic exposure control for the luminance signal supplied from the Y / C separation circuit 22. The luminance signal subjected to the signal processing by the luminance signal processing unit 23 is converted into an analog signal by the D / A converter 25 and output.

  As shown in FIG. 2, the color signal processing unit 22 has at least a primary color separation matrix circuit 221, a data addition circuit 222, and a white balance adjustment circuit 223. The primary color separation matrix circuit 221 separates the color signal supplied from the Y / C separation circuit 22 into R (red), G (green), and B (blue) primary color signals, and these R, G, and B primary color signals. Is supplied to the data addition circuit 222. The function of the data addition circuit 222 will be described later.

  The white balance adjustment circuit 223 includes three amplifiers that process each of the R, G, and B primary color signals, and gain control supplied from a system controller 15 described later to each of these three amplifiers. According to the signal, white balance processing is performed by adjusting the gain so that the ratios of the primary color signals of R, G, and B are equal. The white, balanced R, G, B primary color signals are converted into analog signals by the D / A converter 26 and output.

  The optical detector 27 is configured by an integrator or the like, and the R, G, B primary color signals after white balance is obtained by the white balance adjustment circuit 223 are respectively converted into vertical sync signal periods, that is, each field. The average signal level of each primary color signal in each field is detected by integrating every time. The detection result of the optical detector 27 is given to the system controller 15.

  The system controller 15 is constituted by, for example, a microcomputer including a CPU, a ROM, and a RAM, and the CPU executes various processes by appropriately operating the RAM based on a program stored in the ROM. . One of various processes is an automatic white balance adjustment process. In this automatic white balance adjustment, the system controller 15 is for making the levels of these primary color signals uniform based on the average signal level (integrated value) of the R, G, B primary color signals detected by the optical detector 27. The above-described gain control signal is generated and supplied to the white balance adjustment circuit 223.

  Another example of the various processes executed by the system controller 15 is a color rolling phenomenon detection process. The color rolling phenomenon detection process will be described later. Under the control of the system controller 15, the timing generator 16 starts various drive signals of the imaging device 12, sample hold pulses used for CDS processing in the analog signal processing unit 13, and sampling used in the A / D converter 21. Various timing signals such as pulses are generated and supplied to corresponding circuit units.

  Next, a color rolling phenomenon detection process executed by the system controller 15 will be described. As described above, for example, in the case of the NTSC system in which the field frequency of the solid-state imaging device (the frequency of the vertical synchronization signal) is 59.94 [Hz], this color rolling phenomenon is a fluorescence that operates at a power supply frequency of 60 [Hz]. This is a phenomenon in which the color changes in a long period when an object is imaged under illumination of a lamp, and the object is illuminated under the illumination of a fluorescent lamp operating at a power frequency of 50 [Hz] using a solid-state image sensor compliant with the PAL method. This also occurs in the case of imaging.

  When detecting the color rolling phenomenon, the average signal level of the R, G, B primary color signals detected by the optical detector 27, that is, the integral value is used as in the case of automatic white balance adjustment. When the color rolling phenomenon occurs, the integral value of B becomes extremely small as shown in FIG. This is because the fluorescent lamp repeatedly emits light / extinguishes according to the power supply frequency, but the blue color disappears first, so a yellowish color remains. This is because, when this portion is sampled by the solid-state imaging device, the blue component becomes small in terms of the integral value of the color when the color rolling phenomenon occurs.

  Regarding the integrated values of R and G, no extreme change occurs even if the color rolling phenomenon occurs. Incidentally, there is a possibility that the integral value of B is small even when a yellow subject is imaged. However, when a yellow subject is imaged, a state in which the B integral value becomes small periodically appears. Conversely, a yellow subject is not periodically projected on the entire screen. Therefore, it is easy to differentiate when the color rolling phenomenon occurs and when the yellow subject is imaged, and the system controller 15 detects the color rolling phenomenon by monitoring the integral value of B. can do.

(Example 1)
FIG. 4 is a functional block diagram illustrating an example of functions of the system controller 15 according to the first embodiment.

  As is apparent from FIG. 4, the system controller 15 includes a color rolling phenomenon detection circuit 151, an integrated value hold circuit 152 before color rolling, an integrated value hold circuit 153 during color rolling, a difference circuit 154, and a gain control signal generation circuit 155. doing. The color rolling phenomenon detection circuit 151 constantly monitors the integral value of B, and detects that the color rolling phenomenon has occurred when the integral value of B periodically decreases. The integration value hold circuit 152 before color rolling holds the integration value of B while sequentially updating it in the field period, and stops updating when the color rolling phenomenon detection circuit 151 detects the occurrence of the color rolling phenomenon. The integral value of B before the rolling phenomenon occurs is held.

  The color rolling integration value hold circuit 153 holds the B integration value when the color rolling phenomenon detection circuit 151 detects the occurrence of the color rolling phenomenon. The difference circuit 154 determines the difference between the hold value of the integration value hold circuit 152 before color rolling and the hold value of the integration value hold circuit 153 during color rolling, that is, the integration value of B before the color rolling phenomenon occurs and the color rolling phenomenon. The difference from the integrated value of B when it occurs is calculated. The calculated difference is added to the integral value of B supplied from the optical detector 27 by the adder 156 in the previous stage of the gain control signal generation circuit 155.

  In this way, by performing the interpolation process for adding the difference data calculated by the difference circuit 154 to the B integration value supplied from the optical detector 27, the B integration value does not become extremely small. The gain control signal generation circuit 155 adds the difference calculated by the difference circuit 154 to the R and G integration values supplied from the optical detector 27 and the B integration value supplied from the optical detector 27. Based on the value, a gain control signal for adjusting the gain of each amplifier corresponding to R, G, B of the white balance adjustment circuit 223 is generated.

  FIG. 5 shows the state of the integrated value of B (indicated by the solid line in the figure) and the integrated value of B after the interpolation processing (indicated by the dotted line in the figure) when color rolling occurs. In the figure, as an example of interpolation, a case where the interpolation process is started from the next field where the integral value of B becomes small is shown, and the B integral value after the interpolation process is indicated by a dotted line. As is apparent from FIG. 5, the change in the integral value of B is about ½ as the amplitude before interpolation processing, and the shake from the level where the color rolling phenomenon as a reference does not occur is about ¼. Can be suppressed.

  Note that if the timing at which the color rolling phenomenon occurs can be completely grasped, it is possible to suppress a decrease in the integrated value of B by performing an interpolation process without causing a delay of one field.

  In this way, the occurrence of the color rolling phenomenon is detected based on the B integral value detected by the optical detector 27, and the gain control is performed after suppressing the change in the B integral value caused by the occurrence of the color rolling phenomenon. Based on the gain control signal generated by the signal generation circuit 155, normal white balance adjustment is performed. That is, the gain of each amplifier of the white balance adjustment circuit 223 is adjusted so that the ratio of R, G, B primary color signals is 1: 1: 1. Since the change in the integral value of B is suppressed to be small by the interpolation processing, the gain of the amplifier of B does not change greatly, and as a result, the color noise on the output due to the occurrence of the color rolling phenomenon is suppressed. be able to.

(Example 2)
In the first embodiment, the interpolation process is performed using the integral value of B detected by the optical detector 27. However, in the second embodiment, control for suppressing the color rolling phenomenon itself is performed. That is, in order to suppress the color rolling phenomenon itself, the color signal processing unit 22 performs a process of adding data for canceling the color rolling phenomenon to the primary color signal. This process is also executed under the control of the system controller 15.

  The system controller 15 is the same as in the first embodiment in that the occurrence of the color rolling phenomenon is detected for each field based on the integral value of B detected by the optical detector 27. When the occurrence of the color rolling phenomenon is detected, the system controller 15 converts the interpolation data suitable for the primary color signal of B, for example, when the color rolling phenomenon has occurred into the data addition circuit 222 of the color signal processing unit 22 for each period of the horizontal synchronization signal. The interpolation data is added to the B primary color signal.

  Here, the interpolation data that matches the primary color signal of B when the color rolling phenomenon occurs is, for example, that if the blue component is 80 [h], the color is balanced when R, G, and B are balanced. When the rolling phenomenon occurs and the blue component is only 20 [h], the deficiency is 60 [h]. This deficiency adds the interpolation data 60 [h] to the blue component 20 [h] when the color rolling phenomenon occurs, so that the blue component is corrected to 80 [h], so the color rolling phenomenon itself Can be suppressed.

  Thus, in the second embodiment, for example, interpolation data suitable for the B primary color signal when the color rolling phenomenon occurs is added to the B primary color signal for each period of the horizontal synchronization signal, and the B primary color after this addition is added. By performing automatic white balance adjustment based on each primary color signal including the signal, it is possible to interpolate a change in the primary color signal of B due to the occurrence of the color rolling phenomenon, so that the color signal processing unit 22 performs the color rolling phenomenon itself. Can be suppressed.

  As described above, in the imaging apparatus according to an embodiment of the present invention, when the color rolling phenomenon occurs, the automatic white balance adjustment causes gain to increase and color noise increase due to the color rolling phenomenon. By avoiding (Embodiment 1) or interpolating the change of the primary color signal caused by the occurrence of the color rolling phenomenon, the color signal processing unit 22 suppresses the color rolling phenomenon itself (Embodiment 2). Since the occurrence of the color rolling phenomenon can be reliably suppressed, good image quality can be obtained. In addition, since it is difficult for periodic image changes to occur, the compression rate can be increased when the purpose of performing image compression and data transfer or the like in the subsequent stage of the imaging apparatus is suitable. In addition, special hardware for detecting and suppressing the color rolling phenomenon and an external circuit are not required, so that the intended purpose can be achieved at low cost.

  The control of the first embodiment and the control of the second embodiment are used together, and interpolation is performed using difference data between the B integration value before the color rolling phenomenon occurs and the B integration value when the color rolling phenomenon occurs. After the processing, it is possible to perform processing for further suppressing the color rolling phenomenon by automatic white balance adjustment.

  The imaging apparatus according to the present invention can be used as a camera module for a surveillance camera, a video camera, a digital still camera, a personal computer camera, a mobile phone camera, and the like.

1 is a block diagram illustrating a configuration of an imaging apparatus according to an embodiment of the present invention. It is a block diagram which shows an example of a structure of a color signal processing part. It is a figure which shows the mode of the integral value of B at the time of color rolling generation | occurrence | production. It is a functional block diagram which shows an example of the function of a system controller. It is a figure which shows the mode of the integration value (indicated by a solid line) of B when color rolling occurs and the integrated value of B after interpolation processing (indicated by a dotted line in the drawing). It is a figure where it uses for description of the subject of a prior art.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 11 ... Lens, 12 ... Imaging device, 13 ... Analog signal processing part, 14 ... Digital signal processing part, 15 ... System controller, 16 ... Timing generator (TG), 21 ... A / D converter, 22 ... Y / C separation Circuit, 23 ... Luminance signal processing unit, 24 ... Color signal processing unit, 25, 26 ... D / A converter, 27 ... Optical detector (OPD)

Claims (8)

A solid-state image sensor that converts image light of a subject into an electrical signal;
A computing means for calculating an integral value of the primary color signal obtained by separating from the output signal of the solid-state imaging device;
Detection means for detecting the occurrence of a periodic color change in the subject image that occurs during imaging under fluorescent lamp illumination based on the integral value of the primary color signal calculated by the arithmetic means;
The difference between the integral value before the color change is detected by the detection means and the integral value when the color change is detected by the detection means is added to the integral value when the color change is detected by the detection means. Adding means for
An image pickup apparatus comprising: white balance adjusting means for adjusting white balance based on an integrated value after the difference is added by the adding means. The imaging apparatus according to claim 1, wherein the calculation unit calculates an integrated value of a blue component of the primary color signal. A solid-state image sensor that converts image light of a subject into an electrical signal;
A computing means for calculating an integral value of the primary color signal obtained by separating from the output signal of the solid-state imaging device;
Detection means for detecting the occurrence of a periodic color change in the subject image that occurs during imaging under fluorescent lamp illumination based on the integral value of the primary color signal calculated by the arithmetic means;
Adding means for adding predetermined interpolation data to the primary color signal when the color change is detected by the detecting means;
An image pickup apparatus comprising: white balance adjustment means for adjusting white balance based on a primary color signal after the interpolation data is added by the addition means. The imaging apparatus according to claim 3, wherein the calculation unit calculates an integrated value of a blue component of the primary color signal. A calculation step for calculating an integral value of a primary color signal obtained by separating from an output signal of a solid-state imaging device that converts image light of a subject into an electrical signal;
A detection step of detecting the occurrence of a periodic color change of a subject image that occurs during imaging under fluorescent lighting based on the integral value of the primary color signal calculated in the calculation step;
The difference between the integral value before detection of the color change in the detection step and the integral value at the time of detection of the color change in the detection step is the integral at the time of detection of the color change in the detection step. An addition step to add to the value;
A white balance adjustment step of adjusting a white balance based on an integral value after adding the difference in the addition step. The imaging method according to claim 5, wherein in the calculation step, an integrated value of a blue component of the primary color signal is calculated. A calculation step for calculating an integral value of a primary color signal obtained by separating from an output signal of a solid-state imaging device that converts image light of a subject into an electrical signal;
A detection step of detecting the occurrence of a periodic color change of a subject image that occurs during imaging under fluorescent lighting based on the integral value of the primary color signal calculated in the calculation step;
An adding step of adding predetermined interpolation data to the primary color signal when detecting the color change in the detecting step;
A white balance adjustment step for adjusting a white balance based on a primary color signal after the interpolation data is added in the addition step. The imaging method according to claim 7, wherein in the calculation step, an integrated value of a blue component of the primary color signal is calculated.

Images