JP5335355B2 - Signal processing apparatus, signal processing method, and imaging apparatus - Google Patents

Description

  The present invention relates to a signal processing device, a signal processing method, and an imaging device, and more particularly to a smear correction technique that corrects image quality deterioration due to a smear phenomenon of an imaging element.

  In general, the smear phenomenon that occurs when an image sensor such as a CCD is used is a phenomenon in which a bright streak 52 enters along the vertical transfer path direction (vertical direction) of the image sensor as shown in FIG. Are known. This smear phenomenon is caused by, for example, when there is a strong light subject (light source 51) in the screen, unnecessary charge is accumulated by exposure during vertical transfer of the image sensor, and signal charge transferred through the vertical transfer path. It is generated by mixing in

  Such a phenomenon can be suppressed by closing the mechanical shutter during charge transfer from the image sensor. However, at the time of moving image shooting or EVF (electronic viewfinder) display, it is difficult to use a mechanical shutter because high-speed processing is required to sweep data from the image sensor. For this reason, a smear phenomenon has occurred, which has been a factor that significantly deteriorates the image quality of an image obtained from the image sensor.

  The following methods are known as methods for correcting smear in order to avoid image quality deterioration due to smear. For example, as shown in FIG. 8A, the amount of smear is detected based on a signal from an optical black portion (OB portion) 53 provided outside the effective pixel portion of the image sensor. The smear correction is performed by uniformly subtracting the smear on the video signal in the vertical transfer path direction according to the detected smear amount (see, for example, Patent Documents 1 and 2).

JP-A-5-7335 JP 2000-350032 A JP 2006-324976 A

  However, the above-described method of subtracting the smear amount uniformly in the vertical direction (vertical transfer path direction) assumes that the smear amount added to each pixel during vertical transfer is always constant. Accordingly, depending on a slight amount of smear variation such as fluctuation of the light source 51 and variation of the image sensor, as shown in FIG. It sometimes occurred. Further, minute vertical streaks due to over-correction and residual correction occur at different positions for each time series instead of a fixed pattern, and thus cause a significant deterioration in image quality particularly in moving images.

  On the other hand, a method of controlling the smear correction amount according to the magnitude of the smear detection amount has been proposed (see, for example, Patent Document 3). For example, in a portion where the smear detection amount is large, a method may be considered in which partial overcorrection or residual correction is made inconspicuous by multiplying the detection amount by a gain of 1 or less to weaken the smear correction amount. However, in order to prevent some harmful effects, the effect of smear correction itself was weakened.

  In addition, the adverse effect after smear correction due to such overcorrection and residual correction is particularly noticeable when the smear amount fluctuates greatly from column to column, where the smear amount itself is small and variation from column to column is small. The flicker due to overcorrection is not noticeable.

  The present invention has been made in view of such circumstances, and adjusts the correction amount in the smear correction in accordance with the detected amount of smear and the generated shape so as to suppress image quality deterioration due to overcorrection or residual correction. For the purpose.

The signal processing apparatus according to the present invention includes a smear detection unit that detects a smear amount based on a signal of an optical black unit provided separately from an effective pixel unit of the image sensor, and the image sensor detected by the smear detection unit. Based on the smear amount in the horizontal line, the smear shape determining means for determining the inclination of the smear generation shape in the optical black portion, the smear amount detected by the smear detecting means, and the determination result of the smear shape determining means A smear correction amount determining means for determining a smear correction amount; a smear correction means for performing smear correction on a signal of the effective pixel portion of the image sensor using the smear correction amount determined by the smear correction amount determination means; It is characterized by providing.
The imaging device of the present invention includes the signal processing device, an imaging device that photoelectrically converts incident light and supplies an output to the signal processing device, and a display unit that displays an image related to the output of the signal processing device. It is characterized by that.
The signal processing method of the present invention includes a smear detection step of detecting a smear amount based on a signal of an optical black portion provided separately from an effective pixel portion of the image pickup device, and the image pickup device detected in the smear detection step. Based on the smear amount in the horizontal line, the smear shape determination step for determining the inclination of the smear generation shape in the optical black portion, the smear amount detected in the smear detection step, and the determination result in the smear shape determination step Accordingly, a smear correction amount determining step for determining a smear correction amount, and a smear correction step for performing smear correction on the signal of the effective pixel portion of the image sensor using the smear correction amount determined in the smear correction amount determination step. It is characterized by having.
The program of the present invention includes a smear detection step for detecting a smear amount based on a signal of an optical black portion provided separately from an effective pixel portion of the image pickup device, and a horizontal line of the image pickup device detected in the smear detection step. Based on the amount of smear in the smear shape determination step for determining the inclination of the smear generation shape in the optical black portion, the smear amount detected in the smear detection step, and the determination result in the smear shape determination step A smear correction amount determining step for determining a smear correction amount; and a smear correction step for performing smear correction on a signal of the effective pixel portion of the image sensor using the smear correction amount determined in the smear correction amount determination step. The computer is executed.

  According to the present invention, by determining the smear correction amount based on the smear generation shape in addition to the smear detection amount and performing the smear correction, the image quality deterioration due to the overcorrection or residual correction that occurs when the smear correction is performed. Can be suppressed. It is not necessary to suppress the smear correction amount more than necessary to suppress flicker due to overcorrection and residual correction, and it is possible to perform smear correction using the optimal smear correction amount for the scene, and it is possible to improve image quality It is.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram illustrating a configuration example of an imaging apparatus according to an embodiment of the present invention. FIG. 1 shows a digital camera as an example of an imaging apparatus according to the present embodiment.

  The digital camera in this embodiment includes an optical system 1, a focus lens 2, an image sensor 3, a pre-processing circuit 4, an A / D (analog-digital) converter 5, a smear correction circuit 6, an output processing circuit 7, and a recording medium. 8 and an electronic viewfinder (EVF) 9. In addition, the digital camera according to the present embodiment includes a control unit 10, a switch SWA 11, a switch SWB 12, a focus lens driving circuit 13, an exposure control circuit 14, a white balance control circuit 15, and a distance measurement control circuit 16.

  The light (incident light) imaged by the optical system 1 and the focus lens 2 is subjected to photoelectric conversion by the image sensor 3 and processed by a pre-processing circuit 4 including a CDS circuit and an amplifier circuit for removing output noise. , And digitized by the A / D converter 5. The signal digitized by the A / D converter 5 is subjected to smear correction in the smear correction circuit 6 which is a signal processing device, and is then converted into an image by the output processing circuit 7, and the image is displayed on the EVF 9 which is a display means. Is displayed. The signal smear corrected by the smear correction circuit 6 is recorded on the recording medium 8 via the output processing circuit 7 or used in each control circuit.

  In the digital camera of this embodiment, the focus operation is controlled by the distance measurement control circuit 16. When the switch SWA11 is pressed, control related to the focus operation is started, and the focus lens 2 is driven via the focus lens drive circuit 13. In the focus operation, the contrast of the subject is detected from the captured image signal, and the focus lens is driven to the peak position of the contrast.

  The exposure control is controlled by the exposure control circuit 14. In exposure amount control, the brightness level of a predetermined area of the screen is measured, the exposure amount at which the target brightness level is obtained is calculated, and the shutter speed or aperture is controlled. The white balance control circuit 15 controls white balance.

  When the switch SWB12 is pressed, an image is recorded on the recording medium 8. At this time, since a mechanical shutter is used and smear does not occur, smear correction is not performed.

FIG. 2 is a block diagram illustrating a schematic configuration example of the smear correction circuit 6.
The smear correction circuit 6 includes a smear amount detection circuit 21 as a smear detection unit, a correction amount determination circuit 22 as a smear shape determination unit and a smear correction amount determination unit, and a correction circuit 23 as a smear correction unit.

  The smear amount detection circuit 21 is based on a digital signal output from the A / D converter 5 (specifically, an optical black portion (OB portion) signal shielded so that light does not reach the pixel in the image sensor 3). To detect the amount of smear. Specifically, the smear amount detection circuit 21 outputs the signal (pixel value) of the OB unit 53 provided below the screen (effective pixel unit) in the imaging device 3 as shown in FIG. Integration is performed to calculate a smear amount S (i) (i is a horizontal coordinate position) for each column.

  The correction amount determination circuit 22 determines a smear generation shape based on the smear amount S (i) for one horizontal line related to the image sensor 3 detected by the smear amount detection circuit 21. Further, the correction amount determination circuit 22 determines a smear correction amount to be subtracted from the image signal when performing smear correction processing according to the determination result of the smear occurrence shape and the smear amount S (i).

  The correction circuit 23 is a smear correction amount determined by the correction amount determination circuit 22 from a digitized image signal output from the A / D converter 5 (specifically, a signal of an effective pixel portion of the image sensor 3). Is subtracted and smear correction is performed. The image signal smear corrected by the correction circuit 23 is output to the output processing circuit 7 and the like.

FIG. 3 is a block diagram illustrating a configuration example of the correction amount determination circuit 22.
The correction amount determination circuit 22 includes a smear shape analysis unit 24, a correction gain determination unit 28 as a smear correction amount determination unit, and a multiplier 29.

  The smear shape analysis unit 24 detects smear generated from each light source based on the smear amount S (i) corresponding to 1H (one horizontal line related to the image sensor 3) detected by the smear amount detection circuit 21. Determine the shape. For example, the smear shape analysis unit 24 determines whether the generated smear is a sharp smear (see part a in FIG. 4) or a smear having a gentle slope (see part b in FIG. 4). To do. As a determination algorithm for determining the smear occurrence shape, a method of performing a determination by evaluating pattern matching or a dispersion value based on the amount of smear at each position may be considered.

  The smear shape analysis unit 24 includes a noise removal unit 25, a section detection unit 26, and a shape determination unit (smear shape determination means) 27. The noise removal unit 25 performs noise removal processing on the data of the smear amount S (i) for one horizontal line related to the image sensor 3 detected by the smear amount detection circuit 21. The section detection unit 26 determines the smear generation shape based on the smear amount S (i) for one horizontal line detected by the smear amount detection circuit 21 and subjected to noise removal processing by the noise removal unit 25. Detect areas. Based on the smear amount for one horizontal line detected by the smear amount detection circuit 21, the shape determination unit 27 determines the occurrence shape of smear in each region detected by the section detection unit 26. By determining the smear generation shape in each region detected by the section detection unit 26, even when there are a plurality of light sources, appropriate smear correction can be performed for each light source.

The operation of the smear shape analysis unit 24 will be described below.
First, in order to reduce erroneous determination due to noise, the noise removal unit 25 performs noise removal by a low-pass filter (LPF) or coring on the input smear amount S (i).

  Next, the section detection unit 26 performs threshold processing on the input smear amount S (i), and as shown in FIG. 4, a plurality of regions (smear generation shapes having a smear amount equal to or greater than the threshold TH) are obtained. Divided into areas to be determined). Here, for example, in FIG. 4, the distribution of the smear amount in the horizontal direction is a sharp mountain (first shape) a based on the threshold value TH, and a gentle mountain (second shape) b with a smaller change in the smear amount. It is divided into. Note that the clustering of the data of the smear amount S (i) in the section detection unit 26 is performed by detecting a valley between smear generation regions from pattern matching with a mountain having a certain standard distribution shape or a slope of the data. You may use the method of clustering by the valley part made.

  Next, the shape determination unit 27 determines whether the smear generation shape (smear amount distribution shape) of each divided region is a sharp mountain (first shape) or a gentle mountain (second shape). And the determination result is output to the correction gain determination unit 28. As for the operation of the shape determining unit 27, a method of determining the smear occurrence shape based on the amount of smear in the region and the degree of dispersion thereof may be considered.

  For example, as shown in FIG. 5A, the shape determination unit 27 has smear amounts (a−1, a + 1, b−1, b + 1) before and after each point of the rising edge A and the falling edge B that are both ends of the region. ) To determine the smear amount gradients g_a and g_b at points A and B. If at least one of the obtained smear amount inclinations g_a and g_b is equal to or greater than a predetermined inclination, it is determined that the smeared shape is a sharp mountain. On the other hand, when the inclination is smaller than the predetermined inclination, it is determined that the smeared shape is a gentle mountain. In this case, the slope is used as an evaluation value for determining the occurrence shape of the smear, but the method for determining the threshold value of a sharp peak or a gentle peak is that the designer will notice that the flicker is visually noticeable after smear correction. There is a method of setting a threshold value by a judged value, that is, an empirical method.

  As another determination method, the shape determination unit 27 performs smearing in the section AB based on the threshold value TH and the smear amount S (i) used in the section detection unit 26 as shown in FIG. An integral value H1 and a value H2, which are sums of quantities, are obtained according to equations (1) and (2).

  And the shape determination part 27 calculates | requires ratio R = H1 / H2, and when the ratio R is more than predetermined value, it determines with the generation | occurrence | production shape of the smear being a sharp mountain, and ratio R is predetermined value. If it is smaller, there is a method of determining that the smear generation shape is a gradual mountain.

  It should be noted that the equations (1) and (2) are expressed as TH × (B−A) = ((S (), where S (A) and S (B) are smear amounts at the positions A and B at both ends of the region. Since A) + S (B)) × | B−A |) / 2 is satisfied, it is generalized as shown in equations (3) and (4).

  As another determination method, as shown in FIG. 5C, the shape determination unit 27 determines the maximum value MAX of the smear amount S (i) in the section AB and the width (| BA of the section). The ratio R to (MAX / | BA) is obtained. When the obtained ratio R is greater than or equal to a predetermined value, it is determined that the smear occurrence shape is a sharp mountain. When the ratio R is smaller than the predetermined value, the smear occurrence shape is There is a method to judge that it is a gentle mountain.

In any of the determination methods, the threshold for determining whether the smear generation shape is a sharp peak or a gentle peak is the designer's judgment whether the flicker is visually noticeable in the image after smear correction. Can be set empirically.
The above-described method for determining the smear occurrence shape is merely an example, and the present invention is not limited to this.

  When the shape determination unit 27 determines the smear occurrence shape as described above, the correction gain determination unit 28 receives the determination result from the shape determination unit 27, and the correction gain determination unit 28 determines the region correction gain (correction coefficient) according to the determination result. ) G (G ≦ 1.0) is determined. The smear amount S (i) output from the smear amount detection circuit 21 and the correction gain G determined by the correction gain determination unit 28 are multiplied by the multiplier 29 and corrected by the correction gain (correction coefficient) G. The corrected smear correction amount is output to the correction circuit 23.

  For example, when the determination result of the smear occurrence shape in the shape determination unit 27 is a gradual mountain, the correction gain G = 1.0, that is, the detected smear amount is directly subtracted from the image signal as the correction amount. To do. On the other hand, when the determination result of the smear occurrence shape is a sharp peak, the smear correction amount is suppressed until the flicker is not sufficiently noticeable (the detected smear amount is multiplied by the correction gain G <1), and then the image is displayed. Subtract from signal.

  Here, the correction gain that makes the flickering inconspicuous is that the smear amount fluctuates with the dispersion γ (Sν, Bν) based on the dispersion value γ (Sν, Bν) of the sensor value at the sensitivity Sν and the incident light quantity Bν. However, a method of setting the smear correction amount at the lower limit of dispersion so as not to cause overcorrection can be considered.

  For example, it is assumed that the smear amount varies with time as shown in FIG. 6 at a certain sensitivity and incident light quantity. At this time, assuming that the smear amount is dispersed with the dispersion value γ centered on the average value AVE, the lower limit value MIN is obtained in a sufficient number of samples to perform smear correction so that overcorrection does not occur. If smear correction is performed with this lower limit value, overcorrection will not occur. At this time, the correction gain G may be G = MIN / AVE.

  According to the present embodiment, a smear generation shape is determined based on the detected smear amount, and the smear correction amount is adjusted based on the smear generation shape in addition to the smear amount to perform smear correction on the image signal. As a result, image quality deterioration due to overcorrection or residual correction can be suppressed without suppressing the smear correction amount more than necessary to suppress flicker due to overcorrection or residual correction, and using an appropriate smear correction amount according to the scene. Smear correction can be performed. In addition, by adaptively changing the smear correction amount according to the shape of smear generation, it is possible to provide an imaging apparatus or the like that suppresses flicker due to overcorrection or residual correction and has a large smear correction effect.

(Other embodiments of the present invention)
For realizing the functions of the above-described embodiment for a computer (CPU or MPU) in an apparatus or system connected to the various devices so that the various devices are operated to realize the functions of the above-described embodiments. Supply software programs. And what was implemented by operating the said various devices according to the program stored in the computer of the system or the apparatus is also contained under the category of this invention.
In this case, the software program itself realizes the functions of the above-described embodiments, and the program itself constitutes the present invention. Further, means for supplying the program to the computer, for example, a recording medium storing the program constitutes the present invention. As a recording medium for storing such a program, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.
In addition, such a program is also included in the embodiment of the present invention when the function of the above-described embodiment is realized in cooperation with an operating system running on a computer or other application software. Needless to say.
Further, after the supplied program is stored in a memory provided in a function expansion board or a function expansion unit related to the computer, a CPU or the like provided in the function expansion board or the like based on an instruction of the program may be a part of actual processing or Do everything. Needless to say, the present invention includes the case where the functions of the above-described embodiments are realized by the processing.

For example, the signal processing apparatus described above has a computer function 700 as shown in FIG. 7, and the CPU 701 performs the operation in this embodiment.
As shown in FIG. 7, the computer function 700 includes a CPU 701, a ROM 702, and a RAM 703. Further, a controller (CONSC) 705 of the operation unit (CONS) 709 and a display controller (DISPC) 706 of a display (DISP) 710 as a display unit such as a CRT or LCD are provided. Furthermore, a hard disk (HD) 711, a controller (DCONT) 707 of a storage device (STD) 712 such as a flexible disk, and a network interface card (NIC) 708 are provided. These functional units 701, 702, 703, 705, 706, 707, and 708 are configured to be communicably connected to each other via a system bus 704.
The CPU 701 comprehensively controls each component connected to the system bus 704 by executing software stored in the ROM 702 or the HD 711 or software supplied from the STD 712. That is, the CPU 701 reads out from the ROM 702, the HD 711, or the STD 712 and executes a processing program for performing the operation as described above, thereby performing control for realizing the operation in the present embodiment. The RAM 703 functions as a main memory or work area for the CPU 701.
The CONSC 705 controls an instruction input from the CONS 709. The DISPC 706 controls the display of the DISP 710. The DCONT 707 controls access to the HD 711 and the STD 712 that store a boot program, various applications, user files, a network management program, the processing program in the present embodiment, and the like. The NIC 708 exchanges data with other devices on the network 713 in both directions.

  The above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed as being limited thereto. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

It is a figure which shows the structural example of the imaging device which concerns on embodiment of this invention. It is a figure which shows the structural example of the smear correction circuit in this embodiment. It is a figure which shows the structural example of the correction amount determination circuit in this embodiment. It is a figure for demonstrating the generation | occurrence | production shape of a smear. It is a figure for demonstrating an example of the determination method of the generation | occurrence | production shape of the smear in this embodiment. It is a figure which shows an example of the time fluctuation | variation of a smear amount. It is a block diagram which shows the computer function which can implement | achieve the signal processing apparatus in a present Example. It is a figure for demonstrating the bad effect at the time of smear correction | amendment and smear correction | amendment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 3 Image pick-up element 4 Pre-processing circuit 5 A / D converter 6 Smear correction circuit 7 Output processing circuit 9 Electronic viewfinder 10 Control part 21 Smear amount detection circuit 22 Correction amount determination circuit 23 Correction circuit 24 Smear shape analysis part 25 Noise removal Unit 26 section detection unit 27 shape determination unit 28 correction gain determination unit 29 multiplier

Claims (10)

Smear detection means for detecting a smear amount based on a signal of an optical black portion provided separately from the effective pixel portion of the image sensor;
Smear shape determination means for determining the inclination of the smear occurrence shape in the optical black portion based on the amount of smear in the horizontal line of the image sensor detected by the smear detection means;
A smear correction amount determining means for determining a smear correction amount in accordance with the smear amount detected by the smear detection means and the determination result of the smear shape determination means;
A signal processing apparatus comprising: a smear correction unit that performs smear correction on a signal of the effective pixel portion of the image sensor using the smear correction amount determined by the smear correction amount determination unit. Based on the amount of smear in the horizontal line of the imaging device detected by the smear detection means, further comprising a section detection means for detecting a region for determining a smear occurrence shape in the optical black portion,
The signal processing apparatus according to claim 1, wherein the smear shape determination unit determines a slope of a smear generation shape in each region detected by the section detection unit. The smear shape determination means has a first smear generated shape according to a change in the amount of smear detected by the smear detection means, and a second shape having a gentler slope than the first shape. Determine which of the two shapes it is,
The smear correction amount determination means suppresses the smear correction amount when it is determined that the smear generation shape is the first shape, compared to when it is determined that the second shape is the second shape. Item 3. The signal processing device according to Item 2.   The smear shape determining means evaluates a slope related to the smear amount at both ends of the region detected by the section detecting means, and when the slope at at least one end is equal to or greater than a predetermined slope, a smear generation shape 4 is determined to be the first shape, and when the inclination at both ends is smaller than a predetermined inclination, it is determined that the smear generation shape is the second shape. Signal processing equipment.   The smear shape determining means determines that the smear occurrence shape is the first when the ratio of the integral value of the smear amount in the area to the width of both ends of the area detected by the section detecting means is equal to or greater than a predetermined value. The signal processing apparatus according to claim 3, wherein the signal processing apparatus determines that the smear generation shape is the second shape when the ratio is smaller than a predetermined value.   When the ratio of the maximum value of the smear amount in the region to the width of the both ends of the region detected by the section detection unit is equal to or greater than a predetermined value, the smear shape determining unit determines that the smear generation shape is the first shape. The signal processing apparatus according to claim 3, wherein the signal processing apparatus determines that the smear generation shape is the second shape when the ratio is smaller than a predetermined value.   The smear correction amount determining means determines a correction coefficient according to a smear generation shape determined by the smear shape determining means, and a correction amount according to the smear amount detected by the smear detection means is determined by the correction coefficient. The signal processing apparatus according to claim 1, wherein the signal processing apparatus corrects and determines the smear correction amount. The signal processing device according to any one of claims 1 to 7,
An image sensor that photoelectrically converts incident light and supplies an output to the signal processing device;
An imaging apparatus comprising: display means for displaying an image related to the output of the signal processing apparatus. A smear detection step of detecting a smear amount based on a signal of an optical black portion provided separately from an effective pixel portion of the image sensor;
A smear shape determining step for determining a slope of a smear generated shape in the optical black portion based on a smear amount in a horizontal line of the imaging element detected in the smear detection step;
A smear correction amount determination step for determining a smear correction amount according to the smear amount detected in the smear detection step and the determination result in the smear shape determination step;
And a smear correction step of performing smear correction on the signal of the effective pixel portion of the image sensor using the smear correction amount determined in the smear correction amount determination step. A smear detection step for detecting a smear amount based on a signal of an optical black portion provided separately from an effective pixel portion of the image sensor;
A smear shape determining step for determining a slope of a smear generated shape in the optical black portion based on a smear amount in a horizontal line of the imaging element detected in the smear detection step;
A smear correction amount determining step for determining a smear correction amount according to the smear amount detected in the smear detection step and the determination result in the smear shape determination step;
A program for causing a computer to execute a smear correction step of performing smear correction on a signal of the effective pixel portion of the image sensor using the smear correction amount determined in the smear correction amount determination step.

Images