WO2013162023A1

WO2013162023A1 - Signal processing device, signal processing program, and digital camera

Info

Publication number: WO2013162023A1
Application number: PCT/JP2013/062478
Authority: WO
Inventors: 篤史河合
Original assignee: 株式会社ニコン
Priority date: 2012-04-27
Filing date: 2013-04-26
Publication date: 2013-10-31
Also published as: JP2015130544A

Abstract

In the present invention, a signal processing device is provided with: a first computation unit which calculates a first subtraction value on the basis of signals from calibration pixels among signals from an imaging element having imaging-use pixels and calibration-use pixels; a second computation unit which calculates a second subtraction value on the basis of signals from the calibration-use pixels; a third computation unit which calculates a third subtraction value by adding the first subtraction value and the second subtraction value in a predetermined ratio; a subtraction unit which subtracts the third subtraction value from the signals by the imaging-use pixels that are output via pixels that are shared with the signals from the calibration-use pixels; and a control unit which controls the subtraction unit so as to differentiate the ratios between an occurrence time of a high luminance stripe noise and a non-occurrence time thereof.

Description

Signal processing apparatus, signal processing program, and digital camera

The present invention relates to a signal processing device, a signal processing program, and a digital camera.

In image sensors such as CCD and CMOS, streak pattern noise may occur due to variations in characteristics of elements that amplify and convert accumulated charges. Since such noise causes image quality deterioration, noise correction processing is performed to reduce the image quality deterioration (see Patent Document 1). In this noise correction, a noise shape included in the correction target signal is estimated based on a calibration signal including noise correlated with the noise of the correction target signal, and the correction target signal is corrected using the estimated noise shape correction value. Is.

International Publication No. WO2007 / 111264

However, the inventor has found that the correction of streak pattern noise according to the prior art is not suitable when a phenomenon called high luminance subtraction occurs in the image signal to be corrected. High luminance pulling refers to a phenomenon that occurs when the characteristics of an element that amplifies and converts the accumulated charge of a pixel column including pixels with strong incident light changes when intense light strikes a part of the image sensor. Since it appears like a trace dragged around the high brightness portion of the image, it is called high brightness pull.

According to the first aspect of the present invention, the signal processing device calculates the first subtraction value based on the signal from the calibration pixel among the signals from the imaging element having the imaging pixel and the calibration pixel. A calculation unit, a second calculation unit that calculates a second subtraction value based on a signal from the calibration pixel, and a third subtraction value is calculated by adding the first subtraction value and the second subtraction value at a predetermined ratio. A third calculation unit, a subtraction unit that subtracts the third subtraction value from the signal from the imaging pixel output via a common element with the signal from the calibration pixel, and when high-intensity pulling occurs and when it does not occur And a control unit that controls the subtracting unit so as to vary the ratio.
According to the second aspect of the present invention, in the signal processing device according to the first aspect, it is preferable that the control unit controls the subtraction unit so as to further increase the ratio of the second subtraction value when high luminance pulling occurs. .
According to the third aspect of the present invention, in the signal processing device according to the first or second aspect, based on a signal from a calibration pixel output via a common element and an element other than the common element, A moving average unit is further provided for calculating a first moving average value based on the first moving average distance and a second moving average value based on a second moving average distance shorter than the first moving average distance. And it is preferable that a control part judges whether high-intensity pulling generate | occur | produces based on the comparison result of a 1st moving average value and a 2nd moving average value.
According to the fourth aspect of the present invention, in the signal processing device according to the third aspect, the control unit determines whether or not high luminance pull occurs based on a difference between the first moving average value and the second moving average value. It is preferable to do.
According to the fifth aspect of the present invention, in the signal processing device according to the third aspect, the control unit determines whether or not high luminance pull occurs based on a ratio between the first moving average value and the second moving average value. It is preferable to do.
According to the sixth aspect of the present invention, in the signal processing device according to any one of the third to fifth aspects, the first calculation unit calculates the first subtraction value based on the first moving average value, The arithmetic unit preferably calculates the second subtraction value based on a signal from the calibration pixel output via the common element.
According to the seventh aspect of the present invention, the signal processing device calculates the first correction value based on the signal from the calibration pixel among the signals from the imaging element having the imaging pixel and the calibration pixel. A calculation unit; a second calculation unit that calculates a second correction value using more calibration pixels than the number of calibration pixels used when the first calculation unit calculates the first correction value; and a first correction value. And a third calculation unit that calculates a third correction value by adding the second correction value at a predetermined ratio, a correction unit that corrects a signal from the imaging pixel using the third correction value, and an imaging pixel It is determined whether or not the signal satisfies a predetermined condition, and includes a control unit that controls the correction unit so that the ratio varies between when the predetermined condition is satisfied and when the predetermined condition is not satisfied.
According to the eighth aspect of the present invention, the signal processing program calculates the first subtraction value based on the signal from the calibration pixel among the signals from the imaging element having the imaging pixel and the calibration pixel. A third calculation for calculating a third subtraction value by adding the calculation, a second calculation for calculating the second subtraction value based on the signal from the calibration pixel, and the first subtraction value and the second subtraction value at a predetermined ratio; The ratio is different between the calculation and the subtraction for subtracting the third subtraction value from the signal from the imaging pixel output via the element common to the signal from the calibration pixel, and the time when the high-intensity pull occurs and does not occur. And controlling the subtraction to be executed by the computer.
According to a ninth aspect of the present invention, a digital camera is equipped with the signal processing device according to any one of the first to seventh aspects.

According to the present invention, it is possible to appropriately correct streak pattern noise even when high luminance pull occurs.

It is a figure explaining the principal part structure of the electronic camera by one embodiment of this invention. It is a figure explaining an image sensor. It is a figure explaining the noise waveform of the signal used for a line clamp process. It is a figure explaining the noise waveform of the signal used for a line clamp process. It is a figure which illustrates the noise waveform of the signal after a line clamp process. It is a figure which illustrates the noise waveform of the signal after a moving average clamp process. It is a figure which illustrates the noise waveform of the signal after a noise attenuation correction clamp process. It is a figure which illustrates the relationship between a noise attenuation coefficient and dispersion | distribution. It is a figure explaining high-intensity pulling. It is a figure which illustrates the noise waveform of the signal read from the pixel for imaging in which high-intensity pulling occurred. It is a figure which illustrates the noise waveform of the signal read from the OB pixel in which high-intensity pulling occurred. It is a figure which illustrates the noise waveform of the signal after the noise attenuation correction clamp process at the time of high-intensity pulling generation. It is a figure which illustrates the noise waveform of the signal after the noise attenuation correction clamp process which raised the line clamp contribution rate at the time of high-intensity pulling generation. It is a figure which illustrates the noise waveform of the signal after the noise attenuation correction clamp process which selectively raised the line clamp contribution rate at the time of high-intensity pulling generation. It is a flowchart explaining the flow of a clamp process. It is a figure which illustrates a computer apparatus.

Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. FIG. 1 is a diagram for explaining a main configuration of a digital camera 1 according to an embodiment of the present invention. In FIG. 1, a digital camera 1 includes an imaging optical system 11, a shutter 12, an imaging device 13, an A / D conversion unit 14, a clamping unit 15, a buffer memory 16, an image processing unit 17, and a liquid crystal monitor. 18, an optical system drive unit 19, a shutter drive unit 20, a CPU 21, a flash memory 22, a card interface (I / F) unit 23, and an operation member 24.

The imaging optical system 11 forms a subject image on the imaging surface of the imaging device 13 via the shutter 12. In FIG. 1, the imaging optical system 11 is illustrated as a single lens, but actually includes a plurality of lens groups. The optical system drive unit 19 performs focus adjustment by moving the focusing lens constituting the imaging optical system 11 forward and backward in the optical axis direction (arrow direction) in response to an instruction from the CPU 21.

The shutter 12 is driven to open and close by the shutter drive unit 20. The exposure time of the image sensor 13 is instructed from the CPU 21 to the shutter drive unit 20. The image sensor 13 is configured by a CMOS image sensor or the like. The image sensor 13 photoelectrically converts the formed subject image into an analog image signal. The A / D converter 14 converts the photoelectric conversion signal into digital data.

In accordance with an instruction from the CPU 21, the clamp unit 15 performs a clamp process for aligning the input image data with a reference level based on calibration data corresponding to black of the image (to remove a noise component described later). Do. The clamped image data (so-called RAW data) is stored in the buffer memory 16.

The buffer memory 16 is a memory that temporarily stores image data. The image processing unit 17 performs predetermined image processing (for example, gradation conversion, color space conversion, color interpolation, white balance processing, etc.) on the image data according to an instruction from the CPU 21. The image processing unit 17 further generates an image file (for example, a still image file or a moving image file) in a predetermined format storing the image data.

The liquid crystal monitor 18 displays an image and an operation menu screen in response to an instruction from the CPU 21. The flash memory 22 stores programs executed by the CPU 21, data necessary for execution processing, and the like. The contents of the program and data stored in the flash memory 22 can be added or changed by an instruction from the CPU 21. CPU21 performs a control program and performs various control with respect to each part of a camera.

The card I / F unit 23 records an image file on the memory card 50 and reads an image file from the memory card 50 in accordance with an instruction from the CPU 21. The memory card 50 is a storage medium that is detachably attached to a card slot (not shown).

The operation member 24 includes a menu switch, a shutter button, and the like, and sends a signal corresponding to the operation of each member to the CPU 21. The CPU 21, flash memory 22, card I / F 23, buffer memory 16, image processing unit 17, and liquid crystal monitor 18 are connected via a bus 25.

The present embodiment is characterized by the clamping process performed by the clamping unit 15 of the digital camera 1, and hence the following description will be focused on the clamping process. The clamp processing of this embodiment performs noise attenuation correction clamp that combines a line clamp and a moving average clamp. First, each clamp will be described.

<Line clamp>
FIG. 2 is a diagram illustrating the image sensor 13. The horizontal direction of the image sensor 13 is X, and the vertical direction of the image sensor 13 is Y. The pixel arrangement arranged in the X direction is called “row”, and the pixel arrangement arranged in the Y direction is called “column”. In FIG. 2, the photoelectric conversion elements constituting the pixels are two-dimensionally arranged on the imaging surface of the imaging element 13. The region 13a is a region where imaging pixels are provided. A region 13b formed at the end of the image sensor 13 is a region where a light-shielded pixel called an OB (Optical? Black) pixel is provided. For example, the accumulated charge of each pixel is sequentially read out via the amplification conversion element 13 c provided for each “row”.

In each pixel in the imaging pixel region 13a, charges corresponding to the intensity of the incident subject light are accumulated, but also include charges constituting noise components unrelated to the incident light. Each OB pixel in the OB pixel region 13b is shielded from light so that only charges constituting a noise component are accumulated.

The line clamp averages the signal values from the OB pixels included in the OB pixel area 13b for each “row”, and the average value OB _line (y) is read out from each pixel in the imaging pixel area 13a. x, y)? is subtracted for each "row". The average value OB _line (y) is called an OB clamp subtraction value.

The above-described line clamp means that the noise component included in the signal data (x, y) from the pixel in the imaging pixel region 13a to be corrected is from the OB pixel in the OB pixel region 13b in the same “row”. Estimated to be equivalent to the noise component included in the signal, and subtracting the OB clamp subtraction value OB _line (y) from the signal data (x, y) of the imaging pixel, both black level standardization and noise correction To do.

In the case of the imaging device 13 illustrated in FIG. 2, all the pixels arranged in the same “row” pass through the same amplification conversion element 13 c, and thus the characteristics of the amplification conversion element 13 c are not uniform with other “rows”. In this case, it appears as streak pattern noise on the read image.

3 and 4 are diagrams for explaining a noise waveform of a signal used for the line clamp processing. FIG. 3A is a diagram illustrating a waveform based on the signal? Data (x, y) read from the pixels in the imaging pixel region 13a. In FIG. 3A, the horizontal axis represents the Y direction of the image sensor 13, and the vertical axis represents the average value of read signals (that is, the average value in the X direction) in the same “row” of data (x, y). The vertical scale is arbitrary. In general, the average waveform in FIG. 3A includes a high-frequency component and a low-frequency component, and a random noise average of each imaging pixel. FIG. 3B shows a waveform obtained by disassembling them.

FIG. 4A is a diagram illustrating a waveform based on a signal read from an OB pixel in the OB pixel region 13b. The horizontal axis of FIG. 4A represents the Y direction of the image sensor 13, and the vertical axis represents the average value of readout signals in the same “row” from the OB pixel (that is, the OB clamp subtraction value OB _line (y)). To express. Similar to the case of FIG. 3 (a), the vertical scale is arbitrary. FIG. 4B is a waveform obtained by decomposing the high-frequency component, the low-frequency component, and the random noise average of the OB pixel as in the case of FIG.

Comparing FIG. 3B and FIG. 4B, the imaging pixel and the OB pixel share the high frequency component and the low frequency component of the noise included in the signal, but the random noise component is the OB pixel. It can be seen that the OB clamp subtraction value OB _line (y) based on the OB clamp is largely included. The high frequency component and the low frequency component are common because the signal passes through the same amplification conversion element 13c in the same “row”. On the other hand, the reason why the random noise component is increased by the signal based on the OB pixel is that the average number of OB pixels in the same “row” is much smaller than the number of imaging pixels, and therefore the random noise component is reduced by averaging. Means insufficient.

FIG. 5 is a diagram illustrating the noise waveform of the signal after the line clamp processing. The horizontal axis in FIG. 5 represents the Y direction of the image sensor 13, and the vertical axis represents the value after subtraction of the OB clamp subtraction value OB _line (y) in FIGS. 3 (a) to 4 (a). According to FIG. 5, noise reduction is performed on the high frequency noise component and the low frequency noise component by the line clamp processing, but the random noise component has a low reduction effect. This is because the random noise larger than the random noise included in the signal before the line clamp is subtracted, and it is difficult to remove the random noise component only by the line clamp. Thus, when different noise components remain between “rows”, streak noise in the X direction remains.

<Moving average clamp>
In order to increase the number of OB pixels to be averaged, the moving average clamp calculates a moving average of OB pixel signals over a wide range (for example, 256 rows) including OB pixels in other adjacent “rows”, and calculates the average value. OB clamp subtraction value _OBRM (y). Then, the OB clamp subtraction value _OBRM (y) is subtracted from the signal data (x, y) of the imaging pixel.

The meaning of the moving average clamp is that the noise components included in the signal data (x, y) from the pixels in the imaging pixel region 13a to be corrected are OB pixels in the same “row” and other adjacent “rows”. It is estimated that the noise component is equivalent to the noise component included in the signal from the OB pixel in the region 13b, and the black level reference is obtained by subtracting the OB clamp subtraction value _OBRM (y) from the signal data (x, y) of the imaging pixel. Both noise reduction and noise correction are performed.

FIG. 6 is a diagram illustrating a noise waveform of a signal after the moving average clamping process. The horizontal axis in FIG. 6 represents the Y direction of the image sensor 13, and the vertical axis represents the value after subtracting the OB clamp subtraction value _OBRM (y) from FIG. According to FIG. 6, noise reduction is performed on the low-frequency noise component and the random noise component by the moving average clamp process, but the reduction effect is low on the high-frequency noise component. This is because not only the random noise component but also the high-frequency noise component is smoothed by the moving average, which indicates that it is difficult to remove the high-frequency noise component only by the moving average clamp. When such a high-frequency noise component remains, it is observed as a horizontal stripe in the X direction. 5 and 6, the random noise component has a larger amplitude than the high-frequency noise component, and the variance σ shown in FIG. 5 is larger in FIG.

<Noise attenuation correction clamp>
The noise attenuation correction clamp is performed by combining both the moving average clamp and the line clamp. The OB clamp subtraction value OB _opt (y) for the y row is obtained by the following equation (1).
OB _opt (y)? =? (1-K _opt ) × OB _RM (y) + K _opt × OB _line (y) (1)
Here, K _opt is a noise attenuation coefficient, OB _RM (y) is an OB clamp subtraction value used in the moving average clamp, and OB _line (y) is an OB clamp subtraction value used in the line clamp.

FIG. 7 is a diagram illustrating the noise waveform of the signal after the noise attenuation correction clamp processing. The horizontal axis in FIG. 7 represents the Y direction of the image sensor 13, and the vertical axis represents the value after subtracting the OB clamp subtraction value OB _opt (y) from FIG. According to FIG. 7, both the low-frequency noise component and the high-frequency noise component are reduced, and adverse effects due to the random noise component are suppressed. The dispersion shown in FIG. 7 is smaller than the dispersion shown in FIGS. Further, since the remaining stripes in FIG. 7 are less noticeable because of their small periodicity, they are the most preferable state in terms of image quality compared to the cases of FIGS.

Note that the noise attenuation coefficient K _opt may be obtained based on the data of the same image sensor 13 taken in advance, or a value that empirically minimizes the dispersion σ in FIG. 7 may be selected. In general, the value of the noise attenuation coefficient K _opt that minimizes the variance σ differs depending on the imaging element 13 (including the amplification conversion element 13c) and the set ISO sensitivity. However, as illustrated in FIG. The change of the variance σ with respect to the change of the coefficient K _opt shows a downward convex function form, and can be uniquely determined by a value from 0 to 1. FIG. 8 is a diagram illustrating the relationship between the noise attenuation coefficient K _opt and the variance σ.

In the present embodiment, the value of the noise attenuation coefficient K _opt in the noise attenuation correction clamp is further changed when there is a possibility of high luminance pulling. In general, high luminance pulling occurs when the characteristics of the amplification conversion elements 13c in the “row” including pixels with strong incident light change when strong light hits a part of the image sensor 13. FIG. 9 is a diagram for explaining high-brightness pulling, and represents a case where a high-brightness subject is projected on the central square portion of the image sensor 13 illustrated in FIG.

In FIG. 9, the characteristics of the amplification conversion element 13 c change in a plurality of “rows” including the high input area HL. Therefore, the imaging in the area HLt other than the high input area HL output via these amplification conversion elements 13 The signal from the pixel for use is also biased toward the positive side (or may be biased toward the negative side), resulting in a high luminance pulling state.

FIG. 10 is a diagram illustrating a waveform based on the signal data (x, y) read from the pixel in the imaging pixel region 13a where the high luminance pull has occurred. The horizontal axis in FIG. 10 represents the Y direction of the image sensor 13, and the vertical axis represents the average value of read signals in the same “row” of data (x, y) (that is, the average value in the X direction). The high-intensity pulling generation region in the drawing corresponds to a “row” including the high input region HL. The vertical scale is arbitrary.

FIG. 11 is a diagram illustrating a waveform based on a signal read from the OB pixel in the OB pixel region 13b where the high luminance pull has occurred. The horizontal axis in FIG. 11 represents the Y direction of the image sensor 13, and the vertical axis represents the average value (ie, OB _line (y)) of the readout signal in the same “row” from the OB pixel. As in the case of FIG. 10, the high luminance pull generation area corresponds to a “row” including the high input area HL.

FIG. 12 is a diagram illustrating the noise waveform of the signal after the noise attenuation correction clamp processing when high luminance pull occurs. The horizontal axis of FIG. 12 represents the Y direction of the image sensor 13, and the vertical axis represents the value after subtracting the OB clamp subtraction value OB _opt (y) from FIG. According to FIG. 12, the signal due to high luminance pulling is smoothed by a moving average at a rate of (1−K _opt ) and then reduced. In the high luminance pulling generation region, the signal from the imaging pixel is reduced. The correction for suppressing the bias toward the positive side (or the negative side) becomes insufficient. Such a failure of high luminance pull correction tends to stand out as a bright (or dark) band having a width corresponding to the high input area HL.

In the present embodiment, when high luminance pulling occurs, the second term on the right side becomes dominant by changing the value of the noise attenuation coefficient K _opt of the above formula (1) to a value close to 1, and the ratio of the contribution of the line clamp is determined. Increase. FIG. 13 is a diagram illustrating the noise waveform of the signal after the noise attenuation correction clamp process in which the line clamp contribution rate at the time of occurrence of high luminance pull is increased. The horizontal axis in FIG. 13 represents the Y direction of the image sensor 13, and the vertical axis represents the value after subtracting the OB clamp subtraction value OB _opt (y) from FIG. According to FIG. 13, correction can be performed to suppress the bias of the signal from the imaging pixel to the positive side (or the negative side) even in the high luminance pulling generation region.

However, on the other hand, it becomes difficult to remove random noise components in the non-high luminance pull-out generation region, as in the case illustrated in FIG. Therefore, the value of the noise attenuation coefficient K _opt is changed so as to selectively increase the line clamp contribution rate only for the “row” including the high input region HL. FIG. 14 is a diagram illustrating the noise waveform of the signal after the noise attenuation correction clamp process when the value of the noise attenuation coefficient K _opt is thus controlled. The horizontal axis in FIG. 14 represents the Y direction of the image sensor 13, and the vertical axis represents the value after subtracting the OB clamp subtraction value OB _opt (y) from FIG. According to FIG. 14, in the high luminance pulling generation region, correction for suppressing the bias of the signal from the imaging pixel to the positive side (or negative side) can be performed, and in the non-high luminance pulling generation region, the low frequency noise component and the high frequency are generated. Both noise components are reduced, and adverse effects due to random noise components are suppressed.

<Flow of clamping process>
The flow of the clamping process that the CPU 21 causes the clamp unit 15 to execute will be described with reference to the flowchart illustrated in FIG. The CPU 21 activates the processing shown in FIG. 15 at the time of reading the pixel signal every time the imaging element 13 performs imaging.

In step S10 of FIG. 15, the CPU 21 calculates the average value OB _line (y) of the OB pixel signal from the OB area 13b for each y row, and proceeds to step S20. The clamp unit 15 stores the calculation result in a memory (not shown). In step S20, the CPU 21 determines whether or not the calculation of the average value OB _line (y) has been completed for all rows. If the calculation has been completed for all rows, the CPU 21 makes an affirmative decision in step S20 and proceeds to step S30. If the calculation has not been completed for all rows, the CPU 21 makes a negative determination in step S20 and proceeds to step S40. In step S40, the CPU 21 increments y and returns to step S10. When returning to step S10, similar average value calculation processing is performed for other rows.

In step S30, the CPU 21 calculates the moving average value OB _hlt (y) for high-intensity detection by the following equation (2) based on the average value OB _line (y) stored in the memory (not shown). Proceed to step S50.

However, N _hlt ? Is a moving average distance, and is, for example, 8 (for the line) from the focused line (y) to the lower side.

In step S50, the CPU 21 calculates the moving average clamping moving average value _OBRM (y) by the following equation (3) based on the average value _OBline (y) stored in the memory (not shown). Proceed to step S60. _OBRM (y) is used as the OB clamp subtraction value in the above-described moving average clamp.

Here, N _RM ? Is a moving average distance, for example, 256 (line) from the target line (y) to the lower side.

In step S60, the CPU 21 determines whether or not high luminance pull occurs for each y row. For example, the CPU 21 compares the difference absolute value of OB _hlt (y) −OB _RM (y) with a predetermined determination threshold value TH _hlt ?, And | OB _hlt (y) −OB _RM (y) |> TH _hlt ? If it is established, an affirmative decision is made in step S60 and the process proceeds to step S70. If | OB _hlt (y) −OB _RM (y) |> TH _hlt ? Is not satisfied, the CPU 21 makes a negative determination in step S60 and proceeds to step S80.

In step S70, the CPU 21 estimates that high luminance pulling has occurred, replaces the coefficient K with K _{hlt ?} , and proceeds to step S90. For example, 1 is used as K _hlt , but a value approaching 1 may be obtained according to the magnitude | OB _hlt (y) −OB _RM (y) | In step S80, the CPU 21 estimates that no high luminance pull has occurred, and proceeds to step S90 using the above-described noise attenuation coefficient _Kopt as the coefficient K.

In step S90, the CPU 21 performs a noise attenuation correction clamp using the value obtained by the following equation (4) as the OB clamp subtraction value OB _opt (y) by the following equation (5), and proceeds to step S100.

DATA (x, y)? =? Data (x, y)?-OB _opt (y) ...... (5)
However, data (x, y) is image data read from the imaging pixel region 13a, and DATA (x, y) is image data after clamping.

In step S100, the CPU 21 determines whether or not clamping has been completed for all rows. If all the rows have been completed, the CPU 21 makes an affirmative decision in step S100 and ends the processing in FIG. If the CPU 21 has not finished processing for all the rows, the CPU 21 makes a negative determination in step S100 and proceeds to step S110. In step S110, the CPU 21 increments y and returns to step S60. When returning to step S60, the same processing is repeated for the other rows.

According to the embodiment described above, the following operational effects can be obtained.
(1) The digital camera 1 calculates _OBRM (y) based on the signal from the OB pixel (13b) among the signals from the image pickup device 13 having the imaging pixel (13a) and the OB pixel (13b). The clamp unit 15, the clamp unit 15 that calculates OB _line (y) based on the signal from the OB pixel (13b), and OB _RM (y) and OB _line (y) are added together by a predetermined coefficient K to obtain OB OB _opt (y) is subtracted from the signal from the image pickup pixel (13a) output via the clamp unit 15 for calculating _opt (y) and the common element (13c) with the signal from the OB pixel (13b). Since the clamp unit 15 and the CPU 21 that controls the clamp unit 15 so as to make the coefficient K different between when the high luminance pull occurs and when the high luminance pull does not occur, the streaks are formed even when the high luminance pull occurs. Pattern noise can be corrected appropriately.

(2) In the digital camera 1, the CPU 21 controls the clamp unit 15 so that the coefficient K of OB _line (y) is closer to 1 when high luminance pull occurs. It is possible to appropriately correct streak pattern noise.

(3) In the digital camera 1, a moving average for 256 rows based on signals from the OB pixel (13b) output via the common element (13c) and the elements other than the common element via (13c) distance further comprising a clamp unit 15 that calculates _{OB RM} (y), and the moving average distance shorter than 256 rows 8 rows by OB _hlt a (y), respectively by, CPU 21 may, _{OB RM} (y) and OB _hlt ( Since it is determined whether or not high luminance pulling occurs based on the comparison result with y), it can be appropriately estimated whether or not high luminance pulling can occur.

(4) In the digital camera 1, CPU 21 is, since the determination as to whether the high luminance lookup occurs whether on the basis of the difference between OB RM _(y) and OB _hlt (y), high brightness pulling occurs Whether it is obtained or not can be estimated appropriately with a small amount of calculation.

(5) In the digital camera 1, the clamp unit 15 calculates _OBRM (y) based on the moving average value, and outputs the signal from the OB pixel (13b) output through the common element (13c). Since OB _line (y) is calculated based on this, the above estimation can be performed appropriately.

(Modification 1)
In the above description, the moving average distance N _hlt ? For high-luminance detection is 8 (for a line) as an example. However, the moving average distance N _hlt ?

(Modification 2)
In addition, although 256 (rows) is given as an example of the moving average distance N _RM ? For calculating the moving average value OB _RM (y) for moving average clamping, it is not limited to 256 lines, and may be 128 lines or 512 lines. It may be changed as appropriate.

(Modification 3)
In the above-described step S60, the example in which the presence / absence of high luminance pull is estimated based on the difference between two moving average values OB _hlt (y) and OB _RM (y) having different calculated distances has been described. Instead, the presence / absence of occurrence of high luminance pull may be estimated based on the ratio of two moving average values OB _hlt (y) and OB _RM (y) having different calculated distances.

(Modification 4)
Further, the number of moving averages to be compared is not limited to the two described above. For example, when three moving average values are used, a first moving average value having a moving average distance for high luminance detection corresponding to 8 rows, a second moving average value having a moving average distance corresponding to 64 rows, and moving The third moving average value OB _RM (y) for average clamping is calculated, and the presence / absence of occurrence of high luminance pull is estimated based on the magnitude relationship or ratio of these three moving average values. Good. Also in this case, it goes without saying that each moving average distance may be changed as appropriate.

(Modification 5)
The direction in which the moving average is calculated may be set not only from the lower side from the focused row (y) but also from the focused row (y) to the upper side. In addition, the upper and lower directions may be set across the row (y) of interest.

(Modification 6)
In the example of the image sensor 13 of FIG. 2 of the above-described embodiment, all the pixels arranged in the same “row” pass through the same amplification conversion element 13c, but the image sensor is each of the pixels arranged in the same “row”. Alternatively, different amplification conversion elements may be provided.

(Modification 7)
In the embodiment described above, an example in which a signal processing device that performs clamping processing is mounted on the digital camera 1 has been described. However, a signal processing device that performs OB clamping processing later may be configured using a personal computer. In this case, read data of each pixel included in the imaging pixel area 13a and read data of each OB pixel included in the OB pixel area 13b are stored as image data acquired by the digital camera.

Then, by causing the computer apparatus 200 shown in FIG. 16 to execute the program for performing the processing of the flowchart illustrated in FIG. 15, the OB clamping process is performed on the image data to configure the signal processing apparatus. When the program is used by being loaded into the personal computer 200, the program is loaded into the data storage device of the personal computer 200 and then used as a signal processing device by executing the program.

The loading of the program to the personal computer 200 may be performed by setting a storage medium 204 such as a CD-ROM storing the program in the personal computer 200, or to the personal computer 200 by a method via the communication line 201 such as a network. You may load. When passing through the communication line 201, the program is stored in the hard disk device 203 of the server (computer) 202 connected to the communication line 201. The program can be supplied as various forms of computer program products such as provision via the storage medium 204 or the communication line 201.

The above description is merely an example, and is not limited to the configuration of the above embodiment.

Although various embodiments and modifications have been described above, the present invention is not limited to these contents. Other embodiments conceivable within the scope of the technical idea of the present invention are also included in the scope of the present invention.

The disclosure of the following priority application is hereby incorporated by reference.
Japanese Patent Application No. 2012-61414 (filed on April 27, 2012)

Claims

A first arithmetic unit that calculates a first subtraction value based on a signal from the calibration pixel among signals from an imaging element having an imaging pixel and a calibration pixel;
A second calculation unit that calculates a second subtraction value based on a signal from the calibration pixel;
A third arithmetic unit that calculates the third subtraction value by adding the first subtraction value and the second subtraction value at a predetermined ratio;
A subtraction unit that subtracts the third subtraction value from a signal from the imaging pixel that is output via a common element with a signal from the calibration pixel;
A control unit that controls the subtraction unit so as to vary the ratio between occurrence of high luminance and non-occurrence; and
A signal processing apparatus comprising:
The signal processing device according to claim 1,
The control unit is a signal processing device that controls the subtraction unit so as to further increase the ratio of the second subtraction value when the high luminance subtraction occurs.
The signal processing device according to claim 1 or 2,
Based on a signal from the calibration pixel output via the common element and an element other than the common element, a first moving average value based on a first moving average distance, and the first moving average distance A moving average unit for calculating a second moving average value based on a shorter second moving average distance,
The said control part is a signal processing apparatus which judges whether the said high-intensity pulling generate | occur | produces based on the comparison result of the said 1st moving average value and a said 2nd moving average value.
The signal processing device according to claim 3.
The said control part is a signal processing apparatus which judges whether the said high-intensity pulling generate | occur | produces based on the difference of a said 1st moving average value and a said 2nd moving average value.
The signal processing device according to claim 3.
The said control part is a signal processing apparatus which judges whether the said high-intensity pulling generate | occur | produces based on ratio of the said 1st moving average value and a said 2nd moving average value.
The signal processing apparatus according to any one of claims 3 to 5,
The first calculation unit calculates the first subtraction value based on the first moving average value,
The second processing unit is a signal processing device that calculates the second subtraction value based on a signal from a calibration pixel output through the common element.
A first calculation unit that calculates a first correction value based on a signal from the calibration pixel among signals from an imaging element having an imaging pixel and a calibration pixel;
A second calculation unit that calculates a second correction value by using more calibration pixels than the number of calibration pixels used when the first calculation unit calculates the first correction value;
A third calculation unit that calculates the third correction value by adding the first correction value and the second correction value at a predetermined ratio;
A correction unit that corrects a signal from the imaging pixel using the third correction value;
Control that determines whether or not a signal of the imaging pixel satisfies a predetermined condition, and controls the correction unit to vary the ratio between when the predetermined condition is satisfied and when the predetermined condition is not satisfied And
A signal processing apparatus comprising:
A first calculation for calculating a first subtraction value based on a signal from the calibration pixel among signals from an imaging element having an imaging pixel and a calibration pixel;
A second calculation for calculating a second subtraction value based on a signal from the calibration pixel;
A third operation for calculating a third subtraction value by adding the first subtraction value and the second subtraction value at a predetermined ratio;
Subtraction for subtracting the third subtraction value from the signal from the imaging pixel output via a common element with the signal from the calibration pixel;
Control for controlling the subtraction so that the ratio is different between when high-intensity pulling occurs and when it does not occur;
A signal processing program for causing a computer to execute.
A digital camera equipped with the signal processing device according to any one of claims 1 to 7.