JP6525602B2 - Image pickup apparatus and control method thereof - Google Patents

Description

  The present invention relates to an imaging apparatus and its control method, and more particularly to an imaging apparatus that converts an analog signal read out from an imaging device into a digital signal and its control method.

  Conventionally, a standard called full high-definition television with 1920 horizontal pixels and 1080 vertical pixels has been commonly used as a television standard. However, in recent years, the transition to a television standard called 4K2K called horizontal 3840 pixels and vertical 2160 pixels has been advanced. In the future, a transition to the next-generation television standard, called 8K4K (Super Hi-Vision) with 7680 pixels horizontally and 4320 pixels vertically, is planned. In addition to the increase in the number of pixels, the frame rate has also been steadily increased.

  With the transition of such television standards, an increase in the number of pixels and a increase in frame rate are required for imaging devices that capture television images, and in order to satisfy this, the speed at which the imaging device reads the images One of the issues is to speed up In order to speed up the reading speed, it is essential to speed up the processing speed of the AD converter that the imaging device has, and various techniques for speeding up the AD converter have been proposed.

  Further, in addition to the speeding up of the reading speed, there is also a high demand for the improvement of the gradation accuracy for the purpose of the improvement of the S / N ratio and the expansion of the dynamic range. Therefore, to improve the gradation accuracy without increasing the circuit scale or extending the processing time has become an important issue in order to improve the image performance in the future.

  With respect to such a problem, Patent Document 1 discloses an imaging device having the following configuration. That is, a first pixel signal obtained by amplifying a pixel signal by a first gain, and a second pixel signal amplified by a second gain larger than the first gain, using a column amplifier circuit in the imaging device Are analog-to-digital (AD) converted using different AD converter circuits. Then, according to the level of the pixel signal, one of the first pixel signal and the second pixel signal after AD conversion is selectively output. With such a configuration, it is possible to realize the expansion of the dynamic range and the improvement of the S / N ratio.

  Furthermore, according to Patent Document 1, after the level readout of the first pixel signal and the second pixel signal selectively read out to the same gain level, a gain error or an offset error is detected, and a pixel signal is detected based on the detected value. We also propose a technology to correct the By performing such processing, it is possible to reduce the level difference of the signal level that occurs when one sheet of image is synthesized from the first pixel signal and the second pixel signal that are selectively read out.

JP, 2012-080252, A

  However, in Patent Document 1, the variation (variation) of the correction value calculated using the detection value is not taken into consideration. The main factors that the correction value fluctuates include rapid temperature change, switching of the driving method of the imaging device, noise of the power supply supplied to the column amplifier, wiring noise, disturbance noise (for example, magnetism generated by motor driving etc.) There are noise jumps etc. When a single image is synthesized from the first pixel signal and the second pixel signal as disclosed in Patent Document 1 in a state where the correction value fluctuates due to such a factor, the level difference of the signal level is I can see it.

  For example, in the case of using a correction method in which a correction value is calculated and updated for each frame, there is a frame in which a level difference in signal level occurs and a frame in which a level difference in signal level does not occur. It becomes a certain picture.

  In addition, when using a method of calculating and correcting different correction values for each area obtained by dividing an image for one frame, there is an area in which a step is generated and an area in which no level difference is generated for each area. It becomes an image. In particular, in an image in which the contrast of the subject is low and the luminance level changes gradually, the image is noticeable.

  The present invention has been made in view of the above problems, and it is an unnatural image in which a level difference of signal level occurs or a level difference of level changes between frames or in an area in a screen. The purpose is to prevent

In order to achieve the above object, an imaging device according to the present invention is an analog that converts an analog signal into a digital signal using a pixel unit in which a plurality of pixels are two-dimensionally arranged and a plurality of reference signals having different inclinations. digital conversion means, and voltage supply means for supplying a plurality of different output level analog signals of a previously determined in the analog-digital conversion means, the analog signals of the plurality of different power levels supplied by the voltage supply means the The analog-to-digital conversion of an analog signal output from the pixel unit using the plurality of reference signals based on a plurality of digital signals obtained by converting each of the plurality of reference signals by the analog-to-digital conversion means to calculate the correction value for correcting the slope and offset of the digital signal obtained by converting by means A calculation means, wherein the calculating means performs a filtering process of weighted addition by cyclic coefficient offset correction value for correcting the offset, before newly acquired filtering and offset correction value, before When the difference between the frame and the offset correction value after the filtering process is larger than a threshold, the cyclic coefficient is set to zero .

  According to the present invention, it is possible to prevent an unnatural image in which a level difference in signal level occurs or a level level changes between frames or in an area in a screen.

BRIEF DESCRIPTION OF THE DRAWINGS The block diagram which shows schematic structure of the image pick-up element used with the imaging device in embodiment of this invention. FIG. 7 is a diagram showing a schematic configuration and operation timing of a column amplifier group of an imaging element. FIG. 7 is a timing chart for explaining the operation of AD conversion in the first embodiment. The figure which shows the relationship of an output level and the AD conversion result in the case of using the ramp signal from which inclination differs according to a signal level. FIG. 2 is a view showing an example of the arrangement of a pixel section of an imaging device according to the first embodiment; FIG. 7 is a timing chart when AD conversion processing is performed on the fixed voltage V1 in the readout period of the dummy pixel of the first embodiment with the first ramp signal VRAMP (small). FIG. 7 is a timing chart in the case of performing AD conversion processing of the fixed voltage V1 in the readout period of the dummy pixel of the first embodiment with the second ramp signal VRAMP (large). FIG. 7 is a timing chart when AD conversion processing is performed on the fixed voltage V2 in the readout period of the dummy pixel of the first embodiment with the first ramp signal VRAMP (small). FIG. 7 is a timing chart in the case of performing AD conversion processing of the fixed voltage V2 in the readout period of the dummy pixel of the first embodiment with the second ramp signal VRAMP (large). FIG. 7 is a view showing an example of the fluctuation of the offset correction value β when noise is generated in the first embodiment. FIG. 6 is a view for explaining filtering processing of an offset correction value at the time of disturbance noise occurrence in the first embodiment. FIG. 7 is a diagram for explaining set values of cyclic coefficients at the time of power supply start-up and at the time of switching the driving method of the imaging device in the first embodiment. FIG. 6 is a view showing setting values of cyclic coefficients with respect to temperature change in the first embodiment. FIG. 6 is a view showing an example of the luminance distribution of an image of one frame in the first embodiment. FIG. 6 is a view showing weighting coefficients and cyclic coefficients when the cyclic coefficient is determined according to the contrast in the first embodiment. FIG. 6 is a view showing cyclic coefficients in the case of determining cyclic coefficients according to a frame rate in the first embodiment. FIG. 8 is a view showing setting values of cyclic coefficients in the case of changing cyclic coefficients according to the stability of the offset correction value in the first embodiment. It is a figure regarding a circulation coefficient in the case of changing a circulation coefficient according to the amount of movement of a photographic subject in a 1st embodiment. FIG. 7 is a diagram related to a cyclic coefficient in the case of changing the cyclic coefficient according to the color of the color filter in the first embodiment. FIG. 7 is a diagram relating to a cyclic coefficient in the case where the reference signal level is changed according to the gain switching of the column amplifier in the first embodiment. Explanatory drawing in the case of calculating an offset correction value for every area | region in the image pick-up element of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

First Embodiment
FIG. 1 is a block diagram showing a configuration of an image pickup device 1 used in an image pickup apparatus according to a first embodiment of the present invention, and an image which is an output destination of image data obtained by the image pickup device 1 and the image pickup device 1 The processing unit 2 is shown. The imaging device 1 in the first embodiment is a CMOS image sensor mounted with a parallel AD converter. The image processing unit 2 performs development processing such as white balance processing and gamma processing on the image data output from the imaging device 1 and finally records the image data on a recording medium. Further, the image processing unit 2 incorporates a CPU, and performs communication (for example, serial communication) with the imaging element 1 according to the operation mode of the imaging device by this CPU and performs control.

  In the imaging device 1, the timing control unit 100 supplies an operation clock signal and a timing signal to each block of the imaging device 1 to control the operation.

  The pixel unit 110 includes a plurality of pixels arranged in a two-dimensional manner, and converts a charge obtained by photoelectric conversion according to an incident light amount in a photoelectric conversion element of each pixel into a voltage and outputs the voltage. Color filters and microlenses are mounted on each pixel. In addition, it is general to take a periodic structure of Bayer arrangement by so-called RGB primary color filters of three colors of R (red), G (green) and B (blue) as a color filter, but it is not always necessary to Not as long.

  The vertical scanning circuit 120 performs timing control for sequentially reading out pixel signals obtained by each pixel of the pixel unit 110 within one frame period. In general, reading is sequentially performed row by row from the upper row to the lower row in one frame.

  The column amplifier group 130 includes a plurality of column amplifiers provided in each column, and is used to electrically amplify a pixel signal read from the pixel unit 110. By amplifying the pixel signal by the column amplifier group 130, the S / N ratio to the noise generated by the lamp circuit 140 and the column analog-to-digital converter group (column ADC group) 150 in the subsequent stage is improved. However, in a circuit structure in which the noise generated by the lamp circuit 140 and the column ADC group 150 is sufficiently smaller than the noise generated by the pixel unit 110, the column amplifier group 130 is not necessarily required.

  The constant voltage circuit 400 operates as a voltage supply unit that supplies a fixed voltage to a signal line connecting the pixel unit 110 and the column amplifier group 130. Although the constant voltage circuit 400 is used in the first embodiment, a circuit applicable to a clip circuit or the like that clips a signal at a certain voltage may be used.

  The ramp circuit 140 is a signal generator that generates a ramp-shaped voltage signal (ramp signal) having a constant slope in the time direction. The column ADC group 150 includes a column ADC including a comparator 151 and a counter / latch circuit 152 for each column. The comparison unit 151 compares the pixel signal (analog signal) amplified by the column amplifier group 130 with the ramp signal from the ramp circuit 140, and outputs a signal indicating the magnitude relationship of the signals. Then, the counter / latch circuit 152 latches the counter value in accordance with the signal from the comparison unit 151, thereby performing analog-digital conversion. The detailed operations of the comparison unit 151 and the counter / latch circuit 152 will be described later. The digital image data for one row held in the counter / latch circuit 152 is read by the horizontal transfer circuit 160 sequentially from the end row.

  The image data read by the horizontal transfer circuit 160 is input to the signal processing circuit 170. The signal processing circuit 170 is a circuit that digitally performs signal processing, and can perform gain computation easily by performing shift operation and multiplication in addition to adding a fixed amount of offset value by digital processing. In addition, a light-shielded pixel region (OB pixel unit) may be formed in the pixel unit 110, and a digital black level clamp operation may be performed using a signal obtained from a pixel of the OB pixel unit. Furthermore, based on the input image data, as described later, a ramp signal output from the ramp circuit 140 is controlled via the timing control unit 100.

  The image data processed by the signal processing circuit 170 is passed to the external output circuit 180. The external output circuit 180 has a serializer function, and converts a multi-bit input parallel signal from the signal processing circuit 170 into a serial signal. Further, the serial signal is converted to, for example, an LVDS signal and the like, and is output to the image processing unit 2.

  The controller circuit 300 is an I / F unit with the image processing unit 2, and receives control from the CPU of the image processing unit 2 to the imaging element 1 using a serial communication circuit or the like.

  Next, the principle of basic AD conversion using the row ADC group 150 of the imaging device 1 will be described with reference to FIG. As described above, the column ADC group 150 includes the comparator 151 and the counter / latch circuit 152 for each column. Further, as shown in FIG. 2A, the comparison unit 151 compares the pixel signal VAMP output from the column amplifier group 130 with the ramp signal VRAMP output from the ramp circuit 140, and outputs the result. .

  As shown in FIG. 2B, the operation of the comparison unit 151 is started prior to the start of reading out the pixel signal VAMP from the pixel unit 110 (time t1). When the operation of each column amplifier of the column amplifier group 130 is stabilized, the count value of the counter / latch circuit 152 is reset at time t2. In synchronization with the count / reset timing of the counter / latch circuit 152, the signal level of the ramp signal VRAMP output from the ramp circuit 140 increases with the passage of time from time t2. The output of the comparison unit 151 is inverted when the signal level of the ramp signal VRAMP output from the ramp circuit 140 exceeds the signal level of the pixel signal VAMP output from the column amplifier group 130 (time t3). The counter / latch circuit 152 performs a count operation during a period (time t2 to time t3) after the count value is reset and the output of the comparison unit 151 is inverted. By this operation, a count value proportional to the output level of the pixel signal amplified by the column amplifier group 130 is obtained, and thus the count value obtained in this way becomes the AD conversion result. Note that the comparison method of the pixel signal and the ramp signal described here, the counting method by the counter and latch circuit, and the like are merely examples, and it is possible to detect a period from resetting the count value to inverting the output of the comparing unit 151. It may be implemented by other methods.

  FIG. 3 is a diagram for explaining the operation of the ramp circuit 140 and the column ADC group 150 in the first embodiment. In FIG. 3, the horizontal axis represents time, the vertical axis at the top of the graph represents the output level, and the lower part of the graph represents the output of the comparison unit 151. An example of changing the slope of the ramp signal VRAMP output from the ramp circuit 140 in accordance with the signal level of the output signal VAMP of the column amplifier group 130 will be described with reference to FIG.

  Generally, in the process of reading out a signal from a unit pixel, reading out and AD conversion of an N signal (noise level) is first performed, and thereafter reading out and AD conversion of an S signal (noise level + signal level). Then, the difference between the S signal and the N signal converted by the signal processing circuit 170 is taken to cancel the noise component, thereby obtaining a signal with a good S / N.

  First, in order to perform AD conversion of the N signal, the operation of the comparison unit 151 is started at time t11, and the count of the counter / latch circuit 152 is reset at time t12 and the ramp signal VRAMP output from the ramp circuit 140 Change the signal level. Here, since the signal level of the N signal which is a noise level is small, the first ramp signal VRAMP (small) (first reference signal) having a small inclination is used for AD conversion of the N signal. The N signal is AD converted by performing a count operation during a period (time t12 to time t13) after the count of the counter / latch circuit 152 is reset and the output of the comparison unit 151 is inverted (time t12 to time t13).

  Next, the signal corresponding to the charge stored in the pixel unit 110 is read out, and the ramp circuit 140 outputs a certain determination level Vs during the level determination period with respect to the S signal which is an output signal amplified by the column amplifier group 130. Is output to the comparison unit 151. Then, the comparison with the S signal is performed. Here, at time t14, the count value of the counter / latch circuit 152 is reset, and the ramp circuit 140 starts outputting the determination ramp signal having the predetermined determination level Vs at the maximum level. If the signal level of the S signal is equal to or higher than the determination level Vs (S ≧ Vs), the output of the comparison unit 151 is not inverted, so the count value continues to increase until the level determination period ends at time t16. On the other hand, if the signal level of the S signal is smaller than the determination level Vs (S <Vs), for example, the output of the comparison unit 151 is inverted at time t15, the count value increase is ended. As described above, based on the count value of the counter / latch circuit 152, the signal processing circuit 170 can determine whether the signal level of the S signal is larger or smaller than the determination level Vs. When the output of the ramp circuit 140 is stabilized at the determination level Vs when the count value of the counter / latch circuit 152 is reset, and the signal level of the S signal is smaller than the determination level Vs (S <Vs), Control may be performed so that the count value is zero.

  If the signal level of the S signal is smaller than the determination level Vs, AD conversion of the S signal is performed from time t17 using the same first ramp signal VRAMP (small) as the N signal. Thereby, in the example shown in FIG. 3, the count value between time t17 and time t18 is obtained. On the other hand, when the signal level of the S signal is equal to or higher than the determination level Vs, the second ramp signal VRAMP (large) (second reference signal) whose slope is α times that of the first ramp signal VRAMP (small) is selected. The AD conversion of the S signal is performed using this. Thereby, in the example shown in FIG. 3, the count value between time t17 and time t19 is obtained.

  FIG. 4 is a diagram showing the relationship between the signal level of the output signal and the AD conversion result when using ramp signals having different slopes according to the output level. The horizontal axis of FIG. 4 indicates the output signal level of the column amplifier group 130, and the vertical axis indicates the digital value of the S signal after AD conversion. The solid line represents a digital value (AD conversion value) input to the signal processing circuit 170 after being AD converted by the comparison unit 151 and the counter / latch circuit 152, and passing through the horizontal transfer circuit 160. As described above, the S signal whose signal level is smaller than the determination level Vs uses the first ramp signal VRAMP (small), and the S signal having a signal level higher than the determination level Vs is the second ramp signal VRAMP ( AD conversion is performed using the Therefore, as shown in FIG. 4A, the S signal after AD conversion is not matched before and after the determination level Vs.

  Therefore, with respect to the AD conversion value of the S signal whose signal level is larger than the determination level Vs, the signal processing circuit 170 first performs the first ramp signal VRAMP (small) and the second ramp signal VRAMP (large). Multiply the ratio α of the slopes of Further, by adding the offset amount β so that the step is eliminated at the determination level Vs, correction is performed so that the signal level of the pixel signal according to the incident light amount and the AD conversion value have a primary relationship.

  If the image of the effective pixel is output without performing the above-described correction, the image will have an uncomfortable feeling as if a step remains at a certain luminance. The correction value used for this correction changes the ideal correction target value depending on the temperature of the image sensor, the drive timing of the image sensor (such as the gain and operating state of the column amplifier group 130), and the drive setting (such as power setting). When a change occurs in the condition of (1), it is necessary to reacquire the correction value.

  In addition, a temporary change caused by a sudden change of the correction value occurring at the time of power activation and immediately after the switching of the driving method of the imaging device 1, noise of the power supply supplied to the column amplifier group 130, wiring noise, and disturbance noise. It is necessary to reduce the variation of the correction value as much as possible. Disturbance noise includes, for example, magnetic noise generated by motor drive and the like. The details of the method of reducing the fluctuation of the correction value will be described later.

  Next, an example of processing for calculating the ratio α of the slopes of the first ramp signal VRAMP (small) and the second ramp signal VRAMP (large) and the offset amount β will be described.

  FIG. 5 shows a configuration example of the pixel unit 110. As shown in FIG. As a pixel configuration, a dummy pixel area not having a photodiode is disposed at the top, and in order, an optical black (OB) pixel area shielded from light and an effective pixel area for outputting a signal obtained by photoelectric conversion are arranged. There is. In the first embodiment, the dummy pixels are used to calculate the ratio α of the inclination and the offset amount β. Here, a fixed voltage is input from the constant voltage circuit 400 during the pixel signal readout period of the dummy pixel, and the voltage input from the column amplifier group 130 to the comparison unit 151 is controlled to be a fixed voltage. In the first embodiment, voltages V1 and V2 smaller than the determination level Vs are used as fixed voltages.

  The AD conversion process in the reading period of the dummy pixel will be described with reference to FIG. In FIG. 6A, the fixed voltage V1 is AD converted. Unlike the process described with reference to FIG. 3, there is no need to provide a period for AD converting the N signal. As shown in FIG. 6A, by raising the ramp signal VRAMP output from the ramp circuit 140 in the level determination period to the maximum value VRAMP (MAX), the first ramp signal VAMP (small) having a small inclination has a fixed voltage V1. Perform AD conversion. The result of AD conversion is V1L.

  Subsequently, as shown in FIG. 6B, by setting the ramp signal VRAMP output from the ramp circuit 140 during the level determination period to the minimum value VRAMP (MIN), a fixed voltage is set for the second ramp signal VAMP (large) having a large inclination. AD convert V1. The result of AD conversion is V1H.

  Thereafter, as shown in FIGS. 6C and 6D, the fixed voltage is changed to a voltage V2 larger than the voltage V1, and AD conversion is performed as in FIGS. 6A and 6B. The results are V2L and V2H, respectively.

These are represented as shown in FIG. 4B, where the horizontal axis is an output level and the vertical axis is an AD conversion value. FIG. 4B is an enlarged view of a portion where the output signal level of FIG. 4A is smaller than the determination level Vs. From the coordinates of these four points, the ratio α of the slope and the offset amount β can be determined by Equation (1) and Equation (2), respectively.
α = (V2L−V1L) / (V2H−V1H) (1)
β = (V2L−V1L) / (V2−V1) × Vs
-Α (V2H-V1H) / (V2-V1) x Vs (2)
The calculation of the correction values α and β may be performed inside the imaging device 1 or may be performed by the image processing unit 2. Note that V1L, V1H, V2L, and V2H can be obtained in plural when reading the pixel signal of the dummy pixel from the dummy pixel area, so the ratio α of inclination and the offset amount β are determined by Equations (1) and (2). In each case, the average value of each is used.

  Here, attention is paid to the offset amount β calculated by the equation (2). The offset amount β is calculated by the equation (2) so that two straight lines intersect at the determination level Vs, but as described above, the driving method of the imaging element 1 is switched at the time of power activation. Immediately after or when disturbance noise occurs etc., it may fluctuate greatly. Due to this fluctuation, a level difference occurs at the signal level near the determination level Vs, resulting in an unnatural image. However, in order to suppress the image quality deterioration due to the level difference of the signal level while improving the gradation accuracy for the purpose of the improvement of the S / N ratio and the expansion of the dynamic range, the level level difference does not appear as an image It must be reduced to the level.

  FIG. 7 is a graph showing the fluctuation of the offset correction value β when the offset correction value β is obtained for each frame under the condition where noise causing the fluctuation of the offset amount (offset correction value) β is generated. is there. In the figure, the horizontal axis indicates the number of frames (corresponding to time), and the vertical axis indicates the offset correction value β, and the change of the offset correction value β for each frame is indicated by the graph A. Further, a graph C shows an ideal correction value (ideal value) in a state where there is no influence by noise. The ideal value does not change due to noise components and the like that occur irregularly, but changes due to the drive setting of the imaging device and the temperature change of the imaging device. In the first embodiment, it is assumed that the drive setting of the imaging element remains the same and the temperature is sufficiently stable.

  In the first embodiment, for convenience of explanation, as shown in the graph C, it is ideal that the value of the offset correction value β is always −5 LSB during 1 to 100 frames. In addition, when the value of the graph A exceeds ± 5 LSB (−10 LSB or less or 0 LSB or more) with respect to the value of the graph C which is the ideal value, it is assumed that the level difference of the signal level starts to be visible on the image.

  In the example shown in FIG. 7, there are many frames in which the value of graph A exceeds ± 5 LSB with respect to the value of graph C, which is an ideal value. In other words, an image in which a level difference can be seen in a certain frame and a level difference can not be seen in the next frame is repeated many times. Here, a method of reducing the fluctuation of the offset correction value β will be specifically described.

As one method of reducing the fluctuation of the offset correction value β, there is a method of performing a filtering process using the following equation (3).
βf (n) = β (n) × p + βf (n−1) × (1−p)
(0 ≦ p ≦ 1) (3)
In equation (3), β f (n) is an offset correction value after filtering processing in the nth frame. Also, using the image acquired at the nth frame, a new offset correction value calculated based on the above equation (2) is β (n), and the offset correction value after filtering processing at the n−1th frame is calculated. β f (n−1) and the cyclic coefficient are defined as p (0 ≦ p ≦ 1). The possible range of n is n 0 0, and in the case of n = 0, the offset correction value β f (-1) of the previous frame does not exist, so that p is set to 1.0 and , .Beta. (N-1) = 0.

  The change of the value of the offset correction value βf after the filtering process when the cyclic coefficient p = 0.1 in the above equation (3) is shown by the graph B in FIG. As can be seen from the graph, by performing the filtering process on graph A, the value of offset correction value βf is within the range of ± 5 LSB from graph C, which is the limit value at which the level difference of the signal level begins to appear on the image. It is contained in

  In the example shown in FIG. 7, although the cyclic coefficient p is set to 0.1, it does not necessarily have to be 0.1, and the cyclic coefficient may be determined according to the magnitude of fluctuation of the correction value due to noise. . In addition, as means for performing filtering processing, processing may be performed using a program, or a circuit for filtering processing may be additionally provided inside the imaging device 1 or in the image processing unit 2.

The slope ratio α obtained as described above and the offset correction value βf are used as a correction coefficient, and the following equation (4) is used to perform AD conversion of the second ramp signal VRAMP (large): The digital signal S _D (n) of the S signal is corrected to obtain a corrected digital signal S ′ _D (n).
S ′ _D (n) = S _D (n) × α + β f (4)

  As described above, the filtering process is performed on the offset correction value β calculated using the signal obtained by reading out the fixed voltage during the readout period of the dummy pixel. As a result, it is possible to reduce the fluctuation of the offset correction value β below the limit value at which the step of the signal level starts to appear on the image. Thereby, it is possible to reduce the image quality deterioration due to the level difference of the signal level.

  Hereinafter, preferred filtering methods will be described with some examples.

(1) When the offset correction value β fluctuates momentarily by a large amount As described above, when the correction value fluctuates due to noise of the power supply supplied to the column amplifier group 130, wiring noise, disturbance noise, the cyclic coefficient is Stabilization of the correction value can be realized by performing the filtering process with a smaller size. However, if the value of the cyclic coefficient p is made too small, the value of the offset correction value βf (n) changes rapidly when the ideal value (correction target value) changes rapidly due to a rapid temperature change or switching of the driving method of the imaging device. Takes time to converge to near the ideal value. In addition, when the cyclic coefficient is increased, the reduction effect of the fluctuation of the correction value by the filtering process is weakened, and a frame in which a level difference of the signal level can be seen exists.

  Therefore, if the newly obtained offset correction value β (n) significantly fluctuates with respect to the offset correction value βf (n−1) after the filtering process in the previous frame, the cyclic coefficient p = 0 Set to As a result, the cyclic coefficient p is a set value that can sufficiently follow the variation of the ideal value, and also the rapid change of the correction value β due to disturbance noise etc. can be suppressed, and the influence of noise can be reduced. . In particular, disturbance noise may cause a very large level of fluctuation instantaneously, so optimization of the time for the offset correction value βf (n) after filtering to converge to near the ideal value, disturbance noise, etc. In some cases, stabilization of the offset correction value may not be compatible.

  FIG. 8A is a graph showing the fluctuation of the offset correction value β when disturbance noise occurs. The example shown in the figure shows the case where disturbance noise is generated for three frames as compared with the example shown in FIG. 7, but the other parts are the same as FIG. 7 and thus detailed description of the figure is omitted. Do.

  In FIG. 8A, there are three frames (13, 43, 73 frames) in which the value of the graph A becomes about 60 LSB higher than the value of the graph C which is the ideal value. Therefore, as shown in the graph B, although the filtering process is performed, a frame after ± 5 LSB is generated from the graph C which is the limit value after the process. That is, in 100 frames, a frame in which a level difference can be seen occurs only for a few frames, and thereafter, an image in which the level difference can not be seen is periodically repeated.

  Therefore, in the first embodiment, before performing the filtering process using the above equation (3), the newly obtained offset correction value β (n) and the filtering process result βf (n of the correction values up to the previous frame) Calculate the difference from -1). Then, it is determined whether the difference exceeds the threshold, and if it exceeds the threshold, control is performed to set the cyclic coefficient p = 0. In the first embodiment, the threshold is set to ± 30 LSB, but the threshold is set according to the noise level (random fluctuation due to the power supply voltage and pattern wiring) when disturbance noise is not generated and the set value of the cyclic coefficient. You should decide. Ideally, even if disturbance noise occurs, it is preferable to set the threshold value so that the offset correction value βf (n) after the filtering process does not exceed the limit value at which the step of the signal level starts to be visible on the image.

  The graph B in FIG. 8B shows the result of the filtering process when the threshold determination is added. In the figure, the threshold value is ± 30 LSB, and the cyclic coefficient p = 0 for three frames in which disturbance noise is generated, and the correction value βf (n−1) after the filtering process in the previous frame is taken over. By adding this process, all the offset correction values βf (n) after the filtering process fall within the range below the limit value at which the step of the signal level starts to be visible on the image.

  As described above, before performing the filtering process, the difference between the offset correction value βf (n−1) after the filtering process in the previous frame and the newly acquired offset correction value β (n) is obtained. Then, it is determined whether the difference value exceeds the threshold, and if it exceeds the threshold, the cyclic coefficient p is set to 0. This makes it possible to suppress the influence of the offset correction value βf (n) after the filtering process when a large change occurs instantaneously due to disturbance noise or the like. Furthermore, it is possible to achieve both optimization of the time until the offset correction value βf (n) after the filtering process converges to near the ideal value and stabilization of the correction value for disturbance noise and the like.

  Further, although disturbance noise is dealt with in the present embodiment, the present invention is applicable to all noise generating similar noises.

(2) When the power source is turned on and when the driving method of the imaging element is switched When the power is turned on, the output of the video signal is not stable and the driving method of the imaging element 1 is performed to perform various processing at the starting. Because switching may occur, the ideal value may change significantly. In addition, even when the driving method of the imaging device is switched after the power is turned on, the ideal value instantaneously changes largely immediately after the switching. Due to the above factors, the difference between the offset correction value βf (n) after filtering and the ideal value instantaneously increases.

  If the filtering process is not performed, it is possible to minimize the time until convergence to the vicinity of the ideal value. However, if the filtering process is not performed, after the offset correction value βf (n) converges near the ideal value, it is not possible to reduce the fluctuation of the correction value β (n) due to power supply noise or the like.

  Therefore, in the present embodiment, before performing the filtering process, the difference between the newly acquired correction value β (n) and the offset correction value βf (n−1) after the filtering process in the previous frame is obtained, and The cyclic coefficient is changed according to the magnitude of the difference value. Specifically, if the difference value is large, the cyclic coefficient is set higher, and if the difference value is small, the cyclic coefficient is set lower. As a method of determining the cyclic coefficient with respect to the difference value, it may be determined according to the magnitude of the change of the ideal value and the magnitude of the fluctuation of the correction value due to the power supply noise and the like.

  In the present embodiment, the table data shown in FIG. 9A is provided in advance, and with reference to this, the newly obtained offset correction value β (n) and the offset correction value βf after the filtering process in the previous frame A cyclic coefficient p corresponding to the difference value with n−1) is determined. The difference value shown in the table data of FIG. 9A is ideally (when there is no change in the correction value due to power supply noise etc.), the offset correction value shows a step of the signal level on the image within one frame. The set value is to converge below the limit value to be started. Further, in FIG. 9A, for convenience of explanation, only positive difference values are provided as table data, but negative difference values are also the same. Strictly speaking, absolute value | β (n) −βf (n− 1) |

  FIG. 9B shows the setting value of the cyclic coefficient based on the table data of FIG. 9A and the offset correction difference value β (n) −βf (n−1 before and after switching the driving method of the imaging device 1. It is a figure showing about change of). In the figure, the graph in the case where the setting value of the cyclic coefficient is changed according to the offset correction difference value is indicated by a solid line, and the graph in the case where the setting value of the cyclic coefficient is not changed is indicated by a broken line. In addition, 10 LSB, which is a limit value at which a signal level difference starts to appear on an image, is also described as a reference.

  In the same figure, before switching the driving method of the image pickup device 1, the offset correction difference value is in a state close to 0 LSB (strictly, fluctuation due to power supply noise and the like is superimposed). It is set to .1. In the period until the drive method of the imaging device 1 is switched, the cyclic coefficient p is always set to 0.1.

  Control at the time of switching the driving method of the imaging device 1 will be described. Since an offset correction difference value of 60 LSB is generated in the frame (first frame) immediately after the switching, the cyclic coefficient p = 0.9 is set based on the table data of FIG. 9A. By doing so, even in the frame immediately after the switching of the driving method, it is possible to obtain a correction value close to the ideal value after the switching of the driving method. In the present embodiment, it is possible to reduce to 10 LSB or less ideally (when there is no change in the correction value due to power supply noise or the like) by changing the cyclic coefficient.

  After the switching of the driving method, the offset correction difference value becomes 10 LSB in the second frame, so the cyclic coefficient p is set to 0.1 based on the table data of FIG. 9A. Also for the subsequent frames, if the offset correction difference value is 10 LSB or less, the cyclic coefficient p is always set to 0.1.

  On the other hand, when the driving method is switched while maintaining the setting of the cyclic coefficient p = 0.1, in order to cause the offset correction difference value to converge below the limit value at which the signal level difference starts to be visible on the image: 18 frames were required. However, the offset correction difference value is obtained, and by changing the cyclic coefficient p according to the magnitude, the value becomes less than the limit value after one frame, and after convergence near the ideal value, fluctuation of the correction value due to power supply noise etc. It can be reduced.

  As described above, the difference between the offset correction values β (n) and βf (n−1) is obtained before the filtering process at the time of power activation and when the driving method of the imaging device is switched, and the difference Change the cyclic coefficient according to the value of the value. Thus, even if the difference between the offset correction value βf (n) after filtering processing and the ideal value instantaneously increases, the convergence time of the offset correction value βf (n) can be shortened. In addition, after the offset correction value βf (n) converges near the ideal value, it is possible to reduce the fluctuation of the correction value caused by the power supply noise and the like.

(3) When Temperature Changes A temperature change of the imaging element 1 is one of the factors that cause the ideal value of the offset correction value to change. In particular, when the image pickup element 1 starts to be driven after the power is turned on, the electric power starts to be consumed in the image pickup element 1 so that the temperature rises rapidly, and to suppress this temperature rise, such as a fan or a peltier element The temperature may drop rapidly when cooling is performed using a cooling device. Such rapid temperature change may cause the ideal value of the offset correction value to change rapidly.

  Thus, when the ideal value changes rapidly, as described above, in order to suppress the fluctuation of the offset correction value β f (n) caused by the power supply voltage fluctuation, pattern wiring noise, etc. When processing is performed, there is a possibility that the ideal value can not be followed.

  Therefore, in the embodiment, a temperature change between frames is detected from a temperature measured using a thermistor, a temperature detection sensor or the like, and a process of changing a cyclic coefficient in accordance with a temperature change rate is performed. Specifically, if the rate of change of temperature is large, the cyclic coefficient is set high, and if the rate of change of temperature is small, the cyclic coefficient is set low. As a result, even if the ideal value changes rapidly due to a rapid temperature change, the offset correction value βf (n) follows the ideal value and the power supply does not change rapidly. The variation of the offset correction value β (n) due to noise or the like can be reduced.

  FIG. 10 is an image diagram showing setting values of the temperature, the temperature change rate, and the cyclic coefficient with respect to the time (frame) change. In the figure, the temperature indicates the value of the temperature acquired by a thermistor or a temperature detection sensor. The temperature change rate represents a temperature change amount (differential value of a graph of temperature) increased per unit time (in the present embodiment, a frame). In the present embodiment, the regions are divided into areas A to E in descending order of the temperature change rate. However, when control is actually performed, it is preferable to use a numerical value obtained by dividing the temperature change amount by unit time. The set value of the cyclic coefficient indicates the set value of the cyclic coefficient with respect to the temperature change rates A to E. In the present embodiment, the temperature change rate A = 0.5, the temperature change rate B = 0.4, the temperature change rate C = 0.3, the temperature change rate D = 0.2, and the temperature change rate E = 0.1. Is supported.

  In FIG. 10, since the temperature change rate is in the region of B during the period from immediately after the start to t1, the set value of the cyclic coefficient is set to 0.4. Since the temperature change rate is in the region of C during the period from t1 to t2, the set value of the cyclic coefficient is set to 0.3. In the period from t2 to t3, since the temperature change rate is in the region of D, the setting value of the cyclic coefficient is set to 0.2. After t3, since the temperature change rate is in the region of E, the set value of the cyclic coefficient is set to 0.1.

  As described above, when the rate of change of temperature is large, the cyclic coefficient is increased, and when the rate of change of temperature is small, the cyclic coefficient is decreased. As a result, even when the ideal value suddenly changes due to a rapid temperature change, the offset correction value βf (n) after the filtering process follows the ideal value. In addition, in the case where the ideal value does not change rapidly, it is possible to reduce the fluctuation of the offset correction value βf (n) due to power supply noise or the like.

  Although the example shown in FIG. 10 shows the case where the temperature rises, the same applies to the case where the temperature falls, and strictly speaking, the setting value of the circulation coefficient is set according to the absolute value of the temperature change rate. Do.

(4) Contrast of subject Although the level difference of the signal level is visible on the image when the offset correction value β (n) fluctuates, the conspicuousness changes depending on the contrast and the brightness of the subject. FIG. 11 shows a histogram representing the luminance distribution of an image of one frame, the horizontal axis representing the signal level, and the vertical axis representing the frequency (number of pixels) of luminance generated within the screen of one frame. Further, as described above, the determination level Vs represents a signal level at which the switching between the first ramp signal VRAMP (large) and the second ramp signal VRAMP (small) is performed.

  In an image to be a histogram as shown in FIG. 11A, a large number of pixels having luminance in the vicinity of the determination level Vs are present. That is, it is estimated that the possibility of the subject being low in contrast and having a gradually changing brightness around the luminance level near the determination level Vs is high. Therefore, it can be seen that such an image is an image in which the frequency of occurrence of pixels having luminance near the determination level Vs is high and the level difference of the signal level can be easily seen. For such an image, in order to stabilize the offset correction value βf (n) as much as possible, it is preferable to set a lower cyclic coefficient.

  On the other hand, in an image to be a histogram as shown in FIG. 11B, many pixels in the vicinity of the determination level Vs are not necessarily prominently present. That is, it is presumed that the subject is high in contrast, and has various luminances all over the screen. It can be seen that such an image is an image in which the frequency of occurrence of pixels having luminance in the vicinity of the determination level Vs is low and the level difference of the signal level is difficult to see. For such an image, it is less necessary to stabilize the offset correction value βf (n), and it is preferable to improve the followability to the ideal value by setting the cyclic coefficient higher.

  In the present embodiment, the optimum coefficient of the cyclic filter is determined according to the histogram (brightness distribution). Specifically, weighting is performed on pixels at the signal level in the vicinity of the determination level Vs, and by calculating a weighted integration value obtained by integrating only the number of occurrences of the weighted pixels, how many pixels in the vicinity of the determination level Vs are Find out if it exists. Then, the set value of the cyclic coefficient in the filtering process is determined according to the value of the weighted integrated value.

  FIG. 12A shows an example of a weighting factor used when performing weighting. The horizontal axis is the signal level, and the vertical axis is the weighting factor. In the figure, the coefficient of the signal level of the determination level Vs is 1.0, and the weighting coefficient is reduced as the difference of the signal level with respect to the determination level Vs increases. If the difference between the signal level and the weighting factor of the determination level Vs is large to some extent, the weighting factor is set to 0 because the possibility of being affected by the level difference of the signal level is low. When determining the signal level for setting the weighting factor to 0, the magnitude of the fluctuation of the offset correction value β (n) due to power supply noise or the like that may occur near the S signal level may be taken into consideration.

  A weighted integration value is calculated by integrating each pixel weighted by the above method by the number of pixels. In accordance with the calculated integrated value, the set value of the cyclic coefficient in the filtering process is determined.

  FIG. 12B is an image diagram showing setting values of cyclic coefficients with respect to weighted integrated values. The horizontal axis indicates the weighted integrated value, and the vertical axis indicates the set value of the cyclic coefficient. As shown in the figure, as the weighted integrated value increases, the setting value of the cyclic coefficient decreases.

  By performing the above processing, it is determined whether or not the step of the signal level is easily seen as a subject, and the setting value of the cyclic coefficient is changed according to the determination result. Thus, the stabilization of the offset correction value βf (n) and the followability to the ideal value can be optimized in accordance with the luminance distribution of the photographed subject.

  Further, although the present embodiment has been described using the histogram of the entire screen, the histogram may be created by dividing it into a plurality of areas in the screen. By doing so, the above-described optimization can be realized with higher accuracy even for a subject that is likely to have a step in a certain area in the screen and hard to occur in a certain area.

  As described above, weighting is performed on pixels at the signal level near the determination level, a weighted integration value is calculated by integrating the number of occurrences of the weighted pixels, and a cyclic coefficient is set according to the result of the integration value. Change the value Thereby, the stabilization of the correction value and the followability to the ideal value can be optimized according to the luminance distribution of the photographed subject.

(5) Frame rate As described above, the ideal value changes in accordance with the temperature change. In order to make the offset correction value βf (n) follow the ideal value, it is necessary to frequently acquire the offset correction value β (n) and update the offset correction value βf (n).

  However, at a low frame rate, acquisition of the offset correction value β (n) can not be frequently performed, and therefore, depending on the frame rate, the offset correction value βf (n) delays to follow the change in the ideal value. there is a possibility. In particular, when the fluctuation of the offset correction value β (n) due to power supply noise or the like is large, it is necessary to set the cyclic coefficient lower. Therefore, the offset correction value βf (n ) Is delayed. Therefore, in the present embodiment, the setting value of the cyclic coefficient is changed according to the frame rate.

  FIG. 13 shows setting values of the cyclic coefficient p with respect to the frame rate. As the frame rate is lower, the setting value of the cyclic coefficient p is larger, and as the frame rate is higher, the setting value of the cyclic coefficient p is smaller. That is, as the number of offset correction values β (n) that can be acquired per unit time decreases, the degree of influence of the newly acquired offset correction value β (n) on the offset correction value βf (n) after the filtering process increases. Become. Therefore, the lower the frame rate, the easier it is to follow fluctuations in the ideal value due to temperature changes. The range of the frame rate for each cyclic coefficient may be determined in consideration of the temperature change rate and the fluctuation of the offset correction value β (n) due to power supply noise or the like.

  As described above, by changing the setting value of the cyclic coefficient according to the frame rate, the followability to the change of the ideal value and the stability of the offset correction value even at the low frame rate where the update frequency of the correction value is low. Can be optimized.

(6) When there is a frame not used as an image Although the case where the cyclic coefficient is changed according to the frame rate has been described above, depending on the processing of the imaging device 1, the image is read from the imaging device 1; It may not be used. For example, it is assumed that an image is read from the imaging element 1 at 120 fps, and a video is output at 30 fps. In this case, of the four frames, only one image is used as an actual image.

  As described above, since acquiring the offset correction value as frequently as possible can realize stabilization of the correction value by the filtering process, the correction value β (n) is calculated also for an image of a frame not used as a video output. , Used for filtering processing. Specifically, the offset correction value of a frame not used as a video output in addition to the result of filtering processing using only the frame offset correction value β (n) used as a video output and the frame correction value β (n) Take the difference of the filtering result using β (n). Thereby, the stability of the offset correction value β (n) is determined. If the difference is large, it is determined to be unstable. If the difference is small, it is determined to be stable.

  According to the above determination result, when the degree of stability is high, the fluctuation of the correction value due to power supply noise or the like is small, so the cyclic coefficient is set high in order to accelerate the followability. On the other hand, when the degree of stability is low, since the fluctuation of the offset correction value due to power supply noise or the like is large, the cyclic coefficient is set to a lower value.

  In this embodiment, the offset correction value when the filtering process is performed using only the frames for video output is βf (n), and the offset correction value when the filtering process is performed using the frames for which the video output is not performed is βf ′. (N) Then, to determine the degree of stability, βf (n) −βf ′ (n) is calculated.

  In FIG. 14A, the offset correction value β (n) with respect to the time (frame) change and the offset correction value βf (n) when the filtering process is performed using only the image output image are not output It is the graph which showed offset correction value (beta) f '(n) when performing a filtering process also using the image, and (beta) f (n)-(beta) f' (n) showing stability.

  In the figure, the horizontal axis indicates time (in frame notation, a period is 1 to 100 frames), and the vertical axis indicates offset correction values β (n), βf (n), βf '(n), and offset correction difference values. (beta) f (n)-(beta) f '(n) is shown. In this embodiment, the offset correction value β f (n) when the filtering process is performed using only the image to be video output is set to have a high cyclic coefficient. On the other hand, the offset correction value βf ′ (n) when the filtering process is performed using an image which does not output a video is to obtain a correction value as stable as possible (because it becomes a reference for determining the stability), The cyclic coefficient is set to the lowest. As the difference value βf (n) -βf '(n) at this time is larger, βf (n) is less stable.

  In the present embodiment, as shown in FIG. 14B, the value of the multiplication value q for multiplying the cyclic coefficient p according to the magnitude of the difference value βf (n) −βf ′ (n) is table data , And based on the table data, the cyclic coefficient p is multiplied by the multiplication value q. By doing this, if the cyclic coefficient used to calculate the offset correction value βf (n) when the filtering process is performed using only the image to be output is not appropriate from the viewpoint of stability, the cyclic coefficient is It can be optimized. In FIG. 14B, only positive difference values are provided as table data for convenience of explanation, but negative difference values are also the same. Strictly speaking, absolute value | β f (n)-β f '(n It becomes |).

  As described above, in addition to the result of filtering using only the correction value β (n) of the frame used as the video output, and the correction value β (n) of the frame, the correction value β of the frame not used as the video output. (N) is also used to take the difference of the filtered result. Then, by determining the stability of the correction value of the frame used as the video output and changing the cyclic coefficient according to the stability, it is possible to optimize the cyclic coefficient.

(7) Movement of Subject The manner in which the level difference appears due to the fluctuation of the correction value β (n) due to power supply noise or the like changes depending on the subject. In particular, in the case of a moving image, it is easy to see that the subject has no motion, but when there is motion in the subject, the signal level of each pixel often changes constantly, making it less noticeable.

  Therefore, in the present embodiment, processing is performed to determine a cyclic coefficient according to the amount of movement of the subject. In order to detect the amount of movement, there are a generally used method of using a gyro sensor, a method of calculating a motion vector from the acquired image, and a method of detecting the amount of movement. The detection of the amount of movement of the subject is detected using a known technique, and the description thereof is omitted.

  FIG. 15 is table data representing a multiplication value q by which a cyclic coefficient is multiplied by the amount of movement of an object. In the present embodiment, based on the case where the amount of movement of the object is small (1.0 times), the cyclic coefficient is increased as the amount of movement of the object increases, and the offset correction value βf (n) Perform processing to make it easy to follow. In the present embodiment, the multiplication value q when the amount of movement of the subject is large is 4.0 times, but when the multiplication result of the cyclic coefficient p and the multiplication value q is larger than 1.0, , The multiplication result is 1.0. Although the movement amount is expressed as "large", "medium", and "small" in FIG. 15, the range of the movement amount may be appropriately divided according to the method of detecting the movement amount.

  In this way, the offset correction value β f (n) easily follows the ideal value under the condition that the level difference of the signal level is difficult to see as the object, and the offset correction value under the condition that the level difference is easily visible as the object. Fluctuation of βf (n) can be reduced. As a result, it is possible to optimize the followability of the offset correction value βf (n) to the ideal value and the reduction effect of the fluctuation of the offset correction value caused by the power supply noise and the like.

  As described above, by detecting the amount of movement of the subject and changing the cyclic coefficient according to the detected amount of movement, the ability to follow the ideal value of the offset correction value βf (n) according to the object, and the power supply The reduction effect of the fluctuation of the correction value due to noise or the like can be optimized.

(8) Spectral Characteristics of Color Filter As described above, the color filters used in the image pickup device generally have a periodic structure of Bayer arrangement using three primary color filters of R, G and B. It is. Of the pixel signals output from each pixel provided with the R, G, and B color filters described above, it is very sensitive to G pixels as human visual characteristics, while RB pixels are very sensitive. Is characterized by being less sensitive than G pixels. That is, while the step of the signal level looks noticeable to the G pixel, the RB pixel is less noticeable to the G pixel. As an example, G pixels start to stand out when the signal level difference exceeds ± 10 LSB, but for RB pixels, they do not stand out to ± 20 LSB, so there is a difference in visual level difference (limit value) It occurs.

  Therefore, in the present embodiment, different cyclic coefficients are set for each RGB pixel by multiplying the cyclic coefficient p of the R pixel and the B pixel by the multiplication value q with the G pixel as the reference (1.0). Do. Specifically, by setting the multiplication value q of the RB pixel to be larger than 1.0 with respect to the G pixel, the limit value at which the variation of the offset correction value βf (n) starts to show the step of the signal level on the image It is possible to accelerate the follow-up to the ideal value while maintaining the following condition. In particular, the offset correction value β (n) changes rapidly at the time of power supply start-up or switching of the driving method of the imaging device, etc. However, even the RB pixel with low sensitivity to the rapid change looks as a step of the signal level It will That is, it is preferable as the cyclic coefficient of the RB pixel to keep the tracking speed to the ideal value as fast as possible while suppressing the cyclic coefficient to the limit value (± 20 LSB) at which the step of the signal level starts to be visible on the image.

  FIG. 16 shows the multiplication value q to the cyclic coefficient for each color of the color filter. Since the G pixel is a reference, the multiplication value q of the G pixel is 1.0. Further, in the present embodiment, the multiplication value q is 2.0 for the R pixel and the B pixel. In this embodiment, although the multiplication value q of the RB pixel is 2.0 times, when the multiplication result of the cyclic coefficient p and the multiplication value q is larger than 1.0, the multiplication result is 1 .0.

  By doing this, while suppressing the variation of the offset correction value βf (n) below the limit value at which the level difference of the signal level starts to be visible on the image, the offset correction value βf can be quickly also against sudden changes in the ideal value. (N) can be made to follow.

  As described above, different cyclic coefficients are set for each of the R, G, and B pixels, and the cyclic coefficients are optimized for each pixel. As a result, while suppressing the variation of the offset correction value βf (n) of the pixels of each color below the limit value at which the step of the signal level starts to be visible on the image, the offset correction value can be quickly also against sudden changes in the ideal value. β f (n) can be made to follow.

(9) When the determination level is changed according to the gain switching of the column amplifier group 130: The gain of the column amplifier group 130 of the imaging device 1 is switched to improve the S / N ratio of the image signal at low illuminance. There is. By switching the gain of the column amplifier group 130, noise components generated in the circuits after the column amplifier group 130 can be made smaller relative to the pixel signal, thereby improving the S / N ratio. it can.

  In the present embodiment, the gains of the column amplifier group 130 assume analog gains and have discrete gain settings. Specifically, it has settings of 2x and 4x, and is usually 2x and is switched to 4x at low illumination.

  When a pixel signal having a signal level near the determination level Vs is amplified by switching the gain setting of the column amplifier group 130, it is necessary to change the determination level Vs to a high level to correspond to the amplification factor by the gain switching. . This is because the occurrence position of the level difference of the signal level in the screen instantaneously changes if the level difference is generated. In particular, when there are a large number of pixels in which the level difference of the signal level occurs after switching of the gain of the column amplifier group 130, the image becomes noticeable. In this embodiment, a method of setting a cyclic coefficient when changing the determination level Vs in accordance with gain switching of the column amplifier group 130 will be described.

  In the first embodiment, the set value of the multiplication value q for multiplying the cyclic coefficient p is changed according to the determination level Vs changed after the gain switching of the column amplifier group 130. FIG. 17 shows the multiplication value q by which the cyclic coefficient is multiplied with respect to the determination level Vs after gain switching of the column amplifier group 130. Here, the maximum range of AD conversion is 12 bits (0 to 4095 LSB), and the reference value (× 1.0) of the multiplication value q is the level difference level after the gain switching of the column amplifier group 130. Is the least noticeable 0 to 511 LSB range.

  When the determination level Vs after gain switching of the column amplifier group 130 is 2048 to 4095 LSB with respect to the reference value, the multiplication value q = 4.0 times when the determination level Vs is 512 to 2047 LSB, the multiplication value q = 2. Make it 0 times. That is, as the level difference becomes more noticeable, the cyclic coefficient is set to be higher, and the followability is improved. In this embodiment, when the signal level after column amplifier switching is 2048 to 4095 LSB, the multiplication value q is 4.0 times, but the multiplication result of the cyclic coefficient p and the multiplication value q is 1.0 If the value also becomes large, the multiplication result is set to 1.0.

  By doing this, when the determination level Vs after gain switching of the column amplifier group 130 is large, that is, when the step of the signal level of the S signal is noticeable, the offset correction value can be obtained by temporarily increasing the cyclic coefficient. Make β f (n) converge rapidly to near the ideal value. On the other hand, when the determination level Vs after gain switching of the column amplifier group 130 is small, that is, when the step of the signal level of the S signal is not noticeable, the variation of the offset correction value βf (n) is maintained by keeping the cyclic coefficient low. Reduce Also, the multiplication value may be applied only immediately after the gain switching of the column amplifier group 130.

  As described above, when the determination level is changed according to the gain switching of the column amplifier group 130, the setting value of the cyclic coefficient is set according to the magnitude of the determination level Vs set after the gain switching of the column amplifier group 130. change. Thus, the time until the offset correction value β f (n) converges to the vicinity of the ideal value and the reduction effect of the variation of the offset correction value due to noise are optimized while maintaining a state in which the level difference is hardly visible as an image. Can.

  As mentioned above, although the suitable filtering processing method was mentioned taking several examples and demonstrated, you may determine a circulation coefficient in consideration of these plurality. In addition, although the cyclic coefficient is shown as one example in the present embodiment, the method of setting the cyclic coefficient for each condition is not necessarily limited to this, and noise generation conditions, temperature change rates, drive switching of the imaging device It is good to decide according to the amount of change of the ideal value by.

Second Embodiment
Hereinafter, a second embodiment of the present invention will be described. Since the imaging device used in the second embodiment has the same configuration as the imaging device described with reference to FIG. 1 in the first embodiment, the description will be omitted, and only differences will be described.

  In the second embodiment, as shown in FIG. 18, the effective pixel area is divided into four in the horizontal direction with respect to the configuration of the pixel unit 110 of the first embodiment shown in FIG. 5. The effective pixel area A, the effective pixel area B, the effective pixel area C, and the effective pixel area D are sequentially arranged from the left of the screen. The dummy pixel area is also divided into four in the horizontal direction corresponding to the effective pixel area, and the dummy pixel area A, the dummy pixel area B, the dummy pixel area C, and the dummy pixel area D are sequentially arranged from the left of the screen. As described above, by dividing into several regions in the horizontal direction, even when the performance of the AD converter varies in the horizontal direction, correction closer to the ideal value can be performed.

  In the first embodiment described above, the inclination ratio α and the offset amount β are calculated using dummy pixels on the entire screen, but in the present embodiment, the above equation (3) is calculated for each of the divided areas A to D. Calculate α and β in). The offset correction value βf calculated for each of the regions is processed in the same manner as in the above-described first embodiment to make the level difference between signal levels invisible.

  As described above, the image is divided into a plurality of areas in the horizontal direction, and the offset correction value βf (n) is calculated for each area. Thus, even when the performance of the AD converter varies in the horizontal direction, it is possible to perform correction closer to the ideal value, and it is possible to obtain the same effect as that of the first embodiment.

  In the first and second embodiments, two types of ramp signals having different slopes are described, but the present invention is limited by the number of types of ramp signals because three or more types of slopes can be treated in the same manner. It is not something to be done. In addition, it is essential to perform the filtering process on the correction value calculated using the dummy pixel area, and the method of obtaining the correction value, the calculation method, and the like are the ones described in the first embodiment. Although it is a thing, it is not restricted to this.

  In the first and second embodiments, the imaging device 1 has a configuration in which one row ADC with a small circuit scale is provided for each row, but the present invention is not limited to this. . For example, each column includes a plurality of column ADCs, each of which performs AD conversion using ramp signals having different slopes, and selects one of them, or the configuration described in Japanese Patent Application Laid-Open No. 2013-009087. It may be According to Japanese Patent Application Laid-Open No. 2013-009087, in the column amplifier circuit in the imaging device, the first ramp signal and the second ramp signal having a smaller inclination than the first ramp signal for each pixel according to the level of the pixel signal. It is disclosed that AD conversion is selectively performed using any of the ramp signals. That is, the present invention can be applied as long as the image of each frame is configured by selectively using a pixel signal AD-converted by any of a plurality of ramp signals having different inclinations.

  1: Imaging device 2: Image processing unit 100: Timing control unit 110: Pixel unit 120: Vertical scanning circuit 130: Column amplifier group 140: Lamp circuit 150: Column analog-to-digital converter group (column ADC Groups: 151: Comparison unit, 152: Counter-latch circuit, 160: Horizontal transfer circuit, 170: Signal processing circuit, 180: External output circuit, 400: Constant voltage circuit

Claims (20)

A pixel unit in which a plurality of pixels are two-dimensionally arranged;
Analog-to-digital conversion means for converting an analog signal into a digital signal using a plurality of reference signals having different inclinations from each other;
Voltage supply means for supplying analog signals of a plurality of predetermined different output levels to the analog-to-digital conversion means;
Based on the plurality of digital signals obtained by converting each using the plurality of reference signals by an analog signal of the plurality of different power levels the analog-digital converter means supplied by said voltage supply means, said plurality of Calculating means for calculating a correction value for correcting an inclination and an offset of a digital signal obtained by converting the analog signal output from the pixel unit by the analog-to-digital converter using a reference signal ;
The calculation means performs filtering processing in which the offset correction value for correcting the offset is weighted and added by a cyclic coefficient, and the newly obtained offset correction value before filtering processing and the offset correction after filtering processing in the previous frame An imaging apparatus , wherein the cyclic coefficient is set to 0 when a difference from a value is larger than a threshold . The imaging apparatus according to claim 1, wherein the calculation unit calculates the correction value for each frame. When the offset correction value before the filtering process is β (n) , the offset correction value after the filtering process in the previous frame is βf (n−1), and the cyclic coefficient is p.
β f (n) = β (n) × p + β f (n-1) × (1- p), (0 p p 1 1)
3. The image pickup apparatus according to claim 1, wherein the offset correction value βf (n) after the filtering process is obtained by And offset correction value before the filtering process, when in the range in which the absolute value of the difference between the offset correction value after the filtering process in the preceding frame is predetermined, than smaller than the range, a large value The imaging apparatus according to any one of claims 1 to 3 , wherein Y is set to the cyclic coefficient. Further comprising temperature detection means for measuring the temperature;
When the absolute value of the rate of change of temperature measured by the temperature detecting means is within a predetermined range, a larger value is set as the cyclic coefficient than when it is smaller than the range. Item 4. The imaging device according to any one of Items 1 to 3 . Ago the contrast of one frame of the image output from the Kiga Motobu low, the imaging apparatus according to a value less than any one of claims 1 to 3, characterized in that setting the cyclic coefficient . If within a range in which the frame rate is predetermined for reading pre-outs Motobu, claims 1 to 6 than when higher than the range, and sets a larger value to the cyclic coefficient The imaging device according to any one of the above. An image obtained by reading a front Kiga Motobu, and a frame with and without frame used as a video output,
When the absolute value of the difference between the offset correction value calculated in the frame used as the video output and the offset correction value calculated in the frame not used as the video output is within a predetermined range The imaging apparatus according to any one of claims 1 to 7 , wherein the cyclic coefficient is multiplied by a larger value than when larger than the range. The apparatus further comprises motion amount detection means for detecting the motion amount of the subject,
If within a range that the amount of motion detected by the detecting means of the movement amount is predetermined, according to claim 1 to 8 than smaller than said range, characterized by multiplying the larger value to the cyclic coefficient The imaging device according to any one of the above. Before outs Motobu is covered by primary color filters of the Bayer array,
The correction value for correcting the digital signal corresponding to the pixel covered by the red or blue filter is larger than the correction value for correcting the digital signal corresponding to the pixel covered by the green filter. The imaging apparatus according to any one of claims 1 to 9 , wherein the imaging apparatus is calculated by multiplying a cyclic coefficient. The analog signal output from each pixel constituting the front Kiga Motobu, further comprising amplifying means for amplifying before conversion by the analog-digital converting means,
The analog-to-digital converting means compares the predetermined judgment level output level of the analog signal output from each pixel constituting the front Kiga Motobu,
When the amplification factor by the amplification means changes, the value obtained by changing the determination level to correspond to the change in amplification factor is larger than that in the case where it is smaller than the predetermined range. The imaging apparatus according to any one of claims 1 to 10 , wherein the value is multiplied by the cyclic coefficient. The analog-to-digital converting means, when the output level of the analog signal output from each pixel constituting the front Kiga Motobu is smaller than a predetermined determination level, the digital signal converted using the first reference signal 12. The digital signal according to any one of claims 1 to 11 , wherein the digital signal converted by using a second reference signal having a larger inclination than the first reference signal is output when the output level is higher than the determination level. An imaging device according to claim 1. Using the correction value calculated by the calculation means, the analog signals output from the front Kiga Motobu, further comprising a correction means for correcting the digital signals obtained by analog-digital conversion The imaging device according to any one of claims 1 to 12 . Before outs Motobu has a dummy pixels not including the photoelectric conversion element, said voltage supply means, the reading period of the dummy pixels, supplying a plurality of different output levels analog signal to a predetermined The imaging device according to any one of claims 1 to 13 , characterized by Before outs Motobu is divided into a plurality of regions, the calculation means according to any one of claims 1 to 14, wherein the calculating the correction value for each of a plurality of regions of the divided Imaging device. The analog-to-digital conversion means, the image pickup apparatus according to any one of claims 1 to 15, characterized in that it is arranged in each column of pre-outs Motobu. A control method of an imaging device having a pixel unit in which a plurality of pixels are two-dimensionally arranged,
Voltage supplying means for supplying analog signals of a plurality of different output levels determined in advance to the analog to digital conversion means;
An analog-to-digital conversion step of converting the plurality of different output level analog signals supplied in the voltage supply step into a plurality of digital signals using a plurality of reference signals having different inclinations; ,
Calculating means, on the basis of the analog plurality of digital signal converted by the digital conversion process, and calculates the ratio and the offset amount of the plurality of different slopes, on the basis of the ratio and the offset amount of the tilt, the Calculating a correction value for correcting an inclination and an offset of a digital signal obtained by converting the analog signal output from the pixel unit using the plurality of reference signals by the analog-to-digital converter; And
In the calculation step, filtering processing is performed in which the offset correction value for correcting the offset is weighted and added by a cyclic coefficient, and the newly obtained offset correction value before filtering processing and the offset correction after filtering processing in the previous frame The control method of an imaging apparatus , wherein the cyclic coefficient is set to 0 when a difference from a value is larger than a threshold . The method according to claim 17 , wherein in the calculating step, the correction value is calculated for each frame. Correction means using the correction value calculated in the calculating step, further comprising a correction step of correcting the digital signals obtained an analog signal output from the front Kiga Motobu by analog-to-digital converter The control method of the imaging device according to claim 17 or 18 , characterized by Before outs Motobu is divided into a plurality of regions, and in the calculating step, according to any one of claims 17 to 19, and calculates the correction value for each of a plurality of regions of the divided Control method of imaging device.

Images