JP2007173986A - Imaging apparatus and control method thereof, computer program, and storage medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent the occurrence of a difference from a storage time within a succeeding frame even when a drive method is switched so as to maintain the quality of an output image. <P>SOLUTION: When the drive method is switched in an imaging apparatus disclosed herein using a CMOS sensor, a gain of a pixel value reset in the preceding drive method is corrected on the basis of a ratio of the storage time given to the pixel to the storage time ordinarily given to the pixel so that the variations in the storage time in one frame are eliminated by the gain correction. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a solid-state imaging device having a photoelectric conversion element, and more particularly to an electronic zoom image quality improvement when a CMOS sensor is used.

  A configuration for electronic zoom processing in a CCD camera is, for example, as shown in FIG. In FIG. 1, 100 is a lens, 101 is a charge coupled device (CCD), 102 is a correlated double sampling circuit (CDS), and 103 is a clamp circuit (CLP). Reference numeral 104 denotes an analog / digital converter (A / D), 105 denotes a frame memory, 106 denotes a zoom controller, 108 denotes a pixel complementing unit, and 107 denotes an image output.

  In such a configuration, the optical image passes through the lens 100 and forms an image on the light receiving surface of the CCD 101 which is an image sensor. The optical image formed on the light receiving surface of the CCD 101 is converted into photoelectric charges by a two-dimensionally arranged photoelectric conversion unit and sequentially transferred to the output. The correlated double sampling circuit 102 removes reset noise peculiar to the CCD from the output signal of the CCD 101, and the reset noise is removed to form a sampled and held video signal.

  The clamp circuit 103 performs dark level clamping, and the AD converter 104 converts the input analog signal into a digital signal. The frame memory 105 is a memory for recording all pixel data of one frame. The zoom controller 106 reads out only a partial area from the center of the CCD when, for example, double-zoom image data is desired.

  In recent years, CMOS imaging sensors have been increasingly used for reasons such as requiring no inexpensive and complicated timing generation circuit, operating with a single power source and low power consumption. Further, the CMOS image sensor has a feature that the CCD image sensor does not have, that can capture only an arbitrary area of the CMOS image sensor as an image.

  Next, a description will be given of a high-quality electronic zoom method of a CMOS image sensor that can read out an arbitrary region (see Patent Document 1). FIG. 2 is a conceptual diagram of a CMOS electronic zoom operation. FIG. 2A shows reading in the normal mode, and FIG. 2B shows reading in the zoom mode. In the normal mode, for example, a value obtained by adding four pixels of the solid-state imaging device is read as a pixel value for one pixel. In the range of the thick frame in FIG. 2A, a value obtained by adding the pixel and the four pixel values on the right, diagonally lower right, and lower is read as a pixel signal indicated by filling. That is, a 4 × 4 pixel sentence signal is read out from a vertical and horizontal 8 × 8 pixel range.

On the other hand, in the zoom mode, a continuous 4 × 4 pixel region (filled portion) in the center within the thick frame range of 8 × 8 pixels is read as it is without addition. In this case, the central portion within the thick frame range can be enlarged and displayed. In addition, the number of pixels to be read is the same as that in the normal mode, and it is not necessary to increase the number of pixels by signal processing. As a result, the image quality at the time of electronic zoom can be improved.
JP 2001-78081 A

When photoelectric charges are accumulated in a block-readable CMOS sensor, since the accumulation start timing is controlled for each line, the time for accumulating the photoelectric charges for each line is shifted. This time lag for each line is shifted by the time necessary to read out one line. The time required to read one line can be calculated by the following equation.
1 line readout time = HBLK x α + Skip x β + number of horizontal pixels x reference clock time Formula (1)
(α and β are values determined by the vertical addition method)
As an example, FIG. 3 shows driving for performing vertical two-pixel addition averaging. In this case, α is 2 and β is 1. That is, the readout time for one line is obtained by the sum of the following times. That is, the time HBLK required to transfer the first line, the time Skip required to skip the second line, the time HBLK required to transfer the third line, and the first and third lines are added. This is the time required to transfer the averaged pixels for the horizontal pixels.

  Further, the time required to transfer the horizontal pixels also depends on the reference clock time (drive frequency). That is, one line read time varies depending on the vertical direction addition method and the drive frequency. As a result, the difference between the accumulation start times at the top and bottom of the screen varies depending on the change in the imaging drive.

  FIG. 4 is a diagram for explaining a shift in accumulation time based on one line reading time. 4 (a) and 4 (b), the one-line read time is longer in FIG. 4 (b). Here, the driving method as shown in FIG. 4A in which the one-line readout time is shorter is called “imaging drive A”, and the driving method as shown in FIG. This is referred to as “imaging drive B”. In the imaging drive A, the shift in the accumulation time between the top and bottom of the screen is smaller than that in the imaging drive B.

  Next, the case where the drive changes in EVF, moving image, etc. will be described with reference to FIG. In FIG. 5, reference numeral 501 denotes the sum of the accumulation time and readout time of the pixels of the frame driven by the imaging drive A. In FIG. 5, for the line from which the previous frame has been read, photocharge accumulation for the next frame has started, and the read period for the previous frame and the accumulation start timing for the next frame overlap. Reference numeral 502 denotes the total of the accumulation time and the readout period when the imaging drive A is switched to the imaging drive B at time t1. This drive switching timing is performed during a VBLK (vertical blanking) period after the readout of the pixels in frame 2 is completed. That is, the accumulation start timing corresponding to the imaging drive A is set for the line before the drive switching, and the accumulation start timing corresponding to the imaging drive B is set for the line after the switching.

  In the example shown in FIG. 5, the driving is performed by the imaging drive A before the time t1, but is switched to the imaging driving B at the time t1. Then, since the reset start time of the frame 3 is before the time t1, the inclination based on the shift of the accumulation start timing corresponds to the imaging drive A in the period before the time t1. However, since the time is switched to the imaging drive B after the time t1, the readout time is different from the imaging drive A. As a result, regarding the line where reset is started after time t1, the reset start timing shift corresponds to the imaging drive B, and the inclination based on the shift is different. In this case, if an attempt is made to maintain the frame rate, a difference in accumulation time occurs between the upper and lower frames of the frame 3.

  As described above, as an example, if the driving method is switched when the readout period of the previous frame and the accumulation period of the next frame overlap, there is a difference in the accumulation time within the next frame. It will occur and the quality of the output image will be degraded.

  SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to prevent the difference in accumulation time within the next frame even when the driving method is switched and to maintain the quality of an output image.

  The present invention for solving the above-described problems includes an imaging unit that includes a plurality of lines including a plurality of pixel circuits including a light receiving unit that generates and accumulates charges according to an incident light amount, and generates image data. A first drive for sequentially resetting the light receiving unit for each predetermined line at a first period interval, and a second drive for sequentially resetting for each predetermined line at a second period interval longer than the first period interval. Resetting means for performing, and driving for reading out, as a pixel value, the electric charge accumulated in the light receiving unit for each predetermined line at a first time interval after a predetermined accumulation time has elapsed since the start of the reset in the first driving. The charge accumulated in the light receiving unit for each predetermined line at the second period interval after the predetermined accumulation time has elapsed from the start of the reset in the second drive is defined as a pixel value. Reading means for performing read-out driving and switching accepting means for accepting an instruction to switch from the first drive to the second drive, and finishes reading of pixel values to be read in one frame by the read-out means. In the case where the reset unit has received the switching before the reset unit has reset the predetermined line in the subsequent frame in the subsequent drive, the reset unit is After the reading of the pixel value to be read is completed, the reset operation for the predetermined line in the subsequent frame is switched to the second drive, and the shift in the accumulation time of the plurality of lines included in the subsequent frame is corrected. And a gain correction means for correcting the gain of the pixel value of the subsequent frame. And wherein the door.

  According to the present invention, even when the driving method is switched, it is possible to prevent a difference in accumulation time in the next frame and maintain the quality of the output image.

  Embodiments of the present invention will be described in detail with reference to the drawings.

[First Embodiment]
In the imaging apparatus in FIG. 6, light from the subject passes through the diaphragm blade 1 and forms an image on the imaging element 4 by the lens 2, thereby performing photoelectric conversion. The filter group 3 is configured by combining an optical low-pass filter that cuts a high frequency range of light, a color correction filter, a filter for cutting infrared rays, and the like to prevent moire and the like.

  The optical signal converted by the image sensor 4 is subjected to two-dimensional pixel position selection by the X address selection unit 6 and the Y address selection unit 5 based on a signal from the address designation unit 8, and is read to the timing adjustment unit 7. In the timing adjustment unit 7, timing adjustment of outputs (one to a plurality) from the image sensor 4 is performed.

  The signal output from the timing adjustment unit 7 is controlled in voltage by an AGC (auto gain control) 10 and converted into a digital signal by the A / D converter 11. The camera DSP 12 performs image processing of moving images or still images. The MPU 14 sets parameters used for the image processing in the camera DSP 12, and performs AE and AF processing.

  The AF control is performed by moving a focus lens (not shown) back and forth by the focus motor 51. The DRAM 13 is used as a temporary storage area for image processing, and the image recording medium 18 is used as a nonvolatile storage area. The image recording medium 18 is, for example, a smart medium, a magnetic tape, or an optical disk. In order to perform display after the image processing, a video encoder 15, a CRT 16, and the like are provided.

  Further, the viewfinder 17 is an LCD or the like, and is used for confirming the subject before storing it in the image recording medium 18. These output units are not limited to the CRT 16 and the viewfinder 17, and a printer or the like may be used.

  Next, the image sensor 4 corresponding to the present embodiment is composed of one pixel circuit and a readout circuit. First, the configuration and operation of the one pixel circuit will be described with reference to FIG.

  In FIG. 7, reference numeral 158 denotes a potential barrier for transferring charges accumulated in a photodiode (hereinafter referred to as PD) 150 to a floating diffusion (hereinafter referred to as FD) in which the gate of the amplification MOS transistor 160 is a floating structure. This is a MOS transistor for an operation transfer gate.

  Reference numeral 157 denotes a reset MOS transistor for resetting the charge of the PD 150. A line selection MOS transistor 159 is provided. The gates of these MOS transistors are connected to a transfer signal line 153 that transfers the charge of the PD 150, a reset signal line 156 that resets the FD, and a selection signal line 152, respectively.

  Here, the charge accumulated in the PD 150 is first transferred through the MOS transistor 158 selected by the transfer signal line 153 to the FD that is reset by turning on the reset transistor 157 by the reset signal line 156. Next, the signal is amplified by the source follower MOS transistor 160 via the selection MOS transistor 159 selected by the selection signal line 152 and read to the read line 154.

  In the present embodiment, when performing an electronic zoom operation using an imaging region composed of such a set of one-pixel circuits, in the zoom mode (tele), the pixel value is not added and a part of the imaging region is added. Are read out pixel by pixel. Alternatively, the pixel value is added with a smaller number of additions than in the normal mode (wide) described later. For example, when an imaging region 1301 as shown in FIG. 12 indicates the entire region that can be imaged by the imaging device 4, readout is performed for pixels included in the region 1302 in the region 1301 in the zoom mode. . Such an imaging operation is referred to as “imaging operation A” in the present embodiment. On the other hand, in the normal mode (wide), a predetermined number of pixels (for example, 2 × 2 = 4 pixels) are added and read out with respect to the entire imaging region 1301. Such an imaging operation is referred to as “imaging operation B” in the present embodiment.

  In addition, the number of added pixels in the imaging operation B is not limited to 2 lines (4 pixels), and may be 4 lines (16 pixels) addition. In that case, in the imaging operation A, no addition is performed. Two lines (four pixels) may be added.

  When non-addition readout is performed in the imaging drive A, the pixel value for one line included in the region 1302 is stored in the storage capacitor in the readout circuit, and thereafter, the period from readout to the horizontal output line without addition is 1 It becomes a horizontal period (horizontal period A). On the other hand, in the imaging drive B, the pixel values for two lines are sequentially held in the holding capacitor in the reading circuit, and then added and read out from the horizontal output line. Therefore, the period until the pixel value after addition is read is one horizontal period (horizontal period B). That is, the horizontal period A is shorter than the horizontal period B.

  Next, with reference to FIG. 8, a circuit configuration including a pixel portion 801 in which a plurality of one-pixel circuits in FIG. 7 are arranged and a readout circuit 802 will be described. In FIG. 8, only 2 × 2 pixels are shown for simplicity.

  First, when pixel values are not added in the image pickup drive A (non-addition control), the charge of the PD 150-1 is caused by the conduction of the MOS transistor 161-1 shown in FIG. The charge accumulates to -1. Similarly, the charge of the PD 150-2 is read out to the capacitor 164-1 during the read control of the signal lines 156-1, 153-1, 152-1 and the PD 150-1 of 169. Subsequently, when the signal line 167-1 and the signal line 167-2 are alternately turned on, the imaging signals of the PD 150-1 and PD 150-2 are sequentially read out through the amplifier 171. The operation in the vertical direction is performed by the same operation as described above from the control of 156-2, 153-2, and 152-2.

  Next, when pixel values are added in the imaging drive B (addition control), the PD 150-1 and PD 150- are controlled by the control of the signal lines 156-1, 153-1, 152-1 and 169 shown in FIG. 2 charges are accumulated in the capacitors 162-1 and 164-1, respectively. Subsequently, under the control of the signal lines 156-2, 153-2, 152-2, and 170, the charges of the PD 150-3 and PD 150-4 are accumulated in the capacitors 162-2 and 164-2, respectively. Thereafter, when the signal line 167-1 and the signal line 167-2 are turned on at the same time, an imaging signal obtained by adding the charges of the PD 150-1 to PD 150-4 is read through the amplifier 171.

  With the above configuration, it is possible to perform operations in the imaging drive A and the imaging drive B.

  In the imaging drive A, the pixel value is read by selecting a part of the imaging region 1302. A configuration for selecting such a partial area will be described with reference to FIG.

  FIG. 13 is a diagram showing in more detail the configuration of the image sensor 4, the Y address selection unit 5, and the X address selection unit 6 corresponding to the present embodiment.

  In FIG. 13, the image pickup device 4 includes an image pickup area 1401 constituted by 8 × 8 pixels and a readout circuit 1403 corresponding to eight pixels in the horizontal direction. The imaging area 1402 is a 4 × 4 pixel area in the imaging area 1401, and in the imaging drive A, the pixel value of the imaging area 1402 is read out without addition. On the other hand, in the case of the imaging drive B, the pixel value of the imaging area 1401 is added and read out. Here, the addition is 2 × 2 pixel addition.

  The Y address selection unit 5 includes a vertical decoder unit 1404 and a vertical shift register 1405, and the X address selection unit 6 includes a horizontal decoder unit 1406 and a horizontal shift register 1407. VD0 to VD1 are input to the vertical decoder 1404, and the vertical shift register unit 1405 can receive a clock pulse (CLK) and a vertical reset pulse (VRES). A selection signal 152, a transfer signal 153, and a reset signal 156 are output from the vertical shift register unit 1405. Also, HD0 to HD1 are input to the horizontal decoder 1406, and a clock pulse (CLK) and a horizontal reset pulse (HRES) are input to the horizontal shift register unit 1407.

  The vertical decoder 1404 and the horizontal decoder 1406 are used to determine whether to select the entire imaging area 1401 or a part of the imaging area 1402. Regarding the selection of the imaging region, the horizontal portion and the vertical portion function in substantially the same manner, so the horizontal direction will be described below.

  FIG. 14 shows an example of the configuration of the horizontal decoder unit 1406 and the horizontal shift register unit 1407.

  First, the inputs HD0 to HD1 of the horizontal decoder unit 1406 are two (bit). On the other hand, since the imaging area 1401 is divided into three pixels of 2 pixels, 4 pixels, and 2 pixels in the horizontal direction, when the imaging area 1402 is selected, the central 4 pixels are selected. do it. Therefore, as a combination of (HD0, HD1), for example, the case where the first two pixels are selected is (0, 0), the case where the next four pixels are selected is (0, 1), and the last two pixels are The case of selection can be (1, 0).

  The horizontal shift register unit 1407 is located between the horizontal decoder unit 1406 and the imaging region 1401 and shifts a signal obtained from the horizontal decoder unit 1406 by the clock pulse CLK. At the same time, a signal 167 is output to the pixel circuit in the imaging region 1401 to read out the pixel value to the horizontal output line. When stopping scanning, the contents of the horizontal shift register unit 1407 can be erased by the horizontal reset pulse HRES.

  In FIG. 14, the horizontal decoder unit 1406 has HD0 as a lower digit and HD1 as an upper digit as inputs, and is composed of inverters 1501 and 1502 and AND circuits 1503 and 1504. The horizontal shift register unit 1407 includes D-type flip-flops 1501-1 to 1505-5 and OR circuits 1506-1 and 1506-2. Note that FIG. 14 is illustrated in a simplified manner to explain a case where four pixels corresponding to the imaging region 1402 are selected from the eight pixels in the horizontal direction. Accordingly, only a part of the flip-flops in the horizontal shift register unit 1407 are described, and a configuration in which 8 pixels are originally arranged.

  In addition to the configuration shown in FIG. 14, elements other than AND and an inverter may be used as the circuit of the horizontal decoder unit 1406, or the configuration of the horizontal shift register unit 1407 may be configured by a clocked inverter as in the conventional example. .

  When (0, 1) is input to (HD0, HD1) as the input of the horizontal decoder unit 1406, a signal 167 is output to the readout circuit for the leftmost pixel among the four pixels in the imaging region 1402. At the same time, the flip-flop 1505-1 (FF1) is selected. Thereafter, FF2 (1505-2), FF3 (1505-3), and FF4 (1505-4) are sequentially shifted every clock by the clock pulse CLK. The output of FF4 (1505-4) is input to FF5 (1505-5) via OR circuit 1506-2. Therefore, when only four pixels corresponding to the imaging region 1402 are selected, it is only necessary to input the reset pulse HRES to the AND circuit 1504 without inputting (1, 0) as (HD0, HD1). In order to further select two pixels subsequent to the four pixels, (1, 0) may be input as (HD0, HD1), and the output of the AND circuit 1504 may be set to 1.

  By scanning the vertical shift register unit 1405 in the same manner, a 4 × 4 pixel region 1402 in the imaging region 1401 can be selected.

  By doing so, it is possible to select only a part of the imaging area out of the entire imaging area.

  FIG. 15 shows a timing chart of control signals corresponding to FIG. Here, reference numeral 1601 denotes a readout period in the imaging drive B, and reference numeral 1602 denotes a readout period in the imaging drive A.

  Reference numeral 1603 denotes one horizontal period in the imaging drive B, which is the time required to read out pixel values from two sets of selection signals, for example, 152-1 and 152-2, and add them and read out from the horizontal output line. is there. On the other hand, 1604 represents one horizontal period in the imaging drive A, and is the time required to read from the horizontal output line after reading the pixel value with a single selection signal line, for example, 152-3. Here, one horizontal period 1603 is longer than one horizontal period 1604. Further, the reading period 1601 and the reading period 1602 are configured as a set of such one horizontal period 1603 and one horizontal period 1604. Therefore, as shown in FIG. 15, when the number of pixels read from the horizontal output line is the same, the reading period 1601 becomes longer than the reading period 1602.

  Next, processing in the present embodiment will be described. First, when a change in the electronic zoom magnification is accepted, the imaging drive method is changed accordingly. Correction is performed in order to compensate for the in-frame accumulation time shift that occurs at that time. In the present embodiment, gain correction is performed with a gain correction value for a line in which a light and dark difference occurs. In the present embodiment, when the drive method is changed in response to accepting the change of the electronic zoom magnification, the drive method is changed during the VBLK period immediately after the reading is completed.

  In the present embodiment, the gain correction value can be calculated based on the accumulation time and read time corresponding to the driving method before and after the change. FIG. 9 is a diagram for explaining a method of calculating the gain correction value.

  FIG. 9A shows three cases as the accumulation time. In the accumulation time 1, the reset start timing of each pixel circuit is set as indicated by a thick line 901. In this case, the reset start timing before time t1 corresponds to the imaging drive A, and the reset start timing after time t1 corresponds to the imaging drive B. The accumulation time 1 is given as a difference between the reset start timing 901 and the read timing 904. Note that the time t1 is a time point when reading of all pixel values in the frame 910 in which reading is performed by the drive A is completed.

  Next, in the accumulation time 2, the reset is started at a timing later than the accumulation time 1, and the reset start timing is set as indicated by a thin line 902. Again, the reset start timing is changed around time t1. The accumulation time 1 is given as a difference between the reset start timing 902 and the read timing 904.

  Further, at the accumulation time 3, the reset is started after the time t1, and the reset start timing is set as indicated by a one-dot chain line 903. The reset start timing here corresponds to the imaging drive B as a whole. The accumulation time 3 is given as a difference between the reset start timing 903 and the read timing 904.

  In this embodiment, in order to correct variations in pixel values within one frame in these three accumulation times, gain adjustment according to the difference in accumulation time is performed in this embodiment.

  Prior to time t1, the inclinations of the reset start timings 901 and 902 are inclinations corresponding to the imaging drive A and are known. Therefore, it is possible to count the number of lines where reset starts before time t1 elapses. Further, the accumulated time 1 and accumulated time 2 in each counted line are calculated by the difference between the reset start timings 901 and 902 and the read timing 904, respectively. Therefore, the gain correction value (ga) for correcting the pixel value before time t1 can be obtained as a value obtained by dividing the reference accumulation time 905 in one frame by the accumulation time 1 or the accumulation time 2. Assuming that the gain correction value in the case of the accumulation time 1 is ga1 and the gain correction value in the case of the accumulation time 2 is ga2, it can be calculated as follows.

ga1 = (reference accumulation time 1) / accumulation time 1
ga2 = (reference accumulation time 2) / accumulation time 2
On the other hand, when the accumulation time is shorter than the predetermined value 906 in FIG. 9, no gain correction is performed. This is because the driving method has already been switched because the reset operation is started after the reading of the previous frame is completed. Therefore, no gain correction is performed for the accumulation time 3. Thereby, it is possible to minimize the deterioration of S / N.

  The correspondence relationship between the gain correction value (ga) thus obtained and the line number (I) of the vertical line of one frame is as shown in FIG. In FIG. 9B, in the case of the accumulation time 1, any value on the straight line indicated by 907 is applied as the gain correction value from the top of the screen (I = 1) to the line number I1. “1” is applied to the gain correction values after the line number I1. In the case of the accumulation time 2, any value on the straight line indicated by 908 is applied as the gain correction value from the top of the screen (I = 1) to the line number I2. “1” is applied to the gain correction values after the line number I2. In the case of the accumulation time 3, the gain correction value is always “1” (909).

  In this embodiment, the gain correction value can be set for each color. This is because there is a concern that, for example, the black level changes due to a shift in the accumulation time for each line. As a result, since the color balance composed of the three colors of red (R), green (G), and blue (B) is shifted for each line, the color balance is corrected simultaneously with the luminance correction.

  The gain correction value calculation method for each color will be described with reference to the flowchart of FIG. Note that the processing corresponding to the flowchart of FIG. 10 is executed in the camera DSP 12.

  First, in step S1001, the line number (I) for calculating the gain correction value is reset. In the following step S1002, it is determined whether or not the line number I matches N, which is the number of lines to be corrected. The number of lines N is a value corresponding to the number of lines for which the reset operation was started before time t1 in FIG. If the line number I matches N (“YES” in step S1002), the calculation process has been completed for the line for which the gain correction value needs to be calculated, and thus this process ends. On the other hand, if N does not match (“NO” in step S1002), the process proceeds to step S1003.

  In step S1003, an OB (Optical Black) value is calculated for the line (I) that is the target for calculating the gain correction value. The OB value is a value for correcting the black level of the pixel value, and can be calculated based on the pixel value of the black level detection pixel of the line (I). In step S1004, the reciprocal of the current WB (white balance) coefficient is calculated and converted into a pixel value. In step S1005, an OB value used for signal processing (referred to as “reference OB value”) is added to calculate a pixel value before OB subtraction. The reference OB value is an OB value that is commonly used for pixels in one frame. In the case of the present embodiment, the pixels for which the reset operation has been started before time t1 have different accumulation times, so the reference OB value cannot be applied as it is. Therefore, in step S1005, the value before the subtraction by the OB value is restored.

  In step S1006, the line OB value calculated in step S1003 is subtracted. In step S1007, the WB coefficient is recalculated by obtaining the reciprocal of the newly calculated pixel value. Subsequently, in step S1008, the ratio between the WB coefficient calculated in step S1007 and the WB coefficient used in common for the pixels in one frame is calculated and multiplied by the gain correction value calculated from the accumulation time. . Thereby, a gain correction value for each color is calculated. In step S1009, I is incremented, and the process returns to step S1002.

  The above processing is performed for all lines for which gain correction is performed. By such processing, a gain correction value for each line can be calculated.

  As described above, in this embodiment, even when the imaging drive is switched during shooting, the difference in brightness between the upper and lower portions of the screen can be compensated by applying gain correction to lines having different accumulation times. Further, it is possible to calculate the gain correction value for each color, and to maintain the color balance by suppressing the change in the black level.

[Second Embodiment]
In the first embodiment described above, the case has been described in which the brightness difference between the top and bottom of the screen is completely corrected using the calculated gain correction value. However, such a correction method has a problem that the S / N ratio of the line subjected to gain correction is deteriorated.

  Therefore, in this embodiment, complete correction is performed only when EVF display or the like is not stored in a recording medium, and in the case of moving image shooting or the like, a limit value is provided for the correction gain, and the trade-off between the upper and lower brightness and the S / N ratio is made. The correction is performed in consideration of the above.

  FIG. 11 shows the relationship between the gain correction value and the line number corresponding to this embodiment. In FIG. 11, reference numeral 1101 denotes a transition of gain correction values applied to pixels up to the line n before the limiter is applied. Reference numeral 1102 denotes a transition of the gain correction values applied to the pixels up to the line n, to which the limiter is applied. As described above, in this embodiment, when the gain correction value exceeds the limiter value 1103 corresponding to the predetermined threshold, the gain correction value is set to the limiter value 1103. That is, in the case of FIG. 11, the gain correction value is set to the limiter value 1103 from the top of the screen to the line n. Thereby, the deterioration of the S / N ratio in the said area | region can be suppressed.

  Further, the calculated gain correction value can be applied as it is to the line n to line m in FIG. 11 where the gain correction value is below the limit value 1103 that is a predetermined threshold value. Since the reference accumulation time is secured from the line m to the bottom of the screen, the gain correction value is 1, that is, no gain correction is performed.

  As described above, in the present embodiment, correction can be performed in consideration of the trade-off between the brightness of the upper and lower portions of the screen and the S / N ratio when shooting a moving image.

  In order to completely eliminate the adverse effect of the S / N ratio, a frame in which an accumulation time difference is generated at the top and bottom of the screen due to switching of the imaging drive may be replaced with a frame before the imaging driving is switched.

[Other Embodiments]
Note that the present invention can be applied to a system including a plurality of devices (for example, a host computer, an interface device, a reader, and a printer), and a device (for example, a copying machine and a facsimile device) including a single device. You may apply to.

  The object of the present invention can also be achieved by supplying a storage medium storing software program codes for realizing the above-described functions to the system, and the system reading and executing the program codes. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. In addition, an operating system (OS) running on a computer performs part or all of actual processing based on an instruction of the program code, and the above-described functions are realized by the processing.

  Furthermore, you may implement | achieve with the following forms. That is, the program code read from the storage medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer. Then, based on the instruction of the program code, the case where the above-described functions are realized by the CPU included in the function expansion card or the function expansion unit performing part or all of the actual processing is also included.

  When the present invention is applied to the storage medium, the storage medium stores program codes corresponding to the flowcharts described above.

It is a block diagram which shows the structural example of the electronic zoom process using CCD. It is a figure for demonstrating the concept of the high image quality electronic zoom using partial reading. It is a figure for demonstrating the concept of the line accumulation time difference in a vertical 2 pixel addition average. It is a figure for demonstrating the change of 1 line read time when imaging drive is switched. It is a figure for demonstrating the case where accumulation time shifts up and down within one screen when imaging drive is switched. It is a figure which shows an example of a structure of the imaging device corresponding to embodiment of this invention. It is a figure which shows an example of a structure of 1 pixel circuit corresponding to embodiment of this invention. It is a figure which shows an example of a structure of the pixel circuit and readout circuit corresponding to embodiment of this invention. It is a figure for demonstrating the calculation method of the gain correction value corresponding to the 1st Embodiment of this invention. It is a flowchart which shows an example of the process of the gain correction value calculation method for every color corresponding to embodiment of this invention. It is a figure for demonstrating the calculation method of the gain correction value corresponding to the 2nd Embodiment of this invention. In the embodiment of the present invention, it is a diagram for explaining the difference in the imaging area from which the pixel value is read out in the imaging drive A and the imaging drive B. It is a figure which shows an example of the structure for reading the pixel value of the one part area | region of an imaging area corresponding to embodiment of this invention. It is a figure which shows an example of a structure of the horizontal decoder part 1406 and the horizontal shift register part 1407 corresponding to embodiment of this invention. It is an example of the timing chart of a control signal corresponding to the embodiment of the present invention.

Claims (10)

An imaging unit configured to form a plurality of lines including a plurality of pixel circuits including a light receiving unit that generates and accumulates charges according to the amount of incident light;
A first drive for sequentially resetting the light-receiving unit for each predetermined line at a first period interval; and a second drive for sequentially resetting for each predetermined line at a second period interval longer than the first period interval. Resetting means for performing
Driving for reading out the electric charge accumulated in the light receiving unit for each predetermined line as a pixel value at the first period interval after a predetermined accumulation time has elapsed from the start of the reset in the first driving; A reading unit that performs driving for reading out the electric charge accumulated in the light receiving unit for each of the predetermined lines as a pixel value at the second period interval after the elapse of the predetermined accumulation time from the start of the reset in driving;
Switching acceptance means for accepting an instruction to switch from the first drive to the second drive,
The switching accepting means when the reset means resets the predetermined line in the subsequent frame in the first drive before the readout means finishes reading out the pixel value to be read out in one frame. When the switching is accepted, the reset unit switches the reset operation for the predetermined line in the subsequent frame to the second drive after the reading unit finishes reading the pixel value to be read. ,
An image pickup apparatus, further comprising: a gain correction unit that performs gain correction on a pixel value of the subsequent frame so as to correct a shift in accumulation time of a plurality of lines included in the subsequent frame.   The correction means calculates a first correction value for the gain correction based on a quotient obtained by dividing the predetermined accumulation time by the accumulation time in the line, and the correction value is calculated based on the first correction value. The imaging apparatus according to claim 1, wherein gain correction is performed. The pixel value includes a value for each color of RGB,
The correction unit calculates a first white balance coefficient based on a pixel value of a line reset before switching of driving by the reset unit, and the first white balance coefficient and a pixel value in the one frame Based on the second white balance coefficient used for adjusting the white balance of the first correction value, the first correction value is corrected for each color of RGB to calculate a second correction value, and the second correction value is calculated. The imaging apparatus according to claim 2, wherein the gain correction is performed using a correction value.   The correction means performs the gain correction by replacing a value exceeding a predetermined threshold value among the calculated first correction value or second correction value with the threshold value. Or the imaging device of 3. An imaging apparatus control method for generating image data in an imaging section configured by arranging a plurality of lines including a plurality of pixel circuits including a light receiving section that generates and accumulates charges according to an incident light amount,
A first drive for sequentially resetting the light-receiving unit for each predetermined line at a first period interval; and a second drive for sequentially resetting for each predetermined line at a second period interval longer than the first period interval. A reset process for performing
Driving for reading out the electric charge accumulated in the light receiving unit for each predetermined line as a pixel value at the first period interval after a predetermined accumulation time has elapsed from the start of the reset in the first driving; A readout step of performing a drive for reading out the electric charge accumulated in the light receiving unit for each of the predetermined lines as a pixel value at the second period interval after the elapse of the predetermined accumulation time from the start of the reset in driving;
A switching reception step for receiving a switching instruction from the first drive to the second drive,
The switching is performed when the reset of the predetermined line in the subsequent frame is performed in the reset step in the first drive before the reading of the pixel value to be read in one frame in the readout step is finished. When the switching is accepted in the accepting step, in the reset step, after the reading of the pixel value to be read in the readout step is completed, the reset operation for the predetermined line of the subsequent frame is performed by the second drive. Switched to
Control of an imaging apparatus, further comprising a gain correction step of correcting a gain for a pixel value of the subsequent frame so as to correct a shift in an accumulation time of a plurality of lines included in the subsequent frame. Method.   In the correction step, a first correction value for gain correction is calculated based on a quotient obtained by dividing the predetermined accumulation time by the accumulation time in the line, and the first correction value is used to calculate the first correction value. The method for controlling an imaging apparatus according to claim 5, wherein gain correction is performed. The pixel value includes a value for each color of RGB,
In the correction step, a first white balance coefficient is calculated based on a pixel value of a line reset before switching of driving in the reset step, and the first white balance coefficient and a pixel value in the one frame are calculated. The second correction value is calculated by correcting the first correction value for each color of RGB based on the second white balance coefficient used for adjusting the white balance of The method of controlling an imaging apparatus according to claim 6, wherein the gain correction is performed using a correction value.   In the correction step, the gain correction is performed by replacing a value exceeding a predetermined threshold value among the calculated first correction value or second correction value with the threshold value. Item 8. A method for controlling an imaging apparatus according to Item 6 or 7.   The computer program for making a computer perform the control method of the imaging device in any one of Claims 5 thru | or 8.   A computer-readable storage medium storing the computer program according to claim 9.

Images