JP4269850B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus, and more particularly, to an imaging apparatus capable of performing image processing on a single chip semiconductor chip, for example.

  For example, in a solid-state imaging device using a semiconductor manufacturing technology such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor), an electronic shutter is used to convert the solid-state imaging device. It is possible to electronically control the accumulation time of accumulated charges and take an image with an exposure time shorter than one field time (or one frame time). This electronic shutter function is used to suppress the saturation of the charge due to the excessive light amount, and to capture a sharply moving subject.

  FIG. 1 is a timing chart for explaining the electronic shutter function of a conventional video camera. In FIG. 1, the relationship between the vertical synchronizing signal VD, the horizontal synchronizing signal HD, the substrate clock SUB, the readout signal RD, and the charge (amount) PD1 accumulated in the photodiode constituting the light receiving element of the solid-state image sensor is shown. Show.

  The vertical synchronization signal VD and the horizontal synchronization signal HD are both synchronization signals input to the solid-state image sensor from the outside of the solid-state image sensor. The substrate clock SUB is a signal representing the timing of the electronic shutter, and is input to the solid-state image sensor from the outside of the solid-state image sensor within the horizontal synchronization period. The read signal RD is a signal for reading the charge PD1 accumulated in the solid-state image sensor, and is input to the solid-state image sensor from the outside of the solid-state image sensor within the vertical synchronization period. The charge PD1 represents the charge (amount) of a photodiode that is a light receiving element constituting the solid-state imaging device. The charge PD1 increases corresponding to the amount of light received by the solid-state imaging device, and is extracted (discarded) on the substrate on which the solid-state imaging device is arranged in synchronization with the substrate clock SUB. Furthermore, the charge PD1 is output to the signal line in synchronization with the read signal RD.

  The charge PD1 output on the signal line is output as a pixel value of each pixel as a light receiving element constituting the solid-state image sensor.

  In the video camera, the charge PD1 starts to be accumulated in the light receiving element from the time when the last substrate clock SUB in one field period is supplied. At the time when one field period ends, the read signal RD is supplied to the solid-state imaging device, whereby the charge PD1 accumulated in the light receiving device so far is read from the light receiving device and output to the signal line. In FIG. 1, the time from the timing at which the last substrate clock SUB in one field period is supplied and the accumulation of the charge PD1 is started to the timing at which the charge PD1 is output to the signal line by the read signal RD. Exposure time.

  The electronic shutter function controls the exposure time according to the timing of the substrate clock SUB. That is, for example, when it is desired to increase the exposure time (low-speed shutter), the number of substrate clocks SUB (number of pulses) is reduced. On the other hand, when it is desired to shorten the exposure time (high-speed shutter), the number of substrate clocks SUB is increased.

  On the other hand, Patent Document 1 describes a still camera in which a plurality of frames are added in order to solve the shortage of light amount and an exposure time longer than the frame period is possible.

JP-A-6-296246.

  According to the electronic shutter function of a conventional video camera, the exposure time can be shortened, and the shape of a moving subject, which is a subject moving at high speed, can be clearly captured and imaged one frame at a time. However, when a moving image is taken with a short exposure time, image quality degradation called jerkiness in which subjects are displayed discretely occurs in the image.

  In addition, the electronic shutter function of the conventional video camera only changes the length of the exposure time, so that a fast-moving subject is clearly captured, and an image expression that overlaps with blurring with a tail is superimposed. It could not be realized.

  Thus, there is a method of obtaining an image with a special effect by performing signal processing on an image output from the video camera by an external device, but this method requires a large-scale device.

  The present invention has been made in view of such a situation, and makes it possible to improve image quality deterioration due to jerkiness or the like and to add a special effect without increasing the scale of the apparatus. .

An imaging apparatus according to the present invention includes a photoelectric conversion unit that photoelectrically converts an optical signal from a subject, and a photoelectric conversion unit that has a second period that is shorter than a first period that is a period of a vertical synchronization signal of an image signal. Driving means driven by the above, an amplifying means for amplifying the signal of the pixel output by the photoelectric conversion means driven by the driving means by changing the amplification factor within the period of the first period, and a signal output by the amplifying means And a predetermined signal are added , and the signal obtained by adding to the first period of the first period is output to the output means, and the period of the first period Adding means for outputting a signal obtained by adding to a period of the second cycle other than the first period to the storage means, and storing the stored signal in the first second period of the first cycle period Reset to a period and a second other than the first of the periods of the first period Storing a signal output by the adding means during the period, as a predetermined signal, storage means for the adding means, the first of a second cycle period adding means of the period of the first cycle is output Output means for outputting a signal to be output as an image signal is formed on a one-chip semiconductor chip.

In the imaging apparatus of the present invention, the photoelectric conversion means for photoelectrically converting the optical signal from the subject, and the photoelectric conversion means have a second period shorter than the first period that is the period of the vertical synchronization signal of the image signal . driving means for driving in a cycle, the signals of pixels photoelectric conversion means outputs driven by the drive means, and amplifying means for amplifying by changing the amplification factor in the period of the first cycle, amplifying means outputs The signal obtained by adding the signal and the predetermined signal and adding to the first period of the first period is output to the output means, and the signal of the period of the first period is output. An adding means for outputting a signal obtained by adding to a period of the second period other than the first period to the storage means, and a first second period of the period of the first period in the stored signal And reset to the first period of the first period. Second storing a signal adding means outputs during the period of the predetermined signal, storage means for the adding means, the first period of the second period of the period of the first cycle Output means for outputting the signal output from the adding means as an image signal is formed on a one-chip semiconductor chip.

  According to the present invention, it is possible to improve image quality degradation due to jerkiness or the like, and to add a special effect without increasing the scale of the apparatus.

  Embodiments of the present invention will be described below.

  FIG. 2 is a block diagram showing a configuration example of an embodiment of an imaging apparatus (solid-state imaging device) to which the present invention is applied.

  The imaging device 10 includes a light receiving unit 1, an AD (Analog Digital) conversion unit 2, an amplification unit 3, an addition unit 4, a memory unit 5, an output unit 6, a control unit 7, a drive unit 8, and a camera controller 9. The light receiving unit 1 to the camera controller 9 are formed on, for example, a CMOS as a one-chip semiconductor chip. The imaging device 10 performs high-speed imaging a plurality of times during one field period, and outputs the result of performing a product-sum process for each of the imaging results of the plurality of times.

  The light receiving unit 1 is supplied with an optical signal from the subject and a drive signal supplied from the driving unit 8, and the light receiving unit 1 receives the optical signal from the subject and accumulates charges corresponding to the received light amount. Perform photoelectric conversion. Then, the light receiving unit 1 supplies a current as an analog signal generated by the accumulated charges to the AD conversion unit 2 in synchronization with the drive signal supplied from the drive unit 8. Note that the light receiving unit 1 is configured, for example, by regularly arranging a plurality of light receiving elements as pixels vertically and horizontally. For example, a photodiode can be adopted as the light receiving element.

  The AD conversion unit 2 converts an analog signal corresponding to the current supplied from the light receiving unit 1 into a digital signal and supplies the digital signal to the amplification unit 3.

  The amplification unit 3 is supplied with a control signal from the control unit 7 in addition to a digital signal from the AD conversion unit 2. The amplification unit 3 amplifies the digital signal supplied from the AD conversion unit 2 based on the control signal supplied from the control unit 7 and supplies the amplified signal obtained as a result to the addition unit 4.

  In addition to the amplification signal supplied from the amplification unit 3, the addition unit 4 is supplied with the addition signal from the memory unit 5 and the control signal from the control unit 7. The adding unit 4 adds the amplified signal supplied from the amplifying unit 3 and the added signal supplied from the memory unit 5 to move and add the signal output from the light receiving unit 1 in the time direction. The addition unit 4 determines whether to output the addition signal obtained as a result of the addition to the memory unit 5 or the output unit 6 based on the control signal supplied from the control unit 7. The addition signal is output to the determined one.

  The memory unit 5 temporarily stores (writes) the addition signal supplied from the addition unit 4 based on the control signal supplied from the control unit 7, and supplies the stored addition signal to the addition unit 4 ( Read). Further, the memory unit 5 resets (clears) the stored contents based on the control signal from the control unit 7. The memory unit 5 may be a frame memory that stores signals for one field (or frame).

  The output unit 6 outputs an image signal as an addition signal supplied from the addition unit 4.

  Here, as described above, the addition unit 4 repeatedly adds the amplified signal from the amplification unit 3 and the stored value of the memory unit 5. Therefore, the amplifying unit 3, the adding unit 4, and the memory unit 5 constitute a FIR (Finite Impulse Response) type digital filter, and the filtering process of the signal output from the AD converting unit 2 is performed by the amplifying unit 3. Is performed as a so-called tap coefficient.

  The control unit 7 generates control signals to be supplied to the amplification unit 3, the addition unit 4, and the memory unit 5 based on the control signal supplied from the camera controller 9, and the amplification unit 3, the addition unit 4, and This is supplied to each memory unit 5.

  The drive unit 8 supplies a drive signal to the light receiving unit 1 based on the control signal supplied from the camera controller 9, thereby driving the light receiving unit 1.

  The camera controller 9 includes an MPU (Micro Processing Unit) (not shown) and the like. For example, the control signal is transmitted to the control unit 7 and the drive according to a control command input by a user operating an operation unit (not shown). Supply to part 8.

  FIG. 3 is a flowchart for explaining the operation of the imaging apparatus 10 of FIG. Note that the imaging process in FIG. 3 is started when, for example, a control command for requesting imaging is input to the camera controller 9.

  In step S1, the drive unit 8 supplies a drive signal to the light receiving unit 1 based on the control signal supplied from the camera controller 9, and the process proceeds to step S2.

  In step S2, the light receiving unit 1 accumulates charges according to the optical signal from the subject incident thereon according to the drive signal from the drive unit 8, and converts the current as an analog signal corresponding to the charge into an AD conversion unit. 2 and proceeds to step S3.

  In step S3, the AD conversion unit 2 converts the analog signal supplied from the light receiving unit 1 into a digital signal, supplies the digital signal to the amplification unit 3, and proceeds to step S4.

  In step S4, the amplification unit 3 amplifies the digital signal supplied from the AD conversion unit 2 based on the control signal supplied from the control unit 7, and supplies the amplified signal obtained as a result to the addition unit 4. The process proceeds to step S5.

  Here, the processing of steps S1 to S4 is repeatedly performed in the pipeline in one field period. Therefore, the light receiving unit 1 performs a plurality of times of imaging in one field period, and the amplification unit 3 to the adding unit 4 amplify corresponding to a plurality of (screen) image signals obtained by the plurality of times of imaging. Signals are supplied sequentially.

  In step S <b> 5, the memory unit 5 reads out the stored signal in accordance with the control signal from the control unit 7 and supplies it to the addition unit 4. Here, the signal stored in the memory unit 5 is reset to zero in the initial state. Further, in step S5, the adding unit 4 adds the amplified signal supplied from the amplifying unit 3 and the signal supplied from the memory unit 5, and proceeds to step S6.

  Here, in step S <b> 5, the addition of the signal stored in the memory unit 5 and the amplified signal supplied from the amplifying unit 3 is performed for each pixel having the same spatial position in a plurality of images arranged in the time direction. Is done. Then, the process of step S5 is repeatedly performed as will be described later. Thereby, a plurality of images obtained by a plurality of times of imaging in the light receiving unit 1 and a tap coefficient as an amplification factor in the amplification unit 3 are obtained. Convolution integration is performed.

  In step S <b> 6, the addition unit 4 determines whether to output the addition signal obtained in step S <b> 5 to the memory unit 5 or to the output unit 6 based on the control signal from the control unit 7.

  Here, the control unit 7 adds a control signal that causes the output unit 6 to output the addition signal obtained by the addition unit 4 only at a timing when the addition unit 4 performs the addition a predetermined number of times in one field period. Supply to part 4. On the other hand, the control unit 7 supplies the addition unit 4 with a control signal for outputting the addition signal obtained by the addition unit 4 to the memory unit 5 at another timing.

  In step S6, when it is determined that the addition unit 4 outputs the addition signal to the memory unit 5, the process proceeds to step S7. The addition unit 4 outputs the addition signal to the memory unit 5, and the process proceeds to step S8.

  In step S <b> 8, the memory unit 5 writes (stores) the addition signal supplied from the addition unit 4 based on the control signal supplied from the control unit 7, for example, overwriting, and returns to step S <b> 5. In step S5, the same processing as described above is performed on the amplified signal newly supplied from the amplifying unit 3 to the adding unit 4, and the processing in steps S5 to S8 is repeated thereafter. .

  In step S <b> 6, when it is determined that the addition unit 4 outputs the addition signal to the output unit 6, the process proceeds to step S <b> 9 and outputs the addition signal to the output unit 6. Further, in step S9, the output unit 6 outputs an image signal as an addition signal supplied from the addition unit 4, and the control unit 7 supplies a reset signal to the memory unit 5 and ends the process.

  Note that the processing according to the flowchart of FIG. 3 is repeated, for example, in a field cycle.

  Next, FIG. 4 shows an outline of a configuration example of the light receiving unit 1 to the driving unit 8 when the imaging device 10 of FIG. 2 is, for example, a column parallel type device.

Receiving unit 1, M × N pieces light-receiving element 31 ₁₁ to the light receiving element 31 _MN of (M pieces laterally, N pieces vertically) is constituted is arranged. The AD conversion unit 2 is configured by arranging _M AD converters 121 _{1 to} 121 _M. The amplifying unit 3 includes M multipliers 51 ₁ to 51 _M. Adding section 4 is composed of M adders 71 ₁ to the adder 71 _M and M adders 71 ₁ to the adder 71 _M M-number of selectors 81 ₁ to the selector 81 connected to the respective _M . Memory unit 5 is composed of M number of the line memories 101 ₁ to line memory 101 _M. The output unit 6 includes a horizontal shift register 111. The control unit 7 includes a gain control unit 41, a calculation control unit 61, and a memory control unit 91. The drive unit 8 includes a vertical drive unit 21.

  FIG. 5 is a block diagram illustrating a configuration example of the light receiving unit 1 of FIG.

Receiving unit 1, M × N pieces light-receiving element 31 ₁₁ to the light receiving element 31 _MN of (M pieces laterally, N pieces vertically) is constituted is arranged. The light receiving elements 31 _ij (where i = 1... M, j = 1... N) are connected to the _M vertical signal lines VL _{1 to} VLM for each column. That is, the light receiving elements 31 _{1-1 to} 31 _1-N in the first column are connected to the vertical signal line VL ₁ , and in the same manner, the light receiving elements 31 _{i-1 to} 31 _iN in the i column are connected to the vertical signal line VL _i. Further, the light receiving element 31 _ij is the photoelectric conversion element such as a photodiode, a charge of discharge gate, and is composed of such as a read gate to the vertical signal line VL _i, supplied from the vertical driving unit 21 of the drive unit 8 Controlled by sweep signal or read signal. Here, the drive unit 8 includes a vertical drive unit 21, and the vertical drive unit 21 sweeps out (discards) the electric charge accumulated in the light receiving element 31 _ij as a drive signal that the drive unit 8 supplies to the light receiving unit 1. ) and discharging signal for instructing to output and read signal for causing the output (read) to its charge of the vertical signal line VL _i.

The drive signal from the vertical drive unit 21 is applied to the M light receiving elements 31 _{1-j to} 31 _Mj for each row. Currents generated from the charges of the M light receiving elements 31 _{1-j to} 31 _Mj arranged in one row are simultaneously read onto the vertically connected vertical signal lines VL _{1 to} VL _M , respectively, and are converted into AD converters. 2 are supplied to each of the AD converters 121 _{1 to} 121 _M constituting the AD 2 and AD-converted. In the subsequent stage of the light receiving unit 1, signal processing is performed in parallel.

  FIG. 6 is a block diagram illustrating a configuration example of the amplifying unit 3 in FIG.

The amplifying unit 3 includes M multipliers 51 ₁ to 51 _M. To the multipliers 51 ₁ to the multiplier 51 _M respectively, M-number of the AD converter 121 ₁ to AD converter 121 constituting the AD conversion section 2 connected to the respective vertical signal lines VL ₁ through the vertical signal line VL _{M M} A digital signal supplied from each of them and a control signal g _{1 to a} control signal g _M supplied from a gain control unit 41 constituting the control unit 7 are input. Each of the multipliers 51 _{1 to} 51 _M receives a digital signal as a pixel value supplied from each of the _M AD converters 121 _{1 to} 121 M based on the control signals g _{1 to} g _M. Amplification, that is, the control signal g _{1 to} control signal g _M is multiplied by the digital signal as the pixel value supplied from each of the AD converter 121 _{1 to} AD converter 121 _M, and the digital signal obtained by the multiplication the amplified signal is, and outputs to the vertical signal line VL ₁ to the vertical signal line VL _M for each column, and supplies the M adders 71 ₁ to the adder 71 _M each constituting the adding section 4.

Here, in the amplifying unit 3, multiplication of the control signals g _{1 to} g _M input from the gain control unit 41 and each of the digital signals supplied from the AD conversion unit 2 is performed simultaneously for each row. .

The control unit 7 includes a gain control unit 41, the gain control unit 41, control signals g ₁ to control signals representing the amplification factor for amplifying the pixel values on the vertical signal line VL ₁ to the vertical signal line VL _M g _M is supplied to the amplifying unit 3. The amplification factor, which is the degree to which the digital signal is amplified, corresponds to a tap coefficient in the digital filter.

FIG. 7 is a block diagram illustrating a configuration example of the adding unit 4 of FIG. Adding section 4 is composed of M adders 71 ₁ to the adder 71 _M and M adders 71 ₁ to the adder 71 _M M-number of selectors 81 ₁ to the selector 81 connected to the respective _M . The adders 71 _{1 to} 71 _M are connected to the vertical signal lines VL _{1 to} VL _M, respectively. In the addition unit 4, an addition operation for adding the amplified signal supplied from the amplification unit 3 and the addition signal supplied from the memory unit 5 is performed in parallel in columns for each row.

That is, the adder 71 ₁ to the adder 71 _M, respectively, of the amplifier 3, the M, which is connected to the vertical signal line VL ₁ to each vertical signal line VL _M multipliers 51 ₁ to the multiplier 51 _M, respectively with the amplified signal is supplied, the addition signal from the M line memories 101 ₁ to the line memories 101 _M each constituting the memory unit 5 is supplied. The adder 71 _i adds the amplified signal supplied from the multiplier 51 _i and the addition signal supplied from the line memory 101 _i, and supplies a new addition signal obtained by the addition to the selector 81 _i . .

Each of the selectors 81 _{1 to} 81 _M is supplied with an addition signal from each of the adder 71 _{1 to} 71 _M, and also receives an arithmetic control signal a ₁ to an arithmetic control signal from the arithmetic control unit 61 constituting the control unit 7. each a _M is supplied. The selectors 81 _{1 to} 81 _M are respectively added from the adders 71 ₁ to 71 _M based on the calculation control signals a _{1 to} a _M supplied from the calculation control unit 61 constituting the control unit 7. It is determined whether the supplied addition signal is output to the memory unit 5 or the output unit 6.

Note that the calculation control signal a _i supplied at the time of the next imaging after the last imaging in one field period, that is, the first imaging in the next one field period, outputs an addition signal that has been added a plurality of times. It is a signal that is controlled to be supplied to the unit 6.

When the selector 81 _i determines to output the addition signal to the output unit 6 based on the calculation control signal a _i supplied from the calculation control unit 61, the selector 81 _i outputs the addition signal supplied from the adder 71 _i to the output unit 6. . On the other hand, when the selector 81 _i determines to supply the addition signal to the memory unit 5 based on the calculation control signal a _i supplied from the calculation control unit 61, the selector 81 _i sends the addition signal supplied from the adder 71 _i to the memory unit 5. Output.

In the present embodiment, imaging is performed a plurality of times in one field period (or one frame period), that is, a plurality of samplings of charges accumulated in the light receiving unit 1 (hereinafter referred to as subsampling as appropriate). Done. The selector 81 _i is supplied from the adder 71 _i in the first sub-sampling period in the next one field period, that is, in the sub-sampling period in which the first sub-sampling (charge accumulation) is performed in the next one field period. The addition signal is supplied to the output unit 6. On the other hand, in the sub-sampling period other than the first sub-sampling period, the selector 81 _i supplies the addition signal from the adder 71 _i to the memory unit 5. The addition signal supplied to the memory unit 5 is used as an addition signal as a predetermined signal for the addition operation with the next amplified signal.

  FIG. 8 is a block diagram illustrating a configuration example of the memory unit 5 of FIG.

Memory unit 5 is composed of M number of the line memories 101 ₁ to line memory 101 _M. Each of the line memories 101 _{1 to} 101 _M is connected to an adder 71 _{1 to an} adder 71 _M and a selector 81 _{1 to a} selector 81 _M constituting the adder 4 (FIG. 7). Each of the line memories 101 _{1 to} 101 _M is supplied with an addition signal from each of the selectors 81 _{1 to} 81 _M constituting the addition unit 4 and receives a control signal from the memory control unit 91 constituting the control unit 7. Supplied. In each of the line memories 101 _{1 to} 101 _M , based on the control signal supplied from the memory control unit 91, the addition signal supplied from each of the selectors 81 _{1 to} 81 _M is written and the stored addition signal is stored. Reading or storage contents are reset.

Note that the control signal supplied from the memory control unit 91 is one of a write signal, a read signal, and a reset signal. The memory control unit 91 writes, for example, based on the control signal supplied from the camera controller 9. A signal, a read signal, or a reset signal is supplied to the line memory 101 _i .

The write signal supplied from the memory control unit 91 is the addition signal stored in each of the line memories 101 _{1 to} 101 _M and the addition signal supplied from each of the selectors 81 _{1 to} 81 _M. Is a control signal to be rewritten and stored. When the write signal is supplied from the memory control unit 91, each of the line memories 101 _{1 to} 101 _M receives the stored addition signal from the selector 81 _{1 to the} selector 81 _M of the addition unit 4. Rewrite to

The readout signal supplied from the memory control unit 91 reads out the addition signals stored in the line memories 101 _{1 to} 101 _M and supplies them to the adders 71 _{1 to} 71 _{M of} the addition unit 4. Signal. When a read signal is supplied from the memory control unit 91, each of the line memories 101 _{1 to} 101 _M supplies the stored addition signal to each of the adders 71 _{1 to} 71 _M of the addition unit 4. .

A reset signal supplied from the memory controller 91 is a control signal for clearing the sum signal stored in the line memory 101 ₁ to the line memories 101 _M, respectively. When the reset signal is supplied from the memory control unit 91, a line memory 101 ₁ to the line memories 101 _M, respectively, to clear the sum signal stored in the zero.

Figure 9 is a block diagram showing a configuration example of the line memory 101 ₁ of FIG. The line memory 101 ₁ has storage areas 102 _{1 to} 102 corresponding to the _N light receiving elements 31 _{1-1 to} 31 _{1-N in} one column connected to the vertical signal line VL ₁ in FIG. It is composed of _N. Each of the storage areas 102 _{1 to} 102 _N is connected to the adder 71 ₁ and the selector 81 _{1 in} FIG. In each of the storage areas 102 _{1 to} 102 _N , a row for a signal as a pixel value output from the _N light receiving elements 31 _{1-1 to} 31 _1-N connected to the vertical signal line VL ₁ is provided. Each of the N addition signals as a series of processing results is input from the selector 81 ₁ . In each of the storage areas 102 _{1 to} 102 _N , the addition signal is written, read, or reset based on the control signal supplied from the memory control unit 91 in FIG.

The other line memories 101 _{2 to} 101 _M are configured in the same manner as 101 ₁ in FIG.

  FIG. 10 is a block diagram illustrating a configuration example of the output unit 6 of FIG.

  The output unit 6 includes a horizontal shift register 111. The horizontal shift register 111 is simultaneously supplied with the M addition signals processed for each row from the adder 4, and the horizontal shift register 111 uses the M addition signals as an image signal for one horizontal line at high speed. Transfer and output.

  FIG. 11 is a timing chart for explaining the operation of the imaging apparatus 10 shown in FIGS.

In Figure 11, the video of the vertical synchronizing signal VD, charge accumulated in the photodiode constituting the light-receiving element 31 ₁₁ to the light receiving element 31 _MN of the light receiving portion 1 (amount) PD2, the amplifier 3 from the controller 7 The relationship between the supplied amplification factor G, the addition signal Mem stored in the memory unit 5, and the output signal Output as the pixel value output from the output unit 6 is shown.

  The imaging device 10 picks up an image at a high speed a plurality of times at a cycle shorter than the cycle of the vertical synchronization signal VD, and processes a plurality of image signals obtained as a result, thereby obtaining one image, that is, vertical synchronization. A field image signal corresponding to the signal VD is obtained and output. Note that the maximum number of times of imaging within the period of the vertical synchronization signal VD of the imaging device 10 is limited by the high-speed driving performance of the light receiving unit 1 in the imaging device 10 and the computing performance of the amplification unit 3 and the addition unit 4. In FIG. 11, it is assumed that the imaging device 10 performs imaging eight times within one field period, and the operation of the imaging device 10 will be described focusing on one pixel.

  The vertical synchronization signal VD is one of the control signals supplied from the camera controller 9 to the drive unit 8, and the drive unit 8 drives the light receiving unit 1 with a cycle shorter than the cycle according to the vertical synchronization signal VD. As a result, the light receiving unit 1 performs high-speed imaging.

The charge PD2 represents the charge accumulated in the light receiving unit 1. The charge PD2 accumulated in the light receiving unit 1 is extracted to a substrate (not shown) when a sweep signal is applied from the drive unit 8 to the light receiving unit 1, and when a read signal is applied from the drive unit 8 to the light receiving unit 1. It is read out to the vertical signal line VL _i.

Here, in the imaging apparatus 10, in order to perform imaging a plurality of times within one field period, one field period is divided into the same number of periods as the number of times of imaging. This period is called a sub-sampling period. In this case, since the imaging is performed eight times within one field period, one field period is divided into eight sub-sampling periods. In the exposure periods E1 to E8 in each of the eight sub-sampling periods, the charge PD2 is reset (by the electronic shutter function) by the sweep signal from the drive unit 8 to the light receiving unit 1, and then the drive unit 8 transfers to the light receiving unit 1. by the read signal, which is a time until the charge PD2 are read out to the vertical signal line VL _i.

  The amplification factor G represents the amplification factor supplied from the control unit 7 to the amplification unit 3. FIG. 11 shows the amplification factor by which each pixel value is multiplied when performing a filtering process using a so-called triangular filter. In the present embodiment, the amplification factor G is generated in the control unit 7.

  The addition signal Mem indicates an addition signal stored in the memory unit 5 for a certain pixel.

  The output signal Output is a pixel value of one pixel of an image of one field output from the imaging device 10.

  In the imaging device 10, imaging is performed in the first exposure period E 1 in the first sub-sampling period which is the first sub-sampling period of one field period, and the addition signal Mem stored in the memory unit 5 is Reset. In the first exposure period E1, the light receiving unit 1 accumulates the charge PD2 having the charge amount I1 as the first imaging signal.

In the second sub-sampling period that is the second sub-sampling period, imaging in the second exposure period E2 is performed. Further, the read signal applied to the light receiving unit 1 from the drive unit 8 at the end of the first exposure period E1 is the charge PD2 having the charge amount I1, which is the first imaging signal imaged in the first sub-sampling period. It is read to the vertical signal line VL _i by. Charge PD2 charge amount I1 is a first imaging signal read out to the vertical signal line VL _i is supplied to the AD converter 2, the AD conversion section 2, a digital signal (hereinafter referred to as first Digital signal).

  In a period P1 that is a predetermined time after the end of the first exposure period E1, the AD conversion unit 2 supplies the first digital signal to the amplification unit 3. Further, in the period P1, the amplifying unit 3 multiplies the first digital signal I1 supplied from the AD conversion unit 2 and the amplification factor G, and an amplified signal I1 × G (hereinafter referred to as appropriate) obtained as a result of the multiplication. A first amplified signal) is supplied to the adder 4.

  On the other hand, the memory unit 5 is reset in the first sub-sampling period and is in a state where nothing is stored, that is, zero. In the period P1, the adding unit 4 adds the first amplified signal I1 × G supplied from the amplifying unit 3 and the information stored in the memory unit 5, that is, zero. The adding unit 4 supplies the memory unit 5 with the addition result I1 × G as the first addition signal M1, and stores it.

In the third sub-sampling period that is the third sub-sampling period, imaging in the third exposure period E3 is performed. In addition, the read signal applied to the light receiving unit 1 from the drive unit 8 at the end of the second exposure period E2 is the charge PD2 having the charge amount I2, which is the second imaging signal imaged in the second sub-sampling period. It is read to the vertical signal line VL _i by. The charge PD2 having the charge amount I2, which is the second imaging signal read out on the vertical signal line VL _i , is supplied to the AD converter 2, and the AD converter 2 converts the digital signal (hereinafter referred to as a second signal as appropriate). Digital signal).

  In a period P2 that is a predetermined time after the end of the second exposure period E2, the AD conversion unit 2 supplies the second digital signal to the amplification unit 3. Further, in the period P2, the amplifying unit 3 multiplies the second digital signal I2 supplied from the AD converting unit 2 and the amplification factor G, and an amplified signal I2 × G (hereinafter referred to as appropriate) obtained as a result of the multiplication. (Referred to as a second amplified signal).

  On the other hand, the memory unit 5 stores M1, which is the first addition signal I1 × G obtained in the period P1, as the addition signal Mem. In the period P2, the adding unit 4 receives the second amplified signal I2 × G supplied from the amplifying unit 3 and the information stored in the memory unit 5, that is, the first added signal M1 = I1 × G. to add. The addition unit 4 supplies the addition result M1 + (I2 × G) as the second addition signal M2 to the memory unit 5 for storage.

  Similarly, in each sub-sampling period, the same process as described above is performed in the last sub-sampling period in one field period (in the present embodiment, the eighth sub-sampling period is the eighth sub-sampling period. Sampling period). Processing of the imaging signal obtained in the final (eighth) exposure period E8 is performed after the end of the eighth exposure period E8, that is, within the first sub-sampling period of the next one field period. That is, the amplifying unit 3 multiplies the eighth imaging signal of the charge amount I8 obtained in the eighth exposure period E8 by the amplification factor G to obtain an amplified signal I8 × G and supplies it to the adding unit 4 To do. The adding unit 4 adds the amplified signal I8 × G and the value M7 stored in the memory unit 5 to obtain an added signal M7 + I8 × G. Then, the addition unit 4 supplies the addition signal M7 + I8 × G to the output unit 6 in accordance with the control signal supplied from the control unit 7 as indicated by the output signal Output. The output unit 6 outputs the addition signal M7 + I8 × G to the outside of the imaging device 10 as the pixel value of the image in the previous one field period in the first subsampling period of one field period. This pixel value is a convolution integral between the imaging signal in each sub-sampling period and the amplification factor G, that is, a filtering process of the output signal of the light receiving unit 1 in each sub-sampling period by an FIR filter using the amplification factor G as a tap coefficient. It is the result.

  FIG. 12 is a diagram illustrating an example of the amplification factor G as a tap coefficient used for the filtering process in the imaging apparatus 10.

  FIG. 12 shows a pixel value of a pixel at a certain position obtained in each of eight sub-sampling periods in one field period, with a horizontal axis as time (sub-sampling period), and a tap coefficient (amplification factor) multiplied by the pixel value. G).

  The sample value SAMPLE indicates a signal output by the light receiving unit 1 in each of eight sub-sampling periods of one field period, that is, a pixel value of a pixel at a certain position.

  The tap coefficient BOX is a tap coefficient having the same value constituting the rectangular filter. The tap coefficient BOX constitutes a rectangular filter equivalent to a conventional electronic shutter. However, according to the tap coefficient BOX, the phase (phase characteristic) of the rectangular filter can be changed. That is, in the conventional electronic shutter, since the end timing of the exposure period is determined, it was only possible to change the start timing of the exposure period. However, according to the tap coefficient BOX, the start timing of the exposure period Both the end timing and the end timing can be changed.

  The tap coefficient TENT is a tap coefficient constituting a triangular filter, and the tap coefficient GAUSS is a tap coefficient constituting a Gaussian filter. Since the filter composed of the tap coefficients TENT and GAUSS has a gentle frequency response compared to the rectangular filter, for example, the occurrence of jerkiness can be reduced.

  The tap coefficient COMB is a tap coefficient constituting a comb filter, and can express a special image in which an object moving at high speed in one field period is discretely captured a plurality of times. According to the tap coefficient COMB, for example, when a golf swing is photographed, it is possible to obtain an image in which the trajectory of a continuously moving club is discretely displayed like a disassembled photograph. The tap coefficient COMB is useful not only for moving image shooting but also for still image shooting.

  The tap coefficient COMET configures, for example, a filter that can express a special image that adds a blur that draws a tail while clearly capturing a subject that moves at high speed. The tap coefficient COMET is useful not only for moving image shooting but also for still image shooting.

  As described above, according to the imaging apparatus 10, the process of capturing an image signal in each of a plurality of sub-sampling periods shorter than the period of the vertical synchronization signal of the image signal and performing a filtering process to obtain an image of one field. Since it is performed on a single-chip semiconductor chip such as a CMOS, the processing speed is increased and the power consumption can be reduced. Accordingly, it is possible to reduce image quality degradation due to jerkiness or the like during moving subject shooting by the electronic shutter function and to provide a special effect with low power consumption.

  In addition, the filter comprised by the amplifier part 3, the addition part 4, and the memory part 5 is not limited to what was mentioned above. Further, the gain G as a tap coefficient given from the control unit 7 to the amplification unit 3 can be set in the control unit 7 in advance or can be given from the outside. Also, a plurality of types of amplification factors G as tap coefficients are prepared in advance, and the user can select one to be used.

  Furthermore, the imaging device 10 can capture either a moving image or a still image.

  In the present embodiment, the signal processing is performed in column parallel, but other configurations in which the signal processing is performed in row parallel can be employed.

  Furthermore, in the present embodiment, sub-sampling is performed eight times during one field period, but the number of sub-sampling is not limited to this. Further, the number of sub-sampling can be set from the outside of the imaging apparatus 10.

  In the present embodiment, the convolution integral for the pixel value of one pixel obtained as a result of performing a plurality of sub-samplings is performed in units of fields, but may be performed in units of frames. .

It is a timing chart explaining the conventional electronic shutter function. It is a block diagram which shows the structural example of one Embodiment of the imaging device to which this invention is applied. 3 is a flowchart for explaining the operation of the imaging apparatus 10 in FIG. 2. It is a figure which shows the structural example of the column series type of the imaging device 10 of FIG. 2 is a block diagram illustrating a configuration example of a light receiving unit 1. FIG. 3 is a block diagram illustrating a configuration example of an amplifying unit 3. FIG. 3 is a block diagram illustrating a configuration example of an adding unit 4. FIG. 3 is a block diagram illustrating a configuration example of a memory unit 5. FIG. Is a block diagram illustrating a configuration example of the line memory 101 _1. 3 is a block diagram illustrating a configuration example of an output unit 6. FIG. 3 is a timing chart for explaining the operation of the imaging apparatus 10 in FIG. 2. It is a figure which shows the example of the gain G as a tap coefficient used for the filtering process in the imaging device 10. FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Light-receiving part, 2 AD conversion part, 3 Amplification part, 4 Addition part, 5 Memory part, 6 Output part, 7 Control part, 8 Drive part, 9 Camera controller, 10 Imaging device, 11 Imaging device, 21 Vertical drive part, 31 _{1-1 to} 31 _MN light receiving element, 41 gain control unit, 51 _{1 to} 51 _M multiplier, 61 arithmetic control unit, 71 _{1 to} 71 _M adder, 81 _{1 to} 81 _M selector, 91 memory control unit, 101 ₁ To 101 _M line memory, 102 _{1 to} 102 _N storage area, 111 horizontal shift register, 121 _{1 to} 121 _N AD converter

Claims (3)

Photoelectric conversion means for photoelectrically converting an optical signal from a subject ;
Driving means for driving the photoelectric conversion means in a second cycle that is shorter than a first cycle that is a cycle of a vertical synchronization signal of the image signal;
Amplifying means for amplifying the signal of the pixel output from the photoelectric conversion means driven by the driving means by changing the amplification factor within the period of the first period ;
The signal output from the amplifying unit and a predetermined signal are added , and the signal obtained by adding the signal to the first period of the first period of the first period is output to the output unit. Adding means for outputting a signal obtained by adding to a period of the second cycle other than the first of the periods of the first cycle to a storage unit ;
The stored signal is reset to the first period of the second period in the period of the first period, and the period of the second period other than the first of the periods of the first period It said adding means stores the signal is output to, as the predetermined signal, said storage means for outputting said adding means,
The output means for outputting the signal output from the adding means as an image signal during the first period of the first period of the first period is formed on a one-chip semiconductor chip. An imaging apparatus characterized by that. The amplifying means sets the amplification factor of the predetermined number of the second period from the beginning of the first period and the predetermined number of the second period from the end to zero. , Amplifying the pixel signal output by the photoelectric conversion means
The imaging device according to claim 1. The amplification means uses the maximum amplification factor when the amplification factor of the second cycle period is expressed with the vertical axis representing the amplification factor and the horizontal axis representing the time axis. As a reference, the pixel signal output by the photoelectric conversion means is amplified using the amplification factor that represents the amplification factor of the second period before and after the target.
The imaging device according to claim 1.

Images