GB2401000A - Reset and immediate read method for imaging array - Google Patents

Abstract

A method for operating a solid state image sensor array 1 comprises resetting and then immediately reading an output from each array pixel 10. By reading a signal straight away after reset, and then subtracting the initial reset output from a later, image signal so the noise components can be cancelled. The time periods between each array read cycle may be integral multiples of a lighting flicker period to cancel the effects of illumination flicker. The method allows the array to be continuously exposed, removing the need for shutters or array blanking.

Description

1 2401000 1 Image Sensors with Multiple Integration/Read Cycles 3 The

present invention relates to solid state image 4 sensors. More particularly, the invention relates to CMOS-type image sensors in which multiple 6 integration and read cycles are performed between 7 resetting of the sensor pixels. This provides the 8 basis for obtaining images which are low in noise 9 and/or having a relatively wide dynamic range and/or which are resistant to flicker induced by mains 11 frequency flickering of artificial light sources.

13 Solid state image sensors fabricated using CMOS 14 technology provide a low cost imaging solution, as compared with COD image sensors, for applications 16 such as digital still cameras, camcorders, web 17 cameras etc. Many kinds of CMOS imaging circuit 18 architectures are possible, including three main 19 types as illustrated in Fig. 1: (a) passive pixel type; (b) 3- transistor active pixel type; and (c) 4 21 transistor active pixel type.

23 A disadvantage of types (a) and (b) above is what is 24 known as "reset noise". This type of noise makes the reset potential of the photodiode forming part 1 of each pixel circuit uncertain, and this 2 uncertainty adds to pixel-to-pixel and frame-to 3 frame variable noise in both still and video images.

The rms magnitude of this reset noise is known to 6 be: 8 Nrst = ;(kT/C) volts Where k is Boltzmann's constant, T is absolute 11 temperature in Kelvin, and C is the capacitance of 12 the node being reset.

14 Pixel type (c) above does not suffer from this type of effect if a special type of buried diode is 16 employed which enables the reset node to be entirely 17 depleted of free charge. Photodiode arrays of this 18 type ("pinned" photodiode arrays) require at least 19 one manufacturing process step in addition to the normal CMOS process, and incur an overhead cost of 21 at least one additional transistor per pixel as 22 compared with a 3-transistor active pixel.

24 CMOS image sensors may also suffer a performance disadvantage in environments which are lit by 26 artificial light sources whose intensity varies 27 rapidly in time with the AC mains supply frequency, 28 or at some harmonic of this frequency. For example, 29 fluorescent lights "flicker" at twice the supply the frequency. In these conditions the exposure mode of 31 most un-shuttered CMOS sensors causes horizontal 1 banding interference in the image, which may also be 2 seen to scroll vertically.

4 It is known to correct this flicker effect by making the exposure time substantially equal to one period 6 of the flickering source, or an integer multiple 7 thereof.

9 A third disadvantage, which is common to most image sensors, is that the range of scene luminances that 11 can be captured in one frame (i.e. the dynamic range 12 of the sensor) is significantly limited. In 13 practice, certain noise mechanisms limit the lowest 14 luminance levels, and combinations of supply voltage and circuit design cause areas of the scene which 16 are above a critical luminance to be clipped or 17 saturated. Typically, the dynamic range available 18 in one frame is limited to about 60 dB, but most 19 real scenes contain luminance ranges greater than this.

22 The present invention relates to image sensors 23 having modes of operation in which the operation and 24 timing of certain active pixel sensor arrays substantially eliminates the effects of reset and 26 flicker noise, whilst also expanding the available 27 instantaneous dynamic range.

29 The invention may be applied to any active pixel architecture which supports a non-destructive read 31 of pixel values; e.g. type (b) 3-transistor types as 32 discussed above, but not type (a) passive pixels.

1 The invention exploits the fact that, with pixel 2 types which support non-destructive read operations, 3 it becomes possible to perform multiple, staggered 4 read operations without the pixels being re-set between read operations, so that the data read at 6 any particular point in time represents the 7 cumulative signal integrated up to that point since 8 the last time the pixels were re-set.

The invention is particularly intended for use with 11 CMOS type image sensors, image sensor systems and 12 cameras.

14 In accordance with a first aspect of the invention, there is provided a method of operating a solid 16 state image sensor having an image sensing array 17 comprising a plurality of active pixels, the method 18 comprising: 19 resetting each said pixel; after a first predetermined period of time 21 reading a first output from each said pixel so as to 22 obtain a first set of image data having a first 23 dynamic range; 24 without resetting said pixels, after a second predetermined period of time reading a second output 26 from each said pixel so as to obtain a second set of 27 image data having a second dynamic range; and 28 combining said first and second sets of image 29 data in order to obtain a resultant set of image data having a further dynamic range different from 31 said first and second dynamic ranges.

1 Preferably, the method further comprises, without 2 resetting said pixels, after at least a third 3 predetermined period of time reading at least a 4 third output from each said pixel so as to obtain a third set of image data having a third.dynamic 6 range; and 7 combining at least said first, second and third 8 sets of image data in order to obtain a resultant 9 set of image data having a further dynamic range different from said first, second and third dynamic 11 ranges.

13 In accordance with a second aspect of the invention, 14 there is provided a method of operating a solid state image sensor having an image sensing array 16 comprising a plurality of active pixels, the method 17 comprising: 18 resetting and immediately reading a preliminary 19 output from each said pixel; after a first predetermined period of time, 21 reading a first output from each said pixel.

23 Preferably, the method further includes the step of 24 determining the difference between said preliminary and first outputs so as to obtain a set of image 26 data substantially free of noise components 27 represented by said preliminary outputs.

29 Preferably, the method in accordance with the first aspect of the invention is combined with the method 31 in accordance with the second aspect of the 32 invention, wherein said preliminary outputs of the 1 second aspect are read immediately after performing 2 the resetting step of the first aspect.

4 Preferably, the method further includes the step of determining the difference between said preliminary 6 outputs and each of said first, second and any 7 subsequent outputs so as to obtain a plurality of 8 said sets of image data each of which is 9 substantially free of noise components represented by said preliminary outputs.

12 Preferably, in each of the aforementioned aspects of 13 the invention, the or each said predetermined time 14 period is selected to be an integer multiple of a predetermined lighting flicker period.

17 Preferably, in each of the aforementioned aspects of 18 the invention, said image sensing array remains 19 continuously exposed to incident light while the method is performed.

22 In accordance with a further aspect of the 23 invention, there is provided a solid state image 24 sensor adapted to perform a method in accordance with any one of the first to third aspects of the 26 invention.

27 In accordance with another aspect of the invention, 28 there is provided a solid state image sensor system 29 adapted to perform a method in accordance with any one of the first to third aspects of the invention.

1 In accordance with still another aspect of the 2 invention, there is provided a camera incorporating 3 a solid state image sensor or image sensor system 4 adapted to perform a method in accordance with any one of the first to third aspects of the invention.

7 US-A-5926214 discloses image sensors and methods of 8 operation thereof wherein multiple read cycles are 9 performed between resets of an active pixel sensor array. However, these methods are concerned only 11 with noise reduction and require the use of an 12 optical shutter to mask the array from incident 13 light during an initial reset/read cycle and during 14 subsequent read operations between successive integration periods.

17 Embodiments of the invention will now be described, 18 by way of example only, with reference to the 19 accompanying drawings, in which: 21 Figs. l(a), l(b) and l(c) illustrate, respectively, 22 a passive type image sensor pixel, a 3-transistor 23 active image sensor pixel and a 4-transistor active 24 image pixel; and 26 Fig. 2 illustrates a portion of one example of a 3 27 transistor active pixel image sensor array.

29 Referring now to Fig. 2 of the drawings, a 2x2 pixel portion of a typical active pixel CMOS image sensor 31 array 1 is shown. The array 1 comprises a plurality 32 of rows 3, 5 and columns 4, 6 of active 1 photosensitive pixels 10 defining an image sensing 2 area. The pixels may be addressed sequentially by 3 vertical 12 and horizontal 14 shift registers 4 electronically connected to the pixels as shown, or by any other suitable pixel addressing scheme such 6 as a decoded address scheme. The shift registers 7 12, 14 are electronically connected to scanning 8 circuitry (not shown) for scanning - i.e. reading 9 the pixel outputs to an output O/P. Any of a variety of known types of active pixel may be used 11 for the pixels of the array. In the array of Fig. 12 2, the pixels 10 each comprise a photodiode 11 and 13 associated transistor circuitry for use in 14 amplifying (buffering) the diode outputs and for reading and resetting the diodes 11, as is well 16 known in the art.

18 In conventional use of a sensor array of this type, 19 the pixels would normally be reset and light would impinge on the photodiodes 11 for a predetermined 21 period (the integration period), before the pixels 22 are read in order to capture a set of image data 23 from the array. The pixels would then be reset 24 prior to each integration period for each image to be captured.

27 In accordance with the present invention, multiple 28 reads are performed between successive resets, as 29 follows.

31 Firstly, the pixels are reset, destroying any 32 previous pixel signals and forcing the photodiode of 1 each pixel to a known reset voltage (Vrt), and are 2 read immediately after being reset (preliminary read 3 cycle or Read 0). It will be understood that, as in 4 conventional image sensor operation, lines (rows or columns) of pixels are reset and read sequentially, 6 so that all of the pixels in one line are reset 7 simultaneously and then read simultaneously.

8 However, this preliminary read cycle is performed 9 immediately after resetting the pixels, rather than after a predetermined integration period as in 11 conventional image sensors.

13 The output from each pixel when read immediately 14 after reset is: 16 Out 0 = Vrt + Nrst + Vof f + Vimg 18 Where Vrt is the reset voltage (described above), Nest l9 is the reset noise (described above and different on each reset occasion), Voff is a circuit-induced 21 voltage offset whose value can be unique to each 22 pixel due to local random threshold variations, Vimg 23 is a signal due to any stray light integration which 24 may have occurred between reset and Read 0. That is, Outo comprises a "dark" signal which contains 26 noise comprising the above mentioned components.

27 However, these noise components remain substantially 28 constant over short periods of time and until a new 2 9 reset occurs.

31 Following Read 0, without resetting the pixels, 32 light is integrated during a first predetermined 1 integration period, Tintl, producing a signal, Sig1, 2 within each pixel due to the discharge of pixel 3 capacitance by way of photo-induced leakage current.

4 A first read cycle, Read 1, is performed at the end of Tint1. The output from each pixel when read at the 6 end of Tint1 iS: 8 Out 1 = Vrt + Nrst + Vof f + Vlmg + S. i g1 By calculating the difference between Outo and Out1, 11 the value of Sigl can be determined free of the 12 noise components which comprise Outo, since these 13 components are constant between Read 1 and Read 2; 14 i.e. 16 Out1 - Auto = Sig1.

18 The read process described thus far therefore 19 provides an output signal which is substantially free of noise.

22 Preferably, at least one further read cycle, Read 2, 23 is performed after a second predetermined 24 integration period, Tint2, again without resetting the pixels, resulting in further discharge of the 26 photodiode capacitance and producing a further 27 signal, Sig2, such that 29 Out2 Outs = Sig and 32 Out2 - Auto = Sig3; 2 Where Sig3 = Sigl + Sig2 and where Sig2 and Sig3 are 3 also substantially free of the noise components 4 represented by Outo.

6 It can be seen that Sig1, Sig2 and Sig3 provide sets 7 of image data with different exposure periods so 8 that the sensor outputs obtained from the three read 9 cycles provide three different representations of the same scene taken close together in time and 11 having different dynamic ranges. Sigl, corresponding 12 to the shortest integration period, will contain 13 most information from relatively bright image areas 14 but is likely to be underexposed in relatively dim image areas, Sig2 will provide an intermediate view 16 and Sig3 will contain most information from 17 relatively dim image areas but is likely to be 18 overexposed in relatively bright image areas. The 19 three images may then be combined in order to obtain a composite image having a dynamic range wider than 21 could be obtained by means of a single integration 2 2 period.

24 The three images may be combined in any of a variety of ways. Generally speaking, the signals will be 26 normalised so that the highest luminance values from 27 the first (shortest exposure) image are scaled to 28 the upper end of a predetermined range of output 29 values, the lowest luminance values from the third (longest exposure) image are scaled to the lower end 31 of the range of output values, and intermediate 32 composite output values are determined by combining 1 and/or scaling intermediate values from all three 2 images.

4 The pixels may be reset following the final read cycle. It will be understood that the number of read 6 cycles may vary between resets. The method may be 7 implemented using a substantially conventional image 8 sensor, with suitably adapted control software 9 and/or firmware and/or hardware for controlling the timing of reset and read cycles, sufficient frame 11 storage resources to store the multiple sets of data 12 captured in each read cycle, and suitable image 13 processing software and/or firmware and/or hardware 14 for combining the image data as required.

Obviously, the greater the number of read cycles the 16 greater the overhead of frame storage and data 17 processing.

19 It will be understood that the increased dynamic range obtained by means of multiple integration 21 periods and read cycles may be usefully employed 22 independently of the preliminary read cycle (Read 0) 23 which allows the cancellation of noise from the 24 images. However, it is preferred that these operations are combined in order to obtain images 26 having low noise and wide dynamic range.

28 All of the images obtained by means of such multiple 29 read cycles may be free of lighting flicker effects if the integration periods are each selected to be 31 integer multiples of any lighting flicker period.

1 It will also be understood that the methods and 2 image sensors of the present invention do not 3 require the use of an optical shutter in order to 4 mask the image sensor either during the initial reset/read operation or during subsequent read 6 operations. Read 0 is performed immediately after 7 reset, with resetting and reading being performed on 8 a sequential, line-by-line basis, and subsequent 9 reads are performed in a similar manner while integration continues.

12 The methods of the invention may be implemented by 13 means of a suitably adapted image sensor, or image 14 sensor system, or camera incorporating an image sensor or image sensor system. The methods are also 16 applicable to any type of active pixel architecture, 17 most suitably of the CMOS type, supporting non 18 destructive read operations. Additional examples of 19 such pixel architectures are illustrated in US-A 5926214.

22 Improvements and modifications may be incorporated 23 without departing from the scope of the invention as 24 defined in the Claims appended hereto.

Claims (1)

1 Claims 3 1. A method of operating a solid state image 4 sensor having an

image sensing array comprising a plurality of active pixels, the method comprising: 6 resetting and immediately reading a preliminary 7 output from each said pixel; 8 after a first predetermined period of time, 9 reading a first output from each said pixel.

11 2. A method as claimed in Claim 1, further 12 including the step of determining the difference 13 between said preliminary and first outputs so as to 14 obtain a set of image data substantially free of noise components represented by said preliminary 16 outputs.

18 3. A method as claimed in any preceding Claim, 19 wherein the or each said predetermined time period is selected to be an integer multiple of a 21 predetermined lighting flicker period.

23 4. A method as claimed in any preceding Claim, 24 wherein said image sensing array remains continuously exposed to incident light while the 26 method is performed.

28 5. A solid state image sensor adapted to perform a 29 method as claimed in any one of Claims 1 to 4.

1 6. A solid state image sensor system adapted to 2 perform a method as claimed in any one of Claims 1 3 to 4.

7. A camera incorporating a solid state image 6 sensor or image sensor system adapted to perform a 7 method as claimed in any one of Claims 1 to 4.