GB2298542A - Image motion compensation - Google Patents

Abstract

An image motion-compensation device for a charge coupled device (CCD) sensor of, for example, the frame, frame-interline, or interline transfer type has means (32) for generating control signals to move charge (38) over the imaging area (29) of the CCD during the accumulation period so that at the end of the period the accumulated charge (38) is transferred into the CCD's sensor storage area, whereby the smaller the relative movement between a source of light incident on the CCD imaging area and its corresponding portion of moving charge, the smaller will be the distribution of the accumulated charge (38) of that portion and corresponding blurring of the image (30).

Description

Imaae motion compensation The present invention relates to a method and a device for image motion compensation for use with a charge coupled device (CCD) camera sensor, particularly, although not exclusively a television camera.

It is well known that objects having movement relative to a fixed scene viewed by a television camera can appear with blurred detail on a monitor if significant movement occurs during the integration time of a television field.

In Europe, for example, the integration time for a standard television camera is equivalent to the television field period of 2OmS.

Television cameras using picture sensors based on charge coupled device (CCD) technology can reduce the blurring caused by relative motion by means of electronic shuttering or gating. Shuttering or gating reduces the integration period during the television field period by reducing the time during which the camera is sensitive to light. When shuttered, the CCD sensor integrates the scene for a period of time less than the standard field period; thus less light is received and less signal is available.

Consequently, the camera is less sensitive; for example a camera which is shuttered to imS has only 1/20th of its sensitivity when operated without shuttering.

In certain applications where relative movement within a scene is inevitable, such as, for example, the viewing of objects moving on a conveyor belt past a fixed camera, ground surveillance from a camera mounted in an aircraft, or vehicle identification by number plate or otherwise from an overhead gantry above a road, there may be insufficient light available to allow sufficiently short shuttering to be carried out. This is particularly a problem where it is not possible to provide sufficient additional illumination because of practical limitations and/or safety requirements. For instance, in the case of vehicle surveillance from an overhead gantry, the additional illumination provided by an intense flash of light may disturb and/or distract drivers.

It is therefore an object of the present invention to provide both a method and a device for image motion compensation for use with a charge coupled device (CCD) camera sensor, which overcomes the disadvantages of the prior art, including having improved sensitivity, and obviating or reducing the requirement for additional illumination.

According to a first aspect of the present invention there is provided a method of image motion compensation for use with a CCD sensor having a imaging area for accumulating a set of photo-generated charges, comprising the step of generating control signals to transfer the charge accumulated on said imaging area into a storage area at the end of the charge accumulation period of said sensor, characterised by including an initial step, during the charge accumulation period, of generating additional control signals to move charge across the image area, so that at the end of said accumulation period, the smaller the relative movement between a source of light incident on said imaging area, and its corresponding portion of charge the smaller will be the distribution of that corresponding portion of accumulated charge and corresponding blurring of the image.

Preferably, the method includes the further step of providing timing signals to said control means to adjust the rate of movement of said charge across the image area.

According to a second aspect of the invention, there is provided an image motion compensation device for use with a CCD sensor having an imaging area, a storage area, and means for transferring accumulated charge from said imaging area to said storage area at the end of a accumulation period, the device comprising means for generating control signals to move charge across the imaging area during the accumulation period, so that at the end of said period the accumulated charge is transferred into the sensor storage area.

Preferably, the device includes a timing unit for providing timing signals to said means for generating control signals.

Conveniently, the timing unit may be coupled to detector means for detecting the speed and direction of an object to be viewed by said imaging means so that the speed and direction of charge movement may be varied accordingly to reduce blurring of said viewed image. Advantageously, the timing unit may be capable of adjusting the speed and direction of said charge movement for each accumulation or picture field period.

It is further envisaged that the speed of a viewed object may be determined by processing the resulting video signal.

In order to aid in understanding the invention a specific embodiment thereof will now be described by way of example and with reference to the accompanying drawings, in which: Figure 1 is a diagrammatic view of a typical frame transfer architecture CCD sensor; Figure 2 is a similar view of a typical interline transfer architecture CCD sensor; Figure 3 is a similar view of a typical frameinterline transfer architecture CCD sensor; Figures 4 (a to e) are a series of diagrammatic views illustrating the principle of charge transfer within a CCD sensor using four-phase clocking; Figure 5 is a view similar to that of Figure 4, showing an electrode arrangement for charge transfer utilising four-phase clocking; Figure 6 is a graphical view of drive waveforms for use with the electrode arrangement of Figure 5 to provide one shift cycle forward clocking;; Figure 7 is a similar view of wave forms for one shift cycle reverse clocking; Figure 8 is a perspective view of a conventional CCD sensor having a stationary image projected thereon together with a corresponding charge accumulation profile on the line Z-Z; Figure 9 is a view similar to that of Figure 8, but with an image having moderate movement; Figure 10 is a view similar to Figure 8, but with an image having considerable movement; and Figure 11 is a part perspective, part diagrammatical view of a CCD incorporating a image motion compensation device according to the present invention.

Figures 1, 2 and 3 show three common forms of CCD sensor transfer architecture which are suitable for use with an image compensation device and which are, respectively, a frame transfer architecture A, an interline transfer architecture B, and a frame-interline transfer architecture C.

In each case the basic architecture includes an imaging area 10 which receives and integrates light from an optical system (not shown) to store a charge thereon, and a storage area 12 into which the accumulated charge is rapidly moved at the end of the integration period, thus allowing charge once more to accumulate on the imaging area 10 whilst the charge pattern within the storage register 12 is simultaneously read out to a readout register 14 for conversion into a voltage signal 15 for subsequent display and/or recording.

The fundamental operation of a CCD sensor lies in the ability to move packets of charge in a very efficient manner. Referring to Figures 4 and 5, there is shown a four-phase clocked frame transfer CCD sensor A having a plurality of sets of four-phase electrodes 2,3,4.

Figure 4(a) shows a section through the sensor A along the vertical scanning direction. The electrodes 2,3,4, are evenly spaced apart at regular intervals along the surface of the sensor A, and are connected and driven as sets of four by clock driving circuits (not shown) (Figure 5). When, in use, photons of light impinge on the sensor, a proportion will cause hole-electron pairs to be generated.The electrons will be attracted to a region of the sensor under the electrodes 1 and 2 that are set by their respective clock driving circuit to be at a relatively more positive potential, whilst those electrodes 3 and 4 set by their respective clock driving circuit to be at a relatively lower potential repel the electrons and thus cause them to accumulate under the nearest relatively positive electrodes 1 and 2. Hence, the resulting socalled "multi-phase clocking" defines a set of discrete potential wells which in turn define the vertical position and extent of a pixel or smallest picture element on the sensor. In the case of the frame transfer sensor A, the extent of the pixel in the orthogonal or horizontal direction, is defined by doped channels (not shown).

The sequence of Figures 4(a) to (e) show the steps by which packets of charge 17 may be rapidly moved along the section of the sensor A, at the end of the charge accumulation or integration period, by driving each set of the four electrodes 2, f3, and f4, so that their potentials are varied between a low voltage state Ov and a more positive or high voltage state +V.

Thus, in Figure 4(a), the electrodes 1 and 2 are both at a positive voltage whilst electrodes 03 and f4 are at a lower voltage, these voltages being known as "rest voltages". Consequently, photo-generated negative charge (electrons) accumulate as a packet under electrodes 1 and 2, the packet defining both the size and position of a pixel.

Figure 4(b) shows the next stage in the charge movement operation. Electrode 3 is taken to the positive potential, attracting the charge under electrodes f1 and 2, whilst at the same time f1 is reduced in potential to the lower voltage, so repelling the charge thereunder. The packet of charge therefore moves to the right (in the sense shown on the diagram) by a distance corresponding to the spacing between adjacent ones of the electrodes, i.e. one quarter of the width of the pixel defined by the packet of charge.

Figure 4(c) shows the following stage in which the electrode f4 is taken to the positive potential whilst the electrode 2 is reduced to in potential to the lower voltage so that the packet of charge moves a further distance to the right, which distance corresponds to the spacing between adjacent ones of the electrodes (f).

Figure 4(d) shows the next stage in which the electrode fl is taken to the positive potential +V, whilst the electrode 3 is reduced to the lower voltage Ov, so that the packet of charge moves a further quarter of one pixel to the right, this movement corresponding to the difference in spacing between adjacent ones of the electrodes ().

Figure 4(e) shows the final stage in the sequence in which the potential at each of the electrodes returns to the levels or rest voltages shown in Figure 4(a) thus moving the packet of charge a further quarter of one pixel to the right. Hence the packet of charge which was originally at one pixel position has now moved through a distance corresponding to four clock phases, i.e. a complete pixel to the right.

In a conventional frame transfer CCD sensor A, the above technique is used to transfer charge rapidly from the imaging area 10 to a storage area 12 at the end of the charge accumulation period.

The wave forms provided to each of the four electrodes 2,3,4, of a four-phase clocked device are shown in Figure 6. The wave forms shown are the sequence for one picture field of a 2:1 interlaced standard. The second interlaced field is formed by an alternative sequence of rest voltages applied to the same electrodes. For example, if the first interlaced field has rest voltages at the electrodes 1 and 2 set at a positive voltage whilst electrodes 3 and 4 are at a lower voltage, then the second interlaced field will have voltages at the 1 and 2 electrodes which are set at the lower voltage Ov whilst electrodes 3 and f4 are at a the positive voltage +V.

Accordingly the charge collection area corresponding to each pixel in the second field is displaced by half of one pixel to the right (in Figures 4(a) to (e)).

It will be apparent from the above, that the packets of charge may equally well be moved to the left (in Figure 4(a) to (e)), by applying a suitable sequence of voltages to the four electrodes (as shown in the waveforms of Figure 7).

Turning to the problem of image blur caused by movement of a subject within a picture frame, Figures 8, 9 and 10 illustrate the respective charge patterns 18,20,22 resulting from an image 21 of a stationary letter F, a moderately moving letter F, and a rapidly moving letter F, being projected onto a CCD sensor 24 of the prior art.

From Figure 9 and 10, it will be noted that significant blur 26,28 occurs along the edges of the letter orthogonal to the direction of movement (indicated by the arrow). The blurring is due to the relatively long integration time of a conventional CCD sensor 24 in relation to the velocity of the image 21 across the surface of the sensor 24, such that charge pattern 20,22 is spread over the surface of sensor 24 rather than being concentrated on a fixed set of pixels as shown in Figure 8.

Referring now to Figure 11, there is shown an image 30 of a rapidly moving letter F projected on to a CCD sensor 29 of the present invention, in which clock electrodes f1, 2, 03, and 4, are connected to a multi-phase clock driving circuit 32 which receives input signals from a camera timing generator 31 and an oscillator 34. The oscillator 34 is coupled to a variable speed controller 36.

To compensate for movement of the image, the clock driving circuit 32 is driven during the integration period of the CCD sensor 29 by the oscillator 34 to go through a number of complete charge movement cycles at a frequency related to the movement of the image 30 over the sensor 29. At the end of the integration period, the clock circuit is used to transfer the accumulated charge to a storage area (not shown) in a conventional manner. By effectively moving the charge pattern during the integration period, the edge detail of the image 30 having movement in the direction of the moving charge pattern 38 will remain sharply detailed, whilst stationary objects in the scene will become blurred.

This loss of detail in the background is not important in applications of the present invention such as those mentioned above.

Where the charge is being moved to track a subject, it may be moved in increments corresponding to the distance between adjacent ones of the electrodes (f). It will be clear to a person skilled in the art that this will be some fraction of a pixel depending on the number of clock lines.

Furthermore, the integration or accumulation period for the charge can be an entire field period, hence the sensitivity of the sensor is equal to that of a stationary image.

In a non-illustrated embodiment the above method may be applied to frame-interline transfer sensors, in which case a transfer pulse is applied by the clock circuit at the end of each multi-phase sequence when the charge has been moved down in the storage register by exactly one pixel. Equally, the method may be applied by one skilled in the art to interline transfer sensors.

Finally, it will be apparent to one skilled in the art that the present invention may be applied to any form of camera which utilises a CCD sensor such as, for instance, a still camera where an image is stored as a single frame electronically for later retrieval by a suitably programmed computer or the like.

Claims (16)

Claims:

1. An image motion-compensation device for use with a charge coupled device (CCD) sensor having an imaging area, a storage area, and means for transferring accumulated charge from said imaging area to said storage area at the end of an accumulation period, the device comprising means for generating control signals to move charge over the imaging area during the accumulation period, so that at the end of said period the accumulated charge is transferred into the sensor storage area, whereby the smaller the relative movement between a source of light incident on said imaging area and its corresponding portion of moving charge, the smaller will be the distribution of the accumulated charge of that portion and corresponding blurring of the image.

2. A device as claimed in Claim 1, in which said charge movement over said imaging area is in a substantially transverse direction.

3. A device as claimed in Claim 1, in which said charge movement over said imaging area is in a substantially longitudinal direction.

4. A device as claimed in any preceding Claim, which includes a timing unit for providing timing signals to said means for generating control signals to determine the rate of movement of charge over said imaging area and thereby compensate for movement of said image.

5. A device as claimed in Claim 4, in which the timing unit includes means for adjusting the speed and direction of said charge movement for each accumulation or picture field period.

6. A device as claimed in Claim 4 or Claim 5, in which said timing unit is coupled, in use, to detector means for detecting the speed and direction of said source whereby the speed and direction of charge movement may be varied accordingly to reduce the distribution of charge of said image.

7. A device as claimed in any one of Claims 1 to 5, which further includes processing means for determining the motion of a source from the distribution of its accumulated charge held in said storage area.

8. A device as claimed in Claim 7 as appendant to Claim 4 or Claim 5, in which said timing means varies the frequency of timing signals in response to an output from said processing means said output being proportional to the velocity of a source.

9. A device as claimed in any preceding Claim, in which the CCD is a frame-interline transfer sensor.

10. A device as claimed in any one of Claims 1 to 8, in which the CCD is an interline transfer sensor.

11. A still camera including a CCD sensor and means for storing an image as a single frame, and incorporating a device as claimed in any preceding Claim.

12. A vehicle number plate reader including a CCD sensor, means for storing an image and processing means for deriving a vehicle index number from said stored image, and incorporating a device as claimed in any one of Claims 1 to 10.

13. A method of image motion-compensation for use with a CCD sensor having an imaging area for accumulating a set of photo-generated charges, comprising the step of generating control signals to transfer the charge accumulated on said imaging area into a storage area at the end of the charge accumulation period of said sensor, and including an initial step, during the charge accumulation period, of generating additional control signals to move the charge over the image area, so that at the end of said accumulation period, the smaller the relative movement between a source of light incident on said imaging area and its corresponding portion of moving charge, the smaller will be the distribution of the accumulated charge of that portion and corresponding blurring of the image.

14. A method as claimed in Claim 13, including the further step of providing timing signals to said control means to adjust the rate of movement of said charge over the image area.

15. An image motion compensation device substantially as described herein with reference to Figure 11 of the accompanying drawings.

16. The features herein described or their equivalents in any patentably novel combination.