WO1995027367A1

WO1995027367A1 - Improvements relating to sensor arrays

Info

Publication number: WO1995027367A1
Application number: PCT/GB1995/000750
Authority: WO
Inventors: Donald Robert Sinclair; Philip Michael Gaffney
Original assignee: Seescan Plc
Priority date: 1994-03-31
Filing date: 1995-03-31
Publication date: 1995-10-12
Also published as: GB9406471D0

Abstract

A method of generating a database (7) to enable information derived from sensors (S1-S8) of an array (8) of radiation sensors to be corrected. The method comprises: a) preventing radiation from being incident on the array; b) monitoring the performance of the sensors under at least two different predetermined conditions; and, c) storing data relating to the performance of the sensors.

Description

IMPROVEMENTS RELATING TO SENSOR ARRAYS
The invention relates to sensor arrays, for example arrays of charge coupled device (CCD) sensors.

CCD arrays are used in cameras and the like. o obtain information about images to enable such images to be represented in digital form. Each element of a CCD array, when exposed to radiation such as light, accumulates charge throughout an integration period. At the end of that integration period, the charge is transferred to a shift register which is then downloaded in serial form for further processing. A problem which can arise with such sensors is charge or voltage leakage which has the effect of simulating the effect of incident light even when no light is incident. One approach to minimising this problem is to cool the sensor but this is expensive, particularly if condensation is to be prevented. Another approach is to make use of sensor elements which are permanently obscured from incident light, for example by opaque metallisation.

The values from these obscured sensors are used to apply a
DC adjustment to the values obtained from the other sensors.

In accordance with one aspect of the present invention, a method of generating a database to enable information derived from sensors of an array of radiation sensors to be corrected comprises:
a) preventing radiation from being incident on the array;
b) monitoring the performance of the sensors under at least two different predetermined conditions; and,
c) storing data relating to the performance of the sensors.

We have realised that the previous approaches have assumed that all the sensors are identical and have identical leakage characteristics. In practice, this is not the case, and the method of the invention enables a database to be generated to indicate which sensors are unacceptably liable to the leakage problem so that this can be corrected for when the array is used to scan images.

The different predetermined conditions can take a variety of forms. For example, they could correspond to different temperatures to which the array is subject.

Typically, however, the two different predetermined conditions comprise time intervals during which radiation is prevented from being incident on the array. In some cases, these different time intervals could correspond to different temperatures.

This enables the same sensor array to be characterised for different predetermined intervals and for different adjustments to be made during scanning.

In one arrangement, step b) comprises determining if any sensor indicates the reception of radiation above a predetermined level, and step c) comprises storing data defining the performance of a sensor as unacceptable if it has been monitored as indicating the receipt of radiation above the predetermined level.

Conveniently, step b) is repeated with predetermined intervals of successively longer duration, and step c) comprises storing data only for those sensors which have been found to be acceptable on previous iterations of step b).

The data stored for each sensor can take a variety of forms and in one example a set of values are stored for each predetermined condition indicating whether or not the sensor is unacceptable under that condition. In an alternative approach, a single value could be stored for each sensor, the value indicating the predetermined condition (eg interval) at which it becomes unacceptable, in that the leakage leads to an indication of the receipt of radiation above the predetermined level.

In other examples, the database could store data values relating to the charges built up on the sensors during step (b). That is, values relating to the dark currents under the different conditions.

In a further alternative, the method could comprise, prior to step (c), monitoring charges built up on the sensors during step (b), and computing a set of coefficients for a predetermined algorithm which enables corrected data to be generated for each sensor, step (c) comprising storing the coefficients. In this case, the dark currents can be monitored and then used to generate a set of coefficients for use in the predetermined algorithm.

In some cases, there could be a set of coefficients for each predetermined condition but typically there will be a single set of coefficients for all predetermined conditions.

In a typical example, two coefficients are stored for each sensor and, where appropriate, for each condition, the first coefficient relating to the data obtained from the sensor after an infinite time and the second coefficient relating to the rate at which the data approaches the first coefficient.

In accordance with a second aspect of the present invention, a method of generating an electronic representation of an image using an array of radiation sensors comprises exposing an image to radiation and sensing radiation from the image using an array of sensors which generates a set of electrical signals defining data values corresponding to the radiation incident on the sensors from respective pixels of the image; and modifying the data values in accordance with a predetermined algorithm and data stored in a previously determined database relating to the performance of the sensors under at least two different exposure conditions.

In this aspect, an improved representation of an image is obtained by reference to a database containing data relating to the performance of the sensors.

Preferably, the database has been generated according to the first aspect of the invention.

The predetermined algorithm could take a variety of forms. Preferably, however, the predetermined algorithm calculates the data values from data values of adjacent pixels. This could simply involve utilizing the value from an adjacent pixel as the value for the unacceptable sensor pixel but preferably involves averaging values from adjacent pixels. In some cases, the data values from adjacent pixels are weighted in accordance with the corresponding stored performance data prior to averaging.

In an alternative approach, the database contains a set or sets of coefficients for the sensors, the coefficients being used in a predetermined algorithm to obtain corrected data values from data output by the sensor.

In some cases, there may be two or more adjacent unacceptable pixels. In the first modification step, therefore, the value for one such pixel could involve using data from an adjacent unacceptable pixel. Preferably, therefore, the method further comprises repeating the modification step one or more times. In this way, the contribution from unacceptable pixels is progressively reduced.

Typically, the sensors will be sensitive to radiation in the optical waveband but this is not essential.

The process described for generating a database is useful in many cases but in some cases as the integration time increases, a stage is reached when the image contains too few acceptable pixels to rebuild the entire image from.

At that stage it becomes necessary to characterise the pixels more carefully. By comparing images captured at more than one integration time, we can estimate, for each pixel, what the relative leakage-resistances are to ground and to the effective supply voltage.

This determines, and can be measured by, how fast the pixel tends towards its final value, and what the final value is.

Given these measurements, and a final image, the machine can determine how much useful information is available from each pixel. Often, e. g. when a pixel exceeds the white clipping value used by the camera or framestore, this will be none, and the pixel must be estimated from its neighbours.

However, for many pixels, a useful image can be obtained by estimating what the values of the pixels would have been in the absence of leakage.

In the intermediate case of a moderately leaky pixel, the estimated value (which will not be very accurate) can be given a small (rather than 0) weighting in the average of itself and its four neighbours.

Methods according to the invention can be used in any application of sensor arrays but are particularly useful when such arrays are used in applications such as fluorescence microscopy and gel documentation.

Fluorescence levels can be low and thus sensor arrays susceptible to this"dark noise"problem can yield inaccurate results which can be overcome with the present invention.

An example of a method according to the invention will now be described with reference to the accompanying drawings, in which:
Figure 1 is a block diagram of a CCD camera assembly;
Figure 2 illustrates in block diagram form a CCD array in more detail;
Figure 3 illustrates an example of the content of the database shown in Figure 1; and,
Figures 4A and 4B illustrate an example of the values obtained for pixels from a portion of an image before and after processing respectively.

The apparatus shown in Figure 1 comprises a selfcontained charged coupled device (CCD) camera 1 which is positioned adjacent an image support 2 on which an image such as a transparency is mounted in use. A light source 3 is mounted behind the support 2 so that light from the source, typically within the optical waveband, passes through the support to an image and is received by the CCD camera 1. The construction of the camera 1 will be described in more detail below. The camera 1 generates a digital output which is fed to a separate microprocessor 4 which processes the data in the manner to be described below and is connected to a monitor 5 to display the scanned image, a keyboard 6 to enable the processor 4 to be controlled and to a database 7.

For the sake of this description, a very simple example of a CCD sensor array will be described with reference to Figure 2. Figure 2 illustrates a linear array 8 of eight photosensors (S1-S8) on which light from the source 3 will be incident. In practice, a much larger number of photosensors will be used and typically these will be arranged in a two-dimensional array. Alongside the array of photosensors 8 are provided a pair of shift registers 9,10. The shift register 9 is coupled in parallel to each of the odd numbered photosensors (S1, S3, S5, S7) in the array 8 while the shift register 10 is connected to the even numbered photosensors (S2, S4, S6, S8).

The serial outputs of the registers 9,10 are fed to a multiplexer 11 which generates a single, serial, digital output stream on a line 12 which is fed to the processor 4.

In normal operation, light is incident on each of the photosensors of the array 8 for a predetermined integration time. During that time, charge will build up within each photosensor representing the amount of incident light. At the end of the integration time, the stored charge is transferred to the respective shift registers 9,10 and the transferred values are then shifted out through the multiplexer 11. During this shifting out process, the photosensors in the array 8 start to store further charges during a subsequent integration time.

As has been explained above, one of the problems with existing CCD arrays is that some photosensors can be subject to a leakage problem so that a charge or voltage can build up in a sensor even though it is not being exposed to incident radiation.

In order to compensate for this problem, the results of an analysis of the CCD array 8 are stored in the database 7. This analysis is performed by placing a shutter 13 over the CCD camera 1 so that light cannot fall onto the sensor array 8. The processor 4 then controls the camera 1 to operate with successively longer integration time intervals and monitors the results. Thus, initially, an integration time of two seconds is set and then the content of the photosensors 8 is transferred to the shift registers 9,10, serialised by the multiplexer 11 and fed to the processor 4. The processor 4 reviews the incoming values and compares these with a predetermined value defining an acceptable voltage level. Normally, there may be a small degree of noise but otherwise no charge should be accumulated within the photosensors of the array 8 and so the predetermined level is set relatively low.

If the level is exceeded then this indicates the presence of a leakage problem and indicates that the performance of the particular photosensor is unsatisfactory. The results of this monitoring process are then stored in the database 7.

Part of the contents of the database 7 is shown in
Figure 3. The first column in Figure 3 illustrates the results for an integration time of two seconds and as can be seen the digital value of"0"is stored against each photosensor S1-S8 indicating that all the sensors generated acceptable results.

The processor 4 then repeats the process but with a longer integration time of four seconds. In this case, as can be seen, the sensor S4 generates an output whose level exceeds the predetermined level thus indicating that for an integration time of four seconds the output from the sensor 4 is unacceptable. It is then coded with the value"1"in the database 7.

The process is then repeated for longer time intervals 8,16 and so on up to 1024 seconds. Figure 3 indicates that at an integration time of 16 seconds, the sensor S5 is also found to generate an unacceptable output.

It will be appreciated that some reduction in the monitoring process can be achieved by noting that once a sensor has been found to generate an unacceptable value then it will also be unacceptable for all longer time integration periods.

Typically, this process will be undertaken by the camera user during the initial calibration stage since the acceptability of the photosensors will vary over time.

However, the database could be generated during manufacture of the camera.

Once the database contents have been set up, the camera 1 is ready for normal use. The camera is arranged to scan an image in a conventional manner, for example, by causing movement of the support 2 relative to the camera 1.

An integration time is set, for example, eight seconds, so that a single pixel of the image corresponds to the area of the image scanned by a single photosensor in the array 8 over the integration period of eight seconds. At the end of the integration period, the contents of the photosensor array are transferred to the shift registers 9,10 and then shifted through the multiplexer 11 to the processor 4. The processor 4 then stores the resultant digital values in a store (not shown).

Figure 4A illustrates a part of the contents of this store with each cell corresponding to a pixel of the image and the values being defined in some conventional manner to constitute the colour component content of each pixel. For example, the values may define colour density on a scale of 0-20.

As can be seen in Figure 4A, two of the pixels 14,15 are indicated as having a content"X". This is simply to show that the values of these pixels are unreliable since they have been generated by photosensors which at the appropriate integration time have previously been determined to generate unacceptable outputs. In practice, the processor 4 will review the list in the database 7 corresponding to the next higher integration time, in this case 16 seconds, and it can be seen from Figure 3 that this will indicate that sensors S4, S5 generate unacceptable outputs. The pixels 14,15 correspond therefore to values generated by the sensors S4, S5.

In order to replace the values for the pixels 14,15 by more reliable values, the processor 4 obtains the values for the four closest adjacent pixels and determines an average. In the case of the pixel 14, the four adjacent pixels are those labelled 15-18. For the present purposes, it will be assumed that the value for the pixel 15 (which is unacceptable) is in fact"15"as generated by the sensor. The average of the four values (15 + 8 + 4 + 9) is determined (36/4) and the resultant"9"is stored in the store at the location corresponding to pixel 14.

The processor 4 then repeats this process on the other unacceptable pixel 15 using the adjacent cells 14,19-21.

The value for the cell 14 which is used is that just calculated in the previous processing step. The average of these four values (9 + 9 + 8 + 10) is determined (36/4) and the resultant value 9 stored in the store corresponding to the cell 15. The result of this processing step is shown in Figure 4B which represents the same portion of the image as stored in the store after processing.

It will be appreciated that there are many other ways in which the pixel values could be computed, for example using more than just the four closest pixels and/or using a more complex averaging procedure.

In practice, this processing step is carried out several times to ensure that all unacceptable pixels have been fully corrected.

In the example described above, the data stored in the database defines the acceptability/unacceptability of a sensor output at a particular integration time. An alternative approach is to store coefficients of an algorithm from which a true value for a sensor can be computed from the actual value. Consider the following analysis.

If it is assumed that each sensor has some leakage to a high voltage, V+, and some leakage to a low voltage, V-, and that these leakages are resistive then the effect is of a single resistor connected to a voltage Vc between the two. This is the voltage the sensor tends towards as the integration-time increases to infinity and is defined by the equation
V (T) = Vc (l-exp (-T/(RC))) (1)
where V (T) is the voltage on the sensor at time T, and
C and R are defined by the equations:
I (T) = (Vc-V)/R
V (T) = Q (T)/C
where I (T) is the current flowing into the sensor at time T and
Q (T) = is the charge in the sensor at time T.

Thus, Vc constitutes one coefficient which defines the voltage towards which the voltage of the sensor tends as the integration-time increases to infinity and RC defines a second coefficient constituting a measure of the rate at which this occurs.

During a calibration stage, values of V (T) can be measured for a number of time intervals T from which values for Vc and RC can be computed for each sensor. In general, a single pair of these coefficients will be generated for each sensor and stored in a database (corresponding to the database 7) although it is possible that more than one set of coefficients could be stored for each sensor at different integration times and which are used appropriately in the later analysis.

During an imaging process, a photo-electric current Ip is generated due to the charge built up on the sensor as a result of the image being scanned so that the resultant current from the sensor is given by:
I (T) = Ip + (Vc-V (T))/R
= (RIp + Vc-V)/R (2)
and
V (T) = (Vc+IpR) (1-exp (-T/RC))) (3)
In order to perform noise-correction on a given pixel where the voltage read from the corresponding sensor is Vp, it is necessary to estimate:
1) the value of IpT/C, i. e. the voltage the cell
would have reached without dark-current, and
2) the likely error in that value.

As a first step, the processor 4 computes an estimated value for IpT/C as follows:
Vp (T) = (Vc+IpR) (l-exp (-T/RC)) (4)
The right hand part of this expression can be computed from the stored coefficient RC for that integration time T and will be referred to as K1.

Thus,
Vp (T) = (Vc+IpR) Kl (5)
(Vp (T)/Kl)-Vc = IpR = (Ip/C) (RC) (6)
and thus:
Vest=IpT/C = (Vp/Kl-Vc)/RC (7)
The above computation can be used in a variety of ways to obtain a corrected output voltage.

Method A
In this method, it is assumed that the actual voltage
IpT/C is somewhere between values of Vest estimated using
Vp (T) +l and Vp (T)-l, assuming Vp (T) is digitized to an integer.

This then provides for every pixel a minimum, maximum and expected value of the voltage, the minimum and maximum values being denoted LowerBound and UpperBound.

The initial, estimated value will be used as the starting value in the iterative computation of the resultant value. The least noisy result is obtained by minimising the difference between the value of each pixel and an average of its neighbours, subject to the pixel values not falling outside their constraining values as described above. This can be achieved by carrying out several passes through the image, replacing each pixel by:
min (UpperBound, max (LowerBound, Vavg))
where Vavg is an average, with constant weights (usually 1) of the current result-values of (typically 4 or 8) neighbouring pixels.

MethodJs
This is similar to method A but computes an expected value Vest and an ErrorWeight for that value, and finds
Vout minimising:
(Vout-Vest) 2 (ErrorWeight) = (Vout-Vavg) 2 (1-ErrorWeight)
(Vavg is an average of the output-image-values for the neighbour, which means several passes of the image may be needed for convergence, as is true for method A).

Thus ErrorWeight 1.0 corresponds to very good pixel, and 0.0 to be a very bad one.

Method B has the advantages of
(a) being more"fuzzy", allowing a compromise between evidence from neighbouring pixels and the central one, and
(b) needing less memory than method A.

One choice of ErrorWeight for pixels which are not out-of-range is to use UpperBound and LowerBound as for method A, and set ErrorWeight = LowerBound/UpperBound 1- (UpperBound-LowerBound)/UpperBound.

A simple implementation of method B which can achieve reasonably good performance is to set Vest = Vp and
ErrorWeight = 0 whenever this is not a good estimate for the particular interval being used, and 1.0 elsewhere.

Claims

CLAIMS 1. A method of generating a database to enable information derived from sensors of an array of radiation sensors to be corrected, the method comprising: a) preventing radiation from being incident on the array; b) monitoring the performance of the sensors under at least two different predetermined conditions; and, c) storing data relating to the performance of the sensors.

2. A method according to claim 1, wherein the two different predetermined conditions comprise time intervals during which radiation is prevented from being incident on the array.

3. A method according to claim 2, wherein the predetermined intervals comprise powers of two seconds.

4. A method according to at least claim 2, wherein the longest predetermined interval is 1024 seconds.

5. A method according to any of claims 2 to 4, wherein data is stored for all sensors for each interval.

6. A method according to any of the preceding claims, wherein step b) comprises determining if any sensor indicates the reception of radiation above a predetermined level, and step c) comprises storing data defining the performance of a sensor as unacceptable if it has been monitored as indicating the receipt of radiation above the predetermined level.

7. A method according to claim 6, when dependent on claim 2, wherein step b) is repeated with predetermined intervals of successively longer duration, and step c) comprises storing data only for those sensors which have been found to be acceptable on previous iterations of step b).

8. A method according to any of claims 1 to 5, wherein the data comprises values relating to the charges built up on the sensors during step b).

9. A method according to any of claims 1 to 5, further comprising, prior to step (c), monitoring charges built up on the sensors during step (b), and computing a set of coefficients for a predetermined algorithm which enables corrected data to be generated for each sensor, step (c) comprising storing the coefficients.

10. A method according to claim 9, wherein a single set of coefficients is generated for all predetermined conditions.

11. A method according to claim 9 or claim 10, wherein two coefficients are stored for each sensor and, where appropriate, for each condition, the first coefficient relating to the data obtained from the sensor after an infinite time and the second coefficient relating to the rate at which the data approaches the first coefficient.

12. A method according to any of the preceding claims, wherein the sensors comprise charged coupled devices.

13. A method of generating an electronic representation of an image using an array of radiation sensors, the method comprising exposing an image to radiation and sensing radiation from the image using an array of sensors which generates a set of electrical signals defining data values corresponding to the radiation incident on the sensors from respective pixels of the image; and modifying the data values in accordance with a predetermined algorithm and data stored in a previously determined database relating to the performance of the sensors under at least two different exposure conditions.

14. A method according to claim 13, wherein the database has been generated using a method according to any of claims 1 to 12.

15. A method according to claim 13 or claim 14, wherein the predetermined algorithm calculates a data value from data values of adjacent pixels.

16. A method according to claim 15, wherein the modification step comprises averaging values from adjacent pixels.

17. A method according to claim 16, wherein the data values from adjacent pixels are weighted in accordance with the corresponding stored performance data prior to averaging.

18. A method according to any of claims 13 to 17, comprising repeating the modification step one or more times.

19. Sensor apparatus comprising an array of radiation sensors; a store for storing a database indicating the performance of the sensors in the array; and processing means for monitoring the performance of the sensors under at least two predetermined conditions and for storing data relating to the performance of the sensors.

20. Apparatus according to claim 19, wherein said processing means is adapted to carry out a method according to any of claims 1 to 12.

21. Apparatus for generating an electronic representation of an image, the apparatus comprising an array of radiation sensors; a store containing a database relating to the performance of the sensors under at least two different predetermined conditions; and processing means for modifying data values obtained from the sensors in accordance with a predetermined algorithm and data stored in the database so as to generate an electronic representation of the image.

22. Apparatus according to claim 21, adapted to carry out a method according to any of claims 13 to 18.

23. Apparatus according to any of claims 19 to 22, wherein the sensors comprise charged coupled devices.