GB2061660A - Modifying the contrast of an image - Google Patents

Abstract

A method of image contrast enhancement compares the brightness value of a pixel at a selected position with the individual brightness values of each other pixel in a surrounding local area and determines how many values are greater than, equal to or less than the selected pixel value. The result of this determination is used to produce an output brightness code for the selected pixel. The local area is rippled by at least one pixel at a time over the frame with the brightness value of the pixel at the selected position in the local area being recoded in accordance with the brightness of the pixels around it. The method may be used for infrared systems, medical X-rays and perhaps television.

Description

SPECIFICATION Real time image contrast modification The present invention relates to a real time image contrast modifying technique which may be used in infra-red systems, medical Xrays and perhaps television systems.

Various real time image contrast enhancement techniques are known. For example United States Patent Specification 3,983,320 (corresponding to British Patent Specification No. 1,504,675) discloses a real time raster display histogram equalization sytem in which a local area of 32 X 32 pixels is scanned as a raster over the frame of the picture or scene.

A histogram is formed from the amplitudes of pixels occurring in the local area and then the histogram is equalized to provide the recoding strategy which is applied to four central pixels of the received information so that image contrast of the four pixels is enhanced. The local area is indexed horizontally by 4 pixels and the cycle is repeated. In order to simplify the operation of the system, the picture is held in a frame store and mini-histograms of areas of 4 pixels are made and are summed as required to provide a histogram of 32 pixels X 32 pixels. Although the system is satisfactory as far as histogram equalization is concerned, such an operator is not always suitable for getting the best image enhancement and one may want to try other histogram distributions such as Gaussian, Poisson, exponential and hyperbolic.Additionally this system requires a great deal of hardware, particular stores, which makes the system expensive and complicated.

An object of the invention is to provide a real time image contrast enhancement system which is able to operate on input information and be able to apply different operators as desired and which is simple in its construction and operation.

According to the present invention there is provided a method of modifying the contrast of an image in real time, comprising receiving video information in the form of scanned pixels, comparing in sequence the information from each pixel with that of its surrounding macro group and modifying the information relating to each pixel on a statistical basis by the information derived from its surrounding macro group.

The macro group of pixels may comprise pixels in a local area which is small relative to the area of a frame containing the image, the pixel information being modified relating to a pixel at a selected position in the local area; wherein after each comparison the local area is indexed by at least one pixel in at least one direction.

In an embodiment of the method in accordance with the present invention the results of the comparison are a plurality of decisions indicating whether each of the other pixels amplitudes are greater than, equal to, or less than the pixel of interest at the selected position. These results may be weighted prior to being summed and the sum is adjusted in value in accordance with a selected transform code.

The pixel amplitudes may be processed either as digital or analogue signals and the local area may be moved continuously over the frame, for example in raster fashion. In processing the information digitally it is sometimes desirable to express the adjusted sum signal in a greater number of bits than in the signal prior to being adjusted in order to avoid losing grey-level resolution when using some non-linear recording laws. Alternatively it may be desirable to compress the number of grey levels in the input video information into a lower number in order to be able to display the modified information on a suitable reproducing apparatus.

In the case where a macro group contains information of pixels in unrelated parts of the frame, the pixel information relating to the pixel at the selected position in the local area is processed normally without being modified on a statistical basis by the information derived from the macro group. Such a step is necessary because the pixel information in one part of a frame may be completely different from that in an unrelated second part of a frame in which case the statistical information derived would be unsuitable for recoding the information at a selected position in the macro group.

The present invention also provides an apparatus for modifying the contrast of an image in real time, comprising means for scanning a frame containing and providing video information in the form of scanned pixels, a comparator for comparing in sequence the information from each pixel with that of its surrounding macro group, and means for modifying the information relating to each pixel on a statisti cal basis by the information derived from its surrounding macro group.

The apparatus may further comprise a rip ple store for storing the information relating to the macro group of pixels which are contained in a local area of an image frame, the local area being indexed by at least one pixel in at least one direction in response to the receipt of a pixel value on an input of the ripple store.

The present invention will now be described, by way of example, with reference to the accompanying drawings, wherein: Figure 1 is a frame of 1 2 X 1 2 pixels, Figure 2 is a block schematic diagram of a system for enhancing the central pixel of a local area comprising 5 X 3 pixels, Figure 3 is a more detailed block schematic circuit diagram of one embodiment of an apparatus for modifying an image wherein the pixel amplitude values are expressed in digital form, Figures 4(a) to (m) serve to explain the operation of a ripple store, and Figure 5 is a block schematic circuit diagram of another embodiment of an apparatus for modifying an image wherein the pixel amplitude values are expressed in analogue form.

Referring to Fig. 1, an area 20 of say 1 2 X 1 2 pixels is scanned line by line using a suitable video/electrical transducing apparatus (not shown) e.g. a television camera. As the frame is scanned, a macro group or local area 22 of pixels, say an area of 5 x 3 pixels, is taken and the contrast information in the area is used to modify the contrast of a pixel at a selected position, for example the central pixel A in the rectangle denoted by heavy lines, and the pixel B in the rectangle denoted by broken lines. Conveniently the local area comprises an odd number of pixels so that the pixel whose grey level value is of interest is the central one. However this does preclude a pixel of interest being at another predetermined location in the local area.

The process of modifying the grey level of the pixel at the selected position in each local area is continuous because each new local area is formed at a time interval corresponding to one pixel later than the previously formed local area. In Fig. 1 the first 5 X 3 local area comprises H1 to 5, Vitro 3 and the new local area formed one pixel later is H2 to 6, VI to 3. When the area H8 to 12, VI to 3 has been selected, the next area comprises H9 to 12, Vitro 3 + H1, V2 to 4, accordingly no modification of the pixel at H 11, V2 takes place because the input grey levels are from two unrelated parts of the frame. This also happens during the following four local areas formed until the area H1 to 5, V2 to 4 is selected and the central pixel at H3, V3 is modified.

Fig. 2 is a block schematic diagram of a system for applying contrast enhancement to a selected pixel position in dependence on the grey level information in the local area selected. The input video information is applied to a ripple store 30 which is adapted to store the grey level information of the pixels of the selected local area. For convenience the number of pixels in the local area is N. The grey level information of all but the central pixel is conveyed by a bus 32 to a group 34 of (N-1) comparators. The grey level information concerning the central pixel is applied via a line 36 to a separate input of the group 34 of comparators and serves as a reference against which the grey level information of each of the (N-l) pixels is compared.Each comparator of the group decides for example whether the grey level applied to one input is greater or lesser than, or equal to, the grey level of the reference pixel. The equals condition may encompass a group of grey levels close to that of the particular central pixel under consideration. The (N-1) decisions from the comparators are conveyed by a bus 38 to a network 40 in which they are weighted using for example a non-linear network and summed, the result of this operation is applied, for example to address a recode module 42, in this instance a RAM, and the output of the module is the recoded grey level value to be used to describe the selected pixel in the local area being considered. The recode module 42 may be programmed to recode the incoming video information to provide a grey-level histogram distribution that follows a linear or nonlinear law.In the latter instance Gaussian, Poisson, exponential and hyperbolic distributions are of particular interest. One pixel later the process is repeated for a new local area.

By means of coordinate flags the system is able to determine when the incoming pixel amplitude values are from unrelated areas of the frame and consequently does not modify the grey level of the central pixel which is stored or displayed as it is received.

There is no need to count the distribution of the input grey levels to form a histogram as is done in the prior art, such as United States Patent Specification No. 3,983,320 because in the present invention the position of a single brightness value is identified in the histogram of the input grey levels or brightness by knowing how may values are greater, less than or equal to the value of interest and possessing knowledge of the transfer function of the system.

Fig. 3 is a block schematic circuit diagram of one embodiment of an apparatus for carrying out the method in accordance with the present invention. For convenience of description whenever possible the same reference numerals have been used in Fig. 3 to identify those parts corresponding to the ones shown in Fig. 2. Additionally, for the sake of consistency the macro group or local area will be 5 pixels wide and 3 pixels deep so that N = 15.

In the embodiment to be described the input information will have up to 256 (28) grey-levels and the modified pixel will have one of 32 (25) grey levels. The illustrated apparatus is a digital one and accordingly each line indicating a conductor will in fact comprise a plurality of conductors, the precise number being indicated alongside a short oblique line.

The ripple store 30 is designed to hold 5 X 3 pixels at any one instant following the initial filling of the store. In Fig. 3 the store 30 comprises three sets of delay circuits 52A to 52D, 56A to 56D and 60A to 60D each having a storage or delay time of one pixel and two other delay circuits 54 and 58 each having a storage or delay time of line (1 H).

The delay circuits 52A to 52D, 56A to 56D and 60A to 60D of each set are connected in series. An input 50 of the ripple store 30 is connected to the input of the delay circuits, 52A and 54. The output of the delay circuit 54 is connected to the input of the delay circuits 56A and 58 and the output of the circuit 58 is connected to the input of the delay circuit 60A. Outputs from the store 30 are derived from the locations a to o, the location h representing the information of the central pixel. In a constructional embodiment of the store 30, each of the delay circuits 52A to 52D, 56A to 56D and 60A to 60D may comprise part of a 74 LS 374 element.

In operation as a picture or frame is raster scanned the digital values of the input grey levels are applied to the input 50. Each pixel amplitude value on the input 50 is applied to the delay circuits 52A and 54. As successive pixel values from the first line are received, the pixel values in the first set of delay circuits 52A to 52D are shifted from one delay circuit to the next in a ripple fashion, and also they are accumulated in the delay circuit 54. At the beginning of the next (second) line of the raster scan, the top set of the delay circuits 52A to 52D begin to receive pixel values from the second line of the scan, whereas the middle set of delay circuits 56A to 56D begin to receive the pixel values delayed by one line period from the delay circuit 54. These delayed pixel values are also applied to the delay circuit or store 58.

Atthe beginning of the next following (third) line the newly received pixel values are applied to the delay circuits 52A to 52D and 54. Simultaneously the pixel values stored in the delay circuit 54 and corresponding to the second line are applied to the delay circuits 56A to 56D and 58, and the pixel values stored in the delay circuit 58 and corresponding to the first line scanned are applied to the delay circuits 60A to 60D. Once the values of the first five pixels of the third line scanned have been entered, the ripple store 30 is full.

To compare the value of the central pixel with the other pixel values currently present at the locations a to g and ito o in the ripple store 30, the group 34 of comparators comprises a plurality of digital comparators 64, such as N74 LS85 amplifiers, each having an input A connected to a respective one of the locations a to g and i to o and an input B for receiving the central pixel value derived from the location h. In the comparator 34, each digital comparator 64 determines whether the pixel value on the input A is greater, less than or equal to the central pixel value on the input B. As explained with reference to Fig. 2 "equals" can include a range of values close to the value of interest. The outputs of the digital comparators 64 comprise decisions which can be represented by two binary bits, for example 00 "greater than", 10 "equals" and 01 "less than".These decisions are applied to the sum and weighting network 40 which may comprise a PROM. In the presently described embodiment there is a compression of grey levels between 256 input levels and 32 output levels. Accordingly the output from the network 40 comprises a five bit number relating to an output grey level.

The five bit number is applied to the recode module 42 which may comprise a RAM, where an output grey level relating to the central pixel is produced in accordance with a desired recode strategy. The 5-bit digital output from the module 42 may be converted to an analogue value for display on say a television receiver.

Depending on the resolution required, the number n of bits in the output of the recode module 42 may comprise the same or a different number of bits to that applied to its input. Generally n lies in the range 5 to 8.

Eight bit resolution may be required, for example, when recoding with other than linear laws. Further although the network 40 and the module 42 have been shown as two separate circuit elements, their functions may be combined.

One pixel later, due to the rippling of the store 30, new pixel values are present at the locations e, j and o, the previous pixel values at these locations having moved to locations d, i and n via the delay circuits 52A, 56A and 60A, respectively, the contents of which delay circuits have been moved one delay circuit to the right. In so doing there is a new central value and the process is repeated with the amplitude value of the central pixel being influenced by its related neighbours. When the rippling of the store approaches the right hand side of the frame or the bottom right hand corner of the frame, where the delay circuits 52A to 52D, 56A to 56D and 60A to 60D do not contain related pixel values then the enhanced "central" pixel value is ignored because it is invalid.

The rippling of the store 30 is illustrated in Fig. 4. At the top of the Fig., is a schematic view of the four corner areas of a rectangular or square frame. The pixels are numbered 1, 2, 3. . n-2, n-1, going crosswise and the rows are numbered 1, 2, 3. . X, Y, Z. For ease of identification the first and last pixel in each row will be given the subscript of the row number. In the local areas shown in Figs.

4(a) to 4(m) the pixel numbers to the left of a "1" or to the right of an "n" are the pixels relating to the row indicated by the subscript associated with the "1" or ''n''.

Thus Figs. 4(a) to (d) illustrate the ripple of the store across the first three rows of the frame. The central pixel of interest which is having its amplitude value enhanced or transformed has been surrounded by a square. In the case of (a), (b) and (c) the situation is shown for a change of one pixel at a time.

Figs. 4(e) to 4(i) continue from Fig. 4(d) and illustrate how the store 30 ripples from rows 1 to 3 to rows 2 to 4.

As will be realised the pixel values of row 1 are replaced by pixel values from row 2 as they are read-out of the delay circuit 58, similarly in the middle row of the local area shown, the values of row 2 are replaced by the values of row 3 read-out of the delay store 54 and in the top row, the values of the row 3 are replaced by the newly received values relating to row 4. In Figs 4(e) to 4(h) as the pixel values are taken from unrelated parts of the frame, namely the right and left sides, then the "cental" pixel would not be influenced by the immediately adjacent pixel values. Accordingly these values cannot be used.

Figs 4(j) to (m) illustrated with two (Figs.

(k) and (I)) intervening positions the ripple effect from a complete local area (Fig. 4(j)) in the bottom right hand corner of the frame to a complete local area (Fig. 4(m)) in the top left hand corner of the frame. It is felt unnecessary to set out the numerous 1 pixel steps because a person skilled in the art should have no difficulty in understanding this operation in view of the previously described operation of the pixel delay circuits 52, 56 and 60 and the line delay circuits 54, 58.

Fig. 5 illustrates in block schematic form an embodiment of the invention which processes analogue values of the pixels in the local area, for example a 5 X 3 local area as considered in Figs. 3 and 4. Where appropriate the same reference numerals have been used to illustrate the corresponding parts as in the previous Figures.

The ripple store 30 comprises three tapped shift registers 66, 68 and 70 which may comprise charge-coupled devices or bucket brigade devices and are each capable of storing the pixel values for one row of a frame. In reality the shift registers 68 and 70 may comprise an untapped bucket brigade device 68A, 70A, such as a Reticon SAD-1024A and a tapped bucket brigade device 68B, 70B, such as a Reticon TAD-32A which may also comprise the shift register 66. The input 50 to the ripple store 30 is connected to the shift register 66 and the input of the shift register 68. The output of the shift register 68A is connected to the input of the shift register 70. External connections are made to five selected taps 1 to 5 of each shift register 66, 68B and 70B which connections are coupled to the comparator 34.

The group 34 of comparators comprises fourteen two input comparators 72 having one input a reference input, connected by a common line 74 to the tap 3 of the shift register 68B by which the value of the central pixel is obtained. The other input of each comparator 73 is connected, respectively, to the taps 1 to 5 of the shift registers 66 and 70B and the taps 1, 2, 4 and 5 of the shift register 68B. The comparators 72 may each comprise half of an NE 522 circuit element.

The outputs of the comparators 72 are summed, and weighted if-desired, using an analogue adder network 40 comprising an operation amplifier 76, such as a LH0032, having one input connected to a reference voltage level, e.g. ground, and another input connected to resistors 78 which are connected respectively to the outputs of the comparators 72 and to a feedback resistor 80.

The output of the adder 40 is connected to the recode module 42 which modifies the received signal in accordance with a predetermined transform coding, this coding dictates the grey-level histogram distribution of the output video signal.

The operation of the ripple store 30 is essentially the same as has been described with reference to Fig. 4 and accordingly will not be repeated. Although the five most significant taps have been used in the illustrated embodiment, other groups of taps can be used provided that they correspond with each other and thus define under normal circumstances a 5 X 3 pixel rectangular local area.

Compared with known real-time image modification systems, such as that disclosed in U.S. Patent Specification No. 3,983,320, the local area brightness modification or redistribution is calculated on a point by point basis thereby avoiding possible boundary problems in matching small areas separately calculated.

The local area that influences the redistribution may be varied simply (dynamically if necessary) with no significant increase in complexity. Further the local area that influences the redistribution may be weighted in a nonuniform manner (dynamic if necessary) with no significant increase in complexity. Areas containing a large number of identical input grey levels for instance where there is uniform intensity except for a few isolated points, may be suppressed to near black or near white thus emphasising the differences between the isolated points and thereby provide good point contrast. The hardware required for analogue or hybrid systems is in general less complex and less bulky than the equivalent digital system, and may readily achieve high data rates. Finally it may operate in a purely analogue form by utilising non-sampled delay lines or two dimensional arrays of primary detector elements. For example in the system shown in Fig. 3 quartz delay lines may be used at the locations of the delay circuits 54 and 58 and lumped (L.R.C.) delay lines may be used at the locations of the delay circuits 52, 56 and 60. Alternatively a 2-dimensional array of 1 5 radiation detectors may be directly coupled into the inputs of the comparators in the block 34, for example a detector may be coupled to each of the locations referred a to o in Fig. 3. The individual comparators now taking an analogue form. The detector array could now traverse the field of view in continuous motion while presenting the spatial infor mation in a parallel format, to the compare tors, there being no need for temporal delay networks to produce the required 2-dimensional information from a serial video input.

Such a mosaic array of detectors would scan the scene, say from left to right, in the normal way but would only be indexed downward by one pixel, rather than 3, thus producing the macro grouping for the subsequent line scan.

Such a scan which steps only a single pixel in the vertical direction not only allows for additional integration of the incoming information to take place in the viewer's eye but averages out the differences that exist between the detectors and their amplifiers in the array, i.e.

fixed pattern noise tends to be averaged out.

Claims (20)

1. A method of modifying the contrast of an image in real time, comprising receiving video information in the form of scanned pixels, comparing in sequence the information from each pixel with that of its surrounding macro group and modifying the information relating to each pixel on a statistical basis by the information derived from its surrounding macro group.

2. A method as claimed in Claim 1, wherein the macro group of pixels comprises pixels in a local area which is small relative to the area of a frame containing the image, the pixel information to be modified relates to a pixel at a selected position in the local area and wherein after each comparison the local area is indexed by at least one pixel in at least one direction.

3. A method as claimed in Claim 1 or 2, wherein the results of the comparison are a plurality of decisions indicating whether the information from each pixel in the macro group is greater than, equal to or less than the information of the pixel to be modified.

4. A method as claimed in Claim 1, 2 or 3, wherein the results of the comparison are weighted.

5. A method as claimed in any one of Claims 1 to 4, wherein the results of the comparison are summed and the sum is adjusted in value in accordance with a selected transform code.

6. A method as claimed in Claim 5, wherein the information of each pixel in a macro group is processed as a digital signal and the adjusted value of the sum is expressed in a greater number of bits than occurs in the digital value before adjustment.

7. A method as claimed in anu one of Claim 1 to 5, wherein the information of each pixel in a macro group is processed as a digital signal.

8. A method as claimed in Claim 7 when appended to Claim 5, wherein the adjusted value of the sum is expressed in a lesser number of bits than occurs in the digital information of each pixel in the macro group being considered.

9. A method as claimed in any one of Claims 1 to 5, wherein the information of each pixel in a macro group is processed as an analogue signal.

10. A method as claimed in any one of Claims 1 to 9, wherein the macro group is moved sequentially within the area in a frame of the image.

11. A method as claimed in Claim 2, wherein, when the macro group comprises information of pixels in unrelated parts of the frame containing the image, the pixel information relating to the pixel at the selected position in the local area is processed normally without being modified on a statistical basis of the information derived from the macro group.

1 2. A method of modifying an image in real time, substantially as hereinbefore described with reference to the accompanying drawings.

1 3. An apparatus for modifying the contrast of an image in real time, comprising means for scanning a frame containing and providing video information in the form of scanned pixels, a comparator for comparing in sequence the information from each pixel with that of its surrounding macro group, and means for modifying the information relating to each pixel on a statistical basis by the information derived from its surrounding macro group.

1 4. An apparatus as claimed in Claim 13, further comprising a ripple store for storing the information relating the macro group of pixels which are contained in a local area of an image frame, the local area being indexed by at least one pixel in at least one direction, in response to the receipt of a pixel value on an input of the ripple store.

1 5. An apparatus as claimed in Claim 14, wherein the ripple store has an output for a selected pixel position and other outputs for pixels at other positions of the local area and the comparator comprises a plurality of stages, each stage having one input connected to the selected pixel position output of the ripple store and another input coupled to a respective one of the other outputs.

1 6. An apparatus as claimed Claim 13, 1 4 or 15, further comprising means for weighting the output of the comparator.

1 7. An apparatus as claimed in any one of Claims 1 3 to 16, further comprising summing means for summing the outputs of the comparator and transform coding means connected to the surnming means.

18. An apparatus as claimed in Claim 13, 1 4 or 15, further comprising means for summing and weighting the outputs of the comparator and for recoding the summation in accordance with a predetermined linear or non-linear law.

1 9. An apparatus as claimed in any one of Claims 1 3 to 17, further comprising means for inhibiting the modification of the information relating to each pixel when the macro group comprises information from unrelated parts of a frame containing the image.

20. An apparatus for modifying the contrast of an image in real time, substantially as hereinbefore described with reference to Fig.

2, 3 or 5 of the accompanying drawings.