WO2012038705A1

WO2012038705A1 - Method and system for processing synthetic aperture radar images

Info

Publication number: WO2012038705A1
Application number: PCT/GB2011/001387
Authority: WO
Inventors: Christopher Paul Moate; Darren James Coe; Timothy Michael Lamont-Smith; Robert David Hill; Mohammed Jahangir; James Denton
Original assignee: Qinetiq Limited
Priority date: 2010-09-24
Filing date: 2011-09-26
Publication date: 2012-03-29
Also published as: GB201016230D0

Abstract

A method for processing images having both phase and amplitude information is disclosed, the method allowing convenient highlighting of changes present between a first and a second complex image, the changes showing up as coloured regions within an otherwise monochrome image. The first and second images are each associated with a colour, the colours being complementary. A composite colour image is created, each pixel comprising contributions from corresponding pixels from each of the first and second images with their associated colours. Pixels of equal intensity will therefore produce a shade of grey in the composite, whereas differences in intensity will show as one of the complementary colours. The invention extends to a system, and to a computer program for implementing the method. The invention has utility in synthetic aperture radar image processing of coherent change detection images.

Description

Method and System for Processing Synthetic Aperture Radar Images

This invention relates to a method and system for processing synthetic aperture radar (SAR) images, particularly a sequence of images of a region, to identify changes in the region occurring over the time span of the sequence. The invention relies on phase information available in coherent images.

Synthetic aperture radar systems are known, and have been used for many years to generate high resolution images of the ground. The synthetic aperture is produced by recording a series of radar returns taken from a moving platform over a period of time, applying a phase correction factor to each one that corrects for the platform movement, and integrating them over the time period. Images may be generated from the processed returns that have an equivalent resolution dependent upon the aperture size, i.e. the distance moved by the platform over the time period, or by the aperture beamwidth of the radar's antenna. Wiley, CA, "Synthetic Aperture Radars", IEEE Trns. Aerospace & Electronic Systems, Vol. AES-21 , No. 3, May 1985, pp440-443 describes SAR systems in more detail.

SAR systems usually operate in one of two modes of operation. These are called strip-map mode, and spotlight mode. In strip-map mode the antenna pointing direction is fixed relative to the flight path providing an area of illumination that sweeps along a strip of terrain. In spotlight mode the sensor steers the antenna to continuously illuminate an area of interest allowing a smaller area to be imaged at higher resolution.

The latter approach also provides, for a single pass of the moving platform, a set of images of the same region, albeit that each are from a different aspect, whereas a plurality of different passes are needed to produce multiple images of the same region using the former, strip-map, approach. Of course, images may be recorded from the same region during different passes using the spotlight method also. With each approach therefore a set of images of a region of interest may be built up. By comparing two or more images of the same region, taken at different times and looking for differences between them, changes in the scene, such as may be caused by moving targets can be identified. Various ways of doing this are known.

The most simple is known as incoherent change detection (ICD), and uses changes in image intensity for detecting changes in the interval between the collection of two SAR images of the same region. As the technique operates on intensity information then complex image data (i.e. phase information) is not required.

Another technique is known as coherent change detection (CCD). This technique exploits changes in both the amplitude and phase content between image pairs (i.e. images of the same scene taken at different times). It relies on the image generation process being coherent, wherein each pixel of the image contains an amplitude and a phase, or alternatively a real and imaginary value. As the phase is measured in fractions of a wavelength of the radar signal, which may typically be around 3 cm, CCD has the potential to detect very subtle scene changes that may remain undetected using incoherent techniques.

SAR systems are typically employed on manned, or unmanned aircraft, which may be tasked with recording data from a large area. In consequence, the SAR images produced may be very large, and impractical to analyse manually. The known CCD and ICD techniques simplify this process somewhat by highlighting the changes that have occurred in a time interval. It can still be difficult to analyse the large images however.

It is an object of the present invention to provide an alternative means for highlighting changes in a sequence of coherent SAR images of a region. According to a first aspect of the present invention there is provided a method for indication of changes within a scene by generating a composite coloured image from a plurality of SAR images of the scene, the method comprising the steps of:

a) obtaining a first coherent change detect (CCD) image from a first pair of SAR images;

b) associating a first colour with the first CCD image;

c) obtaining a second CCD image from a second pair of SAR images; d) associating a second colour with the second CCD image, wherein the second colour is complementary to that of the first;

e) creating a composite colour image, each pixel of which comprises a contribution from the corresponding pixels from the first and second CCD images each modified to have their associated colour with a shade depending on the intensity of the corresponding pixel.

The invention provides a means for highlighting changes that have occurred in one set of images as compared to another, whilst also allowing for convenient discrimination of regions of decoherence, i.e. those regions being incoherent in both images. This helps the operator to rapidly identify regions of potential interest in a scene and to distinguish them from natural changes.

A typical CCD image will be brightest (i.e. near white in a black and white image) where the return signal is most coherent. The intensity of any pixel is therefore an indication of the coherence of the image at that point with darker pixels indicative of lower coherence. It will be clear therefore that, when a pair of CCD images processed as described herein are used to create the ICD image, areas in the CCD images that are of similar intensity will tend to appear as shades of grey in the ICD image. This is because the

complementary colours, of similar shade, that each will have will combine to produce the grey colour. Where there are differences, such as where a pixel or region on one of the CCD images is darker than on the corresponding pixel or region on the other CCD image, the combination of complementary colours will result in a residual colour cast of one of the complementary colours used. These will typically be areas where movement has occurred. Where large areas are decoherent in both the first and second CCD images, such as long stemmed crops etc, then the resultant ICD image will tend to contain a random mix of pixels of the two complementary colours. This shows up generally as a greyish region, which may have a tint of one of the colours, but which appears different from the colours resulting from movement occurring in just one of the CCD images, such as vehicle movement. Note that, in this specification, complementary colours are defined as a pair of colours (not themselves grey, white or black) that, when equal in intensity, combine to form a grey colour (including white and black). When the intensities are at maximum, this grey will become white, and when at a minimum, will be black. Advantageously one of the complementary colours will be a primary colour, i.e. red, green or blue. This provides for a simpler calculation of the complementary colour, and also ensures that the two colours (i.e. the primary and its complement) will be visually distinct from the grey (including white) colour resulting from their combination. The invention therefore provides a convenient means for highlighting movement in a way that is simple for an observer to see.

The invention requires three or more SAR images of a region to be recorded. Preferably the SAR images used to create each CCD image are evenly spaced apart in time. This will tend to give better results, as each CCD image will then generally have a similar level of coherence. It has been observed that the level of coherence between a pair of SAR images of general countryside, such as fields, plains etc will decrease over time, caused by wind, rain, thermal cycling etc, and so by collecting image separate by similar time spans as described above the levels of decoherence will tend to be similar. The temporal separation in the SAR image may typically be, for example 30 minutes, 1 hour, 2 hours, 4 hours or 8 hours. The CCD images used should be aligned so that a given pixel in each represents the same point on the ground. Various methods are known for aligning CCD images to the required degree, and so this aspect shall not be described further.

The steps of associating the first and second colours with the first and second CCD images may comprise providing intermediate images each coloured with the respective colours. Alternatively, it may comprise multiplying each pixel of an image by the associated colour, without the generation of an intermediate coloured image. Selection of complementary colours may be done in any convenient manner. For example, the well known Hue, Saturation, Value (HSV) representation provides a convenient means whereby the hue is represented as an angle. The selection of any pair of angles 180° apart therefore guarantee that complementary colours are used.

An alternative approach is to use the well known RGB representation. An image represented in RGB format has, for each pixel, three values (i.e. an RGB triplet). The R value represents the intensity of the red colour of the pixel, and the G and B values represent the intensity of the pixel's green and blue colours respectively. Typically, each value is an eight bit number, although other bit lengths may be used.

Using the RGB representation, the first CCD image may be associated with a colour (e.g. red, although any colour may be used.), and the second CCD image may be associated with the complementary colour (e.g. in this case an equal mix of green and blue), such that the sum of the sum of equal intensities of the two colours will produce a neutral colour, i.e. grey or white (according to the intensity). In generating the composite colour image a scaling of pixel values may be needed. The CCD images may, for example, have pixels that lie in the range 0 (representing black) to 1 (representing white). They may therefore require scaling to provide the full intensity range used in other formats. For example, CCD pixel values lying between 0 and 1 may be multiplied by 255 to bring them into the range used in eight bit RGB representations.

If there is a disparity in the mean intensity of the CCD images being used as first and second images then an image scaling factor may be used to reduce this disparity. This may involve calculating a mean image intensity by summing pixel values and dividing by the number of pixels.

Conveniently, the first or second image may be assigned a primary colour, as then the generation of the complement is simpler. The intensity value of each pixel in the first or second image may be assigned to the chosen colour in the corresponding pixel of the composite image. Image data from both the first and second images is therefore included in the composite image. Where corresponding pixels in the images are the same intensity then the grey (or white) colour will result as described above.

Therefore, as described herein, the contribution to the composite image from the first and second images may be made by summing the corresponding pixel values (these having been associated with their respective

complementary colours), and scaling these if necessary, to produce a composite pixel value. In the HSV case this sum would be a vector sum. The contribution may also be made by using pixel values from the first and second images to generate a pixel, wherein corresponding pixel values from the first and second images each maintain their separate values in forming a pixel of the composite image, such as in the RGB embodiment.

According to a second aspect of the present invention there is provide an image processing system for indication of changes having taken place within a scene, the system comprising a processor, memory, and a display, the system being arranged to generate a composite coloured image from a plurality of complex images stored within memory, each containing amplitude and phase information, and to produce the composite image for storage within memory, and to display the composite image on the display, characterised in that the system has processing means arranged to:

a) associate a first colour with the first complex image;

b) associate a second colour with the second complex image, wherein the second colour is complementary to that of the first;

c) create a composite colour image, each pixel of which comprises a contribution from the corresponding pixels from the first and second complex images each modified to have their associated colour with a shade depending on the intensity of the corresponding pixel.

According to a third aspect of the present invention there is provided a computer program comprising instructions arranged to run on a computer system, the system comprising at least a processor and memory, the steps of the computer program being arranged to process data corresponding to first and second complex images of a region to generate a composite colour image, characterised by

a) associating a first colour with the first complex image;

b) associating a second colour with the second complex image, wherein the second colour is complementary to that of the first;

c) creating a composite colour image, each pixel of which comprises a contribution from the corresponding pixels from the first and second complex images each modified to have their associated colour with a shade depending on the intensity of the corresponding pixel.

The complex images may be CCD images. One of the first and second colours may advantageously be a primary colour, as this is more easily used with some image representations, such as the RGB representation described above.

The invention may also be seen, in another aspect, as a method of processing images using the steps as described in claim 1. The method of processing images comprises the generation of a composite coloured image using the steps as described in claim 1. The invention may also be seen as a method for highlighting changes within a scene, the scene having been imaged with a synthetic aperture radar on at least three occasions, the method comprising the steps as outlined in claim 1. The invention will now be described, by way of example only, with reference to the following Figures, of which:

Figure 1 shows at a top level the process of taking SAR images, producing the CCD images, and then producing a final composite image;

Figure 2 shows a photograph following the processing according to the invention used to produce an ICD image, except for the photograph being in monochrome; and Figure 3 shows a block diagram of a computer system on which the current invention may be implemented.

Figure 1 shows at a top level how the SAR images are used to produce the final composite image. A sequence of SAR images 10, 11 , 12 are provided as an input. At least one of these may come from a radar system, or alternatively they may come from a storage means on which a sequence of previously captured SAR images are stored. The sequence of images are recorded at approximately similar intervals, so that similar naturally occurring

decoherence, such as vegetation movement in wind, is present between each contiguous pair.

A first pair of images, here 10 and 11 , are used to produce a first CCD image 13 in conventional manner. This CCD image is termed a reference image. A second pair, these in this case being images 11 and 12, are used to produce a second CCD image, termed a mission image 14. Note that the reference and mission images need not share one of the original SAR images as has been shown in this example; they may be derived from four or more separate images.

Once the reference and mission images have been produced, they are combined to form the composite image 15. It will be appreciated that the composite image 15 is an ICD image, as the image phase information in the CCD images is not used in its formation.

An embodiment of the present invention implements the generation of the ICD (i.e. the composite) image 15 by summing the first and second CCD images after first colouring them with complementary colours. This may be done as follows. A pair of complementary colours are selected, by choosing, from the HSV colour model, values for H that differ by 180°. Red and cyan are an example of such colours. S and V are both set to 1. The resulting colours are then converted to the RGB colour scheme in conventional fashion (see e.g. Foley and Van Dam, Fundamentals of Interactive Computer Graphics, Addison-Wesley Publishing Co., 1982), to give R_ref, G_ref and B_ref for use with the reference CCD image, and R_mis, G_mi_S and B_mi_S for use with the mission CCD image. For example, using the above approach, with red associated with the reference image, the values following conversion to RGB would be, in an 8 bit system, R_ref=255, G_ref=0 and B_ref=0; and with cyan associated with the mission image the values would be Rr_ef=0, G_ref=255 and B_ref=255. The sum of these two colours will be white, i.e. R_ref=255, G_ref=255 and B_ref=255. The reference and mission images are a greyscale, and so this greyscale information is also included in the final image, which is produced as follows.

A(x, y)R_A _ B(x,y)R_B

255 255

A(x, y)G_A , B(x, y)G_B

c^{, y)} 255 255

c_B(_x,_y) = ^A^ ₊ ^B^

* ^J 255 255

where CR(x,y), C_G(x,y), C_B(x,y) represent the respective red, green and blue elements of the pixels of the composite image; A(x,y) and B(x,y) represent the (greyscale, in range 0 to 1 ) pixels in the first (reference) image and second (mission) image respectively;

R_A, RB represent the red element of the colours associated with the respective first and second images;

G_A, GB represent the green element of the colours associated with the respective first and second images;

B_A, B_B represent the blue element of the colours associated with the respective first and second images. e.g. if red and cyan are chosen as the complementary colours, [RA, GA, BA] will be [255, 0, 0], and [R_B, GB, B_b] will be [0, 255, 255] as described above.

A second embodiment of the present invention implements the generation of the ICD image 15 using the RGB colour representation throughout. Two CCD images are provided as input. A primary colour, e.g. red, is chosen, and its complement calculated. Thus cyan is the complement in this case, this being an equal mix of green and blue. An ICD image is then created, where each pixel comprises an R, G, B triplet. The R value of each pixel is provided by the value of the equivalent (greyscale) pixel in the first CCD image, and the G and B values of each pixel are provided by the value of the equivalent pixel in the second CCD image. Note that, in this case, the G and B values will always be equal. Scaling of the pixel values of the CCD images may be needed, as they may not initially be in the correct numerical range for use in an RGB implementation. For example, the CCD images may have pixel intensity values in the range 0 (for black) to 1 (for white), with intermediate values representing shades of grey. These values would therefore be multiplied by 255 to get them to the correct range for an 8 bit RGB image. Other scaling factors would be used as appropriate for conversion to or from other representations.

The ICD image thus created by the RGB processing will be grey or white in those regions where the first and second images are identical, and will be a shade of red or cyan in those regions where there is a difference. A cyan colouring is indicative of a change occurring in the first image, and a red colouring is indicative of a change occurring in the second image. For any of the embodiments described herein one or both CCD images may be scaled by a constant factor to give equal mean intensity values.

Figure 2 shows an example of the method of the invention having been applied to a pair of CCD images. The Figure is a monochrome copy (for the sake of reproduction herein) of an ICD image where red and cyan were chosen as complementary colours. Much of the image, such as region 22 is made up of shades of grey, with a red or cyan tinged mottling. Track 21 is red which indicates that the change occurred in the mission CCD image. Track 23 is cyan indicating that it occurred in the reference CCD image.

A complex mass of tracks, some red 24 and some cyan 25 are visible, these having likely been made by cattle. Region 26 is mainly a dark red mixed with dark grey. This suggests that it is a region liable to generate decoherent reflections, such as crops. The redness of region 26, as opposed to a plain greyness, may be caused by increased movement of the crops (due perhaps to wind conditions) when the SAR images used in the creation of the CCD image associated with that colour, compared with when the later SAR images were taken.

The invention may be implemented in a computer system comprising a processor and memory. The computer system may have a display for displaying images, or data pertaining to images or other data. The computer system may have an image processing module, an image display module, and image storage means, comprising RAM or permanent storage, such as a hard disk drive (HDD), or a solid state drive. Figure 3 shows an embodiment of a computer system on which the present invention may be implemented. The computer system 20 comprises an image processing module 21 in connection with memory (RAM and HDD) 22 and also in connection with display driver 23. The image processing module comprises a processor and RAM, and is arranged to take complex image data resident in memory 22 and process it according to the steps as described in claim 1 to produce a composite image. It then, at the command of an operator, stores the composite image back to memory 22 and/or passes it to display driver 24, which in turn generates and provides signals to display 25 to present a visual image to the operator. The invention may be arranged to be implemented as a set of steps operative on the computer system, the steps arranged to carry out the method of the invention, with the steps being coded in the form of a computer program. The invention therefore extends to a computer program, which may reside on a carrier, designed to implement the method of the invention when loaded onto a suitable computer system.

Claims

1. A method for indication of changes within a scene by generating a composite coloured image from a plurality of SAR images of the scene, the method comprising the steps of:

b) associating a first colour with the first CCD image;

2. A method as claimed in claim 1 wherein one of the first and second colours is a primary colour.

3. A method as claimed in any of the above claims wherein an RGB (Red, Green, Blue) colour representation is used, with pixel values from the first image providing shades of the first colour, and pixel values from the second image providing shades of the complementary colour.

4. A method as claimed in claim 1 or claim 2 wherein a hue, saturation, value (HSV) colour representation is used, and wherein the first colour and the second colour are each represented as a hue, the hues being separated by 180°

5. A method as claimed in any of the above claims wherein a scaling factor is used to scale at least one of the first and second images to reduce any disparity in mean intensity between the first and second images.

6. A method as claimed in any of the above claims wherein pixel values in the first and second images are scaled to match a pixel intensity range employed in an image format used in the composite colour image.

7. An image processing system for indication of changes having taken place within a scene, the system comprising a processor, memory, and a display, the system being arranged to generate a composite coloured image from a plurality of complex images stored within memory, each containing amplitude and phase information, and to produce the composite image for storage within memory, and to display the composite image on the display, characterised in that the system has processing means arranged to:

a) associate a first colour with the first complex image;

8. A system as claimed in claim 7 wherein the first and second complex images are coherent change detect (CCD) images.

9. A system as claimed in claim 7 or claim 8 wherein one of the first and second colours is a primary colour.

10. A system as claimed in any of claims 7 to 9 wherein an RGB (Red, Green, Blue) colour representation is used, with pixel values from the first image providing shades of the first colour, and pixel values from the second image providing shades of the complementary colour.

11. A system as claimed in any of claims 7 to 9 wherein a hue, saturation, value (HSV) colour representation is used, and wherein the first colour and the second colour are each represented as a hue, the hues being separated by 180°

12. A computer program comprising instructions arranged to run on a computer system, the system comprising at least a processor and memory, the steps of the computer program being arranged to process data

corresponding to first and second complex images of a region to generate a composite colour image, characterised by

a) associating a first colour with the first complex image;

13. A program as claimed in claim 12 wherein the first and second complex images are coherent change detect (CCD) images.

14. A program as claimed in claim 12 or claim 13 wherein one of the first and second colours is a primary colour.

15. A method for highlighting differences in a plurality of SAR images, the method comprising the steps of:

b) associating a first colour with the first CCD image;