WO2006038073A2

WO2006038073A2 - Image processing

Info

Publication number: WO2006038073A2
Application number: PCT/IB2005/002819
Authority: WO
Inventors: Gavin Hough
Original assignee: Gavin Hough
Priority date: 2004-10-04
Filing date: 2005-10-04
Publication date: 2006-04-13
Also published as: WO2006038073A8; WO2006038073A3

Abstract

A method of analysing a series of images containing static image portions and possibly dynamic image portions for changes is provided. The method includes defining at least one image region in the series of images which contains a static image portion and identifying transient occurrences in the image region or regions. At least two images that do not contain transient occurrences in the defined image region or regions are selected, and the defined image region or regions in each of the selected images are compared with one another, whereafter image changes in the defined image regions of each of the selected images are detected.

Description

IMAGE PROCESSING

THIS INVENTION relates to image processing. In particular, the invention relates to a method of analysing a series of images and to an image processor.

The inventor is aware of image processors used to detect motion in an image. However, the disadvantage of such a system is that gradual changes to a static image portion can go undetected.

According to one aspect of the invention, there is provided a method of analysing a series of images containing static image portions and possibly dynamic image portions for changes, the method including defining at least one image region in the series of images, the image region or regions containing a static image portion; identifying transient occurrences in the image region or regions; selecting from the series of images at least two images that do not contain transient occurrences in the defined image region or regions; comparing the defined image region or regions in each of the selected images with one another; and detecting image changes in the defined image regions of each of the selected images.

Transient occurrences in this specification should be interpreted to mean image changes in the defined image regions that can be filtered out over time, i.e. the image stabilize to its former status over a period of time.

It is to be appreciated that the image region containing a static image portion may include transient occurrences in some images in the series of images, but that the frequency of transient occurrences in the static image portions is thus lower than in dynamic image portions. The image region containing a static image portion may thus experience occasional image changes.

Identifying transient occurrences may be effected by detecting the frequency of change in the image region containing a static image portion, where a high frequency of change, i.e. a small number of changes, indicates a transient occurrence.

In a preferred embodiment of the invention, at least two image regions containing a static image portion are defined.

The detection of image changes in the defined image regions of each of the selected images may include performing specific tests which may include one or more of detecting a shift in the image, detecting a persistent or gradual alteration in the spatial structure (i.e. location, orientation and magnitude of spatial intensity gradients) in the defined image region, detecting a change in spatial image intensity distribution, and the like.

The method may include the prior step of receiving a series of images from an image source. The image source may be a camera or may be a stream of data representing a series of images, such as an MPEG 2 data stream, or the like. The data may be in the form of video images, or may be in the form of compressed video images.

The method may include compressing the series of images.

The method may include, prior to identifying transient occurrences, detecting a baseline structure in the defined image region or regions containing a static image portion. Detecting a baseline structure can be performed by means of an adaptive learning algorithm which may be included in an image processor. Detecting image changes may include detecting a change in the baseline structure in the defined image region or regions containing a static image portion. For example, if a loss of structure in the defined image regions is detected from one selected image to the next selected image (i.e. the image is blurred), it may be an indication that a camera has been slightly defocused. Detecting the baseline structure may include any one of the techniques known in the art, e.g. mapping the brightness distribution in the defined image region. Comparing the defined image region or regions may include tracking image registration between the at least two selected images. The tracking of image registration between the at least two selected images may include determining the least square error between the at least two selected images or determining the mutual information between the at least two selected images.

Tracking image registration between two selected image sequences may provide an indication that the image source, i.e. the video camera, has been tampered with, or experienced a non-electrical failure, such as being shifted from the target by maintenance personnel, or the like. Tracking image registration thus provides an indication of the reliability of the information in the image.

Comparing the defined image region or regions may include determining the differences in the image intensity histogram between the at least two selected images in the defined region or regions. In one embodiment of the invention, where the images are in colour, the image intensity distribution is determined in the three primary light colour channels or red, green and blue. In a specific example where the image is defined as an array of pixels, the image intensity distribution can be calculated at a pixel level. By comparing the image intensity distribution between the two selected images, certain image changes may be detected. Thus when the image intensity distribution exceeds a predefined upper or lower threshold, it may indicate that the source of the images is being tampered with, for example the image may be saturated or the image intensity reduced to "dark current" levels. Saturation or loss of intensity may be an indication that a camera is being blinded or that the camera is being obscured.

By way of example, if yellow paint is sprayed over the lens, the intensity histograms for the defined image region (or regions) will change from their characteristic reference distributions to very narrow intensity histograms, uniformly high in both the red and green channels, while remaining low in the blue channel.

A plurality of image regions containing a static image portion may be defined. The method may include identifying any one of several transient occurrences from a selection of images by comparing defined image regions and detecting image changes in the plurality of defined image regions. The method may further include comparing the results of the detected image changes in the defined image regions thereby to obtain an indication of the reliability of the results of the detected image changes. Obtaining an indication of the reliability of the detected image changes is referred to as "obtaining consensus" between the results of the changes in the defined image regions. Advantageously, obtaining consensus between the results of the detected image changes provides a more reliable and robust detection of image changes and may reduce false detections.

In operation, the defined regions are used to establish an image baseline for each region by using an image sequence after filtering out transient events. In each region, adaptive learning techniques may be employed to maintain a set of baseline values which may be adapted on an ongoing basis in order to represent position, structure and intensity distributions (i.e. in terms of spatial distributions and histograms).

Detecting image changes may then use rules based decision criteria to classify deviations from these baseline values which are detected in more than one of the defined regions.

According to another aspect of the invention, there is provided a method of detecting tampering with an image or with a device recording a series of images, the method including analysing a series of images containing static image portions and in response generating an alarm if a predefined image change has been detected in the static image portions of at least some of the images.

The method may include identifying images in the series of images that precede or follow on from the tampered image or images, and preferably recording said images in the series of images. For example, if a camera from which images originate has been tampered with, the instant of tampering can be determined and the images preceding the tampering may be recorded and/or analysed to obtain information relating to the source of the tampering.

The series of images may be analysed according to the method as hereinbefore described. According to a further aspect of the invention, there is provided an image processor, the image processor including region definition means operable to define an image region containing a static image portion in a series of images; a transient detector for detecting, within the defined image region, transient occurrences in the series of images; an image comparator operable to compare the defined image regions of at least two images which do not contain transient occurrences in the defined image regions and then to produce an output corresponding to the differences in the defined image regions; and a trigger operable to trigger an event in response to the output from the image comparator when the output satisfies predefined criteria.

The image processor may include a micro-processor and a set of instructions, the microprocessor being operable to execute the set of instructions, thereby providing the region definition means, the transient detector, the image comparator and the trigger.

The images may originate from a video camera and the image processor may include an input port for receiving the series of images. It is to be appreciated that the video camera may be an analogue camera or a digital camera. However if the camera is an analogue camera, the method of the invention may include digitising the analogue image. The image processor may then include an image digitiser for digitising the analogue image.

The image processor may be operable to define at least two image regions containing a static image portion in a series of images.

When the image processor includes a micro-processor and a set of instructions, the image comparator may include any one of a baseline structure algorithm, an image registration algorithm and an image intensity distribution algorithm. The predefined criteria applied by the trigger to the output from the image comparator may be adjusted automatically in accordance with trends established by the images not containing transient occurrences.

The image processor may include an adaptive learning algorithm to detect a baseline structure in the defined image region or regions containing a static image portion.

The region definition means may be configured to define a region of interest (ROI) within an image. In operation, the processor may then discard all image information outside the ROI. For example, the region definition means may be configured to discard all image information falling outside a defined two dimensional array of image pixels. Advantageously, this will reduce the image data that needs to be compared by the processor.

The image processor may be programmed to perform the method of analysing a series of images, as hereinbefore described.

The invention extends to a camera arrangement which includes more that one image processor.

The invention further extends to a security installation which includes at least one image processor connected to at least one video camera and a display screen also connected to the video camera. The security installation may include a recorder connected to the video camera for recording a series of images. The image processor may be as hereinbefore described.

The invention will now be described, by way of example only, with reference to the following illustrations, in which: Figure 1 shows an image of vehicles moving on a highway, the image being one of a series of images being analysed by a method in accordance with the invention;

Figure 2 shows a display on a screen of an image processor in accordance with the invention;

Figure 3 shows another display on the screen of the image processor of Figure 2; Figure 4 shows yet another display on the screen of the image processor of Figure 2; and

Figure 5 shows a flow diagram of a micro-processor executing instructions in an image processor, in accordance with the invention.

In Figure 1 reference numeral 10 refers to an image in a series of images. In the image 10, two image regions 12, 14 have been defined. The image regions 12, 14 contain static image portions of the image 10. The image regions 12, 14 are shown as image gates which include two sets of pixel arrays in the image. The image in this example is generated by a video camera (not shown). In the image, vehicles, generally referred to by reference numeral 16 are shown. It is to be appreciated that in other images in the series of images from which image 10 was selected, the image regions 12, 14 may include portions of the vehicles 16. However, in the method of analysing a series of images, once the regions 12, 14 have been defined as images containing static image portions of the image 10, a transient occurrence, caused by the vehicle 16.2 moving past the image region 14 and then past the image region 12 can be detected. The method detects the transient occurrence by comparing the two regions 12, 14 after the image content in the image regions 12, 14 has stabilised. It is to be appreciated that the frequency of change in the image regions 12, 14 is high when the vehicle 16.2 passes through the regions 12, 14, while the frequency of change in the image regions 12, 14 is low when one of the vehicles 16 does not pass through the regions. Thus, the method includes selecting images from the series of images which do not contain transient occurrences in the defined regions 12, 14.

in Figure 2, two images are shown and indicated by reference numerals 20 and 22. Four image regions 24, 26, 28, 30 have been identified. The structure in each of the image regions 24, 26, 28, 30 can be determined by determining the contrast ratio in each region 24, 26, 28 and 30. In this example, region 24 is defined over an electric lamp in which the contrast ratio is high, region 26 is defined over a dark counter, in which the contrast ration is low, and regions 28 and 30 are defined in regions which contain sharp lines in which the contrast ratio is high.

As can be seen in image 22, the contrast ratio in each region 24, 28 and 30 will be high, but in image 20, the contrast ratio in each region 24, 28 and 30 will be lower because of the blurring of the image 20. Thus using the method of the invention, by comparing the regions 24, 26, 28, 30 in the two images 22 and 20, the loss of structure in image 20 can be detected, which indicates image blinding. This blinding can be caused by camera defocusing or an obstruction on the lens of the camera. A measure of the detected changes in the regions 24, 26, 28, 30 can be obtained and compared with each other. When the measure of changes in the regions 24, 26, 28, 30 corresponds, (i.e. consensus is obtained), the changes can be an indication that the whole image 20 has deteriorated, which may be caused by tampering with the camera. Thus according to the invention, a method of determining tampering with an image may include analysing a series of images and in response generating an alarm if a predefined image change such as seen in Figure 2 has been detected.

In Figure 3, two images 40, 42 are shown. In the image 40, four regions 44, 46, 48, 50 are shown. Two frames 44.1 , 44.2, 46.1 , 46.2, 48.1 , 48.2 and 50.1 , 50.2, define each of the regions 44, 46, 48, 50. In the method illustrated here, the outer frames 44.2, 46.2, 48.2, 50.2 are defined as the image regions containing static image, portions. The content in the defined images is determined by determining the structure in the inner frames 44.1 , 46.1 , 48.1 and 50.1 in a first image 40 in a series of images. If the content in the inner frames 44.1 , 46.1 , 48.1 and 50.1 is different in a second image 42 in the series of images, the inner frames 44.1 , 46.1 , 48.1 and 50.1 are adjusted to all positions within the outer frames 44.2, 46.2, 48.2, 50.2, to achieve a correlation between the content in the inner frames 44.1 , 46.1 , 48.1 and 50.1 of the first image 40 and the inner frames 44.1 , 46.1 , 48.1 and 50.1 of the second image 42. If a correlation is detected between the inner frames 44.1 , 46.1 , 48.1 and 50.1 of the first image 40 and the second image 42 by moving the inner frames 44.1 , 46.1 , 48.1 and 50.1 in a specific direction within the outer frames 44.2, 46.2, 48.2, 50.2 it is an indication that the first and the second images 40 and 42 in the set of images are out of register with each other. If the direction in which the inner frames 44.1 , 46.1 , 48.1 and 50.1 have been moved to obtain correlation (i.e. consensus exists) can be determined, then it is an indication that the camera from which the image originated may have been moved, which may be an indication that the camera has been tampered with.

In Figure 4, an image 60 is shown. Four image regions 62, 64, 66, 68 have been defined in the image 60. Colour histograms 72, 74, 76, 78, corresponding to the regions 62, 64, 66, 68 are shown. The histograms 72, 74, 76, 78 are representations of pixel intensities of certain colour frequencies. As can be seen, where a large colour variation exist in region 62, the corresponding histogram, 72 shows a wide frequency distribution, while a smaller colour variation in region 64, shows a narrow frequency distribution in histogram 74. The histograms provide a "fingerprint" of the region contents.

By comparing image intensity distributions 72 to 78 in two selected images (of which only the image 60 is shown) and by detecting changes in the image intensity distributions 72 to 78, tampering with an image can be detected. For example, when a light source saturates the image 60, the image intensity distribution in all the regions 62, 64, 66, 68 will change (i.e. consensus exists). Also, if the image is obscured, the image intensity distribution in all the regions 62, 64, 66, 68 will change (i.e. consensus exists). This may provide an indication of tampering with the image by saturating the camera, or by obscuring the camera.

In Figure 5, reference numeral 100 indicates execution flow of a micro¬ processor in an image processor in accordance with the invention executing a set of instructions.

Execution starts at 102, where after image intensity histograms of defined image portions of a present image are compared with the same image portions in a previous image at 104. At 106 one of the regions is tested to determine if the natural light intensity dropped. If the natural light intensity dropped it is concluded at 108 that it is night time. If it is not established at 106 that the natural light intensity dropped, an alarm is raised at 110, indicating that the camera has been blinded or obscured.

Then at 1 12, the structure in the defined image portions of a present image is compared with the same image portions in a previous image, if a structure loss occurred, an alarm is raised at 114, indicating that the camera image has been blurred.

At 116, the registration in the defined image portions of a present image is compared with the same image portions in a previous image. If the present image is out of register with the previous image, but the registration loss is temporary, no alarm is raised at 118. If attempts to re-register the image portions at 120 are successful, no alarm is raised at 122, indicating that the camera has moved slightly. However, if attempts to re-register the image portions at 120 are not successful, an alarm is raised at 124 indicating that the image portions have been shifted.

The inventor believes that the invention, as illustrated, provides a new method of analysing a series of images and a new image processor arranged to implement the method, which may be of particular use in security applications where tampering with images, not resulting in electrical faults, can otherwise go undetected.

Claims

CLAIMS:

1. A method of analysing a series of images containing static image portions and possibly dynamic image portions for changes, the method including defining at least one image region in the series of images, the image region or regions containing a static image portion; identifying transient occurrences in the image region or regions; selecting from the series of images at least two images that do not contain transient occurrences in the defined image region or regions; comparing the defined image region or regions in each of the selected images with one another; and detecting image changes in the defined image regions of each of the selected images.

2. The method as claimed in claim 1 , in which identifying transient occurrences is effected by detecting the frequency of change in the image region containing a static image portion, where a high frequency of change, i.e. a small number of changes, indicates a transient occurrence.

3. The method as claimed in claim 1 or claim 2, in which at least two image regions containing a static image portion are defined.

4. The method as claimed in any one of the preceding claims, in which the detection of image changes in the defined image regions of each of the selected images includes detecting a shift in the image.

5. The method as claimed in any one of the preceding claims, in which the detection of image changes in the defined image regions of each of the selected images includes detecting a persistent or gradual alteration in the spatial structure, i.e. location, orientation and/or magnitude of spatial intensity gradients, in the defined image region or regions.

6. The method as claimed in any one of the preceding claims, in which the detection of image changes in the defined image regions of each of the selected images includes detecting a change in spatial image intensity distribution.

7. The method as claimed in any one of the preceding claims, which includes, prior to identifying transient occurrences, detecting a baseline structure in the defined image region or regions containing a static image portion.

8. The method as claimed in claim 7, in which detecting image changes includes detecting a change in the baseline structure in the defined image region or regions containing a static image portion.

9. The method as claimed in any one of the preceding claims, in which comparing the defined image region or regions includes tracking image registration between the at least two selected images.

10. The method as claimed in claim 9, in which the tracking of image registration between the at least two selected images includes determining the least square error between the at least two selected images or determining the mutual information between the at least two selected images.

1 1. The method as claimed in any one of the preceding claims, in which comparing the defined image region or regions includes determining the differences in the image intensity histogram between the at least two selected images in the defined region or regions.

12. The method as claimed in claim 11 , in which the image intensity distribution is determined in the three primary colour channels or red, green and blue.

13. The method as claimed in any one of the preceding claims, in which a plurality of image regions containing a static image portion are defined, the method including identifying any one of several transient occurrences from a selection of images by comparing defined image regions and detecting image changes in the plurality of defined image regions, the method further including comparing the results of the detected image changes in the defined image regions thereby to obtain an indication of the reliability of the results of the detected image changes.

14. A method of detecting tampering with an image or with a device recording a series of images, the method including analysing a series of images containing static image portions and in response generating an alarm if a predefined image change has been detected in the static image portions of at least some of the images.

15. The method as claimed in claim 14, which includes identifying images in the series of images that precede or follow on from the tampered image or images.

16. The method as claimed in claim 14 or claim 15, in which the series of images is analysed according to the method as claimed in any one of claims 1 to 13 inclusive.

17. An image processor, the image processor including region definition means operable to define an image region containing a static image portion in a series of images; a transient detector for detecting, within the defined image region, transient occurrences in the series of images; an image comparator operable to compare the defined image regions of at least two images which do not contain transient occurrences in the defined image regions and then to produce an output corresponding to the differences in the defined image regions; and a trigger operable to trigger an event in response to the output from the image comparator when the output satisfies predefined criteria.

18. The image processor as claimed in claim 17, which includes a micro¬ processor and a set of instructions, the microprocessor being operable to execute the set of instructions, thereby providing the region definition means, the transient detector, the image comparator and the trigger.

19. The image processor as claimed in claim 17 or claim 18, in which the region definition means is operable to define at least two image regions containing a static image portion in a series of images.

20. The image processor as claimed in claim 18, in which the image comparator includes any one of a baseline structure algorithm, an image registration algorithm and an image intensity distribution algorithm.

21. The image processor as claimed in any one of claims 17 to 20 inclusive, in which the predefined criteria applied by the trigger to the output from the image comparator is adjusted automatically in accordance with trends established by the images not containing transient occurrences.

22. The image processor as claimed in any one of claims 17 to 21 inclusive, which includes an adaptive learning algorithm to detect a baseline structure in the defined image region or regions containing a static image portion.

23. The image processor as claimed in any one of claims 17 to 22 inclusive, in which the region definition means is configured to define a region of interest within an image, the processor being configured to discard all image information outside the region of interest.

24. The image processor as claimed in any one of claims 17 to 23 inclusive, which is programmed to perform the method of analysing a series of images as claimed in any one of claims 1 to 16 inclusive.