GB2442512A - Motion detector using video frame differences with noise filtering and edge change accentuation - Google Patents

Abstract

A motion detector comprises a computer program that uses a digital camera data stream to detect image scene changes; differences between the current and previous video frames are used, e.g. by subtracting (RGB) pixels. A filter can be used to reduce noise in the frame images, and edges of change may be accentuated by a change summation algorithm; that is, edges of true moving features - indicated by contiguous pixels, for example - score more highly than random pixel changes. The amount of noise suppression and the threshold for motion detection may be configured. Upon movement detection, the camera can record or stream live video or save images to a local or remote computer, and optionally alert an operator. Rows and columns are traversed and corrected change scores are summed providing a total change score; previous and current pixel changes are multiplied together so actual motion scores highest. The total sum of change is then used to calculate an average change threshold (rolling basis).

Description

MOTION DETECTOR

This invention relates to a camera motion detector.

Digital cameras are well known devices that convert light intensity into digital values. They are normally based on charge coupled devices (CDD) that are imprinted onto a chip that acts like a digital retina. Light is focused onto the chip using normal lenses, and the light intensity hitting each pixel is converted into a digital value.

There are monochrome and colour CCDs, although most these days are designed to detect human colour: red, green and blue. CCDs have been used for some time for various tasks such as security cameras. Recently however, CCDs have become ubiquitous since the cost of fabrication of the CCDs has fallen.

In the retail market, some cameras are portable and some are used as web cameras attached to a computer. Sometimes they are built into other devices such as mobile phones. Many cameras nowadays combine both standalone picture taking (and video clip recording) with web camera functionality.

When used as a web camera, the camera is usually attached to a computer and a software program grabs the pixels from the camera device (driver). The pixels are then used to display the image onto the screen or sent (via the internet) to another computer that displays the image on its screen (web conferencing).

Some web camera software also allows the recording of video clips and the taking of pictures. Most web cameras support a mode of 320 x 240 (red, green and blue) pixels at 15 frames per second. That means that the data stream is about 3.5 Megabytes per second. Hence, even with compression (e.g. MPEG) video clips are very large files.

When digital cameras are used in the security camera role, the digital data stream is sent to a computer and the images can be displayed on the screen by an operator. One computer can then be used to select to display one or more video streams from the cameras at its disposal.

The computer can also record the video stream if the operator so desires. Most security systems require continuous (periodic) monitoring by an operator who can decide to take appropriate action should something occur. Hence there is a limit as to how many cameras a single operator can cope with.

Many scenes are static scenes: that is, nothing moves in the scene. If nothing is happening, then the operator does not need to monitor the camera. Therefore it makes sense to have some method to detect motion such that the operator need only to monitor those cameras in which something is happening.

According to the present invention there is provided a motion detector whereby a computer program uses the data stream from a digital camera to detect changes in a scene.

Detecting the motion of objects is actually rather difficult because computers have no innate understanding of a scene. However, all the software is required to do is detect changes over time in the pixel intensities. Thus the program uses the differences between two or more frames (current plus previous) to detect motion.

Of course, the data from all cameras contains noise. Therefore a noise filter is used to suppress noise so that sensitivity to actual motion is improved. Different cameras exhibit different characteristics, and hence the noise filter can be configured using the software.

Actual motion is then detected by summing the (corrected) differences between each frame. Various algorithms can be used to excentuate actual motion. Although a simple summation does work, the sensitivity can be increased by introducing various assumptions about what constitutes actual motion.

If we take a scene of a blank background, and move a different colour ball across the front of the camera, the differences look like two crescents, one along the leading edge of motion, and one along the trailing edge. The edges comprise at least several contiguous pixels (of change). It is these edges of change that The detection (summation) algorithm is therefore designed to score edges of change higher than just random change. Hence the sensitivity to actual motion is increased because greater credence is placed on edges of change. Finally, once the sum passes a threshold value, motion is deemed to have occurred, and a motion event is fired.

The invention will now be described by way of example with reference to the accompanying figures in which: Figure 1 shows a screenshot of a typical scene in normal view.

Figure 2 shows a screenshot of a typical scene in diffing mode.

Figure 3 is the source code listing of the camera display/detector class.

Figure 4 is the source code of the camera renderer class.

Figure 5 is the source code of the camera controller class.

Figure 6 is the source code of the top level smooth frame class.

Figure 7 is the source code of the BMP file saving class.

The motion detector comprises a Java program which grabs pixels from a web camera attached to the computer using the standard 320x240 RGB byte array mode at 15 frames per second. Each video frame (or image) is then compared with the previous one to detect changes in the scene (on a pixel by pixel basis).

The first stage of comparison comprises noise suppression since there is normally quite a lot of (random) noise in each image which needs to be filtered out. In general, the amount of noise (red, green and blue) varies on a pixel by pixel basis.

This is because not only are there imperfections in the fabrication of the CCD, the noise actually varies with the intensity of light at each point (pixel). Thus a rolling" noise average needs to be calculated for each (red, green and blue) pixel.

This average is then used to suppress noise in the image by allowing each pixel value to vary by an amount proportional to the average noise, without being classed as a change. This is achieved by subtracting the average change (times a suitable proportion) from the actual change. The proportion will vary on a camera by camera basis and hence the software allows the proportion to be configured.

Although the noise suppression reduces the amount of random change considerably, simply summing the differences between the (previous and current) frames is not particularly sensitive. Remember that when an object moves in the scene, the edges of the object become highlighted by the differencing algorithm. It is these edges of change that we wish to accentuate so that moving objects can be detected with greater sensitivity.

The second stage of comparison is a summation algorithm that emphasises the edges of change. However, because the data rate (3.5 Megabytes per second) is very high, a fast and hence simple algorithm is required. Therefore two further assumptions are introduced: 1) The edges of change comprise contiguous areas of change.

2) Individual objects do not need to be tracked separately.

The initial noise reduction algorithm descends through the rows, traversing each horizontal line (per column). Thus it makes sense to have a second stage process that can be applied at the same time. A pure summation of the changes does work, but random noise and actual change score the same and hence is not particularly sensitive.

With random noise, it is very unlikely that two neighbouring pixels will fire at once, however in actual motion that is very likely. Therefore, for each pixel the previous change is multiplied by the current change. That means that two pixels of change (contiguous horizontally) are required to cause an addition to the change sum.

The algorithm is fast since the noise suppression and change summation can be performed in the same loops. And since actual motion scores much higher than random noise, the sensitivity is excellent. The threshold of motion for the sum can be adjusted on a camera by camera basis.

A "rolling" average total change score is kept, since it the relative amount of motion that is of actual importance. Thus the threshold is actually a proportion of this "rolling" average total change. When the current total change rises above the average total change times the threshold, motion is deemed to have been detected.

Once motion is detected the software then takes a picture by saving the current image into a (BMP) file. Of course, this is a simple example. Other actions could be performed. For example, a remote operator could be alerted over a network and the image streamed over the network (e.g. via RTP) for perusal.

Also, the images could be uploaded to a web site (e.g. via FTP) or sent to an email address or mobile phone so that the "operator" could peruse the scene whilst, for

example, on holiday.

Claims (6)

1) A motion detector comprising a computer program that uses the data stream from a digital camera to detect changes in a scene.

2) A motion detector according to claim 1 whereby the differences between the current and previous video frames are used to detect motion.

3) A motion detector according to claims 1 & 2 whereby a filter can be used to reduce noise in the frame images.

4) A motion detector according to claims 1 & 2 whereby the edges of change are accentuated by the change summation algorithm.

5) A motion detector according to claims 1, 2, 3 & 4 whereby the amount of noise suppression and the threshold of motion may be configured.

6) A motion detector according to claim I whereby when motion occurs the camera can record or stream live video or save still images to a local or remote computer, and optionally alert an operator.