GB2343318A - Video motion detection - Google Patents

Abstract

A video motion detector comprises: a difference detector 401 for detecting an image difference between a test area of a current image of a video signal and a corresponding area of another image of the video signal; a detail detector 402, 403 for detecting an amount of a high spatial frequency component in at least the test area of the current image; and a threshold comparator 430 for comparing the image difference with a threshold image difference to determine whether the test area represents a moving or a stationary part of the image; the motion detector is responsive to the detected amount of the high spatial frequency component so that a minimum image difference which is required for the test area to be detected as a moving part of the image increases with an increasing amount of the high spatial frequency component.

Description

VIDEO MOTION DETECTION This invention relates to video motion detection.

Motion detection in various forms is often used in video processing systems.

In some examples, a"motion vector"is generated, to represent the actual motion of a particular part of an image between successive images of a video signal. However, in other applications a simpler approach is used, in that it is necessary only to detect whether a particular part of an image moving or stationary between successive images.

An example of the use of this latter technique is an image processing system in which pixels of an interlaced output image can be generated from interlaced images of an input video signal by two different algorithms, such as frame-based interpolation or field-based interpolation, in dependence on the detection of motion for those pixels.

Frame-based interpolation, in which pixels from two or more fields are used to generate an output pixel, can give better spatial resolution than field-based interpolation where pixels from only one field are used. So, frame-based interpolation is the better choice when motion is not detected, but field-based interpolation gives a better result for moving portions of an image and so is the better choice where motion is detected.

Previously proposed motion detectors have detected motion by detecting the difference between a block of pixels from a field at time t and the a corresponding block from a field at time t-2, a technique referred to as a frame difference detection (the two test fields being separated by one frame). Figure I schematically illustrates such a motion detector.

The comparison is made between fields of the same polarity field signals to avoid the field interlace giving rise to problems with high vertical frequencies causing aliasing. Successive pairs of pixels at corresponding positions from fields t (i. e. a current field) and t-2 (i. e. the preceding field of the same polarity) are passed to a difference detector 10 where the difference in luminance between each pair of pixels is detected. A low pass filter (LPF) 20 filters the output of the difference detector 10 to smooth out sudden changes in the difference detection.

At the output of the LPF 20, absolute value logic 30 detects an absolute value of the (positive or negative) difference values. The absolute difference values are then averaged over a block of 5 x 3 pixels by an averaging circuit 40. The averaged value is then compared with a threshold value by a comparator 50, to generate a motion detection signal 60.

If the averaged difference value is detected to be greater than the threshold value then the motion detection signal is set to a state indicating"motion". If the averaged difference value is less than the threshold value then the motion detection signal indicates"no motion". (Clearly, if the two signals are equal then the output state of the motion detection signal is set by convention. Also, this description assumes that the averaged difference value increases numerically with increasing pixel difference, although of course the opposite polarity could be used for this signal).

Averaging the difference value over a block of pixels can reduce spurious detections and alleviate the effect of noise, but the averaging also reduces sensitivity to detail and can lead to other problems. In particular, because of the averaging process the motion detector arrives at the same result for a low contrast edge or object moving quickly as for a high contrast edge moving slowly. However, this is not necessarily an appropriate response; in the example given above, frame-based interpolation could well be the better choice for a high contrast edge moving slowly, but the system would be forced to use field-based interpolation because of the erroneous detection of motion.

This invention provides a video motion detector comprising: a difference detector for detecting an image difference between a test area of a current image of a video signal and a corresponding area of another image of the video signal ; a detail detector for detecting an amount of a high spatiatency component in at least the test area of the current image; and a threshold comparator for comparing the image deference with a threshold C) image difference to determine a degree of image motion in the test area; the motion detector being responsive to the detected amount of the high spatial frequency component so that a minimum image difference which is required for the test area to be detected as a moving part of the image increases with an increasing amount of the high spatial frequency component.

The present invention aims to alleviate the problems of the prior art by reducing the sensitivity of (e. g. by reducing the detected image difference or by increasing the threshold image difference when there is more image detail. An output that is more proportional to velocity can thereby be produced.

Further respective aspects and features of the invention are defined in the appended claims.

Embodiments of the invention will now be described, by way of example only, with reference to the accompanying drawings in which: Figure 1 is a schematic diagram of a previously proposed video motion detector; Figure 2 schematically illustrates a video camera apparatus; Figure 3 schematically illustrates a video processing apparatus; and Figure 4 schematically illustrates a motion detector.

Figure 2 schematically illustrates a video camera apparatus comprising a camera body 200 and a lens 210. Inside the camera body are a CCD image pick up device 220, signal processing circuitry 230 for reading a video signal from the CCD image pick up device 220, and lens aberration correction circuitry 240 for correcting aberrations caused by the particular lens in use.

The lens aberration correction circuitry 240 will be described in much greater detail below, but in summary the circuitry receives an aberration signal 250 defining distortions introduced by the lens 210 and applies corresponding corrections to the images from the image pick up device.

The aberration signal 250 from the lens 210 in fact comprises two lens output voltages: Vab, representing lateral chromatic aberration introduced by that lens and Vdis representing the distortion introduced by that lens. The two lens output voltages are generated by circuitry within the lens in dependence on a current zoom, iris and/or focus setting, according to measurement data produced by the lens manufacturer indicating the expected lens aberrations at that particular zoom, iris and focus setting.

Techniques for the generation and output of these lens aberration signals are well established in the art.

The two lens output voltages Vabr and Vd, s are defined by the following equations: Vabr = C (mm) x 72.7 (mV) + 2.5 (V) where C is the aberration at 4.4mm distance from the image centre, mV represents a quantity in millivolts and V represents a quantity in volts; and: Vdjs = 60 x P + 2. 5 (V) where d (mm) = (1. 252xY Y3) xPx44 Y (mm) = (distance from the image centre)/4.4 (mm) and d (mm) is the positional error (deflection) at Y mm from the image centre.

So, the lens aberration correction circuitry determines spatial distortions applied by the lens 210 to the image picked up by the image pickup device 220 and, using horizontal and vertical interpolation techniques applies corrections to restore the image to its undistorted form.

The video signal as corrected by the lens aberration correction circuitry 240 is output for further signal handling, e. g. recording, transmission, mixing etc. It is also routed to a monitor screen 260 within the camera body 200, so that the currently picked up image can be viewed via a viewfinder 270.

Figure 3 schematically illustrates a video processing apparatus embodying the lens aberration circuitry 240. The circuitry of Figure 3 receives a digital video signal from the signal processor 230 and the aberration signal from the lens 210, and generates an output video signal 280 to form the output from the camera apparatus.

The lens aberration circuitry comprises an error correction module 300, a horizontal interpolator 310, a vertical field-based interpolator 320, a vertical framebased interpolator 330, frame and field delays 340, a motion detector 350 and an output mixer 360.

As mentioned above, the lens aberration signal comprises two lens output voltages Vabr and Vdis, In order to make use of these in the circuitry for correcting lens-introduced distortions, these output voltages have to be converted into pixel corrections to be applied at various positions in each field of the video signal.

Equations defining this technique will now be described. The equations make use of a number of variables to be listed below, and for each variable the value applicable to the present embodiment is given in brackets: SIZEX-no of pixels of CCD (1920) SIZEY-no of frame lines of CCD (1080) CCDX-horizontal size in mm of CCD (9.587) CCDY-vertical size in mm of CCD (5. 382) XPOS-horizontal position in pixels from centre of CCD YPOS-vertical position in frame lines from centre of CCD Define x, the distance from the centre of the CCD in mm for XPOS : CCDX x (mm)-. XPOS SIZEX Define y, the distance from the centre of the CCD in mm for YPOS fame lines: CCDYYpOs SIZEY In the error correction module 300, the lens output voltage Vdi, is supplied to a distortion corrector 302 and the lens output Vabr is supplied to a chromatic aberration corrector 304. These each generate values for the distortion (as a number of pixels of image displacement) in the horizontal (x) and vertical (y) image directions for each position within the image. The distortion signals are added together by an adder 306 to form an error signal 308 supplied to the horizontal interpolator 310, the vertical fieldbased interpolator 320 and the vertical frame-based interpolator 330.

The distortion corrector 302 calculates the following equations using the variable P described earlier:

2 2 4.4P. x. (-"- (---)) 4. 4 4.4' Xdistortion (pixels) = CCDX/ /SIZEX 2 2 yz 4.42. y. (1. 4- (4 43)) 4.4 4. 43 Yciistortion (pixels) CCDY /SIZEY The chromatic aberration corrector 304 carries out the following calculations using the variable C defined earlier: C. XPOS Xchromatic(pixels) = 4.4 C. YPOS Ychromatic(pixels) = 4.4 So, the error signal 308 comprises an X component and a Y component giving the horizontal and vertical distortion represented by the lens output voltagesvdi, and Vabr at each position in the image, measured in terms of pixels in the horizontal direction and lines in the vertical direction.

The horizontal interpolator 310 comprises a pixel delay and integer pixel shift circuit 312 and an 11 tap 64 sub-position interpolation filter 314, although larger filters could be used. These operate in a standard fashion to interpolate each pixel along each line of the image from other pixels disposed along the same line, but applying any necessary horizontal shift to correct the horizontal component of the image distortion specified by the error signal 308.

So, for example, if the error signal 308 specifies that at a current image position, the lens has introduced a horizontal shift to the left of, say, 3.125 pixels, the horizontal interpolator 310 will interpolate a current output pixel based on an interpolation position 3.125 pixels to the right of the current output pixel's position.

Horizontal interpolation used in this manner for scaling images is well established in the art.

The particular interpolation filter used in this example is an 11 tap 64 subposition filter, which means that horizontal pixel shifts can be defined to sub-pixel accuracy, and in particular to an accuracy of one sixty-fourth of the separation between two adjacent pixels. Again, this technique is very well established in the art.

Accordingly, the horizontal interpolator 310 outputs lines of pixels in which the horizontal component of distortion defined by the lens output voltages Vdjs and Vabr has been corrected. These lines of pixels are supplied in parallel to the vertical fieldbased interpolator 320 and the vertical frame-based interpolator 330.

The vertical interpolators 320,330 act to correct the vertical component of the distortions specified by the lens output voltages Vdi, and Vab, in a roughly similar manner to the horizontal correction applied by the horizontal interpolator 310. So, if the distortion specified by the lens output voltages at a particular position in the image is a vertical shift of, say, 7. 375 lines upwards, the vertical interpolators will generate pixels at that image position by vertical interpolation about a centre position in the input field which is 7. 375 lines below the position corresponding to the current output pixel.

Each of the vertical field-based interpolator and the vertical frame-based interpolator comprises a series of first-in-first-out buffers (FIFOs) 322, 332 providing line delays for the interpolation process, a multiplexer 324,334 providing integer field line shifts and an 11 tap 32 sub-position interpolation filter 326,336 to provide subline accuracy in the interpolation process. In addition the vertical frame-based interpolator has a field delay 338 to provide two adjacent fields for use in the interpolation process.

The vertical field-based interpolator 320 and the vertical frame-based interpolator 330 operate in parallel to produce respective versions of each output pixel.

The selection of which version to use in forming the output video signal 280 is made by the output mixer 360 under the control of the motion detector 350.

The reason for this choice of the two versions of each output pixel is as follows. The vertical frame-based interpolator uses pixels from two adjacent fields to generate output pixels by interpolation. Because the two adjacent fields are interlaced, this gives much better spatial resolution and avoids alias problems when compared to vertical field interpolation where pixels from only one output field are used. However, if there is motion present in the part of the image currently being interpolated, the use of two temporally separated fields (in the frame-based interpolator) will lead to an inferior output because the image will have moved between the two fields.

Accordingly, if motion is detected by the motion detector 350 in a sub-area surrounding a current output pixel, that output pixel is selected to be the version produced by the field-based interpolator 320. Conversely, if motion is not detected in the sub-areas surrounding the current output pixel, that pixel is selected to be the version generated by the vertical frame-based interpolator 330. This selection is made by the output mixer 360 under the control of a motion signal 370 supplied by the motion detector 350.

The output mixer can work in another way, by operating a"soft threshold".

For example, if the threshold is x and a"degree of softness"is y, then for any degree of motion generated by the motion detector which is less than (x-y), the pixel from the frame-based interpolator will form the output pixel. If the degree of motion is grater than (x+y) then the pixel from the field-based interpolator will be used. Between these two levels, the output mixer combines the two possible output pixels in a normalise additive combination so that in the range ( (x-y) < degree of motion < (x + y)), the proportion of the frame-based pixel varies linearly with respect to degree of motion.

Figure 4 schematically illustrates the motion detector 350 in greater detail.

The motion detector 350 receives pixels of three temporally adjacent fields from the frame/field delays 340, namely a current field at a time t, the temporally preceding field t-1 and the field preceding that, t-2. The motion detector comprises a frame-based motion detector 400, a field-based motion detector 410, a non additive mixer (NAM) 420 and a threshold comparator 430.

In basic terms, the motion detector 350 operates to detect motions by two techniques in parallel. The frame-based motion detector 400 detects motion between corresponding blocks of pixels in two fields of the same polarity, namely field t and t2. The field-based motion detector 410 detects differences between corresponding blocks of pixels in two adjacent fields of opposite polarities, namely fields t and t-1.

After a scaling process (see below) the image differences detected by the two motion detectors 400,410 are combined by the NAM 420 before being compared to a threshold image difference 440 by the comparator 430.

If the comparator 430 detects that the output of the NAM 420 is greater than the threshold 440, an output motion signal 450 is set to a state indicating that motion is present in the block surrounding the current output pixel. If the outpu+ of the NAM 420 is less than the threshold 440, the motion signal 450 is set to a state indicating that motion is not present in the block of pixels surrounding the current output pixel.

(If the output of the NAM 420 is equal to the threshold 440, the setting applies to the motion signal 450 can be selected by convention. In the present embodiment, if the NAM output is equal to the threshold 440, it is considered by convention that motion is indeed present.) Considering first the frame-based motion detector, pixels of the two input fields t and t-2 are supplied to a difference detector 401 and in parallel to a pair of high pass filters 402 (or alternatively high pass vertical filters NAMed with high pass horizontal filters-which may also be used in the field-based detector). The difference detector operates to detect the difference in luminance between pixels at corresponding positions in the two fields. The high pass filters 402 operate to detect a degree of detail present in the two fields by extracting a high frequency component from the bit stream representing the two fields. The output of the two high pass filters 402 are processed 403 to generate a gain control signal 404 which is supplied with the luminance difference detected by the luminance detector 401, to a multiplier 405.

The gain control signal 404 is formed so that as the high frequency component (a representation of image detail) of one or both of the input fields increases, the image difference as detected by the difference detector 401 is scaled down so that the value passed to the low pass filter 406 appears to represent a smaller image difference. This can be achieved by a number of ways, but the example used here is to calculate the gain control signal as follows: gain control signal = k-max (HPFl, HPF2 The use of the gain control signal to reduce the sensitivity of the frame-based motion detector to image difference when there is a large detail (high spatial frequency) component avoids the problem that the block averaging process described below could otherwise lead to the frame-based motion detector arriving at the same result for a low contrast edge or object moving quickly as for a high contrast edge moving slowly. This of course would not be an appropriate response; in the example given above, frame-based interpolation could well be the better choice for a high contrast edge moving slowly, but the system would be forced to use field-based interpolation because of the erroneous detection of motion. So, the sensitivity to image difference is reduced, in this embodiment by scaling down the detected difference signal, as the amount of detail detected by the HPFs increases.

The output of the multiplier 405 is supplied to a low pass filter 406 which filters the gain-controlled difference signal to smooth out sudden changes in the difference detection. Absolute value logic 407 then detects the absolute value of the (positive or negative) difference values. The absolute difference values are then averaged over a block of 5 x 3 pixels by an averaging circuit 408, with the average being supplied to the NAM 420.

Turning to the field-based difference detector 410, the two fields are supplied in parallel to vertical low pass filters 411 which are arranged to remove high frequency components from the fields to avoid alias problems and to provide a fractional line shift to each of the two (opposite polarity) fields so that a proper comparison can be made by a difference detector 412. The filtered pixels are then supplied to the difference detector 412, a low pass filter 413 and an absolute value detector 414 as described above. The output of the absolute value detector 414 is multiplied by a scaling coefficient a before the scaled result is passed to an averaging circuit 415. The averaged output is supplied to the NAM 420.

The coefficient a can be adjusted by the user by means of an adjustment control on the camera body (not shown), so that the user can vary the relative response of the frame-based and field-based motion detectors.

The NAM 420 combines its two inputs so that the output of the NAM represents the greater of the two inputs.

Claims (12)

1. CLAIMS 1. A video motion detector comprising: a difference detector for detecting an image difference between a test area of a current image of a video signal and a corresponding area of another image of the video signal ; a detail detector for detecting an amount of a high spatial frequency component in at least the test area of the current image; and a threshold comparator for comparing the image difference with a threshold image difference to determine a degree of image motion in the test area; the motion detector being responsive to the detected amount of the high spatial frequency component so that a minimum image difference which is required for the test area to be detected as a moving part of the image increases with an increasing amount of the high spatial frequency component.
2. 2. A motion detector according to claim 1, in which: the detail detector is operable to detect the amount of the high spatial frequency component in the test area of the current image and in the corresponding area of the other image; and the motion detector is responsive to the greater of the two detected amounts of the high spatial frequency component.
3. 3. A motion detector according to claim 1 or claim 2, comprising means for varying the detected image difference and/or the threshold image difference in response to the amount of the high spatial frequency component.
4. 4. A motion detector according to any one of the preceding claims, in which the detail detector comprises a high pass spatial filter operable to detect the amount of the high spatial frequency component in at least the test area.
5. 5. A motion detector according to any of the preceding claims, comprising: a low pass spatial filtering arrangement for filtering the detected image difference.
6. 6. Video signal processing apparatus for processing images of an input video signal, the apparatus comprising: a motion detector according to any one of the preceding claims, the motion detector being arranged to detect whether motion is present in each of a plurality of image areas in images of the video signal; and a video signal processor for processing images of the video signal, the video signal processor having at least a first and a second mode of operation, the first or the second mode of operation being selected for processing each image area in response to a detection by the motion detector of a degree of image motion present in that image area.
7. 7. Video camera apparatus comprising : an image pickup device for generating a video signal; a lens arrangement for focusing light onto the image pickup device, the lens being operable to generate a lens aberration signal in response at least to a current focus and/or zoom setting; a video signal processor according to claim 6, the video signal processor being operable to process the video signal in accordance with the lens aberration signal.
8. 8. A method of video motion detection comprising the steps of : detecting an image difference between a test area of a current image of a video signal and a corresponding area of another image of the video signal; detecting an amount of a high spatial frequency component in at least the test area of the current image ; and comparing the image difference with a threshold image difference to determine a degree of image motion in the test area; varying operation of the motion detector in response to the detected amount of the high spatial frequency component so that a minimum image difference which is required for the test area to be detected as a moving part of the image increases with an increasing amount of the high spatial frequency component.
9. 9. A video motion detector substantially as hereinbefore described with reference to Figure 4 of the accompanying drawings.
10. 10. Video camera apparatus substantially as hereinbefore described with reference to Figures 2 to 4 of the accompanying drawings.
11. 11. Video signal processing apparatus substantially as hereinbefore described with reference to Figures 3 and 4 of the accompanying drawings.
12. 12. A method of video motion detection substantially as hereinbefore described with reference to Figure 4 of the accompanying drawings.