GB2276999A - Motion compensated video signal processing; detecting "ridge" motion - Google Patents

Abstract

A correlation surface is formed and stored 300 by comparing search blocks in a first image with a search area, comprising a plurality of search blocks in a second image, and a motion vector determined on the basis of the position of maximum correlation. Movement of a "long" (longer than the dimension of the search block) object along its length causes a ridge of similar maxima to occur on the correlation surface in the direction of motion; these are detected 360, 400 and counted 490 to generate a ridge flag 495 if a predetermined number occur. If ridge motion is detected a true motion vector may be determined from an average of the most widely spaced maximum correlation positions. The correlation surface may be formed by subtracting luminance values in the two images to produce a correlation surface in which a minimum represents a point of maximum correlation. <IMAGE>

Description

MOTION COMPENSATED VIDEO SIGNAL PROCESSING This invention relates to motion compensated video signal processing.

Motion compensated video signal processing is used in applications such as television standards conversion, film standards conversion and conversion between video and film standards.

In a motion compensated television standards converter, such as the converter described in the British Published Patent Application number GB-A-2 231 749, pairs of successive input images are processed to generate sets of motion vectors representing image motion between the pair of input images. The processing is carried out on discrete blocks of the images, so that each motion vector represents the interimage motion of the contents of a respective block.

In the motion vector estimation process a correlation surface representing spatial correlation between blocks of the two input images is examined to detect points of maximum correlation. (The correlation surface actually represents the difference between the two input images; this means that the points of maximum correlation are in fact minima on the correlation surface, and will be referred to herein as "minima"). When a minimum has been detected, a motion vector is generated from the spatial position of the minimum in the correlation surface. Tests are performed to determine whether that minimum represents a significant peak in correlation with respect to the remainder of the correlation surface. If the minimum passes this test it is treated as being "valid", and a validity flag associated with the motion vector is set.

Each set of motion vectors is then supplied to a motion vector reducer which derives a subset of the set of (valid) motion vectors for each block, which is then passed to a motion vector selector which assigns one of the subset of motion vectors to each picture element (pixel) in each block of the image. The selected motion vector for each pixel is supplied to a motion compensated interpolator which interpolates output images from the input images, taking into account the motion between the input images.

A problem can occur if the image contains moving objects (e.g.

lampposts) which, in one direction, are longer than the size of the search blocks used in block matching. In this case, because the moving object extends out of the search block used in block matching, it is not possible (within that search block) to detect the component of the object's motion along the object's longitudinal axis. The result is that a "one-dimensional" or ridge minimum (a ridge correlation maximum) is detected.

Figure 1 of the accompanying drawings schematically illustrates an example of a correlation surface 5 having a ridge minimum 10. The ridge minimum defines a component of the object's motion in a direction perpendicular to the ridge direction (the ridge direction lying along the longitudinal axis of the object), but gives no indication of the motion of the object along the ridge direction.

In previously proposed motion vector estimators, motion vectors generated from ridge minima have failed the validity test and have therefore not been used in interpolation of the output image. Instead, the zero motion vector has been used by default. However, using the zero motion vector to interpolate pixels in a long moving object can result in artifacts such as the omission of portions of the object. An artifact of this type is illustrated in Figure 2 of the accompanying drawings. which shows the interpolation of an output image having a long vertically moving object 15. A correct motion vector 20 is detected at the top and bottom of the moving object, but a default zero motion vector 25 is selected for the central portion of the object. A gap 30 is formed between the upper portion interpolated using the correct motion vector and the central portion interpolated using the zero motion vector.

This invention provides a motion compensated video signal processing apparatus in which motion vectors are generated to represent image motion between a pair of input images of an input video signal, the apparatus comprising: means for comparing a search block within one of the pair of input images with a search area, comprising a plurality of blocks, in the other of the pair of input images, to generate a correlation surface, the correlation surface comprising an array of correlation values representing correlation between the search block and the search area; means for detecting a maximum correlation value in the correlation surface; means for generating a motion vector in dependence on the position of the maximum correlation value in the correlation surface; and ridge detecting means for detecting correlation values in the correlation surface within a threshold amount of the maximum correlation value, indicative of a ridge correlation maximum.

The invention provides a means of detecting ridge correlation maxima in correlation surfaces. Following this detection, various steps can then be taken such as the non-use of the resulting motion vectors, the use of larger search blocks to try to avoid the problem, or the attempted re-qualification of the motion vectors.

In one preferred embodiment, the ridge detecting means comprises means for detecting whether more than a predetermined number of correlation values are within the threshold amount of the maximum correlation value. In order that appropriate action can be taken in dependence on the severity of the ridge, it is also preferred that the ridge detecting means comprises means for detecting whether fewer than a second predetermined number of correlation values are within the threshold amount of the maximum correlation value, the second predetermined number being greater than the firstmentioned predetermined number.

Preferably the apparatus comprises means, responsive to a detection of a ridge correlation maximum, for generating an average motion vector dependent on the positions in the correlation surface of the two most widely spaced correlation values within the threshold amount of the maximum correlation value.

In another preferred embodiment the ridge detecting means comprises means for detecting a region of the correlation surface comprising those correlation values within the threshold amount of the maximum correlation value; means for detecting the ratio of the length of the region in a direction indicative of vertical image motion and the length of the region in a direction indicative of horizontal image motion; means for comparing the ratio with a first predetermined value to detect a ridge correlation maximum in the direction indicative of horizontal image motion; and means for comparing the ratio with a second predetermined value to detect a ridge correlation maximum in the direction indicative of vertical image motion.

In order that the detection of a ridge correlation maximum can be communicated to other parts of the apparatus, it is preferred that the apparatus comprises means for setting a ridge flag indicative of a ridge correlation maximum.

Preferably the ridge flag is indicative of whether the ridge correlation maximum lies in the direction indicative of horizontal image motion or in the direction indicative of vertical image motion.

In order that the component of a motion vector generated from a ridge minimum can be assigned a valid value, and the motion vector then used in motion compensated interpolation, a preferred embodiment comprises means for comparing a motion vector under test, derived from a correlation surface in which a ridge correlation maximum was detected, with a further motion vector generated from the pair of input images; means for detecting whether corresponding components of the further motion vector and the motion vector under test are substantially identical; means for detecting whether the further motion vector was derived from a correlation surface in which a ridge correlation maximum was detected; and means, responsive to a detection that the further motion vector was not derived from a correlation surface in which a ridge correlation maximum was detected, for replacing a component of the motion vector under test in the ridge direction by the corresponding component of the further vector.

Preferably the further motion vector is derived from a correlation surface displaced, in the ridge direction, from the motion vector under test.

In a preferred embodiment the further motion vector is derived from a correlation surface longitudinally adjacent, in the ridge direction, to the motion vector under test.

In order to allow the re-qualification of motion vectors derived from correlation surfaces having ridge correlation maxima representing the motion of long diagonal objects, in a preferred embodiment the further motion vector is derived from a correlation surface diagonally adjacent. in the ridge direction, to the motion vector under test.

Preferably the threshold value is dependent upon the mean of the correlation values in the correlation surface.

Although the correlation surface could comprise correlation values which increase with increasing correlation, in an advantageously simple embodiment the correlation surface comprises an array of correlation values representing the difference between the content of the search block and the search area. In this way a correlation maximum is actually represented by a minimum in the correlation surface.

Although the correlation values could represent the difference between the chrominance, or the luminance and chrominance, components of the search block and the search area, it is preferred that the correlation values represent the difference between the luminance content of the search block and the search area.

Apparatus according to the invention is particularly advantageously employed as part of a motion compensated television standards converter.

Viewed from a second aspect this invention provides a method of motion compensated video signal processing in which motion vectors are generated to represent image motion between a pair of input images of an input video signal, the method comprising the steps of: comparing a search block within one of the pair of input images with a search area, comprising a plurality of blocks, in the other of the pair of input images, to generate a correlation surface, the correlation surface comprising an array of correlation values representing correlation between the search block and the search area; detecting a maximum correlation value in the correlation surface; generating a motion vector in dependence on the position of the maximum correlation value in the correlation surface; and detecting correlation values in the correlation surface within a threshold amount of the maximum correlation value, indicative of a ridge correlation maximum.

The invention will now be described by way of example with reference to the accompanying drawings, throughout which like parts are referred to by like references, and in which: Figure 1 is a schematic diagram of a ridge (one-dimensional) minimum in a correlation surface; Figure 2 is a schematic diagram illustrating image artifacts resulting from the use of a default zero motion vector in motion compensated interpolation of an elongate moving object; Figure 3 is a schematic block diagram of a motion compensated television standards conversion apparatus; Figure 4 illustrates an embodiment of a motion vector estimator for detecting ridge minima; Figures 5a, 5b and 5c illustrate the operation of the motion vector estimator of Figure 4; Figure 6 illustrates a second method of ridge minimum detection; Figures 7, 8 and 9 illustrate three classifications of correlation surface minima; Figure 10 is a schematic diagram showing the re-qualification of a motion vector derived from a ridge minimum; Figure 11 is a schematic block diagram of part of a motion vector reducer; and Figure 12 is a flow chart illustrating the process steps performed by the apparatus of Figure 11.

Figure 3 is a schematic block diagram of a motion compensated television standards conversion apparatus. The apparatus receives an input interlaced digital video signal 50 (e.g. an 1125/60 2:1 high definition video signal (HDVS)) and generates an output interlaced digital video signal 60 (e.g a 1250/50 2:1 HDVS).

The input video signal 50 is first supplied to an input buffer/packer 110. In the case of a conventional definition input signal, the input buffer/packer 110 formats the image data into a high definition (16:9 aspect ratio) format, padding with black pixels where necessary. For a HDVS input the input buffer/packer 110 merely provides buffering of the data.

The data are passed from the input buffer/packer 110 to a matrix circuit 120 in which (if necessary) the input video signal's format is converted to the standard "CCIR recommendation 601" (Y,Cr,Cb) format.

From the matrix circuit 120 the input video signal is passed to a time base changer and delay 130, and via a sub-sampler 170 to a subsampled time base changer and delay 180. The time base changer and delay 130 determines the temporal position of each field of the output video signal, and selects the two fields of the input video signal which are temporally closest to that output field for use in interpolating that output field. For each field of the output video signal, the two input fields selected by the time base changer are appropriately delayed before being supplied to an interpolator 140 in which that output field is interpolated. A control signal t, indicating the temporal position of each output field with respect to the two selected input fields, is supplied from the time base changer and delay 130 to the interpolator 140.

The subsampled time base changer and delay 180 operates in a similar manner, but using spatially subsampled video supplied by the subsampler 170. Pairs of input fields are selected by the subsampled time base changer and delay 180 from the subsampled video, to be used in the generation of motion vectors.

The time base changers 130 and 180 can operate according to synchronisation signals associated with the input video signal, the output video signal, or both. In the case in which only one synchronisation signal is supplied, the timing of fields of the other of the two video signals is generated deterministically within the time base changers 130, 180.

The pairs of fields of the subsampled input video signal selected by the subsampled time base changer and delay 180 are supplied to a motion processor 185 comprising a direct block matcher 190, a data stripper 200, a motion vector estimator 210, a motion vector reducer 220, a motion vector selector 230 and a motion vector post-processor 240. The pairs of input fields are supplied first to the direct block matcher 190 which calculates correlation surfaces representing the spatial correlation between search blocks in the temporally earlier of the two selected input fields and (larger) search areas in the temporally later of the two input fields. Data representing these correlation surfaces are reformatted by the stripper 200 and are passed to the motion vector estimator 210. The motion vector estimator 210 detects points of greatest correlation in the correlation surfaces.

(The correlation surfaces actually represent the difference between blocks of the two input fields; this means that the points of maximum correlation are in fact minima on the correlation surface, and are referred to as "minima"). In order to detect a minimum, additional points on the correlation surfaces are interpolated, providing a degree of compensation for the loss of resolution caused by the use of subsampled video to generate the surfaces. From the detected minimum on each correlation surface, the motion vector estimator 210 generates a motion vector which is supplied to the motion vector reducer 220.

The motion vector estimator 210 also performs a confidence test on each generated motion vector to establish whether that motion vector is significant above the general noise level, and associates a confidence flag with each motion vector indicative of the result of the confidence test. The confidence test, known as the "threshold" test, is described (along with other features of the apparatus of Figure 3) in GB-A-2 231 749.

A test is also performed by the motion vector estimator 210 to detect whether each vector is aliased. In this test, the correlation surface (apart from an exclusion zone around the detected minimum) is examined to detect the next lowest minimum. If this second minimum does not lie at the edge of the exclusion zone, the motion vector derived from the original minimum is flagged as being potentially aliased.

The motion vector reducer 220 operates to reduce the choice of possible motion vectors for each pixel of the output field, before the motion vectors are supplied to the motion vector selector 230. The output field is notionally divided into blocks of pixels, each block having a corresponding position in the output field to that of a search block in the earlier of the selected input fields. The motion vector reducer compiles a group of four motion vectors to be associated with each block of the output field, with each pixel in that block eventually being interpolated using a selected one of that group of four motion vectors.

Vectors which have been flagged as "aliased" are re-qualified during vector reduction if they are identical to non-flagged vectors in adjacent blocks.

As part of its function, the motion vector reducer 220 counts the frequencies of occurrence of "good" motion vectors (i.e. motion vectors which pass the confidence test and the alias test, or which were requalified as non-aliased), with no account taken of the position of the blocks of the input fields used to obtain those motion vectors. The good motion vectors are then ranked in order of decreasing frequency.

The most common of the good motion vectors which are significantly different to one another are then classed as "global" motion vectors.

Three motion vectors which pass the confidence test are then selected for each block of output pixels and are supplied, with the zero motion vector, to the motion vector selector 230 for further processing.

These three selected motion vectors are selected in a predetermined order of preference from: (i) the motion vector generated from the corresponding search block; (ii) those generated from surrounding search blocks ("local" motion vectors); and (iii) the global motion vectors.

The motion vector selector 230 also receives as inputs the two input fields which were selected by the subsampled time base changer and delay 180 and which were used to calculate the motion vectors.

These fields are suitably delayed so that they are supplied to the motion vector selector 230 at the same time as the vectors derived from those fields. The motion vector selector 230 supplies an output comprising one motion vector per pixel of the output field. This motion vector is selected from the four motion vectors for that block supplied by the motion vector reducer 220.

The vector selection process involves detecting the degree of correlation between test blocks of the two input fields pointed to by a motion vector under test. The motion vector having the greatest degree of correlation between the test blocks is selected for use in interpolation of the output pixel. A "motion flagon is also generated by the vector selector. This flag is set to "static" (no motion) if the degree of correlation between blocks pointed to by the zero motion vector is greater than a preset threshold.

The vector post-processor reformats the motion vectors selected by the motion vector selector 230 to reflect any vertical scaling of the picture, and supplies the reformatted vectors to the interpolator 140. Using the motion vectors, the interpolator 140 interpolates an output field from the corresponding two (non-subsampled) interlaced input fields selected by the time base changer and delay 130, taking into account any image motion indicated by the motion vectors currently supplied to the interpolator 140.

If the motion flag indicates that the current output pixel lies in a moving part of the image, pixels from the two selected fields supplied to the interpolator are combined in relative proportions depending on the temporal position of the output field with respect to the two input fields (as indicated by the control signal t), so that a larger proportion of the nearer input field is used. If the motion flag indicates is set to "static" then temporal weighting is not used.

The output of the interpolator 140 is passed to an output buffer 150 for output as a HDVS output signal, and to a down-converter 160 which generates a conventional definition output signal 165, using the motion flag.

The subs ampler 170 performs horizontal and vertical spatial subsampling of the input fields received from the matrix 120.

Horizontal subsampling is a straightforward operation in that the input fields are first pre-filtered by a half-bandwidth low pass filter (in the present case of 2:1 horizontal decimation) and alternate video samples along each video line are then discarded, thereby reducing by one half the number of samples along each video line.

Vertical subsampling of the input fields is complicated by the fact that the input video signal is interlaced. This means that successive lines of video samples in each interlaced field are effectively separated by two video lines of the complete frame, and that the lines in each field are vertically displaced from those in the preceding or following field by one video line of the complete frame.

The method of vertical subsampling actually used involves a first stage of low pass filtering in the vertical direction (to avoid aliasing), followed by a filtering operation which effectively displaces each pixel vertically by half a video line downwards (for even fields) or upwards (for odd fields). The resulting displaced fields are broadly equivalent to progressively scanned frames which have been subsampled vertically by a factor of two.

Figure 4 is a schematic block diagram of an embodiment of the motion vector estimator 210. Correlation surfaces comprising an array of correlation values are received from the data stripper 200 and are stored in a correlation surface store 300. The correlation values are read from the correlation surface store 300 by a "first" minimum detector 310, which scans the correlation surface to detect a single minimum value from the correlation surface. In fact the first minimum detector 310 outputs three items: the actual minimum correlation value (MV) 320, the average correlation value (AV) 330 of the correlation surface, and the coordinates (or position) (MC) 340 of the minimum value 320 in the correlation surface.

After this first scan has been performed to detect the minimum value (MV) 320, a second examination of the correlation surface is performed by a "next" minimum detector 360. The next minimum detector 360 receives the correlation values from the correlation store 300, along with the coordinates of those correlation values. The coordinates of the correlation values are also supplied to an exclusion generator 370 which generates an exclusion flag 380 to mask an exclusion region of, for example, 5x5 correlation values around the coordinates 340 of the minimum value 320. The next minimum detector 360 then detects the minimum correlation value outside of the exclusion region. This detected next minimum value is subjected to a threshold comparison in which the difference between the detected next minimum value and the minimum value 320 is compared with a threshold value derived using the average value 330 of the correlation surface. If the difference between the minimum value 320 and the next minimum value 390 is greater than the threshold value, a "valid vector" flag (W) 390 is set to indicate that the first minimum value 320 represents a significant correlation peak.

A third examination of the correlation surface is performed by a "similar" minimum detector 400. The similar minimum detector detects all correlation values in the correlation surface lying within a threshold value of the minimum value 320. Again, the threshold value used in the similar minimum detector 400 is derived using the average value 330 of the correlation surface. As each of these points is detected by the similar minimum detector 400, a count held by a minima counter and comparator 410 is incremented.

The similar minimum detector 400 detects the position of the two correlation value within the threshold of the minimum value 320 which are most widely spaced in the correlation surface. The coordinates 420 of these most widely spaced minima are passed to a vector averager 440.

The vector averager 440 generates an output 450 representing a motion vector derived from the mean position of the most widely spaced minima 420 as detected by the similar minimum detector 400. This output 450 is passed to an output vector selector 460. The vector averager 440 also detects the spatial relationship of the most widely spaced minima 420, to detect whether the ridge lies in a substantially horizontal or a substantially vertical direction. The result of this test is output as a "ridge direction" flag 445.

The valid vector flag 390 and the minimum coordinates 340 are supplied to a vector generator 470, in which (if the first minimum is valid) a motion vector is generated from the minimum coordinates 340.

This motion vector is passed to the output vector selector 460.

The count held by the minima counter 410 is compared with two threshold values indicative of a ridge range 480. if the count held by the minima counter 430 indicates that more than a first predetermined number, but fewer than a second predetermined number, of minima have been detected in the correlation surface, a "valid ridge" flag (VR) 490 and a "ridge" flag 495 are set to indicate that a ridge minimum has been detected but valid use can be made of the ridge vector 450 generated by the vector averager 440. In this case, the output vector selector supplies the ridge vector 450 as an output vector 465.

If more than the second predetermined number of minima have been detected, the valid ridge flag 490 is not set, but the ridge flag 495 is set. This indicates that the ridge vector 450 should not be used, and the motion vector should either be re-qualified (see below) or not used. In this case, the output vector selector supplies the motion vector generated by the vector generator 470 as the output vector 465.

If fewer than the first predetermined number of minima have been detected, neither the valid ridge flag 490 nor the ridge flag 495 are set. Again, in this case, the output vector selector supplies the motion vector generated by the vector generator 470 as the output vector 465.

Figures 5a, 5b and 5c illustrate the operation of the motion vector estimator of Figure 4 when the valid ridge flag 490 is set.

Figures 5a and 5b illustrate two successive images having an elongate vertical object 500 with a horizontal motion component 510 of y. When blocks 520, 530 of the images shown in Figures 5a and 5b are compared to generate a correlation surface 540 (Figure 5c), a large number of minima 550 are generated having very similar correlation values.

When the correlation surface 540 is processed by the motion vector estimator of Figure 4, the two most widely spaced minima 560.

570 are passed to the vector averager 440. The vector averager 440 generates a ridge vector 450 equal to (y+vb)/2, i.e. a motion vector representing the mean position of the two minima 560, 570. Assuming that the number of minima 550 exceeds the predetermined number, the ridge vector 450 is output by the vector output selector 460 as the output vector 465.

Figure 6 illustrates a second, similar, method of ridge minimum detection. In a first examination of a correlation surface 600, the minimum correlation value 610 is detected. All correlation values within a threshold of the minimum value 610 are then detected, to define a threshold region 620. However, in contrast to the method described with reference to Figure 4, the number of minima within the threshold of the minimum value 610 is not detected. Instead, an 'aspect ratio', i.e. the ratio of length (in the vertical direction) to width (in the horizontal direction) of the threshold region 620 is detected. From the aspect ratio, the threshold region can be classified into one of three classifications illustrated in Figures 7, 8 and 9.

Referring to Figure 7, if the detected aspect ratio lies between a first and a second predetermined value, indicating that the threshold region 630 is not elongate, the minimum is classified as a point minimum having valid coordinates in both the horizontal (x) and vertical (y) directions.

In Figure 8, a minimum region 640 has an aspect ratio of greater than the first predetermined value, indicating that the region is elongate in the vertical direction 650. The minimum is then classified as a vertical ridge minimum and the resulting motion vector has a defined component 660 in the horizontal direction but an undefined component in the vertical direction 650.

Similarly, Figure 9 illustrates a threshold region 670 having an aspect ratio of less than the second predeter vertical direction in the case of a vertical ridge minimum) is undefined. However, an attempt is made as part of the vector reduction process of the present embodiment to re-qualify motion vectors derived from horizontal or vertical ridge minima, by assigning a valid vector component in the ridge direction. This re-qualification of motion vectors derived from ridge minima is illustrated schematically in Figure 10.

In the motion vector reducer, motion vectors are stored with their associated ridge and confidence flags, in an array memory 680 at positions corresponding to the relative positions of the search blocks from which the motion vectors were derived. During an examination of the motion vectors, if a motion vector derived from a ridge minimum is encountered, adjacent motion vectors in the ridge direction are examined to detect whether the vector component at 900 to the ridge direction matches (i.e lies within a threshold of) the corresponding component of the motion vector derived from the ridge minimum.

If a match is found, it is assumed that the adjacent vector relates to the same elongate moving object. If the adjacent vector was itself derived from a ridge minimum, the search is continued in the ridge direction. Eventually, a motion vector may be found which has a substantially identical component at 900 to the ridge direction but was derived from a point minimum (i.e. it has a defined component along the ridge direction). It is assumed that this motion vector represents the motion of the end of the elongate object, and therefore that the component of this motion vector in the ridge direction is equal to the previously undefined component in the ridge direction of the motion vector under examination.

Two examples of this process (one horizontal, one vertical) are illustrated schematically in Figure 10. In the vertical example, a motion vector 700 has a ridge flag indicating that it was generated from a vertical ridge minimum, and accordingly has a good horizontal (x) component but an undefined vertical (y) component. An adjacent motion vector 710 in the ridge direction is examined and is found to have a matching x component but an undefined y component (the vector 710 also being generated from a vertical ridge minimum). Similarly, further searches of vectors 720 and 730 also indicate a matching x component but an undefined y component. Eventually, a motion vector 740 is detected having a matching x component and a defined y component (this motion vector being generated from a point minimum). It is assumed that the motion vector 740 represents the motion of the end of the elongate object, and accordingly the y component of the motion vector 740 is assigned to the motion vector 700. The ridge flag associated with the motion vector 700 is modified to indicate that the motion vector 700 has been re-qualified (by setting the ridge flag to indicate that the motion vector 700 was derived from a point minimum) and can therefore be used in interpolation of an output pixel.

A corresponding process is illustrated for the re-qualification of a motion vector 750 derived from a horizontal ridge minimum. In this case, a first search in a direction from left to right is unsuccessful, because the edge of the array 680 of the motion vectors is reached. A second search is then carried out, in a direction from right to left, in which three motion vectors 760 are detected which have y components matching the motion vector 750 but undefined x components.

Eventually, a motion vector 770 is detected which has a y component matching that of the motion vector 750, and a defined x component. The x component of the motion vector 770 is assigned to the motion vector 750 and the ridge flag modified as described above.

Figure 11 is a schematic block diagram of a part of the motion vector reducer 220 for performing the above process. Motion vectors received from the motion vector estimator 210 are stored in a input vector store 800 along with their confidence and ridge flags. A vector processor 810 reads the stored motion vectors and performs the requalification process described above with reference to Figure 10. The possibly re-qualified motion vectors output by the vector processor 810 are stored in an output vector store 820, before being passed for further vector reduction processing. The use of separate input and output vector stores 800, 820 avoids the propagation of errors through the array of vectors as a result of the re-qualification process.

Figure 12 is a flow chart illustrating the process steps performed by the vector processor 810 of Figure 11.

At a step 900 a motion vector under test (along with its ridge flag) is read from the input vector store 800. At a step 910 the ridge flag associated with the vector under test is examined to detect whether the vector under test was generated from a horizontal or vertical ridge minimum. If the vector under test was generated from a point minimum then, at a step 920, the vector under test and its associated flags are written to the output vector store 820. Control is then returned to the step 900 so that the next motion vector can be read from the input vector store 800.

If the motion vector under test was derived from a ridge minimum, control is passed to a step 930 in which the adjacent motion vector in the ridge direction is read. At a step 940 the component of the adjacent motion vector at 900 to the ridge direction is compared with the corresponding component of the vector under test. If these values differ by more than a threshold amount, control is passed to a step 950 in which the vector under test is written without modification (i.e.

without re-qualification) to the output vector store 820. From the step 950, control is returned to the step 900 where the next motion vector under test is read from the input store 800.

If the result of the test performed at the step 940 is that the adjacent vector does have substantially the same component at 900 to the ridge direction, then control is passed to a step 960 in which the ridge flag associated with the adjacent vector is examined. If the adjacent vector was itself derived from a ridge minimum, then control is returned to the step 930 where the next adjacent vector is examined.

If the test performed at the step 960 indicates that the adjacent vector was not derived from a ridge minimum, then control is passed to a step 970 where the component of the adjacent vector is assigned to the motion vector under test. Then, at a step 980, the ridge flag associated with the motion vector under test is modified to indicate that the motion vector under test has been re-qualified. This is achieved by setting the ridge flag for the motion vector under test to indicate that the motion vector under test was derived from a point minimum.

Finally, at a step 990, the modified value of the vector under test (along with its associated modified flags) is stored in the output vector store 820 and control is returned to the step 900.

In a refinement of the process described above, vectors which are diagonally adjacent to a vector derived from a ridge minimum could be examined to locate an identical vector component at 900 to the ridge direction. At each stage, the adjacent vector along the ridge direction would be examined at the steps 930, 940 and at 960 described above, followed by the vector on either side of that adjacent vector.

This refined embodiment can give better results in the less frequent case of long moving diagonal objects, but at the expense of an increase in the processing overhead of the apparatus.

Claims (19)

1. Motion compensated video signal processing apparatus in which motion vectors are generated to represent image motion between a pair of input images of an input video signal, the apparatus comprising: means for comparing a search block within one of the pair of input images with a search area, comprising a plurality of blocks, in the other of the pair of input images, to generate a correlation surface, the correlation surface comprising an array of correlation values representing correlation between the search block and the search area; means for detecting a maximum correlation value in the correlation surface; means for generating a motion vector in dependence on the position of the maximum correlation value in the correlation surface; and ridge detecting means for detecting correlation values in the correlation surface within a threshold amount of the maximum correlation value, indicative of a ridge correlation maximum.

2. Apparatus according to claim 1, in which the ridge detecting means comprises means for detecting whether more than a predetermined number of correlation values are within the threshold amount of the maximum correlation value.

3. Apparatus according to claim 2, in which the ridge detecting means comprises means for detecting whether fewer than a second predetermined number of correlation values are within the threshold amount of the maximum correlation value, the second predetermined number being greater than the firstmentioned predetermined number.

4. Apparatus according to any one of the preceding claims, comprising means, responsive to a detection of a ridge correlation maximum, for generating an average motion vector dependent on the positions in the correlation surface of the two most widely spaced correlation values within the threshold amount of the maximum correlation value.

5. Apparatus according to claim 1, in which the ridge detecting means comprises: means for detecting a region of the correlation surface comprising those correlation values within the threshold amount of the maximum correlation value; means for detecting the ratio of the length of the region in a direction indicative of vertical image motion and the length of the region in a direction indicative of horizontal image motion; means for comparing the ratio with a first predetermined value to detect a ridge correlation maximum in the direction indicative of horizontal image motion; and means for comparing the ratio with a second predetermined value to detect a ridge correlation maximum in the direction indicative of vertical image motion.

6. Apparatus according to any one of the preceding claims, comprising means for setting a ridge flag indicative of a ridge correlation maximum.

7. Apparatus according to claim 6, comprising means for generating a ridge flag indicative of whether the ridge correlation maximum lies in the direction indicative of horizontal image motion or in the direction indicative of vertical image motion.

8. Apparatus according to any one of claims 5 to 7, comprising: means for comparing a motion vector under test, derived from a correlation surface in which a ridge correlation maximum was detected, with a further motion vector generated from the pair of input images; means for detecting whether corresponding components of the further motion vector and the motion vector under test are substantially identical; means for detecting whether the further motion vector was derived from a correlation surface in which a ridge correlation maximum was detected; and means, responsive to a detection that the further motion vector was not derived from a correlation surface in which a ridge correlation maximum was detected, for replacing a component of the motion vector under test in the ridge direction by the corresponding component of the further vector.

9. Apparatus according to claim 8, in which the further motion vector is derived from a correlation surface displaced, in the ridge direction1 from the motion vector under test.

10. Apparatus according to claim 9, in which the further motion vector is derived from a correlation surface longitudinally adjacent, in the ridge direction, to the motion vector under test.

11. Apparatus according to claim 9, in which the further motion vector is derived from a correlation surface diagonally adjacent, in the ridge direction, to the motion vector under test.

12. Apparatus according to any one of the preceding claims, in which the threshold value is dependent upon the mean of the correlation values in the correlation surface.

13. Apparatus according to any one of the preceding claims, in which the correlation surface comprises an array of correlation values representing the difference between the content of the search block and the search area.

14. Apparatus according to claim 13, in which the correlation values represent the difference between the luminance content of the search block and the search area.

15. A motion compensated television standards converter comprising apparatus according to any one of the preceding claims.

16. A method of motion compensated video signal processing in which motion vectors are generated to represent image motion between a pair of input images of an input video signal, the method comprising the steps of: comparing a search block within one of the pair of input images with a search area, comprising a plurality of blocks, in the other of the pair of input images, to generate a correlation surface, the correlation surface comprising an array of correlation values representing correlation between the search block and the search area; detecting a maximum correlation value in the correlation surface; generating a motion vector in dependence on the position of the maximum correlation value in the correlation surface; and detecting correlation values in the correlation surface within a threshold amount of the maximum correlation value, indicative of a ridge correlation maximum.

17. Motion compensated video signal processing apparatus substantially as hereinbefore described with reference to the accompanying drawings.

18. A method of motion compensated video signal processing, the method being substantially as hereinbefore described with reference to the accompanying drawings.

19. A motion compensated television standards converter substantially as hereinbefore described with reference to the accompanying drawings.