GB2316260A - Detecting motion vectors in a digitised video signal - Google Patents

Abstract

A method and apparatus for detecting the direction and magnitude of motion of a moving object in a television picture by detecting motion vectors in a digitised video signal representative of said picture. Detection is achieved by detecting shifts of areas at which motion exists by using signals representative of fields before and after the field to be detected. According to the shifts of these detected areas, an initial vector Voa, Vob closest to the true motion is selected among prospective initial vectors produced at an initial vector selector 3x, 3y or an initial vector detector by a block matching method, thereby reducing detection errors in motion vectors Vx, Vy occurring at the time of detection of the motion vectors, thereby improving the accuracy of motion vectors Vx, Vy so that image distortion is minimised in an interpolating process.

Description

2316260 "MOTION VECTOR DETECTING METHOD AND MOTION VECTOR DETECTING

APPARATUS" This invention relates to a method and an apparatus for detecting the direction and magnitude of motion of a moving object, or namely, for detecting motion vectors, in a digitised video -signal. This patent application has been divided out of our copending application 9418737.4 (2282021).

Television systems make use of image telecommunications for transmitting visual information such as scenes and pictures to remote viewers using optical and electrical means. In fundamental terms, the television system converts, on its transmitter side, a three dimensional image into a two dimensional (plain) image by means of a lens, transmits optical energy of respective parts of the image in the form of a one dimensional electrical signal, and composes it again on the system's receiving side as a two dimensional image to form a picture. In television, plural pictures are consecutively displayed, thereby reproducing the movement of objects in the picture. Each of the plural pictures being continuously transmitted is referred to as a frame.

The image and the picture composed from the image are composed of an assembly of tiny elements having a different luminance from one another. In a television system, the picture is divided into matrices of tiny elements on the transmitter side, and then, the luminance of each element in the picture is converted into an electrical signal according to a fixed order from left to right and from top to bottom and is transmitted to the receiving side. The electric signal is sequentially composed on the receiving side as pictures according to its order as sent from the transmitting side. Such regular decomposing or composing of the picture is referred to as scanning.

Scanning may be sequential scanning, non-interlaced scanning, or interlaced scanning. In non-interlaced scanning, one picture is completed by scanning sequentially from the top to the bottom of the picture. In interlaced scanning, scanning of a pictue is completed by scanning twice, I namely, by first scanning the picture with space from the top to the bottom and next scanning the middle area over which the first scanning did not pass. Thus, interlaced scanning can be said to form one fine picture, or frame, by substantially superimposing plural coarse pictures. Each of the plural coarse pictures is referred to as a field. Interlaced scanning can reduce the CFR (critical flicker frequency) of a picture without impairing its resolution. Therefore, most television cameras use interlaced scanning these days.

A motion vector indicates the magnitude and direction of a moving object in a scene. The motion vector is used for such as interframe coding during high efficiency coding of a television signal and field interpolation for field number conversion, which is a required process for television standard conversions. A method, generally called a pattern matching method, in which motion vectors are detected using the similarity of signal patterns between frames, is disclosed in Japanese Unexamined Patent Publication Nos. Showa 55-162,683 and Showa 55- 162,684. Inaddition,a method, generally called the iterative gradient method, in which motion vectors are detected from physical correspondence of a signal gradient in a frame or an interframe signal difference value, is disclosed in Japanese Unexamined Patent Publication No. Showa 60 - 158,786. An iterative gradient method using initial vectors improves the detection accuracy of the motion vectors, and is disclosed in Japanese Unexamined Patent Publication Nos. Showa 62 - 206,980 and Heisei 4 - 78,286.

However, even with a method of detecting motion vectors by calculating shift vectors based on a detected motion vector such as an initial vector and rendering the summation of the initial vector and the shift vector as a true motion vector, detection errors may occur due to the initial vectors' inability to respond to a sudden change of the motion vector when the object suddenly moves from a still state, or when it suddenly stops from a moving state. In particular, with a method having low detection accuracy of the motion vectors, likewise the gradient method, sudden changes of the motion vectors induce detection errors.

For example, as shown in Figure 1 (A), where an object M which had been stationary in the picture of the previous field travels in the present field, a background image of the moved object M appears at the hatched area in Figure 1 (B). If the background image is a still picture or has no change between two successive fields, the motion vector for this area must be null. However, the detection result of the motion vector cannot in fact be null. As a result, image distortion may occur in an interpolated image in this area. Although the prospective initial vectors of a block rril showing the object M1 are produced using the detected motion vector, ail the prospective initial vectors become null because the image in Figure 1 (A) is the still picture one. Using the gradient method for such a situation, since only an area at which the image gradient exists allows the operation, detection errors may occur upon detection of motion vectors for the block m 1 in Figure 1 (B).

Where a moving object m2 from the previous field shown in Figure 2(A) to the present field shown in Figure 2(13) in the picture is tiny and moves by a large amount, detection errors of the motion vector may in principle occur in the gradient method. By contrast, where the background is a motion picture, or has changes between two successive fields, and a nearly still object of small area exists therein, detection errors of the motion vectors occur similarly. For example, where a TV camera takes pictures of a volley ball game and when the camera is panned along with the motion of the moving volley ball, the background moves whereas the volley ball itself produces a still image. If the movement of the background is considerable, it affects the data necessary when the still image of the volley ball is corrected using the motion vectors, resulting in image distortion.

Figure 3 shows this example, in which a small ball A is viewed as a still picture portion and a background B moves with a velocity V. Where the ball size is almost equal to a block size for motion vector detection (in Figure 3, a block size is set to 8 pixels x 8 lines), all the initial vectors in the block A become a velocity V because the a 1 read y-detected motion vector is used. Consequently, a shift vector of magnitude -V must be generated to make the block A still, or in particular, to make the motion vector null.

However, if the magnitude V is sufficiently great, the shift vector of -V does not tend to be generated where, likewise with the gradient method, the motion vectors are detected with low accuracy, so that the motion vector of A does not become null.

Furthermore, regarding an image having a small vertical correlation likewise a crosshatching pattern, a problem arises in which detection errors easily occur when a vertical motion vector Vy is detected. Moreover, where, likewise in TV standard converting apparatus, an interpolated image is produced using detected motion vectors, the detection errors of such motion vectors produce image distortion.

The present invention seeks to reduce detection errors of motion vectors occurring when detecting the motion vectors and to improve the accuracy of such motion vectors.

In accordance with a first aspect of the invention, there is provided a method of detecting motion vectors indicative of the magnitude and direction of a moving object in a television picture, said method comprising the steps of dividing a digitised video signal representative of said picture into a plurality of blocks, producing a plurality of prospective initial vectors from previously detected motion vectors in respect of said blocks, in order to detect the motion vector in respect of each of said blocks between signals located at least one field away from one another, seeking an optimum shift vector as an initial vector, and carrying out a summation of said initial vector and said shift vector to produce.the motion vector of said block, wherein:

a) a motion vector is produced by independently detecting the motion vectors with respect to each block; and b) said produced motion vector is used as one of said prospective initial vectors.

1 In accordance with a second aspect of the invention, there is provided apparatus for detecting motion vectors indicative of the magnitude and direction of a moving object in a television picture, said apparatus comprising means for dividing a digitised video signal representative of said picture into a plurality of blocks, means for producing a plurality of prospective initial vectors from previously detected motion vectors in respect of said blocks in order to detect the motion vector in respect of each of said blocks between signals located at least one field away from one another, means for seeking an optimum shift vector as an initial vector, means for summing said initial vector and said shift vector to produce the motion vector of said block, a first prospective initial vector generator for producing a plurality of prospective initial vectors from the previously detected motion vector for the block at which a motion vector is to be detected, a second prospective initial vector generator for producing at least one prospective initial vector by independently detecting the motion vector with respect to each block, and an initial vector selector for selecting, as the initial vector, the optimum vector from among the prospective initial vectors produced at said first and second prospective initial vector generators.

The above and other objects and features of the invention are apparent to those skilled in the art from the following preferred embodiments thereof when considered in conjunction with the accompanying drawings, in which:

Figure 1 is a diagram illustrating changes from a still area to a motion area of an image displayed in a picture; Figure 2 is a diagram illustrating a small area displayed in a picture at a time that the area moves by a large amount; Figure 3 is a diagram used to illustrate problems in the detection of motion vectors of a nearly still small area when the still area exists within a moving background;

Figure 4 is a block diagram illustrating a first embodiment of the invention; Figure 5 is a diagram showing a correspondence of blocks in previous and present fields;

Figure 6 is a diagram illustrating motion vectors; Figure 7 is a diagram showing types of prospective initial vectors; Figure 8 is a block diagram illustrating a second embodiment of the invention; Figure 9 is a block diagram illustrating a third embodiment of the invention; Figure 10 is a block diagram illustrating an initial vector selector, as shown in Figure 9; Figure 11 is a block diagram illustrating a fourth embodiment of the invention; Figure 12 is a diagram showing a searching range and number of blocks for a block matching method at the time of selection of initial vectors VO; Figure 13 is a diagram for describing the operation of the motion area detector shown in Figure 11; Figure 14 is a block diagram illustrating a fifth embodiment of the invention; Figure 15 is a diagram illustrating a motion detecting method by a block matching method; and Figure 16 is a block diagram illustrating an initial vector as shown in Figure 14.

Referring to the drawings in detail, in particular to Figure 4, a detecting method and apparatus of motion vectors according to a preferred embodiment of the invention is shown. This embodiment has the object of reducing detection errors occurring when vertical motion vectors change greatly at a time that the motion vectors are detected from interlaced pictures.

Where the motion vectors are detected from interlaced pictures, detection errors tend to occur at the time of detection of a vertical motion vector Vy for an image, likewise a crosshatching pattern, which has a small vertical correlation, because the motion vectors are detected by comparison of signals between fields. To solve this problem, the motion vectors may be detected between frames. However, the motion vectors of such interframe detection would be inaccurate in comparison with interfield detection due to time expansion of the motion. Vertical motion vectors Vy would be erroneous due to inaccuracy of the motion vectors.

In the first embodiment, motion vectors are detected by the iterative gradient method, using initial vectors. Before the start of the description for the detailed constitution of the embodiment, referring to Figures 5 to 7, there is described a method for selecting an initial vector and a method for seeking a true motion vector using the initial vector.

The detecting method of motion vectors according to this embodiment fractionises one field or one frame of a television signal into block units composed of m-pixels in a horizontal direction and n- lines in a vertical direction (m and n are arbitrary natural numbers), and detects motion vectors with those blocks, each of which is as one unit. This detection based on block units is performed sequentially or in parallel in the order from left to right and from top to bottom, as well as the television signal.

As shown in Figure 5, when a true motion vector is sought for a block to be detected, or a target block (ml, ril), the optimum motion vector is selected among the motion vectors previously detected, and the selected motion vector is rendered as an initial vector VO {= (aO, PO)}. Then, using the iterative gradient method, a motion shift vector V1 {= (ocl, P1)} is sought based on the block (ml + aO, ril + PO) at which the coordinates are shifted by a magnitude of the initial vector and based on the target block. A true motion vector V {= VO + V1} for the target block is sought by adding the initial vector VO and the motion shift vector V1 as shown in Figure 6. It is to be noted that a pattern matching method or the like can be used as well as the iterative gradient method described above to detect the motion shift vector V1.

The detected motion vector is used for the selection of the initial vector VO as described above, but in order to improve the accuracy of the initial vector VO, it is advantageous to select the optimum motion vector from a plurality of previously detected vectors. However, because of the relation with a circuit scale, a simulation result provided by a computer, and whatever, a motion vector selecting method selecting out of six kinds of motion vectors as described below is used in this embodiment. These six kinds of motion vectors are referred to as prospective initial vectors, and they will be described in detail with reference to Figure 7.

As shown in Figure 7, the six motion vectors as cited below 1) to 6) are used as the prospective initial vectors in order to seek the motion vector of the target block, which is shown by means of hatching in the case of the present field.

1) A motion vector VA for a block located immediately above and in the present field as well as the target block.

2) A motion vector V13 for a block located on the fight upper side of and in the present field as well as the target block.

3) A motion vector Vc for a block located on the left side of an in the present field as well as the target block. (Though the Vc, located on the immediate left side would be preferable for one of the prospective initial vectors, the Vc located one block away is used in the light of the circuit constitution and operation time.) 4) A motion vector VN for a block located immediately below the block in the previous field corresponding to the target block.

5) An average vector VE {:: (VG + W + V] + VJ + VK + VL + VM + VN) 18} representing the average of the motion vectors of the block located at the same position in the previous field as the target block and of its surrounding blocks.

6) An acceleration veCtor V. {' VE + (VE-VP)} representing the 1 vector's shift based on the average vector VE and the average vector Vp two fields before.

Based on these prospective initial vectors, six kinds of field signal, in which the co-ordinates of the target block are shifted, are produced. With respect to each of the six kinds of field signals, the absolute values of the difference between the field signal and a signal located one field or one frame away, are sought and are accumulated as much as the number of pixels in the target block. The prospective initial vector whose accumulated value is the minimum is selected among them to render itself as the optimum initial vector VO. In this embodiment, the pixels in the picture are fractionised into blocks of eight pixels and eight lines, and then the motion vector is detected with respect to each block.

A luminance signal S1 of the present field is passed from an input terminal l a to a two dimensional low pass filter (hereinafter abbreviated as "LPF") 2a; a luminance signal S2 of the previous field is passed from an input terminal l b to a two dimensional LPF 2b; a luminance signal S3 of the field before the previous field is passed from an input terminal 1 c to a two dimensional LPF 2c. The two dimensional LPFs 2a, 2b, 2c remove noise and high frequency components from the luminance signals S1, S2, S3, respectively. Among the luminance signals S1, S2, S3 respectively processed through the two dimensional LPFs 2a, 2b, 2c, the luminance signal S1 is fed to initial vector selectors 3x, 3y and shift vector detectors 4x, 4y; the luminance signal S2 is fed to the initial vector selector 3x and the shift vector detector 4x; the luminance signal S3 is fed to the initial vector selector 3y and the shift vector detector 4y. A motion vector memory 5 stores the plural detected motion vectors of blocks adjacent to the target block and produces the prospective initial vectors based on the motion vectors. The memory 5 delivers the plural produced prospective initial vectors to the initial vector selectors 3x, 3y.

The initial vector selector 3x shifts, based on each of the prospective initial vectors provided from the motion vector memory 5, the coordinates of the block, respectively, and calculates the absolute value of the interfield difference in value between the present field signal S1 and the previous field signal S2. The selector 3x renders the prospective initial vector whose accumulated absolute difference values becomes the smallest within the block as the optimum initial vector V0a. The shift vector detector 4x detects a motion shift vector V1 a using the iterative gradient method from the supplied initial vector V0a, the present field luminance signal S1, and the previous field luminance signal S2. An adder 6x finally sums the initial vector V0a and the detected motion shift vector V1 a and yields a horizontal motion vector W.

The initial shift vector selector 3y shifts, based on the prospective initial vectors provided from the motion vector memory 5, the coordinates of the blocks, respectively, and calculates the absolute difference in value between the present field luminance signal S1 and the luminance signal S3 of the field before the previous field, or in other words, between one field and another field located two fields away or interframe. The selector 3y renders the prospective initial vector whose accumulated absolute value becomes the smallest within the block as the optimum initial vector V0b. The shift vector detector 4y detects a motion shift vector V1 b using the iterative gradient method from the supplied initial vector V0b, the present field luminance signal S1, and the luminance signal S3 of the field before the previous field. An adder 6y sums the initial vector V0b and the detected motion shift vector V1 b and yields a vertical motion vector Vy. From those horizontal and vertical motion vectors Vx, Vy separately obtained, a true motion vector V {= (Vx, Vy)} is found and stored in the motion vector memory 5.

Figure 8 is a block diagram showing a second embodiment according to the invention; the same numerals are given to the same portion as those of the first embodiment, and descriptions are omitted.

This embodiment, as well as the first embodiment, has the object of reducing detection errors occurring when vertical motion vectors change 1 greatly at a time that the motion vectors are detected from interlaced pictures.

A comparator 8 compares the magnitude of the vertical motion vector of the plural prospective initial vectors supplied from the motion vector memory 5, with a predetermined threshold T1. The threshold T1 can be arbitrarily set in accordance with the specification of the detecting apparatus, images of the objects to be detected, or whatever. In this embodiment, the threshold T1 is set to, for instance, two lines per field. When the magnitude of the prospective initial vector is equal to or less than the threshold T1, the initial shift vector selector 3y for vertical direction detects a vertical motion vector Vy using the signals of the present field and the field before the previous field. When the magnitude of the prospective initial vector is greater than the threshold T1, the initial shift vector selector 3y for vertical direction detects a vertical motion vector Vy using the signals of the present field and the previous field. A switching circuit 9 performs such switching.

The reason why the second embodiment does thus process for the vertical motion vectors is as follows. When the motion vector is large the detection accuracy would be improved by use of interfield luminance signals rather than interframe luminance signals since the motion area would become small. In contrast, when the motion vector is small the detection accuracy would be improved by use of interframe signals rather than interfield signals, in an interlaced television signal, because the detection accuracy of the motion vectors where the iterative gradient method is used remains at a couple of pixels or lines per one operation of the gradient method.

Although in the second embodiment an initial vector is selected based on the magnitude of the vertical motion vector of the prospective initial vector, it is also possible to select, where vertical motion vectors are detected using an interfield signal and an interframe signal, respectively, either of the motion vectors, depending on whether the magnitude of the

I detected motion vector is greater or less than the predetermined threshold.

At that time, although the motion vector to be compared with the threshold can be workable even either from the interfield signal or from the interframe signal, it is preferable to use the vertical motion vector using the interframe signal, because its detection accuracy will be higher around the threshold.

In the first and second embodiments, detection errors in the vertical motion vectors will be reduced during detection of the motion vectors in the interlaced television signal. Where either of these embodiments is used for an apparatus capable of an interpolation process using such motion vectors, an interpolated image would therefore be produced without any image distortion.

Where the detected motion vectors located around the target block are used as the prospective initial vectors for selection of the initial vector VO, it is presumed that there is the following relation between the selection of prospective initial vectors and the occurrence of detection errors. That is, at the area at which the picture changes from a motion picture to a still one, the correlation is not so strong between the detected motion vector of the previous field and the motion vector to be detected of the present field.

The detection errors would occur less by selection of the prospective initial vectors based on the null motion vector or the detected motion vector of the present field than based on the detected motion vector of the previous field.

This phenomena has been confirmed by experiments conducted by the inventor of this invention. Therefore, errors in the motion vectors would be reduced at the area at which the object suddenly changes from a state of motion to a stationary state, if the motion vectors detected in the previous field such as, particularly, the average vector VE and the acceleration vector VG, among the prospective initial vectors for selection of the initial vector VO, are made to exclude from the prospective initial vectors, or if they are not easily selected.

A third embodiment is constituted based on the above consideration.

That is, the third embodiment reduces detection errors in the motion vectors by providing a priority order for selection of the initial vector VO based on changes, in a time scale, of the summation of the absolute values of the interfield or interframe differences of the target blocks, or namely, based on transitions of the still picture areas and the motion picture areas.

The third embodiment can be used for apparatus having the ability to detect motion vectors such as high efficiency coding apparatus or television standard conversion apparatus. In this embodiment, the prospective initial vectors of six kinds described in the first embodiment and the null motion vector, total seven kinds, are used as the prospective initial vectors for selection of the initial vector VO; an iterative gradient method is used for detection of the motion vectors.

The apparatus for implementing the motion vector detecting method is comprised, as shown in a block diagram in Figure 9, of input terminals 1, 1 b, an initial vector selector 3, a shift vector detector 4, a motion vector memory 5, and an adder 6. The luminance signal S1 of the present field is delivered from the input terminal 1 a to the initial vector selector 3 and the shift vector detector 4. The luminance signal S2 of the previous field is also delivered from the input terminal 1 b to the initial vector selector 3 and the shift vector detector 4. The motion vector memory 5 is composed of a memory portion for storing the motion vectors of the respective blocks of the present field and a memory portion for storing the motion vectors of the respective blocks of the previous field (one field before). The initial vector selector 3 selects as the optimum initial vector VO from the prospective initial vectors produced based on the detected motion vectors stored in the motion vector memory 5. The constitution and operation of the initial vector selector 3 will be separately described with reference to Figure 10.

The shift vector detector 4 detects the motion shift vector V1 using the iterative gradient method from the supplied initial vector VO, the present field luminance signal S1, and the previous field luminance signal S2. The adder 6 adds the supplied initial vector VO with the motion shift vector V1 and detects a true motion vector V. The obtained true motion vector V is 1 further stored at the detected motion vector in the motion vector memory 5 in order to produce another set of prospective initial vectors with respect to the next target block in the present field or the target block in the successive field (one field after).

Figure 10 is a block diagram showing the detailed construction of the initial vector selector 3 of Figure 9. The luminance signals S1, S2 of the present and previous fields supplied to the initial vector selector 3 pass through the two dimensional LPFs, respectively, and are suspecter interlaced signals by a temporal filter, gravity centre correction, or the like.

Such filters are omitted from Figures 9 and 10. The luminance signals S1, S2 are supplied to a motion detector 12. The motion detector 12 is a circuit for detecting motion in block units, and operates on the summation of the absolute values of the difference signals of the present field and the previous field, and, if its result is equal to or less than the threshold T2, delivers a logical high level, or M", to indicate that the block is in a still condition. The threshold T2 is arbitrarily set in accordance with the specification of the detecting apparatus, the image ofthe object to be detected, or the like, and in this embodiment, it is set to, for example, 120, since it is detected on the basis of block units of eight pixels and eight lines.

The output of the motion detector 12 is supplied to a motion area detector 13 by three paths: directly; through a one field delay circuit 10a to delay it for one field; and through a pair of one field delay circuits 1 Oa, 1 Ob to delay it for two fields. The motion area detector 13 is a circuit to detect transition areas from the motion picture to the still picture as shown as the hatched portion in Figure 1 (B). For example, regarding the output results of the motion detector 12 supplied to the motion area detector 13, the direct input is assigned as "a"; the input with one field delay is assigned as "b"; and the input with two fields delay is assigned as "c". If (a, b, c) = (1, 0, 0), it indicates that the motion picture changes to a still picture. The reason why detection of the transition area is based on the output results of three fields, as in this embodiment, is to improve detection accuracy by observing

1 changes of the output results "a" and "c" located before and after the output result 'W' as a centre. For example, when the detection result is (a, b, c) (0, 1, 0), it is judged that the change from the motion picture to the still picture detected at the field of "b" was mistakenly detected.

A prospective initial vector judgement circuit 17 for selection of the initial vector VO supplies a null motion vector to memories 18a to 18f, in lieu of the average vector VE and the acceleration vector Vg of the previous field among the plural prospective initial vectors, in respect to blocks judged by the motion area detector 13 as the transition from the motion picture to the still picture. The motion vector of the block located on the right and lower side of the target block can be supplied even among the motion vectors of the previous field. Even if so, when the direction of the motion vector is from left to right or from top to bottom, a logical low level M" is supplied. In addition, the same condition is possible for the surrounding blocks of the target block when the motion vectors detected from the previous field are not used. The luminance signals S1, S2 are separately supplied, respectively, to delay circuits 11 a, 11 b. The delay circuits 11 a, 11 b delay the luminance signals S1, S2 for one field +(x, respectively. The (X is an amount to compensate for the delay amount of the operation times at the motion detector 12, the one field delay circuits 10a, 10b, and the motion area detector 13. The luminance signal S1 delayed for one field +a, due to passing through the delay circuit 11 a, is supplied to a transpose memory for line to block conversion. This transpose memory 15 converts the scanned signal to block units of m multiplied by n to be read out and delivered.

The luminance signal S2 delayed for one field +a, due to passing through the delay circuit 11 b, is supplied respectively to memories 18a to 18f. The memories 18a to 18f are memories for line to block conversion, for shift of the block's co-ordinates according to the prospective initial vectors of the six kinds, and for reading them out. The outputs of the memories 18a to 18f are respectively fed to corresponding accumulators 20a to 20f functioned with an absolute value conversion, after being subtracted from the output of the memory 15 in corresponding subtractors 19a to 19f. The accumulators 20a to 20f accumulates with conversions of the absolute values; the accumulated results are supplied to a prospective initial vector selector 21. The prospective initial vector selector 21 delivers as the initial vector VO a prospective initial vector which gives the smallest value among the accumulated values.

It is also possible to add a fixed value P according to the results of the motion area detector 13 to the outputs of the corresponding accumulators, after the summation of the absolute interfield differences in use of the motion vectors of the previous field, in lieu of the prospective initial vector judgement circuit 17 in Figure 10 used in this embodiment. As a result, the sum of the accumulated results of the average vector VE and the acceleration vector Vg of the previous field become large, so that they are not easily selected at the prospective initial vector selector 21. Otherwise, the prospective initial vector judgement circuit 17 in Figure 10 could be made to flexibly judge to a certain extent in respect to the switching thereof. Moreover, it is also possible that the motion detection result which is detected with respect to each pixel in the target block is used in lieu of the summation of the absolute difference values to judge from the accumulated values. Although in this embodiment the prospective initial vectors of the six kinds described in the first embodiment and the null motion vector, seven kinds in total, are used as the prospective initial vectors and the iterative gradient method is used to detect the motion vectors, it is also possible to use other prospective initial vectors and detecting methods. By constituting as such the embodiment thus described, detection errors will be reduced at the transition area from the still picture to the motion picture where detection errors in the motion vectors conventionally occur, so that image distortion will be reduced in television standard conversion apparatus or the like which performs interpolation processing in use of the motion vectors.

I Figure 11 is a block circuit diagram showing a fourth embodiment of the invention. In this embodiment, also, the prospective initial vectors of the six kinds described in the first embodiment and the null motion vector, seven kinds in total, are used as the prospective initial vectors for selection of the initial vector VO. Furthermore, as shown in Figure 12, regarding the target block of eight pixels and eight lines in hatching, total forty five blocks given by shifting every four pixels and every four lines over the range of 20 lines in the vertical direction and 36 pixels in the horizontal direction, are used as reference blocks, and the embodiment is thereby constituted so as to select initial vector VO by a block matching method. The reference blocks for the block matching method can be given by shifting every single pixel and line. However, for example, where the reference blocks are to be made over the range of 20 lines in the vertical direction and 36 pixels in the horizontal direction, the method requires 720 blocks in total and thus involves a large amount of hardware. Moreover, since the accuracy of the prospective initial vectors is modified at the succeeding process even if it is coarse more or less, the block is set to every four pixels and every four lines in this embodiment.

For detection of the motion vectors using the construction shown in the block diagram in Figure 11, items such as the two dimensional filter, the temporal filter, or the like, are separately necessary for the purpose of preprocessing prior to detection. However, their descriptions are omitted since they have no direct relation with the essential features of this embodiment. The interlaced luminance signal S supplied from the input terminal 1 is supplied to the motion detector 12 through three routes. The first route is directly to the motion detector 12. The second route is through a delay circuit 1 Oc to the motion detector 12 in order to delay the luminance signal S2 by one field. The third route is through the delay circuit 1 Oc and a delay circuit 1 Od to the motion detector 12 in order to delay the luminance signal S3 by two fields. The motion detector 12 detects motion using the luminance signals S1, S3 located two fields away, or namely one frame away from one another. The motion detector 12 detects motion basically by the difference values between frames, and its description is herein omitted since it has the same constitution as the third embodiment. The motion signal detected by the motion detector 12 is supplied to the motion area detector 13 by three paths: directly; through a delay circuit 1 Oe for one field delay; or through the delay circuit 1 Oe and a delay circuit 1 Of for two fields delay. The luminance signal S1 is thus supplied to the motion detector 12 respectively by three routes.

Using the diagrams shown in Figures 13 (A) to 13 (F), the steps for area judgement at the motion area detector 13 will be described. Figure 13 (A) is a picture based on the input luminance signal S1 supplied to the input terminal 1 in Figure 11, and it is assumed that in the picture an object M1 has been travelling over the background as a still picture. Figure 13 (B) is a picture at that time based on an output luminance signal S2 of the delay circuit 1 Oc and is the picture of the previous field with respect to the picture of the luminance signal S1; Figure 13 (C) is a picture based on an output luminance signal S3 of the delay circuit 1 Od in Figure 11 and is the picture of two fields prior to the picture of the luminance signal S1. The motion area is detected by the motion detector 12 based on the differences of these pictures. Figure 13 (D) is a picture based on an output signal of the motion detector 12; the hatched area represents the detected motion area. Figure 13 (E) is a picture based on an output signal of the delay circuit 10e and shows the motion area of one field before. Figure13(17)is a picture based on an output signal of the delay circuit 1 Of and shows the motion area of two fields before. The judgement for the motion transition area is made by the motion area detector 13 based on the motion areas sought in Figures 13 (D) to 13 (F); Figure 13 (G) indicates a motion transition area. An area al changes from the still picture to the motion picture; an area bl remains as the portion of the motion picture; an area cl changes from the motion picture to the still picture; and an area dl remains as the portion of the still picture. That is, the area al is detectable from 1 Figure 13 (D) and Figure 13 (E); the area cl is detectable from Figure 13 (E) and Figure 13 (F). This detection result is fed to the initial vector selector 3a.

The luminance signal S1 supplied to the input terminal 1 is supplied further to the three dimensional LPF 2d directly and through the delay circuitlOc. The three dimensional LPF 2d is composed of a horizontal LPF, a vertical LPF, and a temporal LPF, and reduces the influence of noise and high frequency components, and suppresses detection errors when the motion vectors are detected. The signal delivered from the three dimensional LPF 2d is fed to a one field delay circuit 1 Oh, an initial vector detector 22, a second initial vector selector 3d, and an iterative gradient method operator 4a after being delayed for one field due to passing through a one field delay circuit 10g. The signal delayed for one field due to passing through the one field delay circuit 1 Oh is also fed to the initial vector detector 22, the second initial vector selector 3d, and the iterative gradient method operator 4a.

The initial vector detector 22 detects initial vectors by the block matching method using the signals separated one field away from one another. Now, the detection of the initial vectors using the block matching method, is described in detail. The absolute values of the differences between the target block in the present field and each of blocks in the previous field, total 9 x 5 = 45 pieces, located in respective positional relations as shown in the diagram in Figure 12 with respect to the block in the previous field located at the same position as the target block in the present field, are sought, and the block whose summation is the smallest is selected as a first initial vector. The number of blocks used for the block matching can be less than the case shown in Figure 12 and conversely can be greater than it. The number of blocks can be determined from the scale of the hardware and the precision of the motion vectors. For example, by assigning eight pixels and eight lines as one block by shifting every eight pixels and every eight lines, blocks of 15 pieces {= 5 x 31 are possible, but I the accuracy of the motion vector becomes at least eight pixels and eight lines in such a case. Detection of the initial vectors at the initial vector detector 22 can be by another method than the block matching method, and a phase detection method using FFT (Fast Fourier Transforms) disclosed in, for example, "G.A. Thomas: Motion Estimation and Its Application to HDTV, SMPTEJ pg 987-992, Dec. 1990" or the like, is applicable.

The first initial vector detected by the initial vector detector 22 is supplied to the first initial vector selector 3a. Three data signals, namely, the motion transition area signal detected at the motion area detector 13, the initial vector detected at the initial vector detector 22, and the second initial vector selected at the second initial vector selector 3b are supplied to the first initial vector selector 3a. The second initial vector selector 3b is to select the optimum as the second initial vector among the initial shift prospective vectors of six kinds produced based on the detected motion vectors stored in the motion vector memory 5. Its constitution is almost the same as that of the initial vector selector 3 in other embodiments described above, so its detailed description is omitted.

In accordance with the area signal detected at the motion area detector 13, the first initial vector selector 3 performs the following selection:

1) Motion picture area -> Still picture area: a null motion vector is set to the initial vector VO.

2) Motion picture area -> Motion picture area: the second prospective initial vector based on the detected motion vectors is set to the initial shift vector VO.

3) Still picture area -> Motion picture area: the first initial vector detected by the block matching is set to the initial vector VO.

4) Still picture area -> Still picture area: a null motion vector is set to the initial vector VO.

When the transition area is detected with respect to each pixel, it is required to convert it into block units to produce area signals. In this embodiment, as an example, the area signals are counted, and the area signal having the largest number in the block is set as the area signal of the block. For example, if the pixel of the area signal indicating the still area signal is the largest among the sixty four pixels existing in a block composed of eight pixels by eight lines, the block is assigned as the still area.

After the initial vector VO is thus detected, the shift vector V1 is sought by the iterative gradient method, and a true motion vector V is sought. Detailed description is omitted since this would be almost the same as that of the third embodiment. This embodiment shows a motion vector detecting method in which the motion vectors detected by the block matching method and the motion vectors based on the already detected motion vectors are selected to be used based on the area signal indicating transitions of motion. In addition to such constitution, the motion vector detected by the block matching method can be added to one of the prospective initial vectors of the initial vector selector in use of the already detected motion vectors, and the priority order for selection can be controlled in accordance with the area signal.

Thus this embodiment is operable to detect independently the motion vector with respect to each of the blocks at an area at which the correlation with the detected motion vectors is scarce while using the detected motion vectors as much as possible. When the detected motion vectors are used, it is desirable to use the motion vectors detected in the field having the same positional relation. That is, in the case of interlaced scanning, it is desirable to use the motion vectors detected two fields before, or one frame before, rather than one field before. Use of the embodiment can improve the detection accuracy of the motion vectors at the area at which motion changes suddenly, and can significantly reduce enhancements of the circuit scale in comparison with where the block matching method is entirely used. This effect becomes more apparent as I the magnitude of the motion vector to be detected becomes larger.

Figure 14 is a block diagram showing a fifth embodiment of the invention. In the fifth embodiment, it is an object to reduce detection errors of the motion vectors even when the background moves by a large amount and an object of small area exists in a nearly still condition. To accomplish this object, the embodiment is such as to use not only the motion vectors produced from the detected motion vectors but also motion vectors detected and produced independently from the motion vector of each block, as one of the prospective initial vectors for selection of the initial vector.

The fifth embodiment is also to detect the motion vector by the iterative gradient method using the prospective initial vectors and has constitution in which the prospective initial vectors of the six kinds described in the first embodiment and the motion vector sought for each block using a block matching method, namely, prospective initial vectors of seven kinds in total, are used as prospective initial vectors for selection of the initial vector VO.

An apparatus to implement the motion vector detecting method, as shown in Figure 14, comprises input terminals 1 a, 1 b, in initial vector selector 3c, a shift vector detector 4, a prospective initial vector generator 23 using a block matching method, a motion vector memory 5, and an adder 6. Hereinafter, the motion vector detecting method using this apparatus is described. The shift vector detector 4, the motion vector memory 5, and the adder 6 have the same purpose as those of the other embodiments, and descriptions are accordingly omitted.

The luminance signal S1 of the present field is fed from the input terminal 1 a to the initial vector selector 3c, the prospective initial vector generator 23 and the shift vector detector 4. The luminance signal S2 of the previous field is also fed from the input terminal 1 b to the initial vector selector 3c, the prospective initial vector generator 23 and the shift vector detector 4. The prospective initial vector generator 23 is a circuit to independently detect the motion vectors with respect to each block and to produce the prospective initial vectors VBr. As shown in Figure 15, the detecting method for motion vectors is to use as reference blocks a total of 81 blocks of the previous field given by shifting the block of 8 pixels and 8 lines by every signal pixel and every signal line over a range of 4 lines in a vertical direction and 4 pixels in a horizontal direction in respect to the target block of the present field, to accumulate the absolute values of the differences between those reference blocks and the target lock, and to select the reference block having the least accumulation value. Then, the prospective initial vector VBr is produced from the positional shift between the selected reference block and the target block. An initial transition vector selector 3c produces plural prospective initial vectors based on the detected motion vectors stored in the motion vector memory 5. The selector 3c then selects the optimum initial vector VO among those prospective initial vectors and the prospective initial vectors VBr produced by the prospective initial vector generator 23.

Referring to Figure 16, the selection of the optimum initial vector VO will be described. Figure 16 is a block diagram showing in detail the initial vector selector 3c. The luminance signals S1, S2 of the present and previous fields fed to the initial vector selector 3c pass in common two dimensional LPFs. The signals S1, S2 are signals pseudo-interlaced by means of, for example, a temporal filter of gravity centre corrections. Descriptions of these filters are omitted. The luminance signal S1 is fed to a transpose memory 15 for line to block conversion. The transpose memory 15 for line to block conversion converts the signal scanned in the scanning direction into read-outs of the block units of m multiplied n and delivers them. In contrast, the luminance signal S2 is respectively fed to memories 18a to 18f and 18g of the vector selector 3c. The memories 18a to 18f are memories to perform conversion of line to block, to shift the coordinates of the blocks corresponding to the conventional prospective initial vectors of the six kinds, and to read them out. The memory 18g is a memory to perform conversion of line to block, to shift the co-ordinates of the blocks corresponding to the prospective initial vector VBr obtained from I the prospective initial vector generator 23, and to read them out. The respective outputs of memories 18a to 1 Bg are fed to corresponding accumulators 20a to 20g having the function of absolute value conversion to be accumulated while made to be absolute value, after being subtracted from the outputs of the transpose memory 15 for line to block conversion at corresponding subtractors 19a to 19g. The accumulated results are respectively fed to the initial shift prospective selector 21. The initial shift prospective selector 21 selects the motion vector, as the initial vector VO, of which the smallest value is give among the accumulated values.

In this embodiment, although the reference blocks are produced by shifting every single pixel and every single line at a time of the detection of the prospective initial vectors by the block matching method, it is also possible to make the pitch a little larger such as, for example, two pixels and two lines, since the accuracy of the prospective initial vector is modified at the following process even if more or less coarse. Where thus constituted, the embodiment can reduce the number of reference blocks and ran simplify the processes required for detection. Moreover, it is also possible to use, as the prospective initial vectors for selection of the initial vector VO, other initial shift prospective vectors than the prospective initial vectors of the six kinds described in the first embodiment.

As described above, upon using the invention, detection errors of the motion vectors occurring when the motion vectors are detected would be reduced, and the accuracy of the motion vectors would be improved. Where an apparatus for producing interpolation images using the detected motion vectors, likewise a television standard conversion apparatus, uses the detecting apparatus for motion vectors according to the invention, occurrence of image distortions due to detection errors of the motion vectors would be reduced.

Claims (4)

1 A method of detecting motion vectors indicative of the magnitude and direction of a moving object in a television picture, said method comprising the steps of dividing a digitised video signal representative of said picture into a plurality of blocks, producing a plurality of prospective initial vectors from previously detected motion vectors in respect of said blocks, in order to detect the motion vector in respect of each of said blocks between signals located at least one field away from one another, seeking an optimum shift vector as an initial vector, and carrying out a summation of said initial vector and said shift vector to produce the motion vector of said block, wherein:

a) a motion vector is produced by independently detecting the motion vectors with respect to each block; and b) said produced motion vector is used as one of said prospective initial vectors.

2. A method as claimed in claim 1 wherein a block matching method is used independently for each block to detect the motion vector.

3. Apparatus for detecting motion vectors indicative of the magnitude and direction of a moving object in a television picture, said apparatus comprising means for dividing a digitised video signal representative of said picture into a plurality of blocks, means for producing a plurality of prospective initial vectors from previously detected motion vectors in respect of said blocks in order to detect the motion vector in respect of each of said blocks between signals located at least one field away from one another, means for seeking an optimum shift vector as an initial vector, means for summing said initial vector and said shift vector to produce the motion vector of said block, a first prospective initial vector generator for producing a plurality of prospective initial vectors from the previously detected motion vector for the block at which a motion vector is to be detected, a second prospective initial vector generator for producing at 0 least one prospective initial vector by independently detecting the motion vector with respect to each block, and an initial vector selector for selecting, as the initial vector, the optimum vector from among the prospective initial vectors produced at said first and second prospective initial vector generators.

4. Apparatus as claimed in claim 3, wherein said second prospective initial vector generator produces a motion vector detected from a block matching method as the prospective initial vector.