JPH0322118B2 - - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION The present invention relates to an image motion vector detection method for detecting motion vectors representing motion of image content in images such as television, and in particular, to an image motion vector detection method that detects motion vectors asymptotically. This has been improved so that motion vectors can be detected smoothly even if the motion of the content is discontinuous.

Regarding this type of image motion vector detection method, the present inventor previously proposed a "motion vector detection method" in Japanese Unexamined Patent Publication No. 124927/1983, but the present invention proposes a method for detecting motion vectors for moving images. This is an improved vector detection method.

In other words, the motion vector detection method described above is
A screen such as a television image is divided into a large number of blocks, and the motion vector of each of these blocks is detected by assuming that the movement of the image within that block is extremely close to parallel movement. The detection method is to set a large number of blocks of the same size and slightly shifted positions in the previous frame image for each block in the current frame image, and then A large number of two-dimensional vectors each representing a positional shift with respect to a corresponding block in the current frame image are called shift vectors, and among these many shift vectors, the image most similar to the image in the block in the current frame image is selected. The shift vector for the block showing the maximum correlation is detected, and the shift vector is extracted as a motion vector.

Therefore, in order to judge the degree of similarity between the image of the current frame and the image of the previous frame, it is necessary to compare the images in the image block set in the image of the current frame and the image block set in the image of the previous frame. Correlation with images is calculated, and when the correlation between the images is strong, it is assumed that the images are very similar. To calculate the correlation, for example, A _i is the signal level of the i-th pixel in the image block set in the image of the previous frame, and the signal level of the i-th pixel in the image block set in the image of the current frame is Assuming that the signal level is B _i and the image correlation value between both image blocks is K, the signal level of each corresponding pixel in both image blocks is calculated by the following calculation formula: K = 〓 ⁱ (A _i - B _i ) ² A squared correlation value is obtained by summing the squared difference values for the entire block.

In the conventional image motion vector detection method based on, for example, the square correlation of image signals between images of two sequential frames as described above, the image motion vector is set in the image of the previous frame corresponding to each image block of the current frame. The more accurately the number of image blocks to be detected, and therefore the number of shift vectors mentioned above, is increased, the more precisely the correlation between image blocks, and therefore the motion vector that closely approximates the actual image movement, can be detected. However, it is impossible to increase the number of image blocks, and therefore the number of shift vectors, indefinitely in terms of the calculation time that can actually be used for image processing performed in real time. The number of image blocks that can be set for this purpose is limited to a relatively small number. Here, if the shift vector between image blocks whose degree of correlation is to be considered is called a sample vector, and the sample vector is set in an area where there is a high possibility that a motion vector will actually be detected, the degree of correlation can be The number of sample vectors to be considered is small. For this reason, in the above-mentioned "motion vector detection method" disclosed in JP-A-54-124927, since the motion of an image generally has almost continuity in time between successive frames, the correlation between the previous The image blocks in the previous frame image, and therefore the entire sample vector, are shifted in the opposite direction to the image motion by the amount of the motion vector detected by the value calculation to create a new sample vector, and these new sample vectors are created by block shifting. The next correlation value calculation was performed for the sample vector.

The conventional motion vector detection method described above is a motion vector detection method that is extremely effective in practice because it can detect a motion vector that closely approximates the movement of an actual image even by using a small number of sample vectors. On the other hand, as mentioned above, an extremely large effect can be obtained for moving images that inherently have continuous movement, such as ordinary television images, but for example, when it comes to television images from film programs, For moving images in which there is discontinuity in image motion, calculation of interframe correlation values for image blocks whose motion is discontinuous, and therefore motion vector detection, cannot be performed smoothly.

For example, moving images from movie film are made up of 24 frames per second, while television images are made up of 30 frames per second, or 60 fields per second. In order to adjust for the difference in the number of images per second, a so-called 2-3 pulldown is usually performed, in which images of the same frame of a motion picture film are continuously captured for three fields only once every five field periods of television images. As a result, the same field image, either odd or even, is repeatedly captured. Therefore, during repeated imaging, although motion vectors should be detected between successive odd-numbered feeds or between successive even-numbered feeds, the same image is repeatedly captured in odd-numbered feeds or even-numbered feeds. Regarding the image block, the squared correlation values for all sample vectors did not converge to zero, and the comparison block with the strongest correlation could not be identified, so the motion vector was not detected, and as described above, it was detected by the previous correlation value calculation. The process of motion vector detection, in which the next correlation value is calculated for a sample vector set according to the motion vector, is continuously and repeatedly interrupted, and motion vector detection using the conventional method cannot be performed smoothly. , a problem arises.

It is an object of the present invention to solve the above-mentioned conventional problems and eliminate their drawbacks, and to partially improve image quality by continuously repeating the same field image, a typical example of which is 2-3 pulldown in movie film imaging. Even if there is discontinuity in motion, the comparison block is shifted based on the motion vector that has been detected asymptotically or approximately for the spatially or temporally nearest image part, such as between odd and even feeds, and An object of the present invention is to provide an image motion vector detection method capable of stably and smoothly detecting motion vectors at a certain stage without interrupting or making constant changes.

That is, in the asymptotic detection method of image motion vectors of the present invention, a block of interest consisting of an image area of a predetermined size is set for each successive frame image, and blocks different from each other in the image of the preceding frame one frame before are set. A plurality of comparison blocks having the same shape and size as the block of interest are set respectively shifted in the direction from a reference position corresponding to the position of the block of interest in the image of the frame concerned, and among the plurality of comparison blocks, In a motion vector detection method that detects a motion vector representing the direction and distance of movement of an image by detecting a comparison block having the maximum correlation with the block of interest, the block of interest is detected in the image of the preceding frame. The plurality of comparison blocks are set around the reference position, which is shifted by the same distance in the opposite direction to the motion vector detected for the preceding frame for the block of interest from a position corresponding to the position of the frame in the image. In order to increase the probability of detecting a comparison block having the maximum correlation with the block of interest from among the plurality of comparison blocks by When the block cannot be continuously detected, the reference position is shifted by the same distance in the opposite direction to the previously detected motion vector for another block located near the position of the block of interest in the image of the frame. The invention is characterized in that the plurality of comparison blocks are set around each of the comparison blocks.

The invention will be explained in detail below by way of example embodiments with reference to the drawings.

In detecting an image motion vector according to the present invention, a plurality of blocks of interest whose positions are sequentially shifted are set in an image of each frame, and for each of these blocks of interest, an image of the preceding frame one frame before is set. A plurality of comparison blocks corresponding to each other and shifted in different directions are set within the block, and among these comparison blocks,
This method is exactly the same as the conventional motion vector detection method described above in that the motion vector of the image is detected by detecting the comparison block having the strongest correlation with each of the plurality of blocks of interest. In the conventional motion vector detection method, as the amount for shifting the sample vector, the amount of temporally adjacent motion that has temporal continuity with respect to the block of interest detected by the previous correlation value calculation with respect to the image of the previous frame is used. Whereas vectors were used, in the asymptotic detection method according to the present invention, there is a difference between the image of the block of interest and the images of other blocks located in its vicinity within the same frame, with similar image movements. Focusing on the fact that there is spatial continuity, the motion vector detected for the spatially nearest block located near the block of interest in the same frame image is calculated from the motion vector that should have been detected previously for the block of interest. The asymptotic detection method of the present invention is essentially different from the conventional motion vector detection method in that it is used instead of a vector.As a result, the block of interest itself is To stably and smoothly continue the intended motion vector detection without interrupting or changing the motion vector, even if the motion vector cannot be directly and continuously detected due to discontinuity of image motion such as when the motion vector is interrupted. becomes possible.

Therefore, in the asymptotic detection method of the present invention, the motion vector used to shift the sample vector for which the correlation value calculation is to be performed is a motion vector that has already been used for another block located as close as possible to the block of interest whose image motion is to be detected. Since it is preferable to use a motion vector that has been detected, it is therefore more preferable to use a motion vector that has already been detected for another block adjacent to the block of interest in the frame, as described above.

Note that, in practice, the motion vector used to shift the position of the sample vector must have already been detected by that time, so for normal scanning images, in other words, the motion vector must be detected. If the order of image blocks to be scanned is the same as the order or direction of normal image scanning, for example, the motion vectors that have already been detected for other blocks located to the left, above, or diagonally above the block of interest. will be used.

In addition, in moving images such as television images, the subject image usually moves left and right, so the motion vector detected for another block adjacent to the left along the direction of movement is used. It is better to use the motion vectors detected for other upper adjacent blocks orthogonal to the direction of motion to set the sample vector, because it is possible to detect a motion vector that closely approximates the actual motion vector for the block of interest. Because it can be done,
This is even more suitable.

Next, an example of the configuration of an image motion vector detection device according to the method of the present invention, which detects motion vectors asymptotically as described above, is shown in FIG. 1. , an example of how to assign addresses is shown in FIG. 2, and the operation of the circuit device for detecting a motion vector according to the present invention will be explained.

In the motion vector detection device having the configuration shown in FIG. 1, the input image signal from the input terminal is first exchanged and written into the frame memory 1 and the buffer memory 2 by the amount necessary for motion vector detection, respectively. Image signals are written to each memory using the so-called sequential addressing method, in which sequential pixel signals are written to sequential memory addresses. The image signal is written so that it corresponds to the memory address. Next, when reading each pixel signal for motion detection, when supplying the address signal generated by the address generator 5 to each memory, the buffer memory 2 into which the image signal of the current frame has been written is The address signal to which the address signal indicating the position of the block of interest from the address generator 5 is supplied is supplied as is, and the image signal of the previous frame has been written together with the image signal of the current frame for at least the second half of the feed. The above-mentioned address signal from the generator 5 and the sample vector signal generated by the sample vector generator 8 indicating the set position of each comparison block are added in the adder 10, and then the block address generator 6 adds the sample vector signal generated by the sample vector generator 8 to The motion vector memory 7 is accessed and read using the block address signal obtained by correcting the address signal of the block by the positional deviation from the block of interest, and the previously detected motion vector signal and the above-described motion vector signal of other nearby blocks, for example, the block immediately above, are read out. The added output of the adder 10 is added in the adder 9, and the added output is supplied as a memory address for reading out the image signal of the previous frame. Therefore, the memory address for reading the previous frame image supplied to the frame memory 1 is an amount obtained by shifting the sample vector from the sample vector generator 8 by the amount of motion vectors already detected for adjacent blocks from the motion vector memory 7. , the memory address is different from the memory address supplied to the buffer memory 2.

The motion vector memory 7 has a memory capacity sufficient to sequentially store motion vector signals detected for each block in a plurality of sequential rows in the block arrangement shown in FIG. Under the control, the block address of the adjacent block in the row located immediately before the row to which the image block of the current frame in the buffer memory 2 for which the current motion vector is to be detected is inputted to the motion vector from the block address generator 6. From among the output motion vector signals supplied to the memory 7 and stored in the motion vector memory 7, the detected motion vector signals for the adjacent block in the immediately previous row corresponding to the block address are read out.

The image motion vector detection device according to the conventional method, which is the object of improvement by the method of the present invention, does not include the block address generator 6, and the address generator 5
The address signal of the block of interest from is applied directly to the motion vector memory 7, and the previously detected motion vector signal of the block of interest is read out and added to the addition output of the adder 10, thereby eliminating the influence of image motion. Due to the continuity, if the previously detected motion vector signal for the block of interest cannot be obtained, the motion vector detection process has to be interrupted.

As described above, the buffer memory 2 and the frame memory 1 are supplied with memory address signals specifying the respective blocks whose mutual correlation values are to be examined, and the current frame and frame memory 1 are respectively read out.
The image signal of the preceding frame before the frame is supplied to the correlation calculation unit 3, and the correlation value between the blocks is calculated by, for example, calculating the squared correlation value using the arithmetic formula described above in connection with the prior art, and the correlation value between the blocks is calculated. The resulting correlation values for each comparison block are supplied to a maximum determiner 4, and the motion vector for each block is sequentially detected from the position of the comparison block where the strongest correlation is obtained.

Therefore, as mentioned above, the present invention detects the motion vector of an image block using the motion vector already detected for the image block directly above the image block whose motion vector is to be detected in the same frame image. In the inventive motion vector asymptotic detection method, since there is no motion vector already detected in the frame for each block at the top of the block array shown in FIG. Unable to use previously used motion vectors. Therefore, in order to apply the present invention to the block at the top, it is assumed that the detected motion vector to be followed is a zero vector, or the motion vector detected for the block at the top in the image of the preceding frame one frame before is assumed. Although the motion vector of the block is detected using the motion vector of the previous block, it is obvious that the latter method allows for more appropriate motion vector detection. However, in any case, for blocks existing at the top or second stage of the screen, the motion vectors are detected around the block, especially the blocks immediately above it, such as in the center of the screen. It is not possible to configure it sufficiently according to your needs,
Therefore, although accurate motion vector detection cannot be performed, the influence of the inaccuracy on the entire screen is small and can be ignored in practice.

As is clear from the above description, according to the present invention, even if there is a discontinuous part in the movement of a moving image such as a television image, it is possible to achieve almost the same accurate movement as for continuous movement. A remarkable effect is obtained in that vector detection can be performed.

[Brief explanation of drawings]

FIG. 1 is a block diagram showing a configuration example of an image motion vector detecting apparatus according to the method of the present invention, and FIG. 2 is a diagram showing an arrangement of blocks into which an image is divided. 1...Frame memory, 2...Buffer memory, 3
...Correlation calculation unit, 4...Maximum judger, 5...Address generator, 6...Block address generator, 7...Movement vector memory, 8...Sample vector generator, 9, 10
...Adder.

Claims (1)

[Scope of Claims] 1. A block of interest consisting of an image area of a predetermined size is set for each successive frame image, and the corresponding block of interest is set in different directions within the image of the preceding frame one frame before. A plurality of comparison blocks having the same shape and size as the block of interest are set respectively shifted from a reference position corresponding to the position of the frame in the image, and among the plurality of comparison blocks, the block with the largest size relative to the block of interest is set. In the motion vector detection method of detecting a motion vector representing the direction and distance of movement of an image by detecting correlated comparison blocks, the block of interest in the image of the preceding frame is detected in the image of the current frame. By setting the plurality of comparison blocks around the reference position, which is shifted by the same distance in the opposite direction to the motion vector detected for the preceding frame for the target block from the position corresponding to the position, the plurality of comparison blocks are set. In order to increase the probability of detecting a comparison block having the maximum correlation with the block of interest from among the comparison blocks of interest, it is determined that the motion vector of the block of interest cannot be detected at least continuously with respect to the preceding frame. Sometimes, the plurality of comparisons are performed around the reference position shifted by the same distance in the opposite direction to the previously detected motion vector for other blocks located near the position of the block of interest in the image of the frame. A method for asymptotic detection of image motion vectors, characterized in that blocks are set individually. 2. An asymptotic method for detecting an image motion vector according to claim 1, characterized in that a block located directly above and close to the position of the block of interest is used as the other block.