JP5286573B2 - Motion vector detection apparatus, motion vector detection method and program - Google Patents

Description

  The present invention relates to a motion vector detection device, a motion vector detection method, and a program.

  As a moving picture coding method for compressing and coding a moving picture, MPEG (Moving Picture Experts Group) coding method, The H.264 / AVC encoding method and the like are known. As an MPEG encoding method, the MPEG-2 encoding method is used for high image quality applications such as digital broadcasting and DVD, and the MPEG-4 encoding method is widely used for low bit rate applications such as mobile videophones. . H. The H.264 / AVC encoding system is an encoding system jointly standardized by ITU-T (The International Telecommunication Union Telecommunication Standardization Sector) and MPEG, and is a new system that achieves both high image quality and high compression rate. It attracts attention.

In these moving image encoding methods, generally, motion prediction (inter prediction) is performed to reduce the code amount. In motion prediction, when a macroblock (MB), which is a part obtained by dividing a single image (frame), is encoded, an image (reference image) positioned in the past or future in time is encoded. Search for a macroblock to be encoded, or a region where the image is similar to a block obtained by further dividing this macroblock, and code the difference between the image detected in the search and the image to be encoded. The amount of code indicating a subject that exists continuously in time can be greatly reduced. This operation of searching for an encoding target macroblock or an area similar to an image of a block is called motion search.
MPEG system, H.264, etc. In the H.264 / AVC format, images to be encoded are classified into three types of I pictures (Intra-coded Pictures), P pictures (Predicated Pictures), and B pictures (Bi-directed Pictures). The I picture is an image for which motion prediction is not performed, and is referred to as a P picture or a B picture. A P picture is an image in which motion prediction in one direction is performed using an I picture or P picture located in the past in time. A B picture is an image that performs bidirectional motion prediction using an I picture or P picture located in the past in time and an I picture or P picture located in the future.

FIG. 7 shows MPEG-2 and H.264. 2 is a diagram illustrating an example of an image referred to in H.264 / AVC motion prediction.
In the motion prediction of MPEG-2, an image referred to by a P picture or a B picture is uniquely determined as an image located immediately before or after. The picture referred to in the motion prediction of the P picture is the immediately preceding I picture or P picture. In the example of FIG. 7A, five images P715 to P711 are shown in time order, P715, P713, and P711 are P pictures, and P714 and P712 are B pictures. Among them, the previous P picture P713 is referred to when motion prediction of the P picture P711 is performed. In addition, the images referred to in the motion prediction of the B picture are the immediately preceding I picture or P picture, and the immediately following I picture or P picture. In the example of FIG. 7A, when the motion prediction of the B picture P712 is performed, the immediately preceding P picture P713 and the immediately following P picture P711 are referred to.

On the other hand, H.H. In the H.264 / AVC encoding scheme, a reference image used for motion prediction can be selected from a large number of candidates in order to increase encoding efficiency.
In the example of FIG. 7B, five images P725 to P721 are shown in time order, P725, P723, and P721 are P pictures, and P724 and P722 are B pictures. Among these, when performing motion prediction of P picture P25, P pictures P725 and P723 are selected as reference pictures from temporally forward I and P pictures (and not deleted from the reference picture memory). ing. In the example of FIG. 7B, when performing motion prediction of the B picture P24, temporally forward P pictures P725 and P723 and temporally backward P picture P721 are selected. .

FIG. 2 is a diagram illustrating the size of a block serving as a unit for performing motion prediction in the H.264 / AVC encoding scheme. FIG.
In the MPEG-2 encoding method, the unit for motion prediction is limited to a macro block unit of horizontal 16 × vertical 16 pixels (or a block unit of horizontal 16 × vertical 8 pixels depending on the encoding mode). On the other hand, H.H. In the H.264 / AVC encoding method, the horizontal 16 × vertical 16 pixel block B811, the horizontal 16 × vertical 8 pixel block B821 and B822, the horizontal 8 × vertical 16 pixel block B831 and B832, and the horizontal 8 × vertical 8 pixel block shown in FIG. B841 to B844, further subdivided into horizontal 8 × vertical 4 pixel sub-blocks B851 and B852, each of horizontal 8 × vertical 8 pixel blocks, sub-blocks B861 and B862 of horizontal 4 × vertical 8 pixels, horizontal 4 × vertical 4 One of the division methods is selected from the pixel sub-blocks B871 to B874. Then, for each block or sub-block, a motion search is performed to search for a region on the reference image similar to the block or sub-block of the image to be encoded, and a motion vector indicating the region (of the image to be encoded) Relative coordinates from the block or sub-block) can be calculated, and motion prediction can be performed for each block or sub-block. However, in an encoder that requires real-time processing, motion prediction is often performed by selecting a block size of 8 × 8 pixels or more in order to avoid processing complexity.

  When the reference image and the encoding target block size can be selected from a large number of candidates as in the H.264 / AVC encoding scheme, all reference images and block sizes are ideally selected in order to improve the encoding efficiency. It is desirable to perform a motion search using as candidates and determine an optimal reference image and block size. However, if motion search is performed for all reference images and block sizes, the amount of computation becomes enormous. Therefore, in a general encoder, the amount of calculation is reduced by performing motion search in two stages. In this case, first, candidates for a reference image and a block size are narrowed down by a primary search that performs a wide range of motion searches for a plurality of reference images for a limited block size. Then, based on the result of the primary search, a secondary search that is a detailed motion search including other block sizes is limited to the vicinity of the vector obtained as a result of the primary search, and the final reference image or Determine the block size.

Further, in the primary search, in order to detect a part that is more similar to the part to be encoded from the reference image, it is necessary to perform a motion search over a wide range on the reference image, and the calculation amount is very large. growing. Therefore, a general encoder performs a primary search by reducing the image to be encoded and the reference image. For example, by reducing the image to be encoded and the reference image by a factor of two both vertically and horizontally, the number of pixels to be compared can be reduced to a quarter, and a wide range is searched for motion with a smaller amount of computation. be able to.
Furthermore, paragraphs 0028 to 0032 of Patent Document 1 show a motion search method called a telescopic search method. In the telescopic search method, it is possible to follow a large movement of an object on a reference image by repeating a search in a relatively small range.

FIG. 9 is a diagram illustrating an example of telescopic search.
In the figure, the encoding target image P91 and the reference image P94 are separated by three images in the time direction with the non-reference images P92 and P93 interposed therebetween. In the telescopic search, when determining a portion in the reference image P94 that is referenced by the encoding target block of the encoding target image P91, the reference image P94 is not directly searched for motion, but is first temporally adjacent non-reference. A search range corresponding to the coordinates of the encoding target block is set in the image P92, and a motion search is performed. If relative coordinates (motion vectors) indicating a region similar to the encoding target block B91 are obtained as a search result R92 by the motion search, a search range is set in the non-reference image P93 with the coordinates indicated by the search result R92 as the center. Set and perform motion search. When the search result R93 is obtained, a search range is set in the reference image P94 around the coordinates indicated by the search result R93, and a motion search is performed. The search result R94 obtained by this motion search is used as the final motion search result.
In this way, by performing motion search sequentially on temporally adjacent images, it is possible to follow the motion of the subject of the encoding target block. Accordingly, the target area for each motion search can be a relatively narrow range, and a wide range can be set as the target area for motion search for the final reference image P94.

H. When motion prediction using a small block size is possible as in the H.264 / AVC encoding method, it is desirable to perform a primary search with as small a block size as possible in order to closely track the motion of an object on the screen. For example, in an encoder that performs a primary search with either a block size of 8 × 8 pixels or a block size of 16 × 16 pixels, it is better to perform a primary search with a block size of 8 × 8 pixels. A more detailed motion search can be performed than when a primary search is performed with 16 horizontal pixels × 16 vertical pixels. Specifically, when the image to be encoded includes a boundary between the object and the background, a small block size is used to distinguish the object block from the background block and to follow each movement in detail. Can do.
However, when the primary search of the block size of horizontal 8 × vertical 8 pixels is performed with an image reduced to half the vertical and horizontal, the image used for motion search has a small size of horizontal 4 × vertical 4 pixels = 16 pixels. Therefore, it may be difficult to accurately follow the movement of the object due to the influence of disturbance due to noise or the like in the image. For example, when using telescopic search, once tracking of a moving object fails, the center point of the search shifts, and there is a high probability that the subsequent search will also fail. In addition, if the telescopic search is performed independently for each of the four blocks per macroblock, the amount of calculation increases.

Therefore, a method of performing motion search in units of macroblocks in the primary search can be considered. For example, a telescopic search of horizontal 8 × vertical 8 pixels = 64 pixels is performed on an image reduced to half the vertical and horizontal dimensions. As a result, disturbance due to noise can be prevented, and the amount of calculation can be reduced.
In this case, only one motion vector is obtained for one macroblock. Therefore, in order to cope with motion prediction such as a block of horizontal 16 × vertical 8 pixels or horizontal 8 × vertical 16 pixels or horizontal 8 × vertical 8 pixels, the vicinity of the motion vector of the primary search result in the secondary search is horizontal 8 Search in units of vertical 8 pixel blocks, or search in the vicinity of zero vectors and motion vector prediction values (predicted motion vectors (PMV)) in units of horizontal 8 × vertical 8 pixel blocks in a secondary search. be able to.

The “zero vector” here is a vector indicating the origin. The zero vector indicates the position of the block to be encoded on the reference image in relative coordinates from the position of the block to be encoded. Searching near the zero vector is effective for detecting a stationary background or the like.
In addition, the “motion vector prediction value” is obtained from the motion vector of each block that has already detected a motion vector adjacent to the top, upper right, upper left, or left of the block to be encoded. It is a vector calculated based on a procedure defined by an encoding standard such as H.264 / AVC. For example, in the case of the horizontal 16 × vertical 8 pixel blocks B821 and B822 described with reference to FIG. 8, the upper block B821 calculates a motion vector prediction value from the motion vector of the block adjacent to the block B821. Further, in the lower block B822, a motion vector prediction value is calculated from the motion vector of the block adjacent to the left of this block B822.
The motion vector prediction value is a vector value that represents the motion of the surrounding block, and when the image of the encoding target block shows the same motion as the image of the surrounding block, the encoding target block Is expected to be almost the same value as the motion vector prediction value. For example, H.M. When a motion vector is encoded according to the H.264 / AVC encoding standard, encoding is performed by taking the difference between the motion vector and the motion vector prediction value. For this reason, the closer the motion vector value is to the motion vector prediction value, the smaller the amount of code for expressing the motion vector.

JP 2000-242554 A

When the primary search is performed in units of macroblocks, there are cases in which a moving object cannot be accurately followed by a macroblock located at the boundary between objects, such as a moving object and a background.
FIG. 10 is a diagram illustrating an example of a macroblock located at the boundary between the moving object and the background.
The macroblock B101 in the figure is an image of a “moving automobile” in which the entire area of 16 × 16 pixels is “moving car”, and the movement of the image can be followed by a motion search in units of macroblocks.
On the other hand, in the macro block B102, the upper half 16 × 8 pixels area is the “background” image, and the lower half 16 × 8 pixels area is “moving car”. It is an image. For this reason, in the search in units of macroblocks, the motion vector of a moving car may not be detected properly due to the influence of the background area.

For the “background” portion of the upper half of the macroblock B102, it can be expected that a motion vector is obtained by searching the vicinity of the zero vector in units of 8 × 8 pixels in the secondary search, for example. Alternatively, even when the background is moving (panning), the motion vector prediction value reflects the motion of the surrounding background. Therefore, by searching for the vicinity of the motion vector prediction value, it is possible to predict the motion of the background portion with a certain degree of accuracy.
However, for the moving object region in the lower half of the macroblock B102 that cannot be accurately searched due to the influence of the background, the original coordinates for performing the secondary search cannot be obtained and cannot be dealt with by the secondary search. As a result, the amount of code increases, for example, motion prediction is performed by taking the difference between a macroblock to be encoded and an image with low similarity, or intra prediction that does not use motion prediction as in the case of an I picture. Or the image quality of the decoded image is degraded.

  The present invention has been made in consideration of such circumstances, and the object thereof is a macroblock that is less affected by disturbances such as noise in an image, has a small amount of calculation, and is located at the boundary of an object. However, it is to provide a motion vector detection apparatus, a motion vector detection method, and a program capable of appropriately detecting a motion vector.

[1] The present invention has been made to solve the above-described problems, and a motion vector detection device according to an aspect of the present invention provides a block on a reference image for each block obtained by dividing a macroblock of an input moving image. a motion vector detecting apparatus for detecting a motion vector indicating the position of the region corresponding to, for each block obtained by dividing the macro block, and surrounding the zero vector indicating the position of the block on the reference image, the for each of the candidate motion vectors of the surrounding motion vector prediction value determined on the basis of the motion vectors of blocks adjacent to the block, calculates the motion vector evaluation value based on a difference between pixels of the pixel and the reference picture of the block A pre-neighbor searcher for determining the presence or absence of a motion vector evaluation value below a predetermined threshold, and a primary for the macroblock The can vector, the pre vicinity searcher is based on the difference between the pixel of the pixel and the reference image of each block is determined that the following motion vector evaluation value without a predetermined threshold value, detected by the motion search in the macro block unit For each of the primary searcher and the blocks determined by the prior neighborhood searcher as having no motion vector evaluation value equal to or less than a predetermined threshold, a motion vector is detected by performing a motion search around the primary motion vector, For each block for which the pre-article neighborhood searcher determines that there is a motion vector evaluation value equal to or less than a predetermined threshold, a secondary motion vector candidate whose motion vector evaluation value is equal to or less than the predetermined threshold is used as the motion vector of the block. And a searcher.
The “primary motion vector” here is a motion vector obtained by a primary search, which is a motion search performed prior to a neighborhood search (secondary search), and indicates the center of the secondary search.

[2] A motion vector detection method according to an aspect of the present invention is a motion vector detection method for detecting a motion vector indicating a position of an area corresponding to a block on a reference image for each block obtained by dividing a macroblock of an input moving image. a motion vector detecting method of the vector detecting device, for each block obtained by dividing the macro block, and surrounding the zero vector indicating the position of the block on the reference image, the motion vectors of blocks adjacent to the block a peripheral motion vector prediction value, which is determined, for each of the candidate motion vectors, and calculates a motion vector evaluation value based on a difference between pixels of the pixel and the reference picture of the block, following the motion predetermined threshold A prior neighborhood search step for determining the presence or absence of a vector evaluation value, and a primary motion vector for the macroblock, Based on the difference between the pixel of the pixel and the reference image of each block is determined that the predetermined following motion vector evaluation value without the threshold at pre vicinity search step, a first search step of detecting the motion search in the macro block unit, For each block determined to have no motion vector evaluation value equal to or less than a predetermined threshold in the prior neighborhood search step, a motion vector is detected by performing a motion search around the primary motion vector, and the prior neighborhood search For each block that is determined to have a motion vector evaluation value less than or equal to a predetermined threshold in the step, a secondary search using the motion vector candidates whose motion vector evaluation value is less than or equal to the predetermined threshold as the motion vector of the block And a step.

[3] A program according to an aspect of the present invention causes a computer to output a motion vector indicating a position of an area corresponding to a block on a reference image for each block obtained by dividing a macroblock of an input moving image. a program, the macro block for each block obtained by dividing the periphery of the zero vector indicating the position of the block on the reference image, the motion vector prediction determined based on motion vectors of blocks adjacent to the block with the surrounding values, for each of the candidate motion vectors, and calculates a motion vector evaluation value based on a difference between pixels of the pixel and the reference picture of the block, whether the following motion vector evaluation value predetermined threshold A pre-neighbor search step for determining, and a primary motion vector for the macroblock, the pre-neighbor search step; Based on the difference between the pixel of the reference image and the pixel of each block is determined that the predetermined following motion vector evaluation value without the threshold at-flop, a first search step of detecting the motion search in the macro block unit, the pre For each of the blocks determined as having no motion vector evaluation value equal to or less than a predetermined threshold in the neighborhood search step, a motion vector is detected by performing a motion search around the primary motion vector, and in the prior neighborhood search step For each block determined to have a motion vector evaluation value less than or equal to a predetermined threshold, a secondary search step in which the motion vector candidate whose motion vector evaluation value is less than or equal to the predetermined threshold is the motion vector of the block; Is a program for causing the computer to execute.

  According to the present invention, it is difficult to be affected by disturbances such as noise in an image, the amount of calculation is small, and a motion vector can be appropriately detected even in a macroblock located at the boundary of an object.

It is a block diagram which shows schematic structure of the motion vector detection apparatus in one Embodiment of this invention. It is a block diagram which shows schematic structure of the primary searcher 6 in the embodiment. It is a figure which shows the example of the evaluation with respect to the position and the motion vector of each block which the neighborhood searcher 8 makes the object of prior neighborhood search in the embodiment. It is a figure which shows the example of evaluation with respect to a motion vector which the primary searcher 6 performs in the embodiment. It is a figure which shows the example in which the primary searcher 6 performs a telescopic search in the same embodiment. 5 is a flowchart illustrating a processing procedure in which the motion vector detection apparatus 100 detects a motion vector for a macro block C121 to be encoded in the embodiment. MPEG-2 and H.264 2 is a diagram illustrating an example of an image referred to in H.264 / AVC motion prediction. H. 2 is a diagram illustrating the size of a block serving as a unit for performing motion prediction in the H.264 / AVC encoding scheme. FIG. It is a figure which shows the example of a telescopic search. It is a figure which shows the example of the macroblock located in the boundary of a moving body and a background.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a configuration diagram showing a schematic configuration of a motion vector detection device according to an embodiment of the present invention.
In the figure, a motion vector detecting device 100 includes an image memory 1, a search range segmenter 2, image reducers 3 and 12, a reduced image cache memory 4, an image cache memory 5, a primary searcher 6, A search center indicator 7 and a neighborhood searcher (pre-neighbor searcher, secondary searcher) 8 are provided.
The image reducer 12 receives an input of an encoding target macroblock image C121 obtained by dividing an input moving image in units of macroblocks, and the encoding target macroblock image C121 is reduced to half the number of pixels both vertically and horizontally. The reduced image C122 is output to the primary searcher 6.

  The image memory 1 receives the input of the local decoded image C11 and accumulates the local decoded image C11. The local decoded image stored in the image memory 1 is an original image from which a reference image used for motion search is cut out. Here, the “local decoded image” is an image obtained by encoding and further decoding an input image by a moving image encoding apparatus including the motion vector detection apparatus 100. The local decoded image is an image generated also by the video decoding device, and by performing motion search using the local decoded image, image errors due to encoding and decoding can be suppressed. Note that the image memory 1 may store the input image instead of the local decoded image. When the motion search is performed using the input image, the motion search can be performed without waiting for the generation and transfer of the local decoded image, so that the degree of freedom of the configuration of the moving image coding apparatus is increased.

The search range extractor 2 reads the image C12 stored in the image memory 1, and a range set around the encoding target macroblock as a range for which the primary searcher 6 and the neighborhood searcher 8 are subject to motion search. Are cut out and output to the image reducer 3 and the image cache memory 5. For example, when the encoding target macroblock is located at the upper left of the image C12, the range that the primary searcher 6 and the neighborhood searcher 8 are subject to motion search is centered on the encoding target macroblock. The search range clipper 2 cuts out an image in this range and outputs it to the image reducer 3 and the image cache memory 5.
The image reducer 3 reduces the image C21 output from the search range extractor 2 to half the number of pixels both vertically and horizontally, generates a reduced image C31, and outputs the reduced image C31 to the reduced image cache memory 4. The reduced image cache memory 4 stores a reduced image C31 output from the image reducer 3 as a reference image for primary search performed by the primary searcher 6. On the other hand, the image cache memory 5 stores an image C21 output from the search range extractor 2 as a reference image for secondary search performed by the neighborhood searcher 8.
The search center indicator 7 outputs the zero vector and the motion vector prediction value C71 to the neighborhood searcher 8. As described above, the zero vector indicates the position of the block to be encoded on the reference image. The motion vector prediction value is a vector value representing the motion of the surrounding blocks, and is calculated based on a method defined by the coding standard. Note that if an accurate motion vector prediction value cannot be obtained, such as when the encoding mode of a macroblock located to the left of the block to be encoded is not fixed, the search center indicator 7 A motion vector prediction value may be generated and output to the neighborhood searcher 8. The “provisional motion vector predicted value” here is an estimated value expected to indicate a value close to the motion vector predicted value. The tentative motion vector prediction value is obtained when, for example, the motion search of the primary searcher 6 for each macroblock adjacent to the upper, upper right, upper left, or left of the macroblock to be encoded has been completed. Using the respective motion vectors C61 obtained as a result of the motion search of 6, encoding is performed assuming that the own macroblock and the upper, upper right, upper left, and left macroblocks all have a block size of horizontal 16 pixels × vertical 16 pixels. Calculate based on the standard procedure. Hereinafter, the motion search performed by the primary searcher 6 is also referred to as “primary search”, and the motion vector obtained as a result of the primary search is also referred to as “primary motion vector”. The primary motion vector is a motion vector obtained with an accuracy of two pixels for the macroblock to be encoded, and indicates the center of the motion search (secondary search) performed by the neighborhood searcher 8.

The neighborhood searcher 8 divides the encoding target macroblock C121 into horizontal 8 × vertical 8 pixel blocks, and based on the zero vector and the motion vector prediction value C71 output from the search center indicator 7 for each of the divided blocks. Thus, the motion search is performed using the positions around the zero vector and the motion vector prediction value on the reference image C51 as candidates. Hereinafter, the motion search performed by the neighborhood searcher 8 using each position around the zero vector and the motion vector prediction value as a candidate is also referred to as “pre-neighbor search”. The neighborhood searcher 8 outputs information C82 indicating a block in which a motion vector cannot be detected as a result of the prior neighborhood search to the primary searcher 6.
Further, the neighborhood searcher 8 uses the primary motion vector C61 output from the primary searcher 6 (specifically, the reference indicated by the primary motion vector C61) for each block for which the motion vector could not be detected by the prior neighborhood search. A search range centered on a position on the image C51 is set), and a motion search on the reference image C51 is performed. Hereinafter, the motion search performed by the neighborhood searcher 8 based on the primary motion vector C61 is also referred to as “secondary search”.
The neighborhood searcher 8 uses the motion vector C201 detected by the prior neighborhood search and the secondary search to the outside of the motion vector detection device 100 (for example, a predicted image generation unit of a video encoding device including the motion vector detection device 100). Output to.

  The primary searcher 6 performs a primary search on the reduced reference image C41 in units of macroblocks using the reduced image C122 of the encoding target macroblock output from the image reducer 12, and generates a primary motion vector C61. Output to the neighborhood searcher 8. When performing the primary search, the primary searcher 6 uses the sum of the absolute values of the differences between the pixels of the block in which the neighborhood searcher 8 could not detect the motion vector in the prior neighborhood search and the pixels of the reduced reference image C41. The motion vector is evaluated, and the primary motion vector C61 is detected based on the evaluation result.

FIG. 2 is a configuration diagram showing a schematic configuration of the primary searcher 6.
In the figure, the primary searcher 6 includes a difference absolute value sum calculator 611, a difference absolute value sum addition controller 621, a difference absolute value sum adder 631, and an evaluator 641.
The difference absolute value sum calculator 611 divides the reduced image C122 of 8 × 8 pixels of the encoding target macroblock output from the image reducer 12 into 4 × 4 pixel blocks. Then, the difference absolute value sum calculator 611 calculates, for each divided block, the sum of absolute values of differences for each pixel between the block and the area corresponding to the block on the reduced reference image C41 (hereinafter referred to as “difference”). (Also referred to as “absolute value sum”), and the calculated difference absolute value sum is output to the difference absolute value sum addition controller 621.

  The difference absolute value sum addition controller 621 is a difference between blocks in which the neighborhood searcher 8 cannot detect a motion vector in the prior neighborhood search among the absolute difference sums for each block output from the difference absolute value sum calculator 611. The absolute value sum is output to the difference absolute value sum adder 631. The difference absolute value sum adder 631 calculates the sum of the difference absolute value sums output from the difference absolute value sum addition controller 621 and outputs the sum to the evaluator 641. The evaluator 641 evaluates each of the primary motion vector candidates based on the sum of the absolute difference sums output from the absolute difference sum adder 631, and selects a primary motion vector. Here, the primary motion vector candidates are, for example, each area of 8 × 8 pixels included in a predetermined range centered on the position of the macroblock to be encoded on the reduced reference image C41. The motion vector shown (relative coordinates from the position of the macroblock to be encoded).

FIG. 3 is a diagram showing an example of the evaluation of the position and the motion vector of each block that are used by the neighborhood searcher 8 as a target for the advance neighborhood search.
When a zero vector is output and indicated by the relative coordinates at the upper left corner of the macroblock from the search center indicator 7, the neighborhood searcher 8 performs a motion search around this zero vector. Specifically, the neighborhood searcher 8 performs a motion search by using a position shifted within a predetermined amount of shift of the pixel in the vertical direction and the horizontal direction as candidates. For example, in the example shown in the figure, the motion search is performed using a total of nine points, which are shifted by one pixel in the vertical direction and the horizontal direction, respectively. Note that the candidates for the search by the neighborhood searcher 8 are not limited to the nine points shown in FIG. For example, more points are selected as candidates for motion search, such as searching for a point shifted by 2 pixels from the center or a point shifted by an image other than an integer value such as 0.5 pixel or 0.25 pixel. May be. Thereby, a motion search with higher accuracy can be performed, for example, a motion vector with 0.5 pixel accuracy or 0.25 pixel accuracy can be obtained. The same applies to the motion vector prediction value.

The neighborhood searcher 8 divides the 16 × 16 macroblock region indicated by the candidate point into four blocks B21 to B24 of 8 × 8 pixels, and calculates a motion vector evaluation value for each block. . Here, the “motion vector evaluation value” is a bit necessary to describe the motion vector and the coding mode in the sum of absolute values of differences between each pixel in the block and each pixel of the corresponding reference image. It is a value obtained by adding a value corresponding to the amount (encoding cost). The motion vector evaluation value is an index indicating a code amount or an image error when motion prediction is performed using the motion vector, and a motion vector having a smaller motion vector evaluation value is used for motion prediction for a macroblock to be encoded. Suitable for use.
In order to determine whether or not a sufficiently accurate motion vector is obtained, the neighborhood searcher 8 determines whether or not there is a motion vector that satisfies Equation (1).

  Here, α represents a threshold value of the evaluation value. This α is a predetermined constant, for example. Alternatively, α may be a value determined according to the analysis result of the immediately preceding image (frame). For example, in the previous image (frame), if there are many blocks whose motion vectors are in the vicinity of the zero vector or near the motion vector evaluation value, the still image block or the image block that moves in the same manner as the surrounding blocks It is thought that there are many. Also, in the image (frame) to be encoded that is temporally adjacent to this image (frame), it is predicted that there are many blocks of still images and images that move in the same manner as the surrounding blocks. The Therefore, by increasing the value of α so that the position near the zero vector or the vicinity of the motion vector evaluation value can be easily selected as the motion vector, it can be expected that the neighborhood searcher 8 detects a more appropriate motion vector. . Conversely, in the previous image (frame), if there are few blocks whose motion vector is in the vicinity of the zero vector or near the motion vector evaluation value, the value of α is decreased to be the same as the previous image (frame). Further, it is difficult to select a position near the zero vector or the position near the motion vector evaluation value as the motion vector.

  Alternatively, α may be a value that is determined according to a pre-analysis result of the encoding target image (frame). For example, the degree of similarity for each pixel from the image (frame) to be encoded using the image (frame) reduced to a small number of pixels, such as 1/8 pixel both vertically and horizontally, from the temporally immediately preceding image (frame) ( For example, if the similarity is high, the value of α is increased, and the neighborhood searcher 8 determines the position near the zero vector or the motion vector evaluation value as a motion vector. Make it easy to select. Conversely, if the similarity is low and low, the value of α is decreased.

Alternatively, the value of α may be changed based on the variance value or activity of the encoding target macroblock. The “activity” of the encoding target macroblock here is one of the values indicating variations in pixel values in the macroblock. The activity is obtained by dividing the macroblock into four blocks of 8 × 8 pixels, calculating a variance value for each block, and adding 1 to the smallest of the calculated variance values.
In general, a macro block with a small variance value or activity has a small difference in motion vector evaluation value due to a motion vector shift, and conversely, a macro block with a large variance value or activity has a difference in motion vector evaluation value due to a motion vector shift. Becomes larger. Accordingly, the value of α is set to a value proportional to the value of the variance or activity, for example, or the variance or activity exceeds or exceeds a certain threshold value S according to the variance or activity value. By setting a different α depending on the case, it is possible to make it less susceptible to the influence of the properties of the macroblock when performing the determination. Note that the threshold S is not limited to one, and by setting a plurality of αk corresponding to each of the plurality of thresholds Sk, the α can be selected from αk according to the variance value or activity of the encoding target macroblock. May be selected.

In the prior neighborhood search, the neighborhood searcher 8 selects a motion vector having the smallest motion vector evaluation value as a motion vector of the block for a block having a motion vector satisfying the expression (1). On the other hand, for a block that does not have a motion vector that satisfies Equation (1), it is determined that the motion vector of the block has not been detected.
For example, when the upper horizontal 8 × vertical 8 pixel blocks B21 and B22 in FIG. 3 are still background image regions, the motion vector evaluation value is minimized when the zero vector is a motion vector, and the expression (1) When satisfying, the neighborhood searcher 8 determines that there is a motion vector satisfying the expression (1) for each of the blocks B21 and B22, and uses the motion vector evaluation value as the motion vector of each of the blocks B21 and B22. The zero vector that minimizes is detected.

On the other hand, in the case where the motion vector satisfying the expression (1) is not detected in the prior neighborhood search, such as when the lower 8 × 8 pixel blocks B23 and B24 on the lower side of FIG. 8 determines that the motion vector of the block cannot be detected by determining that there is no motion vector satisfying the expression (1) for each of the blocks B23 and B24.
The neighborhood searcher 8 outputs information indicating a block determined to have failed to detect a motion vector to the primary searcher 6. In the example of FIG. 3, the neighborhood searcher 8 outputs information indicating the blocks B <b> 23 and B <b> 24 to the primary searcher 6.

FIG. 4 is a diagram illustrating an example of evaluation for a motion vector performed by the primary searcher 6.
Blocks B41 and B42 in the figure are, for example, areas of a stationary background image, and are blocks in which the neighborhood searcher 8 has detected a motion vector in the prior neighborhood search. On the other hand, the blocks B43 and B44 are, for example, moving image regions, and are the blocks that the neighborhood searcher 8 has determined that a motion vector could not be detected in the prior neighborhood search.
When performing a motion search, the primary searcher 6 evaluates a motion search result using a pixel of a block that is determined to have not been detected by the neighborhood searcher 8 in the prior neighborhood search. In the example of FIG. 4, the difference absolute value sum calculator 611 calculates, for each of the blocks B41 to B44, the difference absolute value sum between the pixel on the reduced image C122 of the encoding target macroblock and the pixel on the reduced reference image C41. C341 to C344 are calculated and output to the difference absolute value sum addition controller 621. The difference absolute value sum addition controller 621 determines that the neighborhood searcher 8 cannot detect the motion vector in the prior neighborhood search among the difference absolute value sums C341 to C344 output from the difference absolute value sum calculator 611. And only the difference absolute value sum C343 and C344 of B44 are output to the difference absolute value sum adder 631.

  On the other hand, the difference absolute value sum addition controller 621 does nothing (discards) for the difference absolute value sums C341 and C342 of the blocks B41 and B42 in which the neighborhood searcher 8 has detected the motion vector in the prior neighborhood search. The difference absolute value sum adder 631 calculates the sum of the difference absolute value sums C343 and C344 output from the difference absolute value sum addition controller 621 and outputs the sum to the evaluator 641. The evaluator 641 evaluates the motion vector based on the sum of the absolute difference sums output from the absolute difference sum adder 631.

  In this way, the primary searcher 6 evaluates the motion vector by excluding the pixel of the block from which the neighborhood searcher 8 detected the motion vector in the prior neighborhood search from the motion vector evaluation target. A block of a stationary background image that detects a motion vector by a motion search near the zero vector, or a background image that the neighborhood searcher 8 detects (moves) a motion vector by a motion search near the motion vector evaluation value. The motion vector can be evaluated based on only the block of the moving image without being influenced by the block. Thereby, even when the encoding target macroblock is located at the boundary between the moving object and the background, the moving object can be tracked.

FIG. 5 is a diagram illustrating an example in which the primary searcher 6 performs a telescopic search. In the figure, the encoding target image P51 and the reference image P54 are separated by three images in the time direction with the non-reference images P52 and P53 interposed therebetween.
When performing a telescopic search, the primary searcher 6 determines that, in each motion search performed on adjacent images, the neighborhood searcher 8 cannot detect a motion vector in the prior neighborhood search, as described with reference to FIG. The motion vector is evaluated using only the pixels.
First, the primary searcher 6 sets a search range corresponding to the coordinates of the encoding target block B51 in the non-reference image P52 adjacent to the encoding target image P51, and performs a motion search. At this time, the primary searcher 6 evaluates the motion vector using only the pixels of the block that the neighborhood searcher 8 has determined not to detect the motion vector in the prior neighborhood search. In the example of FIG. 5, among the blocks obtained by dividing the macroblock into four, the upper two blocks are blocks in which the neighborhood searcher 8 has detected a motion vector in the prior neighborhood search, and the primary searcher 6 uses these pixels. Not used for motion vector evaluation. On the other hand, the lower two blocks are blocks that the neighbor searcher 8 has determined to be unable to detect the motion vector in the prior neighbor search, and the primary searcher 6 evaluates the motion vector using these pixels.

Next, the primary searcher 6 sets a search range centered on the coordinates indicated by the search result R52 in the non-reference image P52 in the non-reference image P53 adjacent to the non-reference image P52, and performs a motion search. Again, the primary searcher 6 evaluates the motion vector using the pixels of the lower two blocks.
Further, the primary searcher 6 sets a search range centered on the coordinates indicated by the search result R53 in the non-reference image P53 to the reference image P54 adjacent to the non-reference image P53, and performs a motion search. Again, the primary searcher 6 evaluates the motion vector using the pixels of the lower two blocks. The primary searcher 6 sets the search result R54 obtained by this motion search as the final motion search result (primary motion vector).
As described above, when the primary searcher 6 evaluates the motion vector, the primary searcher 6 performs evaluation by excluding the pixel of the block in which the neighborhood searcher 8 has detected the motion vector in the prior neighborhood search. When the macroblock includes a moving object image and a background image, the movement of the moving object can be tracked without being influenced by the background image.

FIG. 6 is a flowchart illustrating a processing procedure in which the motion vector detection apparatus 100 detects a motion vector for the encoding target macroblock C121.
In step S11, the search center indicator 7 outputs a zero vector to the neighborhood searcher 8, and the neighborhood searcher 8 divides the macro block C121 for encoding into 4 blocks of 8 × 8 pixels, For each block, a motion search is performed around the zero vector of the reference image C51 read from the image cache memory 5.
In step S12, the search center indicator 7 outputs the motion vector prediction value to the neighborhood searcher 8, and the neighborhood searcher 8 performs the motion vector of the reference image C51 for each block obtained by dividing the macro block C121 to be encoded. A motion search is performed around the predicted value. The neighborhood searcher 8 outputs to the primary searcher 6 information C82 indicating a block for which a motion vector could not be detected in steps S11 and S12.

In step S13, the image reducer 12 generates an image C122 in which the encoding target macroblock C121 is reduced to half the number of pixels both vertically and horizontally, and outputs the image C122 to the primary searcher 6. The primary searcher 6 Using the reduced image C122, a motion search on the reduced reference image C41 read from the reduced image cache memory 4 is performed in units of macroblocks, and a primary motion vector C61 is generated and output to the neighborhood searcher 8. At that time, the primary searcher 6 evaluates the motion vector using the pixels of the block in which the neighborhood searcher 8 could not detect the motion vector in steps S11 and S12, and based on the evaluation result, the primary motion vector C61 is detected.
In step S14, the neighborhood searcher 8 performs a motion search around the primary motion vector C61 output by the primary searcher 6 for each block for which a motion vector could not be detected in steps S11 and S12. Specifically, the neighborhood searcher 8 sets a search range corresponding to the primary motion vector on the reference image C51 (that is, a position shifted within a predetermined amount of shift between the primary motion vector and a predetermined pixel). Perform motion search (as a candidate). The neighborhood searcher 8 outputs the motion vector C201 detected in step S11, step S12, or step S14 to the outside of the motion vector detection device 100. Specifically, for a block for which a motion vector has been detected in step S11 or step S12, the motion vector is output. On the other hand, the motion vector detected in step S14 is output for the block whose motion vector could not be detected in step S11 or step S12. Thereafter, the process of detecting a motion vector for the encoding target macroblock C121 ends.

As described above, the motion vector detection apparatus 100 has a block size smaller than the macroblock size, for example, zero in block units of 8 × 8 pixels, prior to the primary search in units of macroblocks of 16 × 16 pixels. A neighborhood search around the vector and the motion vector prediction value is performed in advance. If there is a block in which a motion vector can be detected, the primary search is performed by excluding the pixel of the block from the calculation of the sum of absolute differences.
Since the motion vector detection apparatus 100 performs a primary search in units of macroblocks, it is not easily affected by disturbances such as noise in an image, and the amount of computation is relatively small even when performing a telescopic search. In addition, since a block in which a motion vector was detected in the neighborhood search around the zero vector and the motion vector prediction value is excluded from the calculation of the sum of absolute differences, a primary search is performed. The motion vector can be detected appropriately without being affected by the background.

  Note that the reduction ratio of the reference image used when the primary searcher 6 performs the primary search is not limited to the above-mentioned half of the vertical and horizontal directions. Further, the reference image used when the neighborhood searcher 8 performs the secondary search is not limited to the non-reduced one. For example, the primary searcher 6 performs a primary search using a reference image with a reduction ratio of 1/4 in the vertical and horizontal directions, and the neighborhood searcher 8 performs a secondary search using a reference image with a reduction ratio of 1/2 in the vertical and horizontal directions. You may do it. Alternatively, the primary searcher 6 performs a primary search in units of integer pixels using a non-reduced reference image, and the neighborhood searcher 8 uses a non-reduced reference image to perform a second search in units of 0.5 pixels or 0.25 pixels. A next search may be performed.

Note that a program for realizing all or part of the functions of the motion vector detection apparatus 100 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed. You may process each part by. Here, the “computer system” includes an OS and hardware such as peripheral devices.
Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.
The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory in a computer system serving as a server or a client in that case, and a program that holds a program for a certain period of time are also included. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes design changes and the like within a scope not departing from the gist of the present invention.

  The present invention provides motion prediction such as a video encoder that encodes a moving picture digital signal in accordance with the Moving Picture Experts Group (MPEG) method or the International Telecommunication Union Telecommunication Standardization Sector (ITU-T) H series method. In the video encoder that performs the above, it is suitable for use in a motion vector detection device, a motion vector detection method, and a program that output a motion vector.

DESCRIPTION OF SYMBOLS 1 Image memory 2 Search range cutout device 3, 12 Image reducer 4 Reduced image cache memory 5 Image cache memory 6 Primary searcher 7 Search center indicator 8 Neighbor searcher 100 Motion vector detection apparatus 611 Difference absolute value sum calculator 621 Difference Absolute value sum addition controller 631 Difference absolute value sum adder 641 Evaluator

Claims (3)

A motion vector detection device that detects a motion vector indicating a position of a region corresponding to a block on a reference image for each block obtained by dividing a macro block of an input moving image,
For each block obtained by dividing the macro block, and surrounding the zero vector indicating the position of the block on the reference image, the motion of the surrounding motion vector prediction value determined on the basis of the motion vectors of blocks adjacent to the block for each candidate vector, and calculates a motion vector evaluation value based on a difference between pixels of the pixel and the reference picture of the block, and determines in advance the vicinity searcher whether the following motion vector evaluation value predetermined threshold ,
The primary motion vector for the macroblock is determined in units of macroblocks based on the difference between the pixel of each block determined by the prior neighborhood searcher as having no motion vector evaluation value equal to or less than a predetermined threshold and the pixel of the reference image . A primary searcher to detect by motion search;
For each block determined by the prior neighborhood searcher as having no motion vector evaluation value equal to or less than a predetermined threshold, a motion vector is detected by performing a motion search around the primary motion vector, and the prior neighborhood searcher For each of the blocks determined as having a motion vector evaluation value equal to or less than a predetermined threshold, a secondary searcher that uses a motion vector candidate whose motion vector evaluation value is equal to or less than a predetermined threshold as a motion vector of the block;
A motion vector detection apparatus comprising: A motion vector detection method of a motion vector detection apparatus that detects a motion vector indicating a position of an area corresponding to a block on a reference image for each block obtained by dividing a macro block of an input moving image,
The macro block for each block obtained by dividing the periphery of the zero vector indicating the position of the block on the reference image, the periphery of the motion vector prediction value determined on the basis of the motion vectors of blocks adjacent to the block, for each of the candidate motion vectors, and calculates a motion vector evaluation value based on a difference between pixels of the pixel and the reference picture of the block, pre-neighbor searching step determines the presence or absence of a predetermined threshold value or less of the motion vector evaluation value When,
A primary motion vector for the macroblock is determined in units of macroblocks based on a difference between a pixel of each block determined as having no motion vector evaluation value equal to or less than a predetermined threshold in the prior neighborhood search step and a pixel of the reference image. A primary search step detected by motion search of
For each block determined to have no motion vector evaluation value equal to or less than a predetermined threshold in the prior neighborhood search step, a motion vector is detected by performing a motion search around the primary motion vector, and the prior neighborhood search For each block that is determined to have a motion vector evaluation value less than or equal to a predetermined threshold in the step, a secondary search using the motion vector candidates whose motion vector evaluation value is less than or equal to the predetermined threshold as the motion vector of the block Steps,
A motion vector detection method comprising: A program for causing a computer to output a motion vector indicating a position of an area corresponding to a block on a reference image for each block obtained by dividing a macro block of an input moving image,
The macro block for each block obtained by dividing the periphery of the zero vector indicating the position of the block on the reference image, the periphery of the motion vector prediction value determined on the basis of the motion vectors of blocks adjacent to the block, for each of the candidate motion vectors, and calculates a motion vector evaluation value based on a difference between pixels of the pixel and the reference picture of the block, pre-neighbor searching step determines the presence or absence of a predetermined threshold value or less of the motion vector evaluation value When,
A primary motion vector for the macroblock is determined in units of macroblocks based on a difference between a pixel of each block determined as having no motion vector evaluation value equal to or less than a predetermined threshold in the prior neighborhood search step and a pixel of the reference image. A primary search step detected by motion search of
For each block determined to have no motion vector evaluation value equal to or less than a predetermined threshold in the prior neighborhood search step, a motion vector is detected by performing a motion search around the primary motion vector, and the prior neighborhood search For each block that is determined to have a motion vector evaluation value less than or equal to a predetermined threshold in the step, a secondary search using the motion vector candidates whose motion vector evaluation value is less than or equal to the predetermined threshold as the motion vector of the block Steps,
For causing the computer to execute.

Images