JP2009021864A - Motion vector searching apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve accuracy of motion search by appropriately predicting a position within a search range and to suppress increase in an entire processing speed by suppressing frequency of write into a search range RAM. <P>SOLUTION: For each processing target block group constituted of continuous N (N is an integer of ≥2) blocks, a reference vector is determined which specifies a position within a search range. In the determination of the reference vector, for each of peripheral block groups positioned around the processing target block group to be processed, presence/absence of variation of a motion vector is determined and from each of processing results, presence/absence of variation of the motion vector in the processing target block group is predicted. In the case that presence of variation is predicted from the prediction result, a vector predetermined as the reference vector is set. In the case that absence of variation is predicted, a representative vector corresponding to each of the peripheral block groups is determined and based on a group of the representative vectors, the reference vector is determined. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a motion vector search apparatus, and more particularly to a technique suitable for use in improving processing speed.

  2. Description of the Related Art Conventionally, a method called motion compensation encoding is generally known as means for compressing and encoding a digital moving image. This is because the encoding target image is divided into rectangular blocks, a motion vector is detected for each block, a prediction image by motion compensation is generated based on the motion vector, and a difference from the encoding target image is calculated. It is to do.

  Further, a method is generally used in which the difference image is subjected to DCT (Discrete Cosine Transform) in units of the blocks, and is quantized and entropy encoded. The search for a motion vector in units of the block includes a code consumption amount (hereinafter referred to as “encoding”) based on a correlation value indicating the degree of matching between an input image and a reference image, motion information of surrounding blocks, and the like. , Encoding cost). A method called block matching, in which a vector predicted to have the lowest encoding cost is defined as a motion vector, is generally used.

  FIG. 6 shows a conceptual diagram of motion search by general block matching. Within the frame currently being processed, block motion search processing is performed in the order shown in FIG. Each block searches for a position having the highest similarity within a search range centered on the position in the reference frame corresponding to the position of the block, and determines a motion vector as a relative position from the position of the processing target block. . Since the position of the subsequent block is right adjacent to the block processed immediately before, the corresponding search ranges generally overlap as shown in FIG.

  Next, a configuration example for performing motion search by general block matching as described above will be described with reference to FIG. In FIG. 7, reference numeral 701 denotes a motion search device, which is connected to a reference frame memory 704 via a bus 703. The reference frame memory 704 stores a reference frame to be referenced for motion search of the current processing target frame, and the reference data is accessed from the motion search device 701 as necessary.

  The motion search apparatus 701 further includes a search range RAM 702 that stores reference data of the motion search range necessary for the motion search process of the current processing target block. When performing the motion search process for a block, the motion search device 701 first reads the reference data of the motion search range from the reference frame memory 704 via the bus 703 and stores it in the search range RAM 702. Then, block data in the current processing target frame is input, sequentially matched with reference data in the search range RAM 702, and a position with the highest similarity is output as a motion vector.

  Here, the writing process to the search range RAM 702 prepared in units of blocks has an overlapping portion between the search range of the block currently being processed as shown in FIG. 7 and the search range of the block processed immediately before. Only the part of the search range that is newly required is performed.

  Note that the position of the search range assigned to each block need not be a position centered on the position of the processing target block, and the center of the search position may be shifted according to the situation. For example, there is a method of detecting a vector (GMV) representing the motion of the entire image by some method and shifting the center of the search position from the position of the processing target block to the position indicated by the GMV. This is because the motion in the frame is almost uniformly distributed by panning, the panning speed is high, and the motion vector exceeds the search range when the position of the processing target block is set at the center of the motion search range. Especially effective in cases. However, in the above-described method, the search range is shifted uniformly in the entire frame. Therefore, if there are many portions having movements different from the entire movement in the frame, there is a problem that the motion search accuracy is lowered.

  Therefore, in order to solve this problem, for example, Patent Document 1 evaluates the movement of the entire block group including a plurality of blocks. When the movement of the entire block group can be detected, the search range of the next block group to be searched is shifted in accordance with the movement, and when it cannot be detected, the search range is shifted to a predetermined position. Thereby, it is possible to cope with a movement different from the entire movement in the frame to some extent.

Japanese Patent No. 3335137

  However, in the above-described conventional technology, the position of the search range of the block group to be subjected to motion search processing is based on the motion of the entire block group processed immediately before. For this reason, there is a problem in that it is insufficient as information for predicting an appropriate position of the search range. This is because, as long as the processing order of the blocks is performed in the raster scan order as shown in FIG. 7, the position of the block group processed immediately before the position of the processing block group can only take the left side or the top side. . Therefore, it depends on the correlation of either the horizontal or vertical motion vector. In addition, it is impossible to obtain a correlation with a motion vector of a block existing at a position distant from the adjacent block group, and sufficient prediction cannot be performed.

  In addition, when the block group is made small so as to correspond to a part that moves differently from the whole movement in the frame in a finer unit, an appropriate position of the search range is predicted as described above. Can not get enough information for. For this reason, in places where the motion vectors vary greatly and the correlation between the motion vectors is low, the position setting of the search range is switched more frequently than necessary, and the prediction is likely to deviate and the accuracy of the motion search is reduced. There's a problem.

  Further, when the position setting of the search range is frequently switched in this way, the ratio of occurrence of overlap between the search range of the block currently being processed as shown in FIG. 7 and the search range of the block processed immediately before decreases. . This increases the frequency of writing to the search range RAM 702 and increases the overall processing time.

  In view of the above-described problems, the present invention can improve the accuracy of motion search by appropriately predicting the position of the search range, and can suppress increase in the overall processing speed by suppressing the frequency of writing to the search range RAM. The purpose is to do so.

  The motion vector search device of the present invention is a motion vector search device that divides a frame into a plurality of blocks and determines a motion vector by taking a matching with a pixel group in a reference frame for each of the divided blocks. A first unit block comprising processing block counting means that counts up for each processing of the block, and N (N is an integer of 2 or more) consecutive blocks based on the count value of the processing block counting means A reference vector determining means for determining a reference vector for each group; and evaluation by block matching at a plurality of search points within a search range of motion vectors defined by the reference vector determined by the reference vector determining means; Based on the results of block matching evaluation Motion vector determination means for determining, wherein the reference vector determination means includes a plurality of second blocks composed of a plurality of the blocks located around the first unit block group for performing motion vector search processing. In each unit block group, the presence / absence of motion vector variation is determined, and based on the respective determination results, the presence / absence of motion vector variation in the first unit block group is predicted. In the case of prediction, a predetermined vector is set as the reference vector, and in the case of prediction that there is no variation, a corresponding representative vector is obtained for each of the peripheral second unit block groups, and the group of representative vectors The reference vector is determined based on the following.

  According to the present invention, it is possible to improve the accuracy of motion search by appropriately predicting the position of the search range, and to suppress an increase in the overall processing speed by suppressing the frequency of writing to the search range RAM.

(First embodiment)
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram illustrating a configuration example of a motion search device (motion vector search device) according to the present embodiment.
The motion search apparatus according to the present embodiment includes a processing block count unit 101, a reference vector determination unit 102, and a motion vector determination unit 103, and is further connected to a reference frame memory 104 as a memory for storing a reference frame.

  The processing block counting unit 101 is configured to be counted up each time a block is processed, counts the number of processed blocks, and outputs the counted number to the reference vector determination unit 102. Here, the “block” is a unit of motion vector search composed of a plurality of pixel groups obtained by dividing a frame into rectangular areas. Usually, a block having a size of about 4 × 4 pixels to 16 × 16 pixels is often used.

  The reference vector determination unit 102 determines a reference vector serving as a reference for the search range of the motion vector search assigned to the block group to be processed based on the already determined motion vector group output from the motion vector determination unit 103. . Then, based on the processing block count value output from the processing block counting unit 101, the continuous N (N is an integer of 2 or more) blocks are recognized as a unit of one block group, and the motion vector determining unit 103 is processed. Is output.

  Here, the reference vector will be described with reference to FIG. When performing motion vector search processing for a block to be processed, a search range is set before starting the search. If the search range is set based on the processing target block, the portion indicated as “search range centered on block” in FIG. 3 becomes the search range.

  On the other hand, when the range to be searched is shifted from the position of the processing target block, the shifted displacement is expressed as a vector as a relative position with the position of the processing target block as the origin, This is called a reference vector. The “search range centered on the reference vector” is set based on the position shifted by the reference vector from the position of the processing target block in FIG. Details of the reference vector determination method in the reference vector determination unit 102 will be described later.

  Next, the motion vector determination unit 103 receives a block to be a motion vector search process from the outside, reference data in the reference frame from the reference frame memory 104, and a reference vector output from the reference vector determination unit 102. Thereafter, a motion vector search for the processing target block is performed, a motion vector is determined and output to the external and reference vector determination unit 102.

FIG. 2 is a block diagram illustrating an internal configuration example of the motion vector determination unit 103.
The motion vector determination unit 103 includes a block RAM 1031 that stores a processing target block, and a search range RAM 1032 that stores search range data in a reference frame corresponding to the processing target block. Furthermore, a block matching unit 1033 that performs a motion vector search by block matching processing, determines and outputs a motion vector is provided.

  The block RAM 1031 inputs and stores the processing target block from the outside, and appropriately outputs the data of the processing target block as required by the block matching unit 1033. A search range RAM (search range memory) 1032 stores a reference data group (reference pixel group) set as a search range among the reference data in the reference frame, and appropriately refers to it as needed by the block matching unit 1033. Output data.

  The block matching unit 1033 calculates the position of the search range corresponding to the processing target block based on the input reference vector. Then, only the reference data group included in the position of the search range and new data excluding the reference data already written in the search range RAM 1032 are updated. Further, the block matching unit 1033 reads the data of the processing target block from the block RAM 1031. Then, the degree of similarity is evaluated by matching the read processing target block data with the reference data in the search range corresponding to the processing target block read from the search range RAM 1032.

  This matching process is performed at any time while shifting the search point of the reference data within the search range, and calculates and outputs a motion vector based on the position evaluated as having the highest similarity. As the similarity, for example, a sum of absolute differences (SAD) between pixels is often used.

  Each pixel value of the processing target block (x pixel × y pixel) is set to Org0,0 to Orgx-1, y−1, and each pixel value of the corresponding reference data (x pixel × y pixel) as a matching target is set to Ref0,0. ˜Refx−1, y−1, this SAD value is obtained from the following equation (1).

  That is, the smaller the SAD value, the higher the similarity, and the larger the SAD value, the lower the similarity.

  Returning to the description of FIG. 1, the reference frame memory 104 stores a reference frame corresponding to the processing target frame, and appropriately outputs reference data to the motion vector determination unit 103 as necessary by the motion vector determination unit 103. .

Next, details of a reference vector determination method in the reference vector determination unit 102 will be described with reference to a flowchart illustrating an example of a reference vector determination procedure in FIG.
First, in step S <b> 1, the reference vector determination unit 102 first determines whether or not there is a variation in each motion vector in a plurality of peripheral block groups with respect to the processing target block group.

  Here, the “processing target block group (first unit block group)” and the “plural peripheral block group (second unit block group)” are shown in FIG. The “processing target block group” indicates a rectangular area including a block to start processing from now on, and means a block group to be subjected to motion search processing. In addition, peripheral block groups A to F corresponding thereto correspond to “a plurality of peripheral block groups”.

  However, the size of the peripheral block group, the size of the processing target block group, the number of the peripheral block groups, and the positional relationship with respect to the processing target block group are merely examples, and may be changed as necessary. In addition, the determination of the motion vector variation is, for example, “no variation” if the direction and size of the motion vector determined in the past possessed by each peripheral block group are within a predetermined range. Is determined. Otherwise, it is determined that there is “variation”.

  Next, in step S2, presence / absence of motion vector variation in the processing target block group is predicted based on the determination result of motion vector variation in the plurality of peripheral block groups. In this prediction method, among the plurality of peripheral block groups, among the peripheral block groups adjacent to the processing target block group, the ratio of the number of peripheral block groups determined as “no variation” of the motion vector is: If the value is equal to or greater than a predetermined value, it is predicted that “no variation”. The peripheral block groups are A to D. On the other hand, if not, it is predicted that “there is variation”.

  Next, in step S3, it is determined whether the prediction result in step S2 is “with variation” or “without variation”. If the result of this determination is “no variation”, the process proceeds to step S4 to be described later, and if it is “with variation”, the process proceeds to step S6 to be described later.

  In step S4, a representative vector is obtained for each of the plurality of peripheral block groups A to F. The representative vector can be obtained, for example, by obtaining an average vector of motion vectors of each block belonging to each block group. Then, when step S4 is completed, the process proceeds to step S5.

  In step S5, a reference vector in the processing target block group is obtained from a plurality of representative vectors. In this embodiment, as a method for obtaining a reference vector from a plurality of representative vectors, the representative vectors are respectively weighted according to the relative positions with respect to the processing target block group, and the reference vectors are based on the weighted representative vector group. To decide.

  For example, if the representative vectors of the peripheral block groups A to F are TMVA to TMVF, the relative position relationship of the peripheral block groups A to F with respect to the processing target block group is 1/6, 1/4, 1/6, 1 / A weighting coefficient is assigned such as 4, 1/12, 1/12. Note that these are examples of results of assigning smaller values to those farther from the processing target block group. The weighting coefficients assigned in this way are multiplied by the corresponding representative vectors, and the average vector thereof is set as the reference vector.

  Further, the strength of correlation in the horizontal and vertical directions may be examined and the result reflected in the weighting coefficient. For example, the correlation between the representative vectors of the peripheral block groups E and B and the correlation between the representative vectors of the peripheral block groups F and D are examined. If the former correlation is stronger, the weighting coefficients of the peripheral block groups B and E are set. , Larger than the weighting coefficients of the peripheral block groups F and D. On the other hand, if it is not so, the reference vector can be predicted more accurately by setting it to be smaller.

  In step S4, the number of blocks constituting the processing target block group may be adaptively changed based on representative vectors of a plurality of peripheral block groups. For example, when the directions and sizes of the representative vectors of the peripheral block groups A to C are all within a predetermined range, the number of blocks constituting the processing target block group is set as follows. That is, it is set equal to the total value of the number of blocks constituting the peripheral block group B and the number of blocks constituting the peripheral block group C. By doing so, the update cycle of the reference vector can be adaptively changed according to the state of motion in the frame, and the process for determining the reference vector can be omitted while it is not necessary to update the reference vector. it can.

  When the number of blocks constituting the processing target block group is adaptively changed, a lower limit value of the number of blocks constituting the processing target block group is set according to the integrated value of the writing process to the search range RAM 1032. You may do it. For example, when the integrated value of the writing process to the search range RAM 1032 becomes larger, the lower limit value is set larger to reduce the reference vector update cycle. In this way, even when the write processing time of the search range RAM 1032 affects the entire processing time, it is possible to guarantee the allowable processing time by providing the lower limit value.

  On the other hand, in step S6, a predetermined vector is set as the reference vector in the processing target block group. For example, the origin is set as the reference vector. This is because it is difficult to properly predict the reference vector based on the surrounding motion vectors in the portion where the correlation between the motion vectors is low, so that setting the origin as the center of the search range has the best balance Is taken into account. As described above, it is possible to predict the reference vector more accurately by taking into account the information of the neighboring block groups at various positions with respect to the processing target block group.

  As described above, in the present embodiment, it is possible to obtain the correlation of motion vectors in both the horizontal and vertical directions. In addition, it is possible to obtain a correlation with not only the adjacent block group but also a motion vector of a block that exists at a more distant position, so that a sufficient search range setting position can be predicted, and the motion vector Search accuracy is improved.

  Further, even when the block group is made small, sufficient motion vector correlation information can be obtained. As a result, even in a place where the motion vectors vary greatly and the correlation between the motion vectors is low, the position setting of the search range does not change more frequently than necessary, and the appropriate position of the search range can be predicted. In addition, a motion vector search with sufficient accuracy can be performed, an increase in write access to the search range RAM can be suppressed, and a decrease in the overall processing speed can be suppressed.

(Other embodiments according to the present invention)
Each means constituting the motion vector search apparatus and each step of the motion vector search method in the embodiment of the present invention described above can be realized by operating a program stored in a RAM or ROM of a computer. This program and a computer-readable recording medium recording the program are included in the present invention.

  Further, the present invention can be implemented as, for example, a system, apparatus, method, program, or recording medium. Specifically, the present invention may be applied to a system including a plurality of devices. The present invention may be applied to an apparatus composed of a single device.

  The present invention includes a case where a software program (in the embodiment, a program corresponding to the flowchart shown in FIG. 4) for realizing the functions of the above-described embodiments is directly or remotely supplied to a system or apparatus. This includes the case where the system or the computer of the apparatus is also achieved by reading and executing the supplied program code.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention.

  In that case, as long as it has the function of a program, it may be in the form of object code, a program executed by an interpreter, script data supplied to the OS, and the like.

  Examples of the recording medium for supplying the program include a flexible disk, a hard disk, an optical disk, and a magneto-optical disk. Further, there are MO, CD-ROM, CD-R, CD-RW, magnetic tape, nonvolatile memory card, ROM, DVD (DVD-ROM, DVD-R) and the like.

  As another program supply method, there is a method of connecting to a homepage on the Internet using a browser of a client computer. The computer program itself of the present invention or a compressed file including an automatic installation function can be downloaded from the homepage by downloading it to a recording medium such as a hard disk.

  It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the present invention.

  As another method, the program of the present invention is encrypted, stored in a recording medium such as a CD-ROM, distributed to users, and encrypted from a homepage via the Internet to users who have cleared predetermined conditions. Download the key information to be solved. It is also possible to execute the encrypted program by using the key information and install the program on a computer.

  Further, the functions of the above-described embodiments are realized by the computer executing the read program. Furthermore, based on the instructions of the program, an OS or the like running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments can be realized by the processing.

  As another method, the program read from the recording medium is first written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Then, based on the instructions of the program, the CPU or the like provided in the function expansion board or function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are also realized by the processing.

It is a block diagram which shows the structural example of the motion search apparatus in embodiment of this invention. It is a block diagram which shows the detailed structural example of the motion vector determination part in embodiment of this invention. In an embodiment of the present invention, it is a figure showing a concept of a block, a reference vector, and a search range. It is a flowchart which shows an example of the process sequence which determines the reference | standard vector in embodiment of this invention. In an embodiment of the present invention, it is a figure showing a processing object block group and a plurality of peripheral block groups. It is a figure which shows the outline | summary of the motion search by a general block matching. It is a block diagram which shows the structural example for performing the motion search by general block matching.

Explanation of symbols

101 processing block count unit 102 reference vector determination unit 103 motion vector determination unit 1031 block RAM
1032 Search range RAM
1033 Block matching unit 104 Reference frame memory

Claims (6)

A motion vector search device that divides a frame into a plurality of blocks and determines a motion vector by matching with a pixel group in a reference frame for each of the divided blocks.
Processing block counting means for counting up for each processing of the block;
Reference vector determining means for determining a reference vector for each first unit block group composed of N (N is an integer of 2 or more) consecutive blocks based on the count value of the processing block counting means;
Evaluation is performed by block matching at a plurality of search points within a motion vector search range defined by a reference vector determined by the reference vector determination means, and a motion vector is determined based on at least the result of the block matching evaluation. Motion vector determining means,
The reference vector determination unit is configured to detect a variation in motion vector in each of a plurality of second unit block groups including a plurality of blocks located around the first unit block group that performs a motion vector search process. The presence / absence is determined, the presence / absence of motion vector variation in the first unit block group is predicted from each determination result, and when it is predicted that there is variation based on the prediction result, the reference vector is determined in advance. If a vector is set and it is predicted that there is no variation, a corresponding representative vector is obtained for each of the peripheral second unit block groups, and the reference vector is determined based on the group of representative vectors. A featured motion vector search device.   Prediction of the presence or absence of motion vector variation in the first unit block group by the reference vector determining means is composed of a plurality of the blocks located around the first unit block group for performing motion vector search processing. Among the plurality of second unit block groups, the second unit determined to have no motion vector variation among the second unit block groups adjacent to the first unit block group. The motion vector search apparatus according to claim 1, wherein when the ratio of the number of block groups is equal to or greater than a predetermined value, it is predicted that there is no variation, and otherwise, it is predicted that there is variation.   The reference vector determining means, when the prediction result of the presence or absence of motion vector variation in the first unit block group shows no variation, displays a representative vector corresponding to each of the peripheral second unit block groups. 2. The method according to claim 1, wherein the representative vector is weighted according to a relative position with respect to the first unit block group, and the reference vector is determined based on the weighted representative vector group. 3. The motion vector search device according to 1 or 2.   The reference vector determining means, when the prediction result of the presence or absence of motion vector variation in the first unit block group shows no variation, displays a representative vector corresponding to each of the peripheral second unit block groups. 3. The motion vector search apparatus according to claim 1, wherein the N is adaptively set and the reference vector is determined based on the representative vector group.   The motion vector determination means includes a search range memory for storing a plurality of reference pixel groups within the motion vector search range of the block, and is necessary for the motion search of the processing target block when performing the motion search of the block. The motion vector search according to any one of claims 1 to 4, wherein only a pixel group that needs to be updated is written in the search range memory among reference pixel groups within a range of motion vector search to be apparatus.   6. The motion vector search apparatus according to claim 5, wherein the lower limit value of N is adaptively set in accordance with a write amount to the search range memory.

Images