JP5754120B2 - Motion vector detection apparatus, motion vector detection method and program - Google Patents

Description

  The present invention relates to a motion vector detection device, a motion vector detection method, and a program for performing moving image coding.

  Since moving image data has a large amount of data, compression encoding processing is often performed when it is transmitted from a transmission device to a reception device or stored in a storage device. MPEG (Moving Picture Experts Group) -2 / MPEG-4 established by ISO / IEC (International Standardization Organization / International Electrotechnical Commission) is widely used as a typical moving image compression encoding method. In addition, H.264 / AVC (Advanced Video Coding), which is formulated by ITU (International Telecommunication Union) and ISO / IEC MPEG, is known.

  These compression coding schemes are hybrid coding schemes that combine motion compensation interframe prediction and orthogonal transform. In motion compensation interframe prediction, a frame to be encoded is divided into blocks each consisting of m horizontal pixels and n vertical pixels, and a block (reference block) most similar to each block (encoded block) is selected from the reference frames. To detect. Then, the encoded block itself is directly encoded by encoding the position (motion vector) of the reference block and the difference image (difference block) between the encoded block and the reference block as information for reproducing the encoded block. The amount of information will be greatly reduced. In order to use motion compensation interframe prediction, it is necessary to detect a motion vector for each coding block.

  In the motion vector detection process, an evaluation value is calculated for each candidate motion vector, and a point giving the smallest evaluation value (a point where the predicted image is most similar to the processing target block) is calculated as a motion vector. As an evaluation value, SAD (sum of absolute differences) is generally used.

  The amount of calculation required for the motion vector detection process is enormous, and occupies a large amount with respect to the total amount of calculation of the moving image encoding process. Thus, it is known to reduce the amount of calculation by providing a threshold value truncation process in the motion vector detection process. SAD is sequentially calculated for each motion vector candidate, but when a motion vector candidate having an SAD smaller than the threshold is found on the way, the SAD calculation for the remaining candidates is stopped, and a motion vector that gives an SAD smaller than the threshold Candidates are calculated as motion vectors.

  By the way, MPEG2 and H.264. In H.264 / AVC, it is allowed to select a reference block from a plurality of reference frames. For example, H.M. In H.264 / AVC, it is allowed to select an optimal reference frame for each block from a maximum of 15 reference frames. Therefore, H.H. In motion vector detection processing in H.264 / AVC, it is necessary to determine an optimal motion vector from among a plurality of reference frames and from among a large number of motion vector candidates for each reference frame. The amount of computation required is even greater.

  Even when a method is used in which the SAD calculation for the remaining candidates is stopped when a motion vector candidate having an SAD smaller than the threshold is found and a motion vector candidate that gives an SAD smaller than the threshold is used as a motion vector, The amount of computation increases.

JP 2001-145109 A

  Therefore, in motion vector detection for a plurality of reference frames, it is required that an appropriate reference frame can be selected without reducing image quality while reducing the amount of motion vector detection.

  According to the embodiment, an appropriate reference frame can be selected even when the amount of motion vector detection is reduced.

  According to the first aspect of the present invention, a similarity calculation unit that calculates the similarity between blocks, a determined reference frame data memory that stores information on a reference frame of an already determined similar block, and a processing target frame A search order determination unit that determines, for each block, the order in which a similar block is searched from among a plurality of reference frames, using information on the reference frame of the already determined similar block stored in the determined reference frame data memory. And a search unit that searches for a block whose similarity satisfies a predetermined threshold in a reference frame selected according to the determined order.

  According to a second aspect of the present invention, there is provided a motion vector detection method for detecting a similar block having the highest similarity from a plurality of reference frames for each block in the processing target frame. For each block, the information on the reference frame of the similar block that has already been determined for the block other than this block is used to determine the order in which the similar blocks are searched from the plurality of reference frames, and are selected according to the determined order. Motion vector detection that calculates the similarity for each block of the reference frame, stops the search when the calculated similarity satisfies a predetermined threshold, and determines a block whose similarity exceeds the predetermined threshold as a similar block A method is provided.

  Further, according to the third aspect of the present invention, for each block in the processing target frame, the computer uses the information on the reference frame of the similar block that has already been determined for the block other than this block. The order in which similar blocks are searched is determined, the similarity is calculated for each block of the reference frame selected according to the determined order, and the search is stopped when the calculated similarity exceeds a predetermined threshold. A program is provided that operates to determine a block whose similarity satisfies the predetermined threshold as a similar block.

  According to the embodiment, an optimal reference frame can be found at an early stage after searching for similar blocks, and the amount of calculation can be reduced.

FIG. 1 is a diagram illustrating a schematic configuration of an encoding apparatus according to an embodiment. FIG. 2 is a flowchart showing the motion vector detection process. FIG. 3 is a diagram illustrating processing target frames in a series of frames of video signal data. FIG. 4 is a diagram illustrating a method for determining a threshold value when it is determined that they are similar. FIG. 5 is a diagram illustrating one method for determining the selection order of reference frames. FIG. 6 is a diagram illustrating an example of reference frames determined for the front peripheral block. FIG. 7 is a diagram illustrating an example in which a reference frame is determined from a larger number of front side peripheral blocks. FIG. 8 is a diagram illustrating an example of a search range when the selected reference frame is the previous frame in the processing target frame. FIG. 9 is a diagram showing the selection order of search target points in the search range.

  FIG. 1 is a diagram illustrating a schematic configuration of an encoding apparatus according to an embodiment.

  The encoding apparatus according to the embodiment includes an encoding unit 1, a frame memory 2, and a signal input unit 3. The encoding apparatus according to the embodiment is realized, for example, by operating a computer including a processor, a DSP, a memory, and the like with an installed program.

The signal input unit 3 receives video signal data sequentially supplied from the outside and supplies it to the frame memory 2. The frame memory 2 stores the received video signal data. The frame memory 2 has a capacity for storing data of a plurality of frames used for the encoding process among the video signal data, and discards and uses the data of the frames that are no longer necessary for the encoding process. Replace with new frame data as needed. The encoding unit 1 accesses data of a plurality of frames stored in the frame memory 2, performs encoding processing of video signal data, and outputs encoded encoded data.
The encoding unit 1 includes a motion vector detecting unit 10 and a transform encoding unit that performs transform encoding processing such as DCT, which is a kind of orthogonal transform, using information on similar blocks detected by the motion vector detecting unit 10. Have. In other words, the encoding apparatus according to the embodiment performs an encoding process using a hybrid encoding scheme that combines motion compensation interframe prediction and transform encoding. Since the processing of the transform coding unit is widely known, description thereof is omitted here.

  The motion vector detection unit 10 includes a similarity calculation unit 11, a search unit 12, a determined reference frame data memory 13, and a search order determination unit 14.

  The similarity calculation unit 11 calculates SAD (sum of absolute differences) between blocks. Since this process is widely known, a description thereof will be omitted.

  The search unit 12 searches for a similar block from among a plurality of reference frames for each block in the processing target frame, and outputs the identification number of the reference frame to which the similar block belongs and the coordinates of the similar block in the reference frame.

  The determined reference frame data memory 13 stores information on the reference frame of the similar block determined by the search unit 12.

  For each block in the processing target frame, the search order determination unit 14 uses the information regarding the reference frame of the already determined similar block stored in the determined reference frame data memory 13, and uses the similar information from the plurality of reference frames. Determine the order in which to search for blocks.

  The search unit 12 selects a reference frame according to the order determined by the search order determination unit 14, and searches for a block in which the similarity exceeds a predetermined threshold in the selected reference frame.

  Next, motion vector detection processing in moving picture coding of the coding apparatus according to the present embodiment will be described.

  FIG. 2 is a flowchart showing the motion vector detection process.

  Here, as shown in FIG. 3, there are a series of frames # 0 to # 5 of video signal data, and a frame 22 of # 3 is a processing target frame. As mentioned above, H.M. In H.264 / AVC, a maximum of 15 reference frames can be used as described above. FIG. 2 shows motion vector detection processing of one block in the frame 22 of # 3 that is a processing target frame.

  In step S0, the variable miniSAD is initialized with the maximum value MAX. The maximum value MAX may be a value larger than the maximum value that can be taken by SAD (sum of absolute differences) between blocks.

  In step S1, a threshold value Th when determining that they are similar is set. There are various methods for determining the threshold Th.

  In the first method, in a past frame in which motion vector detection has already been completed, a value obtained by multiplying the average value of SAD corresponding to the motion vector obtained by motion vector detection of each block by 1.1 is set as Th. To do.

  In the second method, a value obtained by multiplying the average value of SAD of peripheral blocks of the processing target frame 22 for which motion vector detection has already been completed by 1.1 is set as Th.

  FIG. 4 is a diagram for explaining these methods. In FIG. 4, when a motion vector is detected for the block 30 of the target frame 22, the motion vector detection is completed for all the blocks in the frame before the frame 21 before the target frame 22, and the SAD is calculated. In the first method, the average value of all blocks in the previous frame is calculated and multiplied by 1.1. The average value of SAD is calculated using, for example, a cumulative adder. In addition, since motion vector detection is completed and SAD is calculated also about the block before the block 30 of the object frame 22, you may make it include them.

  In the second method, in the target frame 22, motion vector detection is completed for the front blocks A, B, C, and D adjacent to the block 30 that detects the motion vector, and the SAD of the blocks A, B, C, and D is completed. Is calculated. Here, the peripheral blocks A, B, C, and D are referred to as front peripheral blocks with respect to the block 30. In the second method, the average value of SAD of the front peripheral blocks A, B, C, and D is calculated, and a value obtained by multiplying the value by 1.1 is set as Th. In this case, it is necessary to store SADs of the front peripheral blocks A, B, C, and D. For example, a memory that stores SAD for two lines of blocks is provided and updated as needed. Although FIG. 4 shows a case where there are four front peripheral blocks, the number is not particularly limited.

  Returning to FIG. 2, in step S <b> 2, the order of selecting a plurality of reference frames is determined.

  FIG. 5 is a diagram illustrating one method for determining the selection order of reference frames.

  The detection of the motion vector of a block in one frame is performed in the raster scan order. In FIG. 5, on the screen of the processing target frame 22, the motion vector detection is performed in order from the left side to the right side block in each block in the row from the top row to the bottom row. Therefore, as shown in FIG. 5A, when detecting the motion vector of the block 30, the peripheral blocks A, B, C, and D are already determined as the motion vector and the reference frame. Here, the peripheral blocks A, B, C, and D are referred to as front peripheral blocks with respect to the block 30.

  The selection order of the reference frames is determined based on the reference frames in the front peripheral blocks A, B, C, and D. For example, the selection order of the reference frames is the order of the reference frames determined for the front peripheral blocks A, B, C, and D.

  FIG. 6 shows an example of reference frames determined for the front peripheral blocks A, B, C, and D. Note that the processing target frame is # 3. In this example, it is determined that block A is reference frame # 1, block B is reference frame # 1, block C is reference frame # 1, and block D is reference frame # 2. In this case, it is determined that the most reference frame # 1 is selected first and the frame # 2 is selected second. Note that the processing target frame is # 3.

  As shown in FIG. 5B, when the block 30 for detecting a motion vector is located at the upper end, the left end, or the right end of the processing target frame 22, there may be a case where some or all of the front peripheral blocks do not exist. . In this case, the reference frame selection order is determined by taking the majority of the reference frames determined among the existing front peripheral blocks. When the block 30 for detecting the motion vector is located at the upper left corner of the processing target frame 22, there is no front peripheral block. In this case, you may determine from the reference frame of the surrounding block also including the back side of the block of the corresponding position (upper left corner) of a front frame.

  5 and 6 show the case where there are four front peripheral blocks, the block for determining the reference frame is not particularly limited, and reference frames are obtained from a larger number of front peripheral blocks. You may decide. For example, as illustrated in FIG. 7, the processing target frame may be determined in descending order of reference frames already determined in the block positioned on the front side of the scan with respect to the block 30 for detecting the motion vector. Moreover, in (A) of FIG. 5, you may determine only with the reference frame of the block A among the front side periphery blocks. When the same maximum number of reference frames exist in the front peripheral block, for example, the reference frame selected by block A is selected. When the reference frame is selected only from the block A, it is only necessary to store the reference frame of the block A processed immediately before, so that the reference frame selection process is simplified. Alternatively, the reference frame may be selected by giving priority to the reference frame selected by the blocks A and B that are close to the block 30 that detects the motion vector. In this case, with respect to the blocks A and B, it is possible to perform the double weighting of the reference frames of the blocks C and D.

  In step S3, one reference frame is selected from a plurality of reference frame candidates # 0 to # 5 as shown in FIG. 3 according to the selection order determined in step S2.

  In step S4, a search target point is selected. Here, the case where the full search method is used will be described as an example. However, in the present invention, the search target point setting method is not limited to the full search method, but includes various methods such as a three-step method, a hexagon search method, a gradient method, and a hierarchical search method. A method can be selected and used.

  FIG. 8 shows an example of a search range when the processing target frame 22 is frame # 3 and the selected reference frame 21 is frame # 2. The block 40 of the reference frame 21 is a block having the same coordinates as the coordinates in the processing target frame 22 of the block 30 for detecting a motion vector. The range indicated by reference numeral 41 is the search range, the horizontal width is H pixels, and the vertical width is V pixels. The block 40 is located at the center of the search range 41.

  FIG. 9 shows the selection order of search target points in the search range 41. In the first step S4, the upper left block 50 of the search range 41 is selected as a motion vector candidate. When performing the second selection in step S4, a block 51 obtained by shifting the block 50 to the right by one pixel is selected as a motion vector candidate. When performing the third and subsequent selections in step S4, blocks obtained by shifting the block 50 to the right by one pixel are sequentially selected as motion vector candidates and selected up to the block 61 at the upper right end. Thereafter, returning to the left end, a block obtained by shifting the upper left block 50 downward by one pixel is selected as a motion vector candidate. After that, a block shifted by one pixel sequentially in the right direction is selected as a motion vector candidate, and when it is selected up to the right end, it returns to the left end and is selected by shifting one pixel downward, and the same selection is repeated. Then, in the last row, the block 62 is selected, and then a block shifted by one pixel in the right direction is selected as a motion vector candidate, and the process ends when the block 63 at the right end is selected. As described above, in step S4, each time the step is repeated, a block is selected like a raster scan. As described above, since the search range 41 is H × V pixels, in order to select all the blocks in the search range 41, step S4 is repeated H × V times.

  In step S5, SAD (absolute difference sum) is calculated for the block of the point selected in step S4. The calculation formula of SAD is shown in the following formula.

  Here, c (i, j) indicates the value of a pixel existing at the i-th position in the horizontal direction and the j-th position in the vertical direction of the processing target block, and r (x + i, y + j) indicates (i of the processing target block in the reference frame. , J) represents the value of a pixel at a position separated by x in the horizontal direction and j in the vertical direction from the pixel at the position. Further, n is the number of pixels in the horizontal direction of a block (processing target block) from which a motion vector is detected, and m is the number of pixels in the vertical direction of the processing target block.

  In step S6, the calculated SAD is compared with the variable miniSAD. When SAD <miniSAD, miniSAD is replaced with the calculated SAD, variable Ref is replaced with the reference frame selected at step S3, and variable Vec is replaced with the coordinates of the motion vector candidate selected at step S4.

  In step S7, it is determined whether or not the calculated SAD is equal to or less than the threshold value Th. If the threshold value Th is equal to or less than the threshold value Th, the process proceeds to step S10.

  In step S8, it is determined whether or not SAD has been calculated at all search target points in the search range 41 for the selected reference frame. If search target points for which SAD has not been calculated remain, the process returns to step S4, and if not, the process proceeds to step S9.

  When returning to step S4, as described above, in the search range 41, the search target point is shifted like a raster scan to select a block.

  In step S9, it is determined whether the search process has been completed for all reference frames. If the search has been completed for all the reference frames, the process proceeds to step S10. If there are remaining reference frames for which the search has not been completed, the process returns to step S3.

  When returning to step S3, as described above, the next reference frame is selected according to the selection order of the reference frames.

  In step S10, the stored variable Ref is stored and output as the reference frame of the similar block in the determined reference frame 13, and the variable Vec is output as the coordinates of the motion vector candidate selected in step S4.

  As described above, the motion vector detection process for one block is completed.

  In the above embodiment, SAD is used as an index as to whether or not the search target point is appropriate as a motion vector, but the following equation MAD may be used instead.

  Alternatively, a value obtained by adding the coding cost of a motion vector to SAD may be used. In this case, the value compared with the threshold value in step S7 is not SAD but a value obtained by adding the encoding cost of the motion vector to MAD or SAD. The threshold Th set in step S1 is set according to a value obtained by adding the motion vector encoding cost to MAD or SAD.

  As described above, in the present embodiment, the optimal reference frame search is performed by determining the order of reference frames to be searched based on the reference frames selected by the surrounding blocks whose motion vectors have already been determined. In the early stage of the motion vector detection process. Therefore, the detection timing of SAD smaller than the threshold can be advanced, the possibility of omitting a search for a reference frame having a high probability of SAD larger than the threshold can be increased, and the average calculation amount of motion vector detection processing is reduced. be able to. When the same subject object is shown across multiple adjacent blocks, there is a high probability that the same reference frame is the optimal reference frame between adjacent blocks. Thus, it is possible to evaluate the motion vector candidate on the optimal reference frame.

  Although the embodiment has been described above, all examples and conditions described herein are described for the purpose of helping understanding of the concept of the invention applied to the invention and the technology. It is not intended to limit the scope of the invention, and the construction of such examples in the specification does not indicate the advantages and disadvantages of the invention. Although embodiments of the invention have been described in detail, it should be understood that various changes, substitutions and modifications can be made without departing from the spirit and scope of the invention.

Hereinafter, the following additional notes will be disclosed with respect to the embodiment.
(Appendix 1)
A similarity calculator for calculating the similarity between blocks;
A determined reference frame data memory for storing information on the reference frame of the already determined similar block;
For each block in the processing target frame, the information on the reference frame of the already determined similar block stored in the determined reference frame data memory is used to search for the similar block from a plurality of reference frames. A search order determination unit for determining;
A motion vector detecting apparatus comprising: a search unit that searches for a block in which the similarity satisfies a predetermined threshold in the reference frame selected according to the determined order.
(Appendix 2)
The search order determination unit
The motion vector detection device according to supplementary note 1, wherein an order is determined for each block in the processing target frame based on a reference frame of a block that is adjacent to the block in the processing target frame and for which a similar block has already been determined.
(Appendix 3)
The search order determination unit
The movement according to appendix 1, wherein for each block in the processing target frame, an order is determined based on a reference frame of a block close to the processing target frame and a similar block already determined and at a position corresponding to the block. Vector detection device.
(Appendix 4)
A motion vector detection method for detecting a similar block having the highest similarity from a plurality of reference frames for each block in a processing target frame,
For each block in the processing target frame, using information related to a reference frame of a similar block that has already been determined for a block other than the block, determine the order in which the similar block is searched from the plurality of reference frames,
Calculating the similarity for each block of the reference frame selected according to the determined order;
The motion vector detection method, wherein the search is stopped when the calculated similarity satisfies a predetermined threshold, and a block whose similarity satisfies the predetermined threshold is determined as a similar block.
(Appendix 5)
The determination of the search order is as follows.
The motion vector detection method according to supplementary note 4, wherein the order is determined for each block in the processing target frame based on a reference frame of a block that is adjacent to the block in the processing target frame and for which a similar block has already been determined.
(Appendix 6)
The determination of the search order is as follows.
The movement according to supplementary note 4, wherein for each block in the processing target frame, the order is determined based on a reference frame of a block close to the processing target frame and a similar block already determined at a position corresponding to the block. Vector detection method.
(Appendix 7)
Computer
For each block in the processing target frame, using information on the reference frame of the similar block that has already been determined for the block other than the block, the order of searching for the similar block from the plurality of reference frames is determined,
Calculating similarity for each block of the reference frame selected according to the determined order;
A program that operates so that the search is stopped when the calculated similarity satisfies a predetermined threshold, and a block that satisfies the predetermined threshold is determined as a similar block.

DESCRIPTION OF SYMBOLS 1 Encoding part 2 Frame memory 3 Signal input part 4 Transform encoding part 10 Motion vector detection part 11 Similarity calculation part 12 Search part 13 Determined reference frame data memory 14 Search order determination part

Claims (4)

A similarity calculation unit that calculates the similarity between blocks with a plurality of reference frames in units of blocks of the processing target frame;
A determined reference frame data memory for storing information on the reference frame of the similar block with respect to the block of the processing target frame that has already been determined;
For each block in the processing target frame, the similar block for a plurality of reference frames is obtained by using information on the reference frame of the similar block for the block of the already determined processing target frame stored in the determined reference frame data memory. A search order determination unit for determining the order of frames to be searched;
A search unit that searches for a block in which the similarity satisfies a predetermined threshold in the reference frame selected according to the determined order of frames to be searched; and
The search order determination unit
For each block in the process target frame, the search order as of the priority of the reference number is large reference frames increases the reference frame blocks similar block is determined in the reference frame already Te the processing target frame smell Determining a motion vector.   The search order determination unit may increase the priority of a reference frame having a high reference count of reference destination frames of blocks around the same position of the previous frame when there is no block whose reference frame is determined in the processing target frame. The motion vector detection apparatus according to claim 1, wherein a search order is determined. For each block in the frame to be processed, a motion vector detecting method for detecting a high have similar block similarity from among a plurality of reference frames,
For each block in the process target frame, the search order as of the priority of the reference number is large reference frames increases the reference frame blocks similar block is determined in the reference frame already Te the processing target frame smell Decide
Calculating the similarity for each block of the reference frame selected according to the determined order of frames to search;
The motion vector detection method, wherein the search is stopped when the calculated similarity satisfies a predetermined threshold, and a block whose similarity satisfies the predetermined threshold is determined as a similar block. Computer
For each block in the frame to be processed, the search order as of the priority of the reference number is large reference frames increases the reference frame blocks similar block is determined in the reference frame already Te the processing target frame smell Decide
Calculating the similarity for each block of the reference frame selected according to the determined order of frames to search;
A program that operates so that the search is stopped when the calculated similarity satisfies a predetermined threshold, and a block that satisfies the predetermined threshold is determined as a similar block.

Images