JP2005323299A - Method for searching motion vector, and encoding device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce operation quantity concerning motion vector search by an encoding device of moving pictures which performs motion compensation, using segmented patterns. <P>SOLUTION: A motion vector of each segmented area in a predetermined segmented pattern, which is one of a plurality of segmented patterns, is searched and the obtained motion vector is stored in a motion vector storing means. For one segmented area in one segmented pattern other than the predetermined segmented pattern, the center of search of the one segmented area is obtained, by using the motion vector of the segmented area in the predetermined segmented pattern, corresponding to the one segmented area, and the motion vector of the one segmented area is searched using the center of search. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a moving picture coding technique, and more particularly to a motion compensation technique in moving picture coding.

  In the moving picture coding technique, a technique for coding only a difference in which a change has occurred between frames is used by paying attention to the nature of a moving picture in which almost the same frame is repeated if the motion of the screen is small. Among them, a motion compensation (MC) technique is generally used as a technique for performing efficient coding even when there is motion in a part of an image.

  Motion compensation refers to the position of an image similar to the image in the encoding target frame, not the difference between the image of the encoding target frame and the image at the same position in the previous frame (hereinafter referred to as a reference frame). The search is performed from the frame, and the difference between the searched image and the image of the encoding target frame is obtained. Searching for a position of an image similar to the image in the encoding target frame from the reference frame is generally called motion estimation (ME), but here it is called motion vector search.

  The motion vector search will be described with reference to FIG. As shown in FIG. 1, the motion vector search is performed in units of regions obtained by cutting out a part of the frame, not the entire frame. That is, a frame is divided into a certain size (for example, a rectangular area of 16 pixels × 16 pixels, called a macroblock in MPEG, etc., which is called a unit area in this specification), and a motion vector having a horizontal component and a vertical component in each area. Search to find Specifically, by comparing the region A with the region B of the reference frame at the same position as the region A of the encoding target frame in FIG. 1 and the region A, the region with the smallest difference with respect to the region A is referred to. Search from the frame. In FIG. 1, the area C is the searched area. The amount of movement from region A to region C is called a motion vector. In the encoding process, the motion vector and the difference between the region A and the region C are encoded.

  The motion vector search will be further described.

  As described above, in the motion vector search, a reference frame region that minimizes the difference from the region of interest of the encoding target frame is searched. As a specific difference, for example, as shown in FIG. Sum of Absolute Difference: Sum of absolute differences of each pixel). Since the time interval between the reference frame and the encoding target frame is generally short, the position of the area in the reference frame that minimizes the difference between the reference frame and the encoding target frame is generally the same as the rectangular area of the encoding target frame. It exists in the vicinity of the position. For this reason, for example, as shown in FIG. 3, an optimal motion vector can be searched by performing a motion vector search from a range of ± 16 pixels around the same position as the region of the encoding target frame. However, in the method shown in FIG. 3, the number of search points is (16 × 2 + 1) 2 = 1089 points, and enormous calculation is required.

  In order to compensate for this drawback, there is a technique in which, when calculating the SAD of a certain area, the calculation of the SAD of the area is terminated when the SAD minimum value calculated so far is exceeded. This method will be described with reference to FIGS.

  Here, for the sake of explanation, three search points, that is, three reference areas (reference area 1 and reference area 2) are defined as rectangular areas (reference areas) of the reference frame with respect to the rectangular area (target area) of the encoding target frame. Focus on the reference region 3). FIG. 4 shows values for each region and each pixel. 5 shows an SAD calculation process between the target area and the reference area 1, FIG. 6 shows an SAD calculation process between the target area and the reference area 2, and FIG. The SAD calculation process between is shown. 5 to 7 show that the SAD is calculated by obtaining the absolute difference value for each pixel and accumulating them.

  When searching in the order of reference area 1 → reference area 2 → reference area 3, since the minimum value of SAD is updated in this order, it is necessary to obtain absolute difference values and accumulations for all pixel combinations. On the other hand, when searching in the order of the reference area 3 → reference area 2 → reference area 1, the SAD of the reference area 3 is 12 which is the minimum value in the 3 reference areas, and the SAD calculation process in the reference area 2 (FIG. 6). ) Exceeds the minimum value at the time of pixel number 3, the subsequent calculation is omitted, and the process proceeds to the SAD calculation process for the reference area 1. Further, in the SAD calculation process in the reference area 1 (FIG. 5), the calculation after pixel number 1 can be omitted. As a result, the amount of calculation in the motion vector search can be reduced.

By utilizing this method, as shown in FIG. 8, the motion vector of the target region is predicted from the peripheral region for which the motion vector has been calculated, and the point indicated by the motion vector is used as a search center, for example, a predetermined shape in a spiral shape. There is a method of scanning a search point in a range to speed up a motion vector search process.
UB Video Inc. “Emerging H.264 Standard: Overview and TMS320DM642-Based Solutions for Real-Time Video Applications” 12-2002 (URL: http://www.ubvideo.com/public/h.264_white_paper.pdf)

  Now, a new video coding standard, H.264. H.264 (see Non-Patent Document 1, for example) is in the standardization stage. H. H.264 uses a new encoding method in order to improve the compression efficiency over the conventional moving image encoding method. As one of them, a rectangular area (macroblock) of 16 × 16 pixels is divided into division patterns (referred to as a division mode in H.264) shown in FIG. Motion compensation is performed by selecting an optimal division mode from a plurality of division modes according to the situation of a complicated portion and a flat portion in the × 16 region. Thereby, the amount of generated codes after quantization is reduced. In this specification, the unit area after dividing a unit area such as a macroblock is referred to as a division pattern or a division mode. A divided area in the unit area is referred to as a divided area.

  H. In H.264, a motion vector search is performed in each divided region of each division mode. Then, the division mode that minimizes the coding cost is selected from among the plurality of division modes. This method has the advantage of reducing the amount of code, but requires a motion vector search in each divided region in each division mode, so that there is a problem that the amount of calculation increases compared to the conventional standard.

  Here, if the motion vector search method shown in FIG. 8 is used, it is possible to reduce the amount of calculation compared to the method of FIG. For example, as shown in FIG. 10, in the motion vector search in the case where the lower half of the division mode 3 is the target region, the horizontal components and vertical components of the motion vectors of the surrounding regions A, B, and C for which motion vectors have been calculated The median value for each component is set as the search center of the target region, and a predetermined range is searched from there by the method described in FIG. However, when this method is used, for example, when the boundary of the object is at the position indicated by the dotted line in FIG. 10, the movement is often different between the upper side and the lower side of the dotted line, and prediction is performed from the surrounding areas A, B, and C. The search center thus obtained is considered to be significantly different from the motion vector of the target region. Therefore, in order to search for an optimal motion vector, a wide range of calculations must be performed, and an increase in the amount of calculations is inevitable.

  The present invention has been made in view of the above points, and an object of the present invention is to reduce the amount of calculation related to motion vector search in a moving image encoding apparatus that performs motion compensation using divided patterns.

  The above-described problem is a motion vector search method executed in an encoding device that performs motion compensation using a plurality of division patterns obtained by dividing a unit region of a frame in a moving image by a plurality of division methods. A step of searching for a motion vector of each divided region in a predetermined divided pattern that is one of the patterns, and storing the obtained motion vector in a motion vector storage means; and another division other than the predetermined divided pattern By using the motion vector of the divided area in the predetermined divided pattern corresponding to the one divided area, which is read from the motion vector storage unit, for one divided area in the pattern, the search center of the one divided area is determined. And searching for a motion vector of the one divided region using the search center. It can be solved by the motion vector search method and a motion vector search step to be executed for each divided region of the divided pattern other than the split pattern.

  In addition, the above-described problem is an encoding apparatus that performs motion compensation using a plurality of division patterns obtained by dividing a unit region in a frame of a moving image by a plurality of division methods, and one of the plurality of division patterns. First motion vector search means for searching for a motion vector of each divided region in a predetermined division pattern, motion vector storage means for storing a motion vector searched by the first motion vector search means, Searched by a second motion vector search means for searching for a motion vector for each divided region of a divided pattern other than a predetermined divided pattern, the first motion vector search means and the second motion vector search means Motion compensation means for performing motion compensation using a motion vector, and the second motion vector search means includes the predetermined division pattern. By using the motion vector of the divided area in the predetermined divided pattern corresponding to the one divided area, which is read from the motion vector storage unit, for the one divided area in the other divided pattern, the one divided area This can also be solved by an encoding device that obtains a search center of a region and searches for the motion vector of the one divided region using the search center.

  According to the motion vector search method of the present invention, the motion vector of each divided region in a predetermined divided pattern is searched in advance, the obtained motion vector is accumulated in the motion vector storage means, and the divided regions in the other divided patterns are stored. Since the search center in the motion vector search is calculated using the motion vector already obtained for the region corresponding to the divided region, the search center close to the optimum motion vector can be obtained. Therefore, the search range can be narrowed down, and as a result, the amount of calculation related to motion vector search can be reduced.

  Next, embodiments of the present invention will be described with reference to the drawings. The embodiment of the present invention is described in H.264. Although the H.264 division mode is used as a base, the present invention can be applied to all moving picture encoding systems that perform motion compensation using a division pattern.

(Outline of processing)
First, an overview of processing according to the present invention will be described. The case of the same division mode 3 as in FIG. 10 will be described as an example with reference to FIG.

  As shown in FIG. 11, when performing motion vector search in the division modes 1 to 7, the motion vector search of each division region in the division mode 7 having the largest number of divisions is performed first, and in the motion vector search in other division modes, The search center obtained from the motion vector already obtained for each divided region constituting the divided mode is used. For example, in the case of the division mode 3, the motion vector already obtained by the motion vector search in the division mode 7 is used for the areas a to h in FIG. That is, each of the divided areas of the division mode 6 to the division mode 1 can be configured by a combination of the divided areas of the division mode 7 and the motion vector search of the division mode 7 has already been performed. In order to determine the search center of each divided region in the division mode 1, the motion vector in the division mode 7 can be used.

  More specifically, when the motion vector of the lower half area of the division mode 3 is obtained, for example, a point composed of the average value of the horizontal components and the average value of the vertical components of the motion vectors already calculated in the areas a to h, or Then, the motion vector search is performed with the point consisting of the median of the horizontal component and the median of the vertical component as the search center. Note that the method for obtaining the search center of the target region from the already calculated motion vector is not limited to this.

  By using the above method, even if the boundary of the object exists at the position shown in FIG. 10, the motion vector already obtained in the corresponding area is used as it is, so that even if there is movement in that area, the movement is reflected. The obtained search center can be obtained and used for motion vector search. Since a highly accurate value can be used as the search center, the motion vector search range can be narrowed, and the amount of calculation can be reduced.

(Device configuration)
An example of an encoding apparatus to which the motion vector search method of the present invention is applied is shown in FIG. FIG. 12 is a block diagram showing the main configuration of a general coding apparatus that performs motion compensation. The motion vector search of the present invention is performed by the motion vector search unit 10 of FIG. In addition, the motion compensation unit 20 in FIG. 12 uses the image data of the immediately preceding frame to create a predicted image of the next frame. The encoding apparatus shown in FIG. 12 can be realized as hardware, or can be realized using a program that causes a computer to execute the processing of each unit. H. The configuration of the encoding device in H.264 is described in Non-Patent Document 1, for example.

  Such an encoding device can be installed in any device that performs image processing of moving images. As an example of an apparatus equipped with an encoding apparatus, there are a mobile phone, a personal computer, a video camera, and the like.

  FIG. 13 shows the configuration of the motion vector search unit 10 in the present embodiment.

  As shown in FIG. 13, the motion vector search unit 10 includes motion vector search units 101 to 107 (abbreviated as ME in the drawing) for each division mode, a mode determination unit 110, a search center setting unit 111, a motion vector. A memory unit 112 is provided. Each of the motion vector search units 101 to 107 receives the encoding target frame, the reference frame, and the search center, and executes a motion vector search in the corresponding division mode. Then, the motion vector obtained as a result of the search and the coding cost (the value obtained by adding the number of coding bits necessary for coding the motion vector and SAD) are input to the mode determination unit 110. The mode determination unit 110 selects a division mode that minimizes the coding cost, and outputs the selected division mode and a motion vector of the division mode. Further, the motion vector output from the motion vector search unit 107 in the division mode 7 is accumulated in the motion vector memory unit 112. The motion vector memory unit 112 also stores motion vectors that have already been encoded. The search center setting unit 111 sets the search center in the motion vector search in each division mode by using the motion vector stored in the motion vector memory unit 112.

(Detailed operation)
Next, the operation of the motion vector search unit 10 will be described in more detail with reference to the flowcharts of FIGS. FIG. 14 shows the overall processing procedure, and FIG. 15 shows the processing procedure of step S2 in FIG. The following process is performed for each unit area in the encoding target frame. Moreover, the execution of the processing according to each flowchart can be performed when a control unit (not shown) issues a command to each processing unit.

  As shown in the flowchart of FIG. 14, first, the motion vector search unit 107 in the division mode 7 calculates a motion vector for each division region in the division mode 7 and stores it in the motion vector memory unit 112 (step S1).

  More specifically, the search center setting unit 111 acquires the already encoded motion vector stored in the motion vector memory unit 112, and for example, the median of the motion vectors of the regions A, B, and C shown in FIG. Is determined as the search center, and is output to the motion vector search unit 107 in the division mode 7. The motion vector search unit 107 in the division mode 7 receives the input of the encoding target frame, the reference frame, and the search center, and performs a motion vector search for each divided region (4 × 4 pixel region). The motion vector search range at this time is a normal range (for example, ± 16 pixels). Then, the motion vector search unit 107 in the division mode 7 adds an encoding cost obtained by adding the number of encoding bits necessary for encoding the motion vector and SAD in addition to the motion vector obtained by the motion vector search. Output to the mode determination unit 110.

  After calculating the motion vector for each region in the division mode 7, the search range is narrowed down for each region in the division mode 1 to the division mode 6, and the calculated motion vector and the coding cost are determined as a mode. Is output to the unit 110 (step S2).

  Then, the mode determination unit outputs a division mode that minimizes the coding cost and a motion vector in the division mode (step S3).

  Next, motion vector search processing for each region in the division mode 1 to division mode 6 will be described with reference to the flowchart of FIG.

  First, a division mode is selected (step S21). For example, if the motion vector search is performed from the division mode 1 to the division mode 6 in order, the selection from the division mode 1 to the division mode 6 is performed in order.

  Next, the motion vector search unit of the selected division mode selects one division region in the division mode (step S22). The selected area is notified to the search center setting unit 111. The search center setting unit 111 acquires a motion vector corresponding to the notified area from the motion vectors already searched for the division mode 7 stored in the motion vector memory unit 112, and averages these values. Alternatively, the median value is determined as a motion vector prediction value serving as a search center, and is output to the motion vector search unit in the corresponding division mode (step S23).

  The motion vector search unit in the division mode receives the search center, the encoding target frame, and the reference frame, and performs a motion vector search for the corresponding divided region (step S24). Since the search center at this time is a value with high prediction accuracy predicted from the motion vector obtained by the motion vector search in the division mode 7, even if the motion vector search range is narrowed down from the normal range, the coding efficiency is hardly lowered. . For this reason, the motion vector search range can be about ± 1 to 4 pixels.

  If the motion vector search for all the divided areas is not completed in the selected division mode (No in step S25), the process returns to step S22, and the motion vector search process is performed for the next area.

  When the motion vector search for all the divided regions is completed (Yes in step S25), if there is a divided mode in which the motion vector search is not completed among the divided modes 1 to 6 (No in step S26), the process proceeds to step S21. Returning, the motion vector search process for the next division mode is executed. When the motion vector search is finished in the division modes 1 to 6 (Yes in step S26), the process in step S2 in FIG. 14 is finished.

  In the above example, the motion vector searched for each divided region of the division mode 7 having the largest number of divisions is stored in the motion vector memory unit 112, and the division of another division mode is performed by using the motion vector. Although the motion vector of the area has been obtained, the motion vector used for obtaining the motion vector of the divided area of another division mode may not be that of the division mode 7.

  For example, by using the motion vector of each divided region in the division mode 6, it is possible to obtain the motion vector of each divided region in the division mode 1 to the division mode 4 having a smaller number of divisions by the same method as described above. In this case, for each of the divided regions in the division mode 5 and the division mode 7, the motion vector of the corresponding divided region in the division mode 6 is used. Therefore, for example, the motion vector of a certain divided area in the dividing mode 6 is used for each of the divided areas in the divided mode 7 corresponding to the divided area. In this case, in FIG. 13, the motion vector searched by the motion vector search unit 106 corresponding to the division mode 6 is stored in the motion vector memory 112, and the search center setting unit 111 performs search center setting processing in another division mode. Will be used.

  Which division mode is the base is determined by an evaluation unit that determines the amount of change in the image within a certain period of time, the situation of the uniform portion and the complicated portion in the frame, and the like. Alternatively, it is determined based on information input from the outside, such as a fast-moving scene.

  As described above, in the method of predicting the search center of the target region using the motion vector of the surrounding region, the search center of the target region is located when the boundary of the object is between the target region and the surrounding region. Since the accuracy is lowered, it is necessary to search a wide range in order to search for an optimal motion vector, and there is a problem that an enormous amount of computation is required. The motion vector search described in this embodiment According to the method, since the search center is obtained by using the already searched motion vector, a highly accurate search center can be obtained, and an optimal motion vector can be calculated even if the search range is narrowed down. Therefore, the amount of calculation can be reduced without reducing the coding efficiency, and the motion vector search process can be speeded up.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made within the scope of the claims.

It is a figure for demonstrating a motion vector search It is a figure for demonstrating the difference of the area | region of an encoding object frame, and the area | region of a reference frame. It is a figure which shows the example of the range of a motion vector search. It is a figure which shows the object area | region and reference area which are used in order to demonstrate the example of the method of performing a motion vector search efficiently. It is a figure which shows the SAD calculation process between an object area | region and the reference area. It is a figure which shows the SAD calculation process between the object area | region and the reference area. FIG. 5 is a diagram illustrating a SAD calculation process between a target area and a reference area 3. It is a figure for demonstrating the example of the method of performing a motion vector search efficiently. It is a figure which shows division mode. It is a figure for demonstrating the conventional motion vector search method when the lower half of the division mode 3 is made into the object area | region. It is a figure for demonstrating the outline | summary of the process which concerns on this invention. It is a block diagram which shows an example of the encoding apparatus with which the motion vector search method of this invention is applied. 3 is a diagram illustrating a configuration of a motion vector search unit 10. FIG. 3 is a flowchart showing an overall processing procedure of a motion vector search unit 10. It is a flowchart which shows the process sequence of step S2 in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Motion vector search part 20 Motion compensation part 101-107 Motion vector search part 110 for each division mode Mode determination part 111 Search center setting part 112 Motion vector memory part

Claims (2)

A motion vector search method executed in an encoding device that performs motion compensation using a plurality of division patterns obtained by dividing a unit region of a frame in a moving image by a plurality of division methods,
A storage step of searching for a motion vector of each divided region in a predetermined divided pattern that is one of the plurality of divided patterns, and storing the obtained motion vector in a motion vector storage means;
By using the motion vector of the divided area in the predetermined divided pattern corresponding to the one divided area read from the motion vector storage means for one divided area in the other divided pattern other than the predetermined divided pattern. The step of obtaining the search center of the one divided region and searching for the motion vector of the one divided region using the search center is executed for each divided region of the divided pattern other than the predetermined divided pattern. A motion vector search method comprising: a motion vector search step. An encoding device that performs motion compensation using a plurality of division patterns obtained by dividing a unit region in a moving image in a frame by a plurality of division methods,
First motion vector search means for searching for a motion vector of each divided region in a predetermined divided pattern that is one of the plurality of divided patterns;
Motion vector storage means for storing a motion vector searched by the first motion vector search means;
Second motion vector search means for searching for a motion vector for each divided region of a divided pattern other than the predetermined divided pattern;
Motion compensation means for performing motion compensation using the motion vectors searched by the first motion vector search means and the second motion vector search means,
The second motion vector search means reads the predetermined division pattern corresponding to the one division area read from the motion vector storage means for one division area in one division pattern other than the predetermined division pattern. A coding apparatus, wherein a search center of the one divided region is obtained by using a motion vector of the divided region, and a motion vector of the one divided region is searched using the search center.

Images