JP6663191B2 - Encoding setting information determination device and program - Google Patents

Description

  The present invention relates to an encoding setting information determining device and a program for determining setting information when encoding a video.

  As a video encoding method, ITU-TH. H.264 / MPEG-4 AVC (hereinafter, simply referred to as “AVC”) and ITU-T H.264. H.265 / MPEG-H HEVC (hereinafter simply referred to as “HEVC”) and the like are known. In these coding systems, inter prediction (inter-frame prediction, inter-screen prediction) and intra prediction (intra-frame prediction, intra-screen prediction) are used. The inter prediction is a prediction encoding method that generates a prediction image from a temporally preceding and succeeding reference frame and encodes a difference from an original image. Intra prediction is a predictive coding method for generating a predictive image from pixels of an adjacent block that has been coded for a block to be intra-coded, and coding a difference from the original image.

  Non-Patent Document 1 discloses that HEVC generates a variable-size coding unit (CU: Coding Unit) by hierarchically dividing a coding tree unit (CTU: Coding Tree Unit) into quadtrees. Have been. The CU has four sizes of 64 × 64 pixels, 32 × 32 pixels, 16 × 16 pixels, and 8 × 8 pixels. When performing inter prediction, each CU is further divided into prediction units (PU: Prediction Unit) for inter prediction, and prediction processing is performed for each PU. When performing intra prediction, each CU is further divided into PUs for intra prediction, and prediction processing is performed for each PU.

  Japanese Patent Application Laid-Open No. H11-163873 discloses a technique for extracting feature points using SURF feature amounts in each of a plurality of camera-captured videos in order to reduce the amount of coding operation of the plurality of camera-captured videos. It is disclosed that block matching is performed for blocks so that the search range is corrected based on the adjacent camera vector so that the search range becomes narrower.

JP 2014-120917 A

Supervised by Okubo Sakae, "Impress Standard Textbook Series H.265 / HEVC Textbook", Impress Japan, October 21, 2013

  However, according to the conventional encoding method, the accuracy of inter prediction is low in a video in which a large number of objects move greatly, such as “a plurality of trains run at high speed” and “a large number of ping-pong balls bounce”. Therefore, the encoded image quality may be reduced.

  Further, the technique described in Patent Document 1 is for reducing the amount of encoding calculation of images captured by a plurality of cameras, and performs motion estimation using matching between a plurality of frames of SURF feature points to reduce the coding efficiency. It does not improve it.

  An object of the present invention made in view of such circumstances is an encoding setting information determining apparatus and a program capable of determining encoding setting information for improving encoding efficiency of a video in which a large number of objects move greatly. Is to provide.

In order to solve the above-mentioned problem, an encoding setting information determining apparatus according to the present invention is an encoding setting information determining apparatus that determines setting information when encoding a video. -Up Robust Features) A motion estimator that generates a motion vector for each SURF feature point by inter-frame matching of feature points, and a SURF feature point is counted for each coding block, and based on the counted number of SURF feature points, A prediction block size setting unit that determines a prediction block having the same size as the coding block or a size obtained by dividing the coding block; and calculating an average value or a median value of the motion vectors for each SURF feature point for each prediction block. An initial displacement vector setting unit that determines an initial displacement vector for motion search for each of the prediction blocks; Using the motion vector for each RF feature points, comprising a motion estimation range setting unit for determining a motion estimation range, and the prediction block size setting unit, the size of the prediction block, the number of SURF feature points is often code When the initial displacement vector is the average value of the motion vectors, the motion search range setting unit sets the start points of all the motion vectors corresponding to the prediction block to the center of the prediction block. When moving to a point, the range including the end point of the motion vector is set as a motion search range for the initial displacement vector of the prediction block, and when the initial displacement vector is set as the median value of the motion vector, Move the start point of the motion vector whose size is the first quartile to the third quartile among the corresponding motion vectors to the center point of the prediction block The range including the end point of the motion vector when the motion vector is set is a motion search range for the initial displacement vector of the prediction block .

  According to another aspect of the present invention, there is provided a program for causing a computer to function as the encoding setting information determination device.

  ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to determine the encoding setting information for improving the encoding efficiency of the video in which many objects move greatly.

FIG. 2 is a block diagram illustrating a configuration example of an encoding setting information determination device according to the first embodiment of the present invention. FIG. 3 is a diagram illustrating processing in a motion estimating unit of the encoding setting information determination device according to the first embodiment of the present invention. FIG. 4 is a diagram illustrating an example of a prediction structure between reference frames in a motion estimator of the encoding setting information determination device according to the first embodiment of the present invention. FIG. 11 is a block diagram illustrating a configuration example of an encoding setting information determination device according to a second embodiment of the present invention. It is a figure explaining processing in the initial displacement vector setting part of the coding setting information deciding device concerning a 2nd embodiment of the present invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(First embodiment)
The encoding setting information determination device according to the first embodiment of the present invention will be described below. FIG. 1 is a block diagram illustrating a configuration example of the encoding setting information determination device according to the first embodiment of the present invention. In the example illustrated in FIG. 1, the coding setting information determination device 1 includes a motion estimating unit 11 and a predicted block size setting unit 12.

  The motion estimating unit 11 receives a video (original image group) and performs SURF feature point matching between frames (between an encoding target frame and a reference frame), thereby obtaining a SURF (Speeded-Up Robust Features). A motion vector starting from the SURF feature point is generated for each feature point. Then, motion vector information indicating the position of the SURF feature point and the direction of the motion vector (the amount of displacement in the horizontal and vertical directions) is output to the prediction block size setting unit 12. Since the information indicating the direction of the motion vector is unnecessary in the predicted block size setting unit 12, only the information indicating the position of the SURF feature point may be output to the predicted block size setting unit 12.

  The motion estimating unit 11 can detect the enlargement / reduction, rotation, illumination fluctuation, and the like by performing the motion estimation using the SURF feature amount. Even if the motion such as “the image in which sparks scatter” is a random image, High-precision motion estimation becomes possible, and it is possible to improve inter prediction accuracy (motion estimation accuracy) in a video having many random motions.

  FIG. 2 is a diagram illustrating a method of calculating a SURF feature amount in the motion estimating unit 11. The SURF feature quantity is obtained by extracting a plurality of SURF feature points by searching for a maximum point of a Hessian matrix value in an image obtained by applying a Gaussian filter with a variable variance value to an original image. In FIG. 2, as shown in the upper left diagram, the center point of a block of 8 × 8 pixels is set as a SURF feature point. Next, at each SURF feature point, a strong angle is obtained at an interval of 3 / π using a Haar wavelet. Then, using the direction of strong intensity as the original direction, the horizontal differential value dx and the vertical differential value dy are obtained using the Haar wavelet for each 2 × 2 pixel region as shown in the lower right diagram of FIG. , Four characteristic amounts of Σdx, Σ | dx |, Σdy, and Σ | dy | are calculated.

  As shown in FIG. 2, when the feature amount is calculated in units of 8 × 8 pixels with the SURF feature point at the center, there are 4 × 4 2 × 2 pixel regions in the 8 × 8 pixel block. For one SURF feature point, 64 (= 4 × 4 × 4) -dimensional feature amounts are obtained. For details of SURF, see, for example, H. Bay, A. Ess, T. Tuytelaars, and LV Gool, "Speeded-Up Robust Features (SURF)", Elsevier, vol. 110, Issue 3, p. 346- 359, Jun. 2008.

  When using the SURF feature, the motion estimating unit 11 obtains a motion vector for each SURF feature point by performing matching of the SURF feature between the encoding target frame and the reference frame. Specifically, a SURF feature point B in a reference frame in which a 64-dimensional Euclidean distance is minimum with respect to a feature amount at an arbitrary SURF feature point A in the encoding target frame is extracted as a matching point, and the SURF feature point is extracted. The amount of horizontal and vertical displacement between AB is calculated as a motion vector.

FIG. 3 is a diagram illustrating an example of a prediction structure between reference frames. The motion estimating unit 11 determines a reference frame for inter prediction based on prediction structure information between the reference frames. In the example shown in FIG. 3, when the coding target frame P ₁ picture, because the I ₀ picture is a reference frame, performing matching by SURF feature quantity between the original image of the P ₁ picture and I ₀ picture position. Further, the B ₂ picture when the encoding target frame, since the I ₀ picture and P ₁ picture is a reference frame, B ₂ picture and I ₀ picture-position, and B ₂ between the picture and P ₁ picture-position of the original image Perform motion estimation.

  The input of the prediction structure information of the reference frame is not essential, and the motion estimation is always performed between the original image and the time position one frame before (past) the encoding target frame without inputting the prediction structure information of the reference frame. It may be performed.

  Returning to FIG. 1, the predicted block size setting unit 12 inputs encoded block size information indicating the size of the encoded block obtained by dividing the frame. Then, the prediction block size setting unit 12 counts the SURF feature points for each coded block of a predetermined size, determines the size of the prediction block based on the counted number of SURF feature points, and indicates the determined size of the prediction block. The prediction block size information is output as setting information to an external encoding device (not shown). The size of the prediction block is the same as the coding block or the size obtained by dividing the coding block.

  The encoding device sets the prediction block size determined by the encoding setting information determination device 1, and performs a prediction process for each prediction block. Although it is considered that the larger the number of SURF feature points is, the larger the number of SURF feature points is in the coding block, the higher the number of SURF feature points is. However, the prediction block size setting unit 12 reduces the size of the prediction block as the number of SURF feature points increases. A minute movement can be detected with high accuracy and high accuracy.

When the coding method is AVC, for example, the coding block size is set to 16 × 16 pixels. Assuming that the total number of SURF feature points is N _S and the number of coding blocks is N _B , if the number of SURF feature points in the coding block is (N _S / N _B ) × 2 or more, the prediction block size (macro block size) Is set to 4 × 4 pixels, and if less than (N _S / N _B ) × 2 and not less than (N _S / N _B ) × 1, the prediction block size is set to 8 × 8 pixels, and (N _S / N) _B ) If less than × 1, the prediction block size is set to 16 × 16 pixels.

When the encoding method is HEVC, the encoding block size is set to the size of the CU. For example, assuming that the total number of SURF feature points is N _S and the number of CU blocks is N _B , if the number of SURF feature points in the CU is (N _S / N _B ) × 2 or more, the prediction block size (PU size) is 4 × If it is set to 4 pixels, and if it is less than (N _S / N _B ) × 2 and (N _S / N _B ) × 1.5 or more, the prediction block size is set to 8 × 8 pixels, and (N _S / N _B) ) × 1.5 and (N _S / N _B ) × 1 or more, the prediction block size is set to 16 × 16 pixels, and if it is less than (N _S / N _B ) × 1, (N _S / N _B ) If it is not less than × 0.5, the prediction block size is set to 32 × 32 pixels, and if it is less than (N _S / N _B ) × 0.5, the prediction block size is set to 64 × 64 pixels. If the PU size exceeds the CU size, the PU and the CU have the same size.

  As described above, the coding setting information determination device 1 counts SURF feature points for each coding block. Then, it can be considered that a coded block having a larger number of SURF feature points has more spatial high-frequency components. Therefore, the size of the prediction block is determined to be smaller as a coded block has a larger number of SURF feature points. Therefore, by performing encoding using the prediction block having the size determined by the encoding setting information determination device 1, it is possible to improve the encoding efficiency of a video in which many objects move greatly.

(Second embodiment)
Next, an encoding setting information determination device according to a second embodiment of the present invention will be described. FIG. 4 is a block diagram illustrating a configuration example of an encoding setting information determination device according to the second embodiment. In the example illustrated in FIG. 4, the encoding setting information determination device 2 includes a motion estimation unit 11, a prediction block size setting unit 12, an initial displacement vector setting unit 13, and a motion search range setting unit 14. The encoding setting information determining device 2 of the second embodiment is different from the encoding setting information determining device 1 of the first embodiment in that it further includes an initial displacement vector setting unit 13 and a motion search range setting unit 14. Different. In the present embodiment, the encoding setting information determination device 2 includes the initial displacement vector setting unit 13 and the motion search range setting unit 14, but may have a configuration including only one of them.

  As in the first embodiment, the motion estimating unit 11 generates a motion vector for each SURF feature point by inputting an image and performing matching of the SURF feature points between frames. Then, the motion vector information is output to the prediction block size setting unit 12, the initial displacement vector setting unit 13, and the motion search range setting unit 14.

  As in the first embodiment, the prediction block size setting unit 12 counts SURF feature points for each coding block of a predetermined size, and based on the counted number of SURF feature points, predicts a size of the coding block or less. Determine the block size. Then, the prediction block size information is output as setting information to an external encoding device, and is also output to the initial displacement vector setting unit 13 and the motion search range setting unit 14.

  The initial displacement vector setting unit 13 receives the motion vector information generated by the motion estimating unit 11 and the predicted block size information generated by the predicted block size setting unit 12, and inputs all the motions included in each predicted block. An average value or a median value of the vectors is calculated and used as an initial displacement vector for motion search. Then, the initial displacement vector setting unit 13 outputs information indicating the initial displacement vector to an external encoding device as setting information. If there is no motion vector corresponding to the prediction block (that is, no SURF feature point exists in the prediction block), information indicating that fact is output to the outside.

  FIG. 5 is a diagram illustrating the process of the initial displacement vector setting unit 13. As shown in FIG. 5, when three SURF feature points A, B, and C exist in the prediction block P, the initial displacement vector setting unit 13 sets the motion vector starting from the SURF feature point A and the SURF feature point B An average value or a median value of the motion vector starting from the starting point and the motion vector starting from the SURF feature point C is calculated as the initial displacement vector.

  The motion search range setting unit 14 inputs the motion vector information generated by the motion estimating unit 11 and the predicted block size information generated by the predicted block size setting unit 12, and for each predicted block, A motion search range is determined using a motion vector starting from a feature point included in the prediction block). Then, the motion search range setting unit 14 outputs information indicating the motion search range as setting information to an external encoding device. If there is no motion vector corresponding to the prediction block (that is, no SURF feature point exists in the prediction block), information indicating that fact is output to the outside.

  Specifically, when the initial displacement vector setting unit 13 sets the initial displacement vector as the average value of the motion vectors, the motion search range setting unit 14 sets the start points of all the motion vectors corresponding to the predicted block to the center of the predicted block. The range including the end point of the motion vector when moved to the point is set as the motion search range for the initial displacement vector of the prediction block. In addition, when the initial displacement vector setting unit 13 sets the initial displacement vector as the median of the motion vectors, the motion search range setting unit 14 sets the size of the motion vector corresponding to the prediction block from the first quartile to the first quartile. The range including the end point of the motion vector when the start point of the motion vector of the third quartile is moved to the center point of the prediction block is defined as motion search range information for the initial displacement vector of the prediction block. Here, the first quartile is the value of the order of 1/4 of the whole when counting the data in ascending order, and the third quartile is the value of sorting the data in ascending order. Is the value of the order of 3/4 of the whole, counting from the smaller one.

  As described above, the coding setting information determination device 2 counts the SURF feature points for each coding block, as in the case of the coding setting information determination device 1. Is encoded using a prediction block of the size determined by the encoding setting information determination device 2 in order to reduce the size of the image, thereby improving the encoding efficiency of a video in which a large number of objects move greatly. It is possible to do.

  In addition, the coding setting information determination device 2 determines an initial displacement vector for motion search using a motion vector included in a prediction block. Therefore, by using the initial displacement vector determined by the encoding setting information determination device 2, even if the motion is large, motion detection can be performed by block matching, and the accuracy of motion estimation can be improved. .

  Further, the coding setting information determination device 2 determines a motion search range using a motion vector included in the prediction block. Therefore, by performing block matching using the motion search range determined by the coding setting information determination device 2, the motion search range can be narrowed, and the speed of motion detection can be improved.

  Note that a computer can be suitably used to function as the above-described encoding setting information determining device 1 or 2, and such a computer performs processing contents for realizing each function of the encoding setting information determining device 1 or 2. Is stored in the storage unit of the computer, and the program is read out and executed by the CPU of the computer. This program can be recorded on a computer-readable recording medium.

  Although the above embodiments have been described as representative examples, it will be apparent to those skilled in the art that many changes and substitutions can be made within the spirit and scope of the present invention. Therefore, the present invention should not be construed as being limited by the above-described embodiment, and various modifications and changes can be made without departing from the scope of the claims. For example, a plurality of constituent blocks described in the embodiment can be combined into one, or one constituent block can be divided.

1, 2 coding setting information determination device 11 motion estimation unit 12 prediction block size setting unit 13 initial displacement vector setting unit 14 motion search range setting unit

Claims (2)

An encoding setting information determination device that determines setting information when encoding a video,
A motion estimator that receives a video and generates a motion vector for each SURF feature point by performing inter-frame matching of SURF (Speeded-Up Robust Features) feature points;
A prediction block size setting unit that counts SURF feature points for each coding block and determines a prediction block having the same size as the coding block or a size obtained by dividing the coding block based on the counted number of SURF feature points,
An initial displacement vector setting unit that calculates an average value or a median value of motion vectors for each SURF feature point for each prediction block, and determines an initial displacement vector for motion search for each prediction block;
A motion search range setting unit that determines a motion search range using a motion vector for each SURF feature point for each of the prediction blocks;
With
The prediction block size setting unit determines the size of the prediction block to be smaller for a coded block having a larger number of SURF feature points ,
The motion search range setting unit,
When the initial displacement vector is the average value of the motion vectors, when moving the start points of all the motion vectors corresponding to the prediction block to the center point of the prediction block, the range including the end point of the motion vector, The motion search range for the initial displacement vector of the prediction block,
When the initial displacement vector is the median of the motion vectors, the start point of the motion vector whose magnitude is from the first quartile to the third quartile among the motion vectors corresponding to the prediction block is An encoding setting information determining device , wherein a range including an end point of the motion vector when the motion vector is moved to a center point is set as a motion search range for an initial displacement vector of the prediction block . A program for causing a computer to function as the encoding setting information determination device according to claim 1 .

Images