JP6612721B2 - Predictive image generation method, predictive image generation apparatus, and computer program - Google Patents

Description

  The present invention relates to a predicted image generation method, a predicted image generation apparatus, and a computer program.

  One of important elemental techniques in moving picture coding is motion compensation interframe prediction (hereinafter also referred to as MC (Motion Compensation) prediction). Motion-compensated interframe prediction is a moving picture coding method with inter-frame prediction that performs prediction between different screens, and refers to a frame that has already been decoded, and motion vectors and residuals are minimized so as to minimize prediction error energy. Find the difference signal. The residual signal is subjected to orthogonal transformation and quantization, and further entropy encoded to generate encoded data. In this method, in order to increase the coding efficiency, it is essential to reduce the prediction error energy, and a highly accurate prediction method is required.

In MC prediction, an encoding target image frame f _t (p) represented by the following equation (1) is divided into several regions.

  Inter-frame prediction as shown by the following equation (2) using motion compensation (MC) is performed for each divided region.

In the equation (2), the region B _i is the i-th region, and the vector v _i is a motion vector that satisfies the following equation (3).

Region R in the formula (3), a search region corresponding to the region B _i, r (v) (Here, v is a vector) is an estimate of the amount of codes of the motion vectors v. For example, r (v) = | v | is applied as r (v). In equation (3), the part of the following equation (4) means obtaining a vector v that minimizes the next value.

In Equation (3), the sum of absolute values (SAD: Sum of Absolute Difference) of the prediction error and the amount of code of the motion vector of the encoding target image frame and the previous encoding target image frame to which the motion vector is applied. vector v that minimizes the weighted sum of the estimated value will be is determined as a motion vector v _i.

  As described above, MC prediction is prediction processing in units of blocks obtained by dividing an area, and a block to be subjected to MC prediction is called an MC prediction block, and a motion vector is detected and associated for each MC prediction block. It is done. However, an error occurs because the motion vectors of the pixels in the MC prediction block are not necessarily the same. This error is called MC prediction error. Although the MC prediction error can be reduced by setting the MC predicted block size small, reducing the MC predicted block size increases the additional information. Thus, the reduction of MC prediction error and the reduction of additional information in MC prediction are in a trade-off relationship.

  On the other hand, an optical flow for each pixel (hereinafter also referred to as OF (Optical Flow)) is calculated on the decoder side without additional information, and the motion vector in the MC prediction block is corrected based on the calculated optical flow. Has been proposed. For example, there is a correction method such as BIO (Bi-directional optical flow) in FRAUC (Frame rate up-conversion mode) (for example, see Non-Patent Document 1).

  When this correction is expressed by an equation, when the motion vector is v, the following equation (5) indicating that the motion vector v is applied to the pixel p becomes the following equation (6) by the correction.

  This process is called OF correction, which is an optical flow calculated by the vector δ (p), and is hereinafter referred to as an OF correction vector. By the OF correction, the prediction signal for MC prediction is expressed by the following equation (7).

  At this time, the OF correction vector δ (p) is obtained by the following equation (8).

In Expression (8), the region Ω is a search range for the correction vector δ, and is given as a fixed region in advance. The region Ψ is a block including p that is a search target of the correction vector δ, and is given as a fixed-size block in advance. Further, as shown in Expression (8), the OF correction vector δ (p) is obtained by correcting f _{t + 1} (p + δ) and f _t−1 (p + δ) obtained by correcting the frames before and after the encoding target image frame f _t (p). It is calculated by.

“Further improvements to HMKTA-1.0”, VCEG-AZ07, ITU-Telecommunications Standardization Sector, STUDY GROUP 16 Question 6, Video Coding Experts Group (VCEG), June 2015.

  However, in the above method, since the amount of calculation increases not only in the encoder but also in the decoder, there is a problem that the video terminal device with limited computer resources cannot support real-time reproduction.

  In view of the above circumstances, an object of the present invention is to provide a technique capable of reducing the amount of calculation.

  One aspect of the present invention is a prediction image generation method that performs motion compensation inter-frame prediction in units of blocks obtained by dividing an encoding target image frame in moving image data, the block of the encoding target image frame, and the block A prediction processing step of detecting a motion vector for each block based on a block of a corresponding reference image frame, and when detecting an optical flow for each pixel included in the block, from the center of the block to the pixel A search range setting step by class for selecting the search range corresponding to the pixel to be detected from a search range that changes according to the distance of the object, and an optical flow for each pixel is calculated based on the selected search range The calculated optical flow is converted into an optical flow correction vector for each pixel. Predictive image generation comprising: an optical flow correction vector calculation processing step, and an optical flow correction image generation processing step of generating a prediction image block based on the motion vector corrected by the optical flow correction vector for each pixel Is the method.

  One aspect of the present invention is the predicted image generation method described above, wherein the search range is preliminarily set so that the search range is expanded as the pixel to be detected by the optical flow moves away from the center of the block. It has been established.

  One aspect of the present invention is the above-described prediction image generation method, wherein the motion vector is detected by performing weighting that decreases in accordance with a distance from the center of the block in the prediction processing step.

  One aspect of the present invention is the predicted image generation method described above, wherein a search range setting step for selecting the same search range for all the pixels, the motion vector of the block, and the periphery of the block A motion vector determination step for determining whether or not the motion vectors of the block are aligned, and in the class-specific search range setting step, the motion vector of the block in the motion vector determination step; When it is determined that the motion vectors of the blocks around the block are aligned, the search range corresponding to the pixel is selected from the search range that changes according to the distance from the center of the block to the pixel, In the search range setting step, the motion vector determination step includes the motion vector of the block and the block. Tsu If click of a motion vector of a neighboring block is determined not to have all, selecting the same search range.

  One aspect of the present invention is a prediction image generation apparatus that performs motion compensation inter-frame prediction in units of blocks obtained by dividing an encoding target image frame in moving image data, the block of the encoding target image frame, and the block A prediction processing unit that detects a motion vector for each block based on a block of a corresponding reference image frame; and when detecting an optical flow for each pixel included in the block, from the center of the block to the pixel The search range setting unit for selecting the search range corresponding to the pixel to be detected from the search range that changes according to the distance, and the optical flow for each pixel based on the selected search range An optical flow correction vector for each pixel is used as the calculated optical flow. A prediction image generation apparatus comprising: an optical flow correction vector calculation processing unit; and an optical flow correction image generation processing unit that generates a prediction image block based on the motion vector corrected by the optical flow correction vector for each pixel. .

  One aspect of the present invention is a computer program for causing a computer to execute the predicted image generation method.

  According to the present invention, the amount of calculation can be reduced.

It is a figure which shows the structure of the moving image encoder in 1st Embodiment of this invention. It is a figure which shows the structure of the inter prediction process part in 1st Embodiment of this invention. It is a flowchart which shows the process by the inter prediction process part of 1st Embodiment. It is a figure which shows the structure of the inter prediction process part in 2nd Embodiment of this invention. It is a flowchart which shows the process by the inter prediction process part of 2nd Embodiment. It is a figure which shows the structure of the inter prediction process part in 3rd Embodiment of this invention. It is a flowchart which shows the process by the inter prediction process part of 3rd Embodiment. It is a figure which shows the structure of the moving image decoding apparatus in one Embodiment of this invention. It is a figure which shows the structure of the inter prediction process part 210 of the moving image decoding apparatus 200. FIG. It is a figure which shows the structure of the inter prediction process part 210b of the moving image decoding apparatus 200. FIG. It is a figure which shows the structure of the inter prediction process part 210c of the moving image decoding apparatus 200. FIG.

(First embodiment)
Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing a configuration of a moving picture coding apparatus 100 according to the first embodiment of the present invention. The moving image encoding apparatus 100 is, for example, an H.264 called High Efficiency Video Coding (HEVC). H.265, H. It is an apparatus applied as an encoder in an encoding method such as H.264.

  In FIG. 1, the moving image encoding apparatus 100 includes a prediction residual generation unit 102, a transformation processing unit 103, a quantization processing unit 104, an inverse quantization processing unit 105, an inverse transformation processing unit 106, a decoded signal generation unit 107, A loop filter processing unit 108, an intra prediction processing unit 109, an inter prediction processing unit 110, a frame memory 111, an intra prediction information storage unit 112, an inter prediction information storage unit 113, a switch 114, and an entropy encoding processing unit 115 are provided.

  The moving image encoding apparatus 100 takes an encoding target video signal as an input signal, encodes each frame of moving image data included in the input video signal, and outputs the encoded bit stream data as encoded stream data. To do.

  In the video encoding device 100, each frame of video data to be encoded included in the input video signal is input to the prediction residual generation unit 102 and the inter prediction processing unit 110. In the following description, the moving image data to be encoded is referred to as encoding target image data, each frame of the encoding target image data is referred to as an encoding target image frame, and further, the encoding target image frame. Each block into which is divided is called an encoding target image block.

  The prediction residual generation unit 102 calculates the difference between the input video signal and the prediction signal output from the intra prediction processing unit 109 or the inter prediction processing unit 110 by an adder provided therein for each input frame. The difference is output as a prediction residual signal. In the following description, the prediction signal is referred to as prediction image data, a prediction image frame, and a prediction image block depending on the configuration to be applied.

  The transform processing unit 103 performs an orthogonal transform of a discrete cosine transform (DCT (Discrete Cosine Transform)) on the prediction residual signal and outputs a transform coefficient obtained by the orthogonal transform. The quantization processing unit 104 quantizes the transform coefficient and outputs the quantized transform coefficient to the entropy coding processing unit 115 and the inverse quantization processing unit 105. The entropy encoding processing unit 115 performs entropy encoding on the quantized change coefficient and outputs encoded stream data. Further, the entropy encoding processing unit 115 reads out information related to intra prediction stored in the intra prediction information storage unit 112 and information related to inter prediction stored in the inter prediction information storage unit 113, and performs entropy encoding and encoding. Output as stream data.

  The inverse quantization processing unit 105 performs inverse quantization on the quantized transform coefficient, returns it to the transform coefficient, and outputs it. The inverse transform processing unit 106 performs inverse orthogonal transform on the transform coefficient output from the inverse quantization processing unit 105, and outputs a prediction residual decoded signal. The decoded signal generation unit 107 adds the prediction signal output from the intra prediction processing unit 109 or the inter prediction processing unit 110 and the prediction residual decoded signal output from the inverse transform processing unit 106 using an adder provided therein. Also, the decoded signal generation unit 107 generates and outputs a decoded image block corresponding to the encoding target image block by the addition. The in-loop filter processing unit 108 performs a filtering process for reducing the coding distortion on the decoded image block output from the decoded signal generation unit 107, and writes the decoded image block after the filtering process into the frame memory 111 for storage.

  The intra prediction processing unit 109 performs intra prediction, that is, prediction within a frame, according to a predetermined prediction mode for each block included in the frame stored in the frame memory 111, and outputs a prediction signal. Further, the intra prediction processing unit 109 writes and stores information related to intra prediction such as the set prediction mode in the intra prediction information storage unit 112.

The inter prediction processing unit 110 reads, from the frame memory 111, the frame f _t−1 (p) _immediately before the input encoding target image frame as a reference image frame. The inter prediction processing unit 110 performs inter prediction, that is, inter-frame prediction, based on the encoding target image block of the input encoding target image frame and the block corresponding to the encoding target image block in the reference image frame. And output a prediction signal. In addition, the inter prediction processing unit 110 detects a motion vector for each block, and performs inter-frame prediction processing to which motion compensation using the detected motion vector is applied. Also, the inter prediction processing unit 110 writes and stores information related to inter prediction such as a detected motion vector and an OF correction vector for correcting a motion vector described later in the inter prediction information storage unit 113.

  The switch 114 switches so that the prediction residual generation unit 102 and the decoded signal generation unit 107 are connected to either the intra prediction processing unit 109 or the inter prediction processing unit 110. When the switch 114 connects the prediction residual generation unit 102, the decoded signal generation unit 107, and the inter prediction processing unit 110, the prediction signal generated by the inter prediction processing unit 110 through the motion compensation inter-frame prediction processing is predicted. It is supplied to the residual generation unit 102. The prediction residual generation unit 102 generates the prediction residual signal by subtracting the prediction signal on which the motion compensation interframe prediction has been performed from the encoding target image block. Thereby, in the moving image encoding device 100, the encoding which applied the inter-frame prediction in which motion compensation was performed is performed.

(Configuration of the inter prediction processing unit of the first embodiment)
FIG. 2 is a block diagram illustrating an internal configuration of the inter prediction processing unit 110 of the above-described video encoding device 100. The inter prediction processing unit 110 includes a motion vector search processing unit 10, a predicted image block generation processing unit 11, a distance calculation unit 12, a class-specific search range setting unit 13, a class classification threshold storage unit 14, an OF correction vector calculation processing unit 15, An OF correction image generation processing unit 16 is provided.

  The inter prediction processing unit 110 performs the following adaptive processing focusing on the uneven distribution of MC prediction errors in the MC prediction block, that is, the block on which motion compensation prediction has been performed. The uneven distribution of MC prediction errors in the MC prediction block is a property that the MC prediction error in the MC prediction block has a tendency to increase as the distance from the center of the block increases. Using this property, the inter prediction processing unit 110 performs processing adapted to the position of the pixel in the MC prediction block when calculating the optical flow for performing OF correction.

For example, the coordinate value of the center position in the i-th MC prediction block B _i is μ _i . Based on the distance d (p, μ _i ) from the coordinate value μ _i to the correction target pixel position p, an optical flow search range of the target pixel position p is set. Specifically, the range of d (p, μ _i ) is classified into C classes, and the search range Ω _c (c = 0,..., C−1) is expressed by the following equation (9) for each class. Determine as follows. In the following description, Ω _c (c = 0,..., C−1) is referred to as a class-specific search range.

In equation (9), when | Ω _c | is an equation representing the number of pixels in the class-specific search range Ω _c , the class-specific search shown in equation (9) is established so that the relationship of the following equation (10) holds. Define the range Ω _c . When the class-specific search range Ω _c is a rectangular region of vertical n pixels and horizontal m pixels, | Ω _c | is = n × m.

The class-specific search range Ω _c , the number of classes C, and the class classification threshold θc (c = 0,..., C−1) are predetermined information. Further, the distance d () described above indicates a length represented by the following expression (11) between two coordinate values p = (p _x , p _y ) and q = (q _x , q _y ).

The above equation (10) means that the optical flow search range is the narrowest near the center of the MC prediction block, and the optical flow search range becomes wider as the outer edge of the MC prediction block is approached. For example, if C = 2 and the search range Ω ₀ is an empty set, OF correction is applied only to the pixels at the outer edge of the MC prediction block, and OF correction is omitted for other pixels. Become.

When the distance from the encoding target image frame to the reference image frame for OF correction is different, for example, just before and immediately after the encoding target image frame, that is, other than the t−1 and t + 1 reference image frames as shown in Expression (8) In the case of using this frame, the calculation of the optical flow applied to the OF correction can be generalized as shown in the following equation (12). In the equation (12), l _f is a parameter that specifies the future direction of the reference image frame, l _b is a parameter that specifies the past direction of the reference image frame.

Hereinafter, the configuration of the inter prediction processing unit 110 will be described with reference to FIG. In the inter prediction processing unit 110, the motion vector search processing unit 10 receives an encoding target image frame and a reference image frame stored in the frame memory 111, and detects a motion vector. For example, the motion vector search processing unit 10 reads from the frame memory 111 a reference image block corresponding to a block that performs MC prediction in the encoding target image frame. That is, the motion vector search processing unit 10 uses the frame f _t−1 (p) _immediately before the captured encoding target image frame as a reference image frame, and corresponds to a block for performing MC prediction in the reference image frame. Read the region as a reference image block. The motion vector search processing unit 10 performs MC prediction processing based on the MC prediction block of the encoding target image frame and the reference image block corresponding to the block, and in the reference image frame most similar to the MC prediction block. A motion vector is detected for each block by searching the area. Here, the block of the reference image frame corresponding to the block for MC prediction reference image as a relationship that satisfies the equation (2) by applying the motion vectors v _i for a pixel p of block for MC prediction It is a block including the pixels of the frame. For example, such a block is detected by matching and read as a reference image block corresponding to a block for performing MC prediction in the reference image frame, and a motion vector is obtained from coordinate information of the block for performing MC prediction and the reference image block. To detect.

  The motion vector search processing unit 10 outputs the detected motion vector to the inter prediction information storage unit 113 and the predicted image block generation processing unit 11. The predicted image block generation processing unit 11 is a prediction processing unit that generates a predicted image block based on a motion vector. The predicted image block generation processing unit 11 outputs information indicating a block on which MC prediction has been performed, that is, an MC predicted block, to the distance calculation unit 12. Further, the predicted image block generation processing unit 11 outputs the generated predicted image block to the OF corrected image generation processing unit 16.

The distance calculation unit 12 detects the central coordinate value μ _i of the block on which the MC prediction has been performed from the information indicating the MC prediction block output from the prediction image block generation processing unit 11. Further, the distance calculation unit 12 calculates the distance d (p, μ _i ) based on the detected central coordinate value μ _i and the coordinate value of the pixel p targeted for the OF correction processing in the block. The calculated distance d (p, μ _i ) is output to the class-specific search range setting unit 13.

The class classification threshold storage unit 14 includes a class-specific search range Ω _c (c = 0,..., C−1) determined by the equation (9) and a class classification threshold θ _c (c = 0,. Information indicating the class-specific search range associated with C-1) is stored in advance. The class-specific search range setting unit 13 reads information indicating the class-specific search range from the class classification threshold storage unit 14, and selects a search range corresponding to the distance d (p, μ _i ) output by the distance calculation unit 12 as a class. A search range by class Ω _{c is} selected according to the classification threshold θ _c . The class-specific search range setting unit 13 outputs the selected search range to the OF correction vector calculation processing unit 15.

The OF correction vector calculation processing unit 15 reads frames before and after the encoding target image frame f _t (p) from the frame memory 111 of the moving image encoding device 100 as reference image frames. Further, the OF correction vector calculation processing unit 15 calculates an optical flow based on the read reference image frame and the search range output by the class-specific search range setting unit 13, and calculates the calculated optical flow as an OF correction vector δ. (P). Further, the OF correction vector calculation processing unit 15 outputs the calculated OF correction vector δ (p) to the OF correction image generation processing unit 16.

  The OF corrected image generation processing unit 16 performs OF correction based on the predicted image block output from the predicted image block generation processing unit 11 and the OF correction vector δ (p) output from the OF correction vector calculation processing unit 15. The predicted image block is generated. Further, the OF corrected image generation processing unit 16 outputs the predicted image block after the OF correction to the prediction residual generation unit 102 and the decoded signal generation unit 107. Further, the OF correction image generation processing unit 16 writes the OF correction vector δ (p) in the inter prediction information storage unit 113 and stores it.

(Processing by the inter prediction processing unit of the first embodiment)
Next, processing by the inter prediction processing unit 110 of the first embodiment will be described. FIG. 3 is a flowchart showing a flow of processing by the inter prediction processing unit 110. In response to the operation of the producer of the moving image coding apparatus 100, the class classification threshold storage unit 14 of the inter prediction processing unit 110 stores the class-specific search range Ω _c (c = 0,..., C−1). , Information indicating the search range by class associated with the class classification threshold θ _c (c = 0,..., C−1) is written and set in advance (step Sa1). The processing of step Sa2 to step Sa6 is repeatedly performed for all MC prediction blocks of the encoding target image frame (loops La1s to La1e).

The motion vector search processing unit 10 captures an encoding target image frame. The motion vector search processing unit 10 uses the frame _ft-1 immediately before the fetched encoding target image frame as a reference image frame, and in the reference image frame, a region corresponding to a block on which MC prediction is performed is a reference image block. Is read from the frame memory 111. The motion vector search processing unit 10 performs MC prediction processing based on the MC prediction block and the reference image block corresponding to the block to detect a motion vector. The motion vector search processing unit 10b outputs the detected motion vector to the predicted image block generation processing unit 11 and the inter prediction information storage unit 113. The predicted image block generation processing unit 11 generates a predicted image block based on the detected motion vector. The predicted image block generation processing unit 11 outputs information indicating the MC predicted block to the distance calculation unit 12. The predicted image block generation processing unit 11 outputs the generated predicted image block to the OF corrected image generation processing unit 16 (step Sa2).

The processing of step Sa3 to step Sa5 is repeated for all the pixels in the MC prediction block (loops La2s to La2e). The distance calculation unit 12 detects the central coordinate value μ _i of the block on which MC prediction has been performed from the information indicating the MC prediction block. Here, the information indicating the MC prediction block may be information indicating the position of the MC prediction block in the encoding target image frame, or may be the central coordinate value μ _i of the MC prediction block itself. . When the information indicating the MC prediction block is the central coordinate value μ _i , the distance calculation unit 12 uses the value output from the prediction image block generation processing unit 11 as it is. On the other hand, when the information indicating the MC prediction block is information indicating the position of the MC prediction block in the encoding target image frame, the distance calculation unit 12 causes the prediction image block generation processing unit 11 to execute the encoding target image frame. The central coordinate value μ _i of the MC prediction block is calculated from information relating to the coordinate position of the encoding target image frame given in advance from the prediction image block generation processing unit 11.

Based on the central coordinate value μ _i of the MC prediction block and the coordinate value of the pixel p targeted for the OF correction processing in the block, the distance calculation unit 12 calculates the distance d (p, μ _i ) is calculated. The distance calculation unit 12 outputs the calculated distance d (p, μ _i ) to the class-specific search range setting unit 13 (step Sa3).

The class-specific search range setting unit 13 stores the class-specific search range Ω _c (c = 0,..., C−1) and the class classification threshold θ _c (c = 0,. , C-1) is read out, the information indicating the class-specific search range is read out. The class-specific search range setting unit 13 sets the search range corresponding to the distance d (p, μ _i ) output by the distance calculation unit 12 as the class classification threshold θ _c based on the read information indicating the class-specific search range. thus selected from the class-specific search range Omega _c. The class-specific search range setting unit 13 outputs the selected search range to the OF correction vector calculation processing unit 15 (step Sa4).

The OF correction vector calculation processing unit 15 transmits the frames before and after the encoding target image frame f _t (p) from the frame memory 111 of the moving image encoding apparatus 100, that is, the time t + l _f and the time The frame at t-l _b is read as a reference image frame. The OF correction vector calculation processing unit 15 obtains the OF correction vector δ (p) by Expression (12) based on the read two reference image frames and the search range output by the class-specific search range setting unit 13. . That is, the OF correction vector calculation processing unit 15 calculates an optical flow for each pixel for the search range in the reference image frame, and sets the calculated optical flow as the OF correction vector δ (p). Further, the OF correction vector calculation processing unit 15 outputs the calculated OF correction vector δ (p) to the OF correction image generation processing unit 16 (step Sa5).

  The OF corrected image generation processing unit 16 includes the predicted image block output from the predicted image block generation processing unit 11, the motion vector stored in the inter prediction information storage unit 113, and the OF output from the OF correction vector calculation processing unit 15. Based on the correction vector δ (p), a predicted image block subjected to OF correction for each pixel according to Expression (7) is generated. The OF corrected image generation processing unit 16 outputs the predicted image block after the OF correction to the prediction residual generation unit 102 and the decoded signal generation unit 107. The OF corrected image generation processing unit 16 writes and stores the motion vector and the OF correction vector δ (p) in the inter prediction information storage unit 113 (step Sa6).

With the configuration of the first embodiment, the prediction image block generation processing unit 11 of the inter prediction processing unit 110 performs motion based on the MC prediction block of the encoding target image frame and the reference image block corresponding to the block. A vector is detected, and a predicted image block is generated based on the detected motion vector. The distance calculation unit 12 calculates the central coordinate value of the MC prediction block and the distance between the pixels to be processed for OF correction. The class-specific search range setting unit 13 associates the class-specific search range Ω _c and the class classification threshold θ _c so that the optical flow search range becomes wider as the outer edge of the MC prediction block is approached. Information indicating the search range for each class is read from the class classification threshold storage unit 14. The class-specific search range setting unit 13 selects a search range corresponding to the distance from the class-specific search range Ω _c based on the distance calculated by the distance calculation unit 12 and the class classification threshold θ _c . The OF correction vector calculation processing unit 15 calculates an optical flow from the selected search range, and outputs the calculated optical flow to the OF correction image generation processing unit 16 as an OF correction vector. The OF corrected image generation processing unit 16 applies the calculated OF correction vector to the predicted image block and the motion vector output from the predicted image block generation processing unit 11, and generates an OF corrected predicted image block.
Thereby, the search range of the optical flow according to the position of the correction target pixel of the MC prediction block can be selected. By making the search range wider as the position of the pixel to be corrected moves away from the vicinity of the center of the MC prediction block, the search range can be narrowed for pixels near the center of the MC prediction block. By narrowing the search range, it is possible to reduce the amount of calculation required for calculating the optical flow. Further, since the search range becomes wider as the position of the pixel to be corrected approaches the outer edge of the MC prediction block, the optical flow can be calculated without reducing accuracy. Therefore, with the configuration of the first embodiment described above, it is possible to reduce the amount of calculation required for OF correction while suppressing a decrease in encoding efficiency.

(Second Embodiment)
FIG. 4 is a diagram illustrating a configuration of the inter prediction processing unit 110b according to the second embodiment of the present invention. The inter prediction processing unit 110b is a functional unit applied in place of the inter prediction processing unit 110 of the first embodiment in the video encoding device 100 illustrated in FIG. Hereinafter, the same components as those of the inter prediction processing unit 110 of the first embodiment are denoted by the same reference numerals, and different configurations will be described. The inter prediction processing unit 110b includes a motion vector search processing unit 10b, a predicted image block generation processing unit 11, a weight coefficient storage unit 21, a distance calculation unit 12, a class-specific search range setting unit 13, a class classification threshold storage unit 14, and an OF correction. A vector calculation processing unit 15 and an OF correction image generation processing unit 16 are provided.

  In the inter prediction processing unit 110 according to the first embodiment, it is assumed that the MC prediction error tends to increase as the distance from the center of the MC prediction block increases. However, depending on the characteristics of the video signal, the assumption may be clearly defined. The case where it does not hold is also assumed. Therefore, the inter prediction processing unit 110b according to the second embodiment multiplies a weight that decreases as the distance from the center of the MC prediction block increases, and detects a motion vector as shown in the following equation (13).

In Expression (13), c (B _i ) is a coordinate value at the center of the MC prediction block B _i , and W (pc (B _i )) is according to the distance from the center position of the MC prediction block. The weighting factor is set to decrease. As the weighting coefficient, for example, a Gaussian function represented by the following equation (14) can be used.

In equation (14), σ is a parameter that determines the amount of weight reduction. Further, in Expression (13), the portion of the following Expression (15) is the Euclidean distance between the pixel p and the central coordinate value c (B _i ). With the weighting shown in Equation (13), the MC prediction error near the center of the MC prediction block is reduced. By detecting the motion vector in this way, the nature of the MC prediction error assumed in the first embodiment becomes clear.

Hereinafter, an internal configuration of the inter prediction processing unit 110b will be described. In the inter prediction processing unit 110b, the weighting factor storage unit 21 stores the weighting factor W (pc (B _i )) calculated based on the equation (14) in advance, such as the producer of the moving picture coding apparatus 100, and the like. Written in response to the operation. That is, the weight coefficient storage unit 21 stores a weight coefficient for each pixel p in advance.
The motion vector search processing unit 10b detects a motion vector based on the encoding target image frame and the weighting factor stored in the weighting factor storage unit 21.

(Processing by the inter prediction processing unit of the second embodiment)
FIG. 5 is a flowchart showing the flow of processing by the inter prediction processing unit 110b. In the flowchart of FIG. 5, the processing by steps Sb1, Sb3, Sb4, Sb5, and Sb6 is the same processing as the processing of steps Sa1, Sa3, Sa4, Sa5, and Sa6 in FIG. Further, the loop processes Lb1s to Lb1e are processes that are repeated for all the MC prediction blocks of the encoding target image frame in the same manner as the loop processes La1s to La1e, and the loop processes Lb2s to Lb2e are the same as the loop processes La2s to La2e. This process is repeated for all the pixels in the prediction block. Hereinafter, processing different from that of the first embodiment will be described.

The motion vector search processing unit 10b reads the weighting factor from the weighting factor storage unit 21 (step Sb2-1). The motion vector search processing unit 10b captures an encoding target image frame. The motion vector search processing unit 10b uses the frame f _t-1 (p) _immediately before the fetched encoding target image frame as a reference image frame, and refers to an area corresponding to a block for performing MC prediction in the reference image frame. Read out from the frame memory 111 as an image block.

  The motion vector search processing unit 10b is based on Expression (13) based on the MC prediction block of the encoding target image frame, the reference image block corresponding to the block, and the weighting coefficient read from the weighting coefficient storage unit 21. Then, MC prediction processing is performed to detect a motion vector. The motion vector search processing unit 10b outputs the detected motion vector to the predicted image block generation processing unit 11 and the inter prediction information storage unit 113. The predicted image block generation processing unit 11 generates a predicted image block using a weighted motion vector. The predicted image block generation processing unit 11 outputs information indicating the MC predicted block to the distance calculation unit 12. The predicted image block generation processing unit 11 outputs the generated predicted image block to the OF corrected image generation processing unit 16 (Step Sb2-2).

  With the configuration of the second embodiment described above, the motion vector search processing unit 10b multiplies the motion vector by multiplying the weighting factor stored in the weighting factor storage unit 21, that is, the weight that decreases with increasing distance from the center position of the MC prediction block. To detect. Therefore, the ubiquity of the MC prediction error in the MC prediction block, that is, the property that the MC prediction error in the MC prediction block tends to increase as the distance from the center of the block increases becomes clear. Under this condition, the class-specific search range setting unit 13 selects a search range in which the optical flow search range becomes wider as the correction target pixel approaches the outer edge of the MC prediction block, and the OF correction vector calculation processing unit 15 However, the optical flow is calculated from the selected search range. As a result, it is possible to reduce the amount of calculation required for OF correction while suppressing a decrease in encoding efficiency with higher accuracy.

(Third embodiment)
FIG. 6 is a block diagram illustrating an internal configuration of the inter prediction processing unit 110c according to the third embodiment of the present invention. The inter prediction processing unit 110c is a functional unit applied in place of the inter prediction processing unit 110 of the first embodiment in the video encoding device 100 illustrated in FIG. The same configurations as those of the inter prediction processing unit 110 of the first embodiment are denoted by the same reference numerals, and different configurations will be described below. The inter prediction processing unit 110c includes a motion vector search processing unit 10, a predicted image block generation processing unit 11c, a motion vector determination unit 22, a switch 23, a distance calculation unit 12, a class-specific search range setting unit 13, and a class classification threshold storage unit 14. A search range setting unit 24, an OF correction vector calculation processing unit 15c, and an OF correction image generation processing unit 16.

  In the inter prediction processing unit 110 according to the first embodiment, it is assumed that the MC prediction error tends to increase as the distance from the center of the MC prediction block increases. However, depending on the characteristics of the video signal, the assumption may be clearly defined. The case where it does not hold is also assumed. Therefore, in the inter prediction processing unit 110c according to the third embodiment, if the motion vector of the target MC prediction block is aligned with the motion vectors of the neighboring blocks, the assumption is satisfied, and the inter prediction processing according to the first embodiment Processing by the unit 110 is performed. On the other hand, when the motion vector of the target MC prediction block is not aligned with the motion vectors of the neighboring blocks, the assumption is not satisfied, and the search range of the optical flow used for the OF correction is determined based on the position in the MC prediction block. Do a fixed search.

As a scale for evaluating that the motion vector of the target MC prediction block is aligned with the motion vectors of the neighboring blocks, for example, the motion vector divergence shown in the following equation (16) can be applied. When the degree of divergence in the equation (16) is less than a predetermined threshold, it is considered that the motion vector of the MC prediction block is aligned with the motion vectors of the neighboring blocks. It is assumed that the motion vectors of the prediction block are not aligned with the motion vectors of the surrounding blocks. In the equation (16), the vector v _i, a motion vector of MC prediction block to be processed, the vector v _n, a motion vector of neighboring blocks of MC prediction block to be processed. N indicates a set of peripheral block indexes. For example, four blocks adjacent to the processing target MC prediction block in the vertical and horizontal directions are set as peripheral blocks. Also, as shown in Expression (16), whether or not the motion vectors are aligned is based on whether or not the motion vectors are aligned in both planes of the vector magnitude and direction, and the motion vector of the MC prediction block When the block vectors have the same magnitude in the same direction, the divergence degree is minimized.

  Hereinafter, an internal configuration of the inter prediction processing unit 110c will be described. In the inter prediction processing unit 110c, the prediction image block generation processing unit 11c outputs the detected motion vector and information indicating the MC prediction block to the motion vector determination unit 22. Also, the predicted image block generation processing unit 11 c outputs the generated predicted image block to the OF corrected image generation processing unit 16.

  Based on the information indicating the MC prediction block output from the prediction image block generation processing unit 11c, the motion vector determination unit 22 uses the inter prediction information storage unit for the motion vectors of the four neighboring blocks adjacent to the MC prediction block in the vertical and horizontal directions. Read from 113. Further, the motion vector determination unit 22 calculates the degree of motion vector divergence from the motion vector of the MC prediction block output from the prediction image block generation processing unit 11c and the motion vector of the peripheral block read from the inter prediction information storage unit 113. To do. Further, the motion vector determination unit 22 determines whether or not the calculated divergence degree is less than a predetermined threshold, and based on the determination result, the connection destination of the switch 23 is the distance calculation unit 12 or The search range setting unit 24 is selected. In addition, the motion vector determination unit 22 outputs information indicating the MC prediction block to the connection destination of the switch 23. The switch 23 receives the instruction information from the motion vector determination unit 22 and connects the connection destination indicated by the instruction information to the motion vector determination unit 22.

  The search range setting unit 24 receives information indicating the MC prediction block output from the motion vector determination unit 22 and sets a predetermined search range that is a search range common to all pixels in the MC prediction block. It outputs to OF correction vector calculation process part 15c. That is, the search range is an area having a fixed size regardless of the position of the pixel.

The OF correction vector calculation processing unit 15c reads out frames before and after the encoding target image frame f _t (p) from the frame memory 111 of the moving image encoding device 100 as reference image frames. Further, the OF correction vector calculation processing unit 15 calculates the OF correction vector δ (p) based on the read reference image frame and the search range output by the class-specific search range setting unit 13 or the search range setting unit 24. Ask. Further, the OF correction vector calculation processing unit 15 outputs the calculated OF correction vector δ (p) to the OF correction image generation processing unit 16.

(Processing by the inter prediction processing unit of the third embodiment)
FIG. 7 is a flowchart showing a flow of processing by the inter prediction processing unit 110c. In the flowchart of FIG. 7, the processes in steps Sc1, Sc3, and Sc4 are the same as the processes in steps Sa1, Sa3, and Sa4 in FIG. 3, and the process in step Sc5 is performed by the OF correction vector calculation processing unit 15c. Similar to the OF correction vector calculation processing unit 15 of the first embodiment, the search range output by the class-specific search range setting unit 13 is received and the same processing as step Sa5 is performed. The loop processes Lc1s to Lc1e are processes that are repeated for all the MC prediction blocks of the encoding target image frame in the same way as the loop processes La1s to La1e, and the loop processes Lc2s to Lc2e are the MC prediction blocks similar to the loop processes La2s to La2e. This process is repeated for all the pixels. Hereinafter, processing different from that of the first embodiment will be described.

The motion vector search processing unit 10 captures an encoding target image frame. The motion vector search processing unit 10 uses the frame _ft-1 immediately before the fetched encoding target image frame as a reference image frame, and in the reference image frame, a region corresponding to a block on which MC prediction is performed is a reference image block. Is read from the frame memory 111.

  The motion vector search processing unit 10 performs MC prediction processing based on the MC prediction block of the encoding target image frame and the reference image block corresponding to the block, detects a motion vector, and detects the detected motion vector. The prediction image block generation processing unit 11 c and the inter prediction information storage unit 113 are output. The predicted image block generation processing unit 11c generates a predicted image block using the motion vector. The predicted image block generation processing unit 11 c outputs the detected motion vector and information indicating the MC predicted block to the motion vector determination unit 22. The predicted image block generation processing unit 11c outputs the generated predicted image block to the OF corrected image generation processing unit 16 (Step Sc2a).

  Based on the information indicating the MC prediction block output from the prediction image block generation processing unit 11c, the motion vector determination unit 22 uses the inter prediction information storage unit for the motion vectors of the four neighboring blocks adjacent to the MC prediction block in the vertical and horizontal directions. Read from 113 (step Sc2b). Based on the motion vector of the MC prediction block output from the prediction image block generation processing unit 11c and the motion vector of the peripheral block read from the inter prediction information storage unit 113, the motion vector determination unit 22 performs motion according to Expression (16). Calculate the degree of vector divergence. The motion vector determination unit 22 determines whether or not the calculated degree of divergence is less than a predetermined threshold value (step Sc2c).

  When the motion vector determination unit 22 determines that the calculated degree of divergence is less than a predetermined threshold value, the motion vector of the MC prediction block output from the prediction image block generation processing unit 11c is aligned with the motion vector of the peripheral block. The instruction information for connecting the switch 23 to the distance calculation unit 12 is output. The switch 23 receives the instruction information and connects the motion vector determination unit 22 and the distance calculation unit 12. Thereafter, the loops Lc2s to Lc2e are processed. On the other hand, when the motion vector determination unit 22 determines that the calculated degree of divergence is greater than or equal to a predetermined threshold, the motion vector of the MC prediction block output from the prediction image block generation processing unit 11c is the motion vector of the surrounding block. The instruction information for connecting the switch 23 to the search range setting unit 24 is output. The switch 23 receives the instruction information and connects the motion vector determination unit 22 and the search range setting unit 24. Then, the process of step Sc10 and step Sc11 is repeatedly performed for all the pixels in the MC prediction block.

The search range setting unit 24 receives information indicating the MC prediction block output from the motion vector determination unit 22 and sets a predetermined search range that is a search range common to all pixels in the MC prediction block. The result is output to the OF correction vector calculation processing unit 15c (step Sc10). The OF correction vector calculation processing unit 15c reads out frames before and after the encoding target image frame f _t (p) from the frame memory 111 of the moving image encoding device 100 as reference image frames. The OF correction vector calculation processing unit 15c calculates the OF correction vector δ (p) based on the read reference image frame and the search range output by the search range setting unit 24. Further, the OF correction vector calculation processing unit 15 outputs the calculated OF correction vector δ (p) to the OF correction image generation processing unit 16 (step Sc11).

  The OF corrected image generation processing unit 16 is based on the predicted image block and motion vector output from the predicted image block generation processing unit 11 and the OF correction vector δ (p) output from the OF correction vector calculation processing unit 15c. According to (7), a predicted image block subjected to OF correction is generated. The OF corrected image generation processing unit 16 outputs the predicted image block after the OF correction to the prediction residual generation unit 102 and the decoded signal generation unit 107. The OF corrected image generation processing unit 16 writes and stores the motion vector and the OF correction vector δ (p) in the inter prediction information storage unit 113 (step Sc15).

  With the configuration of the third embodiment, the motion vector determination unit 22 determines whether or not the motion vectors of the MC prediction block detected by the prediction image block generation processing unit 11c are aligned with the motion vectors of the peripheral blocks of the MC prediction block. Judgment. If the motion vector determination unit 22 determines that the motion vectors are aligned, it is assumed that the tendency of the MC prediction error to increase as the distance from the center of the MC prediction block increases, and the search range is determined by the pixel position. The optical flow is calculated by changing. On the other hand, when the motion vector determination unit 22 determines that the motion vectors are not aligned, the optical flow is calculated based on a fixed search range that does not depend on the pixel position. Thereby, the calculation process of the optical flow can be selected according to the state in which the tendency of increasing the MC prediction error appears as it moves away from the vicinity of the center of the MC prediction block and the state in which it cannot be considered. When the tendency does not appear, even if the optical flow calculation process is performed by changing the search range depending on the pixel position, the calculation amount cannot be reduced. Rather, the distance calculation process by the distance calculation unit 12 or There is also a possibility that the processing amount increases due to the threshold determination process in the class-specific search range setting unit 13. By selectively performing processing as described above, it is possible to reduce the amount of calculation required for OF correction while suppressing an increase in processing amount and suppressing a decrease in encoding efficiency.

(Video decoding device)
FIG. 8 is a block diagram illustrating a configuration of a moving picture decoding apparatus 200 that receives encoded stream data output from the moving picture encoding apparatus 100 of FIG. 1 and decodes a video signal. The video decoding device 200 is, for example, HEVC or H.264. It is an apparatus applied as a decoder in an encoding method such as H.264.

  The moving picture decoding apparatus 200 includes an inverse quantization processing unit 205, an inverse transformation processing unit 206, a decoded signal generation unit 207, an in-loop filter processing unit 208, a frame memory 211, an intra prediction processing unit 209, an inter prediction processing unit 210, an intra A prediction information storage unit 212, an inter prediction information storage unit 213, a switch 214, and an entropy decoding processing unit 215 are provided.

  In the video decoding device 200, the entropy decoding processing unit 215 takes in the encoded stream data output from the entropy encoding processing unit 115 of the video encoding device 100, and information related to intra prediction included in the encoded stream data. Is entropy-decoded and written and stored in the intra prediction information storage unit 212. The entropy decoding processing unit 215 entropy-decodes information related to inter prediction such as motion vectors and OF correction vectors included in the encoded stream data, and writes and stores the information in the inter prediction information storage unit 213. The entropy decoding processing unit 215 entropy-decodes the quantized transform coefficient of the decoding target block included in the encoded stream data.

  The inverse quantization processing unit 205 performs inverse quantization on the quantized transform coefficient output from the entropy decoding processing unit 215 and outputs a decoded transform coefficient. The inverse transform processing unit 206 performs inverse orthogonal transform on the decoded transform coefficient output from the inverse quantization processing unit 205, and outputs a prediction residual decoded signal. The decoded signal generation unit 207 adds the prediction signal output from the intra prediction processing unit 209 or the inter prediction processing unit 210 and the prediction residual decoded signal output from the inverse quantization processing unit 205 using an adder provided therein. Then, a decoded image block is generated. The in-loop filter processing unit 208 performs a filtering process for reducing the coding distortion on the decoded image block output from the decoded signal generation unit 207, and writes the decoded image block after the filtering process in the frame memory 211 for storage.

  The intra prediction processing unit 209 reads the decoded image block after the filtering process stored in the frame memory 211. Further, the intra prediction processing unit 209 generates and outputs a prediction image block, that is, a prediction signal, from the read decoded image block based on information such as a prediction mode stored in the intra prediction information storage unit 212.

  The inter prediction processing unit 210 reads the decoded image block after filtering processing stored in the frame memory 211. Further, the inter prediction processing unit 110 generates and outputs a prediction image block, that is, a prediction signal, from the decoded image block based on information such as a motion vector and an OF correction vector stored in the inter prediction information storage unit 213. .

  The switch 214 switches the prediction signal to be applied by switching between connecting the decoded signal generation unit 207 and the intra prediction processing unit 209 or connecting the decoded signal generation unit 207 and the inter prediction processing unit 210. .

(Application of the inter prediction unit of the first to third embodiments to the video decoding device 200)
FIG. 9 shows a configuration when the configuration of the inter prediction unit according to the first embodiment is applied to the video decoding device 200.
FIG. 9 is a block diagram illustrating an internal configuration of the inter prediction processing unit 210 of the video decoding device 200. The inter prediction processing unit 210 includes a prediction image block generation processing unit 31, a distance calculation unit 32, a class-specific search range setting unit 33, a class classification threshold storage unit 34, an OF correction vector calculation processing unit 35, and an OF correction image generation processing unit 36. Is provided.
The predicted image block generation processing unit 31 is based on the motion vector stored in the inter prediction information storage unit 213 and the reference image frame stored in the frame memory 211. The area is read from the frame memory 211 as a predicted image block. The predicted image block generation processing unit 31, the distance calculation unit 32, the class-specific search range setting unit 33, the class classification threshold storage unit 34, the OF correction vector calculation processing unit 35, and the OF correction image generation processing unit 36 are a moving image encoding device. The same processing as that of the functional unit having the same name included in the 100 inter prediction processing units 110 is performed.

FIG. 10 shows a configuration when the configuration of the inter prediction unit according to the second embodiment is applied to the video decoding device 200.
FIG. 10 is a block diagram illustrating an internal configuration of the inter prediction processing unit 210b of the video decoding device 200. The inter prediction processing unit 210b includes a prediction image block generation processing unit 31b, a distance calculation unit 32, a class-specific search range setting unit 33, a class classification threshold storage unit 34, an OF correction vector calculation processing unit 35, and an OF correction image generation processing unit 36. Is provided.
The predicted image block generation processing unit 31 b uses the weighted motion vector stored in the inter prediction information storage unit 213 and the reference image specified by the motion vector based on the reference image frame stored in the frame memory 211. A region in the frame is read from the frame memory 211 as a predicted image block. The predicted image block generation processing unit 31b, the distance calculation unit 32, the class-specific search range setting unit 33, the class classification threshold storage unit 34, the OF correction vector calculation processing unit 35, and the OF correction image generation processing unit 36 are a moving image encoding device. Processing similar to that of the functional unit having the same name included in the 100 inter prediction processing units 110b is performed.

FIG. 10 shows a configuration when the configuration of the inter prediction unit of the third embodiment is applied to the video decoding device 200.
FIG. 11 is a block diagram illustrating an internal configuration of the inter prediction processing unit 210c of the video decoding device 200. The inter prediction processing unit 210c includes a predicted image block generation processing unit 31c, a motion vector determination unit 42, a switch 43, a distance calculation unit 32, a class-specific search range setting unit 33, a class classification threshold storage unit 34, a search range setting unit 44, An OF correction vector calculation processing unit 35c and an OF correction image generation processing unit 36 are provided.
The predicted image block generation processing unit 31c is based on the motion vector stored in the inter prediction information storage unit 213 and the reference image frame stored in the frame memory 211. The area is read from the frame memory 211 as a predicted image block. The predicted image block generation processing unit 31c, the distance calculation unit 32, the class-specific search range setting unit 33, the class classification threshold storage unit 34, the OF correction vector calculation processing unit 35, and the OF correction image generation processing unit 36 are a moving image encoding device. The same processing as that of the functional unit having the same name included in the 100 inter prediction processing units 110c is performed.

In the first to third embodiments described above, the class classification threshold value storage unit 14 includes a class-specific search range Ω _c (c = 0,..., C−1) defined by the equation (9), and Information indicating the class-specific search range associated with the class classification threshold θ _c (c = 0,..., C−1) is stored in advance. Not limited to. For example, information indicating the search range for each class may be read from an external memory or the like each time processing is started. By doing so, the class-specific search range Ω _c and the class classification threshold θ _c can be easily changed simply by rewriting information in the memory or the like.

  In the first to third embodiments, the MC prediction process calculates the motion vector for the previous frame as shown in the equation (2). Bidirectional MC prediction processing for calculating a motion vector using future and past frames may be used.

  In the first to third embodiments, the optical flow, that is, the OF correction vector is calculated using the reference image frames before and after the encoding target image frame as shown in Expression (12). Any method may be used as long as the optical flow is calculated, and may be calculated from the encoding target image frame and the past reference image frame, or the encoding target image frame and the future reference image frame. You may make it calculate from.

  In the third embodiment, the neighboring blocks are four blocks that are adjacent to the MC prediction block in the upper, lower, left, and right directions, but the upper right, lower right, upper left, and lower left are added to the peripheral blocks. You may make it do.

  Moreover, in said 3rd Embodiment, using the divergence degree of the motion vector shown in following Formula (16) as a scale for evaluating that the motion vector of the target MC prediction block is aligned with the motion vectors of the surrounding blocks. However, the configuration of the present invention is not limited to the embodiment. For example, it may be determined whether only the direction of the vector is observed or not based on the degree of coincidence of the directions.

  Further, the motion vector search processing unit 10b of the second embodiment may be applied as the motion vector search processing unit 10 of the third embodiment. In addition to the predicted image block generation processing unit 11c, a motion vector search processing unit 10b is provided, and when it is selected as a result of determination by the motion vector determination unit 22 to connect to the distance calculation unit 12, You may make it make the vector search process part 10b detect the weighted motion vector.

  In addition, the motion vector determination unit 22 of the third embodiment performs the determination based on whether or not the calculated divergence degree is less than a predetermined threshold value. The determination is not limited to the embodiment, and the determination may be made based on whether or not it is equal to or less than a threshold value. That is, the comparison of the values in the above embodiment is an example. For example, ≦ in formula (9) may be <.

  The prediction image generation device that is the moving image encoding device 100, the moving image decoding device 200, and the inter prediction processing units 110, 110b, 110c, 210, 210b, and 210c in the above-described embodiments may be realized by a computer. . In that case, a program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into a computer system and executed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time, like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory inside a computer system serving as a server or a client in that case may be included and a program held for a certain period of time. Further, the program may be a program for realizing a part of the above-described functions, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system. You may implement | achieve using programmable logic devices, such as FPGA (Field Programmable Gate Array).

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

DESCRIPTION OF SYMBOLS 10 ... Motion vector search process part, 11 ... Predictive image block generation process part, 12 ... Distance calculation part, 13 ... Search range setting part according to class, 14 ... Class classification threshold value memory | storage part, 15 ... OF correction vector calculation process part, 16 ... OF correction image generation processing unit, 110 ... Inter prediction processing unit

Claims (6)

A prediction image generation method for performing motion compensation inter-frame prediction for each block obtained by dividing an encoding target image frame in moving image data,
A prediction processing step of detecting a motion vector for each block based on a block of the encoding target image frame and a block of a reference image frame corresponding to the block;
A class for selecting the search range corresponding to the pixel to be detected from the search range that changes according to the distance from the center of the block to the pixel when detecting an optical flow for each pixel included in the block. Another search range setting step;
An optical flow correction vector calculation processing step of calculating an optical flow for each pixel based on the selected search range, and using the calculated optical flow as an optical flow correction vector for each pixel;
An optical flow correction image generation processing step of generating a predicted image block based on the motion vector corrected by the optical flow correction vector for each pixel;
A predicted image generation method comprising:   The predicted image generation method according to claim 1, wherein the search range is determined in advance such that the search range expands as the pixel to be detected by the optical flow moves away from the center of the block.   The predicted image generation method according to claim 1, wherein in the prediction processing step, the motion vector is detected by performing weighting that decreases in accordance with a distance from a center of the block. A search range setting step for selecting the same search range for all the pixels;
A motion vector determination step of determining whether or not the motion vector of the block and the motion vectors of blocks around the block are aligned;
Further comprising
In the class-specific search range setting step, when it is determined in the motion vector determination step that the motion vector of the block and the motion vectors of blocks around the block are aligned, from the center of the block to the pixel The search range corresponding to the pixel is selected from the search range that changes according to the distance of
In the search range setting step, when it is determined in the motion vector determination step that the motion vector of the block and a motion vector of a block around the block are not aligned, the same search range is selected. Item 4. The predicted image generation method according to any one of Items 1 to 3. A prediction image generation apparatus that performs motion compensation inter-frame prediction in units of blocks obtained by dividing an encoding target image frame in moving image data,
A prediction processing unit that detects a motion vector for each block based on the block of the encoding target image frame and the block of the reference image frame corresponding to the block;
A class for selecting the search range corresponding to the pixel to be detected from the search range that changes according to the distance from the center of the block to the pixel when detecting an optical flow for each pixel included in the block. Another search range setting section;
An optical flow correction vector calculation processing unit that calculates an optical flow for each pixel based on the selected search range, and uses the calculated optical flow as an optical flow correction vector for each pixel;
An optical flow correction image generation processing unit that generates a predicted image block based on the motion vector corrected by the optical flow correction vector for each pixel;
A predicted image generation apparatus comprising:   A computer program for causing a computer to execute the predicted image generation method according to any one of claims 1 to 4.

Images