JP7252016B2 - Video encoding device, video decoding device and program - Google Patents

Description

The present invention relates to a moving image encoding device that encodes a moving image, a moving image decoding device that decodes an encoded signal, and a program.

Conventionally, H.265/HEVC (High Efficiency Video Coding) is known as a standard of a moving image compression method for high-resolution 4K or 8K moving images (see, for example, Non-Patent Document 1). The H.265/HEVC standard is based on motion compensation and orthogonal transform technology that predicts motion using encoded frames, orthogonally transforms prediction residual signals, and encodes them.

FIG. 14 is a block diagram showing a configuration example of a conventional video encoding device, showing the configuration on the encoding side according to the H.265/HEVC standard. This video encoding device 100 includes a subtraction unit 110, an orthogonal transformation unit 111, a quantization unit 112, an inverse quantization unit 113, an inverse orthogonal transformation unit 114, an addition unit 115, an intra-screen prediction unit 116, an in-loop filter 117, It has an inter-frame prediction unit 118 , a switch 119 and an entropy coding unit 120 . The inter-screen prediction unit 118 includes an inter-screen prediction processing unit 121 and a frame memory 122 .

The video encoding device 100 is a device that performs encoding processing on an input video for each picture of one frame, and generates and outputs an encoded signal.

A picture of the input moving image is subtracted from the prediction image generated by the intra prediction section 116 or the inter prediction section 118 by the subtraction section 110 . This produces a residual image.

The residual image is subjected to orthogonal transformation processing by the orthogonal transformation unit 111, quantization processing by the quantization unit 112, and entropy encoding by the entropy encoding unit 120, and is output as an encoded signal.

On the other hand, the quantization index sequence quantized by the quantization section 112 is subjected to inverse quantization processing by the inverse quantization section 113 and inverse orthogonal transformation processing by the inverse orthogonal transformation section 114 . A decoded residual image is thereby generated.

Addition processing is performed between the decoded residual image and the predicted image by the addition unit 115 . The image resulting from the addition by the addition unit 115 is subjected to intra-screen prediction processing by the intra-screen prediction unit 116 . The image resulting from the addition is filtered by the in-loop filter 117 and stored in the frame memory 122 of the inter prediction unit 118 as a locally decoded picture.

The locally decoded picture stored in the frame memory 122 is read as a past locally decoded picture by the inter prediction processing unit 121 . Then, inter prediction processing is performed using the current locally decoded picture (input video picture) as a reference picture and the past locally decoded picture as a reference picture.

One of the image generated by the intra-screen prediction processing by the intra-screen prediction unit 116 and the image generated by the inter-screen prediction processing by the inter-screen prediction processing unit 121 is selected by the switch 119, and the predicted image is is input to the subtraction unit 110 as .

Techniques for improving coding efficiency have been proposed in moving image coding techniques based on such motion compensation and orthogonal transform. For example, in order to improve the coding efficiency of color difference signals, there is a technique of performing color difference interpolation by applying super-resolution processing using multiple frames and converting a color difference thinned image to YCbCr4:4:4 (see, for example, Patent Document 1).

In addition, in order to improve the coding efficiency of moving images, the image is shrunk before being encoded and transmitted, super-resolution restoration is performed after decoding, and the optimal folding frequency is set according to the complexity of the image. There is also a technique for improving the super-resolution image quality by transmitting as such (see Patent Document 2, for example).

Japanese Patent No. 6152642 Japanese Patent No. 6344800

Sakae Okubo, "Impress Standard Textbook Series H.265/HEVC Textbook", Impress Japan

In the conventional moving image coding technology based on motion compensation and orthogonal transformation, an image with a minimum prediction error is selected by block matching, for example, as a predicted image generated by inter-frame prediction processing. As a result, the encoding efficiency can be improved, and a moving image with objectively high image quality can be obtained on the decoding side.

However, region-to-region blurring or sharpening may occur as objects move between pictures of the moving image. Conventional video coding techniques, for example, generate predicted images using normal locally decoded images, even for moving images that are blurred or sharpened.

For this reason, it is not possible to perform inter-frame prediction processing that distinguishes moving images that are blurry or sharpened from normal moving images (moving images that are not blurred or sharpened). In addition, it is not possible to perform inter-frame prediction by distinguishing between a moving image that blurs or sharpens due to horizontal movement and a moving image that blurs or sharpens due to vertical movement.

As described above, the conventional moving image coding technology can objectively obtain a high-quality moving image, but there is a problem that it is not always possible to obtain a subjectively high-quality moving image. rice field.

SUMMARY OF THE INVENTION Accordingly, the present invention has been made to solve the above-described problems, and an object of the present invention is to achieve subjectively high image quality even when blurring or sharpening of objects occurs in each region between pictures of a moving image. It is an object of the present invention to provide a moving image encoding device, a moving image decoding device, and a program capable of obtaining a dynamic image.

In order to solve the above-mentioned problems, the moving image coding apparatus of claim 1 subtracts a predicted image from an input moving image to generate a residual image, and performs orthogonal transformation, quantization, and entropy coding on the residual image. generating and outputting a coded signal, performing inverse quantization and inverse orthogonal transformation on the quantization index sequence generated by the quantization, generating a decoded residual image, and generating the decoded residual image; generates a post-addition image by adding the predicted image to , performs filtering on the post-addition image to generate a locally decoded picture, performs inter prediction using the locally decoded picture, and generates the predicted image generating a plurality of super-resolution pictures based on the locally decoded picture, and predicting the locally decoded picture and the plurality of super-resolution pictures for each predetermined region by the inter-picture prediction; an inter-picture prediction unit that generates prediction images of pictures and selects one of the prediction images, the inter-picture prediction unit performs super-resolution processing on the locally decoded picture, Super-resolution for generating a picture in which a directional high-frequency band component is emphasized, a picture in which a vertical high-frequency band component is emphasized, and a picture in which a diagonal high-frequency band component is emphasized, as the plurality of super-resolution pictures. and a processing unit, using the current locally decoded picture as a reference picture, and using the past local decoded picture and each of the plurality of past super-resolution pictures generated by the super-resolution processing unit as reference pictures, and An inter-screen prediction processing unit that performs prediction, generates a predicted image and a motion vector of the locally decoded picture for each of the predetermined regions, and generates a predicted image and a motion vector of the plurality of super-resolution pictures; a frequency analysis unit that performs frequency analysis on the locally decoded picture and the past locally decoded picture and generates power fluctuation information between the current locally decoded picture and the past locally decoded picture; , the power fluctuation information generated by the frequency analysis unit, or the power fluctuation information and the motion vector of the locally decoded picture and the motion vector of the plurality of super-resolution pictures generated by the inter prediction processing unit and a selection unit that selects one of the prediction image of the locally decoded picture and the prediction images of the plurality of super-resolution pictures based on the prediction image.

Further, the moving image coding apparatus of claim 2 subtracts a predicted image from an input moving image to generate a residual image, performs orthogonal transformation, quantization, and entropy coding on the residual image, and encodes the residual image. Generate and output a signal, perform inverse quantization and inverse orthogonal transformation on the quantization index sequence generated by the quantization, generate a decoded residual image, and add the predicted image to the decoded residual image. a video coding apparatus for generating an image after addition, performing filtering on the image after addition to generate a locally decoded picture, performing inter-picture prediction using the locally decoded picture, and generating the predicted image generating a plurality of blurred pictures based on the locally decoded picture; generating a predicted image of the locally decoded picture and a predicted image of the plurality of blurred pictures for each predetermined region by the inter prediction; An inter-picture prediction unit that selects one of the predicted images, the inter-picture prediction unit performs blurring processing on the locally decoded picture, suppresses horizontal high-frequency band components, a blur processing unit that generates, as the plurality of blurry pictures, a picture in which a high-frequency band component in a direction is suppressed and a picture in which a high-frequency band component in a diagonal direction is suppressed; and the plurality of past blurred pictures generated by the blur processing unit are used as reference pictures to perform the inter-frame prediction, and for each predetermined region, the predicted image and motion vector of the locally decoded picture are obtained. is generated, and an inter-picture prediction processing unit that generates predicted images and motion vectors of the plurality of blurred pictures, performs frequency analysis of the current locally decoded picture and the past locally decoded picture, and performs the current local decoding a frequency analysis unit that generates power fluctuation information between a picture and the previous locally decoded picture; and for each of the predetermined regions, the power fluctuation information generated by the frequency analysis unit, or the power fluctuation information and the one of a predicted image of the locally decoded picture and a predicted image of the plurality of blurred pictures based on the motion vector of the locally decoded picture and the motion vectors of the plurality of blurred pictures generated by an inter prediction processing unit; and a selection unit for selecting one.

Further, the video encoding device according to claim 3 is the video encoding device according to claim 1, wherein the plurality of super-resolution pictures and the plurality of blurred pictures are generated based on the locally decoded picture, By inter-frame prediction, for each of the predetermined regions, a predicted image of the locally decoded picture, a predicted image of the plurality of super-resolution pictures, and a predicted image of the plurality of blurred pictures are generated, and one of these predicted images is generated. An inter-picture prediction unit that selects any one of them, wherein the inter-picture prediction unit further applies blurring processing to the locally decoded picture, a picture in which horizontal high-frequency band components are suppressed, and a vertical high-frequency band and a picture with suppressed high-frequency band components in the diagonal direction as the plurality of blurred pictures, wherein the inter-prediction processing unit converts the current locally decoded picture into As a reference picture, each of the past locally decoded picture, the plurality of past super-resolution pictures generated by the super-resolution processing unit, and the plurality of past blurry pictures generated by the blurring processing unit The inter-prediction is performed as a reference picture, and for each of the predetermined regions, the predicted image and motion vector of the locally decoded picture, the predicted image and motion vector of the plurality of super-resolution pictures, and the predicted image of the plurality of blurred pictures. and a motion vector, and the selection unit selects the power fluctuation information generated by the frequency analysis unit for each predetermined region, or the power fluctuation information and the local region generated by the inter-screen prediction processing unit Based on the motion vector of the decoded picture, the motion vector of the plurality of super-resolution pictures, and the motion vector of the plurality of blurred pictures, the prediction image of the locally decoded picture, the prediction image of the plurality of super-resolution pictures, and the plurality of and selecting any one of the predicted images of the blurred pictures of the .

Further, the moving picture coding apparatus according to claim 4 is the moving picture coding apparatus according to any one of claims 1 to 3, wherein the frequency analysis unit uses the current locally decoded picture and the past For the locally decoded picture, the frequency analysis is performed by first-order wavelet packet decomposition without decimation, and based on the analysis result of the current locally decoded picture and the analysis result of the past locally decoded picture, each frequency band and phase It is characterized by generating the power fluctuation information for each position.

Further, the moving image decoding apparatus according to claim 5 receives a coded signal of a moving image, performs entropy decoding, inverse quantization, and inverse orthogonal transform on the coded signal, generates a decoded residual image, adding a predicted image to a decoded residual image to generate a post-addition image, performing filtering on the post-addition image to generate a decoded picture, thereby restoring the original moving image, and using the decoded picture; In a video decoding device that performs inter-picture prediction and generates the predicted image, the encoded signal includes one predicted image selected from a predicted image of a locally decoded picture and a plurality of predicted images of super-resolution pictures. When a parameter indicating a type is included, a plurality of super-resolution pictures are generated based on the decoded picture, and the inter-prediction is performed to generate the decoded picture and the plurality of super-resolution pictures for each predetermined region according to the parameter. An inter-picture prediction unit for generating a predicted image of any one of the resolution pictures, wherein the inter-picture prediction unit performs super-resolution processing on the decoded picture to emphasize horizontal high-frequency band components. A super-resolution processing unit that generates a picture, a picture in which vertical high-frequency band components are emphasized, and a picture in which diagonal high-frequency band components are emphasized, as the plurality of super-resolution pictures; A screen for performing the inter-screen prediction using a decoded picture and one of the plurality of super-resolution pictures generated by the super-resolution processing unit to generate the predicted image for each of the predetermined regions. and an inter-prediction processing unit.

Further, the moving image decoding apparatus according to claim 6 receives a coded signal of a moving image, performs entropy decoding, inverse quantization, and inverse orthogonal transform on the coded signal to generate a decoded residual image, adding a predicted image to a decoded residual image to generate a post-addition image, performing filtering on the post-addition image to generate a decoded picture, thereby restoring the original moving image, and using the decoded picture; In a moving picture decoding device that performs inter-picture prediction by using the coded signal to generate the predicted picture, the type of one of the predicted pictures selected from the predicted picture of a locally decoded picture and the predicted picture of a plurality of blurry pictures is included in the encoded signal. generating a plurality of blurred pictures based on the decoded picture, and generating a plurality of blurred pictures based on the decoded picture for each predetermined region according to the parameter, among the decoded picture and the plurality of blurred pictures, by the inter prediction An inter-prediction unit that generates one of the predicted images, the inter-prediction unit performs blur processing on the decoded picture, suppresses horizontal high-frequency band components, vertical high-frequency band a blur processing unit that generates, as the plurality of blurred pictures, pictures with suppressed components and pictures with suppressed high-frequency band components in a diagonal direction; an inter-picture prediction processing unit that performs the inter-picture prediction using any one of a plurality of blurred pictures and generates the predicted image for each of the predetermined regions.

Further, the moving image decoding device according to claim 7 is the moving image decoding device according to claim 5, wherein the encoded signal includes a predicted image of the locally decoded picture, a predicted image of the plurality of super-resolution pictures, and a plurality of super-resolution pictures. the plurality of super-resolution pictures and the plurality of blurred pictures are generated based on the decoded picture, and the screen an inter prediction unit that generates a predicted image of any one of the decoded picture, the plurality of super-resolution pictures, and the plurality of blurred pictures according to the parameter for each of the predetermined regions by inter prediction; The inter-frame prediction unit further performs blurring processing on the decoded picture, a picture with suppressed high-frequency band components in the horizontal direction, a picture with suppressed high-frequency band components in the vertical direction, and a high-frequency band component in the diagonal direction. a blur processing unit that generates the suppressed pictures as the plurality of blurred pictures, and the inter-prediction processing unit generates the decoded picture and the plurality of super-resolutions generated by the super-resolution processing unit according to the parameters; The inter prediction is performed using an image picture and any one of the plurality of blurred pictures generated by the blur processing unit to generate the predicted image for each of the predetermined regions. do.

Further, the program according to claim 8 causes a computer to function as the moving image encoding device according to any one of claims 1 to 4.

A program according to claim 9 causes a computer to function as the moving image decoding device according to any one of claims 5 to 7.

As described above, according to the present invention, it is possible to obtain subjectively high-quality moving images even when objects are blurred or sharpened in each area between pictures of the moving images.

1 is a block diagram showing a configuration example of a video encoding device according to an embodiment of the present invention; FIG. It is a block diagram which shows the structural example of the inter-screen prediction part with which the video encoding apparatus was equipped. It is a figure explaining the flow of a process from an inverse orthogonal transformation part to an inter-screen prediction part via an addition part and an in-loop filter. It is a figure explaining the example of a process of a super-resolution process part. It is a figure explaining the example of a process of a blur process part. It is a block diagram which shows the structural example of an inter-screen prediction process part. It is a figure explaining the example of a process of a frequency-analysis part. 4 is a block diagram showing a configuration example of a selection unit; FIG. 8 is a flow chart showing a processing example of a selection unit; FIG. 10 is a diagram illustrating an example of motion amounts CV _H −m _{H, V, and D} of a horizontal super-resolution picture calculated by a motion amount calculator; FIG. 5 is a diagram for explaining an example of power fluctuation amounts p _{H, V, and D} calculated by a power fluctuation amount calculator; 1 is a block diagram showing a configuration example of a video decoding device according to an embodiment of the present invention; FIG. It is a block diagram which shows the structural example of the inter-frame prediction part with which the video decoding apparatus was equipped. 1 is a block diagram showing a configuration example of a conventional video encoding device; FIG.

EMBODIMENT OF THE INVENTION Hereinafter, the form for implementing this invention is demonstrated in detail using drawing. In the video coding technology based on motion compensation and orthogonal transform, the present invention uses a locally decoded picture K as a reference picture for inter-picture prediction processing, as well as a plurality of super-resolution pictures C and/or a plurality of blurring pictures. It is characterized by generating a picture B and selecting an optimum prediction image for each area in consideration of the amount of power fluctuation, the amount of motion, etc. of the picture.

As a result, when blurring of an object occurs between pictures of the moving image from the past picture to the current picture, the prediction image of the blurry picture B is selected for each region. Also, when object sharpening occurs, a super-resolution picture C is selected on a region-by-region basis. Therefore, it is possible to reduce the residual error between the input moving image and the predicted image.

Also, on the decoding side, when an object blur occurs in the original moving image, a predicted image of the blurred picture B' is generated, and the blurred moving image can be restored. Furthermore, if object sharpening occurs in the original video, a super-resolution picture C' can be generated to recover the video in which the sharpening occurs.

Therefore, it is possible to improve the efficiency of inter-picture prediction and obtain subjectively high-quality moving images even when objects are blurred or sharpened in each area between pictures of moving images. becomes.

[Video encoding device]
First, a video encoding device according to an embodiment of the present invention will be described. FIG. 1 is a block diagram showing a configuration example of a video encoding device according to an embodiment of the present invention. This video encoding device 1 includes a subtraction unit 110, an orthogonal transformation unit 111, a quantization unit 112, an inverse quantization unit 113, an inverse orthogonal transformation unit 114, an addition unit 115, an intra-screen prediction unit 116, an in-loop filter 117, It has an inter-frame prediction unit 10 , a switch 119 and an entropy coding unit 120 . Note that FIG. 1 shows only components directly related to the present invention, and components not directly related are omitted.

The subtraction unit 110 receives pictures (I picture, P picture, and B picture) of the input moving picture, and also receives a predicted picture of the picture from the switch 119 . The subtraction unit 110 then subtracts the prediction image from the picture of the input moving image, generates a residual image as a result of the subtraction, and outputs the residual image to the orthogonal transformation unit 111 .

The orthogonal transformation unit 111 receives the residual image from the subtraction unit 110, performs orthogonal transformation on the residual image, and generates a transform coefficient sequence. Orthogonal transform section 111 then outputs the transform coefficient sequence to quantization section 112 .

The quantization unit 112 receives the transform coefficient sequence from the orthogonal transform unit 111, quantizes the transform coefficient sequence, and generates a quantization index sequence. Quantization section 112 then outputs the quantization index sequence to inverse quantization section 113 and entropy coding section 120 .

The inverse quantization unit 113 receives the quantization index sequence from the quantization unit 112 and performs reverse processing of the quantization unit 112 to inversely quantize the quantization index sequence and generate a transform coefficient sequence. Then, inverse quantization section 113 outputs the transform coefficient sequence to inverse orthogonal transform section 114 .

The inverse orthogonal transform unit 114 receives the transform coefficient sequence from the inverse quantization unit 113 and performs the inverse process of the orthogonal transform unit 111 to inverse orthogonal transform the transform coefficient sequence and generates a decoded residual image. Inverse orthogonal transform section 114 then outputs the decoded residual image to addition section 115 .

The adder 115 receives the decoded residual image from the inverse orthogonal transform unit 114 and the predicted image from the switch 119 . The addition unit 115 then adds the decoded residual image to the predicted image, and outputs the image after the addition to the intra-frame prediction unit 116 and the in-loop filter 117 .

The intra-screen prediction unit 116 receives the added image from the addition unit 115, performs intra-screen prediction processing on the added image for each predetermined region, generates an intra-screen prediction result, and generates an intra-screen prediction result for each predetermined region. The prediction result is output to switch 119 .

The in-loop filter 117 receives the added image from the adder 115, performs filtering on the added image, and generates a locally decoded picture K. FIG. The in-loop filter 117 then outputs the locally decoded picture K to the inter prediction unit 10 .

The inter prediction unit 10 receives the locally decoded picture K from the in-loop filter 117 and generates a plurality of super-resolution pictures C and a plurality of blurred pictures B from the locally decoded picture K. FIG. A locally decoded picture K, a plurality of super-resolved pictures C, and a plurality of blurred pictures B are used as reference pictures for inter-prediction processing.

The inter-prediction unit 10 performs inter-prediction processing using a reference picture that is the current local decoded picture K (picture of the input video) and a reference picture such as the local decoded picture K for each predetermined region. The inter-picture prediction unit 10 obtains a motion vector, a predicted image, and a prediction error for each of the locally decoded picture K, multiple super-resolution pictures C, and multiple blurred pictures B for each predetermined region.

The inter-picture prediction unit 10 obtains a weight for each predetermined region from the power fluctuation amount of the picture, etc., multiplies the prediction error by the weight, and selects the image of the picture with the smallest prediction error after weight multiplication as the optimum predicted image. Select as Then, the inter-screen prediction unit 10 outputs the predicted image selected for each predetermined area to the switch 119 as an inter-screen prediction result for each predetermined area.

In addition, in the process of selecting the optimum predicted image for each predetermined region, the inter-screen prediction unit 10 sets parameters such as the type of predicted image actually selected for each predetermined region, the calculated motion vector, and the like. The inter prediction unit 10 then outputs the parameters to the entropy coding unit 120 . Details of the inter-screen prediction unit 10 will be described later.

The switch 119 receives the intra-prediction result for each predetermined region from the intra-prediction unit 116 and the inter-prediction result for each predetermined region from the inter-prediction unit 10, selects one of them, and selects the predicted image. , to the subtraction unit 110 and the addition unit 115 .

The entropy coding unit 120 receives the quantization index sequence from the quantization unit 112, the parameters from the inter-frame prediction unit 10, and the parameters from the orthogonal transformation unit 111 and the like. Then, entropy coding section 120 performs entropy coding on the quantization index sequence and parameters to generate a coded signal. The entropy coding unit 120 outputs the coded signal to the video decoding device 2, which will be described later.

[Inter-screen prediction unit 10]
Next, the inter-screen prediction unit 10 shown in FIG. 1 will be described in detail. FIG. 2 is a block diagram showing a configuration example of the inter-frame prediction unit 10 provided in the video encoding device 1. As shown in FIG. The inter-screen prediction unit 10 includes a super-resolution processing unit 11 , a blur processing unit 12 , a frame memory 13 , an inter-screen prediction processing unit 14 , a frequency analysis unit 15 , a selection unit 16 and a parameter processing unit 17 . FIG. 2 shows only components directly related to the present invention, and components not directly related are omitted.

The inter prediction unit 10 receives the locally decoded picture K from the in-loop filter 117 and stores the locally decoded picture K in the frame memory 13 .

The super-resolution processing unit 11 performs super-resolution processing on the local decoded picture K, and produces three types of super-resolution pictures _{CH, V, D} (horizontal super-resolution picture C _H , vertical super-resolution picture C _V and Generate a diagonal super-resolution picture C _D ). H indicates horizontal, V indicates vertical, and D indicates diagonal. The super-resolution processor 11 stores three types of super-resolution pictures C _{H , V, and D} in the frame memory 13 . Details of the super-resolution processing unit 11 will be described later.

The horizontal super-resolution picture C _H is an image in which horizontal high-frequency band components are emphasized. The vertical super-resolution picture C _V is an image in which components in the vertical high-frequency band are emphasized. The diagonal super-resolution picture C _D is an image in which high-frequency band components in the diagonal direction (horizontal and vertical directions) are emphasized.

The blur processing unit 12 performs blur processing on the locally decoded picture K to generate three types of blurred pictures B _{H , V, D} (horizontal blurred picture B _H , vertical blurred picture B _V and diagonal blurred picture B _D ). . The blur processor 12 stores three types of blurred pictures B _{H , V, and D} in the frame memory 13 . Details of the blur processing unit 12 will be described later.

The horizontally blurred picture _BH is an image in which horizontal high frequency band components are suppressed. The vertically blurred picture B _V is an image in which components in the vertical high frequency band are suppressed. The diagonally blurred picture B _D is an image in which high-frequency components in the diagonal direction (horizontal and vertical directions) are suppressed.

The frame memory 13 stores a locally decoded picture K, three types of super-resolution pictures CH _{, V, D} , and three types of blurred pictures B _{H, V, D} for each frame.

The inter-picture prediction processing unit 14 converts the local decoded picture K, the three types of super-resolution pictures C _{H, V, D} , and the three types of blurred pictures B _{H, V, D} stored in the frame memory 13 into previous local predictions. Read out as decoded picture K, super-resolution picture C _{H, V, D} , and blurred picture B _{H, V, D.} Also, the inter prediction processing unit 14 inputs the current locally decoded picture K (picture of the input moving image).

The inter-screen prediction processing unit 14 uses the current local decoded picture K as a reference picture, and the past local decoded picture K, super-resolution pictures C _{H, V, D} , and blurred pictures B _{H, V, D} as reference pictures, Predicted images KY, CYH _{,V,D, BYH,V, D} , motion vectors KV, CVH _,V,D , BVH _,V,D and prediction error Find KE, CE _H,V,D and BE _H,V,D . The predetermined area is, for example, a slice area, a CTU (Coding Tree Unit) area, or a CU (Coding Unit) area.

The inter-screen prediction processing unit 14 predicts predicted images KY, CYH _,V,D , BYH _,V,D , motion vectors KV, CVH _,V,D , _BVH,V,D and prediction errors for each predetermined region. KE, CE _H,V,D and BE _H,V,D are output to the selector 16 . Details of the inter-screen prediction processing unit 14 will be described later.

The predicted image KY is a predicted image of one type of locally decoded picture. The predicted images CY _{H, V, and D} are predicted images of three types of super-resolution pictures, a predicted image CY _H of a horizontal super-resolution picture, a predicted image CY _V of a vertical super-resolution picture, and a diagonal super-resolution picture. It consists of the predicted image CY _D of the picture.

The predicted images _{BYH, V, and D} are predicted images of three _types of blurred pictures, and consist of a predicted image BYH of a horizontally blurred picture, a predicted image _BYV of a vertically blurred picture, and a predicted image _BYD of a diagonally blurred picture. .

A motion vector KV is a motion vector of one type of locally decoded picture. The motion vectors CV _{H, V, and D} are the motion vectors of three types of super-resolution pictures _{: the motion vector CV H of the horizontal super-resolution picture, the motion vector CV V of} the vertical super-resolution picture, and the diagonal super-resolution picture. It consists of the motion vector CV _D of the picture.

The motion vectors BVH _{, V, D} are the motion vectors of three types of blurred pictures, consisting of the motion vector _BVH of the horizontal blurred picture, the motion vector _BVV of the vertical blurred picture, and the motion vector _BVD of the diagonal blurred picture. .

The prediction error KE is the prediction error of one type of locally decoded picture. The prediction errors CE _{H, V, and D} are the prediction errors of three types of super-resolution pictures, the prediction error CE _H of the horizontal super-resolution picture, the prediction error CE _V of the vertical super-resolution picture, and the diagonal super-resolution picture. It consists of the picture prediction error CE _D .

The prediction errors _BEH,V,D are the prediction errors of three types of blurred pictures, consisting of the prediction error _BEH of the horizontal blurred picture, the prediction error _BEV of the vertical blurred picture and the prediction error _BED of the diagonal blurred picture. .

The frequency analysis unit 15 reads the local decoded picture K stored in the frame memory 13 as the past local decoded picture K, and inputs the current local decoded picture K. FIG.

The frequency analysis unit 15 performs frequency analysis on each of the current locally decoded picture K and the past locally decoded picture K. FIG. Then, the frequency analysis unit 15 subtracts the analysis result of the past locally decoded picture K from the analysis result of the current locally decoded picture K, and obtains power fluctuation information for each frequency band and for each phase position. The frequency analysis unit 15 outputs power fluctuation information for each frequency band and for each phase position to the selection unit 16 . Details of the frequency analysis unit 15 will be described later.

The selection unit 16 selects predicted images KY, CYH _,V,D , _BYH,V,D , motion vectors KV, _CVH,V,D , BVH _,V, Enter _D and prediction errors KE, CE _H,V,D and BE _H,V,D . The selection unit 16 also receives power fluctuation information for each frequency band and for each phase position from the frequency analysis unit 15 .

The selection unit 16 selects super-resolution picture motion vectors CVH _,V,D and super-resolution picture intensity information cpH _,V for three types of super-resolution pictures _CH,V,D for each predetermined region. _{, D} , the weights tC _H,V,D of the super-resolution picture are calculated.

The selection unit 16 may calculate the weights tCH _,V,D of the super-resolution picture based on the power fluctuation information and the intensity information cpH _{,V,D of the} super-resolution picture. Further, the selection unit 16 selects the weight tC _H of the super-resolution picture based on the motion vector CV _{H,V,D of} the super-resolution picture, the power fluctuation information, and the intensity information cp _H,V,D of the super-resolution picture. _{, V and D} may be calculated.

The weights tC _{H , V, and D} are the weights of the three types of super-resolved pictures: the weight tC _H of the horizontal super-resolved picture, the weight tC _V of the vertical super-resolved picture, and the weight tC of the diagonal super-resolved picture. consists of _D.

Further, the selection unit 16 calculates the weight tBH of the blurred picture based on the power fluctuation information and the intensity information bpH _,V,D of the blurred picture for each of the three types of blurred pictures BH _,V,D for each predetermined region _{. , V and D} are calculated.

The selection unit 16 calculates the weights tB _{H,V,D of the blurred picture based on the motion vector BV H,V,D of the} blurred picture, the power fluctuation information, and the intensity information bp _{H,V ,D of the} blurred picture. You may make it

The weights tB _{H , V, D} are the weights of three types of blurred pictures, consisting of a horizontal blurred picture weight tB _H , a vertical blurred picture weight tB _V and a diagonal blurred picture weight tB _D .

The selection unit 16 assigns weights tC _{H , V,} Multiply _D. Further, the selection unit 16 assigns weights tBH _{,V,D corresponding to prediction errors BEH,V,D} of blurred pictures to three types of blurred pictures BH _{,V, D} for each predetermined region. Multiply.

The selection unit 16 selects the prediction error KE of the locally decoded picture, the weight-multiplied prediction error CE _H,V,D of the super-resolution picture, and the weight-multiplied prediction error BE _H,V of the blurred picture for each predetermined region. _{, D} to determine the smallest prediction error. Then, the selection unit 16 selects a predicted image corresponding to the specified prediction error. As a result, one of the predicted images KY, CYH _,V,D , and BYH _,V,D of seven types of pictures is selected.

The selection unit 16 outputs the selected prediction image to the switch 119 as an inter-screen prediction result for each predetermined region. Details of the selection unit 16 will be described later.

The parameter processing unit 17 sets parameters such as the type of predicted image selected for each predetermined region by the selection unit 16 and the calculated motion vector, and outputs the parameters to the entropy encoding unit 120 .

FIG. 3 is a diagram illustrating the flow of processing from the inverse orthogonal transform unit 114 to the inter-frame prediction unit 10 via the addition unit 115 and the in-loop filter 117 .

The adding unit 115 adds the decoded residual image generated by the inverse orthogonal transform unit 114 and the inter prediction result generated by the inter prediction unit 10, that is, the predicted image, and the image after the addition is the in-loop filter 117. is entered in The locally decoded picture K filtered by the in-loop filter 117 is stored in the frame memory 13 and input to the super-resolution processor 11 and the blur processor 12 .

The local decoded picture K is subjected to super-resolution processing in the super-resolution processing unit 11 to generate three types of super-resolution pictures C _{H , V, and D} , which are stored in the frame memory 13 . The blur processor 12 applies blur processing to the locally decoded picture K to generate three types of blurred pictures B _{H , V, and D} , which are stored in the frame memory 13 .

As a result, the frame memory 13 stores seven types of pictures, that is, the locally decoded picture K, three types of super-resolution pictures CH _{, V, D} , and three types of blurred pictures B _{H, V, D.} These pictures are read by the inter prediction processing unit 14 as past local decoded pictures K, super-resolution pictures CH _{, V, D} , and blurred pictures B _{H, V, D.}

The inter-picture prediction processing unit 14 performs inter-picture prediction processing using the current locally decoded picture K as a reference picture and the past locally decoded picture K as a reference picture to generate a predicted picture KY and the like. Further, in the inter-prediction processing unit 14, three types of inter-prediction processing are performed using past three types of super-resolution pictures CH _{, V, D} as reference pictures, and three types of predicted images CY _{H, V, D} etc. are generated. Furthermore, in the inter-picture prediction processing unit 14, three types of inter-picture prediction processing are performed using past three types of blurred pictures _{BH, V, D} as reference pictures, and three types of predicted images BYH _{, V, D} , etc. is generated. Predicted images KY, CY _H,V,D , BY _H,V,D, etc. are input to the selection unit 16 .

The frequency analysis unit 15 generates power fluctuation information based on the current locally decoded picture K and the past locally decoded picture K, and the power fluctuation information is input to the selection unit 16 .

The selection unit 16 selects one of the predicted images KY, CYH _,V,D , and _BYH,V,D based on the power fluctuation information and the like for each predetermined region, and selects the predicted image between the screens. The prediction result is output to addition section 115 .

In this way, in the inter prediction processing unit 14, as reference pictures, a total of 7 types of locally decoded picture K, 3 types of super-resolution pictures CH _{, V, D} , and 3 types of blurred pictures B _{H, V, D} picture is used. Also, one of the seven types of predicted images KY, CYH _,V,D , and _BYH,V,D is selected as a predicted image for each predetermined region.

[Super-resolution processing unit 11]
Next, the super-resolution processing unit 11 shown in FIG. 2 will be described in detail. 4A and 4B are diagrams for explaining a processing example of the super-resolution processing unit 11. FIG.

The super-resolution processing unit 11 receives a locally decoded picture K and, for example, performs first-order wavelet decomposition on the locally decoded picture K (step S401). Then, the super-resolution processing unit 11 converts the element values (pixel values) of all the phase positions of the horizontal (H), vertical (V), and diagonal (D) high-frequency bands to the image after the first-order wavelet decomposition. In contrast, horizontal intensity information cp _{H1, V1, D1} , vertical intensity information cp _{H2, V2, D2} , and diagonal intensity information cp _{H3, V3, D3} of preset super-resolution pictures are multiplied (step S402).

The horizontal intensity information cp _{H1, V1, D1} is the horizontal intensity cp _H1 corresponding to the horizontal high frequency band, the vertical intensity cp _V1 corresponding to the vertical high frequency band, and the diagonal high frequency band in the image after the first-order wavelet decomposition. consists of the diagonal intensity cp _D1 . The horizontal intensity information cp _{H1 , V1, D1} is intensity information for generating a horizontal super-resolution picture C _H that emphasizes horizontal high-frequency band components (high power), and cp _H1 >cp _V1 , cp _D1. Intensity information that satisfies the condition of For example, as shown in FIG. 4, horizontal intensity cp _H1 =1.2, vertical intensity cp _V1 =1.0 and diagonal intensity cp _D1 =1.0 are used.

The vertical intensity information _cpH2,V2,D2 consists of horizontal intensity _cpH2 , vertical intensity _cpV2 and diagonal intensity _cpD2 . The vertical intensity information cp _{H2, V2, D2} is intensity information for generating a vertical super-resolution picture _CV in which vertical high-frequency band components are emphasized (strong power), and cp _V2 >cp _H2 , cp _D2 Intensity information that satisfies the condition of For example, as shown in FIG. 4, horizontal intensity cp _H2 =1.0, vertical intensity cp _V2 =1.2 and diagonal intensity cp _D2 =1.0 are used.

The diagonal strength information cp _{H3 , V3, D3} consists of a horizontal strength cp _H3 , a vertical strength cp _V3 and a diagonal strength cp _D3 . The diagonal intensity information cp _{H3 , V3, D3} is intensity information for generating a diagonal super-resolution picture C _D in which components in the diagonal high-frequency band are emphasized (strong power), and cp _D3 >cp _H3 , cp _V3 are used. For example, as shown in FIG. 4, horizontal intensity cp _H3 =1.0, vertical intensity cp _V3 =1.0 and diagonal intensity cp _D3 =1.2 are used.

The super-resolution processing unit 11 performs first-order wavelet reconstruction on each of the post-multiplication images after first-order wavelet decomposition, and produces three types of super-resolution pictures _{CH, V, D} , that is, horizontal super-resolution pictures _CH , A vertical super-resolution picture C _V and a diagonal super-resolution picture C _D are generated (step S403).

In this way, the super-resolution processing unit 11 generates three types of horizontal super-resolution picture C _H , vertical super-resolution picture C _V and diagonal super-resolution picture C _D from the locally decoded picture K. FIG.

[Blur processing unit 12]
Next, the blur processor 12 shown in FIG. 2 will be described in detail. FIG. 5 is a diagram for explaining an example of processing by the blur processing unit 12. As shown in FIG.

The blur processing unit 12 receives the locally decoded picture K and, for example, performs first-order wavelet decomposition on the locally decoded picture K (step S501). Then, the blur processing unit 12 calculates preset horizontal strength information bp _H1 of a blurred picture for element values of all phase positions in horizontal, vertical, and diagonal high-frequency bands for the image after first-order wavelet decomposition. _{, V1, D1} , vertical strength information bp _{H2, V2, D2} , and diagonal strength information bp _{H3, V3, D3} are multiplied (step S502).

The horizontal intensity information bp _{H1, V1, D1} is the horizontal intensity bp H1 corresponding to the _{horizontal high frequency band, the vertical intensity bp V1 corresponding} to the vertical high frequency band, and the diagonal high frequency band in the image after the first-order wavelet decomposition. consists of the diagonal intensity bp _D1 . The horizontal intensity information bp _{H1 , V1, D1} is intensity information for generating a horizontally blurred picture B _H in which horizontal high frequency band components are suppressed (weak power), and the condition of bp _H1 <bp _V1 , bp _D1 is satisfied. Intensity information that satisfies is used. For example, as shown in FIG. 5, horizontal intensity bp _H1 =0.8, vertical intensity bp _V1 =1.0 and diagonal intensity bp _D1 =1.0 are used.

The vertical intensity information bp _{H2 , V2, D2} consists of horizontal intensity bp _H2 , vertical intensity bp _V2 and diagonal intensity bp _D2 . The vertical intensity information bp _{H2 , V2, D2} is intensity information for generating a vertically blurred picture B _V in which vertical high frequency band components are suppressed (weak power), and the condition of bp _V2 <bp _H2 , bp _D2 is satisfied. Intensity information that satisfies is used. For example, as shown in FIG. 5, horizontal intensity bp _H2 =1.0, vertical intensity bp _V2 =0.8 and diagonal intensity bp _D2 =1.0 are used.

The diagonal intensity information bp _{H3 , V3, D3} consists of horizontal intensity bp _H3 , vertical intensity bp _V3 and diagonal intensity bp _D3 . The diagonal intensity information bp _{H3 , V3, D3} is intensity information for generating a diagonal blurred picture B _D in which components in the diagonal high-frequency band are suppressed (weak power), and bp _D3 <bp _H3 , bp Intensity information that satisfies the conditions of _V3 is used. For example, as shown in FIG. 5, horizontal intensity bp _H3 =1.0, vertical intensity bp _V3 =1.0 and diagonal intensity bp _D3 =0.8 are used.

The blur processing unit 12 performs first-order wavelet reconstruction on each of the post-multiplication images after first-order wavelet decomposition, and produces three types of blurred pictures _{BH, V, and D} , namely, horizontal blurred picture _BH , vertical blurred picture _BV and A diagonally blurred picture B _D is generated (step S503).

In this way, the blur processor 12 generates three types of horizontally blurred picture _BH , vertically blurred picture _BV , and diagonally blurred picture _BD from the locally decoded picture K. FIG.

[Inter-screen prediction processing unit 14]
Next, the inter-screen prediction processing unit 14 shown in FIG. 2 will be described in detail. FIG. 6 is a block diagram showing a configuration example of the inter-screen prediction processing unit 14. As shown in FIG. The inter-picture prediction processing unit 14 includes a decoded picture prediction unit 20 , a super-resolution prediction unit 21 and a blur prediction unit 22 . Note that FIG. 6 shows only components directly related to the present invention, and components not directly related are omitted.

The decoded picture prediction unit 20 reads the locally decoded picture K stored in the frame memory 13 as the past locally decoded picture K, and inputs the current locally decoded picture K. FIG.

The decoded picture prediction unit 20 uses the current local decoded picture K as a reference picture and the past local decoded picture K as a reference picture. A predicted image KY, a motion vector KV, and a prediction error KE are obtained.

The decoded picture prediction unit 20 outputs the predicted image KY, motion vector KV and prediction error KE to the selection unit 16 .

The super-resolution prediction unit 21 reads out the three types of super-resolution pictures C _{H, V, and D} stored in the frame memory 13 as past super-resolution pictures C _{H, V, and D} , and extracts the current local decoded picture. Enter K.

The super-resolution prediction unit 21 uses the current locally decoded picture K as a reference picture, and the past super-resolution pictures _{CH, V, and D} as reference pictures. For example, by the block matching method, they are obtained as a predicted image _CYH,V,D , a motion vector CVH _,V,D , and a prediction error _CEH,V,D .

The super-resolution prediction unit 21 outputs the predicted image CY _H,V,D , the motion vector CV _H,V,D and the prediction error CE _H,V,D to the selection unit 16 .

The blur predictor 22 reads out the three types of blurred pictures _{BH, V, D} stored in the frame memory 13 as past blurred pictures _{BH, V, D,} and inputs the current locally decoded picture K.

The blur prediction unit 22 uses the current locally decoded picture K as a reference picture and the past blurred pictures _{BH, V, and D} as reference pictures, and performs, for example, a block matching method for each predetermined region based on the reference picture and the reference picture. , the predicted image BYH _,V,D , the motion vector BVH _,V,D , and the prediction error _BEH,V,D are obtained.

The blur prediction unit 22 outputs the predicted image BY _H,V,D , the motion vector BV _H,V,D and the prediction error BE _H,V,D to the selection unit 16 .

In this way, the inter-picture prediction processing unit 14, in addition to the predicted image KY, motion vector KV and prediction error KE of the local decoded picture, predictive image _CYH,V,D of the super-resolution picture, motion vector CVH _{,V ,D} and the prediction error CE _H,V,D , as well as the prediction image BY _H,V,D of the blurred picture, the motion vector BV _H, V,D and the prediction error BE _H,V,D are generated.

[Frequency analysis unit 15]
Next, the frequency analysis section 15 shown in FIG. 2 will be described in detail. FIG. 7 is a diagram for explaining a processing example of the frequency analysis unit 15. As shown in FIG.

The frequency analysis unit 15 reads the local decoded picture K stored in the frame memory 13 as the past local decoded picture K, and inputs the current local decoded picture K. FIG.

The frequency analysis unit 15 performs frequency analysis on the current locally decoded picture K by first-order wavelet packet decomposition without decimation, and generates an analysis result of the current locally decoded picture K (step S701). Thereby, the spatial low frequency band signal, horizontal high frequency band signal, vertical high frequency band signal and diagonal high frequency band signal of the current locally decoded picture K are generated.

Further, the frequency analysis unit 15 performs frequency analysis on the past locally decoded picture K by first-order wavelet packet decomposition without decimation, and generates an analysis result of the past locally decoded picture K (step S702). Thereby, a spatial low frequency band signal, a horizontal high frequency band signal, a vertical high frequency band signal and a diagonal high frequency band signal of the past locally decoded picture K are generated.

The frequency analysis unit 15 subtracts the analysis result of the past locally decoded picture K from the analysis result of the current locally decoded picture K for each frequency band and for each phase position, and obtains power fluctuation information for each frequency band and for each phase position. (step S703).

Specifically, the frequency analysis unit 15 converts the element value (pixel value) of the phase position (pixel position) in the horizontal high-frequency band signal of the current locally decoded picture K to the horizontal high-frequency band signal of the previous locally decoded picture K. Subtract the element value of the corresponding phase position. Then, the frequency analysis unit 15 uses the subtraction result as power fluctuation information in the horizontal high frequency band.

Further, the frequency analysis unit 15 subtracts the element value of the corresponding phase position of the vertical high-frequency band signal of the past locally decoded picture K from the element value of the phase position of the vertical high-frequency band signal of the current locally decoded picture K. Then, the frequency analysis unit 15 uses the subtraction result as the power fluctuation information of the vertical high frequency band.

Furthermore, the frequency analysis unit 15 subtracts the element value of the corresponding phase position in the diagonal high-frequency band signal of the past local decoded picture K from the element value of the phase position in the diagonal high-frequency band signal of the current locally decoded picture K. do. Then, the frequency analysis unit 15 uses the subtraction result as the power fluctuation information of the diagonal high frequency band.

The frequency analysis unit 15 analyzes the power fluctuation information at each phase position in the horizontal high frequency band, the power fluctuation information at each phase position in the vertical high frequency band, and the power fluctuation information at each phase position in the diagonal high frequency band into frequency bands. and output to the selector 16 as power fluctuation information for each phase position.

Thus, the frequency analysis unit 15 generates power fluctuation information for each frequency band and for each phase position.

Power variation information is treated as an indicator of whether the current picture is blurry or sharpened compared to previous pictures. If the power fluctuation amount obtained from the power fluctuation information, which will be described later, is larger than a first threshold (positive value), the power of the current picture is higher than that of the past picture. , is judged to be sharpened. If the amount of power variation is smaller than the first threshold (negative value), the power of the current picture is smaller than the power of the past picture, so it is determined that the current picture is more blurred than the past picture. be done. Furthermore, if the power variation is between the first threshold and the second threshold, the power of the current picture and the power of the past picture are approximately the same, so the blurring in the current picture and the past picture is reduced. It is judged that the degree of sharpening and the degree of sharpening do not change so much. Details will be described later.

[Selection unit 16]
Next, the selection unit 16 shown in FIG. 2 will be described in detail. FIG. 8 is a block diagram showing a configuration example of the selection unit 16, and FIG. 9 is a flowchart showing a processing example of the selection unit 16. As shown in FIG.

The selector 16 includes a motion amount calculator 30 , a power variation calculator 31 , a weight calculator 32 and a predicted image selector 33 . Note that FIG. 8 shows only components directly related to the present invention, and components not directly related are omitted.

The selection unit 16 selects predicted images KY, CYH _,V,D , _BYH,V,D , motion vectors KV, _CVH,V,D , BVH _,V for each predetermined region from the inter-screen prediction processing unit 14. _{, D} and prediction errors KE, CE _H,V,D and BE _H,V,D . The selection unit 16 also receives power fluctuation information for each frequency band and for each phase position from the frequency analysis unit 15 (step S901).

(Motion amount calculator 30)
The motion amount calculation unit 30 calculates, for each predetermined region, motion amounts CV _{H,V,D −m} Calculate _{H, V, D.} Further, the motion amount calculation unit 30 calculates motion amounts BV _{H, V,} of the blurred picture, which are components in the horizontal direction, the vertical direction, and the diagonal direction, from the motion vectors BV _{H, V, D of the blurred picture for each predetermined region. D} -m _H,V,D are calculated (step S902). Then, the motion amount calculation unit 30 outputs the motion amounts CV _H,V,D −m _H,V,D and BV _H,V,D −m _H,V,D for each predetermined region to the weight calculation unit 32. .

The amount of motion CVH _,V,D - _mH,V,D of the super-resolution picture is the amount _of horizontal motion CVH-mH, the amount of vertical motion _{CVH - mV} and the amount of diagonal motion of the horizontal super-resolution picture. _CVH - _mD , the horizontal motion amount CVV- _mH of the vertical super-resolution picture, _the vertical motion amount CVV- _mV and _the diagonal motion amount _CVV - _mD , and the horizontal motion amount of the diagonal super-resolution picture It consists of the amount of motion CV _D -m _H , the amount of vertical motion CV _D -m _V and the amount of diagonal motion CV _D -m _D .

The motion amount _BVH,V,D - _mH,V,D of the blurred _picture is the horizontal motion amount BVH-mH, the vertical motion amount _BVH - _mV , and the diagonal motion amount _BVH - _m of the horizontal blurred picture. _D , the horizontal motion amount BV _V -m _H of the vertically blurred picture, the vertical motion amount BV _V -m _V and the diagonal motion amount BV _V -m _D , the horizontal motion amount BV _D -m _H of the diagonal blurred picture, It consists of a vertical motion amount BV _D -m _V and a diagonal motion amount BV _D -m _D .

The horizontal motion amount indicates the magnitude when the motion vector is projected horizontally, the vertical motion amount indicates the magnitude when the motion vector is projected vertically, and the diagonal motion amount indicates the magnitude when the motion vector is projected in the vertical direction. Indicates the size when projected in the angular direction.

In this way, the motion amount calculation unit 30 calculates the horizontal motion amount CV _H -m _H , the vertical motion amount CV _H -m _V , and the diagonal motion amount CV _H - for the horizontal super-resolution picture C _H for each predetermined region. m _D is calculated. Further, for the vertical super-resolution picture C _V , the horizontal motion amount CV _V -m _H , the vertical motion amount CV _V -m _V , and the diagonal motion amount CV _V -m _D are calculated, and the diagonal super-resolution picture C _D is calculated. , a horizontal motion amount CV _D -m _H , a vertical motion amount CV _D -m _V , and a diagonal motion amount CV _D -m _D are calculated.

Further, the motion amount calculator 30 calculates the horizontal motion amount BVH _{- mH} , the vertical motion amount _BVH - _mV , and the diagonal motion amount _{BVH - mD} for the horizontal blurred picture BH for each predetermined region. be done. Also, for the vertically blurred picture B _V , the horizontal motion amount BV _V -m _H , the _vertical motion amount BV _V -m _V , and the diagonal motion amount BV _V -m _D are calculated. A quantity BV _D -m _H , a vertical motion quantity BV _D -m _V and a diagonal motion quantity BV _D -m _D are calculated.

FIG. 10 is a diagram illustrating an example of the motion amount _CVH - _mH,V,D of the horizontal super-resolution picture calculated by the motion amount calculation unit 30. In FIG. The picture at the time of the frame t is taken as the reference picture, and the picture at the time of the frame t-1 is taken as the reference picture. α1 indicates the horizontal direction of the motion vector CV _H of the horizontal super-resolution picture, β1 indicates the vertical direction of the motion vector CV _H of the horizontal super-resolution picture, and γ1 indicates the motion vector CV of the horizontal super-resolution picture. The diagonal direction in _H is shown.

As shown in FIG. 10, the motion amount calculation unit 30 projects the motion vector CV _H of the horizontal super-resolution picture with respect to the reference picture based on the base picture for each predetermined region in the horizontal direction α1. A horizontal motion amount _CVH - _mH , a vertical motion amount _CVH - _mV when projected in the vertical direction β1, and a diagonal motion amount _CVH - _mD when projected in the diagonal direction γ1 are calculated.

(Power fluctuation amount calculator 31)
8 and 9, the power fluctuation amount calculation unit 31 calculates, for each predetermined region, the components of the horizontal high frequency band, the vertical high frequency band, and the diagonal high frequency band from the power fluctuation information for each frequency band and for each phase position. A power fluctuation amount p _H,V,D is calculated (step S903). Then, the power fluctuation amount calculator 31 outputs the power fluctuation amounts p _{H , V, D} for each predetermined region to the weight calculator 32 .

Specifically, the power variation calculation unit 31 extracts the power variation information for each predetermined region from the power variation information for each frequency band and each phase position. Then, the power fluctuation amount calculator 31 totals the element values at all phase positions in the horizontal high-frequency band included in the power fluctuation information of the area, and obtains the horizontal power fluctuation amount p _H of the area.

Also, the power fluctuation amount calculator 31 sums the element values at all phase positions in the vertical high-frequency band included in the power fluctuation information of the area, and obtains the vertical power fluctuation amount p _V of the area. Further, the power fluctuation amount calculator 31 sums up the element values of all the phase positions of the diagonal high frequency band included in the power fluctuation information of the area, and obtains the diagonal power fluctuation amount p _D of the area.

In this way, the power fluctuation amount calculator 31 calculates, for each predetermined region, the horizontal power fluctuation amount p _H that is the component of the horizontal high frequency band, the vertical power fluctuation amount p _V that is the component of the vertical high frequency band, and the diagonal high frequency band. A diagonal power fluctuation amount p _D that is a component of is calculated.

FIG. 11 is a diagram for explaining an example of the power fluctuation amounts p _{H , V, D} calculated by the power fluctuation amount calculator 31 . α2 indicates the direction of the horizontal high frequency band in the power fluctuation information, β2 indicates the direction of the vertical high frequency band in the power fluctuation information, and γ2 indicates the direction of the diagonal high frequency band in the power fluctuation information.

As shown in FIG. 11, the power fluctuation amount calculation unit 31 calculates the horizontal power fluctuation amounts p _H , A vertical power fluctuation amount p _V that is a component in the direction β2 of the vertical high frequency band and a diagonal power fluctuation amount p _D that is a component in the direction γ2 of the diagonal high frequency band are calculated.

(Weight calculator 32)
The weight calculator 32 inputs the motion amounts CV _H,V,D −m _H,V,D and BV _H,V,D −m _H,V,D for each predetermined region from the motion amount calculator 30, and The power fluctuation amounts p _{H , V, and D} for each predetermined region are input from the power fluctuation amount calculator 31 . The weight calculation unit 32 also receives intensity information cp _{H, V, D} of the super-resolution picture and intensity information bp _{H, V, D} of the blurred picture set in advance.

The super-resolution picture intensity information cp _{H, V, D} is the horizontal intensity information cp _H1 used when the super-resolution processing unit 11 shown in FIG. 2 generates the super-resolution pictures C _{H, V, D. , V1, D1} , vertical intensity information cp _{H2, V2, D2} , and diagonal intensity information cp _{H3, V3, D3} . Further, the intensity information bp _{H, V, D} of the blurred picture is the horizontal intensity information bp _{H1, V1, D1} used when the blurred picture B _{H, V, D} is generated by the blur processing unit 12 shown in FIG. , vertical intensity information bp _{H2 , V2, D2} and diagonal intensity information bp _{H3, V3, D3} .

The weight calculation unit 32 calculates the amount of motion CV _{H, V,D} -m _H,V,D and Based on the intensity information _cpH,V,D ( _cpH1,V1,D1 , _cpH2,V2,D2 , _{cpH3,V3,D3 )} of the super-resolution pictures, Weights tC _H,V,D are calculated for each (step S904).
[Number 1]
_tCH =d/{(a* _cpH1 *( _CVH - _mH )+b* _cpV1 *(CVH _{- mV} )+c* _cpD1 *(CVH _{- mD} )} (1)
[Number 2]
_tCV =d/{(a* _cpH2 *( _CVV - _mH )+b* _cpV2 *(CVV _{- mV} )+c* _cpD2 *(CVV _{- mD} )} (2)
[Number 3]
tC _D =d/{(a×cp _H3 ×(CV _D −m _H )+b×cp _V3 ×(CV _D −m _V )+c×cp _D3 ×(CV _D −m _D )} (3)

The weight tC _H indicates the weight of the horizontal super-resolved picture C _H , the weight tC _V indicates the weight of the vertical super-resolved picture C _V , and the weight tC _D indicates the weight of the diagonal super-resolved picture C _D . show. Parameters a, b, c, and d represent preset constants.

In the above equation (1), for the horizontal super-resolution picture C _H , for example, horizontal intensity cp _H1 =1.2, vertical intensity cp _V1 =1.0, and diagonal intensity cp _D1 =1.0 are used as described above.

In the above equation (2), for the vertical super-resolution picture C _V , for example, horizontal intensity cp _H2 =1.0, vertical intensity cp _V2 =1.2, and diagonal intensity cp _D2 =1.0 are used as described above.

In the above equation (3), for the diagonal super-resolution picture C _D , for example, horizontal intensity cp _H3 =1.0, vertical intensity cp _V3 =1.0 and diagonal intensity cp _D3 =1.2 are used as described above.

Thus, the weights tC _H,V,D shown in the formulas (1), (2), and (3) are calculated based on the motion amount CV _H,V,D −m _H,V,D of the super-resolution picture. The larger the motion amount _CVH,V,D - _mH,V,D, the smaller the value, and the smaller the motion amount _CVH,V,D - _mH,V,D , the larger the value. In this case, the larger the amount of motion CV _H,V,D -m _H,V,D, the higher the possibility that the reference picture is more blurred than the reference picture, so the reference picture is sharpened with respect to the reference picture. Probability is high.

Therefore, the weight tC _{H,V, D} is set to a smaller value in the direction (horizontal direction, vertical direction and diagonal direction) in which the motion amount CV H,V,D _{-m H,V,D} is larger. In step S907, the prediction error CEH _,V,D after weight multiplication is set to a small value, and the predicted image CYH _,V,D of the super-resolution picture corresponding to that direction is selected. That is, assuming that the blur is greater in the direction in which the amount of motion CV _H,V,D −m _H,V,D from the reference picture to the reference picture is larger, the predicted image of the super-resolution picture in the direction of strong sharpening CY _{H, V, D} are selected.

In step S904, the weight calculation unit 32 calculates the power fluctuation amount instead of the super-resolution picture motion amount CV _{H ,V,D} −m _H,V,D for the super-resolution picture C H,V,D. _{pH, V, and D} may be used. That is, the weight calculator 32 calculates the power fluctuation amounts p _{H, V, D} and the super-resolution picture intensity information cp _{H, V, D} (cp H1, V1, D1 , cp _{H1, V1, D1} , cp _H2,V2,D2 , cp _H3,V3,D3 ), the weights tC _{H,V,D for each of the super-resolution pictures C H,V, D} may be calculated.
[Number 4]
_tCH =h/(e* _cpH1 * _pH +f* _cpV1 * _pV +g* _cpD1 * _pD ) (4)
[Number 5]
_tCV =h/(e* _cpH2 * _pH +f* _cpV2 * _pV +g* _cpD2 * _pD ) (5)
[Number 6]
tC _D =h/(e×cp _H3 ×p _H +f×cp _V3 ×p _V +g×cp _D3 ×p _D ) (6)

Parameters e, f, g, and h represent preset constants.

In the above equation (4), as in the above equation (1), for example, horizontal intensity cp _H1 =1.2, vertical intensity cp _V1 =1.0 and diagonal intensity cp _D1 =1.0 for the horizontal super-resolution picture C _H are used. .

In the above equation (5), similar to the above equation (2), for example, horizontal intensity cp _H2 =1.0, vertical intensity cp _V2 =1.2 and diagonal intensity cp _D2 =1.0 for the vertical super-resolution picture C _V are used. .

In the above equation (6), as in the above equation (3), horizontal intensity cp _H3 =1.0, vertical intensity cp _V3 =1.0 and diagonal intensity cp _D3 =1.2 for the diagonal super-resolution picture C _D are used. .

Thus, the weights tC _H,V,D shown in the above equations (4), (5), and (6) are calculated based on the power fluctuations _pH,V,D , and the power fluctuations _pH,V,D is larger in the positive direction, the value becomes smaller _. In this case, the larger the power fluctuation amounts p _{H , V, and D} in the positive direction, the higher the possibility that the reference picture is more blurred than the reference picture. expensive.

Therefore, by setting the weights tC _H,V,D to smaller values in the positive direction (horizontal direction, vertical direction, and diagonal direction) in which the power fluctuation amounts p _H,V,D are larger, in step S907 described later, , the prediction error CE _H,V,D after weight multiplication is set to a small value, and the prediction image CYH _,V,D of the super-resolution picture corresponding to that direction is selected. That is, assuming that blurring is greater in the direction in which the power fluctuation amounts p _{H , V, and D} from the reference picture to the reference picture are larger in the positive direction, the predicted image CY _{H, V and D} are selected.

In step S904, the weight calculation unit 32 calculates the motion amount of the super-resolution picture instead of the motion amount CVH _{, V,D} -mH _,V,D for the super-resolution picture CH,V,D. CV _H,V,D −m _H,V,D and power variation p _H,V,D may be used. That is, the weight calculator 32 calculates the motion amount CV _H,V,D −m _{H,V,D of the super-resolution picture, the power fluctuation amount p H,V,D} , and the power fluctuation amount p _H,V,D and Based on the intensity information _cpH,V,D ( _cpH1,V1,D1 , _cpH2,V2,D2 , _{cpH3,V3,D3 )} of the super-resolution pictures, Weights tC _H,V,D may be calculated for each.
[Number 7]
tC _H =o/[{(i × cp _H1 × (CV _H -m _H ) + j × cp _V1 × (CV _H -m _V ) + k × cp _D1 × (CV _H -m _D )} + (l × cp _H1 × _ρH + m × cp _V1 × ρ _V + n × cp _D1 × ρ _D )]
... (7)
[Number 8]
_tCV = o/[{(i x cp _H2 x (CV _V - m _H ) + j x cp _V2 x (CV _V - m _V ) + k x cp _D2 x (CV _V - m _D )} + (l x cp _H2 × ρ _H + m × cp _V2 × ρ _V + n × cp _D2 × ρ _D )]
... (8)
[Number 9]
tC _D =o/[{(i×cp _H3 ×(CV _D -m _H )+j×cp _V3 ×(CV _D -m _V )+k×cp _D3 ×(CV _D -m _D )}+(l×cp _H3 × ρ _H + m × cp _V3 × ρ _V + n × cp _D3 × ρ _D )]
... (9)

Parameters i, j, k, l, m, n, and o indicate preset constants.

In equation (7), similar to equation (1), horizontal intensity cp _H1 =1.2, vertical intensity cp _V1 =1.0 and diagonal intensity cp _D1 =1.0 for horizontal super-resolution picture C _H are used, for example. .

Further, in the above equation (8), as in the above equation (2), for example, the horizontal intensity cp _H2 =1.0, the vertical intensity cp _V2 =1.2, and the diagonal intensity cp _D2 =1.0 for the vertical super-resolution picture C _V are Used.

Further, in the above equation (9), as in the above equation (3), the horizontal intensity cp _H3 =1.0, the vertical intensity cp _V3 =1.0, and the diagonal intensity cp _D3 =1.2 for the diagonal super-resolution picture C _D are Used.

The weight calculation unit 32 shifts from step S904 to calculate the power variation amounts p _{H, V, D and} the intensity information bp Based on _H,V,D (bpH1 _,V1,D1 , bpH2 _,V2,D2 , _bpH3,V3,D3 ), weights tBH _{,V, D for each of the blurred pictures BH,V,} D are Calculate (step S905).
[Number 10]
_tBH =s/(p* _bpH1 * _pH +q* _bpV1 * _pV +r* _bpD1 * _pD ) (10)
[Number 11]
_tBV =s/(p* _bpH2 * _pH +q* _bpV2 * _pV +r* _bpD2 * _pD ) (11)
[number 12]
tB _D =s/(p×bp _H3 ×p _H +q×bp _V3 ×p _V +r×bp _D3 ×p _D ) (12)

Weight tB _H indicates the weight of horizontal blurred picture B _H , weight tB _V indicates the weight of vertical blurred picture B _V , and weight tB _D indicates the weight of horizontal blurred picture _BD . Parameters p, q, r, and s indicate preset constants.

In equation (10) above, for the horizontally blurred picture B _H , for example, horizontal intensity bp _H1 =0.8, vertical intensity bp _V1 =1.0, and diagonal intensity bp _D1 =1.0 are used as described above.

In equation (11) above, for the vertically blurred picture B _V , for example, horizontal intensity bp _H2 =1.0, vertical intensity bp _V2 =0.8, and diagonal intensity bp _D2 =1.0 are used as described above.

In equation (12) above, for the diagonally blurred picture B _D , for example, horizontal intensity bp _H3 =1.0, vertical intensity bp _V3 =1.0, and diagonal intensity bp _D3 =0.8 are used as described above.

In this way, the weights tB _H,V,D shown in the formulas (10), (11), and (12) are calculated based on the power fluctuations _pH,V,D , and the power fluctuations _pH,V,D The larger the negative direction, the smaller the absolute value, and the smaller the power fluctuation amount p _{H, V, D} in the negative direction, the larger the absolute value. In this case, the larger the power fluctuation amounts p _{H , V, and D} in the negative direction, the higher the possibility that the reference picture is sharper than the reference picture. expensive.

Therefore, by setting the absolute values of the weights tB _{H, V, D} to smaller values in the direction (horizontal direction, vertical direction, and diagonal direction) in which the power fluctuation amounts p _{H, V, D} have larger values in the negative direction, In step S907, which will be described later, the prediction error BEH _,V,D after weight multiplication is set to a small value, and the predicted image _BYH,V,D of the blurred picture corresponding to that direction is selected. That is, assuming that sharpening is greater in the direction in which the power fluctuation amounts p _{H , V, and D} from the reference picture to the reference picture have larger negative values, the predicted image BY _{H, V and D} are selected.

For the blurred pictures _{BH,V,D, the weight calculator 32 calculates the amount of motion of the blurred picture BVH,V} ,D- _mH,V instead of the power fluctuation amount pH _{,V, D} in step S905. _{, D} and the power variations p _H,V,D may be used. That is, the weight calculator 32 calculates the motion amount BV _H,V,D −m _H,V,D of the blurred picture, the power fluctuation amount p _H,V,D and the blurred picture _{Weights tBH , _ _ _ V and D} may be calculated.
[Number 13]
tB _H =z/[{(t×bp _H1 ×(BV _H -m _H )+u×bp _V1 ×(BV _H -m _V )+v×bp _D1 ×(BV _H -m _D )}+(w×bp _H1 ×p _H +x×bp _V1 ×p _V +y×bp _D1 ×p _D )]
(13)
[Number 14]
tB _V =z/[{(t x bp _H2 x (BV _V - m _H ) + u x bp _V2 x (BV _V - m _V ) + v x bp _D2 x (BV _V - m _D )} + (w x bp _H2 ×p _H +x×bp _V2 ×p _V +y×bp _D2 ×p _D )]
(14)
[Number 15]
tB _D =z/[{(t×bp _H3 ×(BV _D -m _H )+u×bp _V3 ×(BV _D -m _V )+v×bp _D3 ×(BV _D -m _D )}+(w×bp _H3 ×p _H +x×bp _V3 ×p _V +y×bp _D3 ×p _D )]
(15)

Parameters t, u, v, w, x, y, and z represent preset constants.

In equation (13), as in equation (10) above, for example, horizontal intensity bp _H1 =0.8, vertical intensity bp _V1 =1.0 and diagonal intensity bp _D1 =1.0 for horizontally blurred picture B _H are used.

Also, in the above equation (14), as in the above equation (11), for example, horizontal intensity bp _H2 =1.0, vertical intensity bp _V2 =0.8, and diagonal intensity bp _D2 =1.0 for the vertically blurred picture B _V are used. .

Also, in the equation (15), as in the equation (12), the horizontal intensity bp _H3 =1.0, the vertical intensity bp _V3 =1.0 and the diagonal intensity bp _D3 =0.8 for the diagonal blurred picture B _D are used. .

The weight calculation unit 32 uses the weight tC _H,V,D of the super-resolution picture for each predetermined region calculated in step S904 and the weight tBH _,V,D of the blurred picture calculated in step S905 to select a predicted image. Output to unit 33 .

(Predicted image selection unit 33)
The predicted image selection unit 33 receives from the weight calculation unit 32 the super-resolution picture weights tCH _,V,D and the blurred picture weights tBH _,V,D for each predetermined region. In addition, the predicted image selection unit 33 selects predicted images KY, CYH _,V,D , _BYH,V,D and prediction errors KE, _CEH,V,D , and Enter BE _H,V,D .

The prediction image selection unit 33 shifts from step S905, and for each predetermined region, for each super-resolution picture C _{H , V, D} , the prediction error CE _{H, V, D} of the super-resolution picture is assigned a corresponding weight tC Multiply _H,V,D . Further, the predicted image selection unit 33 multiplies the prediction error BE _H,V,D of the blurred picture by the corresponding weight tB _H,V,D for each of the blurred pictures _BH,V,D for each predetermined region ( step S906).

As a result, the prediction error CE _H,V,D after weight multiplication of the super-resolution picture and the prediction error BE _H,V,D after weight multiplication of the blurred picture are obtained.

The prediction image selection unit 33 selects, for each predetermined region, the prediction error KE of the locally decoded picture, the weight-multiplied prediction error CE _H,V,D of the super-resolution picture, and the weight-multiplied prediction error BE _H of the blurred picture. _{, V, D} to determine the minimum prediction error. Then, the predicted image selection unit 33 selects the predicted image corresponding to the minimum prediction error (step S907). As a result, one of the predicted images KY, CYH _,V,D , and BYH _,V,D of seven types of pictures is selected.

The predicted image selection unit 33 outputs the selected predicted image as an inter-screen prediction result to the switch 119 for each predetermined region (step S908).

For example, for a given region, the prediction error CE _H after weight multiplication of the horizontal super-resolved picture is the prediction error KE of the locally decoded picture, the prediction error CE _H,V,D after weight multiplication of the super-resolution picture, and the blur Suppose that it is the minimum value among the prediction errors BEH _{, V, D} after multiplication of picture weights. In this case, the predicted image selection unit 33 selects the predicted image KY of the local decoded picture, the predicted image _CYH,V,D of the super-resolution picture, and the predicted image _BYH,V,D of the blurred picture for the region. , select the prediction image CY _H of the horizontal super-resolution picture. Then, the predicted image selection unit 33 outputs the predicted image CY _H of the horizontal super-resolution picture for the region to the switch 119 as the inter-frame prediction result.

As described above, according to the video encoding device 1 of the embodiment of the present invention, the super-resolution processing unit 11 of the inter-picture prediction unit 10 performs super-resolution processing on the locally decoded picture K, Super-resolution pictures C _{H , V, D} are generated and stored in the frame memory 13 . The blur processor 12 applies blur processing to the locally decoded picture K to generate three types of blurred pictures B _{H , V, and D} , and stores them in the frame memory 13 .

The inter-picture prediction processing unit 14 uses a reference picture that is the current locally decoded picture K, past locally decoded pictures K read from the frame memory 13, super-resolution pictures C _{H, V, D} , and blurred pictures B _{H, V. , D} , motion vectors KV, CVH _,V,D , BVH _,V, D and predicted images KY, _{CYH,V,D are} obtained by performing inter-frame prediction for each predetermined region. , BYH _,V,D and prediction errors KE, _CEH,V,D , BEH _,V,D .

The frequency analysis unit 15 performs frequency analysis on the current locally decoded picture K and the past locally decoded picture K, subtracts the analysis result of the past locally decoded picture K from the analysis result of the current locally decoded picture K, and determines the frequency band. Power fluctuation information for each phase position is obtained.

_The selection unit 16 selects weights tC _{H ,} Calculate _{V and D.} The selection unit 16 also calculates the weights tB _H,V,D of the blurred picture based on the power fluctuation information and the intensity information bp _{H,V,D of the} blurred picture for each predetermined region.

For each predetermined region, the selection unit 16 multiplies the prediction errors CE _H,V,D of the super-resolution picture by the corresponding weights tC _H,V,D to obtain the prediction errors BE _H,V,D of the blurred picture. , by the corresponding weights tB _H,V,D . The selection unit 16 selects the prediction error KE of the locally decoded picture, the weight-multiplied prediction error CE _H,V,D of the super-resolution picture, and the weight-multiplied prediction error BE _H,V of the blurred picture for each predetermined region. _{, D} , select the predicted image corresponding to the smallest prediction error.

In this way, in addition to the local decoded picture K, three types of super-resolution pictures C _{H, V, D} and three types of blurred pictures B _{H, V, D} are generated as reference pictures used for inter prediction processing. . Then, for each predetermined region, the amount of power fluctuation, the amount of motion, etc. of the picture are taken into consideration, and the optimum picture is selected as the prediction image in consideration of blurring and sharpening, and their directionality.

As a result, when the object is blurred from the past picture to the current picture, the blurred picture B is selected as the prediction image, and when the object is sharpened, the super-resolution picture C is selected. Therefore, it is possible to reduce the residual error between the input moving image and the predicted image.

In the prior art, by performing inter-picture prediction using only the locally decoded picture K as a reference picture, it was possible to obtain an objectively high-quality video. However, with the conventional technology, it is not possible to perform inter-frame prediction that distinguishes between a moving image that is blurred or sharpened and a normal moving image (that is, a moving image that is not blurred or sharpened). In addition, it is not possible to perform inter-frame prediction by distinguishing between a moving image that blurs or sharpens due to horizontal movement and a moving image that blurs or sharpens due to vertical movement.

In the embodiment of the present invention, blurring and sharpening, and inter-frame prediction in consideration of their directionality are performed. Therefore, it is possible to improve the efficiency of inter-picture prediction and obtain subjectively high-quality moving images even when objects are blurred or sharpened in each area between pictures of moving images. becomes.

[Video decoding device]
Next, a video decoding device according to an embodiment of the present invention will be described. FIG. 12 is a block diagram showing a configuration example of the video decoding device according to the embodiment of the present invention. The video decoding device 2 includes an entropy decoding unit 50, an inverse quantization unit 51, an inverse orthogonal transform unit 52, an addition unit 53, an intra-frame prediction unit 54, an in-loop filter 55, an inter-frame prediction unit 56, and a switch 57. ing. FIG. 12 shows only components directly related to the present invention, and components not directly related are omitted.

The entropy decoding unit 50 receives the encoded signal output from the video encoding device 1 shown in FIG. 1 and performs the reverse processing of the entropy encoding unit 120 shown in FIG. is subjected to entropy decoding to generate a quantization index sequence and parameters.

The entropy decoding unit 50 outputs the quantization index sequence to the inverse quantization unit 51, and outputs parameters including the type of predicted image selected for each predetermined region and the calculated motion vector to the inter-screen prediction unit 56. , and other parameters to the inverse quantization unit 51 and the like.

The inverse quantization unit 51 receives the quantization index sequence from the entropy decoding unit 50 and performs the reverse processing of the quantization unit 112 shown in FIG. to generate The inverse quantization unit 51 then outputs the transform coefficient sequence to the inverse orthogonal transform unit 52 .

The inverse orthogonal transform unit 52 receives the transform coefficient sequence from the inverse quantization unit 51, performs the inverse process of the orthogonal transform unit 111 shown in FIG. Generate an image. The inverse orthogonal transform unit 52 then outputs the decoded residual image to the addition unit 53 .

The adder 53 receives the decoded residual image from the inverse orthogonal transform unit 52 and the predicted image from the switch 57 . Then, the addition unit 53 adds the decoded residual image to the predicted image, and outputs the image after the addition to the intra-frame prediction unit 54 and the in-loop filter 55 .

The intra-screen prediction unit 54 receives the image after addition from the addition unit 53 and performs the same processing as the intra-screen prediction unit 116 shown in FIG. Prediction processing is performed, and an intra-screen prediction result for each predetermined area is output to the switch 57 . Further, the intra-screen prediction unit 54 outputs the image after addition as a decoded image. As a result, the original moving image is restored.

The in-loop filter 55 receives the image after addition from the adding unit 53, performs the same processing as the in-loop filter 117 shown in FIG. to generate The in-loop filter 55 then outputs the decoded picture K′ to the inter-picture prediction unit 56 .

The inter-screen prediction unit 56 receives the decoded picture K′ from the in-loop filter 55, and also receives parameters including the type of predicted image selected for each predetermined region and the calculated motion vector from the entropy decoding unit 50. . Then, the inter-picture prediction unit 56 performs a predetermined inter-picture prediction process based on a parameter for each predetermined region to obtain a prediction image KY′ of the local decoded picture, a prediction image CY′ of the three types of super-resolution pictures, _{and , V, D} ' (horizontal super-resolution picture predicted image CY _H ', vertical super-resolution picture predicted image CY _V ' and diagonal super-resolution picture predicted image CY _D '), and three types of blurred pictures any one of the predicted images BY _H,V,D ' (horizontal blurred picture predicted image BY _H ', vertical blurred picture predicted image BY _V ', and diagonal blurred picture predicted image BY _D ') An image is generated and output to the switch 57 as an inter-screen prediction result for each predetermined area. Also, the inter prediction unit 56 outputs the decoded picture K' as a decoded image. As a result, the original moving image is restored. Details of the inter-screen prediction unit 56 will be described later.

The switch 57 receives the intra-prediction result for each predetermined region from the intra-prediction unit 54 and the inter-prediction result for each predetermined region from the inter-prediction unit 56, selects one of them, and selects the predicted image. , and output to the addition unit 53 .

[Inter-screen prediction unit 56]
Next, the inter-screen prediction unit 56 shown in FIG. 12 will be described in detail. FIG. 13 is a block diagram showing a configuration example of the inter-frame prediction section 56 provided in the video decoding device 2. As shown in FIG. The inter-screen prediction unit 56 includes a super-resolution processing unit 60 , a blur processing unit 61 , a frame memory 62 , an inter-screen prediction processing unit 63 and a parameter processing unit 64 . Note that FIG. 13 shows only components directly related to the present invention, and components not directly related are omitted.

The inter prediction unit 56 receives the decoded picture K′ from the in-loop filter 55 and stores the decoded picture K′ in the frame memory 62 . Further, the parameter processing unit 64 of the inter-screen prediction unit 56 receives parameters including the type of predicted image selected for each predetermined region and the calculated motion vector from the entropy decoding unit 50, and applies the parameters to inter-screen prediction processing. Output to unit 63 . The type of prediction image is one of local decoded picture K, three types of super-resolution pictures C _{H, V, D} , and three types of blurred pictures B _{H, V, D} (out of seven types of pictures, ) is shown.

The super-resolution processing unit 60 performs super-resolution processing similar to that of the super-resolution processing unit 11 shown in _FIG . Generate (horizontal super-resolved picture C _H ', vertical super-resolved picture C _V ' and diagonal super-resolved picture C _D '). The super-resolution processing unit 60 stores three types of super-resolution pictures _CH,V,D ' in the frame memory 62. FIG.

_{The blur} processing unit 61 performs blur processing similar to that of the blur processing unit 12 shown in FIG. B _V ' and diagonally blurred picture B _D '). The blur processor 61 stores three types of blurred pictures B _{H , V, D} ′ in the frame memory 62 .

The frame memory 62 stores a decoded picture K', three kinds of super-resolution pictures _CH,V,D ', and three kinds of blurred pictures BH _,V,D ' for each frame.

The inter prediction processing unit 63 receives a parameter from the parameter processing unit 64, and when the type of predicted image included in the parameter indicates the locally decoded picture K, the decoded picture K′ stored in the frame memory 62 is pasted. is read out as a decoded picture K'.

Further, the inter prediction processing unit 63 indicates that the type of predicted image included in the parameter is one of the horizontal super-resolution picture C _H , the vertical super-resolution picture C _V , and the diagonal super-resolution picture C _D . , the corresponding super-resolved picture C′ stored in the frame memory 62 (horizontal super-resolved picture C _H ', vertical super-resolved picture C _V ', and diagonal super-resolved picture corresponding to the type of predicted image C _D ') is read as a past super-resolution picture C'.

Further, when the type of predicted image included in the parameter indicates any one of horizontal blurred picture _BH , vertical blurred picture _BV , and diagonally blurred picture _BD , inter-picture prediction processing unit 63 stores _{the stored corresponding blurred picture B' (any one of horizontal blurred picture BH', vertical blurred picture BV', and diagonal blurred picture BD ' corresponding} to the predicted image type), It is read as a past blurred picture B'.

The inter-screen prediction processing unit 63 performs inter-screen prediction based on the past decoded picture K', the super-resolution picture C' or the blurred picture B', and the motion vector included in the parameter, and performs prediction for each predetermined region. Ask for images. The predetermined area is, for example, a slice area, a CTU (Coding Tree Unit) area, or a CU (Coding Unit) area, as in FIG.

The inter-screen prediction processing unit 63 outputs the predicted image for each predetermined area to the switch 57 as an inter-screen prediction result.

A reading unit (not shown) reads the decoded picture K' from the frame memory 62 and outputs it as the decoded picture O. FIG.

As described above, according to the video decoding device 2 of the embodiment of the present invention, the super-resolution processing unit 60 of the inter-picture prediction unit 56 generates three types of super-resolution pictures CH based on the decoded picture K _{′. , V, D} ′ are generated and stored in the frame memory 62 . The blur processor 61 generates three types of blurred pictures B _{H , V, D} ′ based on the decoded picture K′ and stores them in the frame memory 62 .

The inter-screen prediction processing unit 63 extracts the decoded picture K′ and super-resolution picture C′ (horizontal super-resolution picture C _H ', vertical super-resolution picture C _{V )} from the frame memory 62 according to the type of predicted image included in the parameter. ' and diagonal super-resolution picture C _D ') or blurred picture B' (one of horizontal blurred picture B _H ', vertical blurred picture B _V ' and diagonal blurred picture B _D ') Any one picture) is read, and a predicted image is obtained for each predetermined region based on the read picture and the motion vector included in the parameter.

As described above, in the moving picture encoding device 1, as reference pictures used for inter-picture prediction processing, the locally decoded picture K, the three types of super-resolution pictures CH _{, V, D} , and the three types of blurred pictures B _{H, V , D} are generated, and the amount of power fluctuation, the amount of motion, etc. of the picture is considered for each region, and the optimum picture is selected as the predicted image. A predicted image of the same type as the predicted image selected by the moving image encoding device 1 is generated according to the type of predicted image included in the parameter, and a decoded image is generated.

As a result, when the object is blurred from the past picture to the current picture, the blurred picture B' is selected, and when the object is sharpened, the super-resolution picture C' is selected to generate the predicted image. be done. That is, a predicted image is generated that considers blurring and sharpening, and their directionality.

Therefore, it is possible to improve the efficiency of inter-picture prediction and obtain subjectively high-quality moving images even when objects are blurred or sharpened in each area between pictures of moving images. becomes.

Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments, and can be variously modified without departing from the technical idea thereof.

(Example provided with either one of the super-resolution processing units 11 and 60 and the blurring processing units 12 and 61)
In the above embodiment, as shown in FIG. 2, the inter-frame prediction unit 10 of the video encoding device 1 is provided with the super-resolution processing unit 11 and the blurring processing unit 12 . Further, as shown in FIG. 13 , the inter-screen prediction unit 56 of the video decoding device 2 is provided with a super-resolution processing unit 60 and a blurring processing unit 61 .

On the other hand, the inter-screen prediction unit 10 may include either one of the super-resolution processing unit 11 and the blurring processing unit 12 . Further, the inter-screen prediction unit 56 may include either one of the super-resolution processing unit 60 and the blurring processing unit 61 corresponding to the inter-screen prediction unit 10 .

When the inter-prediction unit 10 of the video encoding device 1 includes the super-resolution processing unit 11 and does not include the blurring processing unit 12, the inter-prediction processing unit 14 of the inter-prediction unit 10 generates a reference picture, and , past local decoded picture K and super-resolution pictures _CH,V,D based on reference pictures, and for each predetermined region, motion vectors KV, CVH _,V,D and predicted image KY , CY _H,V,D and prediction errors KE,CE _H,V,D .

For each predetermined region, the selection unit 16 performs the same processing as described above to determine the minimum prediction error among the prediction error KE of the locally decoded picture and the prediction error CE after weight multiplication of the super-resolution picture _H,V,D. Select the predicted image corresponding to .

When the inter-screen prediction unit 56 of the moving image decoding device 2 includes the super-resolution processing unit 60 and does not include the blurring processing unit 61, the inter-screen prediction processing unit 63 of the inter-screen prediction unit 56 performs prediction included in the parameters. One of the decoded picture K' and the super-resolution picture C' is read from the frame memory 62 according to the type of image, and based on the read picture and the motion vector included in the parameter, for each predetermined region Find the predicted image.

On the other hand, when the inter-prediction unit 10 of the video encoding device 1 includes the blur processing unit 12 and does not include the super-resolution processing unit 11, the inter-prediction processing unit 14 of the inter-prediction unit 10 uses the reference picture , and based on reference pictures, which are past locally decoded pictures K and blurred pictures B _{H, V, D} , perform inter-picture prediction, obtain motion vectors KV, BV _{H, V, D} for each predetermined region, and predict Obtain images KY, BY _H,V,D and prediction errors KE,BE _H,V,D .

The selection unit 16 performs the same processing as described above for each predetermined region, and selects the smallest prediction error among the prediction error KE of the locally decoded picture and the prediction error BEH _,V,D after weight multiplication of the blurred picture. Select the predicted image to use.

When the inter-screen prediction unit 56 of the moving image decoding device 2 includes the blur processing unit 61 and does not include the super-resolution processing unit 60, the inter-screen prediction processing unit 63 of the inter-screen prediction unit 56 performs prediction included in the parameters. Either one of the decoded picture K' and the blurred picture B' is read from the frame memory 62 according to the type of picture, and a predicted picture is generated for each predetermined region based on the read picture and the motion vector included in the parameter. Ask for

(Other selection processing of predicted image)
In the above-described embodiment, the selection unit 16 provided in the inter-picture prediction unit 10 of the video encoding device 1 selects weights tC _H,V,D for super-resolution pictures and weights tB for blurred pictures for each predetermined region. _H,V,D are calculated, prediction error CE after multiplication by local decoded picture prediction error KE, super-resolution picture weight tC _{H,V,D ,} and blurred picture weight tB Among the prediction errors BEH _,V,D multiplied by _H,V,D , the prediction image corresponding to the minimum prediction error is selected.

On the other hand, the selection unit 16 may first determine the type of predicted image to be selected, and select a predicted image from among the types. The type of predicted image is any one of locally decoded picture K, super-resolution picture C, and blurred picture B (any one of three types).

Specifically, using the power fluctuation amounts p _{H, V, and D} calculated from the power fluctuation information, the selection unit 16 selects the horizontal power fluctuation amount p _H , the vertical power fluctuation amount p V , and the corresponding power fluctuation amount p _V for each predetermined region. The sum or average of the angular power fluctuation amounts p _D is obtained as the power fluctuation amount, and the power fluctuation amount is subjected to threshold processing.

When the selection unit 16 determines that the power variation amount is larger than the first threshold value (preset positive value), the selection unit 16 determines super-resolution picture C as the type of predicted image. This is because when the power fluctuation amount is greater than the first threshold, the power of the current picture is greater than that of the past image, so it can be determined that the current image is sharper than the past image. In this case, the selection unit 16, in the same process as described above, selects the smallest prediction error among the prediction errors CE _{H,V,D after being multiplied by the weights tC H, V,D} of the super-resolution picture. Select the predicted image to use.

Further, when the selection unit 16 determines that the power fluctuation amount is between the first threshold value and the second threshold value (negative value set in advance), the selection unit 16 determines the locally decoded picture K as the predicted image type. do. When the power fluctuation amount is between the first threshold and the second threshold, the power of the current picture and the power of the past picture are almost the same, so the current picture is more powerful than the past picture. This is because it can be determined that the image is neither blurred nor sharpened. In this case, the selection unit 16 selects the locally decoded picture K as the prediction image.

Further, when the selection unit 16 determines that the amount of power fluctuation is smaller than the second threshold, the selection unit 16 determines the blurred picture B as the type of predicted image. This is because when the power fluctuation amount is smaller than the second threshold, the power of the current picture is smaller than the power of the past image, so it can be determined that the current image is more blurred than the past image. In this case, the selection unit 16 performs the same process as described above to perform the prediction corresponding to the smallest prediction error among the prediction errors BEH _,V,D multiplied by the weights _{tBH,V,D of the blurred picture.} Select an image.

Embodiments of the present invention are also applicable, for example, when including objects that move in and out of depth of field between pictures, when objects change from stationary to moving, or vice versa. be. For example, when the camera suddenly pans or tilts, when these operations stop, when an object moves from the back to the front in the screen, or vice versa.

A normal computer can be used as the hardware configuration of the moving image encoding device 1 and the moving image decoding device 2 according to the embodiment of the present invention. The moving image encoding device 1 and the moving image decoding device 2 are configured by a computer having a CPU, a volatile storage medium such as a RAM, a nonvolatile storage medium such as a ROM, an interface, and the like.

Subtraction unit 110 provided in video encoding device 1, orthogonal transformation unit 111, quantization unit 112, inverse quantization unit 113, inverse orthogonal transformation unit 114, addition unit 115, intra-screen prediction unit 116, in-loop filter 117, Each function of the inter-screen prediction unit 10, the switch 119, and the entropy coding unit 120 is realized by causing the CPU to execute a program describing these functions.

Also, an entropy decoding unit 50, an inverse quantization unit 51, an inverse orthogonal transform unit 52, an addition unit 53, an intra-frame prediction unit 54, an in-loop filter 55, an inter-frame prediction unit 56, and a switch 57 provided in the video decoding device 2 are realized by causing the CPU to execute a program describing these functions.

These programs are stored in the storage medium and are read and executed by the CPU. In addition, these programs can be stored and distributed in storage media such as magnetic disks (floppy (registered trademark) disks, hard disks, etc.), optical disks (CD-ROM, DVD, etc.), semiconductor memories, etc., and distributed via networks. You can also send and receive

1,100 video encoding device 2 video decoding devices 10, 56, 118, inter-prediction units 14, 63, 121 inter-prediction processing units 11, 60 super-resolution processing units 12, 61 blurring processing units 13, 62 , 122 frame memory 15 frequency analysis unit 16 selection unit 17, 64 parameter processing unit 20 decoded picture prediction unit 21 super-resolution prediction unit 22 blur prediction unit 30 motion amount calculation unit 31 power fluctuation amount calculation unit 32 weight calculation unit 33 predicted image selection unit 50 entropy decoding units 51, 113 inverse quantization units 52, 114 inverse orthogonal transformation units 53, 115 addition units 54, 116 intra-frame prediction units 55, 117 in-loop filters 57, 119 switch 110 subtraction unit 111 orthogonal transformation unit 112 Quantization unit 120 entropy coding unit K local decoded picture K′ decoded pictures C, _CH,V,D ,C′,CH _,V,D ′ super-resolution pictures _CH , _CH′ horizontal super-resolution pictures _CV , _CV ' vertical super-resolution pictures _CD , _CD ' diagonal super-resolution pictures B, BH _{,V,D, B} ', BH _,V,D ' blurred pictures _BH , _BH ' Horizontal blurred pictures _BV , _BV ' Vertical blurred pictures _BD , _BD ' Diagonal blurred pictures KY, KY' Predicted images CYH _,V,D , _CYH,V,D ' of local decoded pictures Super-resolution pictures Predicted image BY _H,V,D , BY _H,V,D ' Predicted image of blurred picture KV Motion vector of locally decoded picture CV _H,V,D Motion vector of super-resolution picture BV _H,V,D Blurry picture KE Prediction error of locally decoded picture CE _H,V,D Prediction error of super-resolution picture BE _H,V,D Prediction error of blurred picture CV _H,V,D -m _H,V,D super-resolution Picture motion amount BV _H,V,D -m _H,V,D Motion amount of blurred picture p _H,V,D Power fluctuation amount cp _H,V,D , cp _H1,V1,D1 , cp _{H2,V2, D2} , cp _{H3, V3, D3} Intensity information bp _{H, V,} D of super-resolution picture, bp _{H1, V1, D1 ,} bp _{H2, V2, D2 ,} bp _{H3, V3, D3} Intensity information t C _H, of blurred picture _V,D , super-resolved picture weight tB _H,V,D , blurred picture weight

Claims (9)

Subtracting a predicted image from an input moving image to generate a residual image, performing orthogonal transformation, quantization, and entropy coding on the residual image, generating and outputting an encoded signal, and performing the quantization performing inverse quantization and inverse orthogonal transformation on the generated quantization index sequence to generate a decoded residual image, adding the predicted image to the decoded residual image to generate an after-addition image, and generating the after-addition image; A video encoding device that performs filtering processing to generate a locally decoded picture, performs inter-screen prediction using the locally decoded picture, and generates the predicted image,
Generate a plurality of super-resolution pictures based on the locally decoded picture, and generate a predicted image of the locally decoded picture and a predicted image of the plurality of super-resolution pictures for each predetermined region by the inter prediction, An inter-screen prediction unit that selects any one of these predicted images,
The inter-screen prediction unit
Super-resolution processing is performed on the locally decoded picture to produce a picture in which horizontal high-frequency band components are emphasized, a picture in which vertical high-frequency band components are emphasized, and a picture in which diagonal high-frequency band components are emphasized, a super-resolution processing unit that generates the plurality of super-resolution pictures;
The current locally decoded picture is used as a reference picture, and the past local decoded picture and each of the plurality of past super-resolution pictures generated by the super-resolution processing unit are used as reference pictures to perform the inter-picture prediction, an inter-frame prediction processing unit that generates predicted images and motion vectors of the locally decoded pictures for each of the predetermined regions, and generates predicted images and motion vectors of the plurality of super-resolution pictures;
a frequency analysis unit that performs frequency analysis on the current locally decoded picture and the past locally decoded picture to generate power variation information between the current locally decoded picture and the past locally decoded picture;
For each of the predetermined regions, the power fluctuation information generated by the frequency analysis unit, or the power fluctuation information and the motion vector of the locally decoded picture generated by the inter-prediction processing unit and the plurality of super-resolutions a selection unit that selects one of the predicted image of the locally decoded picture and the predicted images of the plurality of super-resolution pictures based on the motion vector of the picture;
A video encoding device characterized by comprising: Subtracting a predicted image from an input moving image to generate a residual image, performing orthogonal transformation, quantization, and entropy coding on the residual image, generating and outputting an encoded signal, and performing the quantization performing inverse quantization and inverse orthogonal transformation on the generated quantization index sequence to generate a decoded residual image, adding the predicted image to the decoded residual image to generate an after-addition image, and generating the after-addition image; A video encoding device that performs filtering processing to generate a locally decoded picture, performs inter-screen prediction using the locally decoded picture, and generates the predicted image,
generating a plurality of blurred pictures based on the locally decoded picture; generating a predicted image of the locally decoded picture and a predicted image of the plurality of blurred pictures for each predetermined region by the inter prediction; An inter-screen prediction unit that selects one of the images,
The inter-screen prediction unit
Blurring processing is performed on the locally decoded picture, and a picture with horizontal high-frequency band components suppressed, a picture with vertical high-frequency band components suppressed, and a picture with diagonal high-frequency band components suppressed are selected from the plurality of pictures. a blur processor that generates a blurry picture of
The current locally decoded picture is used as a reference picture, and the past locally decoded picture and each of the plurality of past blurred pictures generated by the blur processing unit are used as reference pictures to perform the inter-picture prediction, and perform the inter prediction for each of the predetermined regions. an inter prediction processing unit that generates predicted images and motion vectors of the locally decoded pictures and generates predicted images and motion vectors of the plurality of blurred pictures;
a frequency analysis unit that performs frequency analysis on the current locally decoded picture and the past locally decoded picture to generate power variation information between the current locally decoded picture and the past locally decoded picture;
For each of the predetermined regions, the power fluctuation information generated by the frequency analysis unit, or the motion vector of the locally decoded picture and the motion vector of the plurality of blurred pictures generated by the power fluctuation information and the inter-prediction processing unit. a selection unit that selects one of the predicted image of the locally decoded picture and the predicted images of the plurality of blurred pictures based on a motion vector;
A video encoding device characterized by comprising: In the video encoding device according to claim 1,
generating the plurality of super-resolution pictures and a plurality of blurred pictures based on the locally decoded picture, and predicting the locally decoded picture and the plurality of super-resolution pictures for each of the predetermined regions by the inter prediction; an inter-screen prediction unit that generates a predicted image of a picture and predicted images of the plurality of blurred pictures, and selects one of the predicted images;
The inter-screen prediction unit further
Blurring processing is performed on the locally decoded picture, and a picture with horizontal high-frequency band components suppressed, a picture with vertical high-frequency band components suppressed, and a picture with diagonal high-frequency band components suppressed are selected from the plurality of pictures. It has a blur processing unit that generates a blurred picture of
The inter-screen prediction processing unit is
Using the current locally decoded picture as a reference picture, the past locally decoded picture, the plurality of past super-resolution pictures generated by the super-resolution processing unit, and the past generated by the blurring processing unit The inter prediction is performed using each of the plurality of blurred pictures as a reference picture, and for each of the predetermined regions, the predicted image and motion vector of the locally decoded picture, the predicted image and motion vector of the plurality of super-resolution pictures, and the generating predicted images and motion vectors for a plurality of blurred pictures;
The selection unit
For each of the predetermined regions, the power fluctuation information generated by the frequency analysis unit, or a motion vector of the locally decoded picture generated by the power fluctuation information and the inter-prediction processing unit, and the plurality of super-resolutions any one of a prediction image of the locally decoded picture, a prediction image of the plurality of super-resolution pictures, and a prediction image of the plurality of blurry pictures, based on the motion vector of the picture and the motion vectors of the plurality of blurry pictures. A video encoding device, characterized in that one is selected. In the video encoding device according to any one of claims 1 to 3,
The frequency analysis unit is
The frequency analysis is performed on the current locally decoded picture and the past locally decoded picture by first-order wavelet packet decomposition without decimation, and the analysis result of the current locally decoded picture and the analysis of the past locally decoded picture are performed. A moving picture encoding apparatus, which generates the power fluctuation information for each frequency band and for each phase position based on the result. Entropy decoding, inverse quantization, and inverse orthogonal transformation are performed on the encoded signal by inputting the coded signal of the video image, generating a decoded residual image, and adding the predicted image to the decoded residual image. generating a post-picture, performing filtering on the post-addition picture to generate a decoded picture, thereby restoring the original moving picture, performing inter-frame prediction using the decoded picture, and generating the predicted picture; In a video decoding device that
When the encoded signal contains a parameter indicating the type of one of the predicted images selected from the predicted image of a locally decoded picture and the predicted images of a plurality of super-resolution pictures,
generating a plurality of super-resolution pictures based on the decoded picture, and predicting any one of the decoded picture and the plurality of super-resolution pictures according to the parameter for each predetermined region by the inter-picture prediction; Equipped with an inter-screen prediction unit that generates
The inter-screen prediction unit
Super-resolution processing is performed on the decoded picture, and a picture with emphasized horizontal high-frequency band components, a picture with emphasized vertical high-frequency band components, and a picture with emphasized diagonal high-frequency band components are obtained as described above. a super-resolution processing unit that generates a plurality of super-resolution pictures;
According to the parameter, the inter-frame prediction is performed using the decoded picture and one of the plurality of super-resolution pictures generated by the super-resolution processing unit, and the prediction is performed for each of the predetermined regions. an inter-screen prediction processing unit that generates an image;
A video decoding device comprising: Entropy decoding, inverse quantization, and inverse orthogonal transformation are performed on the encoded signal by inputting the coded signal of the video image, generating a decoded residual image, and adding the predicted image to the decoded residual image. generating a post-picture, performing filtering on the post-addition picture to generate a decoded picture, thereby restoring the original moving picture, performing inter-frame prediction using the decoded picture, and generating the predicted picture; In a video decoding device that
When the coded signal includes a parameter indicating the type of one of the predicted images selected from the predicted image of a locally decoded picture and the predicted images of a plurality of blurred pictures,
A screen for generating a plurality of blurred pictures based on the decoded picture, and generating a predicted image of any one of the decoded picture and the plurality of blurred pictures according to the parameter for each predetermined region by the inter-frame prediction. with an inter-prediction unit,
The inter-screen prediction unit
Blurring processing is performed on the decoded picture, and a picture in which horizontal high-frequency band components are suppressed, a picture in which vertical high-frequency band components are suppressed, and a picture in which diagonal high-frequency band components are suppressed are divided into the plurality of pictures. a blur processor that generates a blurry picture;
performing the inter-frame prediction using the decoded picture and one of the plurality of blurred pictures generated by the blur processor according to the parameter to generate the predicted image for each of the predetermined regions; an inter-screen prediction processing unit;
A video decoding device comprising: In the video decoding device according to claim 5,
When the coded signal includes a parameter indicating the type of one of the predicted images selected from the predicted image of the locally decoded picture, the predicted image of the plurality of super-resolution pictures, and a plurality of blurred pictures,
generating the plurality of super-resolved pictures and the plurality of blurred pictures based on the decoded picture, and performing the inter-prediction, the decoded picture and the plurality of super-resolved pictures according to the parameters for each of the predetermined regions; and an inter-screen prediction unit that generates a predicted image of any one of the plurality of blurred pictures,
The inter-screen prediction unit further
Blurring processing is performed on the decoded picture, and a picture in which horizontal high-frequency band components are suppressed, a picture in which vertical high-frequency band components are suppressed, and a picture in which diagonal high-frequency band components are suppressed are divided into the plurality of pictures. Equipped with a blur processing unit that generates a blurred picture,
The inter-screen prediction processing unit is
using any one of the decoded picture, the plurality of super-resolution pictures generated by the super-resolution processing unit, and the plurality of blurry pictures generated by the blurring processing unit according to the parameter A moving image decoding device, wherein the inter-frame prediction is performed to generate the predicted image for each of the predetermined regions. A program for causing a computer to function as the video encoding device according to any one of claims 1 to 4. A program for causing a computer to function as the video decoding device according to any one of claims 5 to 7.

Images