JP2020150422A - Moving image encoder, moving image decoder, and program - Google Patents

Abstract

To obtain a moving image having subjectively high image quality even when blurring or sharpening of an object occurs for each region between pictures of a moving image.SOLUTION: A moving image encoder 1 includes an inter-screen prediction unit 10 comprising: a super-resolution processing unit 11 for generating super-resolution pictures CH,V,D; a blurring processing unit 12 for generating blurred pictures BH,V,D; an inter-screen prediction processing unit 14 for performing inter-screen prediction processing to calculate a prediction image and the like of each picture for each predetermined region; a frequency analysis unit 15 for calculating power variation information between pictures; and a selection unit 16 that for each predetermined region and on the basis of the power variation information and the like, calculates weight tCH,V,D, tBH,V,D and selects a prediction image corresponding to the minimum prediction error of a prediction error of a local decoded picture, prediction errors of the super-resolution pictures multiplied by the weight tCH,V,D, and prediction errors of the blurred pictures multiplied by the weight tBH,V,D.SELECTED DRAWING: Figure 2

Description

The present invention relates to a moving image coding device for encoding a moving image, a moving image decoding device for decoding a coded signal, and a program.

Conventionally, H.265 / HEVC (High Efficiency Video Coding) is known as a standard of a moving image compression method for a high-resolution 4K or 8K moving image (see, for example, Non-Patent Document 1). This H.265 / HEVC standard is based on motion compensation and orthogonal conversion techniques that predict motion using encoded frames and orthogonally transform and encode the signal of the predicted residuals.

FIG. 14 is a block diagram showing a configuration example of a conventional moving image coding device, and shows a configuration on the coding side according to the H.265 / HEVC standard. The moving image coding device 100 includes a subtraction unit 110, an orthogonal conversion unit 111, a quantization unit 112, an inverse quantization unit 113, an inverse orthogonal conversion unit 114, an addition unit 115, an in-screen prediction unit 116, and an in-loop filter 117. It includes an inter-screen prediction unit 118, a switch 119, and an entropy coding unit 120. The inter-screen prediction unit 118 includes an inter-screen prediction processing unit 121 and a frame memory 122.

The moving image coding device 100 is a device that performs coding processing for each frame of a picture on an input moving image to generate and output a coded signal.

The picture of the input moving image is subjected to subtraction processing by the subtraction unit 110 with the prediction image generated by the in-screen prediction unit 116 or the inter-screen prediction unit 118. As a result, a residual image is generated.

The residual image is subjected to orthogonal conversion processing by the orthogonal conversion unit 111, quantization processing by the quantization unit 112, and entropy coding by the entropy coding unit 120, and is output as a coded signal.

On the other hand, the quantization index sequence subjected to the quantization processing by the quantization unit 112 is subjected to the inverse quantization processing by the inverse quantization unit 113 and the inverse orthogonal conversion processing by the inverse orthogonal conversion unit 114. As a result, a decoding residual image is generated.

The decoded residual image is subjected to addition processing with the predicted image by the addition unit 115. The image of the addition result by the addition unit 115 is subjected to the in-screen prediction processing by the in-screen prediction unit 116. Further, the image of the addition result is filtered by the in-loop filter 117 and stored in the frame memory 122 of the inter-screen prediction unit 118 as a locally decoded picture.

The locally decoded picture stored in the frame memory 122 is read out as a past locally decoded picture by the inter-screen prediction processing unit 121. Then, the inter-screen prediction processing is performed using the current locally decoded picture (picture of the input moving image) as a reference picture and the past locally decoded picture as a reference picture.

One of the image generated by the in-screen prediction process by the in-screen prediction unit 116 and the image generated by the inter-screen prediction process by the inter-screen prediction processing unit 121 is selected by the switch 119, and the prediction image is selected. Is input to the subtraction unit 110.

In the moving image coding technology based on such motion compensation and orthogonal conversion, a technology for improving the coding efficiency has been proposed. For example, in order to improve the coding efficiency of a color difference signal, there is a technique of applying super-resolution processing by a plurality of frames to perform color difference interpolation and converting a color difference thinned image to YCbCr 4: 4: 4 (for example, Patent Document). 1).

In addition, in order to improve the coding efficiency of moving images, the image is reduced, then coded and transmitted, super-resolution restored after decoding, and the optimum folding frequency is set according to the complexity of the image. There is also a technique for improving super-resolution image quality by transmitting the image (see, for example, Patent Document 2).

Japanese Patent No. 6152642 Japanese Patent No. 6344800

Ei Okubo, "Impress Standard Textbook Series H.265 / HEVC Textbook", Impress Japan

In the conventional moving image coding technique based on motion compensation and orthogonal conversion, an image having the minimum prediction error is selected as the predicted image generated by the inter-screen prediction processing, for example, by a block matching method. As a result, the coding efficiency can be improved, and an objectively high-quality moving image can be obtained on the decoding side.

However, blurring or sharpening may occur in each region due to the movement of objects between pictures of moving images. In the conventional moving image coding technique, for example, even a moving image in which blurring or sharpening occurs, a predicted image is generated by using a normal locally decoded image.

For this reason, it is not possible to perform inter-screen prediction processing that distinguishes between a moving image in which blurring or sharpening occurs and a normal moving image (moving image in which blurring or sharpening does not occur). In addition, it is not possible to perform inter-screen prediction that distinguishes between a moving image in which blurring or sharpening occurs due to horizontal movement and a moving image in which blurring or sharpening occurs due to vertical movement.

As described above, in the conventional moving image coding technique, it is possible to objectively obtain a high-quality moving image, but there is a problem that it is not always possible to obtain a subjectively high-quality moving image. It was.

Therefore, the present invention has been made to solve the above problems, and an object of the present invention is to subjectively obtain high image quality even when an object is blurred or sharpened for each area between pictures of a moving image. It is an object of the present invention to provide a moving image coding device, a moving image decoding device, and a program capable of obtaining a moving image.

In order to solve the above-mentioned problems, the motion image coding apparatus according to claim 1 subtracts a predicted image from the input motion image to generate a residual image, and orthogonally transforms, quantizes, and entropy code the residual image. In addition to generating and outputting a coded signal, the quantization index sequence generated by the quantization is subjected to inverse quantization and inverse orthogonal conversion to generate a decoding residual image, and the decoding residual image is generated. The predicted image is added to to generate an image after addition, the added image is filtered to generate a locally decoded picture, and interscreen prediction is performed using the locally decoded picture to generate the predicted image. A plurality of super-resolution pictures are generated based on the locally decoded picture, and the predicted image of the locally decoded picture and the plurality of super-resolutions are generated for each predetermined area by the inter-screen prediction. An inter-screen prediction unit that generates a predicted image of a picture and selects any one of the predicted images is provided, and the inter-screen prediction unit performs super-resolution processing on the locally decoded picture and horizontally. A super-resolution that generates a picture that emphasizes a component of a high-frequency band in the direction, a picture that emphasizes a component of a high-frequency band in the vertical direction, and a picture that emphasizes a component of a high-frequency band in the diagonal direction as the plurality of super-resolution pictures. Between the screens, the processing unit and the current locally decoded picture are used as reference pictures, and each of the past locally decoded picture and the past plurality of super-resolution pictures generated by the super-resolution processing unit are used as reference pictures. An inter-screen prediction processing unit that performs prediction, generates a prediction image and a motion vector of the locally decoded picture for each predetermined region, and generates a prediction image and a motion vector of the plurality of super-resolution pictures, and a current A frequency analysis unit that performs frequency analysis of the locally decoded picture and the past locally decoded picture and generates power fluctuation information between the current locally decoded picture and the past locally decoded picture, and each of the predetermined regions. , The power fluctuation information generated by the frequency analysis unit, or the motion vector of the power fluctuation information, the locally decoded picture generated by the inter-screen prediction processing unit, and the motion vector of the plurality of super-resolution pictures. Based on this, it is characterized by including a selection unit for selecting any one of the predicted image of the locally decoded picture and the predicted image of the plurality of super-resolution pictures.

Further, the moving image coding device of claim 2 subtracts a predicted image from the input moving image to generate a residual image, performs orthogonal conversion, quantization and entropy coding on the residual image, and encodes the residual image. A signal is generated and output, and the quantization index sequence generated by the quantization is subjected to inverse quantization and inverse orthogonal conversion to generate a decoding residual image, and the predicted image is added to the decoding residual image. To generate an image after addition, filter the added image to generate a locally decoded picture, perform inter-screen prediction using the locally decoded picture, and generate the predicted image. In, a plurality of blurred pictures are generated based on the locally decoded picture, and a predicted image of the locally decoded picture and a predicted image of the plurality of blurred pictures are generated for each predetermined area by the inter-screen prediction. An inter-screen prediction unit for selecting any one of the predicted images is provided, and the inter-screen prediction unit performs blurring processing on the locally decoded picture to suppress components in the high frequency band in the horizontal direction. The blur processing unit that generates a picture in which the component of the high frequency band in the direction is suppressed and the picture in which the component of the high frequency band in the diagonal direction is suppressed as the plurality of blur pictures, and the current locally decoded picture are used as reference pictures in the past. The inter-screen prediction is performed using each of the locally decoded picture and the plurality of past blurred pictures generated by the blur processing unit as reference pictures, and the predicted image and motion vector of the locally decoded picture are performed for each predetermined area. Is generated, and the inter-screen prediction processing unit that generates the predicted images and motion vectors of the plurality of blurred pictures, and the frequency analysis of the current locally decoded picture and the past locally decoded picture are performed to perform the current local decoding. A frequency analysis unit that generates power fluctuation information between the picture and the past locally decoded picture, the power fluctuation information generated by the frequency analysis unit for each predetermined region, or the power fluctuation information and the said One of the predicted image of the locally decoded picture and the predicted image of the plurality of blurred pictures based on the motion vector of the locally decoded picture and the motion vector of the plurality of blurred pictures generated by the inter-screen prediction processing unit. It is characterized by having a selection unit for selecting one.

Further, the motion image coding device according to claim 3 generates the plurality of super-resolution pictures and the plurality of blurry pictures based on the locally decoded picture in the motion image coding device according to claim 1. By inter-screen prediction, a predicted image of the locally decoded picture, a predicted image of the plurality of super-resolution pictures, and a predicted image of the plurality of blurred pictures are generated for each predetermined area, and among these predicted images, the predicted images are generated. An inter-screen prediction unit for selecting any one is provided, and the inter-screen prediction unit further blurs the locally decoded picture to suppress components in the horizontal high-frequency band, and a vertical high-frequency band. A blurring processing unit that generates a picture in which the components of the above are suppressed and a picture in which the components in the high frequency band in the diagonal direction are suppressed as the plurality of blurring pictures are provided, and the inter-screen prediction processing unit produces the current locally decoded picture. As a reference picture, each of the past locally decoded picture, the past plurality of super-resolution pictures generated by the super-resolution processing unit, and the past plurality of blurring pictures generated by the blur processing unit can be used as reference pictures. The screen-to-screen prediction is performed as a reference picture, and the predicted image and motion vector of the locally decoded picture, the predicted image and motion vector of the plurality of super-resolution pictures, and the predicted image of the plurality of blurred pictures are performed for each predetermined area. And the motion vector is generated, and the selection unit generates the power fluctuation information or the power fluctuation information generated by the frequency analysis unit and the local area generated by the inter-screen prediction processing unit for each predetermined region. Based on the motion vector of the decoded picture, the motion vector of the plurality of super-resolution pictures, and the motion vector of the plurality of blurred pictures, the predicted image of the locally decoded picture, the predicted image of the plurality of super-resolution pictures, and the plurality of It is characterized in that one of the predicted images of the blurred picture is selected.

Further, in the moving image coding device according to claim 4, in the moving image coding device according to any one of claims 1 to 3, the frequency analysis unit uses the current local decoding picture and the past said. The frequency analysis of the locally decoded picture is performed by first-order wavelet packet decomposition without decimation, and based on the current analysis result of the locally decoded picture and the past analysis result of the locally decoded picture, each frequency band and phase It is characterized in that the power fluctuation information for each position is generated.

Further, the moving image decoding apparatus according to claim 5 inputs a moving image encoded signal, performs entropy decoding, inverse quantization, and inverse orthogonal conversion on the encoded signal to generate a decoding residual image, and the above-mentioned The predicted image is added to the decoded residual image to generate an image after addition, and the added image is filtered to generate a decoded picture, thereby restoring the original moving image and using the decoded picture. In a moving image decoding device that performs inter-screen prediction and generates the predicted image, the coded signal is a predicted image of one of the predicted images selected from a predicted image of a locally decoded picture and a predicted image of a plurality of super-resolution pictures. When a parameter indicating the type is included, a plurality of super-resolution pictures are generated based on the decoded picture, and according to the inter-screen prediction, the decoded picture and the plurality of super-resolution pictures are generated for each predetermined area according to the parameter. An inter-screen prediction unit that generates a prediction image of any one of the resolution pictures is provided, and the inter-screen prediction unit performs super-resolution processing on the decoded picture to emphasize the components in the high frequency band in the horizontal direction. A super-resolution processing unit that generates a picture, a picture emphasizing a component in a vertical high-frequency band, and a picture emphasizing a component in a diagonal high-frequency band as the plurality of super-resolution pictures, and the above-mentioned according to the parameters. A screen that performs inter-screen prediction using the decoded picture and any one of the plurality of super-resolution pictures generated by the super-resolution processing unit, and generates the predicted image for each predetermined area. It is characterized by having an interval prediction processing unit.

Further, the moving image decoding device according to claim 6 inputs a moving image encoded signal, performs entropy decoding, inverse quantization, and inverse orthogonal conversion on the encoded signal to generate a decoding residual image, and said that. The predicted image is added to the decoded residual image to generate the added image, and the added image is filtered to generate the decoded picture, thereby restoring the original moving image and using the decoded picture. In the moving image decoding device that performs inter-screen prediction and generates the predicted image, the coded signal is a type of one predicted image selected from the predicted image of the locally decoded picture and the predicted image of a plurality of blurred pictures. When the indicated parameters are included, a plurality of blurred pictures are generated based on the decoded picture, and according to the inter-screen prediction, the decoded picture and the plurality of blurred pictures are generated according to the parameters for each predetermined area. An inter-screen prediction unit that generates any one of the predicted images is provided, and the inter-screen prediction unit performs blurring processing on the decoded picture to suppress components in the high frequency band in the horizontal direction, and a high frequency band in the vertical direction. A blur processing unit that generates a picture in which components are suppressed and a picture in which components in a diagonal high frequency band are suppressed as the plurality of blur pictures, and the decoded picture and the blur processing unit generated by the blur processing unit according to the parameters. It is characterized by including an inter-screen prediction processing unit that performs inter-screen prediction using any one of a plurality of blurred pictures and generates the predicted image for each predetermined area.

In addition, the moving image decoding device according to claim 7 is the moving image decoding device according to claim 5, in which the encoded signal includes a predicted image of the locally decoded picture, a predicted image of the plurality of super-resolution pictures, and a plurality of predicted images. When a parameter indicating one type of the predicted image selected from the blurred pictures is included, the plurality of super-resolution pictures and the plurality of blurred pictures are generated based on the decoded picture, and the screen is displayed. The inter-screen prediction unit is provided for each predetermined region to generate a prediction image of any one of the decoded picture, the plurality of super-resolution pictures, and the plurality of blurry pictures according to the parameters. The inter-screen prediction unit further blurs the decoded picture to suppress the horizontal high frequency band component, the vertical high frequency band component suppressed picture, and the diagonal high frequency band component. The inter-screen prediction processing unit includes the blurring processing unit that generates the suppressed picture as the plurality of blurring pictures, and the inter-screen prediction processing unit performs the decoding picture and the plurality of super-resolutions generated by the super-resolution processing unit according to the parameters. The feature is that the inter-screen prediction is performed using any one of the image picture and the plurality of blur pictures generated by the blur processing unit, and the predicted image for each predetermined area is generated. To do.

Further, the program of claim 8 is characterized in that the computer functions as the moving image coding device according to any one of claims 1 to 4.

The program of claim 9 is characterized in that the computer functions as the moving image decoding device according to any one of claims 5 to 7.

As described above, according to the present invention, it is possible to subjectively obtain a high-quality moving image even when the object is blurred or sharpened for each region between the pictures of the moving image.

It is a block diagram which shows the structural example of the moving image coding apparatus by embodiment of this invention. It is a block diagram which shows the structural example of the inter-screen prediction part provided in the moving image coding apparatus. It is a figure explaining the process flow from the inverse orthogonal conversion part to the inter-screen prediction part through an addition part and an in-loop filter. It is a figure explaining the processing example of the super-resolution processing part. It is a figure explaining the processing example of the blurring processing part. It is a block diagram which shows the structural example of the inter-screen prediction processing unit. It is a figure explaining the processing example of the frequency analysis part. It is a block diagram which shows the structural example of a selection part. It is a flowchart which shows the processing example of a selection part. It is a figure explaining the example of the movement amount CV _H- m _{H, V, D} of the horizontal super-resolution picture calculated by the movement amount calculation unit. It is a figure explaining the example of the power fluctuation amount pH _{, V, D} calculated by the power fluctuation amount calculation unit. It is a block diagram which shows the structural example of the moving image decoding apparatus by embodiment of this invention. It is a block diagram which shows the structural example of the interscreen prediction part provided in the moving image decoding apparatus. It is a block diagram which shows the structural example of the conventional moving image coding apparatus.

Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings. In the moving image coding technique based on motion compensation and orthogonal conversion, the present invention includes a plurality of super-resolution pictures C and / and a plurality of blurs as reference pictures used for inter-screen prediction processing in addition to the locally decoded picture K. The feature is that the picture B is generated, and the optimum predicted image is selected for each area in consideration of the power fluctuation amount, the movement amount, and the like of the picture.

As a result, when the object is blurred from the past picture to the current picture between the pictures of the moving image, the predicted image of the blurred picture B is selected for each area. Further, when the object is sharpened, the super-resolution picture C is selected for each area. Therefore, the residual between the input moving image and the predicted image can be reduced.

Further, on the decoding side as well, when the original moving image is blurred, the predicted image of the blurred picture B'is generated, and the blurred moving image can be restored. Further, when the sharpening of the object occurs in the original moving image, a super-resolution picture C'is generated, and the moving image in which the sharpening occurs can be restored.

Therefore, even when the object is blurred or sharpened for each area between the pictures of the moving image, the inter-screen prediction efficiency can be improved and the moving image of high quality can be subjectively obtained. It becomes.

[Video encoding device]
First, a moving image coding device according to an embodiment of the present invention will be described. FIG. 1 is a block diagram showing a configuration example of a moving image coding device according to an embodiment of the present invention. The moving image coding device 1 includes a subtraction unit 110, an orthogonal conversion unit 111, a quantization unit 112, an inverse quantization unit 113, an inverse orthogonal conversion unit 114, an addition unit 115, an in-screen prediction unit 116, and an in-loop filter 117. It includes an inter-screen prediction unit 10, a switch 119, and an entropy coding unit 120. Note that FIG. 1 shows only the components directly related to the present invention, and the components not directly related to the present invention are omitted.

The subtraction unit 110 inputs a picture of the input moving image (I picture, P picture, and B picture), and inputs a predicted image of the picture from the switch 119. Then, the subtraction unit 110 subtracts the predicted image from the picture of the input moving image, generates a residual image of the subtraction result, and outputs the residual image to the orthogonal conversion unit 111.

The orthogonal conversion unit 111 inputs a residual image from the subtraction unit 110, performs orthogonal conversion on the residual image, and generates a conversion coefficient sequence. Then, the orthogonal conversion unit 111 outputs the conversion coefficient sequence to the quantization unit 112.

The quantization unit 112 inputs a conversion coefficient sequence from the orthogonal conversion unit 111, performs quantization on the conversion coefficient sequence, and generates a quantization index sequence. Then, the quantization unit 112 outputs the quantization index sequence to the inverse quantization unit 113 and the entropy coding unit 120.

The inverse quantization unit 113 inputs the quantization index string from the quantization unit 112 and performs the reverse processing of the quantization unit 112 to inversely quantize the quantization index sequence and generate a conversion coefficient sequence. Then, the inverse quantization unit 113 outputs the conversion coefficient sequence to the inverse orthogonal conversion unit 114.

The inverse orthogonal conversion unit 114 inputs the conversion coefficient sequence from the inverse quantization unit 113 and performs the reverse processing of the orthogonal conversion unit 111 to perform inverse orthogonal conversion of the conversion coefficient sequence and generate a decoding residual image. Then, the inverse orthogonal conversion unit 114 outputs the decoding residual image to the addition unit 115.

The addition unit 115 inputs the decoding residual image from the inverse orthogonal conversion unit 114, and inputs the predicted image from the switch 119. Then, the addition unit 115 adds the decoding residual image to the prediction image, and outputs the added image to the in-screen prediction unit 116 and the in-loop filter 117.

The in-screen prediction unit 116 inputs the image after addition from the addition unit 115, performs in-screen prediction processing on the added image for each predetermined area, generates an in-screen prediction result, and in-screen for each predetermined area. The prediction result is output to the switch 119.

The in-loop filter 117 inputs the image after addition from the addition unit 115, filters the image after addition, and generates a locally decoded picture K. Then, the in-loop filter 117 outputs the locally decoded picture K to the inter-screen prediction unit 10.

The inter-screen prediction unit 10 inputs the locally decoded picture K from the in-loop filter 117, and generates a plurality of super-resolution pictures C and a plurality of blurred pictures B from the locally decoded picture K. The locally decoded picture K, the plurality of super-resolution pictures C, and the plurality of blurred pictures B are used as reference pictures for interscreen prediction processing.

The inter-screen prediction unit 10 performs inter-screen prediction processing for each predetermined area by using a reference picture which is a current locally decoded picture K (picture of an input moving image) and a reference picture such as a locally decoded picture K. The inter-screen prediction unit 10 obtains motion vectors, prediction images, and prediction errors for each of the locally decoded picture K, the plurality of super-resolution pictures C, and the plurality of blurry pictures B for each predetermined area.

The inter-screen prediction unit 10 obtains a weight from the power fluctuation amount of the picture for each predetermined area, multiplies the weight by the prediction error, and obtains the image of the picture that minimizes the prediction error after the weight multiplication is the optimum prediction image. Select as. Then, the inter-screen prediction unit 10 outputs the prediction image selected for each predetermined area to the switch 119 as the inter-screen prediction result for each predetermined area.

Further, the inter-screen prediction unit 10 sets parameters such as the type of the predicted image actually selected for each predetermined area, the calculated motion vector, and the like in the process of selecting the optimum predicted image for each predetermined area. Then, the inter-screen prediction unit 10 outputs the parameter to the entropy coding unit 120. The details of the inter-screen prediction unit 10 will be described later.

The switch 119 inputs the in-screen prediction result for each predetermined area from the in-screen prediction unit 116, and also inputs the in-screen prediction result for each predetermined area from the inter-screen prediction unit 10, selects one of them, and selects a prediction image. Is output to the subtraction unit 110 and the addition unit 115.

The entropy encoding unit 120 inputs the quantization index sequence from the quantization unit 112, inputs the parameters from the inter-screen prediction unit 10, and further inputs the parameters from the orthogonal conversion unit 111 and the like. Then, the entropy coding unit 120 performs entropy coding on the quantization index sequence and the parameter, and generates a coded signal. The entropy coding unit 120 outputs the coded signal to the moving image decoding device 2 described later.

[Inter-screen prediction unit 10]
Next, the inter-screen prediction unit 10 shown in FIG. 1 will be described in detail. FIG. 2 is a block diagram showing a configuration example of the inter-screen prediction unit 10 provided in the moving image coding device 1. The inter-screen prediction processing unit 10 includes a super-resolution processing unit 11, a blur processing unit 12, a frame memory 13, an inter-screen prediction processing unit 14, a frequency analysis unit 15, a selection unit 16, and a parameter processing unit 17. Note that FIG. 2 shows only the components directly related to the present invention, and the components not directly related to the present invention are omitted.

The inter-screen prediction unit 10 inputs the locally decoded picture K from the in-loop filter 117, and stores the locally decoded picture K in the frame memory 13.

The super-resolution processing unit 11 performs super-resolution processing on the locally decoded picture K, and three types of super-resolution pictures C _{H, V, D} (horizontal super-resolution picture C _H , vertical super-resolution picture C _V, and generating a diagonal super-resolution picture C _D). H is horizontal, V is vertical, and D is diagonal. The super-resolution processing unit 11 stores three types of super-resolution pictures C _{H, V, and D} in the frame memory 13. The details of the super-resolution processing unit 11 will be described later.

The horizontal super-resolution picture C _H is an image in which components in the high frequency band in the horizontal direction are emphasized. The vertical super-resolution picture _CV is an image in which components in the high frequency band in the vertical direction are emphasized. Diagonal super-resolution picture C _D is an image that emphasizes the high frequency band components in the diagonal direction (horizontal and vertical).

The blur processing unit 12 performs blur processing on the locally decoded picture K to generate three types of blur pictures B _{H, V, D} (horizontal blur picture B _H , vertical blur picture B _V, and diagonal blur picture B _D ). .. The blur processing unit 12 stores three types of blur pictures B _{H, V, and D} in the frame memory 13. The details of the blur processing unit 12 will be described later.

The horizontally blurred picture B _H is an image in which components in the high frequency band in the horizontal direction are suppressed. The vertical blur picture B _V is an image in which components in the high frequency band in the vertical direction are suppressed. The diagonally blurred picture _BD is an image in which components in the high frequency band in the diagonal direction (horizontal and vertical directions) are suppressed.

The frame memory 13 stores locally decoded pictures K, three types of super-resolution pictures C _{H, V, D,} and three types of blurry pictures B _{H, V, D} for each frame.

The inter-screen prediction processing unit 14 displays the locally decoded picture K stored in the frame memory 13, three types of super-resolution pictures C _{H, V, D} and three types of blurred pictures B _{H, V, D} in the past. Read as decoded picture K, super-resolution picture C _{H, V, D} and blurred picture B _{H, V, D.} Further, the inter-screen prediction processing unit 14 inputs the current locally decoded picture K (picture of the input moving image).

The inter-screen prediction processing unit 14 uses the current locally decoded picture K as a reference picture, and the past locally decoded picture K, the super-resolution picture C _{H, V, D} and the blurred picture B _{H, V, D} as reference pictures. Predicted images KY, CY _{H, V, D} , BY _{H, V, D} , motion vectors KV, CV _{H, V, D} , BV _{H, V, D} and prediction error for each predetermined region, for example, by the block matching method. Find KE, CE _{H, V, D} , BE _{H, V, D.} The predetermined area is, for example, a slice area, a CTU (Coding Tree Unit) area, and a CU (Coding Unit) area.

The inter-screen prediction processing unit 14 includes prediction images KY, CY _{H, V, D} , BY _{H, V, D} , motion vectors KV, CV _{H, V, D} , BV _{H, V, D} and prediction errors for each predetermined area. KE, CE _{H, V, D} , BE _{H, V, D} are output to the selection unit 16. The details of the inter-screen prediction processing unit 14 will be described later.

The predicted image KY is a predicted image of one type of locally decoded picture. The predicted images CY _{H, V, and D} are predicted images of three types of super-resolution pictures, the predicted image CY _{H of} the horizontal super-resolution picture, the predicted image CY _V of the vertical super-resolution picture, and the diagonal super-resolution. It consists of a predicted image CY _{D of a} picture.

The predicted image BY _{H, V, D} is a predicted image of three types of blurred pictures, and is composed of a predicted image BY _{H of} a horizontally blurred picture, a predicted image BY _V of a vertically blurred picture, and a predicted image BY _D of a diagonal blurred picture. ..

The motion vector KV is a motion vector of one type of locally decoded picture. The motion vectors CV _{H, V, and D} are motion vectors of three types of super-resolution pictures, the motion vector CV _{H of} the horizontal super-resolution picture, the motion vector CV _V of the vertical super-resolution picture, and the diagonal super-resolution. It consists of the motion vector CV _{D of the} picture.

The motion vector BV _{H, V, D} is a motion vector of three types of blurry pictures, and is composed of a motion vector BV _{H of} a horizontally blurred picture, a motion vector BV _V of a vertically blurred picture, and a motion vector BV _D of a diagonally blurred picture. ..

The prediction error KE is a prediction error of one type of locally decoded picture. The prediction errors CE _{H, V, and D} are the prediction errors of three types of super-resolution pictures, the prediction error CE _{H of} the horizontal super-resolution picture, the prediction error CE _V of the vertical super-resolution picture, and the diagonal super-resolution. It consists of the picture prediction error CE _D.

The prediction error BE _{H, V, D} is a prediction error of three types of blurry pictures, and consists of a prediction error BE _{H of} a horizontally blurred picture, a prediction error BE _V of a vertically blurred picture, and a prediction error BE _D of a diagonally blurred picture. ..

The frequency analysis unit 15 reads the locally decoded picture K stored in the frame memory 13 as the past locally decoded picture K, and inputs the current locally decoded picture K.

The frequency analysis unit 15 performs frequency analysis on each of the current locally decoded picture K and the past locally decoded picture K. Then, the frequency analysis unit 15 subtracts the analysis result of the past locally decoded picture K from the analysis result of the current locally decoded picture K, and obtains the power fluctuation information for each frequency band and each phase position. The frequency analysis unit 15 outputs power fluctuation information for each frequency band and each phase position to the selection unit 16. The details of the frequency analysis unit 15 will be described later.

The selection unit 16 is a prediction image KY, CY _{H, V, D} , BY _{H, V, D} , motion vector KV, CV _{H, V, D} , BV _{H, V, for} each predetermined area from the inter-screen prediction processing unit 14 _. Enter _D and prediction error KE, CE _{H, V, D} , BE _{H, V, D.} Further, the selection unit 16 inputs power fluctuation information for each frequency band and each phase position from the frequency analysis unit 15.

The selection unit 16 has motion vectors CV _{H, V, D} of the super-resolution picture and intensity information cp _{H, V} of the super-resolution picture for three types of super-resolution pictures C _{H, V, D} for each predetermined area. Based on _{, D} , the weights tC _{H, V, D} of the super-resolution picture are calculated.

The selection unit 16 may calculate the weights tC _{H, V, D} of the super-resolution picture based on the power fluctuation information and the intensity information cp _{H, V, D} of the super-resolution picture. Further, the selection unit 16 determines the super-resolution picture weight tC _H based on the motion vector CV _{H, V, D of} the super-resolution picture, the power fluctuation information, and the intensity information cp _{H, V, D} of the super-resolution picture. _{, V, D} may be calculated.

The weights tC _{H, V, and D} are the weights of three types of super-resolution pictures, the weight tC _{H of} the horizontal super-resolution picture, the weight tC _V of the vertical super-resolution picture, and the weight tC of the diagonal super-resolution picture. _{Consists of D.}

Further, the selection unit 16 sets the weight tB _{H of the} blur picture based on the power fluctuation information and the intensity information bp _{H, V, D} of the blur picture for each of the three types of blur pictures B _{H, V, D} for each predetermined area. _{, V, D} are calculated.

The selection unit 16 calculates the weights tB _{H, V, D} of the blurred picture based on the motion vector BV _{H, V, D of} the blurred picture, the power fluctuation information, and the intensity information bp _{H, V, D} of the blurred picture. You may try to do it.

Weight tB _{H, V, D} is 3 a kind of blur weights picture consists weight tB _D horizontal blur weight tB _H picture, vertical blur picture blur weight tB _V and diagonal picture.

The selection unit 16 has weights tC _{H, V,} corresponding to the prediction errors CE _{H, V, D} of the super-resolution pictures for each of the three types of super-resolution pictures C _{H, V, D} for each predetermined region _. Multiply _D. Further, the selection unit 16 assigns corresponding weights tB _{H, V, D} to each of the prediction errors BE _{H, V, D} of the blurry picture for each of the three types of blurry pictures B _{H, V, D} for each predetermined area. Multiply.

The selection unit 16 has a prediction error KE of the locally decoded picture, a prediction error CE _{H, V, D} after weight multiplication of the super-resolution picture, and a prediction error BE _{H, V} after weight multiplication of the blurred picture for each predetermined area. _{, D} , specify the smallest prediction error. Then, the selection unit 16 selects a prediction image corresponding to the specified prediction error. As a result, any one of the predicted images KY, CY _{H, V, D} , BY _{H, V, and D} of the seven types of pictures is selected.

The selection unit 16 outputs the selected predicted image to the switch 119 as an inter-screen prediction result for each predetermined area. The details of the selection unit 16 will be described later.

The parameter processing unit 17 sets the type of the predicted image selected for each predetermined area by the selection unit 16 and the calculated motion vector as parameters, and outputs the parameters to the entropy coding unit 120.

FIG. 3 is a diagram illustrating a processing flow from the inverse orthogonal conversion unit 114 to the inter-screen prediction unit 10 via the addition unit 115 and the in-loop filter 117.

The addition unit 115 adds the decoding residual image generated by the inverse orthogonal conversion unit 114 and the inter-screen prediction result, that is, the prediction image generated by the inter-screen prediction unit 10, and the added image is the in-loop filter 117. Is entered in. The locally decoded picture K filtered by the in-loop filter 117 is stored in the frame memory 13 and input to the super-resolution processing unit 11 and the blur processing unit 12.

The super-resolution processing unit 11 performs super-resolution processing on the locally decoded picture K _, generates three types of super-resolution pictures C _{H, V, and D} , and stores them in the frame memory 13. The blurring processing unit 12 performs blurring processing on the locally decoded picture K _, generates three types of blurry pictures B _{H, V, and D} , and stores them in the frame memory 13.

As a result, the frame memory 13 stores seven types of pictures, which are the locally decoded picture K, the three types of super-resolution pictures C _{H, V, D,} and the three types of blurred pictures B _{H, V, D.} These pictures are read out by the inter-screen prediction processing unit 14 as past locally decoded pictures K, super-resolution pictures C _{H, V, D} and blurry pictures B _{H, V, D.}

In the inter-screen prediction processing unit 14, inter-screen prediction processing is performed using the current locally decoded picture K as a reference picture and the past locally decoded picture K as a reference picture to generate a predicted image KY and the like. Further, in the inter-screen prediction processing unit 14, three types of inter-screen prediction processing are performed using the past three types of super-resolution pictures _{CH, V, D} as reference pictures, and three types of prediction images CY _{H, V, D} etc. are generated. Further, in the inter-screen prediction processing unit 14, three types of inter-screen prediction processing are performed using the past three types of blurred pictures B _{H, V, D} as reference pictures, and three types of prediction images BY _{H, V, D and the} like are performed. Is generated. The predicted images KY, CY _{H, V, D} , BY _{H, V, D and the} like are input to the selection unit 16.

The frequency analysis unit 15 generates power fluctuation information based on the current locally decoded picture K and the past locally decoded picture K, and the power fluctuation information is input to the selection unit 16.

The selection unit 16 selects any one of the predicted images KY, CY _{H, V, D} , BY _{H, V, and D} for each predetermined area based on the power fluctuation information and the like, and between the screens. It is output to the addition unit 115 as a prediction result.

As described above, in the inter-screen prediction processing unit 14, there are a total of seven types of reference pictures, a locally decoded picture K, three types of super-resolution pictures C _{H, V, D,} and three types of blurry pictures B _{H, V, D.} Picture is used. Further, for each predetermined area, any one of the seven types of predicted images KY, CY _{H, V, D} , BY _{H, V, and D} is selected as the predicted image.

[Super-resolution processing unit 11]
Next, the super-resolution processing unit 11 shown in FIG. 2 will be described in detail. FIG. 4 is a diagram illustrating a processing example of the super-resolution processing unit 11.

The super-resolution processing unit 11 inputs the local decoding picture K, and for example, the local decoding picture K is first-order wavelet-decomposed (step S401). Then, the super-resolution processing unit 11 sets the element values (pixel values) of all the phase positions in the horizontal (H), vertical (V), and diagonal (D) high-frequency bands of the image after the first-order wavelet decomposition. On the other hand, the horizontal intensity information cp _{H1, V1, D1 of} the preset super-resolution picture, the vertical intensity information cp _{H2, V2, D2} and the diagonal intensity information cp _{H3, V3, D3} are multiplied respectively (step). S402).

The horizontal intensity information cp _{H1, V1, D1} corresponds to the horizontal intensity cp _H1 corresponding to the horizontal high frequency band in the image after the first-order wavelet decomposition, the vertical intensity cp _V1 corresponding to the vertical high frequency band, and the diagonal high frequency band. It consists of diagonal strength cp _D1 . The horizontal intensity information cp _{H1, V1, D1} is the intensity information for generating a horizontal super-resolution picture C _H that emphasizes the components of the horizontal high-frequency band (strong power), and is cp _H1 > cp _V1 , cp _D1. Strength information satisfying the above conditions is used. For example, as shown in FIG. 4, horizontal strength cp _H1 = 1.2, vertical strength cp _V1 = 1.0, and diagonal strength cp _D1 = 1.0 are used.

The vertical strength information cp _{H2, V2, D2} is composed of the horizontal strength cp _H2 , the vertical strength cp _V2, and the diagonal strength cp _D2 . Vertical intensity information cp _{H2, V2, D2} is intensity information for generating a vertical super-resolution picture C _V that emphasizes the components of the vertical high-frequency band (strong power), and is cp _V2 > cp _H2 , cp _D2. Strength information satisfying the above conditions is used. For example, as shown in FIG. 4, horizontal strength cp _H2 = 1.0, vertical strength cp _V2 = 1.2, and diagonal strength cp _D2 = 1.0 are used.

The diagonal strength information cp _{H3, V3, D3} is composed of the horizontal strength cp _H3 , the vertical strength cp _V3, and the diagonal strength cp _D3 . For diagonal intensity information cp _{H3, V3, D3} emphasized the elements of the diagonal high frequency band is the intensity information for generating the diagonal super-resolution picture C _D (strong power), cp _D3> cp _H3 , Cp _V3 condition information is used. For example, as shown in FIG. 4, horizontal strength cp _H3 = 1.0, vertical strength cp _V3 = 1.0, and diagonal strength cp _D3 = 1.2 are used.

The super-resolution processing unit 11 reconstructs each of the multiplied images after the first-order wavelet decomposition into first-order wavelets, and three types of super-resolution pictures C _{H, V, D} , that is, horizontal super-resolution pictures C _H , A vertical super-resolution picture C _V and a diagonal super-resolution picture C _D are generated (step S403).

In this way, the super-resolution processing unit 11 generates three types of horizontal super-resolution picture C _H , vertical super-resolution picture C _V, and diagonal super-resolution picture C _D from the locally decoded picture K.

[Blur processing unit 12]
Next, the blurring processing unit 12 shown in FIG. 2 will be described in detail. FIG. 5 is a diagram illustrating a processing example of the blur processing unit 12.

The blurring processing unit 12 inputs the locally decoded picture K, and for example, the locally decoded picture K is first-order wavelet-decomposed (step S501). Then, the blur processing unit 12 sets the horizontal intensity information bp _{H1 of} the blur picture preset for the element values of all the phase positions in the horizontal, vertical, and diagonal high frequency bands of the image after the first-order wavelet decomposition. _{, V1, D1} , vertical strength information bp _{H2, V2, D2} and diagonal strength information bp _{H3, V3, D3} , respectively (step S502).

The horizontal intensity information bp _{H1, V1, D1} corresponds to the horizontal intensity bp _H1 corresponding to the horizontal high frequency band in the image after the first-order wavelet decomposition, the vertical intensity bp _V1 corresponding to the vertical high frequency band, and the diagonal high frequency band. It consists of diagonal strength bp _D1 . The horizontal intensity information bp _{H1, V1, D1} is the intensity information for generating a horizontally blurred picture B _{H in} which the components of the horizontal high frequency band are suppressed (weak power), and the conditions of bp _H1 <bp _V1 and bp _D1 . Intensity information that satisfies is used. For example, as shown in FIG. 5, horizontal strength bp _H1 = 0.8, vertical strength bp _V1 = 1.0, and diagonal strength bp _D1 = 1.0 are used.

The vertical strength information bp _{H2, V2, D2} is composed of the horizontal strength bp _H2 , the vertical strength bp _V2, and the diagonal strength bp _D2 . The vertical intensity information bp _{H2, V2, D2} is the intensity information for generating a vertically blurred picture B _{V in} which the components of the vertical high frequency band are suppressed (weak power), and the conditions of bp _V2 <bp _H2 , bp _D2 . Intensity information that satisfies is used. For example, as shown in FIG. 5, horizontal strength bp _H2 = 1.0, vertical strength bp _V2 = 0.8, and diagonal strength bp _D2 = 1.0 are used.

The diagonal strength information bp _{H3, V3, D3} is composed of the horizontal strength bp _H3 , the vertical strength bp _V3, and the diagonal strength bp _D3 . Diagonal intensity information bp _{H3, V3, D3} is intensity information for generating a diagonally blurred picture _{BD in} which components in the diagonal high frequency band are suppressed (weak power), and bp _D3 <bp _H3 , bp. Strength information that satisfies the conditions of _V3 is used. For example, as shown in FIG. 5, horizontal strength bp _H3 = 1.0, vertical strength bp _V3 = 1.0, and diagonal strength bp _D3 = 0.8 are used.

The blur processing unit 12 reconstructs each of the multiplied images after the first-order wavelet decomposition into the first-order wavelet, and three types of blur pictures B _{H, V, D} , that is, horizontal blur picture B _H , vertical blur picture B _V, and A diagonally blurred picture _BD is generated (step S503).

In this way, the blur processing unit 12 generates three types of horizontal blur picture B _H , vertical blur picture B _V, and diagonal blur picture B _D from the locally decoded picture K.

[Inter-screen prediction processing unit 14]
Next, the inter-screen prediction processing unit 14 shown in FIG. 2 will be described in detail. FIG. 6 is a block diagram showing a configuration example of the inter-screen prediction processing unit 14. The inter-screen prediction processing unit 14 includes a decoding picture prediction unit 20, a super-resolution prediction unit 21, and a blur prediction unit 22. Note that FIG. 6 shows only the components directly related to the present invention, and the components not directly related to the present invention are omitted.

The decoding picture prediction unit 20 reads the local decoding picture K stored in the frame memory 13 as the past local decoding picture K, and inputs the current local decoding picture K.

The decoded picture prediction unit 20 uses the current locally decoded picture K as a reference picture and the past locally decoded picture K as a reference picture, and based on the reference picture and the reference picture, for each predetermined area, for example, by a block matching method. The prediction image KY, the motion vector KV, and the prediction error KE are obtained.

The decoded picture prediction unit 20 outputs the prediction image KY, the motion vector KV, and the prediction error KE to the selection unit 16.

The super-resolution prediction unit 21 reads out three types of super-resolution pictures C _{H, V, D} stored in the frame memory 13 as past super-resolution pictures C _{H, V, D} , and reads the current local decoding picture. Enter K.

The super-resolution prediction unit 21 uses the current locally decoded picture K as a reference picture and the past super-resolution pictures C _{H, V, D} as reference pictures, and based on the reference picture and the reference picture, for each predetermined area, For example, by the block matching method, it is obtained as the predicted image CY _{H, V, D} , the motion vector CV _{H, V, D} and the prediction error CE _{H, V, D.}

The super-resolution prediction unit 21 outputs the prediction images CY _{H, V, D} , the motion vectors CV _{H, V, D} and the prediction error CE _{H, V, D} to the selection unit 16.

The blur prediction unit 22 reads out three types of blur pictures B _{H, V, D} stored in the frame memory 13 as past blur pictures B _{H, V, D} , and inputs the current locally decoded picture K.

The blur prediction unit 22 uses the current locally decoded picture K as a reference picture and the past blur pictures B _{H, V, D} as reference pictures, and based on the reference picture and the reference picture, for each predetermined area, for example, a block matching method. The predicted image BY _{H, V, D} , the motion vector BV _{H, V, D} and the prediction error BE _{H, V, D} are obtained.

The blur prediction unit 22 outputs the prediction image BY _{H, V, D} , the motion vector BV _{H, V, D,} and the prediction error BE _{H, V, D} to the selection unit 16.

In this way, the inter-screen prediction processing unit 14 adds the prediction image KY, the motion vector KV, and the prediction error KE of the locally decoded picture, as well as the prediction images CY _{H, V, D} , and the motion vector CV _{H, V of the} super-resolution picture. _{, D} and the prediction error CE _{H, V, D} , and the prediction image BY _{H, V, D} of the blurred picture, the motion vector BV _{H, V, D} and the prediction error BE _{H, V, D} are generated.

[Frequency analysis unit 15]
Next, the frequency analysis unit 15 shown in FIG. 2 will be described in detail. FIG. 7 is a diagram illustrating a processing example of the frequency analysis unit 15.

The frequency analysis unit 15 reads the locally decoded picture K stored in the frame memory 13 as the past locally decoded picture K, and inputs the current locally decoded picture K.

The frequency analysis unit 15 performs frequency analysis on the current locally decoded picture K by first-order wavelet packet decomposition without decimation, and generates an analysis result of the current locally decoded picture K (step S701). As a result, the spatial low frequency band signal, the horizontal high frequency band signal, the vertical high frequency band signal, and the diagonal high frequency band signal of the current locally decoded picture K are generated.

Further, the frequency analysis unit 15 performs frequency analysis on the past locally decoded picture K by first-order wavelet packet decomposition without decimation, and generates an analysis result of the past locally decoded picture K (step S702). As a result, the spatial low frequency band signal, the horizontal high frequency band signal, the vertical high frequency band signal, and the diagonal high frequency band signal of the past locally decoded picture K are generated.

The frequency analysis unit 15 subtracts the analysis result of the past locally decoded picture K from the analysis result of the current locally decoded picture K for each frequency band and each phase position, and obtains the power fluctuation information for each frequency band and each phase position. Obtain (step S703).

Specifically, the frequency analysis unit 15 changes the element value (pixel value) of the phase position (pixel position) in the horizontal high-frequency band signal of the current locally decoded picture K to the horizontal high-frequency band signal of the past locally decoded picture K. Subtract the element values for the corresponding phase positions. Then, the frequency analysis unit 15 uses the subtraction result as power fluctuation information in the horizontal high frequency band.

Further, the frequency analysis unit 15 subtracts the element value of the corresponding phase position in the vertical high frequency band signal of the past locally decoded picture K from the element value of the phase position in the vertical high frequency band signal of the current locally decoded picture K. Then, the frequency analysis unit 15 uses the subtraction result as power fluctuation information in the vertical high frequency band.

Further, the frequency analysis unit 15 subtracts the element value of the corresponding phase position in the diagonal high frequency band signal of the past locally decoded picture K from the element value of the phase position in the diagonal high frequency band signal of the current locally decoded picture K. To do. Then, the frequency analysis unit 15 uses the subtraction result as power fluctuation information in the diagonal high frequency band.

The frequency analysis unit 15 obtains power fluctuation information of each phase position in the horizontal high frequency band, power fluctuation information of each phase position in the vertical high frequency band, and power fluctuation information of each phase position in the diagonal high frequency band in the frequency band. It is output to the selection unit 16 as power fluctuation information for each phase position.

In this way, the frequency analysis unit 15 generates power fluctuation information for each frequency band and each phase position.

The power fluctuation information is treated as an index indicating whether the current picture is blurry or sharper than the past pictures. When the amount of power fluctuation described later obtained from the power fluctuation information is larger than the first threshold value (positive value), the power of the current picture is larger than that of the past picture, so that the current picture is compared with the past picture. , It is judged that it is sharpening. If the power fluctuation amount is smaller than the first threshold value (negative value), it is determined that the current picture is blurred compared to the past picture because the power of the current picture is smaller than that of the past picture. Will be done. Further, when the power fluctuation amount is between the first threshold value and the second threshold value, the power of the current picture and the power of the past picture are almost the same, so that the current picture and the past picture are blurred. It is judged that the degree of sharpening and the degree of sharpening do not change so much. Details will be described later.

[Selection unit 16]
Next, the selection unit 16 shown in FIG. 2 will be described in detail. FIG. 8 is a block diagram showing a configuration example of the selection unit 16, and FIG. 9 is a flowchart showing a processing example of the selection unit 16.

The selection unit 16 includes a movement amount calculation unit 30, a power fluctuation amount calculation unit 31, a weight calculation unit 32, and a prediction image selection unit 33. Note that FIG. 8 shows only the components directly related to the present invention, and the components not directly related to the present invention are omitted.

The selection unit 16 is a prediction image KY, CY _{H, V, D} , BY _{H, V, D} , motion vector KV, CV _{H, V, D} , BV _{H, V for} each predetermined area from the inter-screen prediction processing unit 14. _{, D} and prediction error Enter KE, CE _{H, V, D} , BE _{H, V, D.} Further, the selection unit 16 inputs power fluctuation information for each frequency band and each phase position from the frequency analysis unit 15 (step S901).

(Movement amount calculation unit 30)
The motion amount calculation unit 30 is a motion amount CV _{H, V, D} -m which is a component in the horizontal direction, the vertical direction, and the diagonal direction from the motion vector CV _{H, V, D} of the super-resolution picture for each predetermined region. Calculate _{H, V, D.} Further, the motion amount calculation unit 30 uses the motion vectors BV _{H, V, D} of the blurry picture for each predetermined area to move the motion amount BV _{H, V,} of the blurry picture which is a component in the horizontal, vertical and diagonal directions _{. D-} m _{H, V, D} are calculated (step S902). Then, the movement amount calculation unit 30 outputs the movement amount CV _{H, V, D} −m _{H, V, D} , BV _{H, V, D} −m _{H, V, D} for each predetermined region to the weight calculation unit 32. ..

The amount of movement CV _{H, V, D} -m _{H, V, D} of the super-resolution picture is the amount of horizontal movement CV _H -m _H , the amount of vertical movement CV _H -m _V, and the amount of diagonal movement of the horizontal super-resolution picture. CV _H -m _D , horizontal movement amount CV _V -m _{H of} vertical super-resolution picture, vertical movement amount CV _V -m _V and diagonal movement amount CV _V -m _D , horizontal movement amount of diagonal super-resolution picture It consists of a movement amount CV _D -m _H , a vertical movement amount CV _D -m _V, and a diagonal movement amount CV _D -m _D.

The amount of movement BV _{H, V, D} -m _{H, V, D} of a blurred picture is the amount of horizontal movement BV _H -m _H , the amount of vertical movement BV _H -m _V, and the amount of diagonal movement BV _H -m of a horizontally blurred picture. _D and the horizontal movement amount BV _V -m _{H of} the vertically blurred picture, the vertical movement amount BV _V -m _V and the diagonal movement amount BV _V -m _D, and the horizontal movement amount BV _D -m _{H of} the diagonally blurred picture. It consists of a vertical movement amount BV _D -m _V and a diagonal movement amount BV _D -m _D.

The horizontal movement amount indicates the magnitude when the motion vector is projected in the horizontal direction, the vertical movement amount indicates the magnitude when the motion vector is projected in the vertical direction, and the diagonal movement amount indicates the magnitude when the motion vector is projected in the vertical direction. Shows the size when projected in the angular direction.

In this way, the movement amount calculation unit 30 determines the horizontal movement amount CV _H -m _H , the vertical movement amount CV _H -m _V, and the diagonal movement amount CV _H-for each predetermined region for the horizontal super-resolution picture C _H. m _D is calculated. Further, for the vertical super-resolution picture C _V , the horizontal movement amount CV _V -m _H , the vertical movement amount CV _V -m _V, and the diagonal movement amount CV _V -m _D are calculated, and the diagonal super-resolution picture C _D is calculated. The horizontal movement amount CV _D -m _H , the vertical movement amount CV _D -m _V, and the diagonal movement amount CV _D -m _D are calculated.

Further, the movement amount calculation unit 30 calculates the horizontal movement amount BV _H- m _H , the vertical movement amount BV _H- m _V, and the diagonal movement amount BV _H- m _D for each predetermined area for the horizontally blurred picture B _H. Will be done. Further, the horizontal movement amount BV _V -m _H , the vertical movement amount BV _V -m _V and the diagonal movement amount BV _V -m _D are calculated for the vertical blur picture B _V , and the horizontal movement for the diagonal blur picture B _D. The amount BV _D -m _H , the vertical movement amount BV _D -m _V, and the diagonal movement amount BV _D -m _D are calculated.

FIG. 10 is a diagram illustrating an example of the movement amount CV _H- m _{H, V, D} of the horizontal super-resolution picture calculated by the movement amount calculation unit 30. The picture at the time of frame t is used as the reference picture, and the picture at the time of frame t-1 is used as the reference picture. α1 indicates the horizontal direction in the motion vector CV _H of the horizontal super-resolution picture, β1 indicates the vertical direction in the motion vector CV _H of the horizontal super-resolution picture, and γ1 indicates the motion vector CV of the horizontal super-resolution picture. Indicates the diagonal direction at _H.

As shown in FIG. 10, when the motion amount calculation unit 30 projects the motion vector CV _H of the horizontal super-resolution picture with respect to the reference picture based on the reference picture in each predetermined region in the horizontal direction α1. The horizontal movement amount CV _H- m _H , the vertical movement amount CV _H -m _V when projected in the vertical direction β1, and the diagonal movement amount CV _H -m _D when projected in the diagonal direction γ1 are calculated.

(Power fluctuation amount calculation unit 31)
Returning to FIGS. 8 and 9, the power fluctuation amount calculation unit 31 uses the components of the horizontal high frequency band, the vertical high frequency band, and the diagonal high frequency band for each predetermined region from the power fluctuation information for each frequency band and each phase position. A certain power fluctuation amount pH _{, V, D} is calculated (step S903). Then, the power fluctuation amount calculation unit 31 outputs the power fluctuation amount pH _{, V, D} for each predetermined region to the weight calculation unit 32.

Specifically, the power fluctuation amount calculation unit 31 extracts the power fluctuation information of the region from the power fluctuation information of each frequency band and each phase position for each predetermined region. Then, the power change amount calculation unit 31 sums the element values in all phase positions of the horizontal high frequency band included in the power variation information of the area to determine the horizontal power fluctuation amount p _H of the area.

The power variation amount calculating unit 31 sums the element values in all phase positions of the vertical high-frequency band included in the power variation information of the area, determine the vertical power variation p _V of the region. Furthermore, the power change amount calculation unit 31 sums the element values of all the phase position of the diagonal high-frequency band included in the power variation information of the area, determine the diagonal power fluctuation amount p _D of the region.

Thus, the power fluctuation amount calculating section 31, for each predetermined region, a horizontal high-frequency lateral power variation amount is a component of the band p _H, vertical high-frequency vertical power variation amount is a component of the band p _V, and diagonal high-frequency band The diagonal power fluctuation amount p _D, which is a component of, is calculated.

FIG. 11 is a diagram illustrating an example of power fluctuation amounts pH _{, V, and D} calculated by the power fluctuation amount calculation unit 31. α2 indicates the direction of the horizontal high frequency band in the power fluctuation information, β2 indicates the direction of the vertical high frequency band in the power fluctuation information, and γ2 indicates the direction of the diagonal high frequency band in the power fluctuation information.

As shown in FIG. 11, the power fluctuation amount calculating section 31, from the power variation information of each frequency band and each phase position, for each predetermined region, horizontal power variation amount is a component in the direction α2 of the horizontal high frequency band p _H, The vertical power fluctuation amount p _V , which is a component of the direction β2 of the vertical high frequency band, and the diagonal power fluctuation amount p _D, which is a component of the direction γ2 of the diagonal high frequency band, are calculated.

(Weight calculation unit 32)
The weight calculation unit 32 inputs the movement amount CV _{H, V, D} -m _{H, V, D} , BV _{H, V, D} -m _{H, V, D} for each predetermined area from the movement amount calculation unit 30, and at the same time. The power fluctuation amount pH _{, V, D} for each predetermined area is input from the power fluctuation amount calculation unit 31. Further, the weight calculation unit 32 inputs preset intensity information cp _{H, V, D of} the super-resolution picture and intensity information bp _{H, V, D} of the blurred picture.

The intensity information cp _{H, V, D} of the super-resolution picture is the intensity information cp _H1 for horizontal used when the super-resolution picture C _{H, V, D} was generated by the super-resolution processing unit 11 shown in FIG. _{, V1, D1} , vertical strength information cp _{H2, V2, D2} and diagonal strength information cp _{H3, V3, D3} . Further, the intensity information bp _{H, V, D} of the blurred picture is the horizontal intensity information bp _{H1, V1, D1} used when the blurred picture B _{H, V, D} was generated by the blur processing unit 12 shown in FIG. , Corresponds to vertical strength information bp _{H2, V2, D2} and diagonal strength information bp _{H3, V3, D3} .

The weight calculation unit 32 uses the following formula for each predetermined region for the super-resolution pictures C _{H, V, D} to move the super-resolution pictures CV _{H, V, D} -m _{H, V, D} and Super-resolution picture C _{H, V, D} based on super-resolution picture intensity information cp _{H, V, D} (cp _{H1, V1, D1} , cp _{H2, V2, D2} , cp _{H3, V3, D3} ) The weights tC _{H, V, and D} are calculated for each (step S904).
[Number 1]
_{tC H = d / {(a} × cp H1 × (CV H -m H) + b × cp V1 × (CV H -m V) + c × cp D1 × (CV H -m D)} ··· (1)
[Number 2]
_{tC V = d / {(a} × cp H2 × (CV V -m H) + b × cp V2 × (CV V -m V) + c × cp D2 × (CV V -m D)} ··· (2)
[Number 3]
_{tC D = d / {(a} × cp H3 × (CV D -m H) + b × cp V3 × (CV D -m V) + c × cp D3 × (CV D -m D)} ··· (3)

The weight tC _H indicates the weight of the horizontal super-resolution picture C _H , the weight tC _V indicates the weight of the vertical super-resolution picture C _V , and the weight tC _D indicates the weight of the diagonal super-resolution picture C _D. Shown. The parameters a, b, c, and d indicate preset constants.

In the above equation (1), for the horizontal super-resolution picture C _H , for example, horizontal intensity cp _H1 = 1.2, vertical intensity cp _V1 = 1.0, and diagonal intensity cp _D1 = 1.0 are used as described above.

In the above equation (2), for the vertical super-resolution picture C _V , for example, horizontal intensity cp _H2 = 1.0, vertical intensity cp _V2 = 1.2, and diagonal intensity cp _D2 = 1.0 are used as described above.

In the formula (3), the diagonal super-resolution picture C _D, as described above, for example, a horizontal intensity cp _H3 = 1.0, vertical strength cp _V3 = 1.0 and diagonal intensity cp _D3 = 1.2 is used.

As described above, the weights tC _{H, V, D} shown in the above equations (1), (2) and (3) are calculated based on the movement amount CV _{H, V, D} -m _{H, V, D} of the super-resolution picture. The larger the movement amount CV _{H, V, D} -m _{H, V, D} , the smaller the value, and the smaller the movement amount CV _{H, V, D} -m _{H, V, D} , the larger the value. In this case, as the amount of movement CV _{H, V, D} -m _{H, V, D} is larger, the reference picture is more likely to be blurred than the reference picture, so that the reference picture is sharper than the reference picture. Probability is high.

Therefore, the larger the amount of movement CV _{H, V, D} -m _{H, V, D} (horizontal, vertical, and diagonal directions), the smaller the weights tC _{H, V, D} will be, which will be described later. In step S907, the prediction error CE _{H, V, D} after weight multiplication is set to a small value, and the predicted images CY _{H, V, D of} the super-resolution picture corresponding to the direction are selected. That is, assuming that the larger the amount of movement CV _{H, V, D} -m _{H, V, D} from the reference picture to the reference picture, the greater the blurring, the predicted image of the super-resolution picture in the direction of strong sharpening. CY _{H, V, D} are selected.

In step S904, the weight calculation unit 32 uses the super-resolution pictures C _{H, V, and D} to change the power fluctuation amount instead of the movement amount CV _{H, V, D} -m _{H, V, D} of the super-resolution picture. PH _{, V, D} may be used. That is, the weight calculation unit 32 uses the following equation for each predetermined region to calculate the power fluctuation amount pH _{, V, D} and the intensity information of the super-resolution picture cp _{H, V, D} (cp _{H1, V1, D1} , _D1) . The weights tC _{H, V, D} may be calculated for each of the super-resolution pictures C _{H, V, D} based on cp _{H2, V2, D2} , cp _{H3, V3, D3} ).
[Number 4]
tC _H = h / (e × cp _H1 × p _H + f × cp _V1 × p _V + g × cp _D1 × p _D ) ・ ・ ・ (4)
[Number 5]
tC _V = h / (e × cp _H2 × p _H + f × cp _V2 × p _V + g × cp _D2 × p _D ) ・ ・ ・ (5)
[Number 6]
tC _D = h / (e × cp _H3 × p _H + f × cp _V3 × p _V + g × cp _D3 × p _D ) ・ ・ ・ (6)

The parameters e, f, g, and h indicate preset constants.

In the above equation (4), for example, the horizontal intensity cp _H1 = 1.2, the vertical intensity cp _V1 = 1.0, and the diagonal intensity cp _D1 = 1.0 for the horizontal super-resolution picture C _H are used as in the above equation (1). ..

In the equation (5), similarly to the equation (2), for example, the horizontal intensity cp _H2 = 1.0, the vertical intensity cp _V2 = 1.2 and the diagonal intensity cp _D2 = 1.0 for the vertical super-resolution picture _CV are used. ..

In the formula (6), similarly to the equation (3), the diagonal super-resolution picture C _D horizontal intensity cp _H3 = 1.0 for vertical strength cp _V3 = 1.0 and diagonal intensity cp _D3 = 1.2 is used ..

Thus, the equation (4) (5) weight tC H shown in _{(6), V, D,} the power fluctuation amount p _{H, V,} is calculated based on _D, the power fluctuation amount p _{H, V, D} The larger the value in the positive direction, the smaller the value, and the smaller the power fluctuation amounts pH _{, V, D} in the positive direction, the larger the value. In this case, the larger the power fluctuation amount pH _{, V, D} is in the positive direction, the more likely the reference picture is blurred than the reference picture. Therefore, the reference picture may be sharper than the reference picture. high.

Therefore, in the direction in which the power fluctuation amounts pH _{, V, D} are larger in the positive direction (horizontal direction, vertical direction, and diagonal direction), the weights tC _{H, V, D} are set to smaller values, so that in step S907 described later. , The prediction error CE _{H, V, D} after weight multiplication is set to a small value, and the prediction images CY _{H, V, D of} the super-resolution picture corresponding to the direction are selected. That is, assuming that the larger the power fluctuation amount pH _{, V, D} from the reference picture to the reference picture in the positive direction, the greater the blurring, the predicted image CY _H, of the super-resolution picture in the direction of strong sharpening _{. V and D} are selected.

Further, in step S904, the weight calculation unit 32 uses the super-resolution picture C _{H, V, D} instead of the movement amount CV _{H, V, D} -m _{H, V, D} to move the super-resolution picture. CV _{H, V, D} −m _{H, V, D} and power fluctuations _{pH, V, D} may be used. That is, the weight calculator 32, for each predetermined region, by the following equation, the motion amount of the super-resolution picture _{CV H, V, D -m H} , V, D, power fluctuation amount p _{H, V, D} and Super-resolution picture C _{H, V, D} based on super-resolution picture intensity information cp _{H, V, D} (cp _{H1, V1, D1} , cp _{H2, V2, D2} , cp _{H3, V3, D3} ) The weights tC _{H, V, and D} may be calculated for each.
[Number 7]
tC _H = o / _{[{(i × cp H1 × (} CV H -m H) + j × cp V1 × (CV H -m V) + k × cp D1 × (CV H -m D)} + (l × cp _H1 x p _H + m x cp _V1 x p _V + n x cp _D1 x p _D )]
... (7)
[Number 8]
tC _V = o / _{[{(i × cp H2 × (} CV V -m H) + j × cp V2 × (CV V -m V) + k × cp D2 × (CV V -m D)} + (l × cp _H2 x p _H + m x cp _V2 x p _V + n x cp _D2 x p _D )]
... (8)
[Number 9]
tC _D = o / _{[{(i × cp H3 × (} CV D -m H) + j × cp V3 × (CV D -m V) + k × cp D3 × (CV D -m D)} + (l × cp _H3 x p _H + m x cp _V3 x p _V + n x cp _D3 x p _D )]
... (9)

The parameters i, j, k, l, m, n, o indicate preset constants.

In the above equation (7), similarly to the above equation (1), for example, the horizontal intensity cp _H1 = 1.2, the vertical intensity cp _V1 = 1.0, and the diagonal intensity cp _D1 = 1.0 for the horizontal super-resolution picture C _H are used. ..

Further, in the above equation (8), similarly to the above equation (2), for example, the horizontal intensity cp _H2 = 1.0, the vertical intensity cp _V2 = 1.2 and the diagonal intensity cp _D2 = 1.0 for the vertical super-resolution picture _CV. Used.

Further, in the formula (9), similarly to the equation (3), the horizontal intensity for diagonal super-resolution picture C _D cp _H3 = 1.0, vertical strength cp _V3 = 1.0 and diagonal intensity cp _D3 = 1.2 Used.

The weight calculation unit 32 shifts from step S904, and for the blurred pictures B _{H, V, D} , the power fluctuation amount pH _{, V, D} and the intensity information bp of the blurred picture are expressed by the following formulas for each predetermined area. _{Based on H, V, D} (bp _{H1, V1, D1} , bp _{H2, V2, D2} , bp _{H3, V3, D3} ), the weights tB _{H, V, D} for each of the blurred pictures B _{H, V, D} Calculate (step S905).
[Number 10]
tB _H = s / (p × bp _H1 × p _H + q × bp _V1 × p _V + r × bp _D1 × p _D ) ・ ・ ・ (10)
[Number 11]
tB _V = s / (p × bp _H2 × p _H + q × bp _V2 × p _V + r × bp _D2 × p _D ) ・ ・ ・ (11)
[Number 12]
_{tB D = s / (p ×} bp H3 × p H + q × bp V3 × p V + r × bp D3 × p D) ··· (12)

The weight tB _H indicates the weight of the horizontally blurred picture B _H , the weight tB _V indicates the weight of the vertically blurred picture B _V , and the weight tB _D indicates the weight of the horizontally blurred picture B _D. The parameters p, q, r, s indicate preset constants.

In the above equation (10), for the horizontally blurred picture B _H , for example, horizontal intensity bp _H1 = 0.8, vertical intensity bp _V1 = 1.0, and diagonal intensity bp _D1 = 1.0 are used as described above.

In the above equation (11), for the vertical blur picture B _V , for example, horizontal intensity bp _H2 = 1.0, vertical intensity bp _V2 = 0.8, and diagonal intensity bp _D2 = 1.0 are used as described above.

In the above equation (12), for the diagonally blurred picture _BD , for example, horizontal intensity bp _H3 = 1.0, vertical intensity bp _V3 = 1.0, and diagonal intensity bp _D3 = 0.8 are used as described above.

Thus, the equation (10) (11) (12) weight shown in tB _{H, V, D,} the power fluctuation amount p _{H, V,} is calculated based on _D, the power fluctuation amount p _{H, V, D} The larger the value in the negative direction, the smaller the absolute value, and the smaller the power fluctuation amounts pH _{, V, D} in the negative direction, the larger the absolute value. In this case, the larger the power fluctuation amount pH _{, V, D} is in the negative direction, the more likely the reference picture is sharper than the reference picture, so that the reference picture may be blurred with respect to the reference picture. high.

Therefore, the absolute value of the weights tB _{H, V, D} is set to a smaller value in the direction in which the power fluctuation amount pH _{, V, D} has a larger value in the negative direction (horizontal direction, vertical direction, and diagonal direction). In step S907, which will be described later, the prediction error BE _{H, V, D} after weight multiplication is set to a small value, and the predicted images BY _{H, V, D of} the blurred picture corresponding to the direction are selected. That is, assuming that the sharpening is greater in the direction in which the power fluctuation amount pH _{, V, D} from the reference picture to the reference picture has a larger negative value, the predicted image BY _H, of the blur picture in the direction of strong blur _{. V and D} are selected.

The weight calculation unit 32, blurred picture B _{H, V,} per _D, in step S905, the power fluctuation amount p _{H, V,} the amount of motion blur picture instead of _{D BV H, V, D -m} H, V _{, D} and power fluctuations _{pH, V, D} may be used. That is, the weight calculation unit 32 uses the following formula for each predetermined area to determine the movement amount BV _{H, V, D} −m _{H, V, D} , the power fluctuation amount pH _{, V, D} and the blur picture. Based on the intensity information bp _{H, V, D} (bp _{H1, V1, D1} , bp _{H2, V2, D2} , bp _{H3, V3, D3} ), the weights tB _{H, for} each of the blurred pictures B _{H, V, D. V and D} may be calculated.
[Number 13]
tB _H = z / _{[{(t × bp H1 × (} BV H -m H) + u × bp V1 × (BV H -m V) + v × bp D1 × (BV H -m D)} + (w × bp _H1 x p _H + x x bp _V1 x p _V + y x bp _D1 x p _D )]
... (13)
[Number 14]
tB _V = z / _{[{(t × bp H2 × (} BV V -m H) + u × bp V2 × (BV V -m V) + v × bp D2 × (BV V -m D)} + (w × bp _H2 x p _H + x x bp _V2 x p _V + y x bp _D2 x p _D )]
... (14)
[Number 15]
tB _D = z / _{[{(t × bp H3 × (} BV D -m H) + u × bp V3 × (BV D -m V) + v × bp D3 × (BV D -m D)} + (w × bp _H3 x p _H + x x bp _V3 x p _V + y x bp _D3 x p _D )]
... (15)

The parameters t, u, v, w, x, y, z indicate preset constants.

In the equation (13), similarly to the equation (10), for example, the horizontal intensity bp _H1 = 0.8, the vertical intensity bp _V1 = 1.0, and the diagonal intensity bp _D1 = 1.0 for the horizontally blurred picture B _H are used.

Further, in the equation (14), similarly to the equation (11), for example, the horizontal intensity bp _H2 = 1.0, the vertical intensity bp _V2 = 0.8, and the diagonal intensity bp _D2 = 1.0 for the vertical blur picture B _V are used. ..

Further, in the equation (15), similarly to the equation (12), the horizontal intensity bp _H3 = 1.0, the vertical intensity bp _V3 = 1.0, and the diagonal intensity bp _D3 = 0.8 for the diagonally blurred picture _BD are used. ..

The weight calculation unit 32 selects the predicted image selection of the super-resolution picture weights tC _{H, V, D} calculated in step S904 and the blur picture weights tB _{H, V, D} calculated in step S905. Output to unit 33.

(Prediction image selection unit 33)
The prediction image selection unit 33 inputs the weights tC _{H, V, D} of the super-resolution picture and the weights tB _{H, V, D} of the blurred picture for each predetermined area from the weight calculation unit 32. Further, the prediction image selection unit 33, from the inter-screen prediction processing unit 14, predicts images KY, CY _{H, V, D} , BY _{H, V, D} and prediction errors KE, CE _{H, V, D for} each predetermined area. Enter BE _{H, V, D.}

The prediction image selection unit 33 shifts from step S905 and has a weight tC corresponding to the prediction error CE _{H, V, D} of the super-resolution picture for each predetermined region for the super-resolution pictures C _{H, V, D.} Multiply _{H, V, D.} Further, the prediction image selection unit 33 multiplies the prediction error BE _{H, V, D} of the blurry picture by the corresponding weights tB _{H, V, D} for each of the blurry pictures B _{H, V, D} for each predetermined area ( Step S906).

As a result, the prediction error CE _{H, V, D} after the weight multiplication of the super-resolution picture and the prediction error BE _{H, V, D} after the weight multiplication of the blurred picture can be obtained.

The prediction image selection unit 33 determines the prediction error KE of the locally decoded picture, the prediction error CE _{H, V, D} after weight multiplication of the super-resolution picture, and the prediction error BE _H after weight multiplication of the blurred picture for each predetermined area. _{, V, D} , identify the smallest prediction error. Then, the prediction image selection unit 33 selects the prediction image corresponding to the minimum prediction error (step S907). As a result, any one of the predicted images KY, CY _{H, V, D} , BY _{H, V, and D} of the seven types of pictures is selected.

The prediction image selection unit 33 outputs the selected prediction image to the switch 119 as an inter-screen prediction result for each predetermined area (step S908).

For example, for a region, the prediction error CE _H after weight multiplication of the horizontal super-resolution picture is the prediction error KE of the locally decoded picture, the prediction error CE _{H, V, D} after weight multiplication of the super-resolution picture, and blurring. It is assumed that the prediction error after weight multiplication of the picture is the minimum value among BE _{H, V, and D.} In this case, the prediction image selection unit 33 has the predicted image KY of the locally decoded picture, the predicted image CY _{H, V, D of} the super-resolution picture, and the predicted image BY _{H, V, D} of the blurred picture for the region. , Select the predicted image CY _H of the horizontal super-resolution picture. Then, the prediction image selection unit 33 outputs the prediction image CY _H of the horizontal super-resolution picture to the switch 119 as the inter-screen prediction result for the region.

As described above, according to the moving image coding device 1 of the embodiment of the present invention, the super-resolution processing unit 11 of the inter-screen prediction unit 10 performs super-resolution processing on the local decoding picture K to perform three types of super-resolution processing. The super-resolution pictures C _{H, V, and D} are generated and stored in the frame memory 13. The blur processing unit 12 performs blur processing on the locally decoded picture K, generates three types of blur pictures B _{H, V, and D} , and stores them in the frame memory 13.

The inter-screen prediction processing unit 14 includes a reference picture which is the current locally decoded picture K, and past locally decoded pictures K, super-resolution pictures C _{H, V, D} and blurred pictures B _{H, V} read from the frame memory 13. Based on the reference picture _, which is _D , the motion vector KV, CV _{H, V, D} , BV _{H, V, D} , and the predicted image KY, CY _{H, V, D} are predicted for each predetermined area. , BY _{H, V, D} and prediction error KE, CE _{H, V, D} , BE _{H, V, D.}

The frequency analysis unit 15 performs frequency analysis on the current locally decoded picture K and the past locally decoded picture K, subtracts the analysis result of the past locally decoded picture K from the analysis result of the current locally decoded picture K, and performs frequency band. Obtain power fluctuation information for each phase position.

The selection unit 16 sets the weight tC _H, of the super-resolution picture based on the motion vector CV _{H, V, D} of the super-resolution picture and the intensity information cp _{H, V, D} of the super-resolution picture for each predetermined region _. Calculate _{V and D.} Further, the selection unit 16 calculates the weights tB _{H, V, D} of the blurred picture based on the power fluctuation information and the intensity information bp _{H, V, D} of the blurred picture for each predetermined area.

The selection unit 16 multiplies the prediction error CE _{H, V, D} of the super-resolution picture by the corresponding weights tC _{H, V, D} for each predetermined region to obtain the prediction error BE _{H, V, D} of the blurred picture. , Multiply the corresponding weights tB _{H, V, D.} The selection unit 16 has a prediction error KE of the locally decoded picture, a prediction error CE _{H, V, D} after weight multiplication of the super-resolution picture, and a prediction error BE _{H, V} after weight multiplication of the blurred picture for each predetermined area. Of _{, D} , the predicted image corresponding to the smallest prediction error is selected.

In this way, in addition to the locally decoded picture K, three types of super-resolution pictures C _{H, V, D} and three types of blurry pictures B _{H, V, D} are generated as reference pictures used in the inter-screen prediction processing. .. Then, the power fluctuation amount, the movement amount, and the like of the picture are taken into consideration for each predetermined area, and the optimum picture considering the blurring and sharpening, and these directions is selected as the predicted image.

As a result, when the object is blurred from the past picture to the current picture, the blurred picture B is selected as the predicted image, and when the object is sharpened, the super-resolution picture C is selected. Therefore, the residual between the input moving image and the predicted image can be reduced.

In the prior art, it was possible to objectively obtain a high-quality moving image by performing inter-screen prediction using only the locally decoded picture K as a reference picture. However, in the prior art, it is not possible to perform inter-screen prediction that distinguishes between a moving image in which blurring or sharpening occurs and a normal moving image (moving image in which blurring or sharpening does not occur). In addition, it is not possible to perform inter-screen prediction that distinguishes between a moving image in which blurring or sharpening occurs due to horizontal movement and a moving image in which blurring or sharpening occurs due to vertical movement.

In the embodiment of the present invention, blurring and sharpening, and inter-screen prediction in consideration of these directions are performed. Therefore, even when the object is blurred or sharpened for each area between the pictures of the moving image, the inter-screen prediction efficiency can be improved and the moving image of high quality can be subjectively obtained. It becomes.

[Video decoding device]
Next, the moving image decoding device according to the embodiment of the present invention will be described. FIG. 12 is a block diagram showing a configuration example of a moving image decoding device according to an embodiment of the present invention. The moving image decoding device 2 includes an entropy decoding unit 50, an inverse quantization unit 51, an inverse orthogonal conversion unit 52, an addition unit 53, an in-screen prediction unit 54, an in-loop filter 55, an inter-screen prediction unit 56, and a switch 57. ing. Note that FIG. 12 shows only the components directly related to the present invention, and the components not directly related to the present invention are omitted.

The entropy decoding unit 50 inputs the coded signal output from the moving image coding device 1 shown in FIG. 1 and performs the reverse processing of the entropy coding unit 120 shown in FIG. 1, thereby performing the reverse processing of the coded signal. Is entropy-decoded to generate a quantized index string and parameters.

The entropy decoding unit 50 outputs the quantization index sequence to the inverse quantization unit 51, outputs the parameters including the type of the predicted image selected for each predetermined region and the calculated motion vector to the interscreen prediction unit 56, and outputs the parameters to the interscreen prediction unit 56. , Other parameters are output to the inverse quantization unit 51 and the like.

The inverse quantization unit 51 inputs the quantization index sequence from the entropy decoding unit 50 and performs the reverse processing of the quantization unit 112 shown in FIG. 1 to inversely quantize the quantization index sequence and convert coefficient sequence. To generate. Then, the inverse quantization unit 51 outputs the conversion coefficient sequence to the inverse orthogonal conversion unit 52.

The inverse orthogonal conversion unit 52 inputs the conversion coefficient sequence from the inverse quantization unit 51 and performs the reverse processing of the orthogonal conversion unit 111 shown in FIG. 1 to perform the inverse orthogonal conversion of the conversion coefficient sequence and the decoding residual. Generate an image. Then, the inverse orthogonal conversion unit 52 outputs the decoding residual image to the addition unit 53.

The addition unit 53 inputs the decoding residual image from the inverse orthogonal conversion unit 52, and inputs the predicted image from the switch 57. Then, the addition unit 53 adds the decoding residual image to the prediction image, and outputs the added image to the in-screen prediction unit 54 and the in-loop filter 55.

The in-screen prediction unit 54 inputs the image after addition from the addition unit 53, and performs the same processing as the in-screen prediction unit 116 shown in FIG. 1, so that the image after addition is displayed in the screen for each predetermined area. Prediction processing is performed, and the in-screen prediction result for each predetermined area is output to the switch 57. Further, the in-screen prediction unit 54 outputs the added image as a decoded image. As a result, the original moving image is restored.

The in-loop filter 55 inputs the image after addition from the addition unit 53, and performs the same processing as the in-loop filter 117 shown in FIG. 1 to perform filter processing on the image after addition, and decode picture K'. To generate. Then, the in-loop filter 55 outputs the decoded picture K'to the inter-screen prediction unit 56.

The inter-screen prediction unit 56 inputs the decoded picture K'from the in-loop filter 55, and also inputs the parameter including the type of the predicted image selected for each predetermined area and the calculated motion vector from the entropy decoding unit 50. .. Then, the inter-screen prediction unit 56 performs a predetermined inter-screen prediction process based on the parameters for each predetermined area, so that the prediction image KY of the locally decoded picture ′'and the prediction image CY _{H of the} three types of super-resolution pictures. _{, V, D} '(horizontal prediction image CY _H of the super-resolution picture', 'predicted image CY _D and of diagonal super-resolution picture' prediction image CY _V vertical super-resolution pictures), and 3 types of blurring the picture Prediction of any one of the prediction images BY _{H, V, D} '(horizontal blur picture prediction image BY _H ', vertical blur picture prediction image BY _V ', and diagonal blur picture prediction image BY _D '). An image is generated, and this is output to the switch 57 as an inter-screen prediction result for each predetermined area. Further, the inter-screen prediction unit 56 outputs the decoded picture K'as a decoded image. As a result, the original moving image is restored. The details of the inter-screen prediction unit 56 will be described later.

The switch 57 inputs the in-screen prediction result for each predetermined area from the in-screen prediction unit 54, and also inputs the in-screen prediction result for each predetermined area from the inter-screen prediction unit 56, selects one of them, and predicts the image. Is output to the addition unit 53.

[Inter-screen prediction unit 56]
Next, the inter-screen prediction unit 56 shown in FIG. 12 will be described in detail. FIG. 13 is a block diagram showing a configuration example of the inter-screen prediction unit 56 provided in the moving image decoding device 2. The inter-screen prediction unit 56 includes a super-resolution processing unit 60, a blur processing unit 61, a frame memory 62, an inter-screen prediction processing unit 63, and a parameter processing unit 64. Note that FIG. 13 shows only the components directly related to the present invention, and the components not directly related to the present invention are omitted.

The inter-screen prediction unit 56 inputs the decoded picture K'from the in-loop filter 55, and stores the decoded picture K'in the frame memory 62. Further, the parameter processing unit 64 of the inter-screen prediction unit 56 inputs parameters including the type of the predicted image selected for each predetermined area and the calculated motion vector from the entropy decoding unit 50, and performs inter-screen prediction processing of the parameters. Output to unit 63. The type of the predicted image is one of three types of super-resolution pictures C _{H, V, D} and three types of blurry pictures B _{H, V, D} (out of seven types of pictures). Any one of them) is shown.

The super-resolution processing unit 60 performs the same super-resolution processing as the super-resolution processing unit 11 shown in FIG. 2 on the decoded picture K', and three types of super-resolution pictures C _{H, V, D} '. (Horizontal super-resolution picture C _H ', vertical super-resolution picture C _V ', and diagonal super-resolution picture C _D ') are generated. The super-resolution processing unit 60 stores three types of super-resolution pictures C _{H, V, D'} in the frame memory 62.

The blur processing unit 61 performs the same blur processing as the blur processing unit 12 shown in FIG. 2 on the decoded picture K', and blur pictures B _{H, V, D} '(horizontal blur picture B _H ', vertical blur picture. B _V 'and diagonal blurred picture B _D') to produce a. The blur processing unit 61 stores three types of blur pictures B _{H, V, D'} in the frame memory 62.

The frame memory 62 stores the decoded picture K', three types of super-resolution pictures C _{H, V, D'} and three types of blurry pictures B _{H, V, D'for} each frame.

The inter-screen prediction processing unit 63 inputs parameters from the parameter processing unit 64, and when the type of the predicted image included in the parameters indicates the locally decoded picture K, the decoded picture K'stored in the frame memory 62 is stored in the past. Read as the decoded picture K'of.

Further, the inter-screen prediction processing unit 63 indicates that the type of the predicted image included in the parameter is one of the horizontal super-resolution picture C _H , the vertical super-resolution picture C _V, and the diagonal super-resolution picture C _D. In the case, the corresponding super-resolution picture C'(horizontal super-resolution picture C _H ', vertical super-resolution picture C _V ', and diagonal super-resolution picture corresponding to the type of predicted image) stored in the frame memory 62. 'any one of the picture) in the past of the super-resolution picture C' C _D read out as.

Further, when the type of the predicted image included in the parameter indicates any of the horizontal blur picture B _H , the vertical blur picture B _V, and the diagonal blur picture B _D , the inter-screen prediction processing unit 63 stores the frame memory 62. The stored corresponding blur picture B'(one of the horizontal blur picture B _H ', the vertical blur picture B _V ', and the diagonal blur picture B _D' corresponding to the type of predicted image), Read as a past blurred picture B'.

The inter-screen prediction processing unit 63 performs inter-screen prediction based on the past decoded picture K', super-resolution picture C'or blur picture B', and motion vectors included in the parameters, and predicts each predetermined area. Ask for an image. Similar to FIG. 2, the predetermined region is, for example, a slice region, a CTU (Coding Tree Unit) region, and a CU (Coding Unit) region.

The inter-screen prediction processing unit 63 outputs a prediction image for each predetermined area to the switch 57 as an inter-screen prediction result.

Further, a reading unit (not shown) reads the decoded picture K'from the frame memory 62 and outputs this as the decoded image O.

As described above, according to the video decoding apparatus 2 of the embodiment of the present invention, super-resolution processing unit 60 of the inter prediction unit 56, the three super-resolution picture C _H based on the decoded picture K ' _{, V, D'} are generated and stored in the frame memory 62. The blur processing unit 61 generates three types of blur pictures B _{H, V, D'} based on the decoded picture K'and stores them in the frame memory 62.

The inter-screen prediction processing unit 63 determines from the frame memory 62 the decoded picture K', the super-resolution picture C'(horizontal super-resolution picture C _H ', the vertical super-resolution picture _CV ', according to the type of the predicted image included in the parameter. 'and diagonal super-resolution picture C _D' of any one of the picture) or blurred picture B '(horizontal blur picture B _H', vertical blur picture B _V 'and diagonal blurred picture B _D' of the Any one picture) is read out, and a predicted image is obtained for each predetermined area based on the read out picture and the motion vector included in the parameter.

As described above, in the moving image coding apparatus 1, as the reference picture used for the inter-screen prediction processing, the locally decoded picture K, the three types of super-resolution pictures C _{H, V, D} and the three types of blurred pictures B _{H, V , D} are generated, the power fluctuation amount, the movement amount, etc. of the picture are taken into consideration in each area, and when the optimum picture is selected as the predicted image, the moving image decoding device 2 inputs from the moving image coding device 1. A predicted image of the same type as the predicted image selected by the moving image coding device 1 is generated according to the type of the predicted image included in the parameters, and a decoded image is generated.

As a result, when the object is blurred from the past picture to the current picture, the blurred picture B'is selected, and when the object is sharpened, the super-resolution picture C'is selected and the predicted image is generated. Will be done. That is, a prediction image is generated in consideration of blurring and sharpening, and these directions.

Therefore, even when the object is blurred or sharpened for each area between the pictures of the moving image, the inter-screen prediction efficiency can be improved and the moving image of high quality can be subjectively obtained. It becomes.

Although the present invention has been described above with reference to embodiments, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the technical idea.

(Example of providing either one of the super-resolution processing units 11, 60 and the blur processing units 12, 61)
In the above embodiment, as shown in FIG. 2, the inter-screen prediction unit 10 of the moving image coding device 1 includes a super-resolution processing unit 11 and a blurring processing unit 12. Further, as shown in FIG. 13, the inter-screen prediction unit 56 of the moving image decoding device 2 is provided with a super-resolution processing unit 60 and a blurring processing unit 61.

On the other hand, the inter-screen prediction unit 10 may include either one of the super-resolution processing unit 11 and the blurring processing unit 12. Further, the inter-screen prediction unit 56 may include either one of the super-resolution processing unit 60 and the blur processing unit 61 in response to the inter-screen prediction unit 10.

When the inter-screen prediction processing unit 10 of the motion image coding device 1 includes the super-resolution processing unit 11 and does not include the blurring processing unit 12, the inter-screen prediction processing unit 14 of the inter-screen prediction unit 10 includes a reference picture and a reference picture. , The inter-screen prediction is performed based on the reference picture which is the past locally decoded picture K and the super-resolution picture C _{H, V, D} , and the motion vector KV, CV _{H, V, D} , and the predicted image KY are performed for each predetermined area. , CY _{H, V, D} and prediction error KE, CE _{H, V, D.}

The selection unit 16 performs the same processing as described above for each predetermined area, and has the smallest prediction error among the prediction error KE of the locally decoded picture and the prediction error CE _{H, V, D} after weight multiplication of the super-resolution picture. Select the predicted image corresponding to.

When the inter-screen prediction processing unit 56 of the moving image decoding device 2 includes the super-resolution processing unit 60 and does not include the blurring processing unit 61, the inter-screen prediction processing unit 63 of the inter-screen prediction unit 56 includes prediction included in the parameters. According to the type of image, any one of the decoded picture K'and the super-resolution picture C'is read from the frame memory 62, and the read picture and the motion vector included in the parameter are used for each predetermined area. Obtain a predicted image.

On the other hand, when the inter-screen prediction processing unit 10 of the motion image coding device 1 includes the blurring processing unit 12 and does not include the super-resolution processing unit 11, the inter-screen prediction processing unit 14 of the inter-screen prediction unit 10 is a reference picture. , And, based on the reference picture which is the past locally decoded picture K and the blurred picture B _{H, V, D} , the inter-screen prediction is performed, and the motion vectors KV, BV _{H, V, D} are obtained and predicted for each predetermined area. Find the images KY, BY _{H, V, D} and the prediction errors KE, BE _{H, V, D.}

The selection unit 16 corresponds to the smallest prediction error among the prediction error KE of the locally decoded picture and the prediction error BE _{H, V, D} after weight multiplication of the blurred picture by the same processing as described above for each predetermined area. Select the predicted image to be used.

When the inter-screen prediction processing unit 56 of the moving image decoding device 2 includes the blurring processing unit 61 and does not include the super-resolution processing unit 60, the inter-screen prediction processing unit 63 of the inter-screen prediction unit 56 includes prediction included in the parameters. According to the type of image, any one of the decoded picture K'and the blurred picture B'is read from the frame memory 62, and the predicted image is predicted for each predetermined area based on the read picture and the motion vector included in the parameter. To ask.

(Other selection processing of predicted image)
Further, in the above-described embodiment, the selection unit 16 provided in the inter-screen prediction unit 10 of the moving image coding device 1 has weights tC _{H, V, D of} super-resolution pictures and weights tB of blurry pictures for each predetermined area. _{H, V,} calculates _D, the prediction error KE of the local decoded picture, the weight tC H super-resolution _{picture, V, D} prediction after multiplying the error CE _{H, V, D} and the weight of the blurred picture tB _H, was set to _V, the prediction error after multiplying _D bE _{H, V,} of the _D, and selects a prediction image corresponding to the smallest prediction error.

On the other hand, the selection unit 16 may first determine the type of the predicted image to be selected, and select the predicted image among the types. The type of the predicted image is any one of the locally decoded picture K, the super-resolution picture C, and the blurred picture B (any one of the three types).

More specifically, the selection unit 16, the power fluctuation amount p _H calculated from the power variation _{information, V,} using a _D, and each predetermined area, the lateral power changing amount p _H, vertical power variation p _V and pairs find the total or average of the square power fluctuation amount p _D as a power fluctuation amount, performs threshold processing of the power fluctuation amount.

When the selection unit 16 determines that the power fluctuation amount is larger than the first threshold value (a preset positive value), the selection unit 16 determines the super-resolution picture C as the type of the predicted image. This is because when the power fluctuation amount is larger than the first threshold value, the power of the current picture is larger than that of the past image, so that it can be determined that the current image is sharper than the past image. In this case, the selection unit 16 corresponds to the smallest prediction error among the prediction errors CE _{H, V, D} after multiplying the super-resolution picture weights tC _{H, V, D} by the same processing as described above. Select the predicted image to be used.

Further, when the selection unit 16 determines that the power fluctuation amount is between the first threshold value and the second threshold value (a preset negative value), the selection unit 16 determines the locally decoded picture K as the type of the predicted image. To do. When the power fluctuation amount is between the first threshold value and the second threshold value, the power of the current picture and the power of the past picture are almost the same, so that the current image is compared with the past image. This is because it can be judged that the image is not blurred and is not sharpened. In this case, the selection unit 16 selects the locally decoded picture K as the predicted image.

Further, when the selection unit 16 determines that the power fluctuation amount is smaller than the second threshold value, the selection unit 16 determines the blurred picture B as the type of the predicted image. This is because when the power fluctuation amount is smaller than the second threshold value, the power of the current picture is smaller than that of the past image, so that it can be determined that the current image is blurred as compared with the past image. In this case, the selection unit 16 performs the same processing as described above to perform the prediction corresponding to the smallest prediction error among the prediction errors BE _{H, V, D} after multiplying the blur picture weights tB _{H, V, D.} Select an image.

Further, the embodiment of the present invention is applicable, for example, when an object moving in and out of the depth of field between pictures is included, and when the object changes from a stationary state to an operating state and vice versa. is there. For example, it is applicable when the camera suddenly pans or tilts, when these operations stop, when the object moves from the back to the front on the screen, or vice versa.

As the hardware configuration of the moving image coding device 1 and the moving image decoding device 2 according to the embodiment of the present invention, a normal computer can be used. The moving image coding device 1 and the moving image decoding device 2 are composed of a computer provided with a volatile storage medium such as a CPU and RAM, a non-volatile storage medium such as a ROM, and an interface.

Subtraction unit 110, orthogonal conversion unit 111, quantization unit 112, inverse quantization unit 113, inverse orthogonal conversion unit 114, addition unit 115, in-screen prediction unit 116, in-loop filter 117, provided in the moving image coding device 1. Each function of the inter-screen prediction unit 10, the switch 119, and the entropy coding unit 120 is realized by causing the CPU to execute a program describing these functions.

Further, the entropy decoding unit 50, the inverse quantization unit 51, the inverse orthogonal conversion unit 52, the addition unit 53, the in-screen prediction unit 54, the in-loop filter 55, the inter-screen prediction unit 56, and the switch 57 provided in the moving image decoding device 2 are provided. Is also realized by causing the CPU to execute a program that describes these functions.

These programs are stored in the storage medium, read by the CPU, and executed. In addition, these programs can be stored and distributed in storage media such as magnetic disks (floppy (registered trademark) disks, hard disks, etc.), optical disks (CD-ROM, DVD, etc.), semiconductor memories, etc., and can be distributed via a network. You can also send and receive.

1,100 Video coding device 2 Video decoding device 10, 56, 118, Inter-screen prediction unit 14, 63, 121 Inter-screen prediction processing unit 11, 60 Super-resolution processing unit 12, 61 Blur processing unit 13, 62 , 122 Frame memory 15 Frequency analysis unit 16 Selection unit 17, 64 Parameter processing unit 20 Decoding picture prediction unit 21 Super-resolution prediction unit 22 Blur prediction unit 30 Motion calculation unit 31 Power fluctuation calculation unit 32 Weight calculation unit 33 Prediction image Selection unit 50 Entropy decoding unit 51, 113 Inverse quantization unit 52, 114 Inverse orthogonal conversion unit 53, 115 Addition unit 54, 116 In-screen prediction unit 55, 117 In-loop filter 57, 119 Switch 110 Subtraction unit 111 Orthogonal conversion unit 112 quantization unit 120 entropy encoding unit K local decoded picture K 'decoded picture _{C, C H, V, D} , C', C H, V, D ' super-resolution picture C _H, C _H' horizontal super picture C _V, C _V 'vertical super-resolution picture C _D, C _D' diagonal super picture _{B, B H, V, D} , B ', B H, V, D' blurred picture B _H, B _H ' horizontal blur picture B _V, B _V 'vertical blur picture B _D, B _D' diagonal blurred picture KY, KY 'predicted image CY H of the local decoded _{picture, V, D, CY H,} V, D' super picture Predicted image BY _{H, V, D} , BY _{H, V, D'} Predictive image of blurry picture KV Motion vector of locally decoded picture CV _{H, V, D} Motion vector of super-resolution picture BV _{H, V, D} Blurred picture Motion vector KE Local decoding picture prediction error CE _{H, V, D} Super-resolution picture prediction error BE _{H, V, D} Blurred picture prediction error CV _{H, V, D} -m _{H, V, D} Super-resolution Picture movement BV _{H, V, D} -m _{H, V, D} Blurred Picture movement pH _{H, V, D} Power fluctuation cp _{H, V, D} , cp _{H1, V1, D1} , cp _{H2, V2, D2} , cp _{H3, V3, D3} Super-resolution picture intensity information bp _{H, V, D} , bp _{H1, V1, D1} , bp _{H2, V2, D2} , bp _{H3, V3, D3} Blurred picture intensity information tC _{H, V, D} , super-resolution picture weight tB _{H, V, D} , blur picture weight

Claims (9)

A residual image is generated by subtracting a predicted image from the input moving image, orthogonal conversion, quantization and entropy coding are performed on the residual image, a coded signal is generated and output, and the quantization is performed. The generated quantization index sequence is subjected to inverse quantization and inverse orthogonal conversion to generate a decoded residual image, and the predicted image is added to the decoded residual image to generate an added image, and the added image is generated. In a moving image coding device that generates a locally decoded picture, performs inter-screen prediction using the locally decoded picture, and generates the predicted image.
A plurality of super-resolution pictures are generated based on the locally decoded picture, and a predicted image of the locally decoded picture and a predicted image of the plurality of super-resolution pictures are generated for each predetermined area by the inter-screen prediction. An inter-screen prediction unit for selecting any one of these prediction images is provided.
The inter-screen prediction unit
A picture in which the locally decoded picture is subjected to super-resolution processing to emphasize the component in the high frequency band in the horizontal direction, a picture in which the component in the high frequency band in the vertical direction is emphasized, and a picture in which the component in the high frequency band in the diagonal direction is emphasized The super-resolution processing unit that generates the plurality of super-resolution pictures, and
The screen-to-screen prediction is performed using the current locally decoded picture as a reference picture and each of the past locally decoded picture and the past plurality of super-resolution pictures generated by the super-resolution processing unit as reference pictures. An inter-screen prediction processing unit that generates a prediction image and a motion vector of the locally decoded picture for each predetermined region and also generates a prediction image and a motion vector of the plurality of super-resolution pictures.
A frequency analysis unit that performs frequency analysis of the current locally decoded picture and the past locally decoded picture and generates power fluctuation information between the current locally decoded picture and the past locally decoded picture.
For each predetermined region, the motion vector of the power fluctuation information or the power fluctuation information generated by the frequency analysis unit, the motion vector of the locally decoded picture generated by the inter-screen prediction processing unit, and the plurality of super-resolutions. A selection unit that selects one of the predicted image of the locally decoded picture and the predicted image of the plurality of super-resolution pictures based on the motion vector of the picture.
A moving image encoding device characterized by being equipped with. A residual image is generated by subtracting a predicted image from the input moving image, orthogonal conversion, quantization and entropy coding are performed on the residual image, a coded signal is generated and output, and the quantization is performed. The generated quantization index sequence is subjected to inverse quantization and inverse orthogonal conversion to generate a decoded residual image, and the predicted image is added to the decoded residual image to generate an added image, and the added image is generated. In a moving image coding device that generates a locally decoded picture, performs inter-screen prediction using the locally decoded picture, and generates the predicted image.
A plurality of blurred pictures are generated based on the locally decoded picture, and a predicted image of the locally decoded picture and a predicted image of the plurality of blurred pictures are generated for each predetermined area by the inter-screen prediction, and these predictions are made. Equipped with an inter-screen prediction unit that selects any one of the images
The inter-screen prediction unit
The plurality of pictures in which the locally decoded picture is blurred to suppress the components in the high frequency band in the horizontal direction, the pictures in which the components in the high frequency band in the vertical direction are suppressed, and the pictures in which the components in the high frequency band in the diagonal direction are suppressed are obtained. Blurring processing part generated as a blurring picture of
The screen-to-screen prediction is performed using the current locally decoded picture as a reference picture and each of the past locally decoded picture and the past plurality of blurred pictures generated by the blur processing unit as reference pictures, and for each of the predetermined areas. In addition, an inter-screen prediction processing unit that generates a prediction image and a motion vector of the locally decoded picture and also generates a prediction image and a motion vector of the plurality of blurred pictures.
A frequency analysis unit that performs frequency analysis of the current locally decoded picture and the past locally decoded picture and generates power fluctuation information between the current locally decoded picture and the past locally decoded picture.
For each predetermined region, the motion vector of the power fluctuation information or the power fluctuation information generated by the frequency analysis unit, the motion vector of the locally decoded picture generated by the inter-screen prediction processing unit, and the plurality of blurred pictures. A selection unit that selects one of the predicted image of the locally decoded picture and the predicted image of the plurality of blurred pictures based on the motion vector.
A moving image encoding device characterized by being equipped with. In the moving image coding apparatus according to claim 1,
Based on the locally decoded picture, the plurality of super-resolution pictures and the plurality of blurred pictures are generated, and the predicted image of the locally-decoded picture and the plurality of super-resolutions are generated for each predetermined region by the inter-screen prediction. A screen-to-screen prediction unit that generates a prediction image of a picture and a prediction image of the plurality of blurry pictures and selects any one of the prediction images is provided.
The inter-screen prediction unit further
The plurality of pictures in which the locally decoded picture is blurred to suppress the components in the high frequency band in the horizontal direction, the pictures in which the components in the high frequency band in the vertical direction are suppressed, and the pictures in which the components in the high frequency band in the diagonal direction are suppressed are obtained. Equipped with a blur processing unit that generates as a blur picture of
The inter-screen prediction processing unit
Using the current locally decoded picture as a reference picture, the past locally decoded picture, the past plurality of super-resolution pictures generated by the super-resolution processing unit, and the past said-mentioned generated by the blurring processing unit. The inter-screen prediction is performed using each of the plurality of blurred pictures as a reference picture, and the predicted image and motion vector of the locally decoded picture, the predicted image and motion vector of the plurality of super-resolution pictures, and the said Generates predicted images and motion vectors of multiple blurry pictures,
The selection unit
For each predetermined region, the power fluctuation information generated by the frequency analysis unit, the power fluctuation information, the motion vector of the locally decoded picture generated by the inter-screen prediction processing unit, and the plurality of super-resolutions. One of the predicted image of the locally decoded picture, the predicted image of the plurality of super-resolution pictures, and the predicted image of the plurality of blurred pictures based on the motion vector of the picture and the motion vector of the plurality of blurred pictures. A motion image encoding device characterized in that one is selected. In the moving image coding apparatus according to any one of claims 1 to 3,
The frequency analysis unit
The frequency analysis of the current locally decoded picture and the past locally decoded picture is performed by first-order wavelet packet decomposition without decimation, and the analysis result of the current locally decoded picture and the analysis of the past locally decoded picture are performed. A moving image coding device, characterized in that the power fluctuation information for each frequency band and each phase position is generated based on the result. An encoded signal of a moving image is input, entropy decoding, inverse quantization, and inverse orthogonal conversion are performed on the encoded signal to generate a decoding residual image, and a predicted image is added to the decoded residual image and added. By generating a post-image and filtering the added image to generate a decoded picture, the original moving image is restored, inter-screen prediction is performed using the decoded picture, and the predicted image is generated. In the moving image decoding device
When the coded signal includes a parameter indicating the type of one predicted image selected from the predicted image of the locally decoded picture and the predicted image of a plurality of super-resolution pictures.
A plurality of super-resolution pictures are generated based on the decoded picture, and according to the inter-screen prediction, any one of the decoded picture and the plurality of super-resolution pictures is predicted according to the parameters for each predetermined area. Equipped with an inter-screen prediction unit that generates
The inter-screen prediction unit
The decoded picture is subjected to super-resolution processing to emphasize a picture in which a component in a high frequency band in the horizontal direction is emphasized, a picture in which a component in a high frequency band in a vertical direction is emphasized, and a picture in which a component in a high frequency band in a diagonal direction is emphasized. A super-resolution processing unit that generates multiple super-resolution pictures,
According to the parameters, the inter-screen prediction is performed using the decoded picture and any one of the plurality of super-resolution pictures generated by the super-resolution processing unit, and the prediction for each predetermined area is performed. Inter-screen prediction processing unit that generates images and
A moving image decoding device characterized by being equipped with. An encoded signal of a moving image is input, entropy decoding, inverse quantization, and inverse orthogonal conversion are performed on the encoded signal to generate a decoding residual image, and a predicted image is added to the decoded residual image and added. By generating a post-image and filtering the added image to generate a decoded picture, the original moving image is restored, inter-screen prediction is performed using the decoded picture, and the predicted image is generated. In the moving image decoding device
When the coded signal includes a parameter indicating the type of one predicted image selected from the predicted image of the locally decoded picture and the predicted image of a plurality of blurred pictures.
A screen that generates a plurality of blurred pictures based on the decoded picture, and generates a predicted image of any one of the decoded picture and the plurality of blurred pictures in accordance with the parameters for each predetermined area by the inter-screen prediction. Equipped with an interval prediction unit
The inter-screen prediction unit
A plurality of pictures in which the decoded picture is blurred to suppress the components in the high frequency band in the horizontal direction, the pictures in which the components in the high frequency band in the vertical direction are suppressed, and the pictures in which the components in the high frequency band in the diagonal direction are suppressed are obtained. The blur processing unit generated as a blur picture and
According to the parameters, the inter-screen prediction is performed using the decoded picture and any one of the plurality of blur pictures generated by the blur processing unit, and the predicted image for each predetermined region is generated. Inter-screen prediction processing unit and
A moving image decoding device characterized by being equipped with. In the moving image decoding apparatus according to claim 5.
When the coded signal includes a parameter indicating the type of the predicted image selected from the predicted image of the locally decoded picture, the predicted image of the plurality of super-resolution pictures, and the plurality of blurred pictures.
Based on the decoded picture, the plurality of super-resolution pictures and the plurality of blurred pictures are generated, and according to the inter-screen prediction, the decoded picture and the plurality of super-resolution pictures are generated for each predetermined region according to the parameters. And an inter-screen prediction unit that generates a prediction image of any one of the plurality of blurred pictures.
The inter-screen prediction unit further
A plurality of pictures in which the decoded picture is blurred to suppress the components in the high frequency band in the horizontal direction, the pictures in which the components in the high frequency band in the vertical direction are suppressed, and the pictures in which the components in the high frequency band in the diagonal direction are suppressed are obtained. Equipped with a blur processing unit that generates as a blur picture
The inter-screen prediction processing unit
According to the parameters, any one of the decoded picture, the plurality of super-resolution pictures generated by the super-resolution processing unit, and the plurality of blurring pictures generated by the blurring processing unit is used. A moving image decoding device characterized in that inter-screen prediction is performed and the predicted image for each predetermined area is generated. A program for causing a computer to function as the moving image coding device according to any one of claims 1 to 4. A program for causing a computer to function as the moving image decoding device according to any one of claims 5 to 7.

Images