JPWO2007094329A1 - Moving image processing apparatus, moving image processing method, and moving image processing program - Google Patents

Abstract

The blur intensity determination device (301) determines whether or not to remove the high-frequency signal for each video frame, and when determining to remove the high-frequency signal, determines the high-frequency signal to be removed from the video frame. The image blurring device (302) blurs an image by performing blurring processing, which is processing for removing a high-frequency signal from a video frame, according to the determination of the blur strength determination device (301). The blur intensity determining device (301) periodically changes the high-frequency signal to be removed from the video frame. The image encoding device (303) encodes the video frame from which the high-frequency signal has been removed.

Description

  The present invention relates to a moving image processing apparatus, a moving image processing method, and a moving image processing program that encode and output an input moving image, and in particular, a moving image processing apparatus that removes a high-frequency signal from a video frame constituting a moving image, The present invention relates to a moving image processing method and a moving image processing program.

In order to realize long-time recording on a DVD (registered trademark) and video distribution in a narrow communication network band, it is necessary to realize encoding at a low bit rate. However, encoding at a low bit rate has a problem that image quality is deteriorated. As a method for solving this problem, there is a method of performing encoding after removing in advance a high-frequency signal that is difficult to be detected by human vision included in an image.
When the high frequency signal is removed from the moving image, the code amount for encoding the high frequency signal is not necessary. Therefore, a moving image from which a high-frequency signal has been removed can be encoded at a low bit rate. However, when the high-frequency signal is removed, reproducibility such as an edge (shape contour) in the image is lost, and the image is blurred (meaning that the shape contour is not clear). In particular, if the rate of removing high-frequency signals is large, there is a problem that image quality degradation due to blur is detected by the viewer.
In addition, in a widely used moving picture encoding system such as MPEG (Moving Picture Experts Group) standard, encoding is performed by changing the roughness of quantization for each video frame.
For example, a B picture that performs predictive encoding from both past and future video frames is generally quantized more coarsely than other video frames because the video frame does not become a prediction reference image of a subsequent video frame. is there.
When a video frame is roughly quantized, high-frequency signals tend to be removed by quantization processing. However, since the low-frequency signal is also roughly quantized at the same time, the quantization distortion increases.
When the quantization distortion is large, for example, a difference in pixel value for each pixel block which is a unit for performing quantization becomes large between adjacent pixel blocks. This difference becomes a block distortion, and there is a problem that it is easily detected by the viewer as image quality deterioration.
The following prior art document 1 describes an apparatus for solving this problem. The apparatus described in the prior art document 1 determines an amount of blurring of an image according to the magnitude of motion in a video. This device uses a human visual characteristic that it is difficult to detect a high-frequency signal in a video with a large amount of motion, and blurs a video with a large amount of motion and weakly blurs a video with a small amount of motion. Such a function makes it difficult to detect blurring of an image.
The following prior art document 2 describes an apparatus that performs filter processing on each video frame at regular intervals.
Prior art document 1: Japanese Patent Application Laid-Open No. 2003-333370 (paragraphs 0021 to 0040, FIG. 1)
Prior art document 2: JP-A-2-288788 (page 3-7, FIG. 1)

However, the apparatus described in the above-mentioned prior art document 1 has a problem that blur is easily detected when the motion of the image is large and the blur is too strong. In addition, the high-frequency signal cannot be sufficiently removed because the blurring is weakened in an image with small motion. For this reason, it is impossible to greatly reduce the amount of generated codes during encoding.
Moreover, the apparatus described in the said prior art document 2 filters each video frame for every fixed period. It is preferable that the filtering process performed on the video frame is performed by changing the intensity stepwise.
The present invention has been proposed in view of the above-described problems. That is, an object of the present invention is to provide a moving image processing apparatus, a moving image processing method, and a moving image processing program that can sufficiently remove a high-frequency signal of a video frame.

The moving image processing apparatus according to the first aspect of the present invention determines the intensity of blurring a video frame (including determining whether to blur it), the video frame according to the determination of the blur intensity determining means, and the blur intensity determining means. Image blurring means for blurring the image by removing the high-frequency signal, and the blur intensity determining means changes the intensity of blurring when a predetermined number (one or more) of video frames continue.
The blur intensity determining means changes the intensity of blurring periodically. Also, the blur intensity determining means changes the intensity to be blurred every other frame.
The blur intensity determining means determines an intensity to be blurred according to a change in pixel value between video frames.
The moving image processing apparatus according to the first aspect of the present invention further comprises image encoding means for encoding the video frame output by the image blurring means, and the blur strength determining means is configured such that the image encoding means is a video frame. The intensity to be blurred is determined in accordance with the encoding method when encoding.
The moving image processing apparatus according to the first aspect of the present invention further includes an image encoding unit that encodes the video frame output by the image blurring unit, and the blur intensity determining unit encodes the video frame by the image encoding unit. The intensity to be blurred is determined in accordance with the encoding method and the change in pixel value between video frames.
The moving image processing apparatus according to the first aspect of the present invention further includes a local decoding unit that decodes the video frame encoded by the image encoding unit to generate a predicted reference frame, and a first that stores the predicted reference frame. Storage means, prediction reference frame blurring means for removing the high frequency signal of the prediction reference frame, and prediction image generation means for generating a prediction image of motion compensation prediction from the prediction reference frame from which the prediction reference frame blurring means has removed the high frequency signal The prediction reference frame blurring unit removes the high frequency signal determined to be removed from the video frame by the blur intensity determination unit from the prediction reference frame, and the image encoding unit includes the video frame output by the image blurring unit; The result of the subtraction process with the predicted image generated by the predicted image generation means is encoded.
The moving image processing apparatus according to the first aspect of the present invention further includes a decoding unit that decodes the video frame encoded by the image encoding unit, a second storage unit that stores the video frame decoded by the decoding unit, Second predicted image generating means for generating a motion compensated prediction predicted image, a video frame decoded by the decoding means, and a second predicted image generating means from the video frame stored in the second storage means Adding means for adding and outputting the predicted image.
The moving image processing apparatus according to the first aspect of the present invention further includes a decoding unit that decodes the video frame encoded by the image encoding unit, a second storage unit that stores the video frame decoded by the decoding unit, The second prediction reference frame blurring means for removing the high frequency signal determined to be removed by the blur intensity determination means from the video frame stored in the second storage means, and the second prediction reference frame blurring means for converting the high frequency signal. From the removed video frame, a second predicted image generating unit that generates a motion compensated prediction image, a video frame decoded by the decoding unit, and a predicted image generated by the second predicted image generating unit are added. Adding means for outputting.
The moving image processing apparatus according to the second aspect of the present invention is a moving image processing apparatus that processes a moving image composed of continuous video frames, and determines whether or not to perform filter processing for each video frame, When it is determined that the filter processing is to be performed, the in-loop filter determining means for determining the in-loop filter characteristics for performing the filter processing, and the input video frame are subjected to the filter processing with the filter having the characteristics determined by the in-loop filter determining means. Input image filter processing means for performing, image encoding means for encoding the filtered video frame, local decoding means for generating a prediction reference frame obtained by decoding the video frame encoded by the image encoding means, and prediction reference Fill the predicted reference frame with the first storage means for storing the frame and the filter having the characteristics determined by the in-loop filter determination means. In-loop filter processing means for performing processing, and predicted image generation means for generating a predicted image from a prediction reference frame that has been subjected to filter processing by the in-loop filter processing means, input image filter processing means, and in-loop filter processing means The image encoding means encodes the result of the subtraction process between the video frame output from the input image filter processing means and the prediction image generated by the prediction image generation means. It is characterized by doing.
The moving image processing apparatus according to the second aspect of the present invention further includes a decoding means for decoding the encoded video frame, a second storage means for storing the video frame decoded by the decoding means, and a second storage. The second in-loop filter processing means for performing filter processing on the video frame stored in the means with the filter having the characteristic determined by the in-loop filter determining means, and the second in-loop filter processing means performed the filter processing. From the video frame, a second predicted image generation unit that generates a prediction image for motion compensation prediction, the video frame decoded by the decoding unit, and the predicted image generated by the second predicted image generation unit are added and output. Adding means.
The moving image processing method according to the third aspect of the present invention is a moving image processing method for processing a moving image composed of continuous video frames, and determines the intensity at which the video frame is blurred (it is also determined not to blur). A blur intensity determining step, and an image blur step for blurring the image by removing a high-frequency signal of the video frame in accordance with the determination in the blur intensity determining step. In the blur intensity determining step, a predetermined number (1 The above number) is characterized by changing the intensity of blurring when continuous.
The blur intensity determining step periodically changes the intensity of blurring. The blur intensity determining step changes the intensity to be blurred every other frame.
The blur intensity determining step determines an intensity to be blurred according to a change in pixel value between video frames.
The moving image processing method according to the third aspect of the present invention further includes an image encoding step for encoding the video frame output in the image blurring step, and the video frame is detected in the image encoding step in the blur strength determination step. The intensity to be blurred is determined according to the encoding method at the time of encoding.
The moving image processing method according to the third aspect of the present invention further includes an image encoding step for encoding the video frame output in the image blurring step, and the video frame is detected in the image encoding step in the blur strength determination step. The intensity to be blurred is determined according to the encoding method at the time of encoding and the pixel value change between video frames.
The moving image processing method according to the third aspect of the present invention further includes a local decoding step of generating a prediction reference frame by decoding the video frame encoded in the image encoding step, and removing a high-frequency signal of the prediction reference frame A prediction reference frame blurring step, and a prediction image generation step of generating a prediction image of motion compensation prediction from the prediction reference frame from which a high-frequency signal has been removed in the prediction reference frame blurring step. The high-frequency signal determined to be removed from the video frame in the step is removed from the prediction reference frame, and the result of the subtraction process between the video frame output in the image blurring step and the prediction image generated in the prediction image generation step in the image encoding step Is encoded.
The moving image processing method according to the third aspect of the present invention further includes a decoding step for decoding the video frame encoded in the image encoding step, and a motion compensated prediction predicted image from the video frame decoded in the decoding step. A second predicted image generation step, an addition step of adding and outputting the video frame decoded in the decoding step and the predicted image generated in the second predicted image generation step.
The moving image processing method according to the third aspect of the present invention further includes a decoding step for decoding the video frame encoded in the image encoding step, and a decision to remove from the video frame decoded in the decoding step in the blur intensity determination step. Second prediction reference frame blurring step for removing the high-frequency signal, and second prediction image generation step for generating a prediction image for motion compensation prediction from the video frame from which the high-frequency signal has been removed in the second prediction reference frame blurring step And an addition step of adding and outputting the video frame decoded in the decoding step and the prediction image generated in the second prediction image generation step.
A moving image processing method according to a fourth aspect of the present invention is a moving image processing method for processing a moving image composed of continuous video frames, and determines whether or not to perform filter processing for each video frame, When it is determined that the filter processing is to be performed, the in-loop filter determination step for determining the in-loop filter characteristics for performing the filter processing, and the input video frame are subjected to the filter processing with the filter having the characteristics determined in the in-loop filter determination step. An input image filtering step to be performed; an image encoding step for encoding the filtered video frame; a local decoding step for generating a prediction reference frame obtained by decoding the video frame encoded in the image encoding step; A loop that filters the predicted reference frame with the filter of the characteristic determined in the filter determination step. An inner filter processing step, and a predicted image generation step for generating a predicted image from the prediction reference frame in which the in-loop filter processing step has performed the filter processing. In the input image filter processing step and the in-loop filter processing step, the same The filter processing is performed with a characteristic filter, and the image encoding step encodes the result of the subtraction process between the video frame output in the input image filter processing step and the prediction image generated in the prediction image generation step. To do.
The moving image processing method according to the fourth aspect of the present invention further includes a decoding step for decoding an encoded video frame, and a filter having the characteristics determined in the in-loop filter determination step on the video frame decoded in the decoding step. A second intra-loop filter processing step for performing filter processing; a second predicted image generation step for generating a motion compensated prediction image from the video frame subjected to the filter processing in the second in-loop filter processing step; An addition step of adding and outputting the video frame decoded in the decoding step and the prediction image generated in the second prediction image generation step.
A moving image processing program according to a fifth aspect of the present invention is a moving image processing program for causing a computer to process a moving image composed of continuous video frames, and for determining the strength of blurring the video frames. And determining whether the image is blurred according to the determination in the blur intensity determination process and the blur intensity determination process, and blurring the image by removing the high-frequency signal of the video frame. The intensity of blurring is changed when a predetermined number (one or more) of video frames continues.
The moving image processing program according to the fifth aspect of the present invention causes the computer to change the intensity of blurring periodically in the blur intensity determination process. Further, the computer changes the intensity to be blurred every other frame in the blur intensity determination process.
The moving image processing program according to the fifth aspect of the present invention causes the computer to determine the intensity to be blurred according to the change in the pixel value between the video frames in the blur intensity determination process.
The moving image processing program according to the fifth aspect of the present invention causes a computer to execute an image encoding process for encoding a video frame output by the image blurring process, and in the blurring intensity determination process, the video is processed by the image encoding process. The intensity to be blurred is determined according to the encoding method when encoding the frame.
The moving image processing program according to the fifth aspect of the present invention causes a computer to execute an image encoding process for encoding a video frame output by the image blurring process, and in the blurring intensity determination process, the video is processed by the image encoding process. The intensity to be blurred is determined according to the encoding method used when encoding the frame and the change in the pixel value between the video frames.
The moving image processing program according to the fifth aspect of the present invention includes a computer that further decodes a video frame encoded by the image encoding process to generate a predicted reference frame, and a high-frequency signal of the predicted reference frame In the prediction reference frame blurring process, the prediction reference frame blurring process is executed, and the prediction reference frame blurring process is performed, and the prediction reference frame blurring process is performed to generate a prediction image of motion compensated prediction from the prediction reference frame from which the high frequency signal is removed by the prediction reference frame blurring process The high-frequency signal determined to be removed from the video frame by the blur intensity determination process is removed from the prediction reference frame, and the video frame output by the image blur process and the predicted image generated by the prediction image generation process in the image encoding process The result of the subtraction process is encoded.
The moving image processing program according to the fifth aspect of the present invention is a computer that further decodes a video frame encoded by the image encoding process, and predicts motion compensation prediction from the video frame decoded by the decoding process. A second predicted image generation process for generating an image, a video frame decoded by the decoding process, and an addition process for adding and outputting the predicted image generated by the second predicted image generation process are executed.
The moving image processing program according to the fifth aspect of the present invention is a computer that decodes a video frame encoded by the image encoding process, and removes the video frame decoded by the decoding process by the blur intensity determination process. Then, the second prediction reference frame blurring process for removing the determined high-frequency signal, and the second prediction image for generating a prediction image for motion compensation prediction from the video frame from which the high-frequency signal has been removed by the second prediction reference frame blurring process You may perform a production | generation process and the addition process which adds and outputs the video frame decoded by the decoding process, and the prediction image produced | generated by the 2nd prediction image production | generation process.
A moving image processing program according to a sixth aspect of the present invention is a moving image processing program that causes a computer to process a moving image composed of continuous video frames, and that performs filtering processing for each video frame on the computer. If it is determined whether or not to perform the filter process, the filter determination process in the loop for determining the filter characteristic in the loop to perform the filter process, and the characteristics determined by the filter determination process in the loop for the input video frame An input image filtering execution process for performing a filtering process with a filter, an image encoding process for encoding the filtered video frame, and a local reference frame for decoding a video frame encoded by the image encoding process Prediction reference frame with filter of characteristics determined by decoding process and in-loop filter determination process In-loop filtering execution processing for performing filtering processing, and predicted image generation processing for generating a prediction image from a prediction reference frame subjected to filtering processing in the in-loop filtering execution processing are executed, and input image filtering execution processing and in-loop filtering are executed. In the execution process, the filter process is executed with a filter having the same characteristics, and in the image encoding process, the result of the subtraction process between the video frame output from the input image filtering execution process and the predicted image generated in the predicted image generation process is obtained. It is encoded.
The moving image processing program according to the sixth aspect of the present invention includes a computer that further decodes an encoded video frame, and the characteristics determined by the in-loop filter determination process for the decoded video frame. Second prediction image generation for generating a prediction image of motion compensated prediction from the second intra-loop filtering execution process for performing the filtering process with the first filter and the video frame subjected to the filtering process in the second in-loop filtering execution process A process and an addition process for adding and outputting the video frame decoded by the decoding process and the predicted image generated by the second predicted image generation process are executed.

The first effect of the present invention is to control the intensity of blurring for each video frame, so that the viewer does not easily detect whether or not the blurring process has been performed, and removes high-frequency signals from moving images. Be able to.
In particular, in the case where the video frame that has been subjected to the blurring process and the video frame that has not been subjected to the blurring process are configured to be periodic, the viewer is asked whether or not the blurring process has been performed. It is possible to remove the high-frequency signal of the moving image without being detected more easily.
The second effect of the present invention is that it is difficult to detect that the blurring process has been performed, and the subjective image quality can be improved when a moving image is encoded.
The reason is that blur is hardly detected at the time of reproduction by forming a moving image with a video frame that is first subjected to the blur process and a video frame that is not subjected to the blur process. Second, since the high-frequency signal is removed from the video frame that has been subjected to the blurring process, the amount of generated codes can be reduced, quantization can be performed with a small quantization step, and the quantization error can be reduced. is there.
If the device is configured to determine the high-frequency signal to be removed from the video frame according to the pixel value change between the video frames and the encoding method when encoding the video frame, Utilizing the characteristics, it is possible to make it difficult for the viewer to detect that the blurring process has been performed.
When the apparatus removes the high-frequency signal from the prediction reference frame obtained by decoding the encoded video frame, the apparatus is configured to remove the high-frequency signal and generate a predicted image in the same manner as the blurring process at the time of encoding. In this case, the video frame that has been subjected to the blurring process can be correctly decoded.
When the apparatus is configured to generate a prediction image for motion compensation prediction from a video frame obtained by decoding a video frame encoded with high-frequency signals removed, an encoding scheme such as MPEG-2 is used. A video frame that has been subjected to the blurring process can be decoded with a configuration similar to that of a moving image decoding apparatus that decodes encoded moving image data.

FIG. 1 is a block diagram showing a first embodiment of a moving image processing apparatus.
FIG. 2 is an explanatory diagram illustrating a configuration of a video in which a video frame to be blurred and a video frame to be blurred are alternated.
FIG. 3 is a block diagram showing a second embodiment of the moving image processing apparatus.
FIG. 4 is a flowchart schematically showing the operation of the moving image processing apparatus of the second embodiment.
FIG. 5 is a block diagram illustrating a configuration example of a decoding device.
FIG. 6 is a flowchart schematically showing the operation of the decoding apparatus.
FIG. 7 is a block diagram showing the configuration of the decoding apparatus in the third embodiment.
FIG. 8 is a block diagram showing a fourth embodiment of the moving image processing apparatus.
FIG. 9 is a flowchart schematically showing the operation of the moving image processing apparatus of the fourth embodiment.
FIG. 10 is a block diagram showing the configuration of the decoding apparatus in the fourth embodiment.
FIG. 11 is a flowchart schematically showing the operation of the decoding apparatus in the fourth embodiment.
FIG. 12 is a block diagram illustrating a configuration example when the moving image processing apparatus is realized by a computer system.

A first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a first embodiment of a moving image processing apparatus according to the present invention.
The moving image processing apparatus shown in FIG. 1 includes a blur intensity determining apparatus 201 that determines the intensity at which a video frame is blurred, and an image blur apparatus 202 that performs a process of removing a high-frequency signal from the video frame and blurring the image.
The blur intensity determination device 201 determines whether or not to perform the process of blurring each video frame when the input image is input, and the blur intensity when performing the process of blurring the video frame. In the present embodiment and the following embodiments, the blur intensity corresponds to the frequency of the high frequency signal removed from the video frame.
The intensity of blur is not constant between successive video frames. For example, the blur intensity determining apparatus 201 determines the blur intensity so that a video frame subjected to the blur process and a video frame not subjected to the blur process are alternated, and the video frame which is strongly blurred and the video frame which is weakly blurred alternately. Determine the blur intensity. That is, the blur intensity determination device 201 periodically changes the intensity of the blurring process performed on the video frame. Further, the period of changing the intensity to be blurred by the blur intensity determining apparatus 201 may be constant or not constant. That is, the intensity to blur aperiodically may be changed.
FIG. 2 is an explanatory diagram illustrating a configuration of a video in which a video frame to be blurred and a video frame to be blurred are alternated.
Note that the blur intensity determining apparatus 201 does not have to change the intensity to be blurred for each video frame, as shown in FIG. For example, the blur intensity determining apparatus 201 may change the intensity to be blurred every two video frames.
The blur intensity determining apparatus 201 determines the intensity to be blurred in consideration of the characteristics of the video frame itself such as the amount of change in luminance value. Specifically, the blur intensity determination device 201 has, for example, an average moving amount (motion vector amount) of video of 16 pixels and a change amount of the luminance value considering the amount of motion is 2 per pixel. Decide to blur.
Also, the blur intensity determining apparatus 201 may determine the intensity to be blurred in consideration of the encoding method when the input image is encoded. For example, the blur intensity determining apparatus 201 may change the intensity of blurring between a video frame that employs interframe prediction and a video frame that does not.
The blur intensity determining apparatus 201 determines the intensity for blurring each video frame, for example, in one of 10 levels.
The blur intensity determining apparatus 201 determines whether or not to perform the blur process on each video frame, and when determining to perform the blur process, determines the blur intensity in the video frame blur process. In this case, the image blurring device 202 removes and outputs the high-frequency signal of the video frame determined to be blurred according to the strength to be blurred. The image blurring device 202 outputs the video frame determined not to be subjected to the blur process without performing the blur process. In order to remove the high-frequency signal, for example, a low-pass filter is used.
Specifically, the blur intensity determining apparatus 201, for example, when blurring a video frame strongly, for example, a high-frequency signal having a frequency of about 50% or more of the maximum frequency (for example, 4.2 MHz) (for example, 2.1 MHz or more). Signal).
In addition, the blur intensity determining apparatus 201, when blurring a video frame, for example, outputs a high-frequency signal (for example, a signal of 3.8 MHz or more) having a frequency of about 90% or more of the maximum frequency (for example, 4.2 MHz). Remove.
In this specification, removing a high-frequency signal from a video frame is referred to as blurring processing.
As described above, according to the present embodiment, the intensity of blurring is controlled for each video frame, and video frames that are subjected to the blurring process and video frames that are not subjected to the blurring process are alternately configured. Therefore, it is possible to remove the high-frequency signal of the moving image (video frame) without easily detecting whether or not the process of blurring is performed on the viewer.
Therefore, it is possible to reduce the amount of code for encoding a video frame and prevent encoding at a low bit rate while preventing the viewer from detecting blur.
If no high-frequency signal is removed, that is, if no blurring process is performed, the amount of generated code increases. In that case, it is necessary to increase the quantization step in order to encode at a low bit rate. As a result, a quantization error occurs and the image quality is degraded.
However, in this embodiment, since the high-frequency signal is removed from the video frame subjected to the blurring process, the generated code amount is reduced as a whole. Therefore, even when encoding at a low bit rate, quantization can be performed with a small quantization step, and a quantization error can be reduced. For this reason, it is possible to make it difficult to detect a decrease in image quality due to a quantization error.
In other words, in order to reduce the quantization error when encoding a moving image (video frame), it is difficult to detect that the process of blurring the viewer has been performed. The subjective image quality (image quality perceived by the viewer) can be improved compared to the case where no high frequency signal is removed.
Next, a second embodiment of the present invention will be described with reference to the drawings. FIG. 3 is a block diagram showing a second embodiment of the moving image processing apparatus according to the present invention.
As shown in FIG. 3, the moving image processing apparatus according to the second embodiment is a blur intensity determining device (blur intensity determining means) 301 that determines the blur intensity of a video frame, and an image obtained by removing a high-frequency signal from the video frame. An image blurring device (image blurring unit) 302 that blurs an image, an image encoding device (image encoding unit) 303 that encodes an input image, and a local decoding device (locally) that generates a prediction reference frame of an image to be encoded (Decoding means) 304, a frame memory (first storage means) 305 for storing (storing) a prediction reference frame, and a prediction reference frame blurring apparatus (prediction reference) that blurs the prediction reference frame with the same intensity as the input image being encoded. A frame blurring unit) 306, and a predicted image generating device (predicted image generating unit) 307 that generates a predicted image of motion compensation prediction from the predicted reference frame. That.
In the present embodiment, an apparatus that performs encoding by the MPEG encoding method is assumed as the moving image processing apparatus. However, the moving image processing apparatus according to the present invention is not limited to the one that performs encoding by the MPEG encoding method.
The blur intensity determination device 301 determines whether or not to perform the process of blurring each video frame when an input image is input. When the process of blurring the video frame is performed, the blur strength determining apparatus 301 removes the blur intensity (from the video frame). Signal frequency).
The strength to blur is not constant. The blur intensity determination device 301 determines, for example, that video frames that are to be blurred and video frames that are not to be blurred are alternated, and video frames that are strongly blurred and video frames that are weakly blurred are alternately determined. To do.
The blur intensity determination device 301 determines the blur strength based on the information output from the image encoding device 303. For example, the intensity to be blurred is determined based on the encoding method, the intensity to be blurred is determined based on the change in the pixel value between video frames of the image, and the change in the pixel value between the encoding method and the video frame of the image Determine the strength to blur based on the combination of
The blur intensity determination device 301 outputs the blur strength information indicating the blur strength of the video frame and what kind of blur processing has been performed to the image encoding device 303 and the prediction reference frame blur device 306.
The image blurring device 302 blurs the image frame determined to be blurred by the blur strength determining device 301 in the input image with the blur strength determined by the blur strength determining device 301 (performs the blurring process). The data is output to the encoding device 303. In order to perform the blurring process, for example, a low-pass filter is used.
In addition, the image blurring device 302 outputs, to the image coding device 303, the video frame that is determined not to be subjected to the blurring process performed by the blurring intensity determination device 301 among the input images.
The image encoding device 303 encodes and outputs the video frame output from the image blurring device 302. The image encoding device 303 also encodes the blur intensity information and encodes the encoded moving image data (video frame) so as to include the blur intensity information.
The local decoding device 304 decodes the video frame encoded by the image encoding device 303 for use as a prediction reference frame for motion compensation prediction of a video frame to be encoded later. The frame memory 305 stores the video frame decoded by the local decoding device 304.
The prediction reference frame blurring device 306 performs a process of blurring the video frame (prediction reference frame) stored in the frame memory 305. The prediction reference frame blurring device 306 performs a blurring process using a method similar to the method indicated by the blur strength information based on the blur strength information output from the blur strength determination device 301.
The predicted image generation device 307 generates a predicted image using the predicted reference frame that has been subjected to the blurring process.
The predicted image generated by the predicted image generation device 307 is subtracted from the video frame output from the image blurring device 302, and the subtraction result is input to the image encoding device 303. Further, the predicted image generated by the predicted image generation device 307 is added to the video frame decoded by the local decoding device 304 and stored in the frame memory 305. The subtraction process is performed by, for example, a subtracter, and the addition process is performed by, for example, an adder.
Next, the operation of the moving image processing apparatus of the second embodiment will be described with reference to the drawings. FIG. 4 is a flowchart schematically showing the operation of the moving image processing apparatus of the second embodiment. The process shown in FIG. 4 is a process in a process until a video frame encoded without using a prediction reference frame is output first, and then a video frame encoded using a prediction reference frame is output. It is.
The blur intensity determination device 301 determines whether or not to perform the process of blurring each video frame when an input image is input, and the blur intensity (the signal to be removed from the video) when performing the process of blurring the video frame. (Frequency) is determined (steps S101 and S102).
The image blurring apparatus 302 blurs (performs a blurring process) a video frame determined to be blurred by the blurring intensity determination apparatus 301 in the input image with a blurring intensity determined by the blurring intensity determination apparatus 301 and performs an image encoding apparatus. It outputs to 303 (step S103).
The image encoding device 303 encodes the video frame output from the image blurring device 302 and outputs it as data relating to the first frame (step S104).
Since the video frame output from the image encoding device 303 is used as a prediction reference frame of a video frame to be encoded later, the local decoding device 304 decodes the video frame output from the image encoding device 303 (step S105). ). The prediction reference frame blurring device 306 performs a process (filtering process) to blur the video frame (prediction reference frame) stored in the frame memory 305 (step S106).
The predicted image generation device 307 generates a predicted image using the predicted reference frame that has been subjected to the blurring process (filter process) (step S107). The predicted image generated by the predicted image generation device 607 is subtracted from the video frame output from the image blurring device 302 by the subtracter (step S108).
The image encoding device 303 encodes the output of the subtracter and outputs it as data relating to the second frame (steps S109 and S110).
Next, decoding of the encoded moving image will be described. FIG. 5 is a block diagram illustrating a configuration example of a decoding device.
The decoding device shown in FIG. 5 includes an image decoding device (decoding unit) 401 that decodes an encoded moving image, and a frame memory that stores (stores) the decoded video frame for use as a prediction reference frame ( A second storage unit) 402, a prediction reference frame blurring device (second prediction reference frame blurring unit) 403 that performs processing to blur the prediction reference frame, and a prediction image that generates a motion compensation prediction prediction image from the prediction reference frame A generation device (second predicted image generation means) 404.
The moving image processing apparatus may include the function of the decoding apparatus illustrated in FIG.
The image decoding device 401 decodes the encoded video frame. Note that the image decoding apparatus 401 also decodes the encoded blur intensity information at the same time. Then, the image decoding apparatus 401 outputs the decoded blur intensity information to the prediction reference frame blurring apparatus 403.
The frame memory 402 stores the video frame (prediction reference frame) decoded by the image decoding device 401 in order to use the video frame decoded by the image decoding device 401 as a prediction reference frame.
Based on the blur intensity information decoded by the image decoding apparatus 401, the prediction reference frame blurring apparatus 403 performs a blurring process on the prediction reference frame stored in the frame memory 402 with the same method and the same intensity as when encoding. .
The predicted image generation device 404 generates a predicted image using the predicted reference frame that has been subjected to the processing performed by the predicted reference frame blurring device 403. The predicted image generated by the predicted image generation device 404 is added to the video frame decoded by the image decoding device 401 and output. The addition processing is performed by, for example, an adder (adding unit).
Next, the operation of the decoding device will be described with reference to the drawings. FIG. 6 is a flowchart schematically showing the operation of the decoding apparatus. The process illustrated in FIG. 6 is an example of a case where decoding is performed using a prediction reference frame.
The image decoding apparatus 401 decodes the encoded video frame when the encoded video frame is input without using the prediction reference frame (step S201).
The prediction reference frame blurring apparatus 403 performs the same process and the same blur process (filtering process) as those at the time of encoding on the prediction reference frame (step S202). The predicted image generation device 404 generates a predicted image using the predicted reference frame that has been subjected to the processing (filter processing) performed by the predicted reference frame blurring device 403 (step S203).
The predicted image generated by the predicted image generation device 404 is added to the video frame decoded by the image decoding device 701 by an adder. Then, the addition result is output from the adder (steps S204 and S205).
The operation of each device included in the moving image processing device may be performed by a CPU that executes processing according to program control. Specifically, the moving image processing apparatus is a moving image processing program for causing a computer to process a moving image composed of continuous video frames, and determines the strength to blur the video frame to the computer (determines not to blur). In accordance with the determination in the blur intensity determination process and the blur intensity determination process, an image blur process for blurring the image by removing the high-frequency signal of the video frame is executed. A moving image processing program is mounted that changes the intensity of blurring when a predetermined number (one or more) continues. The CPU may execute each process.
According to the present embodiment, based on the determination of the blur intensity determination device 301, whether or not to perform the blurring process for each video frame, and the blurring intensity at the time of performing the blurring process are changed, thereby encoding and encoding the video frame. Decoding can be performed.
Therefore, similarly to the first embodiment, it is possible to remove the high-frequency signal of the moving image without easily detecting whether or not the blurring process has been performed on the viewer, thereby improving the subjective image quality.
Next, a third embodiment of the present invention will be described. The configuration of the moving image processing apparatus is the same as that of the moving image processing apparatus of the second embodiment shown in FIG. However, in the third embodiment, the decoding device is configured as shown in FIG. In other words, a device corresponding to the prediction reference frame blurring device 403 is not provided.
Note that the moving image processing apparatus that performs the encoding process may include the function of the decoding apparatus illustrated in FIG.
7 includes an image decoding device 501 that decodes encoded video data, a frame memory 502 that stores (stores) the decoded video frame for use as a prediction reference frame, and a prediction reference. A prediction image generation device 503 that generates a prediction image of motion compensation prediction from the frame.
The decoding apparatus shown in FIG. 7 has a configuration in which the prediction reference frame blurring apparatus 403 is removed from the decoding apparatus shown in FIG. The decoding device shown in FIG. 7 has the same configuration as that of a moving image decoding device that decodes moving image data encoded by an encoding method such as MPEG-2.
The input image is encoded by the moving image processing apparatus shown in FIG. Therefore, the prediction reference frame used for motion compensation prediction is blurred with different intensity for each video frame.
On the other hand, the encoded video frame is decoded by the decoding device shown in FIG. The decoding apparatus shown in FIG. 7 does not include a means for blurring the prediction reference frame. Accordingly, since the decoding apparatus shown in FIG. 7 cannot generate a prediction reference frame that matches the prediction reference frame used at the time of encoding, the video frame encoded by the moving image processing apparatus shown in FIG. 3 is correctly decoded. Can not do it.
Therefore, the decoded video frame is different from the video frame before being encoded by the moving image processing apparatus shown in FIG. The prediction residual due to motion compensation prediction is composed of a difference between a prediction reference frame blurred at the time of encoding and a blurred input video frame. When decoding a video frame, this prediction residual is added to a prediction reference frame that is not blurred.
The image data (video frame) when the decoding apparatus shown in FIG. 7 decodes the image data (video frame) when the input image is encoded without blurring as in the case of encoding according to the normal MPEG-2 system or the like. ) And the image data (video frame) obtained by decoding the image data (video frame) encoded by the moving image processing apparatus shown in FIG. Then, the difference in the decoded image data (video frame) is the presence or absence of a high frequency signal of the prediction residual.
Therefore, in this embodiment, when the moving image data encoded by the moving image processing apparatus shown in FIG. 3 is decoded by the decoding apparatus shown in FIG. 7, the image data (video frame) cannot be correctly decoded. However, since high-frequency signals are difficult to detect due to human visual characteristics, the difference (difference between image data (video frame) before encoding and image data (video frame) after decoding) is difficult to be detected by the viewer. be able to.
Further, since the video frame subjected to the blurring process includes only a few high-frequency signals, encoding is simplified. Therefore, the generated code amount can be reduced. Since the reduced code amount can be assigned to a video frame that has not been subjected to the blurring process, the image quality of the video frame that has not been subjected to the blurring process can be improved. Therefore, according to the present embodiment, the subjective image quality of the entire video can be improved.
Next, a fourth embodiment of the present invention will be described with reference to the drawings. A moving image processing apparatus according to the fourth embodiment of the present invention is described in H.264. In an encoding method in which an in-loop filter is adopted, such as the H.261 method (method based on the H.261 recommendation of ITU-T (International Telecommunication Union Standardization Sector)), the input image is filtered by the same filter as the in-loop filter. Filter by the characteristics to reduce the prediction residual and reduce the amount of generated code.
In this embodiment, the video frame is H.264. H.261 encoding, and H.264. A case where a video frame encoded by the H.261 system is decoded will be described as an example. FIG. 8 is a block diagram showing a fourth embodiment of the moving image processing apparatus according to the present invention.
The moving image processing apparatus shown in FIG. 8 includes an in-loop filter determining device (in-loop filter determining means) 601 that determines the type of an in-loop filter, and an in-loop provided in an encoding loop for an input image. Input image filter processing apparatus (input image filter processing means) 602 that performs filter processing using a filter equal to the filter, an image encoding apparatus (image encoding means) 603 that encodes an input image, and an image to be encoded A local decoding device (local decoding means) 604 for generating a prediction reference frame, a frame memory (first storage means) 605 for storing (storing) a prediction reference frame, and a loop for removing a high-frequency signal of the prediction reference frame An intra-filter processing device (in-loop filter processing means) 606 and a prediction image for generating a motion compensated prediction image from the prediction reference frame Image generating apparatus and a (predicted image generating means) 607.
The in-loop filter determination device 601 determines whether or not to perform a blurring process (filter process) on each video frame based on the video frame and the type of the in-loop filter to be used. The in-loop filter determination device 601 determines the type of the in-loop filter in consideration of the generated code amount generated by the output of the image encoding device 603, the complexity of the image data, and the like.
Further, the in-loop filter determination device 601 outputs filter information indicating the type of in-loop filter to be used to the image coding device 603 and the in-loop filter processing device 606.
The input image filter processing device 602 blurs a video frame determined to be blurred by the in-loop filter determining device 601 among the video frames with the in-loop filter determined by the in-loop filter determining device 601 (blurring processing ( Filter processing) and output to the image encoding device 603.
Also, the input image filter processing device 602 sends the video frame determined not to be blurred by the in-loop filter determination device 601 to the image coding device 603 without performing the blurring processing (filter processing). Output.
The image encoding device 603 encodes and outputs the video frame output from the input image filter processing device 602 and the filter information output from the in-loop filter determination device 601.
The local decoding device 604 decodes the video frame encoded by the image encoding device 603 for use as a prediction reference frame for motion compensation prediction of a video frame to be encoded later. The frame memory 605 stores the video frame decoded by the local decoding device 604.
The in-loop filter device 606 performs a process of blurring the video frame (predicted reference frame) stored in the frame memory 605. Note that the in-loop filter device 606 performs processing (filter processing) to blur the predicted reference frame using the same filter as the input image filter processing device 602 based on the filter information output by the in-loop filter determination device 601.
The predicted image generation device 607 generates a predicted image using the predicted reference frame that has been subjected to the blurring process (filter process). The predicted image generated by the predicted image generation device 607 is subtracted from the video frame output from the input image filter processing device 602. Further, the predicted image generated by the predicted image generation device 607 is added to the video frame decoded by the local decoding device 604 and stored in the frame memory 605. In order to perform the filtering process, for example, a low-pass filter is used.
Next, the operation of the moving image processing apparatus of the fourth embodiment will be described with reference to the drawings. FIG. 9 is a flowchart schematically showing the operation of the moving image processing apparatus of the fourth embodiment.
The in-loop filter determination device 601 determines whether or not to apply a blurring process (filter process) to each video frame when image data (video frame) is input, and the type of in-loop filter to be used (step). S301, S302).
The input image filter processing device 602 blurs a video frame determined to be blurred by the in-loop filter determination device 601 out of the video frames with the in-loop filter determined by the in-loop filter determination device 601. That is, a blurring process (filtering process) is performed. Then, the input image filter processing device 602 outputs the video frame subjected to the blurring process to the image coding device 603 (step S303).
The image encoding device 603 encodes the video frame output from the input image filter processing device 602 and outputs the encoded video frame as data relating to the first frame (step S304).
Since the video frame output from the image encoding device 603 is used as a prediction reference frame of a video frame to be encoded later, the local decoding device 604 decodes the video frame output from the image encoding device 603 (step S305). ). The in-loop filter device 606 performs a process (filter process) to blur the video frame (predicted reference frame) stored in the frame memory 605 (step S306).
The predicted image generation device 607 generates a predicted image using the predicted reference frame that has been subjected to the blurring process (filter process) (step S307). The predicted image generated by the predicted image generation device 607 is subtracted from the video frame output from the input image filter processing device 602 by a subtracter (step S308).
The image encoding device 603 encodes the output of the subtracter and outputs encoded video frame data (steps S309 and S310).
Next, decoding of the encoded moving image will be described. In the fourth embodiment, decoding is performed by the decoding device shown in FIG.
The decoding apparatus shown in FIG. 10 includes an image decoding apparatus (decoding unit) 701 that decodes an encoded moving image, and a frame that is stored (stored) to use the decoded video frame as a prediction reference frame. A memory (second storage unit) 702, an in-loop filter processing device (second in-loop filter processing unit) 703 that removes a high-frequency signal from the prediction reference frame, and a motion compensated prediction prediction image is generated from the prediction reference frame A predicted image generation apparatus (second predicted image generation means) 704.
Note that the moving image processing apparatus may include the function of the decoding apparatus illustrated in FIG.
The image decoding device 701 decodes the encoded video frame and filter information. The image decoding device 701 outputs the decoded filter information to the in-loop filter processing device 703. The frame memory 702 stores the video frame decoded by the image decoding device 701 for use as a prediction reference frame. The in-loop filter device 703 performs a blurring process (filter processing) on the prediction reference frame stored in the frame memory 702 based on the filter information output from the image decoding device 701.
The predicted image generation device 704 generates a predicted image using the predicted reference frame that has been subjected to the filter processing. The predicted image generated by the predicted image generation device 704 is added to the video frame decoded by the image decoding device 701 and output.
Next, the operation of the decoding device will be described with reference to the drawings. FIG. 11 is a flowchart schematically showing the operation of the decoding apparatus according to the fourth embodiment.
The image decoding device 701 decodes the encoded video frame (step S401).
The in-loop filter device 703 performs the same process and the same blurring process (filtering process) as the encoding process on the prediction reference frame (step S402). The predicted image generation device 704 generates a predicted image using the predicted reference frame that has been subjected to the processing (filter processing) that is blurred by the in-loop filter device 703 (step S403).
The predicted image generated by the predicted image generation device 704 is added to the video frame decoded by the image decoding device 701 by an adder, and the addition result is output from the adder (steps S404 and S405).
The operation of each device included in the moving image processing device may be performed by a CPU that executes processing according to program control. Specifically, when the moving image processing apparatus determines whether or not to perform the filtering process for each video frame, and determines to perform the filtering process, the moving image processing apparatus determines the in-loop filter characteristics for performing the filtering process. In-loop filter determination processing, input image filtering execution processing for filtering the input video frame with a filter having characteristics determined by the in-loop filter determination processing, and image code for encoding the filtered video frame In the loop that performs the filtering process on the prediction reference frame by the local decoding process that generates the prediction reference frame obtained by decoding the video frame encoded by the encoding process, the image encoding process, and the characteristic filter determined by the filter determination process in the loop Prediction that has been filtered by filtering execution processing and in-loop filtering execution processing A prediction image generation process that generates a prediction image from the reference frame, and a filter process with a filter having the same characteristics in the input image filtering execution process and the in-loop filtering execution process, and the input in the image encoding process A moving image processing program is included, which encodes a result of a subtraction process between a video frame output by the image filtering execution process and a predicted image generated by the predicted image generation process. The moving image processing program may be executed by a CPU.
According to the present embodiment, the filtering process is performed on the input image using a filter having the same characteristics as the in-loop filter obtained by performing the filtering process on the predicted reference frame, so that the high-frequency signal is the same from the predicted reference frame and the input image. To some extent, the prediction residual can be reduced. Therefore, the amount of codes necessary for encoding can be reduced.
In addition, the decoding device in this embodiment is H.264. Since the configuration is the same as that of a decoding device of an encoding method such as the H.261 method, it is possible to implement the present embodiment by changing only the part that performs image encoding.
FIG. 12 is a block diagram illustrating a configuration example when the moving image processing apparatus according to the first embodiment is realized by a computer system.
In the configuration example illustrated in FIG. 12, the computer system includes a processor 801. The processor 801 is connected to an input data buffer 811 and an output data buffer 812, and a program memory 802 that stores necessary programs.
As program modules stored in the program memory 802, a blur intensity determination process 813 and an image blur process 814 are included in addition to the main program. The main program is a main program that executes moving image processing. The blur intensity determination processing program 813 is a program for realizing the function of the blur intensity determining apparatus 201. The image blur processing program 814 is a program for realizing the function of the image blurring apparatus 202. When the processor 801 executes the main program and each functional module, the moving image processing in the moving image processing apparatus of the first embodiment is executed.
Similarly, the moving image processing apparatuses according to the second to fourth embodiments described above are realized by mounting program modules for realizing the functions of the respective blocks in the computer system.
Next, a more specific embodiment of the moving image processing apparatus according to the present invention will be described.
Here, a moving image of SDTV (Standard Definition Television) size (horizontal direction: 720 pixels, vertical direction: 480 pixels, 29.97 frames per second) is input to the moving image processing apparatus shown in FIG. The case where it encodes with an encoding system is made into an example.
The image encoding device 303 includes a frame (I picture) that independently encodes one video frame, a frame (P picture) that predicts a past video frame and encodes a prediction error, Prediction is performed from a future video frame, and the prediction error is encoded into one of three types of pictures (B picture).
Here, only the video frame encoded as a B picture among the input moving images is subjected to the blurring process by the image blurring device 302, and the video frame encoded as the I and P pictures is subjected to the blurring process. An example is a case where encoding is performed without any change. The determination as to whether or not to perform the blurring process is performed by the blurring intensity determination device 301.
In this way, the input image that has been subjected to different blurring processing depending on the type of picture is encoded by the image encoding device 303. The image encoding device 303 encodes information on how to blur the input image in addition to encoding by the same method as the MPEG method using discrete cosine transform, quantization, and variable length encoding. The encoded video frame is decoded by the local decoding device 304 and stored in the frame memory 305 for use as a prediction reference frame. When the encoded video frame is a B picture, the prediction reference frame blurring device 306 performs a blurring process on the prediction reference frame stored in the frame memory 305. At this time, the predicted reference frame is blurred by the same method as the method in which the image blurring device 302 blurs the input image.
The predicted image generation device 307 generates a predicted image using the blurred predicted reference frame. Then, the difference between the input image and the predicted image is taken, and the difference is encoded. When the encoded video frame is a P picture, the prediction reference frame blurring device 306 does not perform the blurring process, the prediction image generation device 307 generates a prediction image, takes the difference from the input image, and calculates the difference. Encoded.
When decoding the encoded moving image data, the image frame is restored by the image decoding device 401 shown in FIG. At this time, information on how the video frame was blurred at the time of encoding is also decoded. The decoded video frame is stored in the frame memory 402 for use as a predicted reference frame.
When the decoded frame is a B picture, the prediction reference frame stored in the frame memory 402 is blurred by the prediction reference frame blurring device 403 in the same manner as when encoding. The predicted image generation apparatus 404 generates a predicted image from the blurred predicted reference frame. Then, the prediction residual as the decoding result of the image decoding apparatus 401 and the prediction image are added, and the decoding is completed. When the frame being decoded is a P picture, the prediction reference frame blurring device 403 does not blur the prediction reference frame. The predicted image generation device 404 generates a predicted image, adds the decoded prediction residual and the predicted image, and completes decoding.

  The present invention can be applied to a moving image processing apparatus that realizes encoding at a low bit rate in order to realize long-time recording on a DVD and video distribution in a narrow communication network band.

Claims (33)

In a moving image processing apparatus that processes a moving image composed of continuous video frames,
Blur intensity determining means for determining the intensity of blurring the video frame;
In accordance with the determination of the blur intensity determination means, the image blur means for blurring the image by removing the high-frequency signal of the video frame,
The moving image processing apparatus, wherein the blur intensity determining means changes an intensity of blurring when a predetermined number of video frames continue. The moving image processing apparatus according to claim 1, wherein the blur intensity determination unit changes the intensity of blurring periodically. The moving image processing apparatus according to claim 1, wherein the blur intensity determining unit changes the intensity to be blurred every other frame. The moving image processing apparatus according to any one of claims 1 to 3, wherein the blur intensity determining unit determines an intensity to be blurred according to a change in a pixel value between video frames. Image encoding means for encoding the video frame output by the image blurring means,
The moving image processing apparatus according to claim 3, wherein the blur intensity determining unit determines an intensity to be blurred according to an encoding method when the image encoding unit encodes a video frame. Image encoding means for encoding the video frame output by the image blurring means,
The blur intensity determining means determines an intensity to be blurred according to an encoding method when the image encoding means encodes a video frame and a pixel value change between the video frames. Moving image processing device. Local decoding means for decoding the video frame encoded by the image encoding means to generate a predicted reference frame;
First storage means for storing the prediction reference frame;
Predicted reference frame blurring means for removing a high-frequency signal of the predicted reference frame;
The prediction reference frame blurring unit includes a prediction image generation unit that generates a prediction image of motion compensation prediction from a prediction reference frame from which a high-frequency signal has been removed,
The prediction reference frame blurring unit removes the high frequency signal determined to be removed from the video frame by the blur intensity determination unit from the prediction reference frame;
The said image encoding means encodes the result of the subtraction process of the image | video frame which the image blurring means output, and the estimated image which the said estimated image generation means produced | generated. The Claim 5 or Claim 6 characterized by the above-mentioned. Moving image processing apparatus. Decoding means for decoding the video frame encoded by the image encoding means;
Second storage means for storing the video frame decoded by the decoding means;
Second predicted image generation means for generating a predicted image of motion compensated prediction from the video frame stored in the second storage means;
The moving image processing apparatus according to claim 7, further comprising: an adding unit that adds and outputs the video frame decoded by the decoding unit and the predicted image generated by the second predicted image generating unit. . Decoding means for decoding the video frame encoded by the image encoding means;
Second storage means for storing the video frame decoded by the decoding means;
Second predictive reference frame blurring means for removing the high-frequency signal determined to be removed by the blur intensity determining means from the video frame stored in the second storage means;
Second predicted image generation means for generating a predicted image of motion compensated prediction from the video frame from which the second predicted reference frame blurring means has removed high-frequency signals;
The moving image processing apparatus according to claim 7, further comprising: an adding unit that adds and outputs the video frame decoded by the decoding unit and the predicted image generated by the second predicted image generating unit. . In a moving image processing apparatus that processes a moving image composed of continuous video frames,
For each video frame, it is determined whether to perform filter processing, and when it is determined to perform the filter processing, an in-loop filter determining means for determining an in-loop filter characteristic for performing the filter processing;
Input image filter processing means for performing filter processing on the input video frame with a filter having characteristics determined by the in-loop filter determination means;
Image encoding means for encoding the filtered video frame;
Local decoding means for generating a prediction reference frame obtained by decoding the video frame encoded by the image encoding means;
First storage means for storing the prediction reference frame;
In-loop filter processing means for performing filter processing on the prediction reference frame with a filter having characteristics determined by the in-loop filter determining means;
A predictive image generating means for generating a predictive image from the predictive reference frame subjected to the filter processing by the in-loop filter processing means;
The input image filter processing means and the in-loop filter processing means perform filter processing with a filter having the same characteristics as described above,
The moving image processing apparatus, wherein the image encoding unit encodes a result of a subtraction process between the video frame output from the input image filter processing unit and the predicted image generated by the predicted image generating unit. Decoding means for decoding the encoded video frame;
Second storage means for storing the video frame decoded by the decoding means;
Second in-loop filter processing means for performing filter processing on the video frame stored in the second storage means with a filter having characteristics determined by the in-loop filter determining means;
Second predicted image generation means for generating a prediction image of motion compensated prediction from the video frame on which the second in-loop filter processing means has performed the filter processing;
The moving image processing apparatus according to claim 10, further comprising an adding unit that adds and outputs the video frame decoded by the decoding unit and the predicted image generated by the second predicted image generating unit. . In a moving image processing method for processing a moving image composed of continuous video frames,
A blur intensity determining step for determining the intensity of blurring the video frame;
An image blur step of blurring the image by removing high frequency signals of the video frame according to the determination in the blur intensity determination step;
The blur intensity determining step includes a step of changing an intensity of blurring when a predetermined number of video frames continue. The moving image processing method according to claim 12, wherein the blur intensity determination step includes a step of periodically changing an intensity to be blurred. The moving image processing method according to claim 12 or 13, wherein the blur intensity determining step includes a step of changing an intensity to be blurred every other frame. The moving image processing method according to claim 14, wherein the blur intensity determination step includes a step of determining an intensity to be blurred according to a change in a pixel value between video frames. An image encoding step for encoding the video frame output in the image blurring step;
The moving image processing method according to claim 14, wherein the blur intensity determining step includes a step of determining an intensity to be blurred according to an encoding method when encoding a video frame in the image encoding step. An image encoding step for encoding the video frame output in the image blurring step;
15. The blur intensity determining step includes a step of determining an intensity to be blurred according to an encoding method when encoding a video frame in the image encoding step and a pixel value change between the video frames. The moving image processing method described. A local decoding step of decoding the video frame encoded in the image encoding step to generate a prediction reference frame;
A predicted reference frame blurring step for removing a high-frequency signal of the predicted reference frame;
A prediction image generation step of generating a prediction image of motion compensation prediction from the prediction reference frame from which the high frequency signal has been removed in the prediction reference frame blurring step;
The predicted reference frame blurring step includes:
Removing the high frequency signal determined to be removed from the video frame in the blur intensity determining step from the predicted reference frame;
The encoding step includes encoding a result of a subtraction process between the video frame output in the image blurring step and the prediction image generated in the prediction image generation step. Item 18. A moving image processing method according to Item 17. A decoding step of decoding the video frame encoded in the image encoding step;
A second predicted image generation step of generating a motion compensated predicted image from the video frame decoded in the decoding step;
The moving image processing method according to claim 18, further comprising: an adding step of adding and outputting the video frame decoded in the decoding step and the predicted image generated in the second predicted image generating step. . A decoding step of decoding the video frame encoded in the image encoding step;
A second prediction reference frame blurring step of removing the high-frequency signal determined to be removed in the blur intensity determination step from the video frame decoded in the decoding step;
A second predicted image generating step of generating a predicted image of motion compensated prediction from the video frame from which the high frequency signal has been removed in the second predicted reference frame blurring step;
The moving image processing method according to claim 18, further comprising: an adding step of adding and outputting the video frame decoded in the decoding step and the predicted image generated in the second predicted image generating step. . In a moving image processing method for processing a moving image composed of continuous video frames,
For each video frame, it is determined whether to perform filter processing, and when it is determined to perform the filter processing, an in-loop filter determination step for determining an in-loop filter characteristic for performing the filter processing;
An input image filter processing step for performing filter processing on the input video frame with a filter having the characteristics determined in the in-loop filter determination step;
An image encoding step for encoding the filtered video frame;
A local decoding step of generating a prediction reference frame obtained by decoding the video frame encoded in the image encoding step;
An in-loop filter processing step for filtering the prediction reference frame with a filter having the characteristics determined in the in-loop filter determination step;
A prediction image generation step of generating a prediction image from the prediction reference frame in which the in-loop filter processing step has performed the filter processing;
The input image filtering step and the in-loop filtering step include
Filtering with a filter having the same characteristics as above,
The video encoding step includes a step of encoding a result of a subtraction process between the video frame output in the input image filter processing step and the prediction image generated in the prediction image generation step. Processing method. A decoding step of decoding the encoded video frame;
A second in-loop filter processing step of performing filtering on the video frame decoded in the decoding step with a filter having the characteristics determined in the in-loop filter determining step;
A second predicted image generation step of generating a predicted image of motion compensated prediction from the video frame subjected to the filter processing in the second in-loop filter processing step;
The moving image processing method according to claim 21, further comprising: an adding step of adding and outputting the video frame decoded in the decoding step and the predicted image generated in the second predicted image generating step. . In a moving image processing program for causing a computer to execute a step of processing a moving image composed of continuous video frames,
The moving image processing step includes
A blur intensity determining step for determining the intensity of blurring the video frame;
An image blur step of blurring the image by removing the high frequency signal of the video frame according to the determination in the blur intensity determination step;
The moving image processing program characterized in that the blur intensity determining step includes a step of changing an intensity of blurring when a predetermined number of video frames continue. 24. The moving image processing program according to claim 23, wherein the blur intensity determination step includes a step of periodically changing an intensity to be blurred. 25. The moving image processing program according to claim 23, wherein the blur intensity determining step includes a step of changing an intensity to be blurred every other frame. 26. The moving image processing program according to claim 25, wherein the blur intensity determining step includes a step of determining an intensity to be blurred according to a change in pixel value between video frames. An image encoding step for encoding the video frame output in the image blurring step;
26. The moving image processing program according to claim 25, wherein the blur intensity determining step includes a step of determining an intensity to be blurred according to an encoding method when encoding a video frame in the image encoding step. An image encoding step for encoding the video frame output in the image blurring step;
26. The blur intensity determination step includes a step of determining an intensity to be blurred according to an encoding method when encoding a video frame in the image encoding step and a change in pixel value between the video frames. The moving image processing program described. A local decoding step of decoding the video frame encoded in the image encoding step to generate a prediction reference frame;
A predicted reference frame blurring step for removing a high-frequency signal of the predicted reference frame;
A prediction image generation step of generating a prediction image of motion compensation prediction from the prediction reference frame from which the high frequency signal has been removed in the prediction reference frame blurring step;
The predicted reference frame blurring step includes:
Removing the high frequency signal determined to be removed from the video frame in the blur intensity determining step from the predicted reference frame;
28. The step according to claim 27, wherein the image encoding step includes a step of encoding a result of a subtraction process between the video frame output in the image blurring step and the prediction image generated in the prediction image generation step. Item 28. A moving image processing program according to Item 28. A decoding step of decoding the video frame encoded in the image encoding step;
A second predicted image generation step of generating a motion compensated predicted image from the video frame decoded in the decoding step;
30. The moving image processing program according to claim 29, further comprising: an adding step of adding and outputting the video frame decoded in the decoding step and the predicted image generated in the second predicted image generating step. . A decoding step of decoding the video frame encoded in the image encoding step;
A second prediction reference frame blurring step of removing the high-frequency signal determined to be removed in the blur intensity determination step from the video frame decoded in the decoding step;
A second predicted image generating step of generating a predicted image of motion compensated prediction from the video frame from which the high frequency signal has been removed in the second predicted reference frame blurring step;
30. The moving image processing program according to claim 29, further comprising: an adding step of adding and outputting the video frame decoded in the decoding step and the predicted image generated in the second predicted image generating step. . In a moving image processing program for causing a computer to execute a step of processing a moving image composed of continuous video frames,
For each video frame, it is determined whether to perform filter processing, and when it is determined to perform the filter processing, an in-loop filter determination step for determining an in-loop filter characteristic for performing the filter processing;
An input image filter processing step for performing filter processing on the input video frame with a filter having the characteristics determined in the in-loop filter determination step;
An image encoding step for encoding the filtered video frame;
A local decoding step of generating a prediction reference frame obtained by decoding the video frame encoded in the image encoding step;
An in-loop filter processing step for filtering the prediction reference frame with a filter having the characteristics determined in the in-loop filter determination step;
A prediction image generation step of generating a prediction image from the prediction reference frame in which the in-loop filter processing step has performed the filter processing;
The input image filtering step and the in-loop filtering step include
Filtering with a filter having the same characteristics as above,
The video encoding step includes a step of encoding a result of a subtraction process between the video frame output in the input image filter processing step and the prediction image generated in the prediction image generation step. Processing program. A decoding step of decoding the encoded video frame;
A second in-loop filter processing step of performing filtering on the video frame decoded in the decoding step with a filter having the characteristics determined in the in-loop filter determining step;
A second predicted image generation step of generating a predicted image of motion compensated prediction from the video frame subjected to the filter processing in the second in-loop filter processing step;
The moving image processing program according to claim 32, further comprising an adding step of adding and outputting the video frame decoded in the decoding step and the predicted image generated in the second predicted image generating step. .

Images