JP6306883B2 - Video encoding method, video decoding method, video encoding device, video decoding device, video encoding program, video decoding program, and recording medium - Google Patents

Description

  The present invention relates to a video encoding method, a video decoding method, a video encoding device, a video decoding device, a video encoding program, a video decoding program, and a recording medium.

  In general video encoding, each frame of video is divided into processing unit blocks using spatial / temporal continuity of the subject, and the video signal is predicted spatially / temporally for each block. By encoding the prediction information indicating the prediction method and the prediction residual signal, the encoding efficiency is greatly improved as compared with the case of encoding the video signal itself. In general 2D video coding, intra prediction for predicting a signal to be encoded with reference to an already encoded block in the same frame, motion compensation with reference to another already encoded frame, etc. Based on the above, inter-frame prediction for predicting the encoding target signal is performed.

  Here, multi-view video encoding will be described. Multi-view video encoding is to encode a plurality of videos obtained by photographing the same scene with a plurality of cameras with high efficiency by using redundancy between the videos. Multi-view video coding is detailed in Non-Patent Document 1. In multi-view video encoding, in addition to the prediction method used in general video encoding, between the viewpoints that predict the encoding target signal based on parallax compensation with reference to video of another viewpoint that has already been encoded. Inter-viewpoint residual prediction that predicts a signal to be encoded by prediction and interframe prediction, and predicts the residual signal with reference to the residual signal at the time of encoding another viewpoint video that has already been encoded The method is used. Inter-view prediction is treated as inter prediction together with inter-frame prediction in multi-view video coding such as MVC (Multiview Video Coding), and two or more predicted images are interpolated into a predicted image in a B picture. It can also be used for direction prediction. As described above, in multi-view video encoding, bi-directional prediction based on inter-frame prediction and inter-view prediction can be performed on a picture that can perform both inter-frame prediction and inter-view prediction.

  When performing inter prediction, it is necessary to obtain reference information such as a reference picture index and a motion vector indicating the reference destination. In general, the reference information is encoded as prediction information and multiplexed with the video, but the reference information may be predicted by some method in order to reduce the code amount. In a general method, the prediction information used when the peripheral block of the encoding target image that has already been encoded is used for encoding and the reference information used for prediction of the encoding target image is used. There is a merge mode in which information is listed as a candidate list and an identifier for identifying a target block from which prediction information is obtained is encoded from the list.

  Another method is residual prediction. Residual prediction is a method for suppressing the amount of code of a prediction residual using the fact that when two images having high correlation are predictively encoded, the prediction residuals are also correlated with each other. The residual prediction is detailed in Non-Patent Document 2. In inter-view residual prediction used in multi-view video encoding, by subtracting the prediction residual signal at the time of encoding the region corresponding to the encoding target image in the video of different viewpoint from the prediction residual signal of the encoding target. It is possible to reduce the energy of the residual signal and improve the encoding efficiency. The correspondence between viewpoints is, for example, a method of setting an area of another viewpoint corresponding to a block to be encoded by the disparity vector when an already encoded peripheral block is encoded by parallax compensation prediction. Is required. The disparity vector obtained by this method is called “Disparity vector from Neighboring blocks” (NBDV). Inter-view residual prediction is used as a further process for the residual separately from the prediction when inter-frame prediction is used in a B picture.

  In addition, when the original prediction method is inter prediction using a reference picture of a viewpoint different from the encoding target viewpoint, the prediction remaining at the time of encoding of the area corresponding to the encoding target image in a frame different from the encoding target frame. There is also a method of performing residual prediction in the same manner using a difference signal. In either case, instead of referring to the prediction residual at the time of encoding the corresponding region, a predicted image is generated using the same motion information as that used in the encoding target image for the corresponding region. There is a method of generating a prediction value of a prediction residual by taking a difference from an image of a corresponding region.

  In the present specification, an image is one frame or a still image of a moving image, and a collection of a plurality of frames (images) (moving image) is referred to as a video.

M. Flierl and B. Girod, "Multiview video compression," Signal Processing Magazine, IEEE, no. November 2007, pp. 66-76, 2007. X. Wang and J. Ridge, "Improved video coding with residual prediction for extended spatial scalability," Communications, Control and Signal Processing, 2008. ISCCSP 2008. 3rd International Symposium on, no. March, pp. 1041-1046, 2008.

  Residual prediction is an effective code amount reduction method in multi-view video encoding. However, the prediction accuracy largely depends on the accuracy of correspondence between viewpoints. When the accuracy of correspondence between viewpoints is not sufficient, there is a gap between the predicted prediction residual and the predicted image, so that the prediction residual of the encoding target cannot be sufficiently reduced or noise occurs in the decoded image. There is a problem that a sufficient effect cannot be obtained. Also, the same problem occurs when the signal characteristics differ greatly between viewpoints, when noise is added to the video to be encoded, or when distortion caused by encoding is added to each reference picture. There is also a problem.

  The present invention has been made in view of such circumstances, and a video encoding method, a video decoding method, a video encoding device, a video decoding device, and a video encoding capable of improving encoding efficiency or decoded video quality. An object is to provide a program, a video decoding program, and a recording medium.

  The present invention is a video encoding method that further performs residual prediction on an image generated by inter-screen prediction from a reference picture when predicting a prediction target region and generating a predicted image, A residual prediction step of generating a prediction prediction residual from the reference picture by the inter-screen prediction, and a prediction prediction residual update step of updating the prediction prediction residual to be a new prediction prediction residual. And

  The present invention provides a primary prediction image generation step for generating a primary prediction image from the reference picture by the inter-screen prediction, a prediction image generation step for generating the prediction image from the primary prediction image and the prediction prediction residual, and It is characterized by having.

  The present invention provides a prediction image generation step of generating the prediction image by the inter-screen prediction from the reference picture, and a prediction residual generation for generating an encoding target prediction residual from the prediction prediction residual and the prediction residual And further comprising steps.

  The present invention is characterized in that in the prediction prediction residual update step, a prediction image is updated by applying a filter to the prediction prediction residual.

  The present invention is an encoding method for further performing residual prediction on an image generated by inter-screen prediction from a reference picture when performing prediction on a prediction target region and generating a prediction image. It has a residual prediction step of generating a prediction prediction residual by inter-screen prediction using a filter from a picture.

  The present invention further includes a filter generation step of generating the filter.

The present invention further includes a filter determination step for determining the filter, wherein the filter determination step determines the filter based on information of the prediction prediction residual or primary prediction image.

The present invention is an encoding method that further performs residual prediction on an image generated by inter-screen prediction from a reference picture when predicting a prediction target region and generating a predicted image,
A filter determining step for determining a filter, and a residual prediction step for generating a prediction prediction residual by inter-screen prediction using the filter from the reference picture, and in the filter determination step, information on the primary prediction image The filter is determined based on:

  The present invention is a video decoding method for further performing residual prediction on an image generated by inter-screen prediction from a reference picture when performing prediction on a prediction target region and generating a prediction image, A residual prediction step for generating a prediction prediction residual from the picture by the inter-screen prediction, and a prediction prediction residual update step for updating the prediction prediction residual to be a new prediction prediction residual, To do.

  The present invention includes a primary prediction image generation step of generating a primary prediction image from the reference picture by the inter-screen prediction, a prediction image generation step of generating the prediction image from the primary prediction image and the prediction prediction residual, Furthermore, it is characterized by having.

  The present invention provides a prediction image generation step of generating the prediction image by the inter-screen prediction from the reference picture, and a prediction residual generation for generating an encoding target prediction residual from the prediction prediction residual and the prediction residual And further comprising steps.

  In the prediction prediction residual update step, the present invention is characterized in that the prediction image is updated by applying a filter to the prediction prediction residual.

  The present invention is a decoding method for further performing residual prediction on an image generated by inter-screen prediction from a reference picture when performing prediction on a prediction target region and generating a predicted image, wherein the reference picture And a residual prediction step of generating a prediction prediction residual by the inter-screen prediction using a filter.

  The present invention further includes a filter generation step of generating the filter.

The present invention further includes a filter determination step for determining the filter, wherein the filter determination step determines the filter based on information of the prediction prediction residual or primary prediction image.

The present invention is an encoding method that further performs residual prediction on an image generated by inter-screen prediction from a reference picture when predicting a prediction target region and generating a predicted image,
A filter determining step for determining a filter, and a residual prediction step for generating a prediction prediction residual by inter-screen prediction using the filter from the reference picture, and in the filter determination step, information on the primary prediction image The filter is determined based on:

  The present invention is a video encoding apparatus that further performs residual prediction on an image generated by inter-screen prediction from a reference picture when performing prediction on a prediction target region and generating a prediction image, A residual prediction unit that generates a prediction prediction residual by inter-screen prediction from a reference picture; and a prediction prediction residual update unit that updates the prediction prediction residual to obtain a new prediction prediction residual. To do.

  The present invention is a video decoding apparatus that further performs residual prediction on an image generated by inter-screen prediction from a reference picture when performing prediction on a prediction target region and generating a prediction image, A residual prediction unit that generates a prediction prediction residual from the picture by the inter-screen prediction, and a prediction prediction residual update unit that updates the prediction prediction residual to be a new prediction prediction residual. To do.

  The present invention is a video encoding program for causing a computer to execute the video encoding method.

  The present invention is a video decoding program for causing a computer to execute the video decoding method.

  The present invention is a computer-readable recording medium on which the video encoding program is recorded.

  The present invention is a computer-readable recording medium on which the video decoding program is recorded.

  According to the present invention, it is possible to improve encoding efficiency or decoded video quality by updating or filtering a residual prediction value by residual prediction and preventing noise mainly derived from the accuracy of residual prediction. An effect is obtained.

It is a block diagram which shows the structure of the video coding apparatus by 1st Embodiment of this invention. It is a flowchart which shows the processing operation of the video coding apparatus 100 shown in FIG. It is a block diagram which shows the structure of the video decoding apparatus by 1st Embodiment of this invention. It is a flowchart which shows the processing operation of the video decoding apparatus 200 shown in FIG. It is a block diagram which shows the structure of the video coding apparatus by 2nd Embodiment of this invention. It is a flowchart which shows the processing operation of the video coding apparatus 100a shown in FIG. It is a block diagram which shows the structure of the video decoding apparatus by 2nd Embodiment of this invention. It is a flowchart which shows the processing operation of the video decoding apparatus 200a shown in FIG.

<First Embodiment>
Hereinafter, a video encoding apparatus according to a first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration of a video encoding device 100 according to the first embodiment of the present invention. As shown in FIG. 1, the video encoding apparatus 100 includes an encoding target video input unit 101, an input video memory 102, a reference picture memory 103, a primary prediction image generation unit 104, a prediction prediction residual generation unit 105, a prediction prediction residual, A difference update unit 106, a predicted image generation unit 107, a subtraction unit 108, a transform / quantization unit 109, an inverse quantization / inverse transform unit 110, an addition unit 111, and an entropy encoding unit 112 are provided.

  The encoding target video input unit 101 inputs a video to be encoded. In the following description, the video to be encoded is referred to as an encoding target video, and a frame to be processed in particular is referred to as an encoding target frame or an encoding target picture. The input video memory 102 stores the input encoding target video. The reference picture memory 103 stores images that have been encoded and decoded so far. Hereinafter, this stored frame is referred to as a reference frame or reference picture.

  The primary prediction image generation unit 104 performs prediction on the encoding target region using the reference picture stored in the reference picture memory 103, and generates a primary prediction image. The prediction prediction residual generation unit 105 generates a prediction prediction residual using the reference picture stored in the reference picture memory 103 and the prediction information at the time of primary prediction image generation. The prediction prediction residual update unit 106 updates the generated prediction prediction residual to obtain a new prediction prediction residual. The predicted image generation unit 107 generates a predicted image from the predicted prediction residual and the primary predicted image. The subtraction unit 108 obtains a difference between the encoding target image and the predicted image, and generates a prediction residual.

  The transform / quantization unit 109 transforms / quantizes the generated prediction residual to generate quantized data. The inverse quantization / inverse transform unit 110 performs inverse quantization / inverse transform on the generated quantized data to generate a decoded prediction residual. The adding unit 111 adds the decoded prediction residual and the predicted image to generate a decoded image. The entropy encoding unit 112 entropy encodes the quantized data to generate code data.

  Next, the processing operation of the video encoding device 100 shown in FIG. 1 will be described with reference to FIG. FIG. 2 is a flowchart showing the processing operation of the video encoding apparatus 100 shown in FIG. Here, it is assumed that the encoding target video is one of the multi-view videos, and the multi-view video has a structure in which videos of all viewpoints are encoded and decoded for each frame. Here, a process of encoding one frame in the video to be encoded will be described. By repeating the processing described for each frame, video encoding can be realized.

  First, the encoding target video input unit 101 inputs an encoding target picture and stores it in the input video memory 102 (step S101). It is assumed that some frames in the video to be encoded have already been encoded and the decoding results are stored in the reference picture memory 103. In addition, it is assumed that a video of another viewpoint that can be referred to up to the same frame as the current picture to be coded is already coded and decoded and stored in the reference picture memory 103.

  After the video input, the encoding target picture is divided into encoding target blocks, and the video signal of the encoding target picture is encoded for each block (repetition loop of steps S102 to S112). Hereinafter, an image of a block to be encoded is referred to as an encoding target block or an encoding target image. The following steps S103 to S110 are repeatedly executed for all the blocks of the picture.

  In the process repeated for each encoding target block, first, the primary prediction image generation unit 104 performs inter prediction that refers to the reference picture in the reference picture memory for the encoding target block, and uses information indicating a reference destination. Certain motion information is determined, and a primary predicted image is generated from the reference destination image (step S103). The prediction may be performed by any method, and the motion information may be any method. Typical reference information includes a combination of reference picture index information for specifying a reference picture and a motion vector indicating a reference position on the reference picture.

  Common prediction methods include matching on candidate reference pictures to determine the reference destination, and prediction when encoding already-encoded peripheral blocks called direct mode or merge mode. There is a method to inherit the motion information. Any other prediction method and motion information may be used. Motion information may be encoded and multiplexed with video code data, or information that can identify motion information may be encoded and multiplexed separately, or can be derived from surrounding motion information or candidate lists as described above Need not be encoded. Further, motion information may be predicted and the residual may be encoded.

  Next, the prediction prediction residual generation unit 105 performs residual prediction on the encoding target region, and generates a prediction prediction residual (step S104). The residual prediction may be performed by any method. As a general method, another region on the reference picture in the reference picture memory corresponding to the encoding target region is used as a reference region, and a prediction residual at the time of encoding in the reference region is obtained. There are methods such as predictive residual. Alternatively, using the same motion information as that used in the encoding target image for the reference region, a prediction image for the reference region is generated, and a prediction prediction residual is generated by taking a difference from the image of the reference region There is a way to do it.

  The reference area may be determined by any method. In a general method, when a region corresponding to an image to be encoded in a video with a different viewpoint from the region to be encoded is set as a reference region, the reference is made by a motion vector used in the parallax compensation prediction of a peripheral block that has already been encoded. There is a method for determining the area.

  Next, the prediction prediction residual update unit 106 updates the generated prediction prediction residual to obtain a new prediction prediction residual (step S105). Any update method may be used. Hereinafter, as an example, an example will be described in which the prediction accuracy by residual prediction is improved when an error is expected in the correspondence between the reference region and the encoding target region. If the correspondence between the reference area and the encoding target area is expected to shift at an integer pixel level or a fractional pixel level, the prediction prediction residual generated by the residual prediction is also expected to be shifted to the same extent. The performance of the system is greatly reduced. In such a case, the error due to the deviation may be averaged by updating the prediction prediction residual by applying a smoothing filter or other low-pass filter.

  As another example, a case will be described in which some noise is present on the video to be encoded, and therefore the same kind of noise remains in the prediction prediction residual. In this case, the error may be reduced by updating the prediction prediction residual by applying some noise removal filter. In this case, the noise model may be estimated using the reference region and its predicted image and used for updating. Alternatively, when encoding distortion is on the reference picture, noise due to encoding distortion may be estimated based on a quantization parameter at the time of encoding the reference area.

  As another example, due to problems such as differences in accessible areas, the same motion information and reference picture of the same frame as those used in the encoding target image are not used for the reference area, and different motion information and different frames are used. When a prediction picture is generated for a reference area using a reference picture and used for prediction prediction residual generation, or when a prediction residual is obtained at the time of encoding in the reference area, a prediction prediction residual in the encoding target area is obtained. The improvement of the accuracy of residual prediction in the case of In this case, a deviation is also expected in the prediction prediction residual due to a difference or deviation in motion information. In such a case, information necessary for updating may be determined from the motion information used in each of the encoding target region and the reference region. For example, the kernel width or strength of the filter may be estimated from the difference between the motion vectors.

  In addition, an appropriate filter may be selected and used from predetermined filters. In any of the above examples, the parameters and filter coefficients used for the update, and information that can identify them may be encoded as additional information and multiplexed with the video.

  After updating the prediction prediction residual, the prediction image generation unit 107 generates a prediction image from the first prediction image and the prediction prediction residual (step S106). The predicted image may be generated in any way. In general, there is a method of generating a prediction image by adding a primary prediction image and a prediction prediction residual. Next, the subtraction unit 108 takes the difference between the prediction image and the encoding target block, and generates a prediction residual (step S107). Next, when the generation of the prediction residual is completed, the transform / quantization unit 109 converts and quantizes the prediction residual, and generates quantized data (step S108). For this transformation / quantization, any method can be used as long as it can be correctly inverse-quantized / inverse-transformed on the decoding side. Then, when the transform / quantization is completed, the inverse quantization / inverse transform unit 110 performs inverse quantization / inverse transform on the quantized data to generate a decoded prediction residual (step S109).

  Next, when the generation of the decoded prediction residual is completed, the addition unit 111 generates a decoded image by adding the decoded prediction residual and the predicted image, and stores the decoded image in the reference picture memory 103 (step S110). If necessary, a loop filter may be applied to the decoded image. In normal video coding, coding noise is removed using a deblocking filter or other filters.

  Next, the entropy encoding unit 112 generates encoded data by entropy encoding the quantized data, and if necessary, also encodes prediction information, residual prediction information, and other additional information, and multiplexes with the encoded data (step S111). ) When all the blocks have been processed, code data is output (step S112).

  Next, the video decoding apparatus according to the first embodiment of the present invention will be described. FIG. 3 is a block diagram showing a configuration of the video decoding apparatus 200 according to the first embodiment of the present invention. As shown in FIG. 3, the video decoding apparatus 200 includes a code data input unit 201, a code data memory 202, a reference picture memory 203, an entropy decoding unit 204, an inverse quantization / inverse transform unit 205, a primary prediction image generation unit 206, A prediction prediction residual generation unit 207, a prediction prediction residual update unit 208, a prediction image generation unit 209, and an addition unit 210 are provided.

  The code data input unit 201 inputs video code data to be decoded. This video code data to be decoded is called decoding target video code data, and a frame to be processed in particular is called a decoding target frame or a decoding target picture. The code data memory 202 stores the input decoding target video. The reference picture memory 203 stores an already decoded image. The entropy decoding unit 204 entropy-decodes the code data of the decoding target picture to generate quantized data, and the inverse quantization / inverse transform unit 205 performs inverse quantization / inverse transformation on the quantized data to obtain a decoded prediction residual. Generate.

  The primary prediction image generation unit 206 performs prediction on the encoding target region using the reference picture stored in the reference picture memory 203, and generates a primary prediction image. The prediction prediction residual generation unit 207 generates a prediction prediction residual using the reference picture stored in the reference picture memory 203 and prediction information at the time of primary prediction image generation. The prediction prediction residual update unit 208 updates the generated prediction prediction residual to obtain a new prediction prediction residual. The predicted image generation unit 209 generates a predicted image from the predicted prediction residual and the primary predicted image. The adding unit 210 adds the decoded prediction residual and the predicted image to generate a decoded image.

  Next, the processing operation of the video decoding apparatus 200 shown in FIG. 3 will be described with reference to FIG. FIG. 4 is a flowchart showing the processing operation of the video decoding apparatus 200 shown in FIG. It is assumed that the decoding target video is one of the multi-view videos, and the multi-view video has a structure in which the videos of all viewpoints are decoded one by one for each frame. Here, a process of decoding one frame in the code data will be described. By repeating the processing described for each frame, video decoding can be realized.

  First, the code data input unit 201 inputs code data and stores it in the code data memory 202 (step S201). It is assumed that some frames in the video to be decoded have already been decoded and the decoding results are stored in the reference picture memory 203. Also, it is assumed that the video of another viewpoint that can be referred to up to the same frame as the decoding target picture has already been decoded, decoded, and stored in the reference picture memory 203.

  Next, after the code data is input, the decoding target picture is divided into decoding target blocks, and the video signal of the decoding target picture is decoded for each block (repetition loop of steps S202 to S210). Hereinafter, an image of a block to be decoded is referred to as a decoding target block or a decoding target image. The processes in steps S203 to S209 are repeatedly executed for all blocks in the frame.

  In the process repeated for each decoding target block, first, the entropy decoding unit 204 performs entropy decoding on the code data (step S203). The inverse quantization / inverse transform unit 205 performs inverse quantization / inverse transformation to generate a decoded prediction residual (step S204). When the prediction data and other additional information are included in the code data, they may be decoded to generate necessary information as appropriate.

  Since the processing from step S205 to step S208 is the same as the processing from step S103 to step S106 in the video encoding device 100, it will be briefly described here. The primary prediction image generation unit 206 performs inter prediction with reference to the reference picture in the reference picture memory for the encoding target block, determines motion information that is information indicating a reference destination, and performs primary prediction from the reference destination image. A predicted image is generated (step S205). Next, the prediction prediction residual generation unit 207 performs residual prediction on the encoding target region, and generates a prediction prediction residual (step S206). Next, the prediction prediction residual update unit 208 updates the generated prediction prediction residual to obtain a new prediction prediction residual (step S207). After updating the prediction prediction residual, the prediction image generation unit 209 generates a prediction image from the primary prediction image and the prediction prediction residual (step S298).

  Next, when the generation of the predicted image is completed, the adding unit 210 adds the decoded prediction residual and the predicted image, generates a decoded image, and stores the decoded image in the reference picture memory (step S209). If necessary, the decoded image may be further subjected to a loop filter. In normal video decoding, a coding noise is removed using a deblocking filter or other filters. When all the blocks are processed, the decoded frame is output (step S210).

Second Embodiment
Next, a video encoding apparatus according to the second embodiment of the present invention will be described. FIG. 5 is a block diagram showing a configuration of a video encoding device 100a according to the second embodiment of the present invention. In this figure, the same parts as those in the apparatus shown in FIG. The apparatus shown in this figure is different from the apparatus shown in FIG. 1 in that the prediction prediction residual update unit 106 is omitted and a filter determination unit 113 is newly provided. The filter determination unit 113 determines a filter to be used for residual prediction. Also, the prediction prediction residual generation unit 105 performs residual prediction using the determined filter and generates a prediction prediction residual.

  Next, the processing operation of the video encoding device 100a shown in FIG. 5 will be described with reference to FIG. FIG. 6 is a flowchart showing the processing operation of the video encoding device 100a shown in FIG. In FIG. 6, the same parts as those shown in FIG.

  First, the encoding target video input unit 101 inputs an encoding target picture and stores it in the input video memory 102 (step S101). After the video input, the encoding target picture is divided into encoding target blocks, and the video signal of the encoding target picture is encoded for each block (repetition loop of steps S102 to S112). Hereinafter, an image of a block to be encoded is referred to as an encoding target block or an encoding target image. The following steps S103 to S111 are repeatedly executed for all blocks of the picture.

  In the process repeated for each encoding target block, first, the primary prediction image generation unit 104 performs inter prediction that refers to the reference picture in the reference picture memory for the encoding target block, and uses information indicating a reference destination. Certain motion information is determined, and a primary predicted image is generated from the reference destination image (step S103).

  Next, the filter determination unit 113 determines a filter to be used for residual prediction (step S104a). After determining the filter, the prediction prediction residual generation unit 105 uses the determined filter to predict the prediction prediction residual. Is generated (step S105a). In residual prediction, an image of a reference region and a predicted image thereof are generated by general inter-screen prediction. In general, inter-screen prediction uses an interpolation filter to acquire an image in units of decimal pixels. As described above for the first embodiment, various noises and shifts are expected in the prediction prediction residual generated by the normal residual prediction. A reference region image and its prediction image in residual prediction are generated by performing inter-screen prediction using a filter that corrects or disperses such noise and deviation, and is used for prediction prediction residual generation. Also good. Alternatively, the reference region image generated by the normal inter-screen prediction and the predicted image may be filtered to obtain a new image, which may be used for generating a predicted prediction residual.

  Or, generally, inter-screen prediction uses a high-accuracy interpolation filter such as 6 taps to 8 taps, but a certain amount of error is expected in the information indicating the reference region and the information indicating the prediction destination in the residual prediction. In this case, the amount of calculation may be reduced by adaptively lowering the accuracy of the interpolation filter. Any method may be used for generating the filter. The method described in the first embodiment may be used, or another method may be used.

  In addition, an appropriate filter may be selected and used from predetermined filters. In any of the above examples, the parameters and filter coefficients used for the update, and information that can identify them may be encoded as additional information and multiplexed with the video. Moreover, you may use the filter determined here for primary prediction image generation.

  The method of generating the prediction prediction residual is as described in the first embodiment, but using a filter in which the image of the reference region, its prediction image, or the prediction residual at the time of encoding the reference region is determined. Or a prediction prediction residual may be generated. A prediction prediction residual may be generated by performing filter processing on those generated in normal inter-screen prediction.

  After updating the prediction prediction residual, the prediction image generation unit 107 generates a prediction image from the first prediction image and the prediction prediction residual (step S106). Next, the subtraction unit 108 takes the difference between the prediction image and the encoding target block, and generates a prediction residual (step S107). Next, when the generation of the prediction residual is completed, the transform / quantization unit 109 converts and quantizes the prediction residual, and generates quantized data (step S108). Then, when the transform / quantization is completed, the inverse quantization / inverse transform unit 110 performs inverse quantization / inverse transform on the quantized data to generate a decoded prediction residual (step S109).

  Next, when the generation of the decoded prediction residual is completed, the addition unit 111 generates a decoded image by adding the decoded prediction residual and the predicted image, and stores the decoded image in the reference picture memory 103 (step S110). If necessary, a loop filter may be applied to the decoded image. In normal video coding, coding noise is removed using a deblocking filter or other filters.

  Next, the entropy encoding unit 112 generates encoded data by entropy encoding the quantized data, and if necessary, also encodes prediction information, residual prediction information, and other additional information, and multiplexes with the encoded data (step S111). ) When all the blocks have been processed, code data is output (step S112).

  Next, a video decoding apparatus according to the second embodiment of the present invention will be described. FIG. 7 is a block diagram showing a configuration of a video decoding apparatus 200a according to the second embodiment of the present invention. In this figure, the same parts as those in the apparatus shown in FIG. The apparatus shown in this figure differs from the apparatus shown in FIG. 3 in that the prediction prediction residual update unit 208 is omitted and a filter determination unit 211 is newly provided. The filter determination unit 211 determines a filter to be used for residual prediction. Also, the prediction prediction residual generation unit 207 performs residual prediction using the determined filter and generates a prediction prediction residual.

  Next, the processing operation of the video decoding apparatus 200a shown in FIG. 7 will be described with reference to FIG. FIG. 8 is a flowchart showing the processing operation of the video decoding apparatus 200a shown in FIG. In FIG. 8, the same parts as those shown in FIG. First, the code data input unit 201 inputs code data and stores it in the code data memory 202 (step S201). It is assumed that some frames in the video to be decoded have already been decoded and the decoding results are stored in the reference picture memory 203. Also, it is assumed that the video of another viewpoint that can be referred to up to the same frame as the decoding target picture has already been decoded, decoded, and stored in the reference picture memory 203.

  Next, after the code data is input, the decoding target picture is divided into decoding target blocks, and the video signal of the decoding target picture is decoded for each block (repetition loop of steps S202 to S210). Hereinafter, an image of a block to be decoded is referred to as a decoding target block or a decoding target image. The processes in steps S203 to S209 are repeatedly executed for all blocks in the frame.

  In the process repeated for each decoding target block, first, the entropy decoding unit 204 performs entropy decoding on the code data (step S203). The inverse quantization / inverse transform unit 205 performs inverse quantization / inverse transformation to generate a decoded prediction residual (step S204). When the prediction data and other additional information are included in the code data, they may be decoded to generate necessary information as appropriate.

  Since the processing from step S205 to step S208 is the same as the processing from step S103 to step S106 in the video encoding device 100, it will be briefly described here. The primary prediction image generation unit 206 performs inter prediction with reference to the reference picture in the reference picture memory for the encoding target block, determines motion information that is information indicating a reference destination, and performs primary prediction from the reference destination image. A predicted image is generated (step S205).

  Next, the filter determination unit 211 determines a filter to be used for residual prediction (step S206a). When the filter is determined, the prediction prediction residual generation unit 207 generates a prediction prediction residual using the determined filter (step S207a). The details are the same as those of the video encoding device 100a.

  Next, the prediction prediction residual generation unit 207 performs residual prediction on the encoding target region, and generates a prediction prediction residual (step S206). Next, the prediction prediction residual update unit 208 updates the generated prediction prediction residual to obtain a new prediction prediction residual (step S207). After updating the prediction prediction residual, the prediction image generation unit 209 generates a prediction image from the primary prediction image and the prediction prediction residual (step S298).

  Next, when the generation of the predicted image is completed, the adding unit 210 adds the decoded prediction residual and the predicted image, generates a decoded image, and stores the decoded image in the reference picture memory (step S209). If necessary, the decoded image may be further subjected to a loop filter. In normal video decoding, a coding noise is removed using a deblocking filter or other filters. When all the blocks are processed, the decoded frame is output (step S210).

  In the above-described embodiment, the case where the prediction image is updated using the generated prediction prediction residual to obtain a new prediction image has been described. However, after the prediction residual is determined using the primary prediction image as it is as the prediction image, the prediction is performed. The prediction residual may be updated using the prediction residual as the encoding target prediction residual. In the above-described embodiment, the case where the video to be encoded is one of the multi-view videos has been described. However, in the case where the video to be encoded is one video of the scalable video, the videos that are correlated with each other are encoded together. In the case of multiplex and multiplexing, noise of residual prediction may be reduced by a similar method for a video to which residual prediction can be applied by a similar method.

  The order of some processes in the first and second embodiments described above may be changed.

  As described above, the encoding efficiency or the decoded video quality can be improved by updating or filtering the residual prediction value based on the residual prediction to prevent noise mainly derived from the accuracy of the residual prediction.

  You may make it implement | achieve the video encoding apparatus and video decoding apparatus in embodiment mentioned above with a computer. In that case, a program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into a computer system and executed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory inside a computer system serving as a server or a client in that case may be included and a program held for a certain period of time. Further, the program may be for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in the computer system. It may be realized using hardware such as PLD (Programmable Logic Device) or FPGA (Field Programmable Gate Array).

  As mentioned above, although embodiment of this invention has been described with reference to drawings, the said embodiment is only the illustration of this invention, and it is clear that this invention is not limited to the said embodiment. is there. Therefore, additions, omissions, substitutions, and other modifications of the components may be made without departing from the technical idea and scope of the present invention.

  The present invention can be applied to applications where it is indispensable to improve encoding efficiency or decoded video quality by updating or filtering residual prediction values based on residual prediction to prevent noise mainly derived from the accuracy of residual prediction.

  DESCRIPTION OF SYMBOLS 100 ... Video coding apparatus, 101 ... Encoding target video input unit, 102 ... Input video memory, 103 ... Reference picture memory, 104 ... Primary prediction image generation unit, 105 ... Prediction prediction residual generation unit, 106 ... Prediction prediction residual update unit, 107 ... Prediction image generation unit, 108 ... Subtraction unit, 109 ... Conversion / quantization unit, 110 ... Inverse quantum Conversion unit, 111 ... adder, 112 ... entropy encoding unit, 113 ... filter determination unit, 200 ... video decoding device, 201 ... code data input unit, 202 ... Code data memory, 203 ... reference picture memory, 204 ... entropy decoding unit, 205 ... inverse quantization / inverse transformation unit, 206 ... primary prediction image generation unit, 207 ... prediction prediction residual Difference generator, 208... Prediction residual updating unit measurement, 209 ... prediction image generation unit, 210 ... adding unit, 211 ... filter determining unit

Claims (12)

A video encoding method that further performs residual prediction on an image generated by inter-screen prediction from a reference picture when predicting a prediction target region and generating a predicted image,
A residual prediction step of generating a prediction prediction residual by the inter-screen prediction from the reference picture;
A prediction prediction residual update step for updating the prediction prediction residual to be a new prediction prediction residual ;
A primary prediction image generating step of generating a primary prediction image by the inter-screen prediction from the reference picture;
A predicted image generation step of generating the predicted image from the primary predicted image and the predicted prediction residual;
A prediction residual generating step for generating an encoding target prediction residual from the prediction prediction residual and the prediction residual;
Have
In the prediction prediction residual update step, the prediction prediction residual is updated by applying a filter to the prediction prediction residual . A filter determining step for determining the filter;
The video encoding method according to claim 1, wherein in the filter determination step, the filter is determined based on information of the prediction prediction residual or primary prediction image. An encoding method that further performs residual prediction on an image generated by inter-screen prediction from a reference picture when predicting a prediction target region and generating a predicted image,
A filter determination step for determining a filter;
A residual prediction step for generating a prediction prediction residual by inter-screen prediction using the filter from the reference picture;
Have
In the filter determining step, the filter is determined based on information of a primary predicted image. A video decoding method that further performs residual prediction on an image generated by inter-screen prediction from a reference picture when predicting a prediction target region and generating a predicted image,
A residual prediction step of generating a prediction prediction residual by the inter-screen prediction from the reference picture;
A prediction prediction residual update step for updating the prediction prediction residual to be a new prediction prediction residual ;
A primary prediction image generating step of generating a primary prediction image by the inter-screen prediction from the reference picture;
A predicted image generation step of generating the predicted image from the primary predicted image and the predicted prediction residual;
A prediction residual generating step for generating an encoding target prediction residual from the prediction prediction residual and the prediction residual;
Have
In the predictive prediction residual update step, the predictive image is updated by applying a filter to the predictive predictive residual . A filter determining step for determining the filter;
5. The video decoding method according to claim 4, wherein, in the filter determination step, the filter is determined based on information of the prediction prediction residual or primary prediction image. An encoding method that further performs residual prediction on an image generated by inter-screen prediction from a reference picture when predicting a prediction target region and generating a predicted image,
A filter determination step for determining a filter;
A residual prediction step for generating a prediction prediction residual by inter-screen prediction using the filter from the reference picture;
Have
In the filter determining step, the filter is determined based on information of a primary predicted image. A video encoding device that further performs residual prediction on an image generated by inter-screen prediction from a reference picture when predicting a prediction target region and generating a predicted image,
Residual prediction means for generating a prediction prediction residual by inter-screen prediction from the reference picture;
A prediction prediction residual update unit that updates the prediction prediction residual to obtain a new prediction prediction residual ;
Primary predicted image generation means for generating a primary predicted image from the reference picture by the inter-screen prediction;
Predicted image generation means for generating the predicted image from the primary predicted image and the predicted prediction residual;
A prediction residual generating means for generating an encoding target prediction residual from the prediction prediction residual and the prediction residual;
With
The predictive prediction residual update unit updates the predictive predictive residual by applying a filter to the predictive predictive residual . A video decoding device that further performs residual prediction on an image generated by inter-screen prediction from a reference picture when predicting a prediction target region and generating a predicted image,
Residual prediction means for generating a prediction prediction residual from the reference picture by the inter-screen prediction;
A prediction prediction residual update unit that updates the prediction prediction residual to obtain a new prediction prediction residual ;
Primary predicted image generation means for generating a primary predicted image from the reference picture by the inter-screen prediction;
Predicted image generation means for generating the predicted image from the primary predicted image and the predicted prediction residual;
A prediction residual generating means for generating an encoding target prediction residual from the prediction prediction residual and the prediction residual;
With
The prediction decoding residual update unit applies a filter to the prediction prediction residual to update the prediction image . A video encoding program for causing a computer to execute the video encoding method according to any one of claims 1 to 3 . A video decoding program for causing a computer to execute the video decoding method according to any one of claims 4 to 6 . A computer-readable recording medium on which the video encoding program according to claim 9 is recorded. The computer-readable recording medium which recorded the video decoding program of Claim 10 .

Images