JP2006237759A - Moving picture re-encoding method, moving picture re-encoding apparatus, moving picture re-encoding program, and computer-readable recording medium with the program recorded therein - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide technology of achieving re-encoding whereby image quality deterioration is hardly caused and efficient re-encoding with less image quality deterioration in the case of carrying out the re-encoding by decoding the coded stream of a moving picture encoded by in-loop filter processing. <P>SOLUTION: The technology adopts a configuration such that when the coded stream of a moving picture encoded by the in-loop filter processing is decoded to carry out re-encoding, an image just before the in-loop filter processing in a decoding section is outputted, and the re-encoding is applied to the video image by completely or almost taking over coding parameters of the pre-stage. Through the configuration above, even the moving picture coding system in which the in-loop filter processing such as the H.264 decodes the image after the in-loop filter processing can achieve the re-encoding wherein image quality deterioration hardly occurs and the efficient re-encoding with less image quality deterioration. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a moving image re-encoding method and apparatus for decoding and re-encoding a moving image encoded stream encoded with in-loop filtering, and to realize the moving image re-encoding method. The present invention relates to a moving image re-encoding program to be used and a computer-readable recording medium on which the program is recorded. Particularly, the re-encoding that realizes re-encoding with little image quality degradation and less image quality degradation is realized. The present invention relates to a moving image re-encoding method and apparatus for realizing the above, a moving image re-encoding program used for realizing the moving image re-encoding method, and a computer-readable recording medium storing the program.

  Conventionally, in a video encoding method such as MPEG-2, re-encoding is performed by inheriting all encoding parameters other than the DCT coefficient in the previous stage, so that re-encoding can be performed with almost no image quality degradation. It is known (for example, refer nonpatent literature 1).

  Taking advantage of this property, the encoded stream is once decoded and re-encoded when the encoded stream is switched in order to prevent the video from being stopped due to path switching when a transmission path failure occurs. The encoded stream is reconstructed by using (see, for example, Non-Patent Document 2).

In addition, in order to realize distribution of high-quality video content that has been converted to MPEG from a distribution center via nodes on a network that is hierarchically distributed, transmission is performed according to changing network loads. Changing the bit rate of video content (transcoding) is performed (see, for example, Non-Patent Document 3).
P. Guillotel et al., "Adaptive Encoders: The New Generation of MPEG-2 Encoders," SMPTE Journal, pp.287-294, April, 2000. Nakamura, Yoshitome, et al., “Examination of video stream switching system configuration method for faults”, FIT (Information Science and Technology Forum) 2003, J-023. Yoshitome, Nakamura, et al., “Examination of high-speed real-time multi-rate transcoder configuration”, FIT (Information Science and Technology Forum) 2003, J-086.

  On the other hand, in the latest encoding standard, H.264 / MPEG-4AVC, a deblocking filter process, which is an in-loop filter, is newly added to remove block noise. Is lost, and the above-described property (property that re-encoding with almost no deterioration in image quality is possible) does not hold.

  For this reason, even if all the encoding parameters are inherited and re-encoding is performed, there is a problem that image quality is greatly deteriorated every time encoding is performed.

  With this, the latest coding standard, H.264 / MPEG-4AVC, cannot use methods such as connection / switching of encoded streams and transcoding using re-encoding of the encoded streams. .

  The present invention has been made in view of such circumstances, and when decoding and re-encoding an encoded stream of a moving image that has been encoded with an in-loop filter process, image quality degradation is almost caused. An object of the present invention is to provide a new moving image re-encoding technique that realizes efficient re-encoding that realizes efficient re-encoding with less image quality degradation.

  In order to achieve this object, according to the present invention, an encoded stream of a moving image encoded with an in-loop filter process is decoded and re-encoded. By taking the configuration of outputting the image immediately before (deblocking filter processing) and re-encoding the video by completely inheriting the encoding parameters of the previous stage, with in-loop filter processing, Even in a moving image coding scheme such as H.264 in which an image after in-loop filtering is used as a decoded image, re-encoding is realized with little image quality degradation.

  In the present invention, when an encoded stream of a moving image encoded with an in-loop filter process is decoded and re-encoded, the in-loop filter process (for example, deblocking filter process) in the decoder is performed. By adopting a configuration that outputs the previous image and inherits and re-encodes the video while changing a part of the encoding parameters in the previous stage. Even in a moving image encoding method such as H.264, in which an image after filter processing is a decoded image, efficient re-encoding is realized with little image quality degradation.

  Next, a theoretical explanation will be given of the fact that image quality degradation hardly occurs due to re-encoding according to the present invention.

  FIG. 1 illustrates a re-encoding operation model constructed according to the present invention.

Here, in the figure, DF () indicates processing of an in-loop filter (for example, deblocking filter), TQ () indicates orthogonal transformation / quantization processing, and TQ ⁻¹ () is inversely converted to TQ (). BUF indicates a reference frame buffer. Also, variable length coding / arithmetic coding (information source coding) and variable length decoding / arithmetic decoding (information source decoding) are omitted.

  As shown in this figure, the re-encoding configured according to the present invention is premised on a moving picture encoding method in which an intra-loop filter process is performed and an image after the intra-loop filter process is used as a decoded image. Below, the decoding unit outputs an image immediately before the deblocking filter process, and the re-encoding unit re-encodes the video while inheriting all the previous encoding parameters. .

When the symbols shown in the figure are used, the signal C and the signal H are converted to the intra-loop filter DF of the decoding unit DF. 1. DF in-loop filter DF of re-encoding unit 2 respectively,
C = DF 1 (E + P ')
H = DF 2 (G + P ")
= DF 2 (TQ ^-1 (TQ (F)) + P ")
= DF 2 (TQ ⁻¹ (TQ (E + P′−P ″)) + P ″)
It can be expressed.

Now, if it is a picture immediately after an I picture, MV (motion vector) and prediction mode inheritance,
P '= P "
Is established,
H = DF 2 (TQ ⁻¹ (TQ (E + P′−P ″)) + P ″)
= DF 2 (TQ ^-1 (TQ (E)) + P ")
It becomes.

Here, “E” is a value that has already been quantized / inverse quantized, and the value does not change substantially by the second quantization / inverse quantization,
H = DF 2 (TQ ^-1 (TQ (E)) + P ")
= DF 2 (E + P ")
It becomes.

If all of MV (motion vector), QP (quantization parameter), and encoding mode are inherited and cbp (encoded block pattern) matches the reproduced image, DF 1 and DF Since the deblocking filter strength of 2 is the same,
H = C
As a result, for the predicted picture of the next picture,
P '= P "
Is established.

Furthermore, the signal A is
A = TQ (XP)
The signal D is
D = TQ (BP ")
= TQ (TQ ^-1 (A) + P'-P ")
= TQ (TQ ^-1 (TQ (X-P)) + P'-P ")
It becomes.

here,
P '= P "
And “TQ (X−P)” is a value that has already been quantized, and considering that the value hardly changes due to the second quantization / inverse quantization,
D = TQ (TQ ⁻¹ (TQ (X−P)) + P′−P ″)
= TQ (X-P)
And therefore
D = A
It becomes.

  That is, it can be seen that the encoded stream at the second stage matches the encoded stream at the first stage, and the image quality is hardly deteriorated by re-encoding.

  Here, the encoding parameters that need to be inherited above are basically other than orthogonal transform coefficients in the stream, such as prediction mode related information, motion vectors, reference picture information, quantization parameters, picture types, macroblock types, etc. Most information needs to be inherited as an encoding parameter at the time of re-encoding.

  Further, in the above theoretical explanation, a configuration has been described in which all of the encoding parameters are inherited, so that the encoded stream of the same bit rate is re-encoded with almost no image quality degradation. It is also possible to adopt a configuration that efficiently re-encodes and transcodes a lower-bit-rate encoded stream with less image quality degradation by changing the quantization parameters, prediction mode, etc. It is.

  Next, the configuration of the present invention will be summarized.

  The moving image re-encoding device of the present invention decodes and re-encodes an encoded stream of a moving image that has been encoded with an in-loop filter process, and (i) decodes the encoded stream. A decoding unit having means for directly outputting an image before in-loop filter processing to the re-encoding unit, temporarily storing it in a memory, and passing it to the re-encoding unit via the memory; (ii) Re-encoding unit having means for performing re-encoding processing using an encoding parameter obtained by changing the same or a part of the encoding parameter of the encoded stream with respect to the image before the in-loop filter processing received from the decoding unit The basic configuration is that

  In order to realize this basic configuration, in the video re-encoding device of the present invention, for example, (a) the decoding unit outputs or stores the decoded encoding parameter of the encoded stream, and receives this, The re-encoding unit receives the encoding parameter from the decoding unit and performs re-encoding processing using an encoding parameter that is the same as or a part of the received encoding parameter, or (b) The encoding unit receives an encoded stream input to the decoding unit, decodes only the encoding parameter from the encoded stream, and changes the same or a part of the decoded encoding parameter. The re-encoding process is performed using.

  In adopting this configuration, the moving image re-encoding device of the present invention further modifies the image to a partial image region of the image before the in-loop filter processing that is output and stored by the decoding unit. An image processing unit may be provided, and when the image processing unit is provided, the re-encoding unit can realize that no image quality degradation occurs in an image area that has not been changed by the image processing unit. Re-encoding processing is performed using an encoding parameter that is the same as or a part of the encoding parameter of the encoded stream, and the image is changed for other image areas. A re-encoding process is performed by performing a normal encoding process that does not use parameters.

  In addition, when adopting this configuration, in the moving image re-encoding device of the present invention, the decoding unit outputs or stores both the image before the in-loop filter processing and the image after the in-loop filter processing, In response, the re-encoding unit, for example, can realize that image quality degradation does not occur during re-encoding for image regions in which the quantization parameters before and after re-encoding are substantially the same. On the other hand, for image regions where the quantization parameter before and after re-encoding changes greatly, image quality degradation cannot be avoided even if an image before in-loop filtering is selected for re-encoding. Depending on the difference in quantization parameters before and after re-encoding and the image contents, such as selecting an image after filter processing, either one of the two images output / stored by the decoding unit Select who, re-encoding process is performed and the selected image as a processing target.

  Here, each processing means for realizing the moving image re-encoding device of the present invention configured as described above can also be realized by a computer program, and this computer program is recorded on an appropriate computer-readable recording medium. Or provided via a network, installed when the present invention is carried out, and operated on a control means such as a CPU, thereby realizing the present invention.

  As described above, in the present invention, an image immediately before an in-loop filter process in a decoding unit when decoding and re-encoding an encoded stream of a moving image encoded with an in-loop filter process. Is output, and the video is completely inherited from the previous encoding parameters, or re-encoded while inheriting most of them.

  In accordance with this configuration, according to the present invention, even in a moving image coding system that includes an in-loop filter process such as H.264 and uses an image after the in-loop filter process as a decoded image, the image quality degradation hardly occurs. It is possible to realize encoding, and thereby, it becomes possible to perform connection, switching, processing, transcoding, and the like of encoded streams in a complete form.

  According to the present invention, according to the present invention, the efficiency of the image coding method with less in image quality degradation is also obtained in the moving picture coding system that uses the in-loop filter processing such as H.264 and uses the image after the in-loop filter processing as a decoded image. Re-encoding can be realized, and this makes it possible to efficiently perform concatenation, switching, processing, transcoding, and the like of encoded streams almost completely.

  Hereinafter, the present invention will be described in detail according to embodiments.

[1] First Embodiment FIG. 2 shows an embodiment of the apparatus configuration of the moving image re-encoding apparatus 1 of the present invention.

  As shown in this figure, the moving image re-encoding device 1 of the present invention receives an encoded stream of a moving image encoded with deblocking filter processing as an input, decodes the encoded stream, and When decoding, a decoding unit 10 configured as the decoding unit illustrated in FIG. 1 that outputs an image before the deblocking filter process, and an image before the deblocking filter process output by the decoding unit 10 are input. A re-encoding unit 11 configured like the re-encoding unit illustrated in FIG. 1 is provided that generates and outputs a new encoded stream by encoding the video.

  FIG. 3 illustrates an example of a processing flow executed by the decoding unit 10 in the present embodiment, and FIG. 4 illustrates an example of a processing flow executed by the re-encoding unit 11 in the present embodiment.

  Next, processing of the moving image re-encoding device 1 of the present invention configured as described above will be described according to these processing flows.

[1-1] Processing of Decoding Unit 10 As shown in the processing flow of FIG. 3, when the decoding unit 10 inputs an encoded stream of a moving image encoded with deblocking filter processing (S10), decoding is performed. A target block is determined (S11), and information source decoding processing is performed on the decoding target block (S12). Here, the information source decoding process refers to variable length decoding or arithmetic decoding.

  Subsequently, the coefficient information of the decoding target block is inversely quantized and inverse orthogonal transformed to generate a difference image (S13). Subsequently, a prediction image of the decoding target block is generated from the motion vector / prediction mode information obtained by the information source decoding process and the reference image stored in the reference frame buffer (S14).

  Subsequently, the predicted image is added to the difference image of the decoding target block (S15), and the added image is output from the decoding unit 10 as an image before the deblocking filter processing (S16). At this time, in this processing flow, it is assumed that the data is stored in the memory A and output from the decoding unit 10, but it is also possible to output directly to the re-encoding unit 11.

  Subsequently, a deblocking filter process is performed on the added image (S17), and an image after the deblocking filter process, that is, a decoded image is stored as a reference image in a reference frame buffer (S18).

  This is performed for all blocks (S19), and this process is repeated until the encoded stream is completed (S20).

[1-2] Processing of Re-Encoding Unit 11 On the other hand, the re-encoding unit 11 inputs an image before deblocking filter processing output from the decoding unit 10 from, for example, the memory A as shown in the processing flow of FIG. Then, a re-encoding target block is determined (S30), and a motion vector / prediction mode is determined for the re-encoding target block by motion search / various prediction mode search (S32).

  Subsequently, a prediction image of the re-encoding target block is generated from the reference image stored in the reference frame buffer and its motion vector / prediction mode (S33). Subsequently, the prediction image is subtracted from the input image of the re-encoding target block to generate a difference image (S34).

  Subsequently, the difference image is orthogonally transformed and quantized to generate coefficient information (S35), and the coefficient information and various encoding parameters are information source encoded (S36). Here, information source coding refers to variable-length coding or arithmetic coding.

  Subsequently, the coefficient information of the re-encoding target block is inversely quantized / inverse orthogonal transformed, the predicted image is added, and the deblocking filter process is performed (S37), and the image after the deblocking filter process, that is, the reproduced image is The reference image is stored in the reference frame buffer as a reference image (S38).

  This is performed for all blocks (S39), and this process is repeated until the input image is completed (S40).

  Thus, in the moving image re-encoding device 1 of the present invention configured as shown in FIG. 2, the re-encoding unit 11 receives the image before the deblocking filter processing output from the decoding unit 10 as an input, and By re-encoding the video, a new encoded stream is generated and output.

  As described above, according to the present embodiment, the re-encoding unit 11 performs re-encoding without inheriting the previous-stage encoding information. Therefore, although the image quality due to re-encoding deteriorates, Since the exchange of information with the re-encoding unit 11 is only an image, there is an advantage that the apparatus configuration is simplified.

[2] Second Embodiment FIG. 5 illustrates another embodiment of the apparatus configuration of the moving image re-encoding apparatus 1 of the present invention.

  The difference between the present exemplary embodiment and the exemplary embodiment illustrated in FIG. 2 is that, in the present exemplary embodiment, the decoding unit 10 outputs an image before the deblocking filter process to the re-encoding unit 11. Thus, the encoding information (information of various encoding parameters) is output, and in response to this, the re-encoding unit 11 outputs the image before the deblocking filter process output from the decoding unit 10 and the encoding information. And a new encoded stream is generated by re-encoding the input image before the deblocking filter process using the input encoded information.

  FIG. 6 illustrates an example of a processing flow executed by the decoding unit 10 in the present embodiment, and FIG. 7 illustrates an example of a processing flow executed by the re-encoding unit 11 in the present embodiment.

  Here, the difference between the processing flow of FIG. 6 and the processing flow of FIG. 3 is that in the processing flow of FIG. 6, a new step 1000 (S1000) is provided between step 18 (S18) and step 19 (S19). The point is that a process is provided.

  Further, the difference between the processing flow of FIG. 7 and the processing flow of FIG. 4 is that in the processing flow of FIG. 7, a process called step 3000 (S3000) is newly added between step 30 (S30) and step 31 (S31). Is provided, the process of step 32 (S32) provided in the process flow of FIG. 4 is not executed, and the process of step 33 (S33) provided in the process flow of FIG. Instead, the process of step 33α is executed.

  Next, processing of the moving image re-encoding device 1 of the present invention configured as described above will be described according to these processing flows.

[2-1] Processing of Decoding Unit 10 As shown in the processing flow of FIG. 6, when the decoding unit 10 inputs an encoded stream of a moving image encoded with deblocking filter processing (S10), decoding is performed. A target block is determined (S11), and information source decoding processing is performed on the decoding target block (S12). Here, the information source decoding process refers to variable length decoding or arithmetic decoding.

  Subsequently, the coefficient information of the decoding target block is inversely quantized and inverse orthogonal transformed to generate a difference image (S13). Subsequently, a prediction image of the decoding target block is generated from the motion vector / prediction mode information obtained by the information source decoding process and the reference image stored in the reference frame buffer (S14).

  Subsequently, the predicted image is added to the difference image of the decoding target block (S15), and the added image is output from the decoding unit 10 as an image before the deblocking filter processing (S16). At this time, in this processing flow, it is assumed that the data is stored in the memory A and output from the decoding unit 10, but it is also possible to output directly to the re-encoding unit 11.

  Subsequently, a deblocking filter process is performed on the added image (S17), and an image after the deblocking filter process, that is, a decoded image is stored as a reference image in a reference frame buffer (S18).

  Subsequently, various encoding parameters of the decoding target block are output as encoding information from the decoding unit 10 (S1000). At this time, in this processing flow, it is assumed that the data is stored in the memory B and output from the decoding unit 10, but it is also possible to output directly to the re-encoding unit 11.

  This is performed for all blocks (S19), and this process is repeated until the encoded stream is completed (S20).

[2-2] Processing of Re-encoding Unit 11 On the other hand, the re-encoding unit 11 inputs an image before deblocking filter processing output from the decoding unit 10, for example, from the memory A, as shown in the processing flow of FIG. Then, for example, when the encoding information output from the decoding unit 10 is input from the memory B (S3000), a re-encoding target block is determined (S31).

  Subsequently, for the re-encoding target block, the re-encoding target block is predicted from the reference image stored in the reference frame buffer and the motion vector / prediction mode of the re-encoding target block included in the input encoding information. An image is generated (S33α). Subsequently, the prediction image is subtracted from the input image of the re-encoding target block to generate a difference image (S34).

  Subsequently, the difference image is orthogonally transformed and quantized to generate coefficient information (S35), and the coefficient information and various encoding parameters are information source encoded (S36). Here, information source coding refers to variable-length coding or arithmetic coding.

  Subsequently, the coefficient information of the re-encoding target block is inversely quantized / inverse orthogonal transformed, the predicted image is added, and the deblocking filter process is performed (S37), and the image after the deblocking filter process, that is, the reproduced image is obtained. The reference image is stored in the reference frame buffer as a reference image (S38).

  This is performed for all blocks (S39), and this process is repeated until the input image is completed (S40).

  In this way, in the video re-encoding device 1 of the present invention configured as shown in FIG. 5, the re-encoding unit 11 outputs the image before the deblocking filter processing output from the decoding unit 10, encoding information, Is input, and the input image before the deblocking filter process is re-encoded using the input encoded information to generate and output a new encoded stream.

  As described above, according to the present embodiment, the re-encoding unit 11 takes the image before the deblocking filter process as a processing target and inherits the previous-stage encoding information and performs re-encoding, thereby degrading the image quality. There is an advantage that re-encoding that does not occur can be performed.

[3] Third Embodiment FIG. 8 illustrates another embodiment of the apparatus configuration of the moving image re-encoding apparatus 1 of the present invention.

  The difference between this exemplary embodiment and the exemplary embodiment shown in FIG. 2 is that, in this exemplary embodiment, the re-encoding unit 11 receives an encoded stream input to the decoding unit 10 as an input and codes included therein Encoding information decoding unit 110 that decodes only encoded information, an image before deblocking filter processing output from decoding unit 10 and encoded information decoded by encoding information decoding unit 110 as inputs, and an input deblocking filter An image before processing is configured to include an encoding unit 111 that generates and outputs a new encoded stream by performing re-encoding using the input encoded information.

  FIG. 9 illustrates an example of a processing flow executed by the decoding unit 10, the encoded information decoding unit 110, and the encoding unit 111 in the present embodiment.

  Here, as shown in this figure, the decoding unit 10 in the present embodiment executes the processing flow of FIG. 3 described above, and the encoding unit 111 in the present embodiment performs the processing flow of FIG. 7 described above. Will be executed.

  Next, processing of the moving image re-encoding device 1 of the present invention configured as described above will be described according to this processing flow.

[3-1] Processing of Decoding Unit 10 As shown in the processing flow of FIG. 3, when the decoding unit 10 inputs an encoded stream of a moving image encoded with deblocking filter processing (S10), decoding is performed. A target block is determined (S11), and information source decoding processing is performed on the decoding target block (S12). Here, the information source decoding process refers to variable length decoding or arithmetic decoding.

  Subsequently, the coefficient information of the decoding target block is inversely quantized and inverse orthogonal transformed to generate a difference image (S13). Subsequently, a prediction image of the decoding target block is generated from the motion vector / prediction mode information obtained by the information source decoding process and the reference image stored in the reference frame buffer (S14).

  Subsequently, the predicted image is added to the difference image of the decoding target block (S15), and the added image is output from the decoding unit 10 as an image before the deblocking filter processing, for example, stored in the memory A (S16). .

  Subsequently, a deblocking filter process is performed on the added image (S17), and an image after the deblocking filter process, that is, a decoded image is stored as a reference image in a reference frame buffer (S18).

  This is performed for all blocks (S19), and this process is repeated until the encoded stream is completed (S20).

[3-2] Process of Encoded Information Decoding Unit 110 On the other hand, when the encoded information decoding unit 110 receives the encoded stream input to the decoding unit 10 as shown in the processing flow in FIG. 9 (S50). The decoding target block is determined (S51), and the information source decoding process is performed on the decoding target block (S52). Here, the information source decoding process refers to variable length decoding or arithmetic decoding.

  Subsequently, various encoding parameters of the decoded block to be decoded are stored as encoding information in, for example, the memory B and output from the encoding information decoding unit 110 (S53).

  This is performed for all blocks (S54), and this process is repeated until the encoded stream is completed (S55).

[3-3] Processing of Encoding Unit 111 On the other hand, the encoding unit 111 inputs, for example, an image before deblocking filter processing output from the decoding unit 10 from the memory A as shown in the processing flow of FIG. S30) When, for example, the encoding information output from the encoding information decoding unit 110 is input from the memory B (S3000), a re-encoding target block is determined (S31).

  Subsequently, for the re-encoding target block, the re-encoding target block is predicted from the reference image stored in the reference frame buffer and the motion vector / prediction mode of the re-encoding target block included in the input encoding information. An image is generated (S33α). Subsequently, the prediction image is subtracted from the input image of the re-encoding target block to generate a difference image (S34).

  Subsequently, the difference image is orthogonally transformed and quantized to generate coefficient information (S35), and the coefficient information and various encoding parameters are information source encoded (S36). Here, information source coding refers to variable-length coding or arithmetic coding.

  Subsequently, the coefficient information of the re-encoding target block is inversely quantized / inverse orthogonal transformed, the predicted image is added, and the deblocking filter process is performed (S37), and the image after the deblocking filter process, that is, the reproduced image is obtained. The reference image is stored in the reference frame buffer as a reference image (S38).

  This is performed for all blocks (S39), and this process is repeated until the input image is completed (S40).

  In this way, in the video re-encoding device 1 of the present invention configured as shown in FIG. 8, the re-encoding unit 11 decodes only the encoded information of the encoded stream input to the decoding unit 10. At the same time, the image before the deblocking filter process output from the decoding unit 10 is input, and the input image before the deblocking filter process is re-encoded using the decoded encoding information, thereby performing a new encoding It is processed to generate and output a stream.

  As described above, according to the present embodiment example, as in the embodiment example illustrated in FIG. 5, the re-encoding unit 11 inherits the previous encoded information with the image before the deblocking filter processing as a processing target. Thus, the re-encoding is performed, so that re-encoding that hardly causes image quality degradation can be performed, and the encoding information decoding unit 110 that decodes the preceding-stage encoded information and the encoding that performs re-encoding Since the unit 111 can be tightly mounted, there is an advantage that it is not necessary to newly define a transmission format of re-encoded information that has not been standardized.

[4] Fourth Embodiment FIG. 10 shows another embodiment of the apparatus configuration of the moving image re-encoding apparatus 1 of the present invention.

  The difference between the present exemplary embodiment and the exemplary embodiment illustrated in FIG. 5 is that in the present exemplary embodiment, an image before the deblocking filter process output by the decoding unit 10 is provided between the decoding unit 10 and the re-encoding unit 11. And the encoded information are input, and the image processing unit 12 is provided to perform processing for superimposing another image information on a partial image area of the input image. The image processing unit 12 is provided. In response, the re-encoding unit 11 receives the image before the deblocking filter process (processed image) output from the image processing unit 12 and the encoded information as input, and the input image before the deblocking filter process It is a point of adopting a configuration in which a new encoded stream is generated and output by re-encoding (processed image) using the input encoded information.

  FIG. 11 illustrates an example of a processing flow executed by the image processing unit 12 in the present embodiment. FIGS. 12 and 13 illustrate an example of a processing flow executed by the re-encoding unit 11 in the present embodiment. To do.

  Here, the decoding unit 10 in the present embodiment executes the processing flow of FIG. 6 described above.

  Next, processing of the moving image re-encoding device 1 of the present invention configured as described above will be described according to these processing flows.

[4-1] Processing of Decoding Unit 10 As shown in the processing flow of FIG. 6, the decoding unit 10 receives an encoded video stream that has been encoded with deblocking filtering (S10). A target block is determined (S11), and information source decoding processing is performed on the decoding target block (S12). Here, the information source decoding process refers to variable length decoding or arithmetic decoding.

  Subsequently, the coefficient information of the decoding target block is inversely quantized and inverse orthogonal transformed to generate a difference image (S13). Subsequently, a prediction image of the decoding target block is generated from the motion vector / prediction mode information obtained by the information source decoding process and the reference image stored in the reference frame buffer (S14).

  Subsequently, the predicted image is added to the difference image of the decoding target block (S15), and the added image is output from the decoding unit 10 as an image before the deblocking filter processing, for example, stored in the memory A (S16). .

  Subsequently, a deblocking filter process is performed on the added image (S17), and an image after the deblocking filter process, that is, a decoded image is stored as a reference image in a reference frame buffer (S18).

  Subsequently, various encoding parameters of the decoding target block are output as encoding information from the decoding unit 10 by storing them in, for example, the memory B (S1000).

  This is performed for all blocks (S19), and this process is repeated until the encoded stream is completed (S20).

[4-2] Processing of Image Processing Unit 12 On the other hand, as shown in the processing flow of FIG. 11, the image processing unit 12 inputs, for example, an image before deblocking filter processing output from the decoding unit 10 from the memory A ( S60), for example, the encoding information output from the decoding unit 10 is input from the memory B (S61), and the superimposed image is input (S62).

  Subsequently, a processing target block is determined (S63), and it is determined whether or not there is an input superimposed image signal in the processing target block (S64).

  When it is determined by this determination processing that there is a superimposed image signal in the processing target block, the superimposed image is superimposed on the input image of the processing target block (S65), and image processing is performed on the encoded information of the processing target block. A flag is added (S66). On the other hand, when it is determined that there is no superimposed image signal in the processing target block, the image processing flag is not added to the encoding information of the processing target block (S67).

  Subsequently, various encoding parameters of the processing target block are stored as encoding information in, for example, the memory B 'and output from the image processing unit 12 (S68). Subsequently, the image before the deblocking filter process, which is an image of the processing target block, is output from the image processing unit 12 by storing it in the memory A ′, for example (S69).

  This is performed for all blocks (S70), and this process is repeated until the input image is completed (S71).

[4-3] Processing of Re-encoding Unit 11 On the other hand, the re-encoding unit 11 performs, for example, deblocking filter processing output from the image processing unit 12 from the memory A ′ as shown in the processing flow of FIGS. When a previous image (processed image) is input (S80) and, for example, encoding information output from the image processing unit 12 is input from the memory B ′ (S81), a re-encoding target block is determined (S82).

  Subsequently, it is determined whether an image processing flag is added to the re-encoding target block (S83). If it is determined that an image processing flag is added, the image is processed. It is determined that the encoding information cannot be inherited, and the motion vector / prediction mode is determined by motion search / various prediction mode search for the re-encoding block (S84) and stored in the reference frame buffer. A predicted image of the re-encoding target block is generated from the reference image being present and the motion vector / prediction mode (S85).

  On the other hand, when it is determined that the image processing flag is not added to the re-encoding target block, it is determined that the encoding information can be inherited because the image is not processed, and the reference is made. A predicted image of the re-encoding target block is generated from the reference image stored in the frame buffer and the motion vector / prediction mode of the re-encoding target block of the input encoding information (S86).

  Subsequently, the prediction image is subtracted from the input image of the re-encoding target block to generate a difference image (S87). Subsequently, the difference image is orthogonally transformed and quantized to generate coefficient information (S88), and the coefficient information and various encoding parameters are information source encoded (S89). Here, information source coding refers to variable-length coding or arithmetic coding.

  Subsequently, the coefficient information of the re-encoding target block is inversely quantized / inverse orthogonally transformed and the predicted image is added, and the deblocking filter process is performed (S90), and the image after the deblocking filter process, that is, the reproduced image is obtained. The reference image is stored in the reference frame buffer as a reference image (S91).

  This is performed for all blocks (S92), and this process is repeated until the input image is completed (S93).

  In this way, the moving image re-encoding device 1 of the present invention configured as shown in FIG. 10 does not perform image processing on the decoded image decoded by the decoding unit 10, but outputs the output of the decoding unit 10. Image processing is performed on the image before the deblocking filter processing, and the normal re-encoding is performed on the processed image region, and the non-processed image region is re-inherited from the previous encoding information. Processing is performed to perform encoding.

  As described above, according to the present embodiment, there is an advantage in that re-encoding with little image quality degradation can be performed even when an image is processed during re-encoding.

[5] Fifth Embodiment FIG. 14 shows another embodiment of the apparatus configuration of the moving image re-encoding apparatus 1 of the present invention.

  The difference between the present exemplary embodiment and the exemplary embodiment illustrated in FIG. 5 is that, in the present exemplary embodiment, the decoding unit 10 outputs the image before the deblocking filter processing and the encoded information, and also deblocks. In response to this, the re-encoding unit 11 is configured to output a decoded image that is an image after filter processing, for each re-encoding block, according to the image content and the quantization parameter. The point is to adopt a configuration in which whether to input a decoded image or an image before deblocking filter processing is adaptively selected, and the input image is re-encoded.

  FIG. 15 illustrates an example of a processing flow executed by the decoding unit 10 in the present embodiment. FIGS. 16 and 17 illustrate an example of a processing flow executed by the re-encoding unit 11 in the present embodiment. .

  Next, processing of the moving image re-encoding device 1 of the present invention configured as described above will be described according to these processing flows.

[5-1] Processing of Decoding Unit 10 As shown in the processing flow of FIG. 15, when the decoding unit 10 inputs an encoded stream of a moving image encoded with deblocking filter processing (S100), decoding is performed. A target block is determined (S101), and information source decoding processing is performed on the decoding target block (S102). Here, the information source decoding process refers to variable length decoding or arithmetic decoding.

  Subsequently, the coefficient information of the decoding target block is inversely quantized and inverse orthogonal transformed to generate a difference image (S103). Subsequently, a prediction image of the decoding target block is generated from the motion vector / prediction mode information obtained by the information source decoding process and the reference image stored in the reference frame buffer (S104).

  Subsequently, the prediction image is added to the difference image of the decoding target block (S105), and the added image is output from the decoding unit 10 as an image before the deblocking filter processing, for example, stored in the memory A (S106). .

  Subsequently, a deblocking filter process is performed on the added image (S107), and the image after the deblocking filter process, that is, a decoded image is stored in, for example, the memory C and output from the decoding unit 10 (S108). .

  Subsequently, the image after the deblocking filter processing, that is, the decoded image is stored in the reference frame buffer as a reference image (S109). Subsequently, various encoding parameters of the decoding target block are output as encoding information from the decoding unit 10 by storing them in, for example, the memory B (S110).

  This is performed for all blocks (S111), and this process is repeated until the encoded stream is completed (S112).

[5-2] Processing of Re-encoding Unit 11 On the other hand, when the re-encoding unit 11 receives the encoding information output from the decoding unit 10 from the memory B, for example, as shown in the processing flow of FIGS. (S200), a re-encoding target block is determined (S201).

  Subsequently, it is determined whether the difference between the quantization parameters before and after the re-encoding of the re-encoding target block is smaller than a predetermined threshold (S202). The image before the deblocking filter processing for the re-encoding target block output by 10 is input (S203). If it is determined that the image is not small, the decoding unit 10 outputs the image from the memory C, for example. A decoded image for the re-encoding target block is input (S204).

  Subsequently, a prediction image of the re-encoding target block is generated from the reference image stored in the reference frame buffer and the motion vector / prediction mode of the re-encoding target block included in the input encoding information (S205).

  Subsequently, the prediction image is subtracted from the input image of the re-encoding target block to generate a difference image (S206). Subsequently, the difference image is orthogonally transformed and quantized to generate coefficient information (S207), and the coefficient information and various encoding parameters are information source encoded (S208). Here, information source coding refers to variable-length coding or arithmetic coding.

  Subsequently, the coefficient information of the re-encoding target block is inversely quantized / inverse orthogonal transformed, the predicted image is added, and the deblocking filter process is performed (S209), and the image after the deblocking filter process, that is, the reproduced image is The reference image is stored in the reference frame buffer as a reference image (S210).

  This is performed for all blocks (S211), and this process is repeated until the input image is completed (S212).

  In this way, in the moving image re-encoding device 1 of the present invention configured as shown in FIG. 14, image quality deterioration occurs in re-encoding for blocks with substantially the same quantization parameters before and after re-encoding. The image before the deblocking filter processing is selected, while for the block whose quantization parameter before and after the re-encoding changes greatly, the image before the deblocking filter processing is selected for the re-encoding. However, since image quality degradation is unavoidable, an image after the deblocking filter process is selected, and the selected image is re-encoded.

  As described above, according to this embodiment, re-encoding can be performed with less image quality according to the difference in quantization parameter before and after re-encoding and the image content.

  Although the illustrated embodiment has been described, the present invention is not limited to this. For example, in the embodiment, it is assumed that a deblocking filter is used as the in-loop filter. However, the present invention is not limited to the deblocking filter, and any in-loop filter is used. It can be applied as it is.

  Also, in the embodiment, a configuration has been described in which all of the encoding parameters are inherited, so that the encoded stream of the same bit rate is re-encoded with almost no image quality degradation. It is also possible to adopt a configuration in which, for example, by changing a quantization parameter, a prediction mode, and the like, an encoded stream with a lower bit rate is efficiently re-encoded with less image quality degradation.

It is explanatory drawing of the re-encoding operation | movement model comprised by this invention. It is a device block diagram of the moving image re-encoding device in the first embodiment. It is a processing flow which the decoding part in a 1st example of execution performs. It is a processing flow which the re-encoding part in a 1st embodiment example performs. It is an apparatus block diagram of the moving image re-encoding apparatus in 2nd Embodiment. It is a processing flow which the decoding part in a 2nd example of execution performs. It is a processing flow which the re-encoding part in a 2nd embodiment example performs. It is an apparatus block diagram of the moving image re-encoding apparatus in the example of 3rd Embodiment. It is the processing flow which the decoding part in a 3rd example of an embodiment, an encoding information decoding part, and an encoding part perform. It is an apparatus block diagram of the moving image re-encoding apparatus in the example of 4th Embodiment. It is a processing flow which the image processing part in a 4th example of execution performs. It is a processing flow which the re-encoding part performs in the fourth embodiment. It is a processing flow which the re-encoding part performs in the fourth embodiment. It is a device block diagram of the moving image re-encoding device in the fifth embodiment. It is a processing flow which the decoding part in a 5th example of execution performs. It is a processing flow which the re-encoding part in a 5th example of execution performs. It is a processing flow which the re-encoding part in a 5th example of execution performs.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Moving image re-encoding apparatus 10 Decoding part 11 Re-encoding part 12 Image processing part 110 Encoding information decoding part 111 Encoding part

Claims (16)

A moving image re-encoding method for decoding and re-encoding an encoded stream of a moving image that has been encoded with an in-loop filtering process,
When decoding the encoded stream, output or save the image before in-loop filtering,
Performing re-encoding processing on the image before the in-loop filter processing,
A moving image re-encoding method as a feature. A moving image re-encoding method for decoding and re-encoding an encoded stream of a moving image that has been encoded with an in-loop filtering process,
When decoding the encoded stream, output or save the image before in-loop filtering,
Re-encoding processing is performed on the image before the in-loop filter processing using an encoding parameter that is the same as or part of the encoding parameter of the encoded stream,
A moving image re-encoding method as a feature. A moving image re-encoding method for decoding and re-encoding an encoded stream of a moving image that has been encoded with an in-loop filtering process,
When decoding the encoded stream, output or save the image before the in-loop filter processing, and output or save the decoded encoding parameter of the encoded stream,
Performing re-encoding processing on the image before the in-loop filter processing using an encoding parameter that is the same as or part of the encoding parameter is changed,
A moving image re-encoding method as a feature. A moving image re-encoding method for decoding and re-encoding an encoded stream of a moving image that has been encoded with an in-loop filtering process,
When decoding the encoded stream, output or save the image before in-loop filtering,
Only the encoding parameters are decoded from the encoded stream, and the image before the in-loop filter processing is re-encoded using an encoding parameter that is the same as or changed from the decoded encoding parameter. To do the processing,
A moving image re-encoding method as a feature. The moving image re-encoding method according to any one of claims 2 to 4,
Change the image to a partial image area of the image before the in-loop filter processing,
For the image area that has not been changed, re-encoding processing is performed using an encoding parameter that is the same as or part of the encoding parameter of the encoded stream, and for other image areas, Performing the re-encoding process by performing a normal encoding process that does not use the encoding parameter of the encoded stream,
A moving image re-encoding method as a feature. The moving image re-encoding method according to any one of claims 1 to 4,
Select either the image before the in-loop filter processing obtained when decoding the encoded stream or the image after the in-loop filter processing obtained when decoding the encoded stream. To re-encode the image as the processing target,
A moving image re-encoding method as a feature. The moving image re-encoding method according to claim 6,
For image areas where the quantization parameters before and after re-encoding are substantially the same, select an image before in-loop filtering, and for other image areas, select an image after in-loop filtering,
A moving image re-encoding method as a feature. A moving image re-encoding device that decodes and re-encodes an encoded stream of a moving image that has been encoded with an in-loop filtering process,
A decoding unit having means for outputting or storing an image before in-loop filtering when decoding the encoded stream;
Comprising a re-encoding unit having means for performing re-encoding processing on the image before the in-loop filter processing,
A moving image re-encoding device. A moving image re-encoding device that decodes and re-encodes an encoded stream of a moving image that has been encoded with an in-loop filtering process,
A decoding unit having means for outputting or storing an image before in-loop filtering when decoding the encoded stream;
A re-encoding unit having means for performing re-encoding processing on the image before the in-loop filter processing using an encoding parameter that is the same as or part of the encoding parameter of the encoded stream; To prepare,
A moving image re-encoding device. A moving image re-encoding device that decodes and re-encodes an encoded stream of a moving image that has been encoded with an in-loop filtering process,
A decoding unit having means for outputting or saving an image before the in-loop filter processing and outputting or saving a decoded encoding parameter of the encoded stream when decoding the encoded stream;
A re-encoding unit having means for performing re-encoding processing using an encoding parameter that is the same as or a part of the encoding parameter, with respect to the image before the in-loop filter processing,
A moving image re-encoding device. A moving image re-encoding device that decodes and re-encodes an encoded stream of a moving image that has been encoded with an in-loop filtering process,
A decoding unit having means for outputting or storing an image before in-loop filtering when decoding the encoded stream;
Only the encoding parameters are decoded from the encoded stream, and the image before the in-loop filter processing is re-encoded using an encoding parameter that is the same as or changed from the decoded encoding parameter. Comprising a re-encoding unit having means for performing processing,
A moving image re-encoding device. The moving image re-encoding device according to any one of claims 9 to 11,
An image processing unit for changing an image with respect to a partial image region of the image before the in-loop filter processing;
The re-encoding unit re-encodes an image area that has not been changed by the image processing unit, using an encoding parameter that is the same as or part of the encoding parameter of the encoded stream. For other image regions, re-encoding processing is performed by performing normal encoding processing that does not use the encoding parameters of the encoded stream.
A moving image re-encoding device. The moving image re-encoding device according to any one of claims 8 to 11,
The decoding unit outputs or stores both the image before the in-loop filter processing and the image after the in-loop filter processing,
The re-encoding unit selects one of the two images to be output / stored by the decoding unit, and performs the re-encoding process on the selected image as a processing target.
A moving image re-encoding device. The moving image re-encoding device according to claim 13,
The re-encoding unit selects an image before the in-loop filter processing for an image region where the quantization parameters before and after the re-encoding are substantially the same, and an image after the in-loop filter processing for the other image regions. To choose
A moving image re-encoding device.   A moving image re-encoding program for causing a computer to execute processing used to realize the moving image re-encoding method according to claim 1.   A computer-readable recording medium having recorded thereon a moving image re-encoding program for causing a computer to execute processing used to realize the moving image re-encoding method according to claim 1.

Images