JP2002152735A - Image re-coder - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image re-coder or the like that attains optimum bit assignment by taking a temporal succeeding frame going to be re-coded into account without the need for newly adding a delay memory when coding again a received coded image under a revised coding condition. SOLUTION: In the image re-coder 100 re-codes moving picture data having been coded under a prescribed coding condition by revising the coding condition, a user data separation section 200 acquires externally moving picture data with coded auxiliary information with respect to coding attached to them and extracts the coded auxiliary information from the moving picture data, a decoding section 400 decodes a moving picture from the moving picture data, and a re-coding section 500 re-codes the moving picture under a different coding condition from the coding condition on the basis of the acquired coded auxiliary information under the control of a re-coding control section 300.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image re-encoding device, an image re-encoding method, and a recording medium for re-encoding encoded moving image data by changing encoding conditions.

[0002]

2. Description of the Related Art MPEG2 is used as a moving picture coding method.
Has become an international standard and is used in current CS digital broadcasting.
It has been decided to be used also in digital broadcasting. In digital broadcasting, regarding the recording method of each content at home, conventional analog broadcasting is
Analog recording similar to that recorded on the R is conceivable, but digital recording using an MPEG2 bit stream is considered separately. When digitally recording an MPEG2 bit stream, the broadcasted M
In addition to recording without changing the PEG2 bit stream, in order to realize long-time recording in the limited storage capacity of media, it is necessary to change the received MPEG2 bit stream to a different spatial resolution. It is necessary to record a re-encoded MPEG2 bit stream under different encoding conditions, such as changing to a higher compression ratio.

[0003] In the future, it is expected that a product recording in an MPEG2 bit stream will appear in a home video camera, and an image taken by this camera is edited.
It is also conceivable that the edited image is recorded as an MPEG2 bit stream. In this case as well, the M2 re-encoded from the original MPEG2 bit stream under different encoding conditions
It needs to be converted to a PEG2 bit stream.

Incidentally, the control of the bit rate in the MPEG2 encoding system is basically a variable rate, and is basically defined by the average bit rate although there are restrictions on the maximum bit rate and the minimum bit rate. Therefore, the bit usage at each time differs for each frame, and a large number of bits are allocated to an image having a complicated pattern, and a small number of bits are allocated to an image having a simple pattern. Bit allocation has been done. Even if the average bit rate is the same, the reason why the image quality differs depending on the encoding device is that the method of allocating the bits is different, and how optimal this bit allocation is. It can be said that the performance of the encoding device is determined.

In such a situation, the same applies to a re-encoding device for re-encoding an original MPEG 2 bit stream into an MPEG 2 bit stream having a different bit rate, and an optimal bit allocation method in re-encoding is determined. It becomes important. Therefore, since the re-encoding device receives an MPEG2 bit stream to which bits have been assigned once as input, the re-encoding device analyzes the bit usage per frame, the average quantization step size, and the like from the bit stream input so far. However, a method of allocating bits at the time of re-encoding based on the analysis result has been considered.

For example, the re-encoding device shown in FIG. 5A includes a bit analysis unit 910 for analyzing an input bit stream, and a decoding unit 93 for decoding a bit stream.
0, a re-encoding unit 940 for re-encoding the decoded image and a re-encoding control unit 920. The bit analysis unit 910 sequentially analyzes the bit usage for each frame based on the input bit stream, outputs the analysis result to the re-encoding control unit 920, and sequentially analyzes the analyzed bit stream. 930. Decoding section 93
0 sequentially decodes the input bit stream and outputs it to the re-encoding unit 940, and outputs the quantization step size used at the time of decoding to the re-encoding control unit 920. The re-encoding unit 940 sequentially re-encodes the decoded data from the decoding unit 930 under the control of the re-encoding control unit 920 and outputs the result. The re-encoding controller 920 includes a bit analyzer 910
And the bit allocation in the re-encoding unit 940 based on the bit usage per frame input from the decoding unit 930 and the average quantization step size calculated from the quantization step size input from the decoding unit 930. Perform control.

The re-encoding device shown in FIG. 5 (b) has the same structure as the re-encoding device shown in FIG. 5 (a), and further includes a delay buffer for delaying the encoding timing. 960, from the decoding unit 930 to the re-encoding unit 9
The bit stream is re-encoded while delaying the transmission of the decoded data to 40. By delaying this, the re-encoding control unit 920 controls the bit allocation using not only the bit analysis result of the frame to be re-encoded but also the bit analysis result of the future frame of the frame to be re-encoded. It is something that can be done.

[0008]

However, FIG.
In the method shown in (a), bits can be allocated only from the result of analyzing the MPEG2 bit stream input in the past before the frame to be re-encoded, and the time is shorter than the frame to be re-encoded. There is a problem that it is not possible to perform bit allocation of a frame to be re-encoded at present in consideration of bit allocation to a previous frame.

Further, in the method shown in FIG. 5B, in order to consider the bit allocation to a frame temporally ahead of the frame to be re-encoded, the input M
There is a problem that a delay memory for delaying the re-encoding is required for the analysis of the PEG2 bit stream. The present invention has been made in view of such a situation, and when re-encoding a received encoded image by changing the encoding conditions, re-encode without adding a new delay memory. It is an object of the present invention to provide an image re-encoding device, an image re-encoding method, and a recording medium capable of performing optimal bit allocation in consideration of a temporal point of a frame to be set.

[0010]

In order to solve the above-mentioned problems, an image re-encoding apparatus according to the present invention re-encodes moving image data encoded according to predetermined encoding conditions by changing the encoding conditions. An image re-encoding device that obtains, from outside, moving image data to which encoding auxiliary information relating to encoding is added, and an extracting unit that extracts the encoding auxiliary information from the moving image data, A decoding unit for decoding a moving image from the moving image data, and a re-encoding unit for re-encoding the moving image under an encoding condition different from the encoding condition based on the acquired encoding auxiliary information. Prepare.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A re-encoding device according to an embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing the configuration of the re-encoding device of the present embodiment.
In FIG. 1, the re-encoding device 100 includes a user data separating unit 200, a re-encoding control unit 300, a decoding unit 400, and a re-encoding unit 500.

The user data separation unit 200 receives M
When an encoded image encoded by the PEG2 method is input, re-encoding information is extracted from the encoded image and output to the re-encoding control unit 300. The re-encoding information is included in a user data area in the encoded image. In the case of the MPEG2 system, the user data area is a data area existing in the head area of the coding unit, such as in a GOP header or a picture header. FIG. 2A shows the relationship between the user data and the GOP. FIG. 2A shows a case where a user data area including re-encoding information exists in the GOP header. In the figure, user data 201 and 202 exist in the GOP header at the head of the GOP, and each frame in each GOP {KI01, KP03, KB01, KB02, KP0}
6, KB04, KB05｝, {KI12, KB10, KB1
1, KP13} as re-encoding information. As shown in FIG.
The header portion of P may include re-encoding information on all frames in the GOP. Alternatively, the re-encoding information on the picture may be included in a picture header portion on a picture-by-picture basis. FIG.
(B) shows an example. In FIG.
, 213 exist in the picture header at the head of each picture, and each holds information relating to the re-encoding of this picture as re-encoding information.

Specifically, in the present embodiment, the re-encoding information is the bit usage of each frame and the average quantization step size of each frame. Decoding section 400 decodes the encoded image input from user data separating section 200 and then sequentially outputs the decoded image to re-encoding section 500. Further, it outputs the quantization step size obtained at the time of decoding to re-encoding control section 300 and the motion vector information to re-encoding section 500.

More specifically, the decoding section 400 includes a variable length decoding section 401 for performing variable length decoding on an encoded image, an inverse quantization section 402 for inversely quantizing a quantized DCT coefficient obtained by variable length decoding, An inverse DCT unit 403 that performs an inverse DCT on the DCT coefficient obtained by the quantization, an addition unit 404 that adds a prediction error obtained by the inverse DCT and a reference image indicated by a motion vector included in the encoded image to obtain a decoded image, It comprises a decoded image memory unit 405 for storing a decoded image to be a reference frame. Decoding by these components is the same as that of a general MPEG decoder, and a detailed description thereof will be omitted.

The image decoded by the decoding unit 400 and output to the re-encoding unit 500 is a DCT coefficient obtained by performing only the variable-length decoding and the inverse quantization on only the variable-length decoding unit 401 and the inverse quantization unit 402. It may be. In that case, the inverse DCT section 403, the addition section 404, and the decoded image memory section 405 are not required due to the configuration of the decoding section 400. Re-encoding unit 50
0 re-encodes the decoded image sequentially input from the decoding unit 400 at a bit rate different from the bit rate of the original encoded image. In the present embodiment, the different bit rate is a bit rate lower than the bit rate of the original coded image.

More specifically, the re-encoding unit 500 rearranges and stores the frames of the decoded image in the order of encoding as necessary, and performs a motion compensation prediction for each block of the decoded image to calculate a motion vector. A motion vector detecting unit 502 for detecting, a predicted image memory unit 510 for storing a reference frame, a subtraction unit 503 for calculating a prediction error from a reference block of the reference frame and a coding target block of a decoded image, DCT section 504 for performing DCT, quantization section 505 for performing quantization on DCT coefficients, and inverse quantization section for performing inverse quantization and inverse DCT on quantized DCT coefficients for use in motion compensation prediction to obtain a reference frame 507 and the inverse DCT section 508, and the variable-length coding of the quantized DCT coefficient and the motion vector to output the final re-coded image to the outside. Composed of the variable length coding unit 506.

Here, the motion vector detecting section 502 may use the motion vector information input from the decoding section 400 as it is as a motion vector in re-encoding,
The periphery around the motion vector indicated by the motion vector information may be searched, or a new motion vector may be searched regardless of the motion vector information. The quantization unit 505 performs quantization using the quantization step size of each block indicated in the quantization control information notified from the re-encoding control unit 300.

Since the configuration is the same as that of other general MPEG encoding devices in the re-encoding unit 500, a detailed description is omitted. The re-encoding control unit 300 analyzes the re-encoding information input from the user data separating unit 200, calculates an optimal allocated bit amount for a frame to be encoded, and outputs the calculation result and the decoding unit 400 The bit rate control of the re-encoding unit 500 is performed based on the notified quantization step size. Specifically, the re-encoding control unit 30
0 determines the quantization step size of each block to be quantized by the quantization unit 505 and notifies the quantization unit 505 as quantization control information.

The re-encoding device 100 thus configured
Hereinafter, an example of the operation will be described focusing on the re-encoding control unit 300. FIG. 3 is a flowchart showing a processing procedure of the re-encoding control unit 300. In the figure, the re-encoding control section 300 first obtains the re-encoding information output from the user data separating section 200 (step S30).
1). Here, it is assumed that the re-encoding information is stored in the user data area of the GOP header as shown in FIG. 2A, the bit usage of each frame in the GOP, and the average quantization step size of each frame. Shall be included.

Next, the re-encoding control unit 300 determines in step S
The processing from step 302 to step S305 is repeated for the number of frames in the GOP. Specifically, the re-encoding control unit 300
Read out the re-encoded information for one frame (step S3
02). Then, based on the bit usage of the frame and the average quantization step, the reduction ratios of the allocated bits to the frame are A, B, C, and D (step S30).
3). These A, B, C, and D are only for roughly discriminating the level of the reduction rate of the allocated bit amount in step S303, and it is assumed that specific numerical values have not been determined. It is assumed that a specific value is determined in S306.

Before proceeding to the description of step S304, the process of step S303 will be described in detail. FIG.
9 is a flowchart showing a detailed process of step S303. First, in step S401 in FIG. 4, the re-encoding control unit 300 determines whether or not the bit usage of the frame is larger than a predetermined threshold S. Further, in steps S402 and S403, it is determined whether or not the average quantization step size of the frame is larger than a predetermined threshold value P.

As a result of the determination, if the bit usage of the frame is larger than the predetermined threshold value S and the average quantization step size is larger than the predetermined threshold value P, the reduction rate of the allocated bit amount to be allocated to the frame is determined. Is set to A (step S404). If the bit usage of the frame is larger than the predetermined threshold value S and the average quantization step size is equal to or smaller than the predetermined threshold value P, the reduction ratio of the allocated bit amount to be allocated to the frame is set to B (step S40
5). Also, the bit usage of this frame is equal to a predetermined threshold S.
Hereinafter, when the average quantization step size is larger than the predetermined threshold value P, the reduction ratio of the allocated bit amount to be allocated to the frame is set to C (step S406). When the bit usage of the frame is equal to or less than a predetermined threshold value S,
If the average quantization step size is equal to or smaller than the predetermined threshold value P, the reduction ratio of the allocated bit amount to be allocated to the frame is set to D (step S407).

Here, the relationship among A, B, C, and D is, for example, A = C, B = D, and B> A. In this case, the frame whose average quantization step is equal to or smaller than the threshold value P is
The amount of allocated bits is significantly reduced as compared with a frame larger than the threshold value P. A, B, C, and D have such a relationship, that is, A = C, B = D, and B> A for the following reason.

That is, considering the amount of bits used and the average quantization step size, when the amount of bits used is large and the average quantization step size is large, it is considered that the image is a very complicated pattern. Can be Also, if the bit usage is large and the average quantization step size is small, the image is a complicated pattern or the image is not a very complicated pattern but the accumulated bit usage up to that point is less than necessary. It is considered that a bit is generated in In addition, when the bit usage is small and the average quantization step size is large, the image is a simple pattern or the image is not a simple pattern but the accumulated bit usage up to that point is too large. It is considered that the number of bits is reduced. If the bit usage is small and the average quantization step size is small, the image may be a very simple pattern.

Therefore, for a frame having a large bit usage and a small average quantization step size or a frame having a small bit usage and a small average quantization step size, it is assumed that the bits are used more than necessary. Deemed,
At the time of re-encoding, the rate at which bit allocation is reduced is increased. Frames with a large bit usage and a large average quantization step size or frames with a small bit usage and a large average quantization step size are considered to have reduced bits more than necessary, and At the time of encoding, the rate at which bit allocation is reduced is reduced.

Referring back to FIG. 3, in step S304, re-encoding control section 300 sets reduction rates A, B, C, D
The number of frames is separately accumulated (step S305), and GO
Steps S302 to S for all frames of P
After performing the process of step 304, the process proceeds to step S306 (step S305). As a result, the number of frames is determined for each reduction ratio, for example, three frames determined to be A and four frames determined to be B among all frames of one GOP.

Finally, the re-encoding control unit 300 determines whether A, B,
From the relationship between C and D, that is, the relationship between A = C and B = D and B> A, the number of frames for each reduction ratio, and the target bit rate of the GOP, the values of A, B, C, and D satisfying these conditions are obtained. Specific values are determined, and from these values, the allocated bit amount of each frame is determined. When the re-encoding control unit 300 determines the amount of bits to be allocated to each frame as described above, the re-encoding control unit 300 further determines the quantization step size for each block of each frame so that the amount of allocated bits is further reduced. As a method for determining the quantization step size for each block, a conventional method such as TM5 may be used. Briefly, in TM5, the allocated bit amount is compared with the actual generated code amount in macroblock units, and the quantization step size in macroblock units is sequentially determined so that the total generated code amount falls within the allocated bit amount. decide. As another method, a quantization step size for each macro block obtained from the decoding unit 400 is used, and a value obtained by weighting each quantization step size at a reduction rate is used as a quantization step size for re-encoding. May be used.

Then, based on the calculation result, the encoding unit 4
The quantization step size supplied from 00 is controlled so as to be large for a frame for which bit allocation is reduced, and conversely, such control as to be small for a frame for which bit allocation is increased. While adjusting to match the average bit rate of re-encoding
The re-encoding unit 500 is controlled.

With such a configuration, the re-encoding device
When re-encoding a received encoded image by changing the encoding conditions, it is possible to optimize the frame to be re-encoded in consideration of a temporally earlier frame without adding a new delay memory. Bit allocation can be performed.
As described above, the re-encoding device of the present invention has been described based on the embodiment. However, the present invention is not limited to the above-described embodiment, and may be as follows.

(1) If the bit usage per frame indicated in the recoding information included in the GOP header tends to increase with time,
Because the accumulated bit usage in the first half of the GOP is small,
It may be considered that bits are generated more than necessary in the latter half, and the allocated bit amount of each frame is determined so that the reduction ratio of the allocated bit amount increases with time. If the bit usage per frame indicated in the re-encoding information is decreasing with time, the bits are reduced more than necessary in the second half because the accumulated bit amount in the first half of the GOP is large. In such a case, the allocation bit amount of each frame may be determined so that the reduction ratio of the allocation bit amount decreases with time. (2) The relationship among the reduction ratios A, B, C, and D need not be limited to the relationship in the above embodiment. For example, a relationship of all different values such as B>D>A> C may be used. (3) The re-encoding control unit 300 determines the reduction ratios A, B, C, and B from the bit usage of each frame indicated in the re-encoding information.
A configuration in which a value obtained by reducing D is determined as each frame allocation bit amount may be adopted. (4) The above embodiment and (1) to (3) may be executed in combination.

[0031]

An image re-encoding device according to the present invention is an image re-encoding device for re-encoding moving image data encoded according to predetermined encoding conditions by changing the encoding conditions. Acquiring means for externally acquiring moving image data to which encoding auxiliary information relating to encoding is added, extracting means for extracting the encoding auxiliary information from the moving image data, and decoding for decoding a moving image from the moving image data Means, and re-encoding means for re-encoding the moving image based on the obtained encoding auxiliary information under an encoding condition different from the encoding condition.

According to this configuration, the image re-encoding device performs re-encoding on the basis of the encoding auxiliary information. Therefore, before performing the re-encoding, the entire frame, that is, the temporally earlier frame than the frame to be encoded. Optimum bit allocation can be performed in consideration of the bit allocation of the entire frame including the frame of (i). That is, it is possible to realize an image re-encoding device that can efficiently allocate bits and perform high-quality re-encoding. Also, in order to consider the bit allocation of the frame ahead in time,
There is also an effect that it is not necessary to newly provide a delay memory as in the related art.

Further, the acquiring means acquires moving image data to which encoding auxiliary information including a quantization step size for each frame used in the original encoding has been added, and the extracting means comprises: The quantization step size is extracted as the encoding auxiliary information, and the re-encoding unit calculates the allocated bit amount for each frame in the re-encoding,
Assuming that the smaller the quantization step size extracted by the extracting means, the smaller the bit amount allocated in the original encoding, the more the bit amount is reduced from the allocated bit amount in the original encoding, with the encoding condition being set to be smaller than the allocated bit amount in the original encoding. The amount of bits assigned to the
The frame may be encoded using the bit amount as a target generated bit amount. According to this configuration, there is no frame in which the allocated bit amount becomes uselessly large, efficient bit allocation is performed for the entire frame, and high-quality re-encoding is performed.

The acquiring means acquires moving image data to which encoding auxiliary information including the generated bit amount for each frame in the original encoding is added, and the extracting means determines the generated bit amount for each frame. Extracted as the encoding auxiliary information, the re-encoding unit re-encodes a bit amount obtained by subtracting a larger amount of bits from the allocated bit amount in the original encoding as the generated bit amount for each frame is larger. Since the frame is coded with the bit amount as the target generated bit amount as the allocated bit amount in, the same effect as described above is obtained.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a re-encoding device according to an embodiment;

FIG. 2A shows a data structure when a GOP header includes a user data area including recoding information. (B) shows a data structure in the case where a data area including the re-encoding information exists in the picture header.

FIG. 3 is a flowchart illustrating a processing procedure of a re-encoding control unit 300;

FIG. 4 is a flowchart illustrating a detailed process of step S303.

FIG. 5A shows an example of a conventional re-encoding device. (B) Another example of the conventional re-encoding device is shown.

[Explanation of symbols]

 Reference Signs List 100 re-encoding device 200 user data separation unit 300 re-encoding control unit 400 decoding unit 401 variable length decoding unit 402 inverse quantization unit 403 inverse DCT unit 404 addition unit 405 decoded image memory unit 500 re-encoding unit 501 Replacement memory unit 502 Motion vector detection unit 503 Subtraction unit 504 DCT unit 505 Quantization unit 507 Inverse quantization unit 508 Inverse DCT unit 506 Variable length encoding unit 510 Predictive image memory unit

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 5C059 KK00 KK34 MA00 MA23 ME01 NN01 RC09 RC26 RC28 SS06 SS11 TA53 TA60 TA62 TB03 TC19 TC25 TC38 TD12 UA02 UA34 5J064 BA09 BA16 BB03 BC01 BC02 BC08 BC16 BC21 BD03

Claims (5)

[Claims] 1. An image re-encoding device for re-encoding moving image data encoded according to a predetermined encoding condition by changing an encoding condition, wherein encoding auxiliary information relating to encoding is added. Acquiring means for acquiring moving image data from outside; extracting means for extracting the encoding auxiliary information from the moving image data; decoding means for decoding a moving image from the moving image data; and the acquired encoding auxiliary information. Re-encoding means for re-encoding the moving image based on a coding condition different from the coding condition. 2. The method according to claim 1, wherein the acquiring unit acquires moving image data to which encoding auxiliary information including a quantization step size for each frame used in the original encoding is added. The quantization step size is extracted as the encoding auxiliary information, and the re-encoding means encodes that the allocated bit amount for each frame in the re-encoding is smaller than the allocated bit amount in the original encoding. As a condition, for a frame having a larger quantization step size extracted by the extraction means, a bit amount obtained by subtracting a larger bit amount from the allocated bit amount in the original encoding is set as the allocated bit amount in the re-encoding, and 2. The frame is coded with the target bit amount as a target generation bit amount.
An image re-encoding device according to claim 1. 3. The acquiring unit acquires moving image data to which encoding auxiliary information including an amount of generated bits for each frame in the original encoding is added, and the extracting unit includes an amount of generated bits for each frame. Is extracted as the encoding auxiliary information, and the re-encoding unit determines that the larger the amount of generated bits for each frame,
The frame is coded by setting a bit amount obtained by subtracting a larger bit amount from the allocated bit amount in the original coding as an allocated bit amount in the re-encoding, and using the bit amount as a target generated bit amount. 2. The image re-encoding device according to 1. 4. A re-encoding method for re-encoding moving image data encoded according to a predetermined encoding condition by changing an encoding condition, wherein the moving image to which encoding auxiliary information related to encoding is added An acquisition step of acquiring image data from outside; an extraction step of extracting the encoding auxiliary information from the moving image data; a decoding step of decoding a moving image from the moving image data; A re-encoding step of re-encoding the moving image based on an encoding condition different from the encoding condition based on the encoding condition. 5. A computer-readable recording medium storing a program for re-encoding moving image data encoded according to a predetermined encoding condition while changing the encoding condition, wherein the program is a computer-readable recording medium. An acquisition step of externally acquiring moving image data to which encoding auxiliary information relating to encoding is added; an extraction step of extracting the encoding auxiliary information from the moving image data; and a moving image from the moving image data. A decoding step of decoding, and a re-encoding step of re-encoding the moving image under an encoding condition different from the encoding condition based on the acquired encoding auxiliary information. A readable recording medium.