JP3724202B2 - Image data processing apparatus and method, and recording medium - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image data processing apparatus and method, and Record With regard to the medium, in particular, a moving image signal is recorded on a recording medium such as a magneto-optical disk or a magnetic tape, and is reproduced and displayed on a display or the like that can be viewed in stereo. An image data processing apparatus and method suitable for use when transmitting a moving image signal from a transmission side to a reception side via a transmission line, and receiving and displaying the reception side on a reception side, etc. Record It relates to the medium.
[0002]
[Prior art]
For example, in a system that transmits a moving image signal to a remote place such as a video conference system and a videophone system, in order to efficiently use a transmission path, line correlation and interframe correlation of video signals are used. The image signal is compression encoded.
[0003]
When the image signal is compression-encoded, encoding is performed so that the generated bit stream has a predetermined bit rate. However, in actual operation, it may be necessary to convert the bit rate of the bit stream due to the circumstances of the transmission path. In such a case, a method is generally used in which the encoded information is once decoded by a transcoder 131 as shown in FIG. 68 and is encoded again so that the bit rate becomes a predetermined value. In the case of the example in FIG. 68, the bit stream transmitted at 10 Mbps is decoded by the decoding device 132 and supplied to the encoding device 133 as a digital video signal, and the bit rate is 5 Mbps by the encoding device 133. Is encoded and output.
[0004]
[Problems to be solved by the invention]
When re-encoding the video signal in this way, the encoding device 133 requires a motion detection unit 134 for detecting line correlation and inter-frame correlation of the video signal, as shown in FIG. There was a problem of increasing the scale of.
[0005]
For example, in a broadcasting station, video editing is performed in units of seconds, so it is preferable that image information of a frame is independent of image information of other frames. Therefore, as shown in FIG. 70, frames constituting a GOP (Group of Picture) that is a set of frames in which information is correlated so that image quality does not deteriorate even when transferred at a low bit rate (3 to 9 Mbps). The bit stream output from the long GOP encoding device 133-1 having a large number is converted into a short GOP with a small number of frames constituting the GOP by the encoding device 133-2 of the broadcasting station, and the high bit rate (18 To 50 Mbps), and after the editing is completed, the encoding device 133-3 converts the data again into a Long GOP and outputs the result. Thus, when encoding and decoding of image information are repeated, there is a problem that the image information is deteriorated because an encoding parameter used for each encoding changes.
[0006]
The present invention has been made in view of such a situation, and by performing re-encoding using a motion vector calculated in the past, the scale of the apparatus is reduced, and image degradation due to re-encoding is reduced. It is possible to suppress.
[0007]
[Means for Solving the Problems]
The image data processing apparatus of the present invention extracts a first encoding parameter from a first bit stream encoded with a first encoding parameter, and uses the first encoding parameter to From the decoding means for decoding the bit stream into image data, and the first bit stream encoded with the first encoding parameter, An encoding parameter obtained by excluding the first encoding parameter from the past plural generations is used as an encoding parameter obtained by removing the first encoding parameter from the past plural generation encoding parameters as a first history encoding parameter. As the first history encoding parameter Included in the history information extracting means for extracting the first history information including the first encoding parameter extracted by the decoding means and the first history information extracted by the history information extracting means. First history encoding parameter Output together with the image data decoded by the decoding means, the first encoding parameter output by the output means, First history encoding parameter For reuse in the encoding process Coding parameters as reuse coding parameters An acquisition means for acquiring the image data output by the output means, a first encoding parameter output by the output means, and Second history information including, as a second history coding parameter, a coding parameter obtained by removing the reuse coding parameter obtained by the obtaining unit from the first history coding parameter. History information generating means for generating and acquired by the acquiring means Reuse coding parameters And encoding the image data acquired by the acquisition unit to generate a second bit stream, and the acquisition unit acquires Reuse coding parameters And encoding means for multiplexing the second history information extracted by the history information extraction means into the generated second bit stream.
[0008]
The image data processing method of the present invention extracts a first encoding parameter from a first bit stream encoded with a first encoding parameter, and uses the first encoding parameter to From the decoding step of decoding the bit stream into image data, and the first bit stream encoded with the first encoding parameter, An encoding parameter obtained by removing the first encoding parameter from the past multiple generation encoding parameters is used as the first history encoding parameter. The first history information extracted by the history information extraction step of extracting the first history information including the first encoding parameter extracted by the processing of the decoding step and the processing of the history information extraction step. First history encoding parameter Are output together with the image data decoded by the process of the decoding step, the first encoding parameter output by the process of the output step, and First history encoding parameter For reuse in the encoding process Coding parameters as reuse coding parameters An acquisition step for acquiring the image data output by the output step, a first encoding parameter output by the processing of the output step, and Second history information including, as a second history coding parameter, a coding parameter obtained by removing the reuse coding parameter obtained by the processing of the obtaining step from the first history coding parameter. Obtained by the history information generation step and the acquisition step processing Reuse coding parameters Based on the above, the image data acquired by the process of the acquisition step is encoded to generate the second bit stream, and the image data acquired by the process of the acquisition step Reuse coding parameters And an encoding step of multiplexing the second history information generated by the history information generation step into the generated second bit stream.
[0009]
The recording medium of the present invention extracts the first encoding parameter from the first bit stream encoded with the first encoding parameter, and uses the first encoding parameter to generate the first bit stream. A decoding step of decoding the image data into a first bitstream encoded with the first encoding parameter, An encoding parameter obtained by removing the first encoding parameter from the past multiple generation encoding parameters is used as the first history encoding parameter. The first history information extracted by the history information extraction step of extracting the first history information including the first encoding parameter extracted by the processing of the decoding step and the processing of the history information extraction step. First history encoding parameter Are output together with the image data decoded by the process of the decoding step, the first encoding parameter output by the process of the output step, and First history encoding parameter For reuse in the encoding process. Coding parameters as reuse coding parameters An acquisition step for acquiring the image data output by the output step, a first encoding parameter output by the processing of the output step, and Second history information including, as a second history coding parameter, a coding parameter obtained by removing the reuse coding parameter obtained by the processing of the obtaining step from the first history coding parameter. Obtained by the history information generation step and the acquisition step processing Reuse coding parameters Based on the above, the image data acquired by the process of the acquisition step is encoded to generate the second bit stream, and the image data acquired by the process of the acquisition step Reuse coding parameters And a computer-readable program for executing a process including an encoding step of multiplexing the second history information generated by the history information generation step into the generated second bitstream. It is characterized by that.
[0010]
In the image data processing apparatus and method and the recording medium of the present invention, the first encoding parameter is extracted from the first bit stream encoded with the first encoding parameter, and the first encoding parameter is extracted. The first bitstream is decoded using and from the first bitstream encoded with the first encoding parameter, An encoding parameter obtained by removing the first encoding parameter from the past multiple generation encoding parameters is used as the first history encoding parameter. The first history information including is extracted and included in the extracted first encoding parameter and the extracted first history information. First history encoding parameter Are output together with the decoded image data, and the output first encoding parameter and First history encoding parameter For reuse in the encoding process Reuse coding parameters Is obtained, the output image data is obtained, and the output first encoding parameter and Second history information including a coding parameter obtained by removing a reuse coding parameter from the first history coding parameter as a second history coding parameter Was generated and retrieved Reuse coding parameters The image data is encoded and a second bit stream is generated, and the generated second bit stream is Reuse coding parameters And the second history information is multiplexed.
[0011]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below, but in order to clarify the correspondence between each means of the invention described in the claims and the following embodiments, in parentheses after each means, The features of the present invention will be described with the corresponding embodiment (however, an example) added.
[0012]
The image data processing apparatus of the present invention extracts a first encoding parameter from a first bit stream encoded with a first encoding parameter, and uses the first encoding parameter to From decoding means (for example, the decoding device 102 in FIG. 15) for decoding the bit stream into image data, and the first bit stream encoded with the first encoding parameter, An encoding parameter obtained by removing the first encoding parameter from the past multiple generation encoding parameters is used as the first history encoding parameter. The history information extracting means for extracting the first history information including the history information extracting means (for example, the history decoding apparatus 104 in FIG. 15), the first encoding parameter extracted by the decoding means, and the first information extracted by the history information extracting means. Included in history information First history encoding parameter Are output together with the image data decoded by the decoding means (for example, the encoding parameter multiplexer 103 in FIG. 15), the first encoding parameter output by the output means, First history encoding parameter For reuse in the encoding process Coding parameters as reuse coding parameters Acquisition means (for example, the encoding parameter separation device 105 in FIG. 15) for acquiring the image data output by the output means, the first encoding parameter output by the output means, and Second history information including, as a second history coding parameter, a coding parameter obtained by removing the reuse coding parameter obtained by the obtaining unit from the first history coding parameter. Obtained by the history information generation means (for example, the history encoding device 107 in FIG. 15) and the acquisition means. Reuse coding parameters And encoding the image data acquired by the acquisition unit to generate a second bit stream, and the acquisition unit acquires Reuse coding parameters And encoding means (for example, the encoding device 106 in FIG. 15) that multiplexes the second history information generated by the history information generation means into the generated second bit stream. To do.
[0013]
However, of course, this description does not mean that each means is limited to the description.
[0014]
Before describing a transcoder to which the present invention is applied, compression coding of a moving image signal will be described. In this specification, the term “system” means an overall apparatus constituted by a plurality of apparatuses and means.
[0015]
For example, in a system that transmits a moving image signal to a remote place such as a video conference system and a video phone system, in order to efficiently use a transmission path, the line correlation of video signals and the correlation between frames are used. An image signal is compressed and encoded.
[0016]
When line correlation is used, an image signal can be compressed by, for example, DCT (discrete cosine transform) processing.
[0017]
Further, when the inter-frame correlation is used, the image signal can be further compressed and encoded. For example, as shown in FIG. 1, when frame images PC1 to PC3 are generated at times t1 to t3, the difference between the image signals of the frame images PC1 and PC2 is calculated to generate PC12. The difference between the images PC2 and PC3 is calculated to generate PC23. Normally, images of frames that are temporally adjacent do not have such a large change. Therefore, when the difference between them is calculated, the difference signal has a small value. Therefore, if this difference signal is encoded, the code amount can be compressed.
[0018]
However, if only the differential signal is transmitted, the original image cannot be restored. Therefore, the image of each frame is set to one of three types of pictures, ie, I picture, P picture, or B picture, and the image signal is compressed and encoded.
[0019]
That is, for example, as shown in FIG. 2, the image signals of 17 frames from frames F1 to F17 are set as a group of pictures (GOP), which is a unit of processing. The image signal of the leading frame F1 is encoded as an I picture, the second frame F2 is processed as a B picture, and the third frame F3 is processed as a P picture. Hereinafter, the fourth and subsequent frames F4 to F17 are alternately processed as a B picture or a P picture.
[0020]
As an image signal of an I picture, the image signal for one frame is transmitted as it is. On the other hand, as the picture signal of the P picture, basically, as shown in FIG. 2, the difference from the picture signal of the I picture or P picture preceding in time is transmitted. Further, as an image signal of a B picture, basically, as shown in FIG. 3, a difference from the average value of both the temporally preceding frame and the succeeding frame is obtained, and the difference is encoded.
[0021]
FIG. 4 shows the principle of a method for encoding a moving image signal in this way. As shown in the figure, since the first frame F1 is processed as an I picture, it is transmitted as it is to the transmission path as transmission data F1X (intra-picture coding). In contrast, since the second frame F2 is processed as a B picture, the difference between the temporally preceding frame F1 and the average value of the temporally following frame F3 is calculated, and the difference is calculated as It is transmitted as transmission data F2X.
[0022]
However, there are four types of processing as the B picture in more detail. The first processing is to transmit the data of the original frame F2 as it is as transmission data F2X (SP1) (intra coding), and is the same processing as in the case of an I picture. The second process is to calculate a difference from the temporally subsequent frame F3 and transmit the difference (SP2) (backward predictive coding). The third process is to transmit a difference (SP3) from the temporally preceding frame F1 (forward prediction coding). Further, the fourth process is to generate a difference (SP4) between the average value of the temporally preceding frame F1 and the succeeding frame F3 and transmit this as transmission data F2X (bidirectional predictive coding). .
[0023]
In practice, the method that minimizes the transmission data among the four methods described above is employed.
[0024]
Note that when transmitting difference data, a motion vector x1 (motion vector between frames F1 and F2) between the frame image (predicted image) for which the difference is to be calculated (forward prediction) or x2 (Motion vector between frames F3 and F2) (for backward prediction) or both x1 and x2 (for bidirectional prediction) are transmitted along with the difference data.
[0025]
In addition, the frame F3 of the P picture uses the temporally preceding frame F1 as a predicted image, the difference signal (SP3) from this frame, and the motion vector x3 are calculated and transmitted as transmission data F3X (forward prediction) Coding). Alternatively, the data of the original frame F3 is transmitted as it is as data F3X (SP1) (intra coding). As to which method is used for transmission, a method with less transmission data is selected as in the case of B pictures.
[0026]
FIG. 5 shows an example of the configuration of an apparatus that encodes and transmits a moving image signal and decodes it based on the principle described above. The encoding device 1 encodes an input video signal and transmits it to a recording medium 3 as a transmission path. The decoding device 2 reproduces the signal recorded on the recording medium 3, decodes it, and outputs it.
[0027]
In the encoding device 1, the input video signal is input to the preprocessing circuit 11, where the luminance signal and the color signal (in this embodiment, the color difference signal) are separated, and each of the A / D converter 12, At 13, the analog signal is converted into a digital signal. The video signals converted into digital signals by the A / D converters 12 and 13 are supplied to the frame memory 14 and stored therein. The frame memory 14 stores the luminance signal in the luminance signal frame memory 15 and the color difference signal in the color difference signal frame memory 16, respectively.
[0028]
The format conversion circuit 17 converts the frame format signal stored in the frame memory 14 into a block format signal. That is, as shown in FIG. 6, the video signal stored in the frame memory 14 is frame format data as shown in FIG. 6A in which V lines of H dots are collected per line. Yes. The format conversion circuit 17 divides the signal of one frame into M slices in units of 16 lines as shown in FIG. 6B. Each slice is divided into M macroblocks. As shown in FIG. 6C, the macro block is composed of a luminance signal corresponding to 16 × 16 pixels (dots), and this luminance signal is further divided into blocks Y [1 in units of 8 × 8 dots. ] To Y [4]. The 16 × 16 dot luminance signal corresponds to an 8 × 8 dot Cb signal and an 8 × 8 dot Cr signal.
[0029]
In this way, the data converted into the block format is supplied from the format conversion circuit 17 to the encoder 18 where encoding (encoding) is performed. Details thereof will be described later with reference to FIG.
[0030]
The signal encoded by the encoder 18 is output to the transmission path as a bit stream. For example, it is supplied to the recording circuit 19 and recorded on the recording medium 3 as a digital signal.
[0031]
Data reproduced from the recording medium 3 by the reproducing circuit 30 is supplied to the decoder 31 of the decoding device 2 and decoded. Details of the decoder 31 will be described later with reference to FIG.
[0032]
The data decoded by the decoder 31 is input to the format conversion circuit 32 and converted from the block format to the frame format. The luminance signal in the frame format is supplied to and stored in the luminance signal frame memory 34 of the frame memory 33, and the color difference signal is supplied to and stored in the color difference signal frame memory 35. The luminance signal and the color difference signal read from the luminance signal frame memory 34 and the color difference signal frame memory 35 are converted into analog signals by the D / A converters 36 and 37, respectively, and supplied to the post-processing circuit 38. The post-processing circuit 38 synthesizes and outputs the luminance signal and the color difference signal.
[0033]
Next, the configuration of the encoder 18 will be described with reference to FIG. The encoded image data is input to the motion vector detection circuit 50 in units of macro blocks. The motion vector detection circuit 50 processes the image data of each frame as an I picture, P picture, or B picture according to a predetermined sequence set in advance. It is predetermined whether the image of each frame that is sequentially input is processed as an I, P, or B picture (for example, as shown in FIGS. 2 and 3, frames F1 to F17). Are processed as I, B, P, B, P,... B, P).
[0034]
Image data of a frame processed as an I picture (for example, frame F1) is transferred from the motion vector detection circuit 50 to the front original image portion 51a of the frame memory 51, stored, and processed as a B picture (for example, a frame) The image data of F2) is transferred and stored in the original image portion 51b, and the image data of a frame (for example, frame F3) processed as a P picture is transferred and stored in the rear original image portion 51c.
[0035]
At the next timing, when an image of a frame to be further processed as a B picture (frame F4) or a P picture (frame F5) is input, the first P picture stored in the rear original image portion 51c until then. The image data of (frame F3) is transferred to the forward original image portion 51a, the image data of the next B picture (frame F4) is stored (overwritten) in the original image portion 51b, and the next P picture (frame F5) Is stored (overwritten) in the rear original image portion 51c. Such an operation is sequentially repeated.
[0036]
The signal of each picture stored in the frame memory 51 is read therefrom, and the prediction mode switching circuit 52 performs frame prediction mode processing or field prediction mode processing.
[0037]
Furthermore, under the control of the prediction determination circuit 54, the calculation unit 53 performs intra-image prediction, forward prediction, backward prediction, or bidirectional prediction. Which of these processes is performed is determined corresponding to a prediction error signal (a difference between a reference image to be processed and a predicted image corresponding thereto). For this reason, the motion vector detection circuit 50 generates the absolute value sum (or sum of squares) of the prediction error signal used for this determination.
[0038]
Here, the frame prediction mode and the field prediction mode in the prediction mode switching circuit 52 will be described.
[0039]
When the frame prediction mode is set, the prediction mode switching circuit 52 supplies the four luminance blocks Y [1] to Y [4] supplied from the motion vector detection circuit 50 to the calculation unit 53 in the subsequent stage as they are. Output. In other words, in this case, as shown in FIG. 8, the data of the odd-numbered field lines and the data of the even-numbered field lines are mixed in each luminance block. In this frame prediction mode, prediction is performed in units of four luminance blocks (macro blocks), and one motion vector corresponds to the four luminance blocks.
[0040]
On the other hand, in the field prediction mode, the prediction mode switching circuit 52 receives a signal input from the motion vector detection circuit 50 with the configuration shown in FIG. 8 among four luminance blocks as shown in FIG. The luminance blocks Y [1] and Y [2] are composed of, for example, only odd-field line dots, and the other two luminance blocks Y [3] and Y [4] are even-field line dots. And output to the calculation unit 53. In this case, one motion vector corresponds to the two luminance blocks Y [1] and Y [2], and the other two luminance blocks Y [3] and Y [4]. Thus, one other motion vector is associated.
[0041]
The motion vector detection circuit 50 outputs the sum of absolute values of prediction errors in the frame prediction mode and the sum of absolute values of prediction errors in the field prediction mode to the prediction mode switching circuit 52. The prediction mode switching circuit 52 compares the absolute value sum of the prediction errors in the frame prediction mode and the field prediction mode, performs a process corresponding to the prediction mode having a small value, and outputs the data to the calculation unit 53.
[0042]
However, such processing is actually performed by the motion vector detection circuit 50. That is, the motion vector detection circuit 50 outputs a signal having a configuration corresponding to the determined mode to the prediction mode switching circuit 52, and the prediction mode switching circuit 52 outputs the signal as it is to the calculation unit 53 in the subsequent stage.
[0043]
In the case of the frame prediction mode, the color difference signal is supplied to the arithmetic unit 53 in a state where data of odd field lines and data of even field lines coexist as shown in FIG. In the case of the field prediction mode, as shown in FIG. 9, the upper half (four lines) of the color difference blocks Cb and Cr are the odd field color difference signals corresponding to the luminance blocks Y [1] and Y [2]. The lower half (four lines) is the color difference signal of the even field corresponding to the luminance blocks Y [3] and Y [4].
[0044]
In addition, the motion vector detection circuit 50 uses a prediction error for determining whether to perform intra prediction, forward prediction, backward prediction, or bidirectional prediction in the prediction determination circuit 54 as shown below. Generate sum of absolute values.
[0045]
That is, as the sum of the absolute values of prediction errors in intra-picture prediction, the absolute value | ΣAij | of the sum ΣAij of the macroblock signal Aij of the reference picture and the sum Σ | Aij | of the absolute value | Aij | of the macroblock signal Aij Find the difference. Further, as the absolute value sum of the prediction errors of the forward prediction, the sum Σ | Aij− of the absolute value | Aij−Bij | of the difference Aij−Bij between the macroblock signal Aij of the reference image and the macroblock signal Bij of the predicted image Bij | is obtained. Also, the absolute value sum of the prediction errors of the backward prediction and the bidirectional prediction is obtained in the same manner as in the forward prediction (by changing the prediction image to a prediction image different from that in the forward prediction).
[0046]
These sums of absolute values are supplied to the prediction determination circuit 54. The prediction determination circuit 54 selects the smallest one of the absolute value sums of the prediction errors of the forward prediction, the backward prediction, and the bidirectional prediction as the absolute value sum of the prediction errors of the inter prediction. Further, the absolute value sum of the prediction error of the inter prediction and the absolute value sum of the prediction error of the intra prediction are compared, and the smaller one is selected, and the mode corresponding to the selected absolute value sum is set as the prediction mode. select. That is, if the sum of the absolute values of the prediction errors of intra prediction is smaller, the intra prediction mode is set. If the absolute value sum of the prediction errors of inter prediction is smaller, the mode with the smallest corresponding absolute value sum among the forward prediction, backward prediction, and bidirectional prediction modes is set.
[0047]
As described above, the motion vector detection circuit 50 has the configuration corresponding to the mode selected by the prediction mode switching circuit 52 out of the frame or field prediction modes for the signal of the macroblock of the reference image. The motion vector between the prediction image corresponding to the prediction mode selected by the prediction determination circuit 54 of the four prediction modes and the reference image is detected, and the variable-length encoding circuit 58 is detected. And output to the motion compensation circuit 64. As described above, the motion vector having the minimum absolute value sum of the corresponding prediction errors is selected.
[0048]
When the motion vector detection circuit 50 reads I-picture image data from the front original image portion 51a, the prediction determination circuit 54 uses a frame or field (image) prediction mode (a mode in which motion compensation is not performed) as a prediction mode. And the switch 53d of the calculation unit 53 is switched to the contact a side. As a result, the I-picture image data is input to the DCT mode switching circuit 55.
[0049]
As shown in FIG. 10 or FIG. 11, the DCT mode switching circuit 55 is configured to separate the data of four luminance blocks in a state where odd-numbered field lines and even-numbered field lines coexist (frame DCT mode). Any one of the states (field DCT mode) is output to the DCT circuit 56.
[0050]
That is, the DCT mode switching circuit 55 compares the coding efficiency when DCT processing is performed with mixed odd-numbered field data and even-numbered field data, and the coding efficiency when DCT processing is performed in a separated state. Choose a good mode.
[0051]
For example, as shown in FIG. 10, the input signal has a configuration in which odd-numbered field and even-numbered field lines coexist, and calculates the difference between the signal of the odd-numbered line and the even-numbered line adjacent to each other. Further, the sum (or sum of squares) of the absolute values is obtained.
[0052]
Further, as shown in FIG. 11, the input signal has a configuration in which the odd-numbered field and even-numbered field lines are separated from each other, and the difference between the signals of the odd-numbered adjacent fields and the signals of the even-numbered lines are The sum of absolute values (or the sum of squares) of each is calculated.
[0053]
Furthermore, both (absolute value sum) are compared, and a DCT mode corresponding to a small value is set. That is, if the former is smaller, the frame DCT mode is set, and if the latter is smaller, the field DCT mode is set.
[0054]
Then, data having a configuration corresponding to the selected DCT mode is output to the DCT circuit 56, and a DCT flag indicating the selected DCT mode is output to the variable length encoding circuit 58 and the motion compensation circuit 64.
[0055]
As is apparent from a comparison between the prediction mode (FIGS. 8 and 9) in the prediction mode switching circuit 52 and the DCT mode (FIGS. 10 and 11) in the DCT mode switching circuit 55, the luminance block has both of them. The data structure in the mode is substantially the same.
[0056]
When the frame prediction mode (mode in which odd lines and even lines are mixed) is selected in the prediction mode switching circuit 52, the frame DCT mode (mode in which odd lines and even lines are mixed) also in the DCT mode switching circuit 55. When the prediction mode switching circuit 52 selects the field prediction mode (the mode in which the data of the odd field and the even field are separated), the DCT mode switching circuit 55 selects the field DCT mode ( There is a high possibility that the mode in which the data of the odd field and the even field are separated is selected.
[0057]
However, the mode is not always selected in this way, and the prediction mode switching circuit 52 determines the mode so that the absolute value sum of the prediction errors is small, and the DCT mode switching circuit 55 performs the encoding. The mode is determined so that the efficiency is good.
[0058]
The I-picture image data output from the DCT mode switching circuit 55 is input to the DCT circuit 56, subjected to DCT processing, and converted into DCT coefficients. This DCT coefficient is input to the quantization circuit 57, quantized with a quantization scale corresponding to the data storage amount (buffer storage amount) of the transmission buffer 59, and then input to the variable length encoding circuit 58.
[0059]
The variable length encoding circuit 58 corresponds to the quantization scale (scale) supplied from the quantization circuit 57, and converts image data (in this case, I picture data) supplied from the quantization circuit 57, for example, It is converted into a variable length code such as a Huffman code and output to the transmission buffer 59.
[0060]
The variable length encoding circuit 58 also determines whether a quantization scale (scale) is set by the quantization circuit 57 and a prediction mode (intra-picture prediction, forward prediction, backward prediction, or bidirectional prediction) is set by the prediction determination circuit 54. Mode), a motion vector from the motion vector detection circuit 50, a prediction flag (a flag indicating whether the frame prediction mode or the field prediction mode is set) from the prediction mode switching circuit 52, and the DCT output from the DCT mode switching circuit 55 A flag (a flag indicating whether the frame DCT mode or the field DCT mode is set) is input, and these are also variable-length encoded.
[0061]
The transmission buffer 59 temporarily stores input data and outputs data corresponding to the storage amount to the quantization circuit 57. When the remaining data amount increases to the allowable upper limit value, the transmission buffer 59 increases the quantization scale of the quantization circuit 57 by the quantization control signal, thereby reducing the data amount of the quantized data. On the other hand, when the remaining data amount decreases to the allowable lower limit value, the transmission buffer 59 reduces the quantization scale of the quantization circuit 57 by the quantization control signal, thereby reducing the data amount of the quantized data. Increase. In this way, overflow or underflow of the transmission buffer 59 is prevented.
[0062]
The data stored in the transmission buffer 59 is read at a predetermined timing, output to the transmission path, and recorded on the recording medium 3 via the recording circuit 19, for example.
[0063]
On the other hand, the I picture data output from the quantization circuit 57 is input to the inverse quantization circuit 60 and inversely quantized in accordance with the quantization scale supplied from the quantization circuit 57. The output of the inverse quantization circuit 60 is input to an IDCT (Inverse Discrete Cosine Transform) circuit 61, subjected to inverse discrete cosine transform processing, and then supplied to and stored in the forward prediction image unit 63a of the frame memory 63 via the calculator 62. Is done.
[0064]
The motion vector detection circuit 50, when processing the image data of each frame sequentially input as, for example, pictures of I, B, P, B, P, B. After the image data is processed as an I picture, the image data of the next input frame is further processed as a P picture before the image of the next input frame is processed as a B picture. This is because a B picture is accompanied by backward prediction, and therefore cannot be decoded unless a P picture as a backward predicted image is prepared first.
[0065]
Therefore, the motion vector detection circuit 50 starts processing the image data of the P picture stored in the rear original image portion 51c after the processing of the I picture. As in the case described above, the absolute value sum of the inter-frame difference (prediction error) in units of macroblocks is supplied from the motion vector detection circuit 50 to the prediction mode switching circuit 52 and the prediction determination circuit 54. The prediction mode switching circuit 52 and the prediction determination circuit 54 correspond to the absolute value sum of the prediction errors of the macroblock of this P picture, or the frame / field prediction mode, or intra-picture prediction, forward prediction, backward prediction, or bidirectional prediction. Set the prediction mode.
[0066]
When the intra prediction mode is set, the calculation unit 53 switches the switch 53d to the contact a side as described above. Accordingly, this data is transmitted to the transmission line via the DCT mode switching circuit 55, the DCT circuit 56, the quantization circuit 57, the variable length coding circuit 58, and the transmission buffer 59, similarly to the I picture data. Further, this data is supplied to and stored in the backward predicted image unit 63b of the frame memory 63 via the inverse quantization circuit 60, the IDCT circuit 61, and the calculator 62.
[0067]
When the forward prediction mode is set, the switch 53d is switched to the contact point b, and image (in this case, an I picture image) data stored in the forward prediction image portion 63a of the frame memory 63 is read. Then, the motion compensation circuit 64 performs motion compensation corresponding to the motion vector output from the motion vector detection circuit 50. That is, the motion compensation circuit 64, when the setting of the forward prediction mode is instructed from the prediction determination circuit 54, the read address of the forward prediction image unit 63a, and the macroblock currently being output by the motion vector detection circuit 50. Data is read out from the position corresponding to the position by an amount corresponding to the motion vector, and predicted image data is generated.
[0068]
The predicted image data output from the motion compensation circuit 64 is supplied to the calculator 53a. The computing unit 53a subtracts the prediction image data corresponding to the macroblock supplied from the motion compensation circuit 65 from the macroblock data of the reference image supplied from the prediction mode switching circuit 52, and the difference (prediction error). ) Is output. The difference data is transmitted to the transmission line via the DCT mode switching circuit 55, the DCT circuit 56, the quantization circuit 57, the variable length coding circuit 58, and the transmission buffer 59. The difference data is locally decoded by the inverse quantization circuit 60 and the IDCT circuit 61 and input to the calculator 62.
[0069]
The calculator 62 is also supplied with the same data as the predicted image data supplied to the calculator 53a. The calculator 62 adds the predicted image data output from the motion compensation circuit 64 to the difference data output from the IDCT circuit 61. As a result, image data of the original (decoded) P picture is obtained. The image data of the P picture is supplied to and stored in the backward predicted image unit 63b of the frame memory 63.
[0070]
As described above, the motion vector detection circuit 50 stores the data of the I picture and the P picture in the forward predicted image unit 63a and the backward predicted image unit 63b, respectively, and then executes the process of the B picture. The prediction mode switching circuit 52 and the prediction determination circuit 54 set a frame / field mode corresponding to the magnitude of the absolute value sum of inter-frame differences in units of macroblocks, and set the prediction mode to an intra-picture prediction mode, Set to either the forward prediction mode, backward prediction mode, or bidirectional prediction mode.
[0071]
As described above, in the intra prediction mode or the forward prediction mode, the switch 53d is switched to the contact point a or b. At this time, the same processing as in the case of the P picture is performed, and data is transmitted.
[0072]
On the other hand, when the backward prediction mode or the bidirectional prediction mode is set, the switch 53d is switched to the contact c or d, respectively.
[0073]
In the backward prediction mode in which the switch 53d is switched to the contact point c, the image data (in this case, a P picture image) stored in the backward prediction image unit 63b is read out, and the motion compensation circuit 64 Motion compensation is performed corresponding to the motion vector output from the vector detection circuit 50. That is, the motion compensation circuit 64, when the setting of the backward prediction mode is instructed by the prediction determination circuit 54, the read address of the backward prediction image unit 63b, the macro block currently output by the motion vector detection circuit 50. Data is read out from the position corresponding to the position by an amount corresponding to the motion vector, and predicted image data is generated.
[0074]
The predicted image data output from the motion compensation circuit 64 is supplied to the calculator 53b. The computing unit 53b subtracts the predicted image data supplied from the motion compensation circuit 64 from the macroblock data of the reference image supplied from the prediction mode switching circuit 52, and outputs the difference. The difference data is transmitted to the transmission line via the DCT mode switching circuit 55, the DCT circuit 56, the quantization circuit 57, the variable length coding circuit 58, and the transmission buffer 59.
[0075]
In the bidirectional prediction mode in which the switch 53d is switched to the contact point d, the image (in this case, the image of I picture) data stored in the forward predicted image unit 63a and the backward predicted image unit 63b are stored. Image (in this case, P picture image) data is read out, and motion compensation is performed by the motion compensation circuit 64 corresponding to the motion vector output from the motion vector detection circuit 50.
[0076]
That is, in the motion compensation circuit 64, when the setting of the bidirectional prediction mode is instructed by the prediction determination circuit 54, the motion vector detection circuit 50 now outputs the read addresses of the forward prediction image unit 63a and the backward prediction image unit 63b. The data is read out from the position corresponding to the position of the macroblock being read by shifting the motion vector by the amount corresponding to the motion vector (in this case, the motion vector is for the forward prediction image and the backward prediction image), and prediction image data is generated. To do.
[0077]
The predicted image data output from the motion compensation circuit 64 is supplied to the calculator 53c. The computing unit 53c subtracts the average value of the predicted image data supplied from the motion compensation circuit 64 from the macroblock data of the reference image supplied from the motion vector detection circuit 50, and outputs the difference. The difference data is transmitted to the transmission line via the DCT mode switching circuit 55, the DCT circuit 56, the quantization circuit 57, the variable length coding circuit 58, and the transmission buffer 59.
[0078]
The B picture image is not stored in the frame memory 63 because it is not a predicted image of another image.
[0079]
Note that, in the frame memory 63, the forward predicted image unit 63a and the backward predicted image unit 63b are subjected to bank switching as necessary, and those stored in one or the other with respect to a predetermined reference image, It can be switched and output as a predicted image or a backward predicted image.
[0080]
In the above description, the luminance block is mainly described, but the chrominance block is also processed and transmitted in units of macroblocks shown in FIGS. Note that the motion vector when processing the color difference block is obtained by halving the motion vector of the corresponding luminance block in the vertical direction and the horizontal direction.
[0081]
FIG. 12 is a block diagram showing a configuration of the decoder 31 of FIG. The encoded image data transmitted through the transmission path (recording medium 3) is received by a receiving circuit (not shown), reproduced by a reproducing device, temporarily stored in the receiving buffer 81, and then decoded by a decoding circuit 90. Is supplied to the variable length decoding circuit 82. The variable length decoding circuit 82 performs variable length decoding on the data supplied from the reception buffer 81, outputs a motion vector, a prediction mode, a prediction flag, and a DCT flag to the motion compensation circuit 87, and dequantizes the quantization scale. In addition to outputting to the circuit 83, the decoded image data is output to the inverse quantization circuit 83.
[0082]
The inverse quantization circuit 83 inversely quantizes the image data supplied from the variable length decoding circuit 82 according to the quantization scale also supplied from the variable length decoding circuit 82 and outputs the result to the IDCT circuit 84. The data (DCT coefficient) output from the inverse quantization circuit 83 is subjected to inverse discrete cosine transform processing by the IDCT circuit 84 and supplied to the computing unit 85.
[0083]
When the image data supplied from the IDCT circuit 84 to the computing unit 85 is I picture data, the data is output from the computing unit 85, and image data (P or B picture data that is input later to the computing unit 85). ) Is supplied to and stored in the forward predicted image portion 86a of the frame memory 86. The data is output to the format conversion circuit 32 (FIG. 5).
[0084]
When the image data supplied from the IDCT circuit 84 is P picture data having the image data of the previous frame as predicted image data and is data in the forward prediction mode, the image data is stored in the forward predicted image unit 86a of the frame memory 86. The stored image data of the previous frame (I picture data) is read out, and motion compensation corresponding to the motion vector output from the variable length decoding circuit 82 is performed by the motion compensation circuit 87. The calculator 85 adds the image data (difference data) supplied from the IDCT circuit 84 and outputs the result. This added data, that is, the decoded P picture data is stored in the rear of the frame memory 86 in order to generate predicted picture data of the picture data (B picture or P picture data) to be input later to the calculator 85. The predicted image unit 86b is supplied and stored.
[0085]
Even in the case of P picture data, the intra prediction mode data is not processed by the computing unit 85 and stored in the backward predicted image unit 86b as is the case with the I picture data.
[0086]
Since this P picture is an image to be displayed next to the next B picture, at this point of time, it is not yet output to the format conversion circuit 32 (as described above, the P picture input after the B picture is Processed and transmitted before B picture).
[0087]
When the image data supplied from the IDCT circuit 84 is B picture data, it is stored in the forward predicted image unit 86a of the frame memory 86 corresponding to the prediction mode supplied from the variable length decoding circuit 82. I-picture image data (in the forward prediction mode), P-picture image data stored in the backward prediction image portion 86b (in the backward prediction mode), or both image data (in the bidirectional prediction mode) In the motion compensation circuit 87, motion compensation corresponding to the motion vector output from the variable length decoding circuit 82 is performed, and a predicted image is generated. However, when motion compensation is not required (in the case of intra-picture prediction mode), a predicted image is not generated.
[0088]
In this way, the data subjected to motion compensation by the motion compensation circuit 87 is added to the output of the IDCT circuit 84 in the computing unit 85. This addition output is output to the format conversion circuit 32.
[0089]
However, since this addition output is B picture data and is not used for generating a predicted image of another image, it is not stored in the frame memory 86.
[0090]
After the B picture image is output, the P picture image data stored in the backward predicted image unit 86 b is read out and supplied to the computing unit 85 via the motion compensation circuit 87. However, at this time, motion compensation is not performed.
[0091]
The decoder 31 does not show circuits corresponding to the prediction mode switching circuit 52 and the DCT mode switching circuit 55 in the encoder 18 of FIG. 5, but processes corresponding to these circuits, that is, odd fields and even numbers. The motion compensation circuit 87 executes a process of returning the configuration in which the signal of the field line is separated to the original configuration as necessary.
[0092]
In the above description, the luminance signal processing has been described, but the color difference signal processing is performed in the same manner. However, the motion vector in this case is obtained by halving the luminance signal motion vector in the vertical and horizontal directions.
[0093]
FIG. 13 shows the quality of the encoded image. Image quality (SNR: Signal to Noise Ratio) is controlled according to the picture type, I picture and P picture are of high quality, and B picture is inferior to I and P pictures. Is transmitted. This is a technique using human visual characteristics, and the visual image quality is better when the quality is oscillated than when all the image qualities are averaged. The image quality control corresponding to this picture type is executed by the quantization circuit 57 of FIG.
[0094]
FIG. 14 shows a configuration of a transcoder 101 to which the present invention is applied, and FIG. 15 shows a more detailed configuration thereof. The decoding apparatus 102 includes (multiplexed) the encoded image signal included (multiplexed) in the bitstream of a predetermined bit rate (in this example, 10 Mbps). Decoding using the current coding parameters (frame / field DCT flag, frame / field prediction flag, prediction mode, picture type, motion vector, macroblock information, and quantization scale) of the bitstream, a coding parameter multiplexing device In addition, the current encoding parameter is also output to the encoding parameter multiplexer 103.
[0095]
The decoding device 102 also decodes and separates user data included in the bitstream, and outputs it to the history decoding device 104. Although details will be described later, this user data includes generation history information including encoding parameters for the three most recent generations. On the other hand, the current encoding parameter is included in, for example, group_of_pictures_header (1), extension_and_user_data (1), picture_header (), picture_coding_extension (), extensions_data (2), picture_data (), or sequence_extension () ( FIG. 38 to be described later). The history decoding apparatus 104 decodes the input user data and outputs generation history information including encoding parameters for three generations to the encoding parameter multiplexing apparatus 103.
[0096]
Note that the decoding device 102 is obtained by changing the decoder 31 (FIG. 12) of the decoding device 2 in FIG. 5 to a decoder 111 shown in FIG. The variable length decoding circuit 112 of the decoder 111 extracts the current encoding parameter from the bit stream, supplies it to a predetermined circuit, extracts user data including generation history information, and outputs it to the history decoding device 104. Has been made. Since the other configuration of the decoder 111 is the same as that of the decoder 31, the description thereof is omitted.
[0097]
The encoding parameter multiplexing apparatus 103 writes (multiplexes) four generations of encoding parameters in an empty area of the decoded image data (details will be described with reference to FIG. 18), and performs baseband digital video. As a signal, the signal is output to the coding parameter separation device 105 which is roughly coupled (no dedicated bus for coding parameter transmission or the like is provided). The coding parameter separation device 105 separates image data and coding parameters used for coding by the coding device 106 from the baseband digital video signal and supplies them to the coding device 106.
[0098]
The coding parameter separation device 105 also extracts coding parameters for three generations excluding the coding parameters used in the coding device 106 from the input baseband digital video signal, and outputs the extracted parameters to the history coding device 107. . The history encoding device 107 writes the input encoding parameters for three generations into user data, and outputs the user data to the encoding device 106.
[0099]
The format of the image data in which the code parameter is written will be described with reference to FIGS. One macro block is composed of 16 × 16 pixels as shown in FIG. The 16 × 16 pixel data includes 8 × 8 pixel luminance signals Y [0] [x] to Y [4] [x] and 8 × 8 pixel color difference signals Cr [0] [x], Cr [ 1] [x], Cb [0] [x], and Cb [1] [x] (x = 2 to 9). For example, the luminance signal Y [0] [9] indicates the luminance signal of the pixels (8 pixels) in the first row of 8 × 8 pixels. Since the information amount of the luminance signal per pixel is 8 bits, the information amount of the luminance signal Y [0] [9] is 8 (pixels) × 8 (bits) = 64 bits. The same applies to the color difference signal.
[0100]
On the other hand, as shown in FIG. 18, the format of the image data is provided with the areas (D0 to D9) for 10 rows, so that the areas (D0, D1) for 2 rows are not necessary. Since information of 64 bits × 16 = 1024 bits can be recorded in this empty area, an encoding parameter other than the original image data is written in the area for the two rows. Note that since the encoding parameter corresponding to one macroblock has a 256-bit information amount, the encoding parameter used for the past four encodings can be recorded in this area.
[0101]
In the image data (digital video signal) transmitted from the encoding parameter multiplexing apparatus 103 to the encoding parameter separation apparatus 105, 10 lines (D0 to D9) are provided as areas for describing the luminance signal Y and the color difference signals Cr and Cb. This area is provided. However, the area where the luminance signal Y or the like is actually written is an area corresponding to eight rows D2 to D9, and the areas D0 and D1 are not used. Therefore, this 2-bit area is used as a coding parameter writing area. As a result, the encoding parameter is written into the lower 2 bits of the pixel at a predetermined position of 16 × 16 pixels in FIG.
[0102]
The encoding device 106 encodes the image data using the current encoding parameter supplied as an encoding parameter for encoding to be performed in the future, and converts the user data supplied from the history encoding device 107 into a bitstream. And output to SDTI (Serial Data Transfer Interface) 108-i (i = 1, 2,..., N) (FIG. 30 described later) at a predetermined bit rate (5 Mbps in this example). It is made like that.
[0103]
The encoding device 106 is obtained by changing the encoder 18 (FIG. 7) of the encoding device 1 in FIG. 5 to an encoder 121 shown in FIG. 19. The encoder 121 deletes the motion vector detection circuit 50, the frame memory 51, the prediction mode switching circuit 52, the prediction determination circuit 54, and the DCT mode switching circuit 55 that generate encoding parameters from the encoder 18, and the history encoding device 107 The user data to be output is variable length encoded by the variable length encoding circuit 58. Since the other configuration of the encoder 121 is the same as that of the encoder 18, the description thereof is omitted.
[0104]
Next, history decoding apparatus 104 and history encoding apparatus 107 in FIG. 15 will be further described. As shown in the figure, the history decoding device 104 includes a user data decoder 201 that decodes user data supplied from the decoding device 102, a converter 202 that converts the output of the user data decoder 201, and history information from the output of the converter 202. Is constituted by a history decoder 203 for reproducing.
[0105]
The history encoding device 107 also formats the history formatter 211 that formats the encoding parameters for three generations supplied from the encoding parameter separation device 105, the converter 212 that converts the output of the history formatter 211, and the output of the converter 212. The user data formatter 213 formats the user data format.
[0106]
The user data decoder 201 decodes the user data supplied from the decoding device 102 and outputs it to the converter 202. Although details will be described later, user data (user_data ()) is composed of user_data_start_code and user_data, and in the MPEG standard, it is prohibited to generate continuous 23-bit “0” in user_data. This is to prevent erroneous detection of start_code. In the history information, there may be such “0” of 23 bits or more which are continuous, and it is necessary to process this and convert it into converted_history_stream () (FIG. 38 described later). This conversion is performed by the converter 212 of the history encoding device 107. The converter 202 of the history decoding device 104 performs a conversion process opposite to that of the converter 212.
[0107]
The history decoder 203 generates history information from the output of the converter 202 and outputs the history information to the encoding parameter multiplexer 103.
[0108]
On the other hand, in the history encoding device 107, the history formatter 211 converts the encoding parameters for three generations supplied from the encoding parameter separation device 105 into a history information format. This format includes a fixed-length format (FIGS. 40 to 46 described later) and a variable-length format (FIG. 47 described later). Details of these will be described later.
[0109]
The history information formatted by the history formatter 211 is converted into converted_history_stream () by the converter 212. As described above, this is to prevent the start_code of user_data () from being erroneously detected. That is, although “0” having 23 or more consecutive bits exists in the history information, since “0” having 23 or more consecutive bits cannot be arranged in user_data, do not touch this prohibited item. The converter 212 converts the data.
[0110]
The user data formatter 213 adds Data_ID to converted_history_stream () supplied from the converter 212 based on FIG. 38 to be described later, and further adds user_data_stream_code to generate user_data that can be inserted into the video stream. Is output to the conversion device 106.
[0111]
FIG. 20 shows a configuration example of the history formatter 211. In the codeword converter 301 and the code length converter 305, an encoding parameter (an encoding parameter to be transmitted this time as history information) (item data) and information for specifying a stream in which the encoding parameter is arranged (for example, , Syntax name) (for example, name of sequence_header described later) (item NO.) Is supplied from the encoding parameter separation device 105. The codeword converter 301 converts the input encoding parameter into a codeword corresponding to the instructed syntax, and outputs the codeword to the barrel shifter 302. The barrel shifter 302 shifts the code word input from the code word converter 301 by an amount corresponding to the shift amount supplied from the address generation circuit 306, and outputs the code word to the switch 303 as a byte-by-byte code word. A switch 303 that is switched by a bit select signal output from the address generation circuit 306 is provided for each bit, and the code word supplied from the barrel shifter 302 is supplied to the RAM 304 for storage. The write address at this time is designated from the address generation circuit 306. In addition, when a read address is designated from the address generation circuit 306, data (codeword) stored in the RAM 304 is read and supplied to the subsequent converter 212 and, if necessary, via the switch 303. The data is again supplied to the RAM 304 and stored.
[0112]
The code length converter 305 determines the code length of the encoding parameter from the input syntax and the encoding parameter, and outputs it to the address generation circuit 306. The address generation circuit 306 generates the above-described shift amount, bit select signal, write address, or read address corresponding to the input code length, and supplies them to the barrel shifter 302, the switch 303, or the RAM 304, respectively. .
[0113]
As described above, the history formatter 211 is configured as a so-called variable length encoder, and performs variable length encoding on the input encoding parameter and outputs it.
[0114]
FIG. 21 shows a configuration example of the history decoder 203 that decodes data formatted in the history format as described above. In the history decoder 203, the encoding parameter data supplied from the converter 202 is supplied to the RAM 311 and stored therein. The write address at this time is supplied from the address generation circuit 315. The address generation circuit 315 also generates a read address at a predetermined timing and supplies it to the RAM 311. At this time, the RAM 311 reads the data stored at the read address and outputs it to the barrel shifter 312. The barrel shifter 312 shifts the input data by an amount corresponding to the shift amount output from the address generation circuit 315 and outputs the shifted data to the inverse code length converter 313 and the inverse codeword converter 314.
[0115]
The inverse code length converter 313 is also supplied from the converter 202 with the name of the syntax of the stream in which the encoding parameters are arranged. Based on the syntax, the inverse code length converter 313 obtains the code length from the input data (code word) and outputs the obtained code length to the address generation circuit 315.
[0116]
Further, the inverse codeword converter 314 decodes the data supplied from the barrel shifter 312 based on the syntax (inverse codeword) and outputs the decoded data to the encoding parameter multiplexing apparatus 103.
[0117]
Further, the inverse codeword converter 314 extracts information necessary for specifying what codeword is included (information necessary for determining a codeword delimiter), and generates an address generation circuit. It outputs to 315. The address generation circuit 315 generates a write address and a read address based on this information and the code length input from the inverse code length converter 313, outputs it to the RAM 311, generates a shift amount, and outputs it to the barrel shifter 312. To do.
[0118]
FIG. 22 shows a configuration example of the converter 212. In this example, 8-bit data is read from the read address output from the controller 326 in the buffer memory 320 arranged between the history formatter 211 and the converter 212, and a D-type flip-flop (D-FF) 321 is read. Are supplied and held. The data read from the D-type flip-flop 321 is supplied to the stuff circuit 323 and also supplied to and held in the 8-bit D-type flip-flop 322. The 8-bit data read from the D-type flip-flop 322 is combined with the 8-bit data read from the D-type flip-flop 321 and supplied to the stuff circuit 323 as 16-bit parallel data.
[0119]
The stuff circuit 323 inserts a code “1” at the position of a stuff position signal (stuff position) supplied from the controller 326 (stuffing), and outputs the data to the barrel shifter 324 as 17-bit data.
[0120]
The barrel shifter 324 shifts the input data based on the signal (shift) indicating the shift amount supplied from the controller 326, extracts 8-bit data, and outputs it to the 8-bit D-type flip-flop 325. Data held in the D-type flip-flop 325 is read from the data and supplied to the user data formatter 213 in the subsequent stage via the buffer memory 327. At this time, the controller 326 generates a write address together with the output data and supplies the write address to the buffer memory 327 interposed between the converter 212 and the user data formatter 213.
[0121]
FIG. 23 illustrates a configuration example of the stuff circuit 323. The 16-bit data input from the D-type flip-flops 322 and 321 are input to the contacts a of the switches 331-16 to 331-1, respectively. The data of the switch adjacent to the MSB side (upper side in the figure) is supplied to the contact c of the switch 331-i (i = 0 to 15). For example, the thirteenth data from the LSB supplied to the contact a of the switch 331-13 adjacent to the MSB side is supplied to the contact c of the switch 331-12, and the contact c of the switch 331-13 is supplied to the contact c of the switch 331-13. The 14th data from the LSB side supplied to the contact a of the switch 331-13 adjacent to the MSB side is supplied.
[0122]
However, the contact a of the switch 331-0 further below the switch 331-1 corresponding to the LSB is open. In addition, the contact c of the switch 331-16 corresponding to the MSB is open because there is no higher-order switch.
[0123]
Data “1” is supplied to the contact b of each of the switches 331-0 to 331-16.
[0124]
The decoder 332 switches one of the switches 331-0 to 331-16 to the contact b side in response to the signal stuff position indicating the position to insert the data “1” supplied from the controller 326, The switch on the LSB side is switched to the contact c side, and the switch on the MSB side is switched to the contact a side.
[0125]
FIG. 23 shows an example in which data “1” is inserted in the thirteenth from the LSB side. Accordingly, in this case, the switches 331-0 to 331-12 are all switched to the contact c side, the switch 331-13 is switched to the contact b side, and the switches 331-14 to 331-16 are switched to the contact c side. It has been switched to the a side.
[0126]
The converter 212 shown in FIG. 22 converts the 22-bit code into 23-bit and outputs the result.
[0127]
FIG. 24 shows the timing of the output data of each part of the converter 212 of FIG. When the controller 326 of the converter 212 generates a read address (FIG. 24A) in synchronization with the clock in units of bytes, the corresponding data is read out from the buffer memory 320 in units of bytes, and a D-type flip-flop 321 is temporarily held. The data read from the D-type flip-flop 321 (FIG. 24B) is supplied to the stuff circuit 323 and also supplied to the D-type flip-flop 322 and held therein. The data held in the D-type flip-flop 322 is further read therefrom (FIG. 24C) and supplied to the stuff circuit 323.
[0128]
Therefore, the input of the stuff circuit 323 (FIG. 24D) is the first 1-byte data D0 at the timing of the read address A1, and the 1-byte data D0 and 1 byte at the timing of the next read address A2. 2 bytes of data D1 and at the timing of the read address A3, it becomes 2 bytes of data composed of data D1 and data D2.
[0129]
The stuff circuit 323 is supplied from the controller 326 with a signal stuff position (FIG. 24E) indicating a position where data “1” is inserted. The decoder 332 of the stuff circuit 323 switches the switch corresponding to this signal stuff position to the contact b among the switches 331-16 to 331-0, switches the switch on the LSB side to the contact c side, and further switches the MSB from it. Switch the side switch to the contact a side. As a result, since data “1” is inserted, the stuff circuit 323 outputs data in which data “1” is inserted at the position indicated by the signal stuff position (FIG. 24F).
[0130]
The barrel shifter 324 barrel-shifts the input data by the amount indicated by the signal shift (FIG. 24 (G)) supplied from the controller 326 and outputs it (FIG. 24 (H)). This output is further held once by the D-type flip-flop 325 and then output to the subsequent stage (FIG. 24I).
[0131]
Data “1” is inserted into the data output from the D-type flip-flop 325 after the 22-bit data. Therefore, even if all the bits between the data “1” and the next data “1” are 0, the number of consecutive 0 data is 22.
[0132]
FIG. 25 illustrates a configuration example of the converter 202. The configuration including the D-type flip-flop 341 to the controller 346 of the converter 202 is basically the same as the configuration of the D-type flip-flop 321 to the controller 326 of the converter 212 shown in FIG. Instead of this, the point in which the delete circuit 343 is inserted is different from that in the converter 212. Other configurations are the same as those in the converter 212 of FIG.
[0133]
That is, in this converter 202, the delete circuit 343 deletes the bit (data “1” inserted by the stuff circuit 323 in FIG. 22) according to the signal delete position output from the controller 346 and indicating the position of the bit to be deleted. Is done.
[0134]
Other operations are the same as those in the converter 212 of FIG.
[0135]
FIG. 26 shows a configuration example of the discrete circuit 343. In this configuration example, among the 16-bit data input from the D-type flip-flops 342 and 341, 15 bits on the LSB side are supplied to the contacts a of the corresponding switches 351-0 to 351-14, respectively. . MSB side data is supplied to the contact b of each switch by 1 bit. The decoder 352 deletes the bit specified by the signal delete position supplied from the controller 346 and outputs it as 15-bit data.
[0136]
FIG. 26 shows a state where the 13th bit from the LSB is deleted. Accordingly, in this case, the switches 351-0 to 351-11 are switched to the contact a side, and 12 bits from the LSB to the 12th are selected and output as they are. Further, since the switches 351-12 to 351-14 are respectively switched to the contact b side, the 14th to 16th data are selected and output as the 13th to 15th bit data. The
[0137]
The input of the stuff circuit 323 of FIG. 23 and the input of the delete circuit 343 of FIG. 26 is 16 bits. The input of the stuff circuit 323 of the converter 212 of FIG. 22 is supplied from the D-type flip-flops 322 and 321, respectively. This is because, in the converter 202 of FIG. 25, the input of the delete circuit 343 is 16 bits by the D-type flip-flops 342 and 341. In FIG. 22, the 17 bits output from the stuff circuit 323 are barrel-shifted by the barrel shifter 324, so that, for example, 8 bits are finally selected and output. In the converter 202 of FIG. The 15-bit data output from 343 is barrel-shifted by a predetermined amount by the barrel shifter 344 to obtain 8-bit data.
[0138]
FIG. 27 illustrates another configuration example of the converter 212. In this configuration example, the counter 361 counts the number of consecutive 0 bits in the input data and outputs the count result to the controller 326. For example, when the counter 361 counts 22 consecutive zero bits, the controller 326 outputs the signal stuff position to the stuff circuit 323. At this time, the controller 326 resets the counter 361 and causes the counter 361 to count the number of consecutive 0 bits again.
[0139]
Other configurations and operations are the same as those in FIG.
[0140]
FIG. 28 illustrates another configuration example of the converter 202. In this configuration example, the counter 371 counts the number of consecutive 0s in the input data and outputs the count result to the controller 346. When the count value of the counter 371 reaches 22, the controller 346 outputs the signal delete position to the delete circuit 343, resets the counter 371, and causes the counter 371 to count the number of new consecutive 0 bits again. . Other configurations are the same as those in FIG.
[0141]
Thus, in this configuration example, data “1” as a marker bit is inserted and deleted based on a predetermined pattern (the number of consecutive data “0”).
[0142]
The configuration shown in FIGS. 27 and 28 enables more efficient processing than the configuration shown in FIGS. 22 and 25. However, the converted length depends on the original history information.
[0143]
FIG. 29 shows a configuration example of the user data formatter 213. In this example, when the controller 383 outputs a read address to a buffer memory (not shown) disposed between the converter 212 and the user data formatter 213, the data read from the read address is changed to the user data formatter 213. The switch 382 is supplied to the contact a side. The ROM 381 stores data necessary for generating user_data () such as a user data start code and a data ID. The controller 313 switches the switch 382 to the contact a side or the contact b side at a predetermined timing, and appropriately selects and outputs data stored in the ROM 381 or data supplied from the converter 212. As a result, data in the format of user_data () is output to the encoding device 106.
[0144]
Although not shown, the user data decoder 201 can be realized by outputting input data through a switch that is read from the ROM 381 in FIG. 29 and deletes inserted data. .
[0145]
FIG. 30 shows a state where a plurality of transcoders 101-1 to 101-N are connected in series and used in a video editing studio, for example. The coding parameter multiplexing apparatus 103-i of each transcoder 101-i (i = 1 to N) is used by itself in a section in which the oldest coding parameter in the above-described coding parameter area is recorded. Overwrite the latest encoding parameters. As a result, the latest four generations of encoding parameters (generation history information) corresponding to the same macroblock are recorded in the baseband image data.
[0146]
The encoder 121-i (FIG. 19) of each encoding device 106-i performs quantization in the variable length encoding circuit 58 based on the encoding parameter used this time supplied from the encoding parameter separation device 105-i. The video data supplied from the circuit 57 is encoded. In the bit stream generated in this way (for example, picture_header ()), the current encoding parameter is multiplexed.
[0147]
The variable length encoding circuit 58 also multiplexes user data (including generation history information) supplied from the history encoding device 107-i into the output bit stream (in the embedding process as shown in FIG. 18). Not multiplexed into the bitstream). The bit stream output from the encoding device 106-i is input to the subsequent transcoder 101- (i + 1) via the SDTI 108-i.
[0148]
The transcoder 101-i and the transcoder 101- (i + 1) are configured as shown in FIG. Therefore, the process is the same as that described with reference to FIG.
[0149]
When it is desired to change what is currently encoded as an I picture to P or B picture as the encoding using the actual encoding parameter history, the past encoding parameter history is referred to and P or A case where the picture is a B picture is searched. If these histories exist, the picture type is changed using parameters such as a motion vector. On the other hand, if there is no history in the past, the change of the picture type without motion detection is abandoned. Of course, even if there is no history, the picture type can be changed by performing motion detection.
[0150]
In the format shown in FIG. 18, encoding parameters for four generations are embedded. However, parameters of picture types of I, P, and B can be embedded. FIG. 31 shows an example of the format in this case. In this example, one generation of encoding parameters (picture history information) is recorded for each picture type when the same macroblock has been encoded with a change in picture type in the past. Accordingly, the decoder 111 shown in FIG. 16 and the encoder 121 shown in FIG. 19 are arranged in place of the encoding parameters of the current (latest), one generation before, two generations, and three generations before, I picture, P picture. , And the encoding parameters for one generation corresponding to the B picture are input / output.
[0151]
In this example, the Cb [1] [x] and Cr [1] [x] regions are not used, so the Cb [1] [x] and Cr [1] [x] regions are not used. The present invention can also be applied to image data in 4: 2: 0 format.
[0152]
In this example, the decoding apparatus 102 extracts the encoding parameter simultaneously with decoding, determines the picture type, and writes (multiplexes) the encoding parameter in a location corresponding to the picture type of the image signal. Output to the separation device 105. The encoding parameter separation apparatus 105 can separate the encoding parameters, and can perform re-encoding while changing the picture type in consideration of the picture type to be encoded from now and the input past encoding parameters. .
[0153]
Next, processing for determining a picture type that can be changed in each transcoder 101 will be described with reference to the flowchart of FIG. This process is premised on that the picture type change in the transcoder 101 is executed without performing motion detection because it uses a past motion vector. Further, the processing described below is executed by the encoding parameter separation device 105.
[0154]
In step S 1, one generation of encoding parameters (picture history information) is input to the encoding parameter controller 122 for each picture type.
[0155]
In step S2, the coding parameter separation apparatus 105 determines whether there is a coding parameter when the picture history information is changed to the B picture. If it is determined that there is an encoding parameter when the picture history information is changed to the B picture, the process proceeds to step S3.
[0156]
In step S3, the coding parameter separation apparatus 105 determines whether there is a coding parameter when the picture history information is changed to the P picture. If it is determined that there is an encoding parameter when the picture history information is changed to the P picture, the process proceeds to step S4.
[0157]
In step S4, the encoding parameter separation apparatus 105 determines that the changeable picture types are I picture, P picture, and B picture.
[0158]
If it is determined in step S3 that there is no coding parameter when the picture history information is changed to the P picture, the process proceeds to step S5.
[0159]
In step S5, the encoding parameter separation apparatus 105 determines that the changeable picture types are I picture and B picture. Furthermore, the encoding parameter separation apparatus 105 can change to a P picture in a pseudo manner by performing a special process (only the forward prediction vector is used, not the backward prediction vector included in the B picture history information). Judge.
[0160]
If it is determined in step S2 that there is no coding parameter when the picture history information is changed to the B picture, the process proceeds to step S6.
[0161]
In step S6, the coding parameter separation apparatus 105 determines whether there is a coding parameter when the picture history information is changed to the P picture. If it is determined that there is an encoding parameter when the picture history information is changed to the P picture, the process proceeds to step S7.
[0162]
In step S7, the coding parameter separation apparatus 105 determines that the picture types that can be changed are I picture and P picture. Furthermore, the coding parameter separation apparatus 105 determines that the picture can be changed to the B picture by performing a special process (using only the forward prediction vector included in the history information for the P picture).
[0163]
If it is determined in step S6 that there is no coding parameter when the picture history information is changed to the P picture, the process proceeds to step S8. In step S8, since there is no motion vector, the coding parameter separation apparatus 105 determines that the picture type that can be changed is only an I picture (since it is an I picture, it cannot be changed to other than an I picture).
[0164]
Following step S4, S5, S7, and S8, in step S9, the encoding parameter separation device 105 displays a changeable picture type on a display device (not shown) and notifies the user.
[0165]
FIG. 33 shows an example of changing the picture type. The picture type is changed by changing the number of frames constituting the GOP. That is, in this example, a Long GOP (first frame) composed of frames with N = 15 (GOP frame number N = 15) and M = 3 (I or G picture appearance period M = 3 in the GOP). Generation) to Short GOP (second generation) consisting of N = 1, M = 1 frames, and again Long GOP (third generation) consisting of N = 15, M = 3 frames Has been converted. In the figure, a broken line indicates a GOP boundary.
[0166]
When the picture type is changed from the first generation to the second generation, as is clear from the description of the changeable picture type determination process described above, it is possible to change the picture type to I picture for all frames. is there. When the picture type is changed, all motion vectors calculated when the moving image (0th generation) is converted to the 1st generation are stored (remained) in the picture history information. Next, when converting to Long GOP again (the picture type is changed from the second generation to the third generation), the motion vector for each picture type when converted from the zeroth generation to the first generation is stored. Therefore, by reusing this, it is possible to suppress degradation in image quality and convert it back to a Long GOP again.
[0167]
FIG. 34 shows another example of picture type change. In this example, N = 14, M = 2 Long GOP (first generation) is converted to N = 2, M = 2 Short GOP (second generation), and N = 1, M 1 is converted into a Short GOP (third generation) with the number of frames being 1, and a frame number N is converted into a random GOP (fourth generation).
[0168]
Also in this example, the motion vector for each picture type when converted from the 0th generation to the 1st generation is stored until the conversion from the 3rd generation to the 4th generation. Therefore, as shown in FIG. 34, even when the picture type is changed in a complicated manner, the stored image parameters can be reused, so that image quality deterioration can be suppressed to a small level. Furthermore, if the stored quantization parameter quantization scale is used effectively, it is possible to realize encoding with little image quality degradation.
[0169]
The reuse of the quantization scale will be described with reference to FIG. FIG. 35 shows that a predetermined frame is always converted into an I picture from the first generation to the fourth generation, and only the bit rate is changed to 4 Mbps, 18 Mbps, or 50 Mbps.
[0170]
For example, when converting from the first generation (4 Mbps) to the second generation (18 Mbps), the image quality does not improve even if re-encoding is performed with a fine quantization scale as the bit rate increases. This is because data quantized in the past in a coarse quantization step is not restored. Therefore, as shown in FIG. 35, even if the bit rate is increased in the middle, the quantization in fine quantization steps is accompanied by an increase in the amount of information and does not lead to an improvement in image quality. Therefore, if control is performed so as to maintain the coarsest (larger) quantization scale in the past, the most efficient and efficient encoding is possible.
[0171]
As described above, when the bit rate is changed, it is very effective to encode using the history of the past quantization scale.
[0172]
This quantization control process will be described with reference to the flowchart of FIG. In step S11, the encoding parameter separation apparatus 105 determines whether or not the input picture history information includes a picture type encoding parameter to be converted. If it is determined that there is a coding parameter of the picture type to be converted, the process proceeds to step S12.
[0173]
In step S12, the coding parameter separation apparatus 105 extracts the quantization scale (Q_history) from the coding parameter which is the contrast of the picture history information.
[0174]
In step S <b> 13, the encoding parameter separation apparatus 105 reads the quantization scale candidate value Q_feedback fed back from the transmission buffer 59 to the quantization circuit 57.
[0175]
In step S14, the encoding parameter separation apparatus 105 determines whether Q_history is larger (coarse) than Q_feedback. When it is determined that Q_history is larger than Q_feedback, the process proceeds to step S15.
[0176]
In step S15, the encoding parameter separation apparatus 105 outputs Q_history to the quantization circuit 57 as a quantization scale. The quantization circuit 57 performs quantization using Q_history.
[0177]
In step S16, it is determined whether all macroblocks included in the frame have been quantized. When it is determined that all the macroblocks are not quantized, the process returns to step S13, and the processes of steps S13 to S16 are repeated until all the macroblocks are quantized.
[0178]
If it is determined in step S14 that Q_history is not larger (fine) than Q_feedback, the process proceeds to step S17.
[0179]
In step S <b> 17, the coding parameter separation apparatus 105 outputs Q_feedback to the quantization circuit 57 as a quantization scale. The quantization circuit 57 performs quantization using Q_feedback.
[0180]
If it is determined in step S11 that there is no encoding parameter of the picture type to be converted, the process proceeds to step S18.
[0181]
In step S <b> 18, the quantization circuit 57 receives the quantization scale candidate value Q_feedback fed back from the transmission buffer 59.
[0182]
In step S19, the quantization circuit 57 performs quantization using Q_feedback.
[0183]
In step S20, it is determined whether all macroblocks included in the frame have been quantized. If it is determined that all the macroblocks are not quantized, the process returns to step S18, and the processes of steps S18 to S20 are repeated until all the macroblocks are quantized.
[0184]
In the transcoder 101 in the present embodiment, as described above, the decoding side and the code side are roughly coupled, and the encoding parameters are multiplexed and transmitted to the image data. As shown, the decoding apparatus 102 and the encoding apparatus 106 may be connected (tightly coupled) by a high-speed bus 111 for encoding parameter transmission.
[0185]
FIG. 38 is a diagram showing a syntax for decoding an MPEG video stream. The decoder extracts a plurality of meaningful data items (data elements) from the bitstream by decoding the MPEG bitstream according to this syntax. In the drawing, the syntax described below has functions and conditional statements expressed in small letters, and data elements are shown in bold letters. The data item is described by a mnemonic indicating its name, bit length, type, and transmission order.
[0186]
First, functions used in the syntax shown in FIG. 38 will be described.
[0187]
The next_start_code () function is a function for searching for a start code described in the bitstream. Therefore, in the syntax shown in FIG. 38, the sequence_header () function and the sequence_extension () function are arranged in order after the next_start_code () function, so that this sequence_header () is included in this bitstream. Data elements defined by the function and the sequence_extension () function are described. Therefore, when decoding the bitstream, the next_start_code () function uses the next_start_code () function to find the start code (a type of data element) described at the beginning of the sequence_header () function and sequence_extension () function from the bitstream. Then, the sequence_header () function and the sequence_extension () function are further found, and each data element defined by them is decoded.
[0188]
The sequence_header () function is a function for defining header data of the sequence layer of the MPEG bit stream, and the sequence_extension () function is a function for defining extension data of the sequence layer of the MPEG bit stream. .
[0189]
The do {} while syntax placed next to the sequence_extension () function is a data element written based on the function in {} of the do statement while the condition defined by the while statement is true. This is a syntax for extracting from the stream. That is, with the do {} while syntax, while the condition defined by the while statement is true, a decoding process is performed to extract the data element described based on the function in the do statement from the bit stream.
[0190]
The nextbits () function used in the while statement is a function for comparing a bit or a bit string appearing in the bit stream with a data element to be decoded next. In the syntax example of FIG. 38, the nextbits () function compares the bit string in the bit stream with sequence_end_code indicating the end of the video sequence. When the bit string in the bit stream does not match the sequence_end_code, this while The sentence condition is true. Therefore, the do {} while syntax placed next to the sequence_extension () function means that the data element defined by the function in the do statement is not bitstreamed while the sequence_end_code indicating the end of the video sequence does not appear in the bitstream It shows that it is described in.
[0191]
In the bitstream, after each data element defined by the sequence_extension () function, a data element defined by the extension_and_user_data (0) function is described. The extension_and_user_data (0) function is a function for defining extension data and user data in the sequence layer of the MPEG bitstream.
[0192]
The do {} while syntax placed next to this extension_and_user_data (0) function is a data element described based on the function in {} of the do statement while the condition defined by the while statement is true. Is a function for extracting from the bitstream. The nextbits () function used in this while statement is a function for determining a match between a bit or a bit string appearing in the bit stream and picture_start_code or group_start_code, and the bit or bit string appearing in the bit stream, If picture_start_code or group_start_code matches, the condition defined by the while statement is true. Therefore, in this do {} while syntax, when picture_start_code or group_start_code appears in the bitstream, the code of the data element defined by the function in the do statement is described next to the start code. By searching for the start code indicated by this picture_start_code or group_start_code, the data element defined in the do statement can be extracted from the bitstream.
[0193]
The if statement described at the beginning of the do statement indicates a condition that group_start_code appears in the bitstream. When the condition by this if statement is true, the data elements defined by the group_of_picture_header (1) function and the extension_and_user_data (1) function are sequentially described in the bitstream after this group_start_code.
[0194]
The group_of_picture_header (1) function is a function for defining the header data of the GOP layer of the MPEG bitstream. The extension_and_user_data (1) function is the extension data (extension_data) and user data (extension_data) of the GOP layer of the MPEG bitstream. This is a function for defining (user_data).
[0195]
Furthermore, in this bitstream, the data elements defined by the picture_header () function and the picture_coding_extension () function are described after the data elements defined by the group_of_picture_header (1) function and the extension_and_user_data (1) function. Yes. Of course, if the condition of the if statement described above is not true, the data element defined by the group_of_picture_header (1) function and the extension_and_user_data (1) function is not described, so it is defined by the extension_and_user_data (0) function. The data element defined by the picture_header () function and the picture_coding_extension () function is described after the data element.
[0196]
The picture_header () function is a function for defining the header data of the picture layer of the MPEG bit stream, and the picture_coding_extension () function is a function for defining the first extension data of the picture layer of the MPEG bit stream It is.
[0197]
The next while statement is a function for determining the condition of the next if statement while the condition defined by the while statement is true. The nextbits () function used in this while statement is a function for determining a match between a bit string appearing in the bitstream and extension_start_code or user_data_start_code, and a bit string appearing in the bitstream and extension_start_code or user_data_start_code and If they match, the condition defined by this while statement is true.
[0198]
The first if statement is a function for determining whether the bit string appearing in the bitstream matches extension_start_code. When the bit string appearing in the bitstream matches the 32-bit extension_start_code, it indicates that the data element defined by the extension_data (2) function is described next to the extension_start_code in the bitstream. .
[0199]
The second if statement is a syntax for determining a match between the bit string appearing in the bitstream and user_data_start_code. If the bit string appearing in the bitstream matches the 32-bit user_data_start_code, the third if statement Condition judgment of if statement is performed. This user_data_start_code is a start code for indicating the start of the user data area in the picture layer of the MPEG bit stream.
[0200]
The third if statement is a syntax for determining whether the bit string appearing in the bitstream matches History_Data_ID. If the bit string appearing in the bit stream matches this 8-bit History_Data_ID, then in the user data area of the picture layer of this MPEG bit stream, the converted_history_stream () function is next to the code indicated by this 8-bit History_Data_ID. Indicates that the data element defined by is described.
[0201]
The converted_history_stream () function is a function for describing history information and history data for transmitting all the encoding parameters used at the time of MPEG encoding. Details of the data element defined by the converted_history_stream () function will be described later. The History_Data_ID is a start code indicating the history information and history data described in the user data area of the picture layer of the MPEG bitstream.
[0202]
The else statement is a syntax for indicating that the condition is not true in the third if statement. Therefore, when the data element defined by the converted_history_stream () function is not described in the user data area of the picture layer of the MPEG bit stream, the data element defined by the user_data () function is described.
[0203]
The picture_data () function is a function for describing data elements related to the slice layer and the macroblock layer after the user data of the picture layer of the MPEG bit stream. Normally, the data element indicated by this picture_data () function is the data element defined by the converted_history_stream () function described in the user data area of the picture layer of the bit stream or the data element defined by the user_data () function. As described below, if there is no extension_start_code or user_data_start_code in the bitstream indicating the data element of the picture layer, the data element indicated by this picture_data () function is defined by the picture_coding_extension () function It is described after the data element.
[0204]
Next to the data element indicated by the picture_data () function, data elements defined by the sequence_header () function and the sequence_extension () function are arranged in order. The data elements described by the sequence_header () function and the sequence_extension () function are exactly the same as the data elements described by the sequence_header () function and the sequence_extension () function described at the beginning of the video stream sequence. The reason why the same data is described in the stream in this way is that the data of the sequence layer is received when the reception starts from the middle of the data stream (for example, the bit stream portion corresponding to the picture layer) on the bit stream receiver side. This is to prevent the stream from being able to be decoded and being unable to decode the stream.
[0205]
Following the data element defined by the last sequence_header () function and sequence_extension () function, that is, at the end of the data stream, 32-bit sequence_end_code indicating the end of the sequence is described.
[0206]
An outline of the basic configuration of the above syntax is as shown in FIG.
[0207]
Next, the history stream defined by the converted_history_stream () function will be described.
[0208]
This converted_history_stream () is a function for inserting a history stream indicating history information into the user data area of the MGEG picture layer. The meaning of “converted” is a conversion process that inserts a marker bit (1 bit) at least every 22 bits of a history stream composed of history data to be inserted into the user area in order to prevent start emulation. It means that it is a stream.
[0209]
This converted_history_stream () is described in either a fixed-length history stream (FIGS. 40 to 46) or a variable-length history stream (FIG. 47) described below. When a fixed-length history stream is selected on the encoder side, there is an advantage that a circuit and software for decoding each data element from the history stream on the decoder side are simplified. On the other hand, when a variable length history stream is selected on the encoder side, history information (data elements) described in the user area of the picture layer can be arbitrarily selected in the encoder as needed. Can be reduced, and as a result, the data rate of the entire encoded bitstream can be reduced.
[0210]
The “history information”, “history data”, and “history parameter” described in the present invention are coding parameters (or data elements) used in past coding processing, and are the current (final stage) codes. It is not the encoding parameter used in the encoding process. For example, in the first generation encoding process, a certain picture is encoded and transmitted with an I picture, and in the next second generation encoding process, this picture is encoded and transmitted as a P picture. In the third generation encoding process, an example will be described in which this picture is encoded with a B picture and transmitted. The encoding parameters used in the third generation encoding process are set to predetermined positions in the sequence layer, GOP layer, picture layer, slice layer, and macroblock layer of the encoded bitstream generated in the third generation encoding process. It has been described. On the other hand, the encoding parameters used in the first generation and second generation encoding processes, which are past encoding processes, are sequence layers and GOP layers in which the encoding parameters used in the third generation encoding process are described. Is described in the user data area of the picture layer as the history information of the encoding parameter according to the syntax described above.
[0211]
First, the fixed-length history stream syntax will be described with reference to FIGS.
[0212]
The user area of the picture layer of the bitstream generated in the final stage (for example, third generation) encoding process is first used in the past (for example, first generation and second generation) encoding process. The coding parameters related to the sequence header of the sequence layer are inserted as a history stream. Note that history information such as the sequence header of the sequence layer of the bit stream generated in the past encoding process is not inserted into the sequence header of the sequence layer of the bit stream generated in the encoding process of the final stage. It should be noted that.
[0213]
Data elements related to the sequence header used in the past encoding processing are sequence_header_code, sequence_header_present_flag, horizontal_size_value, vertical_size_value, aspect_ratio_information, frame_rate_code, bit_rate_value, marker_bit, VBV_buffer_size_value, constrained_parameter_flag, load_nontrix_intra_quantizer_traiter
[0214]
The sequence_header_code is data representing the start synchronization code of the sequence layer. The sequence_header_present_flag is data indicating whether the data in the sequence_header is valid or invalid. horizontal_size_value is data consisting of the lower 12 bits of the number of pixels in the horizontal direction of the image. vertical_size_value is data consisting of the lower 12 bits of the number of vertical lines of the image. Aspect_ratio_information is data representing the pixel aspect ratio (aspect ratio) or display screen aspect ratio. The frame_rate_code is data representing an image display cycle.
[0215]
bit_rate_value is lower 18 bits (rounded up in units of 400 bsp) of the bit rate for limiting the amount of generated bits. The marker_bit is bit data inserted to prevent start code emulation. VBV_buffer_size_value is lower 10-bit data of a value that determines the size of the generated code amount control virtual buffer (video buffer verifier). constrained_parameter_flag is data indicating that each parameter is within the limit. The load_intra_quantizer_matrix is data indicating the presence of intra MB quantization matrix data. intra_quantizer_matrix is data indicating the value of the intra MB quantization matrix. load_non_intra_quantizer_matrix is data indicating the presence of non-intra MB quantization matrix data. non_intra_quantizer_matrix is data representing a value of a non-intra MB quantization matrix.
[0216]
Next, in the user area of the picture layer of the bit stream generated in the encoding process at the final stage, a data element representing a sequence extension of the sequence layer used in the past encoding process is described as a history stream. .
[0217]
Data elements representing sequence extensions used in the past encoding process are extension_start_code, extension_start_code_identifier, sequence_extension_present_flag, profile_and_level_indication, progressive_sequence, chroma_format_horizontal_size_extension, vertical_size_extension, bit_rate_extension, vbv_buffer_size_extframe, extension_start_code,
[0218]
extension_start_code is data representing a start synchronization code of extension data. extension_start_code_identifier is data indicating which extension data is sent. The sequence_extension_present_flag is data indicating whether the data in the sequence extension is valid or invalid. Profile_and_level_indication is data for designating the profile and level of video data. progressive_sequence is data indicating that the video data is sequentially scanned. chroma_format is data for designating the color difference format of the video data.
[0219]
The horizontal_size_extension is upper 2 bits data added to the horizntal_size_value of the sequence header. vertical_size_extension is upper 2 bits of data to be added to the vertical_size_value of the sequence header. bit_rate_extension is upper 12-bit data added to bit_rate_value of the sequence header. vbv_buffer_size_extension is upper 8-bit data to be added to vbv_buffer_size_value of the sequence header. low_delay is data indicating that a B picture is not included. Frame_rate_extension_n is data for obtaining a frame rate in combination with frame_rate_code of the sequence header. Frame_rate_extension_d is data for obtaining a frame rate in combination with frame_rate_code of the sequence header.
[0220]
Subsequently, in the user area of the picture layer of the bit stream, a data element representing a sequence layer sequence display extension used in the past encoding process is described as a history stream.
[0221]
The data element described as the sequence display extension includes extension_start_code, extension_start_code_identifier, sequence_display_extension_present_flag, video_format, color_description, color_primaries, transfer_characteristics, matrix_coeffients, display_horizontal_size, and display_vertical_size.
[0222]
extension_start_code is data representing a start synchronization code of extension data. extension_start_code_identifier is a code indicating which extension data is sent. The sequence_display_extension_present_flag is data indicating whether the data element in the sequence display extension is valid or invalid. video_format is data representing the video format of the original signal. color_description is data indicating that there is detailed data of the color space. color_primaries is data indicating details of the color characteristics of the original signal. transfer_characteristics is data indicating details of how photoelectric conversion is performed. Matrix_coeffients is data indicating details of how the original signal is converted from the three primary colors of light. display_horizontal_size is data representing the active area (horizontal size) of the intended display. display_vertical_size is data representing the active area (vertical size) of the intended display.
[0223]
Subsequently, macroblock assignment data (macroblock_assignment_in_user_data) indicating phase information of the macroblock generated in the past encoding process is stored in the user area of the picture layer of the bitstream generated in the encoding process of the final stage, It is described as a history stream.
[0224]
Macroblock_assignment_in_user_data indicating the phase information of the macroblock is composed of data elements such as macroblock_assignment_present_flag, v_phase, and h_phase.
[0225]
This macroblock_assignment_present_flag is data indicating whether the data element in macroblock_assignment_in_user_data is valid or invalid. v_phase is data indicating vertical phase information when a macroblock is cut out from image data. h_phase is data indicating horizontal phase information when a macroblock is cut out from image data.
[0226]
Subsequently, in the user area of the picture layer of the bitstream generated by the encoding process at the final stage, a data element representing the GOP header of the GOP layer used in the past encoding process is described as a history stream. Yes.
[0227]
The data element representing this GOP header is composed of group_start_code, group_of_picture_header_present_flag, time_code, closed_gop, and broken_link.
[0228]
group_start_code is data indicating the start synchronization code of the GOP layer. group_of_picture_header_present_flag is data indicating whether a data element in group_of_picture_header is valid or invalid. time_code is a time code indicating the time from the beginning of the sequence of the first picture of the GOP. closed_gop is flag data indicating that an image in a GOP can be reproduced independently from other GOPs. The broken_link is flag data indicating that the first B picture in the GOP cannot be accurately reproduced for editing or the like.
[0229]
Subsequently, in the user area of the picture layer of the bitstream generated by the encoding process at the final stage, a data element representing the picture header of the picture layer used in the past encoding process is described as a history stream. Yes.
[0230]
Data elements related to this picture header are composed of picture_start_code, temporal_reference, picture_coding_type, vbv_delay, full_pel_forward_vector, forward_f_code, full_pel_backward_vector, and backward_f_code.
[0231]
Specifically, picture_start_code is data representing the start synchronization code of the picture layer. temporal_reference is a number indicating the display order of pictures and is data to be reset at the top of the GOP. picture_coding_type is data indicating a picture type. vbv_delay is data indicating the initial state of the virtual buffer at the time of random access. full_pel_forward_vector is data indicating whether the accuracy of the forward motion vector is an integer unit or a half pixel unit. forward_f_code is data representing the forward motion vector search range. full_pel_backward_vector is data indicating whether the accuracy of the backward motion vector is an integer unit or a half pixel unit. backward_f_code is data representing the backward motion vector search range.
[0232]
Subsequently, in the user area of the picture layer of the bit stream generated by the encoding process at the final stage, the picture coding extension of the picture layer used in the past encoding process is described as a history stream.
[0233]
The data elements related to this picture coding extension are extension_start_code, extension_start_code_identifier, f_code [0] [0], f_code [0] [1], f_code [1] [0], f_code [1] [1], intra_dc_precision, picture_structure, top_field_first , Frame_predictive_frame_dct, concealment_motion_vectors, q_scale_type, intra_vlc_format, alternate_scan, repeat_firt_field, chroma_420_type, progressive_frame, composite_display_flag, v_axis, field_sequence, sub_carrier, burst_amplitude, and sub_carrier_phase.
[0234]
extension_start_code is a start code indicating the start of extension data of the picture layer. extension_start_code_identifier is a code indicating which extension data is sent. f_code [0] [0] is data representing the horizontal motion vector search range in the forward direction. f_code [0] [1] is data representing a vertical motion vector search range in the forward direction. f_code [1] [0] is data representing the horizontal motion vector search range in the backward direction. f_code [1] [1] is data representing a vertical motion vector search range in the backward direction.
[0235]
intra_dc_precision is data representing the precision of the DC coefficient. Picture_structure is data indicating a frame structure or a field structure. In the case of a field structure, the data indicates whether the upper field or the lower field. top_field_first is data indicating whether the first field is upper or lower in the case of a frame structure. In the case of a frame structure, frame_predictive_frame_dct is data indicating that the prediction of the frame mode DCT is only the frame mode. concealment_motion_vectors is data indicating that a motion vector for concealing a transmission error is attached to an intra macroblock.
[0236]
q_scale_type is data indicating whether to use a linear quantization scale or a nonlinear quantization scale. The intra_vlc_format is data indicating whether another two-dimensional VLC is used for the intra macroblock. The alternate_scan is data representing a selection between using a zigzag scan or an alternate scan. repeat_firt_field is data used for 2: 3 pull-down. The chroma_420_type is data representing the same value as the next progressive_frame when the signal format is 4: 2: 0, and 0 otherwise. progressive_frame is data indicating whether or not this picture can be sequentially scanned. composite_display_flag is data indicating whether the source signal is a composite signal.
[0237]
v_axis is data used when the source signal is PAL. The field_sequence is data used when the source signal is PAL. sub_carrier is data used when the source signal is PAL. burst_amplitude is data used when the source signal is PAL. sub_carrier_phase is data used when the source signal is PAL.
[0238]
Subsequently, the quantization matrix extension used in the past encoding process is described as a history stream in the user area of the picture layer of the bit stream generated by the encoding process at the final stage.
[0239]
Data elements related to the quantization matrix extension, extension_start_code, extension_start_code_identifier, quant_matrix_extension_present_flag, load_intra_quantizer_matrix, intra_quantizer_matrix [64], load_non_intra_quantizer_matrix, non_intra_quantizer_matrix [64], load_chroma_intra_quantizer_matrix, chroma_intra_quantizer_matrix [64], load_chroma_non_intra_quantizer_matrix, and consists chroma_non_intra_quantizer_matrix [64].
[0240]
extension_start_code is a start code indicating the start of the quantization matrix extension. extension_start_code_identifier is a code indicating which extension data is sent. quant_matrix_extension_present_flag is data for indicating whether the data element in the quantization matrix extension is valid or invalid. load_intra_quantizer_matrix is data indicating the presence of quantization matrix data for intra macroblocks. Intra_quantizer_matrix is data indicating the value of a quantization matrix for an intra macroblock.
[0241]
load_non_intra_quantizer_matrix is data indicating the presence of quantization matrix data for non-intra macroblocks. non_intra_quantizer_matrix is data representing the value of a quantization matrix for a non-intra macroblock. load_chroma_intra_quantizer_matrix is data indicating the presence of quantization matrix data for the color difference intra macroblock. chroma_intra_quantizer_matrix is data indicating the value of the quantization matrix for the color difference intra macroblock. load_chroma_non_intra_quantizer_matrix is data indicating the presence of quantization matrix data for a color difference non-intra macroblock. chroma_non_intra_quantizer_matrix is data indicating the value of the quantization matrix for the chrominance non-intra macroblock.
[0242]
Subsequently, the copyright extension used in the past encoding process is described as the history stream in the user area of the picture layer of the bit stream generated by the encoding process in the final stage.
[0243]
The data elements related to the copyright extension are composed of extension_start_code, extension_start_code_itentifier, copyright_extension_present_flag, copyright_flag, copyright_identifier, original_or_copy, copyright_number_1, copyright_number_2, and copyright_number_3.
[0244]
extension_start_code is a start code indicating the start of the copyright extension. This code indicates which extension data of extension_start_code_itentifier is sent. The copyright_extension_present_flag is data for indicating whether the data element in this copyright extension is valid or invalid. copyright_flag indicates whether or not a copy right is given to the encoded video data until the next copyright extension or sequence end.
[0245]
The copyright_identifier is data for identifying the registration organization of the copy right designated by ISO / IEC JTC / SC29. original_or_copy is data indicating whether the data in the bitstream is original data or copy data. copyright_number_1 is data representing bits 44 to 63 of the copyright number. copyright_number_2 is data representing bits 22 to 43 of the copyright number. copyright_number_3 is data representing bits 0 to 21 of the copyright number.
[0246]
Subsequently, the picture display extension (picture_display_extension) used in the past encoding process is described as a history stream in the user area of the picture layer of the bit stream generated by the encoding process at the final stage.
[0247]
Data elements representing this picture display extension include extension_start_code, extension_start_code_identifier, picture_display_extension_present_flag, frame_center_horizontal_offset_1, frame_center_vertical_offset_1, frame_center_horizontal_offset_2, frame_center_vertical_offset_2, frame_center_horizontal_offset_3, and _offcenter_3.
[0248]
extension_start_code is a start code for indicating the start of the picture display extension. extension_start_code_identifier is a code indicating which extension data is sent. picture_display_extension_present_flag is data indicating whether a data element in the picture display extension is valid or invalid. The frame_center_horizontal_offset is data indicating a horizontal offset of the display area, and can be defined up to three offset values. The frame_center_vertical_offset is data indicating the vertical offset of the display area, and can be defined up to three offset values.
[0249]
In the user area of the picture layer of the bit stream generated in the encoding process at the final stage, user data used in the past encoding process is stored in the history stream next to the history information indicating the picture display extension described above. It is described as.
[0250]
Next to the user data, information on the macroblock layer used in the past encoding process is described as a history stream.
[0251]
Information about this macroblock layer includes data elements related to macroblock positions such as macroblock_address_h, macroblock_address_v, slice_header_present_flag, skipped_macroblock_flag, macroblock_quant, macroblock_motion_forward, macroblock_motion_backward, macroblock_pattern, macroblock_type_block_type_data, And data elements related to quantization step control such as quantiser_scale_code, PMV [0] [0] [0], PMV [0] [0] [1], motion_vertical_field_select [0] [0], PMV [0] [1 ] [0], PMV [0] [1] [1], motion_vertical_field_select [0] [1], PMV [1] [0] [0], PMV [1] [0] [1], motion_vertical_field_select [1] Data on motion compensation such as [0], PMV [1] [1] [0], PMV [1] [1] [1], motion_vertical_field_select [1] [1], and data on macroblock patterns such as coded_block_pattern Jere And cement, num_mv_bits, is configured Num_coef_bits, and the data elements relating to the generated code amount of such Num_other_bits.
[0252]
Hereinafter, data elements related to the macroblock layer will be described in detail.
[0253]
macroblock_address_h is data for defining the absolute position of the current macroblock in the horizontal direction. macroblock_address_v is data for defining the absolute position of the current macroblock in the vertical direction. The slice_header_present_flag is data indicating whether or not this macroblock is the head of the slice layer and is accompanied by a slice header. skipped_macroblock_flag is data indicating whether or not to skip this macroblock in the decoding process.
[0254]
The macroblock_quant is data derived from a macroblock type (macroblock_type) shown in FIGS. 65 to 67 to be described later, and indicates whether quantizer_scale_code appears in the bitstream. The macroblock_motion_forward is data derived from the macroblock type shown in FIGS. 65 to 67 and used in the decoding process. The macroblock_motion_backward is data derived from the macroblock type shown in FIGS. 65 to 67 and used in the decoding process. The mocroblock_pattern is data derived from the macroblock type shown in FIGS. 65 to 67, and indicates whether the coded_block_pattern appears in the bitstream.
[0255]
The macroblock_intra is data derived from the macroblock type shown in FIGS. 65 to 67 and used in the decoding process. spatial_temporal_weight_code_flag is data derived from the macroblock type shown in FIGS. 65 to 67, and spatial_temporal_weight_code indicating the upsampling method of the lower layer image with temporal scalability is data indicating whether or not the bitstream exists. It is.
[0256]
frame_motion_type is a 2-bit code indicating the prediction type of the macroblock of the frame. If the number of prediction vectors is two and the field-based prediction type is “00”, if the number of prediction vectors is one and the field-based prediction type is “01”, the number of prediction vectors is one and the frame base The prediction type is “10”, and if the prediction type is one and the prime prediction type is “11”. field_motion_type is a 2-bit code indicating motion prediction of a macroblock in a field. If the prediction vector is one and the field-based prediction type is “01”, if the prediction vector is two and the 18 × 8 macroblock-based prediction type is “10”, the prediction vector is 1 It is “11” if the prediction type is individual and prime prime. dct_type is data indicating whether the DCT is a frame DCT mode or a field DCT mode. quantiser_scale_code is data indicating the quantization step size of the macroblock.
[0257]
Next, data elements relating to motion vectors will be described. The motion vector is encoded as a difference with respect to the previously encoded vector in order to reduce the motion vector required during decoding. In order to perform motion vector decoding, the decoder must maintain four motion vector prediction values (with horizontal and vertical components, respectively). This predicted motion vector is expressed as PMV [r] [s] [v]. [r] is a flag indicating whether the motion vector in the macroblock is the first vector or the second vector, and is “0” when the vector in the macroblock is the first vector. Thus, when the vector in the macroblock is the second vector, “1” is obtained. [s] is a flag indicating whether the direction of the motion vector in the macroblock is the forward direction or the backward direction, and is “0” in the case of the forward motion vector, and the backward motion vector In this case, it is “1”. [v] is a flag indicating whether the vector component in the macroblock is the horizontal direction or the vertical direction, and is “0” in the case of the horizontal component, and in the case of the vertical component Becomes “1”.
[0258]
Therefore, PMV [0] [0] [0] represents the horizontal component data of the forward motion vector of the first vector, and PMV [0] [0] [1] represents the first vector. PMV [0] [1] [0] represents the vertical component data of the forward motion vector, PMV [0] [1] [0] represents the horizontal component data of the backward motion vector of the first vector, and PMV [0] [ 1] [1] represents the vertical component data of the backward motion vector of the first vector, and PMV [1] [0] [0] represents the horizontal motion vector of the forward direction of the second vector. Represents the direction component data, PMV [1] [0] [1] represents the vertical component data of the forward motion vector of the second vector, and PMV [1] [1] [0] PMV [1] [1] [1] represents the vertical component data of the backward motion vector of the second vector, and represents the horizontal component data of the backward motion vector of the second vector. Yes.
[0259]
motion_vertical_field_select [r] [s] is data indicating which reference field is used for the prediction format. When the motion_vertical_field_select [r] [s] is “0”, the top reference field is used, and when it is “1”, the bottom reference field is used.
[0260]
Therefore, motion_vertical_field_select [0] [0] indicates a reference field for generating the forward motion vector of the first vector, and motion_vertical_field_select [0] [1] indicates the backward motion vector of the first vector. , Motion_vertical_field_select [1] [0] indicates a reference field for generating a forward motion vector of the second vector, and motion_vertical_field_select [1] [1] indicates the second A reference field for generating a backward motion vector of the vector is shown.
[0261]
The coded_block_pattern is variable length data indicating which DCT block has a significant coefficient (non-zero coefficient) among a plurality of DCT blocks storing DCT coefficients. num_mv_bits is data indicating the code amount of the motion vector in the macroblock. num_coef_bits is data indicating the code amount of the DCT coefficient in the macroblock. num_other_bits is data indicating the code amount of the macroblock and the code amount other than the motion vector and the DCT coefficient.
[0262]
Next, a syntax for decoding each data element from a variable-length history stream will be described with reference to FIGS.
[0263]
This variable length history stream consists of next_start_code () function, sequence_header () function, sequence_extension () function, extension_and_user_data (0) function, group_of_picture_header () function, extension_and_user_data (1) function, picture_header () function, picture_coding_extension () function, It consists of data elements defined by the extension_and_user_data (2) function and the picture_data () function.
[0264]
Since the next_start_code () function is a function for searching for a start code existing in the bit stream, the top of the history stream is a data element used in the past encoding process as shown in FIG. A data element defined by the sequence_header () function is described.
[0265]
The data elements defined by the sequence_header () function are sequence_header_code, sequence_header_present_flag, horizontal_size_value, vertical_size_value, aspect_ratio_information, frame_rate_code, bit_rate_value, marker_bit, VBV_buffer_size_value, constrained_parameter_flag, load_intra_intra_izer_izer, and so on.
[0266]
The sequence_header_code is data representing the start synchronization code of the sequence layer. The sequence_header_present_flag is data indicating whether the data in the sequence_header is valid or invalid. horizontal_size_value is data consisting of the lower 12 bits of the number of pixels in the horizontal direction of the image. vertical_size_value is data consisting of the lower 12 bits of the number of vertical lines of the image. Aspect_ratio_information is data representing the pixel aspect ratio (aspect ratio) or display screen aspect ratio. The frame_rate_code is data representing an image display cycle. bit_rate_value is lower 18 bits (rounded up in units of 400 bsp) of the bit rate for limiting the amount of generated bits.
[0267]
The marker_bit is bit data inserted to prevent start code emulation. VBV_buffer_size_value is lower 10-bit data of a value that determines the size of the generated code amount control virtual buffer (video buffer verifier). constrained_parameter_flag is data indicating that each parameter is within the limit. The load_intra_quantizer_matrix is data indicating the presence of intra MB quantization matrix data. intra_quantizer_matrix is data indicating the value of the intra MB quantization matrix. load_non_intra_quantizer_matrix is data indicating the presence of non-intra MB quantization matrix data. non_intra_quantizer_matrix is data representing a value of a non-intra MB quantization matrix.
[0268]
Next to the data element defined by the sequence_header () function, the data element defined by the sequence_extension () function as shown in FIG. 49 is described as a history stream.
[0269]
Data elements defined by the sequence_extension () function are extension_start_code, extension_start_code_identifier, sequence_extension_present_flag, profile_and_level_indication, progressive_sequence, chroma_format, horizontal_size_extension, vertical_size_extension, bit_rate_extension, _delay_rate_rate_delay_rate_delay_rate
[0270]
extension_start_code is data representing a start synchronization code of extension data. extension_start_code_identifier is data indicating which extension data is sent. The sequence_extension_present_flag is data indicating whether the data in the sequence extension is valid or invalid. Profile_and_level_indication is data for designating the profile and level of video data. progressive_sequence is data indicating that the video data is sequentially scanned. chroma_format is data for designating the color difference format of the video data. The horizontal_size_extension is upper 2 bits data added to the horizntal_size_value of the sequence header. vertical_size_extension is upper 2 bits of data added to vertical_size_value of the sequence header. bit_rate_extension is upper 12-bit data added to bit_rate_value of the sequence header. vbv_buffer_size_extension is upper 8-bit data to be added to vbv_buffer_size_value of the sequence header.
[0271]
low_delay is data indicating that a B picture is not included. Frame_rate_extension_n is data for obtaining a frame rate in combination with frame_rate_code of the sequence header. Frame_rate_extension_d is data for obtaining a frame rate in combination with frame_rate_code of the sequence header.
[0272]
Next to the data element defined by the sequence_extension () function, the data element defined by the extension_and_user_data (0) function as shown in FIG. 50 is described as a history stream. The extension_and_user_data (i) function describes only the data element defined by the user_data () function as a history stream without describing the data element defined by the extension_data () function when "i" is other than 2. . Therefore, the extension_and_user_data (0) function describes only the data element defined by the user_data () function as a history stream.
[0273]
The user_data () function describes user data as a history stream based on the syntax as shown in FIG.
[0274]
Next to the data element defined by the extension_and_user_data (0) function, the data element defined by the group_of_picture_header () function as shown in FIG. 52 and the data element defined by the extension_and_user_data (1) function are used as a history stream. It has been described. However, the data element defined by the group_of_picture_header () function and the data element defined by the extension_and_user_data (1) function are described only when the group_start_code indicating the GOP layer start code is described in the history stream. ing.
[0275]
The data element defined by the group_of_picture_header () function is composed of group_start_code, group_of_picture_header_present_flag, time_code, closed_gop, and broken_link.
[0276]
group_start_code is data indicating the start synchronization code of the GOP layer. group_of_picture_header_present_flag is data indicating whether a data element in group_of_picture_header is valid or invalid. time_code is a time code indicating the time from the beginning of the sequence of the first picture of the GOP. closed_gop is flag data indicating that an image in a GOP can be reproduced independently from other GOPs. The broken_link is flag data indicating that the first B picture in the GOP cannot be accurately reproduced for editing or the like.
[0277]
Similar to the extension_and_user_data (0) function, the extension_and_user_data (1) function describes only the data element defined by the user_data () function as a history stream.
[0278]
If group_start_code indicating the start code of the GOP layer does not exist in the history stream, the data elements defined by these group_of_picture_header () and extension_and_user_data (1) functions are not described in the history stream. Absent. In this case, after the data element defined by the extension_and_user_data (0) function, the data element defined by the picture_headr () function is described as a history stream.
[0279]
Data elements defined by the picture_headr () function are picture_start_code, temporal_reference, picture_coding_type, vbv_delay, full_pel_forward_vector, forward_f_code, full_pel_backward_vector, backward_f_code, extra_bit_picture, and extra_information_picture, as shown in FIG.
[0280]
Specifically, picture_start_code is data representing the start synchronization code of the picture layer. temporal_reference is a number indicating the display order of pictures and is data to be reset at the top of the GOP. picture_coding_type is data indicating a picture type. vbv_delay is data indicating the initial state of the virtual buffer at the time of random access. full_pel_forward_vector is data indicating whether the accuracy of the forward motion vector is an integer unit or a half pixel unit. forward_f_code is data representing the forward motion vector search range. full_pel_backward_vector is data indicating whether the accuracy of the backward motion vector is an integer unit or a half pixel unit. backward_f_code is data representing the backward motion vector search range. extra_bit_picture is a flag indicating the presence of subsequent additional information. When this extra_bit_picture is “1”, there is next extra_information_picture, and when extra_bit_picture is “0”, it indicates that there is no subsequent data. extra_information_picture is information reserved in the standard.
[0281]
Next to the data element defined by the picture_headr () function, the data element defined by the picture_coding_extension () function as shown in FIG. 54 is described as a history stream.
[0282]
The data elements defined by this picture_coding_extension () function are extension_start_code, extension_start_code_identifier, f_code [0] [0], f_code [0] [1], f_code [1] [0], f_code [1] [1], intra_dc_precision, picture_structure, top_field_first, frame_predictive_frame_dct, concealment_motion_vectors, q_scale_type, intra_vlc_format, alternate_scan, repeat_firt_field, chroma_420_type, progressive_frame, composite_display_flag, v_axis, field_sequence, sub_mplitude, phase_st, sub_carrier, burst
[0283]
extension_start_code is a start code indicating the start of extension data of the picture layer. extension_start_code_identifier is a code indicating which extension data is sent. f_code [0] [0] is data representing the horizontal motion vector search range in the forward direction. f_code [0] [1] is data representing a vertical motion vector search range in the forward direction. f_code [1] [0] is data representing the horizontal motion vector search range in the backward direction. f_code [1] [1] is data representing a vertical motion vector search range in the backward direction. intra_dc_precision is data representing the precision of the DC coefficient.
[0284]
Picture_structure is data indicating a frame structure or a field structure. In the case of a field structure, the data indicates whether the upper field or the lower field. top_field_first is data indicating whether the first field is upper or lower in the case of a frame structure. In the case of a frame structure, frame_predictive_frame_dct is data indicating that the prediction of the frame mode DCT is only the frame mode. concealment_motion_vectors is data indicating that a motion vector for concealing a transmission error is attached to an intra macroblock. q_scale_type is data indicating whether to use a linear quantization scale or a nonlinear quantization scale. The intra_vlc_format is data indicating whether another two-dimensional VLC is used for the intra macroblock.
[0285]
The alternate_scan is data representing a selection between using a zigzag scan or an alternate scan. repeat_firt_field is data used for 2: 3 pull-down. The chroma_420_type is data representing the same value as the next progressive_frame when the signal format is 4: 2: 0, and 0 otherwise. progressive_frame is data indicating whether or not this picture can be sequentially scanned. composite_display_flag is data indicating whether the source signal is a composite signal. v_axis is data used when the source signal is PAL. The field_sequence is data used when the source signal is PAL. sub_carrier is data used when the source signal is PAL. burst_amplitude is data used when the source signal is PAL. sub_carrier_phase is data used when the source signal is PAL.
[0286]
Next to the data element defined by the picture_coding_extension () function, the data element defined by extensions_and_user_data (2) is described as a history stream. As shown in FIG. 50, the extension_and_user_data (2) function describes a data element defined by the extension_data () function when an extension start code (extension_start_code) exists in the bitstream. Next to this data element, when a user data start code (user_data_start_code) exists in the bitstream, a data element defined by the user_data () function is described. However, when the extension start code and the user data start code do not exist in the bit stream, the data elements defined by the extension_data () function and the user_data () function are not described in the bit stream.
[0287]
As shown in FIG. 55, the extension_data () function records a data element indicating extension_start_code and a data element defined by the quant_matrix_extension () function, copyright_extension () function, and picture_display_extension () function in the bitstream. This is a function to describe as a stream.
[0288]
Data elements defined by the quant_matrix_extension () function, as shown in FIG. 56, extension_start_code, extension_start_code_identifier, quant_matrix_extension_present_flag, load_intra_quantizer_matrix, intra_quantizer_matrix [64], load_non_intra_quantizer_matrix, non_intra_quantizer_matrix [64], load_chroma_intra_quantizer_matrix, chroma_intra_quantizer_matrix [64], load_chroma_non_intra_quantizer_matrix, and chroma_non_intra_quantizer_matrix [64].
[0289]
extension_start_code is a start code indicating the start of the quantization matrix extension. extension_start_code_identifier is a code indicating which extension data is sent. quant_matrix_extension_present_flag is data for indicating whether the data element in the quantization matrix extension is valid or invalid. load_intra_quantizer_matrix is data indicating the presence of quantization matrix data for intra macroblocks. Intra_quantizer_matrix is data indicating the value of a quantization matrix for an intra macroblock.
[0290]
load_non_intra_quantizer_matrix is data indicating the presence of quantization matrix data for non-intra macroblocks. non_intra_quantizer_matrix is data representing the value of a quantization matrix for a non-intra macroblock. load_chroma_intra_quantizer_matrix is data indicating the presence of quantization matrix data for the color difference intra macroblock. chroma_intra_quantizer_matrix is data indicating the value of the quantization matrix for the color difference intra macroblock. load_chroma_non_intra_quantizer_matrix is data indicating the presence of quantization matrix data for a color difference non-intra macroblock. chroma_non_intra_quantizer_matrix is data indicating the value of the quantization matrix for the chrominance non-intra macroblock.
[0291]
As shown in FIG. 57, the data element defined by the copyright_extension () function is composed of extension_start_code, extension_start_code_itentifier, copyright_extension_present_flag, copyright_flag, copyright_identifier, original_or_copy, copyright_number_1, copyright_number_2, and copyright_number_3.
[0292]
extension_start_code is a start code indicating the start of the copyright extension. extension_start_code_itentifier This code indicates which extension data is sent. The copyright_extension_present_flag is data for indicating whether the data element in this copyright extension is valid or invalid.
[0293]
copyright_flag indicates whether or not a copy right is given to the encoded video data until the next copyright extension or sequence end. The copyright_identifier is data for identifying the registration organization of the copy right designated by ISO / IEC JTC / SC29. original_or_copy is data indicating whether the data in the bitstream is original data or copy data. copyright_number_1 is data representing bits 44 to 63 of the copyright number. copyright_number_2 is data representing bits 22 to 43 of the copyright number. copyright_number_3 is data representing bits 0 to 21 of the copyright number.
[0294]
Data elements defined by the picture_display_extension () function are extension_start_code_identifier, frame_center_horizontal_offset, frame_center_vertical_offset, etc., as shown in FIG.
[0295]
extension_start_code_identifier is a code indicating which extension data is sent. The frame_center_horizontal_offset is data indicating the horizontal offset of the display area, and the number of offset values defined by number_of_frame_center_offsets can be defined. The frame_center_vertical_offset is data indicating the vertical offset of the display area, and the number of offset values defined by number_of_frame_center_offsets can be defined.
[0296]
Returning again to FIG. 47, after the data element defined by the extension_and_user_data (2) function, the data element defined by the picture_data () function is described as a history stream.
[0297]
The data element defined by the picture_data () function is a data element defined by the slice () function as shown in FIG. However, when slice_start_code indicating the start code of the slice () function does not exist in the bitstream, the data element defined by this slice () function is not described in the bitstream.
[0298]
As shown in FIG. 60, the slice () function includes data elements such as slice_start_code, slice_quantiser_scale_code, intra_slice_flag, intra_slice, reserved_bits, extra_bit_slice, extra_information_slice, and extra_bit_slice, and data elements defined by the macroblock () function. It is a function to describe as.
[0299]
The slice_start_code is a start code indicating the start of the data element defined by the slice () function. The slice_quantiser_scale_code is data indicating the quantization step size set for the macroblock existing in this slice layer. However, when quantiser_scale_code is set for each macroblock, the macroblock_quantiser_scale_code data set for each macroblock is used preferentially.
[0300]
intra_slice_flag is a flag indicating whether or not intra_slice and reserved_bits exist in the bitstream. intra_slice is data indicating whether or not a non-intra macroblock exists in the slice layer. If any of the macroblocks in the slice layer is a non-intra macroblock, intra_slice is “0”, and if all of the macroblocks in the slice layer are non-intra macroblocks, intra_slice is “1”. Become. reserved_bits is 7-bit data and takes a value of “0”. extra_bit_slice is a flag indicating that additional information exists as a history stream, and is set to “1” when extra_information_slice exists next. If there is no additional information, it is set to “0”.
[0301]
Next to these data elements, data elements defined by the macroblock () function are described as a history stream.
[0302]
As shown in FIG. 61, the macroblock () function is a function for describing data elements such as macroblock_escape, macroblock_address_increment, and macroblock_quantiser_scale_code, and data elements defined by the macroblock_modes () function and macroblock_vecters (s) function. .
[0303]
macroblock_escape is a fixed bit string indicating whether or not the horizontal difference between the reference macroblock and the previous macroblock is 34 or more. If the horizontal difference between the reference macroblock and the previous macroblock is 34 or more, 33 is added to the value of macroblock_address_increment. The macroblock_address_increment is data indicating a horizontal difference between the reference macroblock and the previous macroblock. If there is one macroblock_escape before this macroblock_address_increment, the value obtained by adding 33 to the value of this macroblock_address_increment is the data indicating the horizontal difference between the actual reference macroblock and the previous macroblock. .
[0304]
macroblock_quantiser_scale_code is a quantization step size set for each macroblock. In each slice layer, slice_quantiser_scale_code indicating the quantization step size of the slice layer is set, but when macroblock_quantiser_scale_code is set for the reference macroblock, this quantization step size is selected.
[0305]
Next to macroblock_address_increment, a data element defined by the macroblock_modes () function is described. As shown in FIG. 62, the macroblock_modes () function is a function for describing data elements such as macroblock_type, frame_motion_type, field_motion_type, and dct_type as a history stream.
[0306]
macroblock_type is data indicating the coding type of the macroblock. Specifically, as shown in FIGS. 65 to 67, macroblock_type is variable-length data generated from flags such as macroblock_quant, dct_type_flag, macroblock_motion_forward, and macroblock_motion_backward. macroblock_quant is a flag indicating whether or not macroblock_quantiser_scale_code for setting the quantization step size for the macroblock is set. When macroblock_quantiser_scale_code exists in the bitstream, macroblock_quant is a value of “1”. I take the.
[0307]
dct_type_flag is a flag for indicating whether or not dct_type indicating whether the reference macroblock is encoded by the frame DCT or the field DCT exists (in other words, a flag indicating whether or not the DCT is DCT), and is a bit. When dct_type exists in the stream, this dct_type_flag takes a value of “1”. The macroblock_motion_forward is a flag indicating whether or not the reference macroblock is predicted forward, and takes a value of “1” when the reference macroblock is predicted forward. macroblock_motion_backward is a flag indicating whether or not the reference macroblock is predicted backward, and takes a value of “1” when backward prediction is performed.
[0308]
If the macroblock_motion_forward or the macroblock_motion_backward is “1”, the picture structure is a frame, and the frame_period_frame_dct is “0”, the data element representing the frame_motion_type is described next to the data element representing the macroblock_type. This frame_period_frame_dct is a flag indicating whether or not frame_motion_type exists in the bitstream.
[0309]
frame_motion_type is a 2-bit code indicating the prediction type of the macroblock of the frame. If the number of prediction vectors is two and the field-based prediction type is “00”, if the number of prediction vectors is one and the field-based prediction type is “01”, the number of prediction vectors is one and the frame base The prediction type is “10”, and if the prediction type is one and the prime prediction type is “11”.
[0310]
If macroblock_motion_forward or macroblock_motion_backward is “1” and a picture structure does not appear in a frame, a data element representing field_motion_type is described next to a data element representing macroblock_type.
[0311]
field_motion_type is a 2-bit code indicating motion prediction of a macroblock in a field. If the prediction vector is one and the field-based prediction type is “01”, if the prediction vector is two and the 18 × 8 macroblock-based prediction type is “10”, the prediction vector is 1 It is “11” if the prediction type is individual and prime prime.
[0312]
If the picture structure is a frame, frame_period_frame_dct indicates that frame_motion_type is present in the bitstream, and frame_period_frame_dct indicates that dct_type is present in the bitstream, then dct_type after the data element representing macroblock_type A data element representing is described. Dct_type is data indicating whether the DCT is a frame DCT mode or a field DCT mode.
[0313]
Referring back to FIG. 61, if the reference macroblock is a forward prediction macroblock or the reference macroblock is an intra macroblock and is a concealment macroblock, the motion_vectors (0) function Describes the data element defined by. When the reference macroblock is a backward prediction macroblock, a data element defined by the motion_vectors (1) function is described. The motion_vectors (0) function is a function for describing a data element relating to the first motion vector, and the motion_vectors (1) function is for describing a data element relating to the second motion vector. Is a function of
[0314]
The motion_vectors (s) function is a function for describing data elements relating to motion vectors, as shown in FIG.
[0315]
If there is one motion vector and the dial prime prediction mode is not used, a data element defined by motion_vertical_field_select [0] [s] and motion_vector (0, s) is described.
[0316]
In this motion_vertical_field_select [r] [s], the first motion vector (which may be either forward or backward vector) is a vector created by referring to the bottom field or refers to the top field. Is a flag indicating whether the vector is a This index “r” is an index indicating whether the vector is the first vector or the second vector, and “s” is whether the prediction direction is forward or backward prediction. It is an indicator that shows.
[0317]
As shown in FIG. 64, the motion_vector (r, s) function includes a data string related to motion_code [r] [s] [t], a data string related to motion_residual [r] [s] [t], and dmvector [t ] Is a function for describing the data representing [].
[0318]
motion_code [r] [s] [t] is variable-length data representing the magnitude of the motion vector in the range of −16 to +16. motion_residual [r] [s] [t] is variable-length data representing a motion vector residual. Therefore, a detailed motion vector can be described by the values of motion_code [r] [s] [t] and motion_residual [r] [s] [t]. dmvector [t] is an existing value according to the time distance in order to generate a motion vector in one field (for example, the top field is one field with respect to the bottom field) in the dual prime prediction mode. The motion vector is scaled, and the vertical direction is corrected in order to reflect the vertical shift between the lines of the top field and the bottom field. This index “r” is an index indicating whether the vector is the first vector or the second vector, and “s” is whether the prediction direction is forward or backward prediction. It is an indicator that shows. “S” is data indicating whether the motion vector is a vertical component or a horizontal component.
[0319]
64. First, a data string representing motion_coder [r] [s] [0] in the horizontal direction is described as a history stream by the motion_vector (r, s) function shown in FIG. Since the number of bits of both motion_residual [0] [s] [t] and motion_residual [1] [s] [t] is indicated by f_code [s] [t], f_code [s] [t] is not 1 In this case, motion_residual [r] [s] [t] is present in the bitstream. The motion_residual [r] [s] [0] of the horizontal component is not “1” and the motion_code [r] [s] [0] of the horizontal component is not “0”. Since there is a data element representing [r] [s] [0], which means that there is a horizontal component of the motion vector, in that case, motion_residual [r] [s ] A data element representing [0] is described.
[0320]
Subsequently, a data string representing motion_coder [r] [s] [1] in the vertical direction is described as a history stream. Similarly, since the number of bits of both motion_residual [0] [s] [t] and motion_residual [1] [s] [t] is indicated by f_code [s] [t], f_code [s] [t] If is not 1, it indicates that motion_residual [r] [s] [t] is present in the bitstream. motion_residual [r] [s] [1] is not “1” and motion_code [r] [s] [1] is not “0”. This means that motion_residual [r] [s] [1] Means that there is a vertical component of the motion vector, and in this case, the data element that represents the vertical component motion_residual [r] [s] [1] Is described.
[0321]
In the variable length format, history information can be reduced in order to reduce the transmission bit rate.
[0322]
That is, when macroblock_type and motion_vectors () are transferred but quantizer_scale_code is not transferred, the bit rate can be reduced by setting slice_quantiser_scale_code to “00000”.
[0323]
In addition, when only macroblock_type is transferred and motion_vectors (), quantizer_scale_code, and dct_type are not transferred, the bit rate can be reduced by using “not coded” as macroblock_type.
[0324]
Furthermore, when only picture_coding_type is transferred and all information below slice () is not transferred, the bit rate can be reduced by using picture_data () without slice_start_code.
[0325]
In the above, “1” is inserted every 22 bits in order to prevent the continuous “0” of 23 bits in user_data from being output. However, it may not be every 22 bits. Further, instead of counting the number of consecutive “0” s and inserting “1”, it is possible to check and insert Byte_allign.
[0326]
Furthermore, in MPEG, the generation of 23 bits of continuous “0” is prohibited, but in reality, only the case where 23 bits are continued from the beginning of the byte is a problem. If 0 continues for 23 bits, this is not a problem. Therefore, for example, “1” may be inserted at a position other than the LSB every 24 bits.
[0327]
In the above, the history information is in a format close to a video elementary stream, but may be in a format close to a packetized elementary stream or a transport stream. In addition, although the location of user_data in Elementary Stream is in front of picture_data, it can be other locations.
[0328]
The computer program for performing each of the above processes can be provided to the user via a network providing medium such as the Internet or a digital satellite in addition to a providing medium made of an information recording medium such as a magnetic disk or a CD-ROM.
[0329]
【The invention's effect】
As described above, according to the image data processing device and method and the recording medium of the present invention, the first encoding parameter is extracted from the first bit stream encoded with the first encoding parameter, and the first A first bit stream is decoded into image data using an encoding parameter of 1, and from the first bit stream encoded with the first encoding parameter, An encoding parameter obtained by removing the first encoding parameter from the past multiple generation encoding parameters is used as the first history encoding parameter. First history information including is extracted, and included in the extracted first encoding parameter and first history information First history encoding parameter Are output together with the decoded image data, and the output first encoding parameter and First history encoding parameter To obtain a reuse encoding parameter for reuse in the encoding process, and output the first encoding parameter and Second history information including a coding parameter obtained by removing a reuse coding parameter from the first history coding parameter as a second history coding parameter Is generated, the output image data is acquired, and the acquired Reuse coding parameters And encoding the image data to generate a second bit stream, and to the generated second bit stream, Reuse coding parameters In addition, since the second history information is multiplexed, it is possible to suppress image degradation due to re-encoding with a small-scale apparatus.
[Brief description of the drawings]
FIG. 1 is a diagram for explaining the principle of high-efficiency encoding.
FIG. 2 is a diagram for explaining a picture type when image data is compressed.
FIG. 3 is a diagram illustrating picture types when image data is compressed.
FIG. 4 is a diagram for explaining the principle of encoding a moving image signal.
FIG. 5 is a block diagram illustrating a configuration of an apparatus that encodes and decodes a moving image signal.
FIG. 6 is a diagram illustrating format conversion.
7 is a block diagram showing a configuration of the encoder 18 of FIG. 5. FIG.
8 is a diagram for explaining the operation of a prediction mode switching circuit 52 in FIG. 7;
9 is a diagram for explaining the operation of a prediction mode switching circuit 52 in FIG. 7;
10 is a diagram for explaining the operation of a prediction mode switching circuit 52 in FIG. 7;
11 is a diagram for explaining the operation of a prediction mode switching circuit 52 in FIG. 7;
12 is a block diagram showing a configuration of the decoder 31 in FIG. 5. FIG.
FIG. 13 is a diagram illustrating SNR control corresponding to a picture type.
FIG. 14 is a block diagram showing a configuration of a transcoder 101 to which the present invention is applied.
15 is a block diagram showing a more detailed configuration of the transcoder 101 of FIG.
16 is a block diagram showing a configuration of a decoder 111 built in the decoding device 102 of FIG.
FIG. 17 is a diagram illustrating macroblock pixels.
FIG. 18 is a diagram for explaining an area in which an encoding parameter is recorded.
FIG. 19 is a block diagram showing a configuration of an encoder 121 built in the encoding device 106 of FIG.
20 is a block diagram illustrating a configuration example of a history formatter 211 in FIG.
FIG. 21 is a block diagram illustrating a configuration example of the history decoder 203 of FIG.
22 is a block diagram illustrating a configuration example of a converter 212 in FIG.
23 is a block diagram illustrating a configuration example of a stuff circuit 323 in FIG.
24 is a timing chart illustrating the operation of converter 212 in FIG.
25 is a block diagram illustrating a configuration example of a converter 202 in FIG.
26 is a block diagram showing a configuration example of a discrete circuit 343 in FIG. 25. FIG.
27 is a block diagram showing another configuration example of the converter 212 in FIG.
28 is a block diagram showing another configuration example of the converter 202 in FIG.
29 is a block diagram illustrating a configuration example of a user data formatter 213 in FIG.
30 is a diagram showing a state in which the transcoder 101 in FIG. 14 is actually used.
FIG. 31 is a diagram for explaining an area in which an encoding parameter is recorded.
Fig. 32 is a flowchart for describing changeable picture type determination processing of the encoding device 106 of Fig. 14;
FIG. 33 is a diagram illustrating an example in which a picture type is changed.
FIG. 34 is a diagram illustrating another example in which the picture type is changed.
FIG. 35 is a diagram for describing quantization control processing of the encoding device 106 in FIG. 14;
36 is a flowchart for describing quantization control processing of the encoding device 106 in FIG. 14; FIG.
FIG. 37 is a block diagram showing a configuration of the transcoder 101 that is tightly coupled.
Fig. 38 is a diagram for describing the syntax of an MPEG stream.
FIG. 39 is a diagram for explaining the configuration of the syntax of FIG. 38;
Fig. 40 is a diagram for describing the syntax of history_stream () for recording history information of fixed length.
Fig. 41 is a diagram for describing the syntax of history_stream () for recording history information of fixed length.
Fig. 42 is a diagram illustrating the syntax of history_stream () for recording history information of fixed length.
FIG. 43 is a diagram illustrating the syntax of history_stream () for recording fixed-length history information.
Fig. 44 is a diagram for describing the syntax of history_stream () for recording history information of fixed length.
Fig. 45 is a diagram for describing the syntax of history_stream () for recording history information of fixed length.
FIG. 46 is a diagram for describing the syntax of history_stream () for recording fixed-length history information.
Fig. 47 is a diagram for describing the syntax of history_stream () for recording variable-length history information.
Fig. 48 is a diagram for describing the syntax of sequence_header ().
Fig. 49 is a diagram for describing the syntax of sequence_extension ().
Fig. 50 is a diagram for describing the syntax of extension_and_user_data ().
Fig. 51 is a diagram illustrating the syntax of user_data ().
Fig. 52 is a diagram for describing the syntax of group_of_pictures_header ().
Fig. 53 is a diagram for describing the syntax of picture_header ().
Fig. 54 is a diagram for describing the syntax of picture_coding_extension ().
[Fig. 55] Fig. 55 is a diagram for describing the syntax of extension_data ().
Fig. 56 is a diagram for describing the syntax of quant_matrix_extension ().
Fig. 57 is a diagram for describing the syntax of copyright_extension ().
Fig. 58 is a diagram for describing the syntax of picture_display_extension ().
Fig. 59 is a diagram for describing the syntax of picture_data ().
Fig. 60 is a diagram for describing the syntax of slice ().
Fig. 61 is a diagram for describing the syntax of macroblock ();
Fig. 62 is a diagram for describing the syntax of macroblock_modes ().
Fig. 63 is a diagram for describing the syntax of motion_vectors (s).
Fig. 64 is a diagram for describing the syntax of motion_vector (r, s).
Fig. 65 is a diagram for describing a variable length code of macroblock_type for an I picture.
Fig. 66 is a diagram for describing a variable length code of macroblock_type for a P picture.
[Fig. 67] Fig. 67 is a diagram for describing a variable length code of macroblock_type for a B picture.
FIG. 68 is a block diagram showing an example of a configuration of a conventional transcoder 131.
FIG. 69 is a block diagram showing an example of a configuration of a conventional transcoder 131.
Fig. 70 is a diagram for explaining an arrangement of a conventional encoding device and decoding device.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Encoding apparatus, 2 Decoding apparatus, 3 Recording medium, 12, 13 A / D converter, 14 Frame memory, 15 Luminance signal frame memory, 16 Color difference signal frame memory, 17 Format conversion circuit, 18 Encoder, 31 Decoder, 32 format conversion circuit, 33 frame memory, 34 luminance signal frame memory, 35 color difference signal frame memory, 36, 37 D / A converter, 50 motion vector detection circuit, 51 frame memory, 52 prediction mode switching circuit, 53 arithmetic unit, 54 prediction judgment circuit, 55 DCT mode switching circuit, 56 DCT circuit, 57 quantization circuit, 58 variable length coding circuit, 59 transmission buffer, 60 inverse quantization circuit, 61 IDCT circuit, 62 arithmetic unit, 63 frame memory, 64 Motion compensation circuit, 81 receive buffer, 82 variable length decoding Path, 83 inverse quantization circuit, 84 IDCT circuit, 85 arithmetic unit, 86 frame memory, 87 motion compensation circuit, 101 transcoder, 102 decoding device, 103 coding parameter multiplexing device, 105 coding parameter separation device, 106 coding Device, 106 SDTI, 111 decoder, 112 variable length decoding circuit, 121 encoder, 122 encoding parameter controller, 131 transcoder, 132 decoding device, 133 encoding device, 134 motion detection unit, 135 encoding unit

Claims (12)

In an image data processing apparatus that converts a bit rate of a bit stream of input image data or a GOP (Group of Picture) structure and outputs the converted data,
The first encoding parameter is extracted from the first bit stream encoded with the first encoding parameter, and the first bit stream is converted into the image data using the first encoding parameter. Decoding means for decoding into
A coding parameter obtained by removing the first coding parameter from a plurality of past generation parameters from the first bitstream coded with the first coding parameter is used as a first history coding parameter. History information extracting means for extracting first history information including;
The first coding parameter extracted by the decoding unit and the first history coding parameter included in the first history information extracted by the history information extraction unit are decoded by the decoding unit. Output means for outputting together with the image data;
An encoding parameter to be reused in an encoding process is acquired as a reuse encoding parameter from either the first encoding parameter or the first history encoding parameter output by the output unit. And an acquisition means for acquiring the image data output by the output means;
An encoding parameter obtained by removing the reuse encoding parameter acquired by the acquisition unit from the first encoding parameter and the first history encoding parameter output by the output unit is a second history code. History information generating means for generating the second history information included as the activation parameter ;
Based on the reuse encoding parameter acquired by the acquisition unit, the image data acquired by the acquisition unit is encoded to generate a second bit stream, and the re- transmission acquired by the acquisition unit. An image data processing apparatus comprising: an encoding unit that multiplexes the second encoding information generated by the use encoding parameter and the history information generation unit into the generated second bitstream. . The output means decodes the first coding parameter extracted by the decoding means and the first history coding parameter included in the first history information extracted by the history information extraction means. The image data processing apparatus according to claim 1, wherein the image data decoded by the means is multiplexed and output. The history information generating means includes a first generation parameter and a first history encoding parameter that are included in the first history encoding parameter for each picture type in a previous generation of encoding processing than the encoding processing by the encoding means. Generating the second history information from a picture history encoding parameter which is a used encoding parameter ;
When the encoding unit changes the picture type and encodes, the picture history encoding parameter of the generation in which the picture type is changed in the encoding process of the past generation than the encoding process by the encoding unit If is contained in the picture history encoding parameters, according to claim 1, characterized in that reusing picture history encoding parameter generation changing the picture type as the re-use encoding parameters Image data processing device. The first encoding parameter and the first history encoding parameter include a quantization scale;
The encoding means includes calculation means for calculating an encoding parameter. When changing the bit rate of an I picture, the encoding means includes the first quantization scale calculated by the calculation means and the reuse encoding parameter . The image data processing apparatus according to claim 1, wherein a larger one of the second quantization scales is used as a quantization scale used for encoding. The image data processing apparatus according to claim 1, wherein the history information generating unit converts the second history information into a user data format for multiplexing the second bit stream. The encoding means describes the second history information in a user data area in a picture layer of the second bitstream, and sets the reuse encoding parameter to a sequence layer of the second bitstream, GOP 6. The image data processing device according to claim 5 , wherein the image data processing device is described at a predetermined position in a layer, a picture layer, a slice layer, and a macroblock layer. The image data processing apparatus according to claim 5 , wherein the history information generation unit uses a fixed-length format as the format. The image data processing apparatus according to claim 5 , wherein the history information generation unit uses a variable-length format as the format. The image data processing apparatus according to claim 6 , wherein the history information generation unit inserts a marker bit in the second history information at a fixed period. The image data processing apparatus according to claim 7 , wherein the history information generation unit inserts a marker bit corresponding to a predetermined appearance pattern in the second history information. In an image data processing method for converting a bit rate of a bit stream of input image data or a GOP (Group of Picture) structure and outputting it,
The first encoding parameter is extracted from the first bit stream encoded with the first encoding parameter, and the first bit stream is converted into the image data using the first encoding parameter. A decoding step of decoding into
A coding parameter obtained by removing the first coding parameter from a plurality of past generation parameters from the first bitstream coded with the first coding parameter is used as a first history coding parameter. A history information extraction step for extracting first history information including;
The first coding parameter extracted by the processing of the decoding step and the first history coding parameter included in the first history information extracted by the processing of the history information extraction step are the decoding step. An output step of outputting together with the image data decoded by the processing of
Either from reuse coding parameter encoding parameters for reuse in the encoding process of said first encoding parameter and the first hysteresis encoded parameter outputted by the processing of the outputting step And obtaining the image data output by the output step, and
An encoding parameter obtained by removing the reuse encoding parameter acquired by the acquisition step from the first encoding parameter and the first history encoding parameter output by the output step processing is a first parameter. A history information generating step for generating second history information including the two history encoding parameters ;
Based on the reuse encoding parameter acquired by the process of the acquisition step, the image data acquired by the process of the acquisition step is encoded to generate a second bitstream, and the process of the acquisition step An encoding step of multiplexing the reuse encoding parameter acquired by step 2 and the second history information generated by the history information generation step into the generated second bitstream. A featured image data processing method. A program for causing a computer to execute a process of converting and outputting a bit rate or a GOP (Group of Picture) structure of a bit stream of input image data,
The first encoding parameter is extracted from the first bit stream encoded with the first encoding parameter, and the first bit stream is converted into the image data using the first encoding parameter. A decoding step of decoding into
A coding parameter obtained by removing the first coding parameter from a plurality of past generation parameters from the first bitstream coded with the first coding parameter is used as a first history coding parameter. A history information extraction step for extracting first history information including;
The first coding parameter extracted by the processing of the decoding step and the first history coding parameter included in the first history information extracted by the processing of the history information extraction step are the decoding step. An output step of outputting together with the image data decoded by
Reuse coding parameters for reusing coding parameters for reuse in coding processing from any of the first coding parameters and the first history coding parameters output by the processing of the output step And obtaining the image data output by the output step, and
An encoding parameter obtained by removing the reuse encoding parameter acquired by the acquisition step from the first encoding parameter and the first history encoding parameter output by the output step is a first parameter. A history information generating step for generating second history information including the two history encoding parameters ;
Based on the reuse encoding parameter extracted by the acquisition step, the image data acquired by the processing of the acquisition step is encoded to generate a second bit stream, and acquired by the processing of the acquisition step And a coding step of multiplexing the second history information extracted by the reuse encoding parameter and the history information generation step generated into the generated second bitstream. A recording medium on which a computer-readable program for recording is recorded.

Images